From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794912992; b=o1pQeUhUafxGErU+CC/4+357+8ITP/TaUh6uWLpbzK5wy6BUgzCGqBpw+hb3HjTQv0je HTtRpZ3DOmzs9vaTQAzEotidWJxI9zEMz/5a+XuW44j1+nKGQa+Zj+rhSNYBTWzv1j6UG bKBRcf95u1OcgCzBClAQzdmD4JbvgO0H8QZEA8RTrTl9mLg4DSTtUpSTlddpobONm8SxB mXUpe+ytcVXvVnwK9sXppJyTMOrCE6J8RMUHBmULrOwx8D2nzf2vuO9Jhyo9y6Ch4TqzN Oy4Uk3eALcUwLJ62UKE/PBdC9NJdOThi+FGc9RFQzy2hoKPJT+mdFf+yuThC8hWnMUQ== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794912992; h=Received:DKIM-Signature:From:To:Cc:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:MIME-Version:Content-Type: Content-Transfer-Encoding; bh=J4b2Wz72Orm8MbTc8cc4ySJ6ZN5RgfcfZKHevMSvilE=; b=OUtv51IL8s8PPXEJAsDTRY0slLSHLyabO79AE+JHCqtl+xeaF6BrZ/mkaNv9w8BuEaCK 2m5JkYWRZUfVBDaFqejUPSogyj/XcNNbWqrdjrq8bSw9/Va43+Jq3JI30rkcKH8YFJxkl WvYZlowQVC16hvjC0nKaaSK4M7g3OOsbn1tMebLQ8wNiR9dL929BOh7ujzapk6SGUHS6D bHL9mOhlGl6MuRzq7ksAk0cs6MvfhfoEJsUnpTyb6VR0+5/1WV16WoFKsuuBllBj0Vhwx TiIH0vLoQHr3cXP3IAqdyoA7XgBAcknyrbISAXtgkm6ZoZ+SWCZoF2ieq/wnxc3TcsA== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=linuxfoundation.org header.i=@linuxfoundation.org header.a=rsa-sha256 header.s=korg header.b=g8VJpBn9; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72309606A31C; Tue, 17 Nov 2026 10:56:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.a=rsa-sha256 header.s=korg header.b=g8VJpBn9 Received: by smtp.kernel.org (Postfix) with ESMTPSA id AA6071F000C5; Tue, 17 Nov 2026 10:51:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1794912992; bh=J4b2Wz72Orm8MbTc8cc4ySJ6ZN5RgfcfZKHevMSvilE=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=g8VJpBn9EE2ObuSpRi9GZH1kqPoi5u22RymoNCvccrYBosZCjRs3gwYqEJfb6hjTF O0pje2xXYUrJ0Ytae3HdxndsOrI9r6Q1SLjKJ0b/4ew2l5JePja/T3q2ct3G7ZfEal tZr11yv/u/L9H7+RrdKcxgAu1pJNZN8DznqA2R3k= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org, akpm@linux-foundation.org, torvalds@linux-foundation.org, stable@vger.kernel.org Cc: lwn@lwn.net, jslaby@suse.cz, Greg Kroah-Hartman Subject: Re: Linux 7.3.5 Date: Tue, 17 Nov 2026 11:45:30 +0100 Message-ID: <2026111729-armory-overjoyed-70fd@gregkh> X-Mailer: git-send-email 2.56.0 In-Reply-To: <2026111729-brim-nest-fa62@gregkh> References: <2026111729-brim-nest-fa62@gregkh> X-Mailing-List: stable@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit diff --git a/Documentation/ABI/testing/sysfs-fs-f2fs b/Documentation/ABI/testing/sysfs-fs-f2fs index 0cebc89799dd..4f3915948176 100644 --- a/Documentation/ABI/testing/sysfs-fs-f2fs +++ b/Documentation/ABI/testing/sysfs-fs-f2fs @@ -271,7 +271,7 @@ Description: Shows all enabled kernel features. inode_crtime, lost_found, verity, sb_checksum, casefold, readonly, compression, test_dummy_encryption_v2, atomic_write, pin_file, encrypted_casefold, linear_lookup, - fserror. + fserror, resizable_tail. What: /sys/fs/f2fs//inject_rate Date: May 2016 diff --git a/Documentation/admin-guide/pm/intel_idle.rst b/Documentation/admin-guide/pm/intel_idle.rst index 996e5bd83c42..d1a6251790b8 100644 --- a/Documentation/admin-guide/pm/intel_idle.rst +++ b/Documentation/admin-guide/pm/intel_idle.rst @@ -145,11 +145,13 @@ for the other idle state types (C2 and C3) the target residency value is 3 times the exit latency (again, that is because it reflects the target residency to exit latency ratio in the majority of cases for the processor models recognized by ``intel_idle``). The name of the idle state (to be returned by the ``name`` -idle state attribute in ``sysfs``) is either "Cx_LPI" (if it comes from ``_LPI`` -processing) or "Cx_ACPI", where x is the index of that idle state in the final -list (note that the minimum value of x is 1, because 0 is reserved for the -"polling" state), and its target residency is based on the exit latency value. -All of the idle states in the final list are enabled by default in this case. +idle state attribute in ``sysfs``) either is equal to the state name value that +comes from ``_LPI`` processing (with the possible fallback to "Cx_LPI" if the +state name is not provided) or it is "Cx_ACPI", where x is the index of that +idle state in the final list (note that the minimum value of x is 1, because 0 +is reserved for the "polling" state), and its target residency is based on the +exit latency value. All of the idle states in the final list are enabled by +default in this case. .. _intel-idle-initialization: diff --git a/Documentation/arch/arm64/silicon-errata.rst b/Documentation/arch/arm64/silicon-errata.rst index 99a1eb4b833d..95faec78b7f7 100644 --- a/Documentation/arch/arm64/silicon-errata.rst +++ b/Documentation/arch/arm64/silicon-errata.rst @@ -273,7 +273,8 @@ stable kernels. +----------------+-----------------+-----------------+-----------------------------+ | ARM | MMU L1 | #3878312 | N/A | +----------------+-----------------+-----------------+-----------------------------+ -| ARM | MMU S3 | #3995052 | N/A | +| ARM | MMU S3 | #3995052, | N/A | +| | | #3673557 | | +----------------+-----------------+-----------------+-----------------------------+ | ARM | GIC-700 | #2941627 | ARM64_ERRATUM_2941627 | +----------------+-----------------+-----------------+-----------------------------+ diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst index 16c35b491dad..4c241d220bc2 100644 --- a/Documentation/filesystems/overlayfs.rst +++ b/Documentation/filesystems/overlayfs.rst @@ -731,7 +731,8 @@ When decoding an overlay file handle, the following steps are followed: 1. Find underlying layer by UUID and path type information. 2. Decode the underlying filesystem file handle to underlying dentry. - 3. For a lower file handle, lookup the handle in index directory by name. + 3. For a lower file handle, encode the decoded underlying inode again and + lookup that handle in index directory by name. 4. If a whiteout is found in index, return ESTALE. This represents an overlay object that was deleted after its file handle was encoded. 5. For a non-directory, instantiate a disconnected overlay dentry from the diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst index e4b66af55ffb..6b641c3cee64 100644 --- a/Documentation/filesystems/resctrl.rst +++ b/Documentation/filesystems/resctrl.rst @@ -351,12 +351,12 @@ with the following files: "mbm_assign_mode": The supported counter assignment modes. The enclosed brackets indicate which mode is enabled. The MBM events associated with counters may reset when "mbm_assign_mode" - is changed. + is changed. See architecture specific notes below. :: # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode - [mbm_event] - default + [default] + mbm_event "mbm_event": @@ -374,20 +374,17 @@ with the following files: Moving to mbm_event counter assignment mode requires users to assign the counters to the events. Otherwise, the MBM event counters will return 'Unassigned' when read. - The mode is beneficial for AMD platforms that support more CTRL_MON - and MON groups than available hardware counters. By default, this - feature is enabled on AMD platforms with the ABMC (Assignable Bandwidth - Monitoring Counters) capability, ensuring counters remain assigned even - when the corresponding RMID is not actively used by any processor. + It is intended for deployments that need to manage counter assignment on + platforms where the number of monitoring groups exceeds the available + hardware counters. Hardware counters are scarce on these platforms, so + mbm_event mode suits workflows that monitor a subset of groups at a time and + rotate assignments as needed. The mbm_event mode ensures counters remain + assigned even when the corresponding RMID is not actively used. "default": In default mode, resctrl assumes there is a hardware counter for each - event within every CTRL_MON and MON group. On AMD platforms, it is - recommended to use the mbm_event mode, if supported, to prevent reset of MBM - events between reads resulting from hardware re-allocating counters. This can - result in misleading values or display "Unavailable" if no counter is assigned - to the event. + event within every CTRL_MON and MON group. * To enable "mbm_event" counter assignment mode: :: @@ -399,6 +396,24 @@ with the following files: # echo "default" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode + In default mode on AMD platforms that support more CTRL_MON and MON groups + than hardware counters, hardware dynamically shares a smaller pool of + counters among RMIDs. One counter from this pool is used per monitoring + group and counts every MBM event of that group's RMID. The size of the pool + is not enumerated to software (unlike "num_mbm_cntrs"), and "num_rmids" may + be much larger. For example, 64 counters in this pool monitor 64 groups. + Newer hardware may provide a larger pool. + + On such platforms, while the number of monitoring groups does not exceed + that pool, a counter stays attached to each RMID and readings remain + accurate. Creating more groups than the pool can cause hardware to + re-allocate those counters between successive reads of an event. Bandwidth + values may then be misleading, or a read may return "Unavailable" if no + counter is allocated to the RMID. There is no user-visible indication when + this begins. Users who need stable readings beyond that pool should switch + to mbm_event mode, if supported, and assign counters to the groups of + interest (rotating assignments as needed). + "num_mbm_cntrs": The maximum number of counters (total of available and assigned counters) in each domain when the system supports mbm_event mode. @@ -1783,17 +1798,19 @@ View the llc occupancy snapshot:: Examples on working with mbm_assign_mode ======================================== -a. Check if MBM counter assignment mode is supported. +a. Check if MBM counter assignment mode is supported. Enable "mbm_event" +mode. Switching the mbm_assign_mode may reset all the MBM counters (and +thus all MBM events) of all the resctrl groups. :: - # mount -t resctrl resctrl /sys/fs/resctrl/ - + # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode + [default] + mbm_event + # echo "mbm_event" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode [mbm_event] default -The "mbm_event" mode is detected and enabled. - b. Check how many assignable counters are supported. :: @@ -1804,36 +1821,39 @@ c. Check how many assignable counters are available for assignment in each domai :: # cat /sys/fs/resctrl/info/L3_MON/available_mbm_cntrs - 0=30;1=30 + 0=32;1=32 -d. To list the default group's assign states. +d. To list the default group's assign states. Counters are unassigned when the mode +is switched to "mbm_event". :: # cat /sys/fs/resctrl/mbm_L3_assignments - mbm_total_bytes:0=e;1=e - mbm_local_bytes:0=e;1=e + mbm_total_bytes:0=_;1=_ + mbm_local_bytes:0=_;1=_ -e. To unassign the counter associated with the mbm_total_bytes event on domain 0. +e. To assign the counter associated with the mbm_total_bytes event on domain 0 +in exclusive mode. :: - # echo "mbm_total_bytes:0=_" > /sys/fs/resctrl/mbm_L3_assignments + # echo "mbm_total_bytes:0=e" > /sys/fs/resctrl/mbm_L3_assignments # cat /sys/fs/resctrl/mbm_L3_assignments - mbm_total_bytes:0=_;1=e - mbm_local_bytes:0=e;1=e + mbm_total_bytes:0=e;1=_ + mbm_local_bytes:0=_;1=_ f. To unassign the counter associated with the mbm_total_bytes event on all domains. :: # echo "mbm_total_bytes:*=_" > /sys/fs/resctrl/mbm_L3_assignments - # cat /sys/fs/resctrl/mbm_L3_assignment + # cat /sys/fs/resctrl/mbm_L3_assignments mbm_total_bytes:0=_;1=_ - mbm_local_bytes:0=e;1=e + mbm_local_bytes:0=_;1=_ -g. To assign a counter associated with the mbm_total_bytes event on all domains in -exclusive mode. +g. To assign a counter associated with the mbm_total_bytes and mbm_local_bytes +events on all domains in exclusive mode. :: # echo "mbm_total_bytes:*=e" > /sys/fs/resctrl/mbm_L3_assignments + # echo "mbm_local_bytes:*=e" > /sys/fs/resctrl/mbm_L3_assignments # cat /sys/fs/resctrl/mbm_L3_assignments mbm_total_bytes:0=e;1=e mbm_local_bytes:0=e;1=e @@ -1890,13 +1910,8 @@ the MBM counters (and thus all MBM events) of all the resctrl groups. # echo "default" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode - mbm_event [default] - -m. Unmount the resctrl filesystem. -:: - - # umount /sys/fs/resctrl/ + mbm_event Intel RDT Errata ================ diff --git a/Documentation/hwmon/asus_rog_ryujin.rst b/Documentation/hwmon/asus_rog_ryujin.rst index b0d7ce8dd921..24dccef2d2df 100644 --- a/Documentation/hwmon/asus_rog_ryujin.rst +++ b/Documentation/hwmon/asus_rog_ryujin.rst @@ -33,6 +33,12 @@ Usage notes As these are USB HIDs, the driver can be loaded automatically by the kernel and supports hot swapping. +The pump of the RYUJIN III also has a 4-pin cable for a motherboard fan header. +The cooler either hands the pump to its internal controller, which follows the +PWM of the header its 4-pin cable is plugged into, or runs it at the pump duty +set over USB. Writing pwm1 selects the duty set over USB. Writing pwm2 sets +only the internal fan duty and leaves the pump control as it is. + Sysfs entries ------------- diff --git a/Documentation/mm/vmemmap_dedup.rst b/Documentation/mm/vmemmap_dedup.rst index 9fa8642ded48..66d58fbea5b1 100644 --- a/Documentation/mm/vmemmap_dedup.rst +++ b/Documentation/mm/vmemmap_dedup.rst @@ -131,8 +131,19 @@ pages of ``struct page`` (page 1 to page 7) are tail pages. The optimization is only applied when the size of the struct page is a power of 2. In this case, all tail pages of the same order are identical. See compound_head(). This allows us to remap the tail pages of the vmemmap to a -shared, read-only page. The head page is also remapped to a new page. This -allows the original vmemmap pages to be freed. +shared page. The head page is also remapped to a new page. This allows the +original vmemmap pages to be freed. + +The shared page is mapped writable. A lockless get of a page reference can +read a non-zero refcount before the remap and attempt its atomic update +after it. The refcount of a tail page is zero, so the update fails without +changing the page. Nothing else may write to the shared page: it is shared +by all HugeTLB pages of the same size in the zone, and a stray write would +corrupt all of them. + +HugeTLB pages whose vmemmap is optimized when the memmap is first set up at +boot keep a read-only mapping of the shared page. Nothing can do a lockless +get on them before they are handed to HugeTLB. Here is how things look after remapping:: diff --git a/Makefile b/Makefile index 05c5b682ad50..51e278ccea9c 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 VERSION = 7 PATCHLEVEL = 3 -SUBLEVEL = 4 +SUBLEVEL = 5 EXTRAVERSION = NAME = Baby Opossum Posse diff --git a/arch/arm64/include/asm/sdei.h b/arch/arm64/include/asm/sdei.h index b2248bd3cb58..a78a5337ff85 100644 --- a/arch/arm64/include/asm/sdei.h +++ b/arch/arm64/include/asm/sdei.h @@ -44,7 +44,8 @@ asmlinkage unsigned long __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg); unsigned long do_sdei_event(struct pt_regs *regs, - struct sdei_registered_event *arg); + struct sdei_registered_event *arg, + unsigned long vbar); unsigned long sdei_arch_get_entry_point(int conduit); #define sdei_arch_get_entry_point(x) sdei_arch_get_entry_point(x) diff --git a/arch/arm64/include/asm/suspend.h b/arch/arm64/include/asm/suspend.h index e9ce68d50ba4..a5abf0615594 100644 --- a/arch/arm64/include/asm/suspend.h +++ b/arch/arm64/include/asm/suspend.h @@ -2,7 +2,7 @@ #ifndef __ASM_SUSPEND_H #define __ASM_SUSPEND_H -#define NR_CTX_REGS 14 +#define NR_CTX_REGS 15 #define NR_CALLEE_SAVED_REGS 12 /* diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 72c03ccea59f..92b2fe0b2784 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -30,6 +30,7 @@ #include #include #include +#include /* * Handle IRQ/context state management when entering from kernel mode. @@ -1002,6 +1003,7 @@ asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs) asmlinkage noinstr unsigned long __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg) { + unsigned long elr, spsr, esr, far, vbar; irqentry_state_t state; unsigned long ret; @@ -1027,10 +1029,46 @@ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg) else if (cpu_has_pan()) set_pstate_pan(0); + /* + * Nothing saved the interrupted context's exception registers on the + * way in, and it may be in the middle of exception entry or return + * (SDEI ignores DAIF). Save them before anything can take an exception + * and overwrite them. + */ + elr = read_sysreg(elr_el1); + spsr = read_sysreg(spsr_el1); + esr = read_sysreg(esr_el1); + far = read_sysreg(far_el1); + + /* + * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to + * EL0 until the next exception from EL0 maps the kernel. They only + * handle exceptions from EL0: an exception taken here would run their + * empty current-EL half and never come back. Switch to the kernel + * vectors. The IRQ-like resume keeps using the interrupted VBAR, so it + * still goes through the trampoline. + */ + vbar = read_sysreg(vbar_el1); + if (arm64_kernel_unmapped_at_el0() && + vbar == (unsigned long)raw_cpu_read(this_cpu_vector)) { + write_sysreg((unsigned long)vectors, vbar_el1); + isb(); + } + state = irqentry_nmi_enter(regs); - ret = do_sdei_event(regs, arg); + ret = do_sdei_event(regs, arg, vbar); irqentry_nmi_exit(regs, state); + /* + * Put them back unconditionally. An unchanged ELR does not rule out a + * nested exception: one may have returned to the same address. + */ + write_sysreg(elr, elr_el1); + write_sysreg(spsr, spsr_el1); + write_sysreg(esr, esr_el1); + write_sysreg(far, far_el1); + write_sysreg(vbar, vbar_el1); + return ret; } #endif /* CONFIG_ARM_SDE_INTERFACE */ diff --git a/arch/arm64/kernel/machine_kexec_file.c b/arch/arm64/kernel/machine_kexec_file.c index 854d872dfd0f..14f453dd0946 100644 --- a/arch/arm64/kernel/machine_kexec_file.c +++ b/arch/arm64/kernel/machine_kexec_file.c @@ -112,6 +112,8 @@ int load_other_segments(struct kimage *image, vfree(headers); goto out_err; } + + vfree(image->elf_headers); image->elf_headers = headers; image->elf_load_addr = kbuf.mem; image->elf_headers_sz = headers_sz; diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 778f2a1faac8..1b902da8ba43 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -195,21 +195,49 @@ unsigned long sdei_arch_get_entry_point(int conduit) return 0; } +/* + * Exceptions an SDEI handler may take without harm once __sdei_handler() has + * restored the interrupted context's exception registers. Their handlers are + * safe in NMI context: + * - EL1 data aborts on translation, permission and access-flag faults: + * faulthandler_disabled() is true in NMI context, so do_page_fault() never + * takes the mmap lock. This covers the _nofault() accessors used by the + * generic dump_stack()/show_regs() code (e.g. print_worker_info()). + * - BRK: kprobes, WARN/BUG, kCFI and UBSAN traps, all NMI-capable. + * Anything else taken at EL1 is fatal on its own, or unexpected here. + */ +static bool sdei_nested_exception_ok(unsigned long esr) +{ + switch (ESR_ELx_EC(esr)) { + case ESR_ELx_EC_DABT_CUR: + return esr_fsc_is_translation_fault(esr) || + esr_fsc_is_permission_fault(esr) || + esr_fsc_is_access_flag_fault(esr); + case ESR_ELx_EC_BRK64: + return true; + default: + return false; + } +} + /* * do_sdei_event() returns one of: * SDEI_EV_HANDLED - success, return to the interrupted context. * SDEI_EV_FAILED - failure, return this error code to firmware. * virtual-address - success, return to this address. + * + * vbar is the interrupted context's VBAR_EL1, for the IRQ-like resume. */ unsigned long __kprobes do_sdei_event(struct pt_regs *regs, - struct sdei_registered_event *arg) + struct sdei_registered_event *arg, + unsigned long vbar) { u32 mode; int i, err = 0; int clobbered_registers = 4; u64 elr = read_sysreg(elr_el1); + u64 nested_elr, nested_esr, nested_far; u32 kernel_mode = read_sysreg(CurrentEL) | 1; /* +SPSel */ - unsigned long vbar = read_sysreg(vbar_el1); if (arm64_kernel_unmapped_at_el0()) clobbered_registers++; @@ -221,18 +249,27 @@ unsigned long __kprobes do_sdei_event(struct pt_regs *regs, } err = sdei_event_handler(regs, arg); - if (err) - return SDEI_EV_FAILED; - if (elr != read_sysreg(elr_el1)) { - /* - * We took a synchronous exception from the SDEI handler. - * This could deadlock, and if you interrupt KVM it will - * hyp-panic instead. - */ - pr_warn("unsafe: exception during handler\n"); + /* + * A synchronous exception taken by the handler has overwritten the + * exception registers; __sdei_handler() puts them back. Only the last + * one's syndrome survives; report it if it is not one known to be + * harmless. + * + * If the event interrupted a KVM world switch, VBAR points at the KVM + * hyp vectors and any such exception is a hyp panic instead. + */ + nested_elr = read_sysreg(elr_el1); + nested_esr = read_sysreg(esr_el1); + nested_far = read_sysreg(far_el1); + if (nested_elr != elr && !sdei_nested_exception_ok(nested_esr)) { + pr_warn_once("unexpected exception during handler: ESR %#llx FAR %#llx near %pS\n", + nested_esr, nested_far, (void *)nested_elr); } + if (err) + return SDEI_EV_FAILED; + mode = regs->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK); /* diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index b8baba281166..44305b997a02 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -470,6 +470,12 @@ static void amu_read_core_const_ctrs(void *val) cpu_read_corecnt(&ctrs->corecnt); } +static bool cpc_ffh_reg_valid(const struct cpc_reg *reg) +{ + return reg->bit_width && reg->bit_width <= 64 && + reg->bit_offset <= 64 - reg->bit_width; +} + static u64 cpc_ffh_extract_bits(const struct cpc_reg *reg, u64 val) { val &= GENMASK_ULL(reg->bit_offset + reg->bit_width - 1, @@ -506,7 +512,8 @@ int cpc_read_ffh_fb_ctrs(int cpu, struct cpc_reg *reg1, u64 *val1, struct amu_ffh_ctrs ctrs; int ret; - if (!is_amu_ctr_reg(reg1) || !is_amu_ctr_reg(reg2)) + if (!is_amu_ctr_reg(reg1) || !is_amu_ctr_reg(reg2) || + !cpc_ffh_reg_valid(reg1) || !cpc_ffh_reg_valid(reg2)) return -EINVAL; ret = counters_read_on_cpu(cpu, amu_read_core_const_ctrs, &ctrs); @@ -530,6 +537,9 @@ int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val) { int ret = -EOPNOTSUPP; + if (!cpc_ffh_reg_valid(reg)) + return -EINVAL; + switch ((u64)reg->address) { case CPC_FFH_CTR_CORE: ret = counters_read_on_cpu(cpu, cpu_read_corecnt, val); diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index bf6d6526639f..cae133805c76 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1362,21 +1362,32 @@ static void kvm_send_hwpoison_signal(unsigned long address, short lsb) send_sig_mceerr(BUS_MCEERR_AR, (void __user *)address, lsb, current); } -static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot, - unsigned long hva, +struct kvm_s2_fault_desc { + struct kvm_vcpu *vcpu; + phys_addr_t fault_ipa; + struct kvm_s2_trans *nested; + struct kvm_memory_slot *memslot; + unsigned long hva; +}; + +static bool fault_supports_stage2_huge_mapping(const struct kvm_s2_fault_desc *s2fd, unsigned long map_size) { - gpa_t gpa_start; + struct kvm_memory_slot *memslot = s2fd->memslot; + unsigned long hva = s2fd->hva; hva_t uaddr_start, uaddr_end; + gpa_t gpa_start; size_t size; /* The memslot and the VMA are guaranteed to be aligned to PAGE_SIZE */ if (map_size == PAGE_SIZE) return true; - /* pKVM only supports PMD_SIZE huge-mappings */ - if (is_protected_kvm_enabled() && map_size != PMD_SIZE) - return false; + /* pKVM only supports PMD_SIZE huge-mappings for non-protected VMs */ + if (is_protected_kvm_enabled()) { + if (vcpu_is_protected(s2fd->vcpu) || map_size != PMD_SIZE) + return false; + } size = memslot->npages * PAGE_SIZE; @@ -1436,9 +1447,9 @@ static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot, * Returns the size of the mapping. */ static long -transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot, - unsigned long hva, kvm_pfn_t *pfnp, gfn_t *gfnp) +transparent_hugepage_adjust(const struct kvm_s2_fault_desc *s2fd, kvm_pfn_t *pfnp, gfn_t *gfnp) { + struct kvm *kvm = s2fd->vcpu->kvm; kvm_pfn_t pfn = *pfnp; gfn_t gfn = *gfnp; @@ -1447,8 +1458,8 @@ transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot, * sure that the HVA and IPA are sufficiently aligned and that the * block map is contained within the memslot. */ - if (fault_supports_stage2_huge_mapping(memslot, hva, PMD_SIZE)) { - int sz = get_user_mapping_size(kvm, hva); + if (fault_supports_stage2_huge_mapping(s2fd, PMD_SIZE)) { + int sz = get_user_mapping_size(kvm, s2fd->hva); if (sz < 0) return sz; @@ -1585,14 +1596,27 @@ static enum kvm_pgtable_prot adjust_nested_exec_perms(struct kvm *kvm, return prot; } -struct kvm_s2_fault_desc { - struct kvm_vcpu *vcpu; - phys_addr_t fault_ipa; - struct kvm_s2_trans *nested; - struct kvm_memory_slot *memslot; - unsigned long hva; +struct kvm_s2_fault_vma_info { + unsigned long mmu_seq; + long vma_pagesize; + vm_flags_t vm_flags; + unsigned long max_map_size; + struct page *page; + kvm_pfn_t pfn; + gfn_t gfn; + bool device; + bool mte_allowed; + bool is_vma_cacheable; + bool map_writable; + bool map_non_cacheable; }; +static int kvm_s2_fault_get_vma_info(const struct kvm_s2_fault_desc *s2fd, + struct kvm_s2_fault_vma_info *s2vi); + +static gfn_t get_canonical_gfn(const struct kvm_s2_fault_desc *s2fd, + const struct kvm_s2_fault_vma_info *s2vi); + static int gmem_abort(const struct kvm_s2_fault_desc *s2fd) { bool write_fault, exec_fault; @@ -1600,12 +1624,10 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd) enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_SHARED; enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; struct kvm_pgtable *pgt = s2fd->vcpu->arch.hw_mmu->pgt; - unsigned long mmu_seq; - struct page *page; + struct kvm_s2_fault_vma_info s2vi = {}; struct kvm *kvm = s2fd->vcpu->kvm; void *memcache = NULL; - kvm_pfn_t pfn; - gfn_t gfn; + gfn_t canonical_gfn; int ret; if (!perm_fault) { @@ -1615,24 +1637,23 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd) return ret; } - if (s2fd->nested) - gfn = kvm_s2_trans_output(s2fd->nested) >> PAGE_SHIFT; - else - gfn = s2fd->fault_ipa >> PAGE_SHIFT; + s2vi.vma_pagesize = PAGE_SIZE; + s2vi.gfn = ALIGN_DOWN(s2fd->fault_ipa, s2vi.vma_pagesize) >> PAGE_SHIFT; + canonical_gfn = get_canonical_gfn(s2fd, &s2vi); write_fault = kvm_is_write_fault(s2fd->vcpu); exec_fault = kvm_vcpu_trap_is_exec_fault(s2fd->vcpu); VM_WARN_ON_ONCE(write_fault && exec_fault); - mmu_seq = kvm->mmu_invalidate_seq; + s2vi.mmu_seq = kvm->mmu_invalidate_seq; /* Pairs with the smp_wmb() in kvm_mmu_invalidate_end(). */ smp_rmb(); - ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, gfn, &pfn, &page, NULL); + ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, canonical_gfn, &s2vi.pfn, &s2vi.page, NULL); if (ret) { - kvm_prepare_memory_fault_exit(s2fd->vcpu, s2fd->fault_ipa, PAGE_SIZE, - write_fault, exec_fault, false); + kvm_prepare_memory_fault_exit(s2fd->vcpu, gfn_to_gpa(canonical_gfn), + s2vi.vma_pagesize, write_fault, exec_fault, false); return ret; } @@ -1649,7 +1670,7 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd) prot = adjust_nested_exec_perms(kvm, s2fd->nested, prot); kvm_fault_lock(kvm); - if (mmu_invalidate_retry(kvm, mmu_seq)) { + if (mmu_invalidate_retry(kvm, s2vi.mmu_seq)) { ret = -EAGAIN; goto out_unlock; } @@ -1660,48 +1681,34 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd) * PTE, which will be preserved. */ prot &= ~KVM_NV_GUEST_MAP_SZ; - ret = KVM_PGT_FN(kvm_pgtable_stage2_relax_perms)(pgt, s2fd->fault_ipa, + ret = KVM_PGT_FN(kvm_pgtable_stage2_relax_perms)(pgt, gfn_to_gpa(s2vi.gfn), prot, flags); } else { - ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, s2fd->fault_ipa, PAGE_SIZE, - __pfn_to_phys(pfn), prot, + ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, gfn_to_gpa(s2vi.gfn), + s2vi.vma_pagesize, + __pfn_to_phys(s2vi.pfn), prot, memcache, flags); } out_unlock: - kvm_release_faultin_page(kvm, page, !!ret, prot & KVM_PGTABLE_PROT_W); + kvm_release_faultin_page(kvm, s2vi.page, !!ret, prot & KVM_PGTABLE_PROT_W); kvm_fault_unlock(kvm); if ((prot & KVM_PGTABLE_PROT_W) && !ret) - mark_page_dirty_in_slot(kvm, s2fd->memslot, gfn); + mark_page_dirty_in_slot(kvm, s2fd->memslot, canonical_gfn); return ret != -EAGAIN ? ret : 0; } -struct kvm_s2_fault_vma_info { - unsigned long mmu_seq; - long vma_pagesize; - vm_flags_t vm_flags; - unsigned long max_map_size; - struct page *page; - kvm_pfn_t pfn; - gfn_t gfn; - bool device; - bool mte_allowed; - bool is_vma_cacheable; - bool map_writable; - bool map_non_cacheable; -}; - static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd) { unsigned int flags = FOLL_HWPOISON | FOLL_LONGTERM | FOLL_WRITE; + struct kvm_s2_fault_vma_info s2vi = {}; struct kvm_vcpu *vcpu = s2fd->vcpu; struct kvm_pgtable *pgt = vcpu->arch.hw_mmu->pgt; struct mm_struct *mm = current->mm; struct kvm *kvm = vcpu->kvm; void *hyp_memcache; - struct page *page; int ret; hyp_memcache = get_mmu_memcache(vcpu); @@ -1709,12 +1716,16 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd) if (ret) return -ENOMEM; + ret = kvm_s2_fault_get_vma_info(s2fd, &s2vi); + if (ret) + return ret; + ret = account_locked_vm(mm, 1, true); if (ret) return ret; mmap_read_lock(mm); - ret = pin_user_pages(s2fd->hva, 1, flags, &page); + ret = pin_user_pages(s2fd->hva, 1, flags, &s2vi.page); mmap_read_unlock(mm); if (ret == -EHWPOISON) { @@ -1724,7 +1735,7 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd) } else if (ret != 1) { ret = -EFAULT; goto dec_account; - } else if (!folio_test_swapbacked(page_folio(page))) { + } else if (!folio_test_swapbacked(page_folio(s2vi.page))) { /* * We really can't deal with page-cache pages returned by GUP * because (a) we may trigger writeback of a page for which we @@ -1744,8 +1755,8 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd) } write_lock(&kvm->mmu_lock); - ret = pkvm_pgtable_stage2_map(pgt, s2fd->fault_ipa, PAGE_SIZE, - page_to_phys(page), KVM_PGTABLE_PROT_RWX, + ret = pkvm_pgtable_stage2_map(pgt, gfn_to_gpa(s2vi.gfn), PAGE_SIZE, + page_to_phys(s2vi.page), KVM_PGTABLE_PROT_RWX, hyp_memcache, 0); write_unlock(&kvm->mmu_lock); if (ret) { @@ -1756,7 +1767,7 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd) return 0; unpin: - unpin_user_pages(&page, 1); + unpin_user_page(s2vi.page); dec_account: account_locked_vm(mm, 1, false); return ret; @@ -1779,7 +1790,7 @@ static short kvm_s2_resolve_vma_size(const struct kvm_s2_fault_desc *s2fd, switch (vma_shift) { #ifndef __PAGETABLE_PMD_FOLDED case PUD_SHIFT: - if (fault_supports_stage2_huge_mapping(s2fd->memslot, s2fd->hva, PUD_SIZE)) + if (fault_supports_stage2_huge_mapping(s2fd, PUD_SIZE)) break; fallthrough; #endif @@ -1787,7 +1798,7 @@ static short kvm_s2_resolve_vma_size(const struct kvm_s2_fault_desc *s2fd, vma_shift = PMD_SHIFT; fallthrough; case PMD_SHIFT: - if (fault_supports_stage2_huge_mapping(s2fd->memslot, s2fd->hva, PMD_SIZE)) + if (fault_supports_stage2_huge_mapping(s2fd, PMD_SIZE)) break; fallthrough; case CONT_PTE_SHIFT: @@ -2027,9 +2038,7 @@ static int kvm_s2_fault_map(const struct kvm_s2_fault_desc *s2fd, if (perm_fault_granule > PAGE_SIZE) { mapping_size = perm_fault_granule; } else { - mapping_size = transparent_hugepage_adjust(kvm, s2fd->memslot, - s2fd->hva, &pfn, - &gfn); + mapping_size = transparent_hugepage_adjust(s2fd, &pfn, &gfn); if (mapping_size < 0) { ret = mapping_size; goto out_unlock; diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c index 8e4c6e4bec12..c298a5b9b12a 100644 --- a/arch/arm64/kvm/pkvm.c +++ b/arch/arm64/kvm/pkvm.c @@ -414,6 +414,9 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, u64 end = addr + size; int ret; + if (WARN_ON_ONCE(!PAGE_ALIGNED(addr | size))) + return -EINVAL; + lockdep_assert_held_write(&kvm->mmu_lock); mapping = pkvm_mapping_iter_first(&pgt->pkvm_mappings, addr, end - 1); diff --git a/arch/arm64/kvm/vgic/vgic-mmio-v3.c b/arch/arm64/kvm/vgic/vgic-mmio-v3.c index 5ab789accc9d..9e6200c5adce 100644 --- a/arch/arm64/kvm/vgic/vgic-mmio-v3.c +++ b/arch/arm64/kvm/vgic/vgic-mmio-v3.c @@ -303,9 +303,19 @@ static void vgic_mmio_write_v3r_ctlr(struct kvm_vcpu *vcpu, if (ctlr != GICR_CTLR_ENABLE_LPIS) return; + /* + * Yes, disabling LPIs is painful, since it can be done from + * a *remote* vcpu! So let's not take any chance, and make + * sure that the target vCPU has written their LRs back to the + * irq structures, and release any reference they would have. + * + * If it hurts, don't do it. + */ + kvm_arm_halt_guest(vcpu->kvm); vgic_flush_pending_lpis(vcpu); vgic_its_invalidate_all_caches(vcpu->kvm); atomic_set_release(&vgic_cpu->ctlr, 0); + kvm_arm_resume_guest(vcpu->kvm); } else { ctlr = atomic_cmpxchg_acquire(&vgic_cpu->ctlr, 0, GICR_CTLR_ENABLE_LPIS); diff --git a/arch/arm64/mm/mmap.c b/arch/arm64/mm/mmap.c index 32e0771d6477..e212270d4e09 100644 --- a/arch/arm64/mm/mmap.c +++ b/arch/arm64/mm/mmap.c @@ -61,7 +61,9 @@ int valid_phys_addr_range(phys_addr_t addr, size_t size) */ int valid_mmap_phys_addr_range(unsigned long pfn, size_t size) { - return !(((pfn << PAGE_SHIFT) + size) & ~PHYS_MASK); + const phys_addr_t limit = PHYS_MASK + 1; + + return size <= limit && pfn <= (limit - size) >> PAGE_SHIFT; } static int __init adjust_protection_map(void) diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 12aacc74f764..b1516630904d 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -113,6 +113,10 @@ SYM_FUNC_START(cpu_do_suspend) alternative_if ARM64_HAS_TCR2 mrs x2, REG_TCR2_EL1 str x2, [x0, #104] +alternative_else_nop_endif +alternative_if ARM64_SME + mrs_s x2, SYS_TPIDR2_EL0 + str x2, [x0, #112] alternative_else_nop_endif ret SYM_FUNC_END(cpu_do_suspend) @@ -152,6 +156,10 @@ alternative_if ARM64_HAS_TCR2 ldr x2, [x0, #104] msr REG_TCR2_EL1, x2 alternative_else_nop_endif +alternative_if ARM64_SME + ldr x2, [x0, #112] + msr_s SYS_TPIDR2_EL0, x2 +alternative_else_nop_endif msr sctlr_el1, x12 set_this_cpu_offset x13 diff --git a/arch/mips/mm/cex-gen.S b/arch/mips/mm/cex-gen.S index e528583d1331..6989fe274aab 100644 --- a/arch/mips/mm/cex-gen.S +++ b/arch/mips/mm/cex-gen.S @@ -8,6 +8,7 @@ * * Cache error handler */ +#include #include #include #include @@ -37,6 +38,21 @@ nop nop +#if defined(CONFIG_64BIT) && VMLINUX_LOAD_ADDRESS >= XKPHYS && \ + VMLINUX_LOAD_ADDRESS < XKSEG + /* + * An XKPHYS kernel may lie outside the 256MB region a j from the + * vector reaches, so jump to the handler's uncached XKPHYS address. + */ + PTR_LA k0, cache_parity_error + PTR_LI k1, TO_PHYS_MASK + and k0, k0, k1 + PTR_LI k1, UNCAC_BASE + or k0, k0, k1 + jr k0 + nop +#else j cache_parity_error nop +#endif END(except_vec2_generic) diff --git a/arch/mips/pci/ops-mace.c b/arch/mips/pci/ops-mace.c index 951d8070fb48..8827e5f2ea46 100644 --- a/arch/mips/pci/ops-mace.c +++ b/arch/mips/pci/ops-mace.c @@ -35,15 +35,27 @@ static inline int mkaddr(struct pci_bus *bus, unsigned int devfn, (reg & 0xfc); } +/* + * On CRIME 1.1, a MACE PCI register read followed by two register writes + * can hang the machine under DMA load, so every register write in this + * file is followed by a read. Config cycles to empty slots end in a + * master abort; ack it with interrupts still off so the bridge error + * interrupt does not report it. + */ +static void mace_pci_ack_abort(void) +{ + u32 error = mace->pci.error; + + if (error & MACEPCI_ERROR_MASTER_ABORT) { + mace->pci.error = error & ~MACEPCI_ERROR_MASTER_ABORT; + mace->pci.error; + } +} static int mace_pci_read_config(struct pci_bus *bus, unsigned int devfn, int reg, int size, u32 *val) { - u32 control = mace->pci.control; - - /* disable master aborts interrupts during config read */ - mace->pci.control = control & ~MACEPCI_CONTROL_MAR_INT; mace->pci.config_addr = mkaddr(bus, devfn, reg); switch (size) { case 1: @@ -56,9 +68,7 @@ mace_pci_read_config(struct pci_bus *bus, unsigned int devfn, *val = mace->pci.config_data.l; break; } - /* ack possible master abort */ - mace->pci.error &= ~MACEPCI_ERROR_MASTER_ABORT; - mace->pci.control = control; + mace_pci_ack_abort(); /* * someone forgot to set the ultra bit for the onboard * scsi chips; we fake it here @@ -77,6 +87,7 @@ mace_pci_write_config(struct pci_bus *bus, unsigned int devfn, int reg, int size, u32 val) { mace->pci.config_addr = mkaddr(bus, devfn, reg); + mace->pci.error; switch (size) { case 1: mace->pci.config_data.b[(reg & 3) ^ 3] = val; @@ -88,6 +99,7 @@ mace_pci_write_config(struct pci_bus *bus, unsigned int devfn, mace->pci.config_data.l = val; break; } + mace_pci_ack_abort(); DPRINTK("write%d: reg=%08x,val=%02x\n", size * 8, reg, val); diff --git a/arch/mips/pci/pci-ip32.c b/arch/mips/pci/pci-ip32.c index 7ae89d0c7099..41ec8453c865 100644 --- a/arch/mips/pci/pci-ip32.c +++ b/arch/mips/pci/pci-ip32.c @@ -77,6 +77,8 @@ static irqreturn_t macepci_error(int irq, void *dev) } mace->pci.error = flags; + /* follow the write with a read, see mace_pci_ack_abort() */ + mace->pci.error; return IRQ_HANDLED; } @@ -125,10 +127,13 @@ static int __init mace_init(void) { PCIBIOS_MIN_IO = 0x1000; - /* Clear any outstanding errors and enable interrupts */ + /* + * Clear any outstanding errors and enable interrupts. PCI interrupts + * are masked in CRIME, so the per-slot enables stay on. + */ mace->pci.error_addr = 0; mace->pci.error = 0; - mace->pci.control = 0xff008500; + mace->pci.control = 0xff0085ff; printk("MACE PCI rev %d\n", mace->pci.rev); diff --git a/arch/mips/sgi-ip32/ip32-irq.c b/arch/mips/sgi-ip32/ip32-irq.c index 29d04468a06b..b9c72af481ed 100644 --- a/arch/mips/sgi-ip32/ip32-irq.c +++ b/arch/mips/sgi-ip32/ip32-irq.c @@ -160,38 +160,6 @@ static struct irq_chip crime_edge_interrupt = { .irq_unmask = crime_enable_irq, }; -/* - * This is for MACE PCI interrupts. We can decrease bus traffic by masking - * as close to the source as possible. This also means we can take the - * next chunk of the CRIME register in one piece. - */ - -static unsigned long macepci_mask; - -static void enable_macepci_irq(struct irq_data *d) -{ - macepci_mask |= MACEPCI_CONTROL_INT(d->irq - MACEPCI_SCSI0_IRQ); - mace->pci.control = macepci_mask; - crime_mask |= 1 << (d->irq - CRIME_IRQ_BASE); - crime->imask = crime_mask; -} - -static void disable_macepci_irq(struct irq_data *d) -{ - crime_mask &= ~(1 << (d->irq - CRIME_IRQ_BASE)); - crime->imask = crime_mask; - flush_crime_bus(); - macepci_mask &= ~MACEPCI_CONTROL_INT(d->irq - MACEPCI_SCSI0_IRQ); - mace->pci.control = macepci_mask; - flush_mace_bus(); -} - -static struct irq_chip ip32_macepci_interrupt = { - .name = "IP32 MACE PCI", - .irq_mask = disable_macepci_irq, - .irq_unmask = enable_macepci_irq, -}; - /* This is used for MACE ISA interrupts. That means bits 4-6 in the * CRIME register. */ @@ -299,8 +267,8 @@ static struct irq_chip ip32_maceisa_edge_interrupt = { .irq_unmask = enable_maceisa_irq, }; -/* This is used for regular non-ISA, non-PCI MACE interrupts. That means - * bits 0-3 and 7 in the CRIME register. +/* This is used for regular non-ISA MACE interrupts. That means bits 0-3 + * and 7-15 in the CRIME register. */ static void enable_mace_irq(struct irq_data *d) @@ -436,20 +404,13 @@ void __init arch_init_irq(void) mips_cpu_irq_init(); for (irq = CRIME_IRQ_BASE; irq <= IP32_IRQ_MAX; irq++) { switch (irq) { - case MACE_VID_IN1_IRQ ... MACE_PCI_BRIDGE_IRQ: + case MACE_VID_IN1_IRQ ... MACEPCI_SHARED2_IRQ: irq_set_chip_and_handler_name(irq, &ip32_mace_interrupt, handle_level_irq, "level"); break; - case MACEPCI_SCSI0_IRQ ... MACEPCI_SHARED2_IRQ: - irq_set_chip_and_handler_name(irq, - &ip32_macepci_interrupt, - handle_level_irq, - "level"); - break; - case CRIME_CPUERR_IRQ: case CRIME_MEMERR_IRQ: irq_set_chip_and_handler_name(irq, diff --git a/arch/openrisc/kernel/or32_ksyms.c b/arch/openrisc/kernel/or32_ksyms.c index 84a937a64e2a..5aabef30ca09 100644 --- a/arch/openrisc/kernel/or32_ksyms.c +++ b/arch/openrisc/kernel/or32_ksyms.c @@ -35,6 +35,7 @@ DECLARE_EXPORT(__divsi3); DECLARE_EXPORT(__umodsi3); DECLARE_EXPORT(__modsi3); DECLARE_EXPORT(__muldi3); +DECLARE_EXPORT(__mulsi3); DECLARE_EXPORT(__ashrdi3); DECLARE_EXPORT(__ashldi3); DECLARE_EXPORT(__lshrdi3); diff --git a/arch/x86/boot/startup/sme.c b/arch/x86/boot/startup/sme.c index c07a2c381ed1..2d5cb3f7eee6 100644 --- a/arch/x86/boot/startup/sme.c +++ b/arch/x86/boot/startup/sme.c @@ -99,7 +99,7 @@ static void __init sme_clear_pgd(struct sme_populate_pgd_data *ppd) pgd_t *pgd_p; pgd_start = ppd->vaddr & PGDIR_MASK; - pgd_end = ppd->vaddr_end & PGDIR_MASK; + pgd_end = (ppd->vaddr_end - 1) & PGDIR_MASK; pgd_size = (((pgd_end - pgd_start) / PGDIR_SIZE) + 1) * sizeof(pgd_t); diff --git a/arch/x86/entry/syscall_32.c b/arch/x86/entry/syscall_32.c index 91123a90710c..4c36e8db0e34 100644 --- a/arch/x86/entry/syscall_32.c +++ b/arch/x86/entry/syscall_32.c @@ -77,6 +77,10 @@ static __always_inline void do_syscall_32_irqs_on(struct pt_regs *regs, unsigned if (likely(nr < IA32_NR_syscalls)) { nr = array_index_nospec(nr, IA32_NR_syscalls); regs->ax = ia32_sys_call(regs, (unsigned int)nr); + } else { + /* See comment in do_syscall_64() */ + if (nr != -1UL) + regs->ax = -ENOSYS; } } diff --git a/arch/x86/entry/syscall_64.c b/arch/x86/entry/syscall_64.c index 966d3d1b7586..bb95ad0ab786 100644 --- a/arch/x86/entry/syscall_64.c +++ b/arch/x86/entry/syscall_64.c @@ -64,15 +64,20 @@ static __always_inline bool do_syscall_x64(struct pt_regs *regs, unsigned long n return false; } -static __always_inline void do_syscall_x32(struct pt_regs *regs, unsigned long nr) +static __always_inline bool do_syscall_x32(struct pt_regs *regs, unsigned long nr) { + if (!IS_ENABLED(CONFIG_X86_X32_ABI)) + return false; + /* Adjust the starting offset of the table */ nr -= __X32_SYSCALL_BIT; - if (IS_ENABLED(CONFIG_X86_X32_ABI) && likely(nr < X32_NR_syscalls)) { + if (likely(nr < X32_NR_syscalls)) { nr = array_index_nospec(nr, X32_NR_syscalls); regs->ax = x32_sys_call(regs, (unsigned int)nr); + return true; } + return false; } /* Returns true to return using SYSRET, or false to use IRET */ @@ -81,8 +86,21 @@ __visible noinstr bool do_syscall_64(struct pt_regs *regs, long nr) if (likely(syscall_enter_from_user_mode_randomize_stack(regs, &nr))) { instrumentation_begin(); - if (!do_syscall_x64(regs, nr)) - do_syscall_x32(regs, nr); + if (unlikely(!do_syscall_x64(regs, nr))) { + /* + * The special treatment of syscall number -1 is a + * historical bogosity as the original code used -1 to + * indicate syscall abort, which is not distinguishable + * from a syscall which is invoked with syscall number + * -1. The historic behavior is to keep the return value + * in regs->ax unmodified in that case, which means it's + * either the preset value -ENOSYS or a value which was + * written by a tracer. If the syscall number is invalid + * but not -1 then overwrite any tracer modification. + */ + if (!do_syscall_x32(regs, nr) && nr != -1L) + regs->ax = -ENOSYS; + } instrumentation_end(); } diff --git a/arch/x86/include/asm/checksum_32.h b/arch/x86/include/asm/checksum_32.h index 17da95387997..f74873a5e351 100644 --- a/arch/x86/include/asm/checksum_32.h +++ b/arch/x86/include/asm/checksum_32.h @@ -159,7 +159,7 @@ static inline __sum16 csum_ipv6_magic(const struct in6_addr *saddr, "adcl $0, %0 ;\n" : "=&r" (sum) : "r" (saddr), "r" (daddr), - "r" (htonl(len)), "r" (htonl(proto)), "0" (sum) + "g" (htonl(len)), "g" (htonl(proto)), "0" (sum) : "memory"); return csum_fold(sum); diff --git a/arch/x86/include/asm/vmware.h b/arch/x86/include/asm/vmware.h index 4220dae14a2d..fb011cef01e7 100644 --- a/arch/x86/include/asm/vmware.h +++ b/arch/x86/include/asm/vmware.h @@ -276,20 +276,22 @@ unsigned long vmware_hypercall_hb_out(unsigned long cmd, unsigned long in2, asm_inline volatile ( UNWIND_HINT_SAVE + "mov %[in6], %%" _ASM_AX "\n\t" "push %%" _ASM_BP "\n\t" UNWIND_HINT_UNDEFINED - "mov %[in6], %%" _ASM_BP "\n\t" + "mov %%" _ASM_AX ", %%" _ASM_BP "\n\t" + "mov %[magic], %%eax\n\t" "rep outsb\n\t" "pop %%" _ASM_BP "\n\t" UNWIND_HINT_RESTORE - : "=a" (out0), "=b" (*out1) - : "a" (VMWARE_HYPERVISOR_MAGIC), - "b" (cmd), + : "=&a" (out0), "=b" (*out1) + : "b" (cmd), "c" (in2), "d" (in3 | VMWARE_HYPERVISOR_PORT_HB), "S" (in4), "D" (in5), - [in6] VMW_BP_CONSTRAINT (in6) + [in6] VMW_BP_CONSTRAINT(in6), + [magic] "i" (VMWARE_HYPERVISOR_MAGIC) : "cc", "memory"); return out0; } @@ -304,20 +306,22 @@ unsigned long vmware_hypercall_hb_in(unsigned long cmd, unsigned long in2, asm_inline volatile ( UNWIND_HINT_SAVE + "mov %[in6], %%" _ASM_AX "\n\t" "push %%" _ASM_BP "\n\t" UNWIND_HINT_UNDEFINED - "mov %[in6], %%" _ASM_BP "\n\t" + "mov %%" _ASM_AX ", %%" _ASM_BP "\n\t" + "mov %[magic], %%eax\n\t" "rep insb\n\t" "pop %%" _ASM_BP "\n\t" UNWIND_HINT_RESTORE - : "=a" (out0), "=b" (*out1) - : "a" (VMWARE_HYPERVISOR_MAGIC), - "b" (cmd), + : "=&a" (out0), "=b" (*out1) + : "b" (cmd), "c" (in2), "d" (in3 | VMWARE_HYPERVISOR_PORT_HB), "S" (in4), "D" (in5), - [in6] VMW_BP_CONSTRAINT (in6) + [in6] VMW_BP_CONSTRAINT(in6), + [magic] "i" (VMWARE_HYPERVISOR_MAGIC) : "cc", "memory"); return out0; } diff --git a/arch/x86/kernel/acpi/cppc.c b/arch/x86/kernel/acpi/cppc.c index bbade0da5130..174b734e8261 100644 --- a/arch/x86/kernel/acpi/cppc.c +++ b/arch/x86/kernel/acpi/cppc.c @@ -6,6 +6,8 @@ #include +#include + #include #include #include @@ -45,10 +47,20 @@ bool cpc_ffh_supported(void) return true; } +static bool cpc_ffh_reg_valid(const struct cpc_reg *reg) +{ + return reg->address <= U32_MAX && reg->bit_width && + reg->bit_width <= 64 && + reg->bit_offset <= 64 - reg->bit_width; +} + int cpc_read_ffh(int cpunum, struct cpc_reg *reg, u64 *val) { int err; + if (!cpc_ffh_reg_valid(reg)) + return -EINVAL; + err = rdmsrq_safe_on_cpu(cpunum, reg->address, val); if (!err) { u64 mask = GENMASK_ULL(reg->bit_offset + reg->bit_width - 1, @@ -65,6 +77,9 @@ int cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val) u64 rd_val; int err; + if (!cpc_ffh_reg_valid(reg)) + return -EINVAL; + err = rdmsrq_safe_on_cpu(cpunum, reg->address, &rd_val); if (!err) { u64 mask = GENMASK_ULL(reg->bit_offset + reg->bit_width - 1, diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index 000ebee2d247..4ced10c6ba0d 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -2088,14 +2088,12 @@ static void __init bhi_select_mitigation(void) if (bhi_mitigation != BHI_MITIGATION_AUTO) return; - if (cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST)) { - if (cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL)) - bhi_mitigation = BHI_MITIGATION_ON; - else - bhi_mitigation = BHI_MITIGATION_VMEXIT_ONLY; - } else { + if (cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL)) + bhi_mitigation = BHI_MITIGATION_ON; + else if (cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST)) + bhi_mitigation = BHI_MITIGATION_VMEXIT_ONLY; + else bhi_mitigation = BHI_MITIGATION_OFF; - } } static void __init bhi_update_mitigation(void) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index e3cfa0c10e92..ffd74a68671b 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -214,8 +214,8 @@ union l3_qos_abmc_cfg { bw_src :12, reserved1: 3, is_clos : 1, - cntr_id : 5, - reserved : 9, + cntr_id :12, + reserved : 2, cntr_en : 1, cfg_en : 1; } split; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 3838e0a13d36..a6a9090c4808 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -470,8 +470,8 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r) r->mon.mbm_cntr_assignable = true; r->mon.mbm_cntr_configurable = true; cpuid_count(0x80000020, 5, &eax, &ebx, &ecx, &edx); - r->mon.num_mbm_cntrs = (ebx & GENMASK(15, 0)) + 1; - hw_res->mbm_cntr_assign_enabled = true; + /* cntr_id is 12 bits and can only encode 4096 counters. */ + r->mon.num_mbm_cntrs = min((ebx & GENMASK(15, 0)) + 1, BIT(12)); } r->mon_capable = true; diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index 3c9f60d6ca5a..cb46da482c85 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -115,6 +115,14 @@ static void nmi_check_duration(struct nmiaction *action, u64 duration) { int remainder_ns, decimal_msecs; + /* sched_clock() went backwards? */ + if ((s64)duration < 0) { + pr_warn_once("NMI handler (%ps): sched_clock() went backwards by %lld ns on CPU%d\n", + action->handler, -(s64)duration, + smp_processor_id()); + return; + } + if (duration < nmi_longest_ns || duration < action->max_duration) return; diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c index 252c5827d063..3b9b41820c84 100644 --- a/arch/x86/kernel/x86_init.c +++ b/arch/x86/kernel/x86_init.c @@ -37,7 +37,7 @@ static void iommu_shutdown_noop(void) { } bool __init bool_x86_init_noop(void) { return false; } void x86_op_int_noop(int cpu) { } int set_rtc_noop(const struct timespec64 *now) { return -EINVAL; } -void get_rtc_noop(struct timespec64 *now) { } +void get_rtc_noop(struct timespec64 *now) { *now = (struct timespec64){0}; } static __initconst const struct of_device_id of_cmos_match[] = { { .compatible = "motorola,mc146818" }, diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e56497089b59..2c2ff20f8ad0 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2310,6 +2310,16 @@ static struct kvm_mmu_page *kvm_mmu_find_shadow_page(struct kvm *kvm, continue; } + /* + * Note! Unlike fully direct MMUs, direct SPs in indirect MMUs + * can have an arbitrary number of parents, e.g if the guest is + * mapping a huge PTE at many points in its page tables. + */ + if (pte_list_count(&sp->parent_ptes) >= RMAP_RECYCLE_THRESHOLD) { + kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list); + break; + } + /* unsync and write-flooding only apply to indirect SPs. */ if (sp->role.direct) goto out; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 6e0153f23136..7454dac59b3a 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -339,6 +339,13 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, gpa_t real_gpa; gfn_t gfn; + /* + * KVM might sleep/resched when retrying the walk if the updating A/D + * fails because the gPTE changed between reading the gPTE and trying + * the CMPXHG to set A/D bits. + */ + might_sleep(); + trace_kvm_mmu_pagetable_walk(addr, access); retry_walk: walker->level = w->cpu_role.base.level; @@ -479,8 +486,10 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, addr, write_fault); if (unlikely(ret < 0)) goto error; - else if (ret) + else if (ret) { + cond_resched(); goto retry_walk; + } } return 1; diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 5165275128fc..a1450c55e946 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -5059,6 +5059,7 @@ static void nested_vmx_restore_host_state(struct kvm_vcpu *vcpu) __func__, j, h.index, h.value); goto vmabort; } + break; } } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c76498d1a162..56a19a7260d3 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6515,10 +6515,12 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, } else if (vcpu->arch.complete_userspace_io) { writeback = false; r = 0; - } else if (r == EMULATION_RESTART) + } else if (r == EMULATION_RESTART) { + cond_resched(); goto restart; - else + } else { r = 1; + } writeback: if (writeback) { @@ -9661,18 +9663,23 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) kvm_x86_call(set_efer)(vcpu, 0); kvm_x86_call(update_exception_bitmap)(vcpu); + if (old_cr0 & X86_CR0_PG) + kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); + /* - * On the standard CR0/CR4/EFER modification paths, there are several - * complex conditions determining whether the MMU has to be reset and/or - * which PCIDs have to be flushed. However, CR0.WP and the paging-related - * bits in CR4 and EFER are irrelevant if CR0.PG was '0'; and a reset+flush - * is needed anyway if CR0.PG was '1' (which can only happen for INIT, as - * CR0 will be '0' prior to RESET). So we only need to check CR0.PG here. + * Unconditionally reset the MMU, even though super strictly speaking, + * a MMU reset is only necessary if and only one of the page role bits + * is changing. All of the CR0/CR4/EFER bits effectively get covered + * by checking CR0.PG=1, as CR0.WP and the paging-related bits in CR4 + * and EFER are irrelevant if CR0.PG was '0'. Subtly, if the vCPU was + * in guest mode (L2) or SMM (although emulating INIT in SMM _should_ + * be impossible), an MMU reset is also necessary as "guest mode" and + * "smm" are both encoded in the shadow page role. As a result, the + * only scenarios where an MMU reset is *not* needed is if the vCPU is + * in Real Mode or Unpaged Protected Mode, and micro-optimizing those + * paths are not worth the risk of running with a stale MMU context. */ - if (old_cr0 & X86_CR0_PG) { - kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); - kvm_mmu_reset_context(vcpu); - } + kvm_mmu_reset_context(vcpu); /* * Intel's SDM states that all TLB entries are flushed on INIT. AMD's diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 0c4da320831c..53dcab39566f 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -320,13 +320,19 @@ static u16 allocate_global_asid(void) lockdep_assert_held(&global_asid_lock); - /* The previous allocation hit the edge of available address space */ - if (last_global_asid >= MAX_ASID_AVAILABLE - 1) - reset_global_asid_space(); - asid = find_next_zero_bit(global_asid_used, MAX_ASID_AVAILABLE, last_global_asid); - if (asid >= MAX_ASID_AVAILABLE && !global_asid_available) { + /* + * Nothing is free between last_global_asid and MAX_ASID_AVAILABLE. + * Flush lazy-freed ASIDs and reset the search space. + */ + if (asid >= MAX_ASID_AVAILABLE) { + reset_global_asid_space(); + asid = find_next_zero_bit(global_asid_used, MAX_ASID_AVAILABLE, + last_global_asid); + } + + if (asid >= MAX_ASID_AVAILABLE) { /* This should never happen. */ VM_WARN_ONCE(1, "Unable to allocate global ASID despite %d available\n", global_asid_available); diff --git a/block/bio.c b/block/bio.c index f95b63c0604a..90104e6abbcf 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1285,6 +1285,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter, unsigned mem_align_mask, unsigned len_align_mask) { iov_iter_extraction_t flags = 0; + struct iov_iter saved_iter; if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED))) return -EIO; @@ -1303,6 +1304,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter, if (bio->bi_bdev && blk_queue_pci_p2pdma(bio->bi_bdev->bd_disk->queue)) flags |= ITER_ALLOW_P2PDMA; + saved_iter = *iter; do { ssize_t ret; @@ -1320,6 +1322,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter, bio_release_pages(bio, false); bio_clear_flag(bio, BIO_PAGE_PINNED); bio->bi_vcnt = 0; + *iter = saved_iter; } if (!bio->bi_vcnt) return ret; diff --git a/block/blk-iocost.c b/block/blk-iocost.c index 7b8af094ab37..23445d701433 100644 --- a/block/blk-iocost.c +++ b/block/blk-iocost.c @@ -2851,6 +2851,9 @@ static void ioc_rqos_done(struct rq_qos *rqos, struct request *rq) if (!ioc->enabled || !rq->alloc_time_ns || !rq->start_time_ns) return; + if (unlikely(rq->rq_flags & RQF_FLUSH_SEQ)) + return; + switch (req_op(rq)) { case REQ_OP_READ: pidx = QOS_RLAT; diff --git a/block/blk-mq.c b/block/blk-mq.c index 5a5dc66fc3b1..e21d69767fd4 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -3193,16 +3193,15 @@ void blk_mq_submit_bio(struct bio *bio) blk_mq_bio_to_request(rq, bio, nr_segs); + if (bio_zone_write_plugging(bio)) + blk_zone_write_plug_init_request(rq); + ret = blk_crypto_rq_get_keyslot(rq); if (ret != BLK_STS_OK) { - bio_endio_status(bio, ret); - blk_mq_free_request(rq); + blk_mq_end_request(rq, ret); return; } - if (bio_zone_write_plugging(bio)) - blk_zone_write_plug_init_request(rq); - if (op_is_flush(bio->bi_opf) && blk_insert_flush(rq)) return; @@ -3485,8 +3484,10 @@ void blk_mq_free_rqs(struct blk_mq_tag_set *set, struct blk_mq_tags *tags, if (blk_mq_is_shared_tags(set->flags)) drv_tags = set->shared_tags; - else + else if (hctx_idx < set->nr_hw_queues) drv_tags = set->tags[hctx_idx]; + else + drv_tags = NULL; if (tags->static_rqs && set->ops->exit_request) { int i; diff --git a/block/elevator.c b/block/elevator.c index 2161b6eea680..66edf1a5768a 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -812,25 +812,29 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf, * reference during concurrent disk deletion: * update_nr_hwq_lock -> kn->active (via del_gendisk -> kobject_del) * kn->active -> update_nr_hwq_lock (via this sysfs write path) - * - * Use the writer lock instead of the reader lock of update_nr_hwq_lock - * to serialize the two-stage elevator switch steps in - * elevator_change(): the core switch step under the elevator lock and - * the elevator_change_done() step outside the elevator lock. */ - if (!down_write_trylock(&set->update_nr_hwq_lock)) { + if (!down_read_trylock(&set->update_nr_hwq_lock)) { ret = -EBUSY; goto out; } - if (!blk_queue_no_elv_switch(q)) { - ret = elevator_change(q, &ctx); - if (!ret) - ret = count; - } else { - ret = -ENOENT; + + /* + * The reader lock is shared, so use QUEUE_FLAG_NO_ELV_SWITCH to allow + * only one switch of this queue at a time, and none once the disk is + * going away. + */ + if (test_and_set_bit(QUEUE_FLAG_NO_ELV_SWITCH, &q->queue_flags)) { + ret = -EBUSY; + goto out_unlock; } - up_write(&set->update_nr_hwq_lock); + ret = elevator_change(q, &ctx); + if (!ret) + ret = count; + + blk_queue_flag_clear(QUEUE_FLAG_NO_ELV_SWITCH, q); +out_unlock: + up_read(&set->update_nr_hwq_lock); out: if (ctx.type) elevator_put(ctx.type); diff --git a/block/ioctl.c b/block/ioctl.c index 64b4e6c0f696..fb2f4b4d2505 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -835,8 +835,8 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg) return compat_put_ulong(argp, bdev_nr_sectors(bdev)); /* The data is compatible, but the command number is different */ - case BLKBSZGET_32: /* get the logical block size (cf. BLKSSZGET) */ - return put_int(argp, bdev_logical_block_size(bdev)); + case BLKBSZGET_32: /* get the soft block size (cf. BLKSSZGET) */ + return put_int(argp, block_size(bdev)); case BLKBSZSET_32: return blkdev_bszset(file, mode, argp); case BLKGETSIZE64_32: diff --git a/drivers/acpi/battery.c b/drivers/acpi/battery.c index 670853ec3a4d..7169ac7e83f3 100644 --- a/drivers/acpi/battery.c +++ b/drivers/acpi/battery.c @@ -583,9 +583,8 @@ static int acpi_battery_get_info(struct acpi_battery *battery) struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL }; acpi_status status = AE_ERROR; - status = acpi_evaluate_object(battery->device->handle, - use_bix ? "_BIX":"_BIF", - NULL, &buffer); + status = acpi_evaluate_object_exclusive(battery->device->handle, + use_bix ? "_BIX" : "_BIF", NULL, &buffer); if (ACPI_FAILURE(status)) { acpi_handle_info(battery->device->handle, @@ -624,8 +623,7 @@ static int acpi_battery_get_state(struct acpi_battery *battery) msecs_to_jiffies(cache_time))) return 0; - status = acpi_evaluate_object(battery->device->handle, "_BST", - NULL, &buffer); + status = acpi_evaluate_object_exclusive(battery->device->handle, "_BST", NULL, &buffer); if (ACPI_FAILURE(status)) { acpi_handle_info(battery->device->handle, "_BST evaluation failed: %s", @@ -675,8 +673,8 @@ static int acpi_battery_set_alarm(struct acpi_battery *battery) !test_bit(ACPI_BATTERY_ALARM_PRESENT, &battery->flags)) return -ENODEV; - status = acpi_execute_simple_method(battery->device->handle, "_BTP", - battery->alarm); + status = acpi_execute_simple_method_exclusive(battery->device->handle, "_BTP", + battery->alarm); if (ACPI_FAILURE(status)) return -ENODEV; diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c index fef54fcd00b7..8aa0ce556863 100644 --- a/drivers/acpi/cppc_acpi.c +++ b/drivers/acpi/cppc_acpi.c @@ -36,6 +36,7 @@ #include #include #include +#include #include #include #include @@ -93,6 +94,16 @@ static DEFINE_PER_CPU(int, cpu_pcc_subspace_idx); */ static DEFINE_PER_CPU(struct cpc_desc *, cpc_desc_ptr); +/* Protect immutable capability queries against descriptor removal. */ +static DEFINE_MUTEX(cpc_desc_lock); + +static void cpc_set_desc(unsigned int cpu, struct cpc_desc *desc) +{ + mutex_lock(&cpc_desc_lock); + per_cpu(cpc_desc_ptr, cpu) = desc; + mutex_unlock(&cpc_desc_lock); +} + /* pcc mapped address + header size + offset within PCC subspace */ #define GET_PCC_VADDR(offs, pcc_ss_id) (pcc_data[pcc_ss_id]->pcc_channel->shmem + \ 0x8 + (offs)) @@ -999,12 +1010,12 @@ int acpi_cppc_processor_probe(struct acpi_processor *pr) } /* Plug PSD data into this CPU's CPC descriptor. */ - per_cpu(cpc_desc_ptr, pr->id) = cpc_ptr; + cpc_set_desc(pr->id, cpc_ptr); ret = kobject_init_and_add(&cpc_ptr->kobj, &cppc_ktype, &cpu_dev->kobj, "acpi_cppc"); if (ret) { - per_cpu(cpc_desc_ptr, pr->id) = NULL; + cpc_set_desc(pr->id, NULL); kobject_put(&cpc_ptr->kobj); goto out_free; } @@ -1056,6 +1067,8 @@ void acpi_cppc_processor_exit(struct acpi_processor *pr) if (!cpc_ptr) return; + cpc_set_desc(pr->id, NULL); + /* Free all the mapped sys mem areas for this CPU */ for (i = 2; i < cpc_ptr->num_entries; i++) { addr = cpc_ptr->cpc_regs[i-2].sys_mem_vaddr; @@ -1133,6 +1146,9 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val) } *val = 0; + if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) + return cpc_read_ffh(cpu, reg, val); + size = GET_BIT_WIDTH(reg); if (IS_ENABLED(CONFIG_HAS_IOPORT) && @@ -1160,8 +1176,6 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val) } else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY) vaddr = reg_res->sys_mem_vaddr; - else if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) - return cpc_read_ffh(cpu, reg, val); else return acpi_os_read_memory((acpi_physical_address)reg->address, val, size); @@ -1208,6 +1222,9 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val) unsigned long flags; bool locked = false; + if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) + return cpc_write_ffh(cpu, reg, val); + size = GET_BIT_WIDTH(reg); if (IS_ENABLED(CONFIG_HAS_IOPORT) && @@ -1233,8 +1250,6 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val) } else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY) vaddr = reg_res->sys_mem_vaddr; - else if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) - return cpc_write_ffh(cpu, reg, val); else return acpi_os_write_memory((acpi_physical_address)reg->address, val, size); @@ -1400,6 +1415,11 @@ static int cppc_set_reg_val(int cpu, enum cppc_regs reg_idx, u64 val) reg = &cpc_desc->cpc_regs[reg_idx]; + /* Integer 1 describes autonomous selection that is always enabled. */ + if (reg_idx == AUTO_SEL_ENABLE && reg->type == ACPI_TYPE_INTEGER && + reg->cpc_entry.int_value == 1) + return val == 1 ? 0 : -EOPNOTSUPP; + /* if a register is writeable, it must be a buffer and not null */ if ((reg->type != ACPI_TYPE_BUFFER) || IS_NULL_REG(®->cpc_entry.reg)) { pr_debug("CPC register is not supported\n"); @@ -1613,6 +1633,9 @@ bool cppc_perf_ctrs_in_pcc_cpu(unsigned int cpu) { struct cpc_desc *cpc_desc = per_cpu(cpc_desc_ptr, cpu); + if (!cpc_desc) + return false; + return CPC_IN_PCC(&cpc_desc->cpc_regs[DELIVERED_CTR]) || CPC_IN_PCC(&cpc_desc->cpc_regs[REFERENCE_CTR]) || CPC_IN_PCC(&cpc_desc->cpc_regs[CTR_WRAP_TIME]); @@ -1924,6 +1947,32 @@ int cppc_get_auto_sel(int cpu, bool *enable) } EXPORT_SYMBOL_GPL(cppc_get_auto_sel); +/** + * cppc_auto_sel_is_immutable - Check for always-enabled autonomous selection. + * @cpu: CPU whose _CPC descriptor to check. + * + * Context: Process context. + * Return: true for Integer 1, false for a register or an absent descriptor. + */ +bool cppc_auto_sel_is_immutable(int cpu) +{ + struct cpc_desc *cpc_desc; + struct cpc_register_resource *reg; + bool immutable = false; + + mutex_lock(&cpc_desc_lock); + cpc_desc = per_cpu(cpc_desc_ptr, cpu); + if (cpc_desc) { + reg = &cpc_desc->cpc_regs[AUTO_SEL_ENABLE]; + immutable = reg->type == ACPI_TYPE_INTEGER && + reg->cpc_entry.int_value == 1; + } + mutex_unlock(&cpc_desc_lock); + + return immutable; +} +EXPORT_SYMBOL_GPL(cppc_auto_sel_is_immutable); + /** * cppc_set_auto_sel - Write autonomous selection register. * @cpu : CPU to which to write register. diff --git a/drivers/acpi/utils.c b/drivers/acpi/utils.c index d499b72574ab..ea8cb876c12d 100644 --- a/drivers/acpi/utils.c +++ b/drivers/acpi/utils.c @@ -10,6 +10,7 @@ #include #include +#include #include #include #include @@ -243,6 +244,28 @@ acpi_extract_package(union acpi_object *package, EXPORT_SYMBOL(acpi_extract_package); +static DEFINE_MUTEX(acpi_exclusive_eval_lock); + +/** + * acpi_evaluate_object_exclusive - Evaluate an object without interleaving + * @handle: ACPI handle of the object or its scope. + * @pathname: Path of the object relative to @handle. + * @arguments: Arguments to pass, or NULL. + * @buffer: Buffer for the return value, or NULL. + * + * Same as acpi_evaluate_object(), but serialized with other exclusive + * evaluations. Use it only for methods known to break when interleaved. + */ +acpi_status acpi_evaluate_object_exclusive(acpi_handle handle, acpi_string pathname, + struct acpi_object_list *arguments, + struct acpi_buffer *buffer) +{ + guard(mutex)(&acpi_exclusive_eval_lock); + + return acpi_evaluate_object(handle, pathname, arguments, buffer); +} +EXPORT_SYMBOL_GPL(acpi_evaluate_object_exclusive); + acpi_status acpi_evaluate_integer(acpi_handle handle, acpi_string pathname, @@ -277,6 +300,16 @@ acpi_evaluate_integer(acpi_handle handle, EXPORT_SYMBOL(acpi_evaluate_integer); +acpi_status acpi_evaluate_integer_exclusive(acpi_handle handle, acpi_string pathname, + struct acpi_object_list *arguments, + unsigned long long *data) +{ + guard(mutex)(&acpi_exclusive_eval_lock); + + return acpi_evaluate_integer(handle, pathname, arguments, data); +} +EXPORT_SYMBOL_GPL(acpi_evaluate_integer_exclusive); + int acpi_get_local_u64_address(acpi_handle handle, u64 *addr) { acpi_status status; @@ -685,6 +718,14 @@ acpi_status acpi_execute_simple_method(acpi_handle handle, char *method, } EXPORT_SYMBOL(acpi_execute_simple_method); +acpi_status acpi_execute_simple_method_exclusive(acpi_handle handle, char *method, u64 arg) +{ + guard(mutex)(&acpi_exclusive_eval_lock); + + return acpi_execute_simple_method(handle, method, arg); +} +EXPORT_SYMBOL_GPL(acpi_execute_simple_method_exclusive); + /** * acpi_evaluate_ej0: Evaluate _EJ0 method for hotplug operations * @handle: ACPI device handle diff --git a/drivers/ata/libata-pmp.c b/drivers/ata/libata-pmp.c index 904f418387d1..43991e54eb08 100644 --- a/drivers/ata/libata-pmp.c +++ b/drivers/ata/libata-pmp.c @@ -482,6 +482,20 @@ static void sata_pmp_quirks(struct ata_port *ap) if (link->pmp == 4) link->flags |= ATA_LFLAG_DISABLED; } + } else if (vendor == 0x1b4b && devid == 0x9705) { + /* + * Marvell 88SM9705 quirks + * + * 1-to-5 PMP with an extra SEMB port. It reports 6 ports and + * port 5 is the SEMB, which never answers SRST: EH spends about + * a minute per round retrying it, and disk discovery waits for + * that. Seen behind a Marvell 88SE9235 in the Synology DX1215 + * and DX1222. + */ + ata_for_each_link(link, ap, EDGE) { + if (link->pmp == 5) + link->flags |= ATA_LFLAG_DISABLED; + } } } diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c index b86ac50f16b9..023de0e633b7 100644 --- a/drivers/ata/libata-scsi.c +++ b/drivers/ata/libata-scsi.c @@ -1984,6 +1984,18 @@ static int ata_scsi_translate(struct ata_device *dev, struct scsi_cmnd *cmd, if (xlat_func(qc)) goto done; + /* + * A data-out request may contain pages pinned without write access. + * Reject a translated data-in command before issuing it, since its + * completion handler may write to the buffer even if the device fails. + */ + if (unlikely(cmd->sc_data_direction == DMA_TO_DEVICE && + ata_is_data(qc->tf.protocol) && + !(qc->tf.flags & ATA_TFLAG_WRITE))) { + ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0x4b, 0x0a); + goto done; + } + return ata_scsi_qc_issue(ap, qc); done: @@ -2015,6 +2027,12 @@ static void ata_scsi_rbuf_fill(struct ata_device *dev, struct scsi_cmnd *cmd, unsigned long flags; unsigned int len; + /* A data-out request may contain pages pinned without write access. */ + if (cmd->sc_data_direction == DMA_TO_DEVICE) { + ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0x4b, 0x0a); + return; + } + spin_lock_irqsave(&ata_scsi_rbuf_lock, flags); memset(ata_scsi_rbuf, 0, ATA_SCSI_RBUF_SIZE); @@ -3108,6 +3126,13 @@ static unsigned int atapi_xlat(struct ata_queued_cmd *qc) qc->complete_fn = atapi_qc_complete; + /* atapi_fixup_inquiry() writes to the buffer on successful completion. */ + if (scmd->sc_data_direction == DMA_TO_DEVICE && + scmd->cmnd[0] == INQUIRY) { + ata_scsi_set_sense(dev, scmd, ABORTED_COMMAND, 0x4b, 0x0a); + return 1; + } + qc->tf.flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_DEVICE; if (scmd->sc_data_direction == DMA_TO_DEVICE) { qc->tf.flags |= ATA_TFLAG_WRITE; diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index 1ef01ce18f4f..959e6594013a 100644 --- a/drivers/block/ublk_drv.c +++ b/drivers/block/ublk_drv.c @@ -5833,6 +5833,7 @@ static int ublk_ctrl_uring_cmd_permission(struct ublk_device *ub, case UBLK_CMD_STOP_DEV: case UBLK_CMD_ADD_DEV: case UBLK_CMD_DEL_DEV: + case UBLK_CMD_DEL_DEV_ASYNC: case UBLK_CMD_SET_PARAMS: case UBLK_CMD_START_USER_RECOVERY: case UBLK_CMD_END_USER_RECOVERY: diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c index c8bfbf8435ac..10ba8b0b863b 100644 --- a/drivers/block/virtio_blk.c +++ b/drivers/block/virtio_blk.c @@ -1596,7 +1596,7 @@ static int virtblk_freeze_priv(struct virtio_device *vdev) /* Ensure no requests in virtqueues before deleting vqs. */ memflags = blk_mq_freeze_queue(q); - blk_mq_quiesce_queue_nowait(q); + blk_mq_quiesce_queue(q); blk_mq_unfreeze_queue(q, memflags); /* Ensure we don't receive any more interrupts */ diff --git a/drivers/bluetooth/btqca.c b/drivers/bluetooth/btqca.c index 22b08ab05b82..002307000d3c 100644 --- a/drivers/bluetooth/btqca.c +++ b/drivers/bluetooth/btqca.c @@ -426,7 +426,8 @@ static int qca_tlv_check_data(struct hci_dev *hdev, tlv_nvm->data[0] |= 0x80; /* UART Baud Rate */ - if (soc_type >= QCA_WCN3991) + if (soc_type == QCA_WCN3950 || + soc_type >= QCA_WCN3991) tlv_nvm->data[1] = nvm_baud_rate; else tlv_nvm->data[2] = nvm_baud_rate; diff --git a/drivers/char/hw_random/virtio-rng.c b/drivers/char/hw_random/virtio-rng.c index 5e83ffa105e4..a9ff520975a7 100644 --- a/drivers/char/hw_random/virtio-rng.c +++ b/drivers/char/hw_random/virtio-rng.c @@ -190,6 +190,7 @@ static int probe_common(struct virtio_device *vdev) err_find: ida_free(&rng_index_ida, index); err_ida: + vdev->priv = NULL; kfree(vi); return err; } @@ -198,6 +199,11 @@ static void remove_common(struct virtio_device *vdev) { struct virtrng_info *vi = vdev->priv; + if (!vi) { + virtio_reset_device(vdev); + return; + } + vi->hwrng_removed = true; vi->data_avail = 0; vi->data_idx = 0; @@ -207,6 +213,7 @@ static void remove_common(struct virtio_device *vdev) virtio_reset_device(vdev); vdev->config->del_vqs(vdev); ida_free(&rng_index_ida, vi->index); + vdev->priv = NULL; kfree(vi); } diff --git a/drivers/clk/nuvoton/clk-ma35d1.c b/drivers/clk/nuvoton/clk-ma35d1.c index 9f65d0623517..c914079cee2d 100644 --- a/drivers/clk/nuvoton/clk-ma35d1.c +++ b/drivers/clk/nuvoton/clk-ma35d1.c @@ -17,6 +17,7 @@ static DEFINE_SPINLOCK(ma35d1_lock); #define PLL_MAX_NUM 5 +#define MA35D1_CLK_COUNT 238 /* Clock Control Registers Offset */ #define REG_CLK_PWRCTL 0x00 @@ -467,12 +468,12 @@ static int ma35d1_clocks_probe(struct platform_device *pdev) int ret; ma35d1_hw_data = devm_kzalloc(dev, - struct_size(ma35d1_hw_data, hws, CLK_MAX_IDX), + struct_size(ma35d1_hw_data, hws, MA35D1_CLK_COUNT), GFP_KERNEL); if (!ma35d1_hw_data) return -ENOMEM; - ma35d1_hw_data->num = CLK_MAX_IDX; + ma35d1_hw_data->num = MA35D1_CLK_COUNT; hws = ma35d1_hw_data->hws; clk_base = devm_platform_ioremap_resource(pdev, 0); @@ -543,6 +544,8 @@ static int ma35d1_clocks_probe(struct platform_device *pdev) hws[HCLK3] = ma35d1_clk_fixed_factor(dev, "hclk3", "sysclk1_mux", 1, 2); hws[PCLK3] = ma35d1_clk_fixed_factor(dev, "pclk3", "sysclk1_mux", 1, 2); hws[PCLK4] = ma35d1_clk_fixed_factor(dev, "pclk4", "sysclk1_mux", 1, 2); + hws[PCLK3_DIV4096] = ma35d1_clk_fixed_factor(dev, "pclk3_div4096", "pclk3", 1, 4096); + hws[PCLK4_DIV4096] = ma35d1_clk_fixed_factor(dev, "pclk4_div4096", "pclk4", 1, 4096); hws[USBPHY0] = ma35d1_clk_fixed("usbphy0", 480000000); hws[USBPHY1] = ma35d1_clk_fixed("usbphy1", 480000000); diff --git a/drivers/cpufreq/amd-pstate.c b/drivers/cpufreq/amd-pstate.c index 6e00d200920a..28b9d4d062b9 100644 --- a/drivers/cpufreq/amd-pstate.c +++ b/drivers/cpufreq/amd-pstate.c @@ -1796,7 +1796,7 @@ EXPORT_SYMBOL_FOR_PSTATE_UT(amd_pstate_get_status); int amd_pstate_update_status(const char *buf, size_t size) { - int mode_idx; + int cpu, mode_idx, ret = 0; if (size > strlen("passive") || size < strlen("active")) return -EINVAL; @@ -1805,12 +1805,27 @@ int amd_pstate_update_status(const char *buf, size_t size) if (mode_idx < 0) return mode_idx; - if (mode_state_machine[cppc_state][mode_idx]) { - guard(mutex)(&amd_pstate_driver_lock); - return mode_state_machine[cppc_state][mode_idx](mode_idx); + guard(mutex)(&amd_pstate_driver_lock); + + if (!mode_state_machine[cppc_state][mode_idx]) + return 0; + + if (mode_idx == AMD_PSTATE_PASSIVE && + !cpu_feature_enabled(X86_FEATURE_CPPC)) { + cpus_read_lock(); + /* Check offline CPUs too, before changing or removing the driver. */ + for_each_present_cpu(cpu) { + if (cppc_auto_sel_is_immutable(cpu)) { + ret = -EOPNOTSUPP; + break; + } + } + cpus_read_unlock(); + if (ret) + return ret; } - return 0; + return mode_state_machine[cppc_state][mode_idx](mode_idx); } EXPORT_SYMBOL_FOR_PSTATE_UT(amd_pstate_update_status); diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c index 5f20c4a7ba2f..782b97acf5dc 100644 --- a/drivers/cpufreq/cppc_cpufreq.c +++ b/drivers/cpufreq/cppc_cpufreq.c @@ -42,6 +42,7 @@ MODULE_PARM_DESC(fie_disabled, "Disable Frequency Invariance Engine (FIE)"); /* Frequency invariance support */ struct cppc_freq_invariance { int cpu; + bool pcc_work_initialized; struct irq_work irq_work; struct kthread_work work; struct cppc_perf_fb_ctrs prev_perf_fb_ctrs; @@ -152,7 +153,7 @@ static struct scale_freq_data cppc_sftd_pcc = { static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy) { - struct scale_freq_data *sftd = &cppc_sftd; + struct scale_freq_data *sftd; struct cppc_freq_invariance *cppc_fi; int cpu, ret; @@ -169,7 +170,7 @@ static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy) kthread_init_work(&cppc_fi->work, cppc_scale_freq_workfn); init_irq_work(&cppc_fi->irq_work, cppc_irq_work); - sftd = &cppc_sftd_pcc; + cppc_fi->pcc_work_initialized = true; } ret = cppc_get_perf_ctrs(cpu, &cppc_fi->prev_perf_fb_ctrs); @@ -185,8 +186,15 @@ static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy) } } - /* Register for freq-invariance */ - topology_set_scale_freq_source(sftd, policy->cpus); + /* A shared policy may contain both PCC and non-PCC counters. */ + for_each_cpu(cpu, policy->cpus) { + cppc_fi = &per_cpu(cppc_freq_inv, cpu); + if (cppc_fi->pcc_work_initialized) + sftd = &cppc_sftd_pcc; + else + sftd = &cppc_sftd; + topology_set_scale_freq_source(sftd, cpumask_of(cpu)); + } } /* @@ -209,11 +217,12 @@ static void cppc_cpufreq_cpu_fie_exit(struct cpufreq_policy *policy) topology_clear_scale_freq_source(SCALE_FREQ_SOURCE_CPPC, policy->related_cpus); for_each_cpu(cpu, policy->related_cpus) { - if (!cppc_perf_ctrs_in_pcc_cpu(cpu)) - continue; cppc_fi = &per_cpu(cppc_freq_inv, cpu); + if (!cppc_fi->pcc_work_initialized) + continue; irq_work_sync(&cppc_fi->irq_work); kthread_cancel_work_sync(&cppc_fi->work); + cppc_fi->pcc_work_initialized = false; } } @@ -890,6 +899,7 @@ static ssize_t store_auto_select(struct cpufreq_policy *policy, const char *buf, size_t count) { struct cppc_cpudata *cpu_data = policy->driver_data; + bool old_auto_sel = cpu_data->perf_ctrls.auto_sel; bool val; int ret; @@ -918,8 +928,8 @@ static ssize_t store_auto_select(struct cpufreq_policy *policy, if (ret) { cpu_data->perf_ctrls.min_perf = old_min_perf; cpu_data->perf_ctrls.max_perf = old_max_perf; - cppc_set_auto_sel(policy->cpu, false); - cpu_data->perf_ctrls.auto_sel = false; + cppc_set_auto_sel(policy->cpu, old_auto_sel); + cpu_data->perf_ctrls.auto_sel = old_auto_sel; return ret; } } diff --git a/drivers/cxl/core/features.c b/drivers/cxl/core/features.c index ba6d2a5acb74..754fe5eb4b76 100644 --- a/drivers/cxl/core/features.c +++ b/drivers/cxl/core/features.c @@ -97,6 +97,7 @@ get_supported_features(struct cxl_features_state *cxlfs) kvmalloc_flex(*entries, ent, count); if (!entries) return NULL; + entries->num_features = count; struct cxl_mbox_get_sup_feats_out *mbox_out __free(kvfree) = kvmalloc(cxl_mbox->payload_size, GFP_KERNEL); @@ -174,7 +175,6 @@ get_supported_features(struct cxl_features_state *cxlfs) start += num_entries; } while (remain_feats); - entries->num_features = count; entries->num_user_features = user_feats; return no_free_ptr(entries); diff --git a/drivers/dma-buf/dma-buf-mapping.c b/drivers/dma-buf/dma-buf-mapping.c index 833be519e1e6..ef0f22854d01 100644 --- a/drivers/dma-buf/dma-buf-mapping.c +++ b/drivers/dma-buf/dma-buf-mapping.c @@ -156,7 +156,7 @@ struct sg_table *dma_buf_phys_vec_to_sgt(struct dma_buf_attachment *attach, phys_vec[i].paddr); } else if (dma_use_iova(dma->state)) { ret = dma_iova_link(attach->dev, dma->state, - phys_vec[i].paddr, 0, + phys_vec[i].paddr, mapped_len, phys_vec[i].len, dir, DMA_ATTR_MMIO); if (ret) diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c index 05a3b1f9e0c2..f67a93a37d48 100644 --- a/drivers/dma/qcom/bam_dma.c +++ b/drivers/dma/qcom/bam_dma.c @@ -1332,8 +1332,8 @@ static int bam_dma_probe(struct platform_device *pdev) for (i = 0; i < bdev->num_channels; i++) bam_channel_init(bdev, &bdev->channels[i], i); - ret = devm_request_irq(bdev->dev, bdev->irq, bam_dma_irq, - IRQF_TRIGGER_HIGH, "bam_dma", bdev); + ret = request_irq(bdev->irq, bam_dma_irq, IRQF_TRIGGER_NONE, + "bam_dma", bdev); if (ret) goto err_bam_channel_exit; @@ -1366,7 +1366,7 @@ static int bam_dma_probe(struct platform_device *pdev) ret = dma_async_device_register(&bdev->common); if (ret) { dev_err(bdev->dev, "failed to register dma async device\n"); - goto err_bam_channel_exit; + goto err_free_irq; } ret = of_dma_controller_register(pdev->dev.of_node, bam_dma_xlate, @@ -1385,6 +1385,8 @@ static int bam_dma_probe(struct platform_device *pdev) err_unregister_dma: dma_async_device_unregister(&bdev->common); +err_free_irq: + free_irq(bdev->irq, bdev); err_bam_channel_exit: for (i = 0; i < bdev->num_channels; i++) tasklet_kill(&bdev->channels[i].vc.task); @@ -1410,7 +1412,7 @@ static void bam_dma_remove(struct platform_device *pdev) /* mask all interrupts for this execution environment */ writel_relaxed(0, bam_addr(bdev, 0, BAM_IRQ_SRCS_MSK_EE)); - devm_free_irq(bdev->dev, bdev->irq, bdev); + free_irq(bdev->irq, bdev); for (i = 0; i < bdev->num_channels; i++) { bam_dma_terminate_all(&bdev->channels[i].vc.chan); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c index 8b8acf98fdfe..53475f8a3ff1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c @@ -405,7 +405,11 @@ int amdgpu_atif_query_backlight_caps(struct amdgpu_atif *atif) characteristics.max_input_signal; atif->backlight_caps.ac_level = characteristics.ac_level; atif->backlight_caps.dc_level = characteristics.dc_level; - atif->backlight_caps.data_points = characteristics.number_of_points; + if (characteristics.number_of_points > ATIF_QBTC_MAX_DATA_POINTS) + pr_warn(FW_BUG "ATIF: backlight curve has %u points, using the first %u\n", + characteristics.number_of_points, ATIF_QBTC_MAX_DATA_POINTS); + atif->backlight_caps.data_points = min_t(u8, characteristics.number_of_points, + ATIF_QBTC_MAX_DATA_POINTS); memcpy(atif->backlight_caps.luminance_data, characteristics.data_points, sizeof(atif->backlight_caps.luminance_data)); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index f462b40220eb..8e0b3db5c15e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -345,7 +345,8 @@ int amdgpu_amdkfd_gpuvm_map_bo_to_kernel(struct kgd_mem *mem, void **kptr, u64 *size, u32 domain); void amdgpu_amdkfd_gpuvm_unmap_bo_from_kernel(struct kgd_mem *mem); -int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart); +int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart, + struct amdkfd_process_info *process_info); int amdgpu_amdkfd_gpuvm_restore_process_bos(void *process_info, struct dma_fence __rcu **ef); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index c5c2a73ddd94..6c657d6060d6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -975,11 +975,11 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, ret = amdgpu_vm_lock_pd(vm, &exec, 0); drm_exec_retry_on_contention(&exec); if (unlikely(ret)) - goto unwind; + goto unlock; ret = drm_exec_lock_obj(&exec, &bo[i]->tbo.base); drm_exec_retry_on_contention(&exec); if (unlikely(ret)) - goto unwind; + goto unlock; } bo_va = amdgpu_vm_bo_find(vm, bo[i]); @@ -1005,6 +1005,8 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, return 0; +unlock: + drm_exec_fini(&exec); unwind: for (; i >= 0; i--) { if (!attachment[i]) @@ -2261,11 +2263,13 @@ int amdgpu_amdkfd_gpuvm_sync_memory( * amdgpu_amdkfd_map_gtt_bo_to_gart - Map BO to GART and increment reference count * @bo: Buffer object to be mapped * @bo_gart: Return bo reference + * @process_info: KFD process info of the queue the BO is for * * Before return, bo reference count is incremented. To release the reference and unpin/ * unmap the BO, call amdgpu_amdkfd_free_kernel_mem. */ -int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart) +int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart, + struct amdkfd_process_info *process_info) { int ret; @@ -2287,8 +2291,8 @@ int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo goto err_map_bo_gart_failed; } - amdgpu_amdkfd_remove_eviction_fence( - bo, bo->vm_bo->vm->process_info->eviction_fence); + /* bo->vm_bo can be the VM of any file which opened the BO */ + amdgpu_amdkfd_remove_eviction_fence(bo, process_info->eviction_fence); amdgpu_bo_unreserve(bo); @@ -2473,6 +2477,9 @@ static int import_obj_create(struct amdgpu_device *adev, err_remove_mem: remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info); + amdgpu_sync_free(&(*mem)->sync); + mutex_destroy(&(*mem)->lock); + dma_buf_put(dma_buf); drm_vma_node_revoke(&obj->vma_node, drm_priv); err_free_mem: kfree(*mem); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index 86a3dc3655b3..a03a15cfca96 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -149,7 +149,12 @@ static int amdgpu_cs_p1_user_fence(struct amdgpu_cs_parser *p, if (size != PAGE_SIZE || data->offset > (size - 8)) return -EINVAL; - if (amdgpu_ttm_tt_get_usermm(p->uf_bo->tbo.ttm)) + if (amdgpu_ttm_tt_is_userptr(p->uf_bo->tbo.ttm)) + return -EINVAL; + + /* The GPU writes the user fence through the GART */ + if (p->uf_bo->preferred_domains != AMDGPU_GEM_DOMAIN_GTT && + p->uf_bo->allowed_domains != AMDGPU_GEM_DOMAIN_GTT) return -EINVAL; *offset = data->offset; @@ -188,18 +193,22 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p, int ret; int i; + if (cs->in.flags) + return -EINVAL; + chunk_array = memdup_array_user(u64_to_user_ptr(cs->in.chunks), cs->in.num_chunks, sizeof(uint64_t)); if (IS_ERR(chunk_array)) return PTR_ERR(chunk_array); - p->nchunks = cs->in.num_chunks; - p->chunks = kvmalloc_objs(struct amdgpu_cs_chunk, p->nchunks); + /* parser_fini() walks nchunks chunks, set it once they exist */ + p->chunks = kvmalloc_objs(struct amdgpu_cs_chunk, cs->in.num_chunks); if (!p->chunks) { ret = -ENOMEM; goto free_chunk; } + p->nchunks = cs->in.num_chunks; for (i = 0; i < p->nchunks; i++) { struct drm_amdgpu_cs_chunk __user *chunk_ptr = NULL; @@ -1071,7 +1080,7 @@ static int amdgpu_cs_patch_ibs(struct amdgpu_cs_parser *p, if (r) return r; - kptr += va_start - (m->start * AMDGPU_GPU_PAGE_SIZE); + kptr += va_start - (m->start * AMDGPU_GPU_PAGE_SIZE) + m->offset; if (ring->funcs->parse_cs) { memcpy(ib->ptr, kptr, ib->length_dw * 4); @@ -1309,10 +1318,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p, fence = &p->jobs[i]->base.s_fence->scheduled; dma_fence_get(fence); r = drm_sched_job_add_dependency(&leader->base, fence); - if (r) { - dma_fence_put(fence); + if (r) return r; - } } if (p->gang_size > 1) { diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 5b72e4afb650..948886fffb4a 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -5515,7 +5515,21 @@ static void amdgpu_device_halt_activities(struct amdgpu_device *adev, struct amdgpu_device *tmp_adev = NULL; int i; - /* block all schedulers and reset given job's ring */ + /* Stop job submission first, the preparation below can take seconds */ + list_for_each_entry(tmp_adev, device_list, reset_list) { + for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { + struct amdgpu_ring *ring = tmp_adev->rings[i]; + + if (!amdgpu_ring_sched_ready(ring)) + continue; + + drm_sched_wqueue_stop(&ring->sched); + + if (need_emergency_restart) + amdgpu_job_stop_all_jobs_on_sched(&ring->sched); + } + } + list_for_each_entry(tmp_adev, device_list, reset_list) { amdgpu_device_set_mp1_state(tmp_adev); @@ -5551,18 +5565,6 @@ static void amdgpu_device_halt_activities(struct amdgpu_device *adev, amdgpu_ras_suspend(tmp_adev); amdgpu_userq_pre_reset(tmp_adev); - - for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { - struct amdgpu_ring *ring = tmp_adev->rings[i]; - - if (!amdgpu_ring_sched_ready(ring)) - continue; - - drm_sched_wqueue_stop(&ring->sched); - - if (need_emergency_restart) - amdgpu_job_stop_all_jobs_on_sched(&ring->sched); - } atomic_inc(&tmp_adev->gpu_reset_counter); } } @@ -5797,8 +5799,10 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, amdgpu_device_halt_activities(adev, job, reset_context, &device_list, hive, need_emergency_restart); - if (need_emergency_restart) - goto skip_sched_resume; + if (need_emergency_restart) { + amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart); + goto reset_unlock; + } /* * Must check guilty signal here since after this point all old * HW fences are force signaled. @@ -5808,18 +5812,17 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, if (job && (dma_fence_get_status(&job->hw_fence->base) > 0)) { job_signaled = true; dev_info(adev->dev, "Guilty job already signaled, skipping HW reset"); - goto skip_hw_reset; + goto post_resume; } r = amdgpu_device_asic_reset(adev, &device_list, reset_context); if (r) goto reset_unlock; -skip_hw_reset: +post_resume: + amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart); r = amdgpu_device_sched_resume(&device_list, reset_context, job_signaled); if (r) goto reset_unlock; -skip_sched_resume: - amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart); reset_unlock: amdgpu_device_recovery_put_reset_lock(adev, &device_list); amdgpu_ras_post_reset(adev, &device_list); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c index 7ab2ad2c5ba9..6d9a3e3b3546 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c @@ -1402,6 +1402,40 @@ static int amdgpu_discovery_sysfs_recurse(struct amdgpu_device *adev, return 0; } +#define list_to_kobj(el) container_of(el, struct kobject, entry) + +static void amdgpu_discovery_sysfs_update_harvest(struct amdgpu_device *adev) +{ + struct ip_discovery_top *ip_top = adev->discovery.ip_top; + struct ip_die_entry *ip_die_entry; + struct list_head *el_die, *el_hw_id, *el_hw_inst; + struct ip_hw_id *hw_id; + struct kset *die_kset; + struct ip_hw_instance *ip_inst; + + if (!ip_top) + return; + + die_kset = &ip_top->die_kset; + + guard(spinlock)(&die_kset->list_lock); + list_for_each(el_die, &die_kset->list) { + ip_die_entry = to_ip_die_entry(list_to_kobj(el_die)); + + list_for_each(el_hw_id, &ip_die_entry->ip_kset.list) { + hw_id = to_ip_hw_id(list_to_kobj(el_hw_id)); + + list_for_each(el_hw_inst, &hw_id->hw_id_kset.list) { + ip_inst = to_ip_hw_instance(list_to_kobj(el_hw_inst)); + ip_inst->harvest = + amdgpu_discovery_get_harvest_info( + adev, ip_inst->hw_id, + ip_inst->num_instance); + } + } + } +} + static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev) { uint8_t *discovery_bin = adev->discovery.bin; @@ -1412,9 +1446,13 @@ static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev) if (!discovery_bin) return -EINVAL; - /* If early init already created sysfs in standalone mode, skip normal init */ - if (adev->discovery.ip_top && adev->discovery.ip_top->standalone_mode) + /* If early init already created sysfs in standalone mode, update harvest info + * now that adev is fully initialized, then skip the rest of normal init. + */ + if (adev->discovery.ip_top && adev->discovery.ip_top->standalone_mode) { + amdgpu_discovery_sysfs_update_harvest(adev); return 0; + } ip_top = kzalloc_obj(*ip_top); if (!ip_top) @@ -1469,8 +1507,6 @@ static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev) /* -------------------------------------------------- */ -#define list_to_kobj(el) container_of(el, struct kobject, entry) - static void amdgpu_discovery_sysfs_ip_hw_free(struct ip_hw_id *ip_hw_id) { struct list_head *el, *tmp; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c index 9adf3eed8822..ccadbcf38d3f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c @@ -547,6 +547,11 @@ amdgpu_dma_buf_move_notify(struct dma_buf_attachment *attach) continue; } + if (!amdgpu_vm_ready(vm)) { + dma_resv_unlock(resv); + continue; + } + /* Reserve fences for two SDMA page table updates */ r = dma_resv_reserve_fences(resv, 2); if (!r) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c index afd8502a2bc4..cd05921aede5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c @@ -1033,6 +1033,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) { + struct amdgpu_device *adev = drm_to_adev(dev); struct drm_amdgpu_gem_op *args = data; struct drm_gem_object *gobj; struct amdgpu_vm_bo_base *base; @@ -1139,7 +1140,8 @@ int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data, amdgpu_vm_bo_va_for_each_valid_mapping(bo_va, mapping) { if (num_mappings < args->num_entries) { - vm_entries[num_mappings].addr = mapping->start * AMDGPU_GPU_PAGE_SIZE; + vm_entries[num_mappings].addr = + amdgpu_gmc_sign_extend(mapping->start * AMDGPU_GPU_PAGE_SIZE); vm_entries[num_mappings].size = (mapping->last - mapping->start + 1) * AMDGPU_GPU_PAGE_SIZE; vm_entries[num_mappings].offset = mapping->offset; vm_entries[num_mappings].flags = mapping->flags; @@ -1149,7 +1151,8 @@ int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data, amdgpu_vm_bo_va_for_each_invalid_mapping(bo_va, mapping) { if (num_mappings < args->num_entries) { - vm_entries[num_mappings].addr = mapping->start * AMDGPU_GPU_PAGE_SIZE; + vm_entries[num_mappings].addr = + amdgpu_gmc_sign_extend(mapping->start * AMDGPU_GPU_PAGE_SIZE); vm_entries[num_mappings].size = (mapping->last - mapping->start + 1) * AMDGPU_GPU_PAGE_SIZE; vm_entries[num_mappings].offset = mapping->offset; vm_entries[num_mappings].flags = mapping->flags; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index 35eb87591740..37cf1b808950 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -1485,7 +1485,8 @@ int amdgpu_ras_reset_error_count(struct amdgpu_device *adev, return -EOPNOTSUPP; } - if (!amdgpu_ras_is_supported(adev, block)) + if (!amdgpu_ras_is_supported(adev, block) || + !amdgpu_ras_mgr_get_debug_mode(adev)) return -EOPNOTSUPP; if (amdgpu_sriov_vf(adev)) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c index d6ae9974c952..4f31021b2152 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c @@ -447,10 +447,8 @@ int amdgpu_sync_push_to_job(struct amdgpu_sync *sync, struct amdgpu_job *job) dma_fence_get(f); r = drm_sched_job_add_dependency(&job->base, f); - if (r) { - dma_fence_put(f); + if (r) return r; - } } return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c index b3dc5dd49a64..053ead0d690f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c @@ -365,6 +365,8 @@ static int amdgpu_userq_unmap_helper(struct amdgpu_usermode_queue *queue) adev->userq_funcs[queue->queue_type]; int r; + /* The reset changes the queue state and the MES is unusable meanwhile */ + guard(rwsem_read)(&adev->reset_domain->sem); if ((queue->state == AMDGPU_USERQ_STATE_MAPPED) || (queue->state == AMDGPU_USERQ_STATE_PREEMPTED)) { trace_amdgpu_userq_state_start(queue); @@ -391,6 +393,8 @@ static int amdgpu_userq_map_helper(struct amdgpu_usermode_queue *queue) adev->userq_funcs[queue->queue_type]; int r; + /* The reset changes the queue state and the MES is unusable meanwhile */ + guard(rwsem_read)(&adev->reset_domain->sem); if (queue->state == AMDGPU_USERQ_STATE_UNMAPPED) { trace_amdgpu_userq_state_start(queue); @@ -751,12 +755,7 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args) if (!adev->userq_halt_for_enforce_isolation || ((queue->queue_type != AMDGPU_HW_IP_GFX) && (queue->queue_type != AMDGPU_HW_IP_COMPUTE))) { - /* Serialize the map against an in-progress GPU reset (MES is - * unresponsive during recovery), matching amdgpu_userq_detach_doorbell(). - */ - down_read(&adev->reset_domain->sem); r = amdgpu_userq_map_helper(queue); - up_read(&adev->reset_domain->sem); if (r) { drm_file_err(uq_mgr->file, "Failed to map Queue\n"); trace_amdgpu_userq_create_end(queue, r); @@ -973,7 +972,11 @@ static int amdgpu_userq_validate_vm(void *param, struct amdgpu_bo *bo) return ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); } -/* Handle all BOs on the invalidated list, validate them and update the PTs */ +/* + * Handle all individual BOs which were moved, lock and validate them and + * update the PTs. An evicted individual BO lands on the needs_update list, + * the evicted one only holds imported DMA-bufs of compute VMs. + */ static int amdgpu_userq_bo_validate(struct amdgpu_device *adev, struct drm_exec *exec, struct amdgpu_vm *vm) @@ -984,8 +987,8 @@ amdgpu_userq_bo_validate(struct amdgpu_device *adev, struct drm_exec *exec, int ret; spin_lock(&vm->individual_lock); - while (!list_empty(&vm->individual.evicted)) { - bo_va = list_first_entry(&vm->individual.evicted, + while (!list_empty(&vm->individual.needs_update)) { + bo_va = list_first_entry(&vm->individual.needs_update, struct amdgpu_bo_va, base.vm_status); bo = bo_va->base.bo; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index fc3a8d38d514..c07f1115f654 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -269,19 +269,27 @@ static void amdgpu_vm_bo_reset_state_machine(struct amdgpu_vm *vm) /* * Don't use list splice here, we need the special handling for the root - * PD and set the moved flag appropriately. + * PD. All mappings must be written again, also of the vm_bos which are + * already waiting for an update of some of them, e.g. after a map with + * a delayed update, so set the moved flag for everything that needs an + * update. */ amdgpu_vm_assert_locked(vm); list_for_each_entry_safe(vm_bo, tmp, &vm->kernel.idle, vm_status) amdgpu_vm_bo_needs_update(vm_bo); + list_for_each_entry(vm_bo, &vm->kernel.needs_update, vm_status) + vm_bo->moved = true; + list_for_each_entry_safe(vm_bo, tmp, &vm->always_valid.idle, vm_status) amdgpu_vm_bo_needs_update(vm_bo); + list_for_each_entry(vm_bo, &vm->always_valid.needs_update, vm_status) + vm_bo->moved = true; spin_lock(&vm->individual_lock); - list_for_each_entry_safe(vm_bo, tmp, &vm->individual.idle, vm_status) { - vm_bo->moved = true; + list_for_each_entry_safe(vm_bo, tmp, &vm->individual.idle, vm_status) list_move(&vm_bo->vm_status, &vm->individual.needs_update); - } + list_for_each_entry(vm_bo, &vm->individual.needs_update, vm_status) + vm_bo->moved = true; spin_unlock(&vm->individual_lock); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c index eb8cf0d9d9c5..652266e93766 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c @@ -234,7 +234,6 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p, dma_fence_get(fence); r = drm_sched_job_add_dependency(&p->job->base, fence); if (r) { - dma_fence_put(fence); dma_resv_iter_end(&cursor); return r; } diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c index f2977fe6d824..04b18056a906 100644 --- a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c +++ b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c @@ -1705,8 +1705,9 @@ static void dce_v10_0_afmt_setmode(struct drm_encoder *encoder, WREG32(mmAFMT_RAMP_CONTROL2 + dig->afmt->offset, 0x00000001); WREG32(mmAFMT_RAMP_CONTROL3 + dig->afmt->offset, 0x00000001); - /* enable audio after to setting up hw */ - dce_v10_0_audio_enable(adev, dig->afmt->pin, true); + /* Only enable audio if the display supports it */ + if (connector->display_info.has_audio) + dce_v10_0_audio_enable(adev, dig->afmt->pin, true); } static void dce_v10_0_afmt_enable(struct drm_encoder *encoder, bool enable) diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c index c68de0fe1d7d..585b0a50279f 100644 --- a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c @@ -1748,8 +1748,9 @@ static void dce_v6_0_afmt_setmode(struct drm_encoder *encoder, dce_v6_0_audio_dp_enable(encoder, 1); } - /* enable audio after setting up hw */ - dce_v6_0_audio_enable(adev, dig->afmt->pin, true); + /* Only enable audio if the display supports it */ + if (connector->display_info.has_audio) + dce_v6_0_audio_enable(adev, dig->afmt->pin, true); } static void dce_v6_0_afmt_enable(struct drm_encoder *encoder, bool enable) diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c index c3906270f25e..1390efa4ea51 100644 --- a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c @@ -1653,8 +1653,9 @@ static void dce_v8_0_afmt_setmode(struct drm_encoder *encoder, WREG32(mmAFMT_RAMP_CONTROL2 + offset, 0x00000001); WREG32(mmAFMT_RAMP_CONTROL3 + offset, 0x00000001); - /* enable audio after setting up hw */ - dce_v8_0_audio_enable(adev, dig->afmt->pin, true); + /* Only enable audio if the display supports it */ + if (connector->display_info.has_audio) + dce_v8_0_audio_enable(adev, dig->afmt->pin, true); } static void dce_v8_0_afmt_enable(struct drm_encoder *encoder, bool enable) diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c index d69c43f1e194..60debaa23762 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c @@ -4080,6 +4080,7 @@ static void gfx_v12_0_update_spm_vmid(struct amdgpu_device *adev, { u32 reg, data; + amdgpu_gfx_off_ctrl(adev, false); reg = SOC15_REG_OFFSET(GC, 0, regRLC_SPM_MC_CNTL); if (amdgpu_sriov_is_pp_one_vf(adev)) data = RREG32_NO_KIQ(reg); @@ -4093,6 +4094,7 @@ static void gfx_v12_0_update_spm_vmid(struct amdgpu_device *adev, WREG32_SOC15_NO_KIQ(GC, 0, regRLC_SPM_MC_CNTL, data); else WREG32_SOC15(GC, 0, regRLC_SPM_MC_CNTL, data); + amdgpu_gfx_off_ctrl(adev, true); if (ring && amdgpu_sriov_is_pp_one_vf(adev) diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c index 333e9c30c091..535c84e55558 100644 --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c @@ -168,8 +168,19 @@ static int ih_v6_0_toggle_ring_interrupts(struct amdgpu_device *adev, tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0)); if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) { - if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) + /* + * PSP programming of IH_RB_CNTL is slow enough for MSI to + * arrive before this call returns. Publish ih->enabled first + * so amdgpu_ih_process() consumes that IV and rearms. + */ + if (enable) { + ih->enabled = true; + smp_wmb(); + } + if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) { + ih->enabled = false; return -ETIMEDOUT; + } } else { WREG32(ih_regs->ih_rb_cntl, tmp); } diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c index c57bcb9a98b1..10bed6f38f30 100644 --- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c @@ -218,8 +218,10 @@ static int mes_v11_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes, r = amdgpu_fence_wait_polling(ring, seq - ring->fence_drv.num_fences_mask, timeout); - if (r < 1) + if (r < 1) { + r = -ETIMEDOUT; goto error_undo; + } api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off); api_status->api_completion_fence_addr = status_gpu_addr; diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c index ee72afdb0a1d..bb9892d1bd05 100644 --- a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c +++ b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c @@ -195,8 +195,10 @@ static int mes_v12_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes, r = amdgpu_fence_wait_polling(ring, seq - ring->fence_drv.num_fences_mask, timeout); - if (r < 1) + if (r < 1) { + r = -ETIMEDOUT; goto error_undo; + } api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off); api_status->api_completion_fence_addr = status_gpu_addr; diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 4cd325149b63..bed0eb4b6ec3 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -161,13 +161,52 @@ static int navi10_ih_toggle_ring_interrupts(struct amdgpu_device *adev, tmp = RREG32(ih_regs->ih_rb_cntl); tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_ENABLE, (enable ? 1 : 0)); tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_GPU_TS_ENABLE, 1); + + if (enable) { + /* Unset the CLEAR_OVERFLOW bit to make sure the next step + * is switching the bit from 0 to 1 + */ + tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 0); + if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) { + if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) + return -ETIMEDOUT; + } else { + WREG32_NO_KIQ(ih_regs->ih_rb_cntl, tmp); + } + + /* Clear RB_OVERFLOW bit */ + tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 1); + if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) { + if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) + return -ETIMEDOUT; + } else { + WREG32_NO_KIQ(ih_regs->ih_rb_cntl, tmp); + } + + /* Unset the CLEAR_OVERFLOW bit immediately so new overflows + * can be detected. + */ + tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 0); + } + /* enable_intr field is only valid in ring0 */ if (ih == &adev->irq.ih) tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0)); if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) { - if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) + /* + * PSP programming of IH_RB_CNTL is slow enough for MSI to + * arrive before this call returns. Publish ih->enabled first + * so amdgpu_ih_process() consumes that IV and rearms. + */ + if (enable) { + ih->enabled = true; + smp_wmb(); + } + if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) { + ih->enabled = false; return -ETIMEDOUT; + } } else { WREG32(ih_regs->ih_rb_cntl, tmp); } diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c index 87fb42d3c713..64673007ccde 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c @@ -303,7 +303,8 @@ static int init_user_queue(struct process_queue_manager *pqm, } retval = amdgpu_amdkfd_map_gtt_bo_to_gart(q_properties->wptr_bo, - &(*q)->wptr_bo_gart); + &(*q)->wptr_bo_gart, + pqm->process->kgd_process_info); if (retval) { pr_err("Failed to map wptr bo to GART\n"); goto free_gang_ctx_bo; diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_freesync.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_freesync.c index 7e484adb4120..a51cf46d0e44 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_freesync.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_freesync.c @@ -128,6 +128,14 @@ void amdgpu_dm_get_freesync_config_for_crtc( vrefresh >= aconnector->min_vfreq && vrefresh <= aconnector->max_vfreq; + /* + * DC reads allow_freesync while validating the new state, for example + * to keep SubVP off a single FreeSync display, but it is otherwise set + * only on the first flip after the commit. Set it here, so that a + * stream created by this commit is not validated without it. + */ + new_crtc_state->stream->allow_freesync = new_crtc_state->vrr_supported; + if (new_crtc_state->vrr_supported) { new_crtc_state->stream->ignore_msa_timing_param = true; fs_vid_mode = new_crtc_state->freesync_config.state == VRR_STATE_ACTIVE_FIXED; @@ -154,6 +162,18 @@ void amdgpu_dm_get_freesync_config_for_crtc( } EXPORT_IF_KUNIT(amdgpu_dm_get_freesync_config_for_crtc); +#define DP_HDR_SDP_EOTF_IDX 2 + +static bool dm_dp_in_hdr_mode(const struct dc_stream_state *stream) +{ + const struct dc_info_packet *hdr = &stream->hdr_static_metadata; + + if (!dc_is_dp_signal(stream->signal) || !hdr->valid) + return false; + + return hdr->sb[DP_HDR_SDP_EOTF_IDX] == HDMI_EOTF_SMPTE_ST2084; +} + void amdgpu_dm_update_freesync_state_on_stream( struct amdgpu_display_manager *dm, struct dm_crtc_state *new_crtc_state, @@ -169,6 +189,7 @@ void amdgpu_dm_update_freesync_state_on_stream( bool pack_sdp_v1_3 = false; struct amdgpu_dm_connector *aconn; enum vrr_packet_type packet_type = PACKET_TYPE_VRR; + enum color_transfer_func app_tf; if (!new_stream) return; @@ -220,12 +241,16 @@ void amdgpu_dm_update_freesync_state_on_stream( &new_stream->adaptive_sync_infopacket); } + app_tf = dm_dp_in_hdr_mode(new_stream) ? + TRANSFER_FUNC_PQ2084 : + TRANSFER_FUNC_UNKNOWN; + mod_freesync_build_vrr_infopacket( dm->freesync_module, new_stream, &vrr_params, packet_type, - TRANSFER_FUNC_UNKNOWN, + app_tf, &vrr_infopacket, pack_sdp_v1_3); diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c index 6ee3da89d058..135eeef68ac4 100644 --- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c +++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c @@ -538,6 +538,9 @@ static void dcn401_update_clocks_update_dtb_dto(struct clk_mgr_internal *clk_mgr otg_master = resource_get_otg_master_for_stream( &context->res_ctx, context->streams[i]); ASSERT(otg_master); + if (!otg_master) + continue; + ASSERT(otg_master->clock_source); ASSERT(otg_master->clock_source->funcs->program_pix_clk); ASSERT(otg_master->stream_res.pix_clk_params.controller_id >= CONTROLLER_ID_D0); diff --git a/drivers/gpu/drm/amd/display/dc/core/dc.c b/drivers/gpu/drm/amd/display/dc/core/dc.c index 519ac878ada1..60b903edff2f 100644 --- a/drivers/gpu/drm/amd/display/dc/core/dc.c +++ b/drivers/gpu/drm/amd/display/dc/core/dc.c @@ -2452,6 +2452,8 @@ static enum dc_status dc_commit_state_no_check(struct dc *dc, struct dc_state *c * update pending complete before optimizing bandwidth. */ hwss_wait_for_odm_update_pending_complete(dc, context); + /* No DET may stay allocated to a HUBP without a plane (e.g. left over from VBIOS) */ + hwss_reset_unused_det(dc, context); /* pplib is notified if disp_num changed */ dc->hwss.optimize_bandwidth(dc, context); /* Need to do otg sync again as otg could be out of sync due to otg @@ -4885,8 +4887,10 @@ static void commit_planes_for_stream(struct dc *dc, dc_exit_ips_for_hw_access(dc); dc_z10_restore(dc); - if (update_type == UPDATE_TYPE_FULL && dc->optimized_required) + if (update_type == UPDATE_TYPE_FULL && dc->optimized_required) { + hwss_reset_unused_det(dc, dc->current_state); hwss_process_outstanding_hw_updates(dc, dc->current_state); + } if (update_type != UPDATE_TYPE_FAST && dc->res_pool->funcs->prepare_mcache_programming) dc->res_pool->funcs->prepare_mcache_programming(dc, context); diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c index ef4987d449cb..d4dbdf7d0506 100644 --- a/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c +++ b/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c @@ -3047,6 +3047,37 @@ void hwss_wait_for_odm_update_pending_complete(struct dc *dc, struct dc_state *c hwss_wait_for_all_blank_complete(dc, context); } +void hwss_reset_unused_det(struct dc *dc, struct dc_state *context) +{ + struct hubbub *hubbub = dc->res_pool->hubbub; + unsigned int i, j; + + if (!hubbub || !hubbub->funcs->program_det_segments) + return; + + for (i = 0; i < dc->res_pool->pipe_count; i++) { + struct hubp *hubp = dc->res_pool->hubps[i]; + bool hubp_in_use = false; + + if (!hubp) + continue; + + for (j = 0; j < dc->res_pool->pipe_count; j++) { + const struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[j]; + + /* A plane-less pipe renders from the DPG, its HUBP (assigned with the stream) needs no DET */ + if (pipe->plane_state && pipe->plane_res.hubp && + pipe->plane_res.hubp->inst == hubp->inst) { + hubp_in_use = true; + break; + } + } + + if (!hubp_in_use) + hubbub->funcs->program_det_segments(hubbub, hubp->inst, 0); + } +} + void hwss_wait_for_no_pipes_pending(struct dc *dc, struct dc_state *context) { int i; diff --git a/drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h b/drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h index cdf2562a9a40..0d64a0c1913d 100644 --- a/drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h +++ b/drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h @@ -1807,6 +1807,9 @@ void hwss_wait_for_odm_update_pending_complete(struct dc *dc, void hwss_wait_for_no_pipes_pending(struct dc *dc, struct dc_state *context); +void hwss_reset_unused_det(struct dc *dc, + struct dc_state *context); + void hwss_wait_for_outstanding_hw_updates(struct dc *dc, struct dc_state *dc_context); diff --git a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c index 24e3bbec15ff..8ff364d465de 100644 --- a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c @@ -2601,12 +2601,8 @@ bool dp_is_sink_present(struct dc_link *link) * which indicates we need additional delay */ - if (dal_ddc_open(ddc, GPIO_MODE_INPUT, - GPIO_DDC_CONFIG_TYPE_MODE_I2C) != GPIO_RESULT_OK) { - dal_ddc_close(ddc); - + if (dal_ddc_open(ddc, GPIO_MODE_INPUT, GPIO_DDC_CONFIG_TYPE_MODE_I2C) != GPIO_RESULT_OK) return present; - } /* * Read GPIO: DP sink is present if both clock and data pins are zero diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c b/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c index 73cc8a713556..2cdfe3d7a0ea 100644 --- a/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c +++ b/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c @@ -449,6 +449,13 @@ void optc2_lock_doublebuffer_disable(struct timing_generator *optc) void optc2_setup_manual_trigger(struct timing_generator *optc) { struct optc *optc1 = DCN10TG_FROM_TG(optc); + uint32_t v_total, v_total_max; + + REG_GET(OTG_V_TOTAL_MAX, OTG_V_TOTAL_MAX, &v_total_max); + if (!v_total_max) { + REG_GET(OTG_V_TOTAL, OTG_V_TOTAL, &v_total); + optc->funcs->set_vtotal_min_max(optc, v_total, v_total); + } /* Set the min/max selectors unconditionally so that * DMCUB fw may change OTG timings when necessary diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn30/dcn30_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn30/dcn30_optc.h index 16c5610b49ac..b801989f10cf 100644 --- a/drivers/gpu/drm/amd/display/dc/optc/dcn30/dcn30_optc.h +++ b/drivers/gpu/drm/amd/display/dc/optc/dcn30/dcn30_optc.h @@ -91,6 +91,12 @@ SRI(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst),\ SRI(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst),\ SRI(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst),\ + SRI(OTG_CRC1_DATA_RG, OTG, inst),\ + SRI(OTG_CRC1_DATA_B, OTG, inst),\ + SRI(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst),\ + SRI(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst),\ + SRI(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst),\ + SRI(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst),\ SR(GSL_SOURCE_SELECT),\ SRI(OTG_TRIGA_MANUAL_TRIG, OTG, inst),\ SRI(OTG_DRR_CONTROL, OTG, inst) @@ -229,6 +235,7 @@ SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\ + SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\ SF(OTG0_OTG_CRC_CNTL2, OTG_CRC_DSC_MODE, mask_sh),\ SF(OTG0_OTG_CRC_CNTL2, OTG_CRC_DATA_STREAM_COMBINE_MODE, mask_sh),\ @@ -245,6 +252,17 @@ SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\ SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\ SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\ SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\ SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\ SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\ diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h index 119a954fe3e5..85223414c909 100644 --- a/drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h +++ b/drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h @@ -83,6 +83,12 @@ SRI(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst),\ SRI(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst),\ SRI(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst),\ + SRI(OTG_CRC1_DATA_RG, OTG, inst),\ + SRI(OTG_CRC1_DATA_B, OTG, inst),\ + SRI(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst),\ + SRI(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst),\ + SRI(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst),\ + SRI(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst),\ SR(GSL_SOURCE_SELECT),\ SRI(OTG_TRIGA_MANUAL_TRIG, OTG, inst),\ SRI(OTG_GLOBAL_CONTROL1, OTG, inst),\ @@ -205,7 +211,9 @@ SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\ + SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\ + SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\ SF(OTG0_OTG_CRC0_DATA_RG, CRC0_R_CR, mask_sh),\ SF(OTG0_OTG_CRC0_DATA_RG, CRC0_G_Y, mask_sh),\ SF(OTG0_OTG_CRC0_DATA_B, CRC0_B_CB, mask_sh),\ @@ -217,6 +225,17 @@ SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\ SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\ SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\ SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\ SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\ SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\ @@ -264,8 +283,7 @@ SF(OTG0_INTERRUPT_DEST, OTG0_IHC_OTG_VERTICAL_INTERRUPT2_DEST, mask_sh) #define OPTC_COMMON_MASK_SH_LIST_DCN31X(mask_sh)\ - OPTC_COMMON_MASK_SH_LIST_DCN3_1(mask_sh),\ - SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh) + OPTC_COMMON_MASK_SH_LIST_DCN3_1(mask_sh) void dcn31_timing_generator_init(struct optc *optc1); diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.h index 7b43c3ecd46b..956eecf0a43a 100644 --- a/drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.h +++ b/drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.h @@ -84,6 +84,12 @@ SRI(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst),\ SRI(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst),\ SRI(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst),\ + SRI(OTG_CRC1_DATA_RG, OTG, inst),\ + SRI(OTG_CRC1_DATA_B, OTG, inst),\ + SRI(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst),\ + SRI(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst),\ + SRI(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst),\ + SRI(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst),\ SR(GSL_SOURCE_SELECT),\ SRI(OTG_TRIGA_MANUAL_TRIG, OTG, inst),\ SRI(OTG_GLOBAL_CONTROL1, OTG, inst),\ @@ -204,6 +210,7 @@ SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\ + SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\ SF(OTG0_OTG_CRC0_DATA_RG, CRC0_R_CR, mask_sh),\ @@ -217,6 +224,17 @@ SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\ SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\ SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\ SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\ SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\ SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\ diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.h index 60c01ec28b65..492760779226 100644 --- a/drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.h +++ b/drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.h @@ -146,6 +146,17 @@ SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\ SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\ SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\ SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\ SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\ SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\ diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.h index 82e0818c8f9f..196596cc9fb6 100644 --- a/drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.h +++ b/drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.h @@ -32,23 +32,12 @@ #define OPTC_COMMON_MASK_SH_LIST_DCN3_5(mask_sh)\ OPTC_COMMON_MASK_SH_LIST_DCN3_2(mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_WINDOW_DB_EN, mask_sh),\ - SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\ - SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\ - SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\ SF(OTG0_OTG_CRC2_DATA_RG, CRC2_R_CR, mask_sh),\ SF(OTG0_OTG_CRC2_DATA_RG, CRC2_G_Y, mask_sh),\ SF(OTG0_OTG_CRC2_DATA_B, CRC2_B_CB, mask_sh),\ SF(OTG0_OTG_CRC3_DATA_RG, CRC3_R_CR, mask_sh),\ SF(OTG0_OTG_CRC3_DATA_RG, CRC3_G_Y, mask_sh),\ SF(OTG0_OTG_CRC3_DATA_B, CRC3_B_CB, mask_sh),\ - SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\ - SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\ - SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\ - SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\ - SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\ - SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\ - SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\ - SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\ SF(OTG0_OTG_CRC0_WINDOWA_X_CONTROL_READBACK, OTG_CRC0_WINDOWA_X_START_READBACK, mask_sh),\ SF(OTG0_OTG_CRC0_WINDOWA_X_CONTROL_READBACK, OTG_CRC0_WINDOWA_X_END_READBACK, mask_sh),\ SF(OTG0_OTG_CRC0_WINDOWA_Y_CONTROL_READBACK, OTG_CRC0_WINDOWA_Y_START_READBACK, mask_sh),\ diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.h index 3add972ca2ff..9e4e9d75400a 100644 --- a/drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.h +++ b/drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.h @@ -110,6 +110,7 @@ SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\ + SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\ SF(OTG0_OTG_CRC0_DATA_RG, CRC0_R_CR, mask_sh),\ @@ -123,6 +124,17 @@ SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\ SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\ SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\ + SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\ + SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\ SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\ SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\ SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\ diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn42/dcn42_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn42/dcn42_optc.h index bc641f6ecd7e..bdda94489a23 100644 --- a/drivers/gpu/drm/amd/display/dc/optc/dcn42/dcn42_optc.h +++ b/drivers/gpu/drm/amd/display/dc/optc/dcn42/dcn42_optc.h @@ -137,6 +137,7 @@ SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\ + SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\ SF(OTG0_OTG_CRC0_DATA_R, CRC0_R_CR, mask_sh),\ @@ -316,6 +317,7 @@ SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\ + SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\ SF(OTG0_OTG_CRC0_DATA_R, CRC0_R_CR, mask_sh),\ diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn60/dcn60_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn60/dcn60_optc.h index ba207c0e498e..ec40c9180030 100644 --- a/drivers/gpu/drm/amd/display/dc/optc/dcn60/dcn60_optc.h +++ b/drivers/gpu/drm/amd/display/dc/optc/dcn60/dcn60_optc.h @@ -109,7 +109,9 @@ SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_FORCE_DELAY, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\ + SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\ SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\ + SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\ SF(OTG0_OTG_CRC0_DATA_R, CRC0_R_CR, mask_sh),\ SF(OTG0_OTG_CRC0_DATA_G, CRC0_G_Y, mask_sh),\ SF(OTG0_OTG_CRC0_DATA_B, CRC0_B_CB, mask_sh),\ diff --git a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c index a2e66761b8ec..81227df68680 100644 --- a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c +++ b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c @@ -1941,8 +1941,21 @@ static bool dml1_validate(struct dc *dc, struct dc_state *context, enum dc_valid out = dcn32_internal_validate_bw(dc, context, pipes, &pipe_cnt, &vlevel, validate_mode); DC_FP_END(); - if (pipe_cnt == 0) + if (pipe_cnt == 0) { + /* The context is a copy of the previous state, so without this the + * clocks the last display needed stay applied with all displays off. + */ + context->bw_ctx.bw.dcn.clk.socclk_khz = 0; + context->bw_ctx.bw.dcn.clk.dppclk_khz = 0; + context->bw_ctx.bw.dcn.clk.dcfclk_khz = 0; + context->bw_ctx.bw.dcn.clk.dcfclk_deep_sleep_khz = 0; + context->bw_ctx.bw.dcn.clk.dramclk_khz = 0; + context->bw_ctx.bw.dcn.clk.fclk_khz = 0; + context->bw_ctx.bw.dcn.clk.p_state_change_support = true; + context->bw_ctx.bw.dcn.clk.fclk_p_state_change_support = true; + context->bw_ctx.bw.dcn.clk.num_ways = 0; goto validate_out; + } if (!out) goto validate_fail; diff --git a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.h b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.h index 68e7140f1505..fa8cc89cb3f7 100644 --- a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.h +++ b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.h @@ -1054,6 +1054,12 @@ unsigned int dcn32_get_max_hw_cursor_size(const struct dc *dc, SRI_ARR(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst), \ SRI_ARR(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst), \ SRI_ARR(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst), \ + SRI_ARR(OTG_CRC1_DATA_RG, OTG, inst), \ + SRI_ARR(OTG_CRC1_DATA_B, OTG, inst), \ + SRI_ARR(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst), \ + SRI_ARR(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst), \ + SRI_ARR(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst), \ + SRI_ARR(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst), \ SR_ARR(GSL_SOURCE_SELECT, inst), \ SRI_ARR(OTG_TRIGA_MANUAL_TRIG, OTG, inst), \ SRI_ARR(OTG_GLOBAL_CONTROL1, OTG, inst), \ diff --git a/drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h b/drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h index 8e3473f86490..3cfc1a39a103 100644 --- a/drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h +++ b/drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h @@ -535,6 +535,12 @@ int dcn401_get_power_profile(const struct dc_state *context); SRI_ARR(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst), \ SRI_ARR(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst), \ SRI_ARR(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst), \ + SRI_ARR(OTG_CRC1_DATA_RG, OTG, inst), \ + SRI_ARR(OTG_CRC1_DATA_B, OTG, inst), \ + SRI_ARR(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst), \ + SRI_ARR(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst), \ + SRI_ARR(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst), \ + SRI_ARR(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst), \ SR_ARR(GSL_SOURCE_SELECT, inst), \ SRI_ARR(OTG_TRIGA_MANUAL_TRIG, OTG, inst), \ SRI_ARR(OTG_GLOBAL_CONTROL1, OTG, inst), \ diff --git a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c index 4bd060e30799..f154a619b533 100644 --- a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c +++ b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c @@ -1281,10 +1281,8 @@ void mod_freesync_handle_v_update(struct mod_freesync *mod_freesync, * If VRR is inactive, set vtotal min and max to nominal vtotal */ if (in_out_vrr->state == VRR_STATE_INACTIVE) { - in_out_vrr->adjust.v_total_min = - mod_freesync_calc_v_total_from_refresh(stream, - in_out_vrr->max_refresh_in_uhz); - in_out_vrr->adjust.v_total_max = in_out_vrr->adjust.v_total_min; + in_out_vrr->adjust.v_total_min = stream->timing.v_total; + in_out_vrr->adjust.v_total_max = stream->timing.v_total; return; } } diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c index 8285d4a91589..13e7e33fa0eb 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c @@ -2912,6 +2912,25 @@ static int smu_v13_0_7_mode1_reset(struct smu_context *smu) return ret; } +static void smu_v13_0_7_get_unique_id(struct smu_context *smu) +{ + SmuMetricsExternal_t metrics_external; + SmuMetrics_t *metrics = &metrics_external.SmuMetrics; + struct amdgpu_device *adev = smu->adev; + u32 upper32 = 0, lower32 = 0; + int ret; + + ret = smu_cmn_get_metrics_table(smu, &metrics_external, false); + if (ret) + goto out; + + upper32 = metrics->PublicSerialNumberUpper; + lower32 = metrics->PublicSerialNumberLower; + +out: + adev->unique_id = ((u64)upper32 << 32) | lower32; +} + static void smu_v13_0_7_init_msg_ctl(struct smu_context *smu) { struct amdgpu_device *adev = smu->adev; @@ -2995,6 +3014,7 @@ static const struct pptable_funcs smu_v13_0_7_ppt_funcs = { .enable_uclk_shadow = smu_v13_0_enable_uclk_shadow, .set_wbrf_exclusion_ranges = smu_v13_0_set_wbrf_exclusion_ranges, .interrupt_work = smu_v13_0_interrupt_work, + .get_unique_id = smu_v13_0_7_get_unique_id, }; void smu_v13_0_7_set_ppt_funcs(struct smu_context *smu) diff --git a/drivers/gpu/drm/bridge/nwl-dsi.c b/drivers/gpu/drm/bridge/nwl-dsi.c index 09992529f3d8..1aab46e1a168 100644 --- a/drivers/gpu/drm/bridge/nwl-dsi.c +++ b/drivers/gpu/drm/bridge/nwl-dsi.c @@ -110,6 +110,8 @@ struct nwl_dsi { int error; struct nwl_dsi_transfer *xfer; + + unsigned int endpoint; }; static const struct regmap_config nwl_dsi_regmap_config = { @@ -821,10 +823,13 @@ static int nwl_dsi_bridge_atomic_check(struct drm_bridge *bridge, struct drm_connector_state *conn_state) { struct drm_display_mode *adjusted_mode = &crtc_state->adjusted_mode; + struct nwl_dsi *dsi = bridge_to_dsi(bridge); /* At least LCDIF + NWL needs active high sync */ - adjusted_mode->flags |= (DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC); - adjusted_mode->flags &= ~(DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC); + if (dsi->endpoint == NWL_DSI_ENDPOINT_LCDIF) { + adjusted_mode->flags |= (DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC); + adjusted_mode->flags &= ~(DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC); + } /* * Do a full modeset if crtc_state->active is changed to be true. @@ -1088,13 +1093,12 @@ static int nwl_dsi_parse_dt(struct nwl_dsi *dsi) static int nwl_dsi_select_input(struct nwl_dsi *dsi) { struct device_node *remote; - u32 use_dcss = 1; int ret; remote = of_graph_get_remote_node(dsi->dev->of_node, 0, NWL_DSI_ENDPOINT_LCDIF); if (remote) { - use_dcss = 0; + dsi->endpoint = NWL_DSI_ENDPOINT_LCDIF; } else { remote = of_graph_get_remote_node(dsi->dev->of_node, 0, NWL_DSI_ENDPOINT_DCSS); @@ -1103,11 +1107,12 @@ static int nwl_dsi_select_input(struct nwl_dsi *dsi) "No valid input endpoint found\n"); return -EINVAL; } + dsi->endpoint = NWL_DSI_ENDPOINT_DCSS; } DRM_DEV_INFO(dsi->dev, "Using %s as input source\n", - (use_dcss) ? "DCSS" : "LCDIF"); - ret = mux_control_try_select(dsi->mux, use_dcss); + (dsi->endpoint == NWL_DSI_ENDPOINT_DCSS) ? "DCSS" : "LCDIF"); + ret = mux_control_try_select(dsi->mux, dsi->endpoint); if (ret < 0) DRM_DEV_ERROR(dsi->dev, "Failed to select input: %d\n", ret); diff --git a/drivers/gpu/drm/display/drm_dp_helper.c b/drivers/gpu/drm/display/drm_dp_helper.c index 488fd123a01a..7676285c19f8 100644 --- a/drivers/gpu/drm/display/drm_dp_helper.c +++ b/drivers/gpu/drm/display/drm_dp_helper.c @@ -2192,7 +2192,8 @@ static int drm_dp_i2c_xfer(struct i2c_adapter *adapter, struct i2c_msg *msgs, msg.buffer = msgs[i].buf + j; msg.size = min(transfer_size, msgs[i].len - j); - if (j + msg.size == msgs[i].len && aux->no_zero_sized) + if (aux->no_zero_sized && i == num - 1 && + j + msg.size == msgs[i].len) msg.request &= ~DP_AUX_I2C_MOT; err = drm_dp_i2c_drain_msg(aux, &msg); diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c index 63ead8ba6756..5295ec122e0d 100644 --- a/drivers/gpu/drm/drm_crtc.c +++ b/drivers/gpu/drm/drm_crtc.c @@ -154,11 +154,8 @@ static void drm_crtc_crc_fini(struct drm_crtc *crtc) #endif } -static const struct dma_fence_ops drm_crtc_fence_ops; - static struct drm_crtc *fence_to_crtc(struct dma_fence *fence) { - BUG_ON(rcu_access_pointer(fence->ops) != &drm_crtc_fence_ops); return container_of(fence->extern_lock, struct drm_crtc, fence_lock); } @@ -493,14 +490,23 @@ EXPORT_SYMBOL(__drmm_crtc_alloc_with_planes); * drm_crtc_cleanup - Clean up the core crtc usage * @crtc: CRTC to cleanup * - * This function cleans up @crtc and removes it from the DRM mode setting - * core. Note that the function does *not* free the crtc structure itself, - * this is the responsibility of the caller. + * This function cleans up @crtc and removes it from the DRM mode setting core, + * after first waiting an RCU grace period to ensure @crtc->dev can safely be + * dereferenced by our dma_fence_ops. + * + * Note that the function does *not* free the crtc structure itself, this is the + * responsibility of the caller. */ void drm_crtc_cleanup(struct drm_crtc *crtc) { struct drm_device *dev = crtc->dev; + /* Ensure our dma_fence_ops remain valid for an RCU grace period after + * the fence is signaled. This is necessary because our dma_fence_ops + * dereference crtc->dev. + */ + synchronize_rcu(); + /* Note that the crtc_list is considered to be static; should we * remove the drm_crtc at runtime we would have to decrement all * the indices on the drm_crtc after us in the crtc_list. diff --git a/drivers/gpu/drm/drm_panic.c b/drivers/gpu/drm/drm_panic.c index d28ce8878e05..fef47625c526 100644 --- a/drivers/gpu/drm/drm_panic.c +++ b/drivers/gpu/drm/drm_panic.c @@ -558,7 +558,8 @@ static void draw_panic_screen_kmsg(struct drm_scanout_buffer *sb) const struct font_desc *font = get_default_font(sb->width, sb->height, NULL, NULL); struct drm_rect r_screen = DRM_RECT_INIT(0, 0, sb->width, sb->height); struct kmsg_dump_iter iter; - char kmsg_buf[512]; + /* the size printk formats one record into, too big for the stack */ + static char kmsg_buf[2048]; size_t kmsg_len; struct drm_panic_line line; int yoffset; @@ -576,6 +577,10 @@ static void draw_panic_screen_kmsg(struct drm_scanout_buffer *sb) char *start; char *end; + /* a record that does not fit comes back empty, every time */ + if (!kmsg_len) + break; + /* ignore terminating NUL and newline */ start = kmsg_buf + kmsg_len - 2; end = kmsg_buf + kmsg_len - 1; diff --git a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c index 03b2576af67f..99c91a9eecf1 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c @@ -168,6 +168,17 @@ u16 etnaviv_buffer_config_pta(struct etnaviv_gpu *gpu, unsigned short id) return buffer->user_size / 8; } +static void etnaviv_cmd_flush_ts(struct etnaviv_cmdbuf *buffer, bool has_blt) +{ + CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE, VIVS_TS_FLUSH_CACHE_FLUSH); + + if (has_blt) { + CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1); + CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1); + CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0); + } +} + void etnaviv_buffer_end(struct etnaviv_gpu *gpu) { struct etnaviv_cmdbuf *buffer = &gpu->buffer; @@ -191,7 +202,7 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu) unsigned int dwords = 7; if (has_blt) - dwords += 10; + dwords += 11; link_target = etnaviv_buffer_reserve(gpu, buffer, dwords); @@ -204,16 +215,8 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu) CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0); } CMD_LOAD_STATE(buffer, VIVS_GL_FLUSH_CACHE, flush); - if (gpu->exec_state == ETNA_PIPE_3D) { - if (has_blt) { - CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1); - CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1); - CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0); - } else { - CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE, - VIVS_TS_FLUSH_CACHE_FLUSH); - } - } + if (gpu->exec_state == ETNA_PIPE_3D) + etnaviv_cmd_flush_ts(buffer, has_blt); CMD_SEM(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE); CMD_STALL(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE); if (has_blt) { @@ -400,12 +403,12 @@ void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, u32 exec_state, return_dwords = 7; /* - * When the BLT engine is present we need 6 more dwords in the return + * When the BLT engine is present we need 7 more dwords in the return * target: 3 enable/flush/disable + 4 enable/semaphore stall/disable, - * but we don't need the normal TS flush state. + * on top of the normal TS flush state. */ if (has_blt) - return_dwords += 6; + return_dwords += 7; return_target = etnaviv_buffer_reserve(gpu, buffer, return_dwords); CMD_LINK(cmdbuf, return_dwords, return_target); @@ -422,14 +425,7 @@ void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, u32 exec_state, VIVS_GL_FLUSH_CACHE_DEPTH | VIVS_GL_FLUSH_CACHE_COLOR | VIVS_GL_FLUSH_CACHE_SHADER_L1); - if (has_blt) { - CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1); - CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1); - CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0); - } else { - CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE, - VIVS_TS_FLUSH_CACHE_FLUSH); - } + etnaviv_cmd_flush_ts(buffer, has_blt); } CMD_SEM(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE); CMD_STALL(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE); diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.c b/drivers/gpu/drm/etnaviv/etnaviv_drv.c index 08aca9035fc1..ec1352c1b09f 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_drv.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.c @@ -601,8 +601,10 @@ static void etnaviv_unbind(struct device *dev) component_unbind_all(dev, drm); - etnaviv_cmdbuf_free(priv->flop_reset_data_ppu); - kfree(priv->flop_reset_data_ppu); + if (priv->flop_reset_data_ppu) { + etnaviv_cmdbuf_free(priv->flop_reset_data_ppu); + kfree(priv->flop_reset_data_ppu); + } etnaviv_cmdbuf_suballoc_destroy(priv->cmdbuf_suballoc); diff --git a/drivers/gpu/drm/i915/display/intel_ddi.c b/drivers/gpu/drm/i915/display/intel_ddi.c index 02a53c9848e1..8de6e13cf2dd 100644 --- a/drivers/gpu/drm/i915/display/intel_ddi.c +++ b/drivers/gpu/drm/i915/display/intel_ddi.c @@ -4594,7 +4594,7 @@ intel_ddi_port_sync_transcoders(const struct intel_crtc_state *ref_crtc_state, * From, xe3lpd onwards we have defeatured this with reference to * Wa_16024710867 */ - if (!IS_DISPLAY_VER(display, 9, 20)) + if (!IS_DISPLAY_VER(display, 9, 30)) return 0; if (!intel_crtc_has_type(ref_crtc_state, INTEL_OUTPUT_DP)) diff --git a/drivers/gpu/drm/i915/display/intel_display_device.c b/drivers/gpu/drm/i915/display/intel_display_device.c index f17fc2c68472..c74256359c25 100644 --- a/drivers/gpu/drm/i915/display/intel_display_device.c +++ b/drivers/gpu/drm/i915/display/intel_display_device.c @@ -1813,7 +1813,10 @@ static void __intel_display_device_info_runtime_init(struct intel_display *displ if (intel_display_wa(display, INTEL_DISPLAY_WA_14011765242)) for_each_pipe(display, pipe) display_runtime->num_scalers[pipe] = 0; - else if (DISPLAY_VER(display) >= 11) { + else if (DISPLAY_VERx100(display) == 3002) { + for_each_pipe(display, pipe) + display_runtime->num_scalers[pipe] = 1; + } else if (DISPLAY_VER(display) >= 11) { for_each_pipe(display, pipe) display_runtime->num_scalers[pipe] = 2; } else if (DISPLAY_VER(display) >= 9) { diff --git a/drivers/gpu/drm/i915/display/intel_dp.c b/drivers/gpu/drm/i915/display/intel_dp.c index 3152122e6aef..d79b6367cd57 100644 --- a/drivers/gpu/drm/i915/display/intel_dp.c +++ b/drivers/gpu/drm/i915/display/intel_dp.c @@ -475,9 +475,9 @@ bool intel_dp_has_joiner(struct intel_dp *intel_dp) !connector->panel.vbt.edp.pipe_joiner_enable) return false; - return DISPLAY_VER(display) >= 12 || + return ((DISPLAY_VER(display) >= 12 && DISPLAY_VERx100(display) != 3002) || (DISPLAY_VER(display) == 11 && - encoder->port != PORT_A); + encoder->port != PORT_A)); } static int dg2_max_source_rate(struct intel_dp *intel_dp) diff --git a/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c b/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c index 266e042e0023..d84def79c4bf 100644 --- a/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c +++ b/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c @@ -41,6 +41,7 @@ #include "intel_display_types.h" #include "intel_dp.h" #include "intel_dp_aux_backlight.h" +#include "intel_quirks.h" /* * DP AUX registers for Intel's proprietary HDR backlight interface. We define @@ -687,11 +688,30 @@ int intel_dp_aux_init_backlight_funcs(struct intel_connector *connector) struct drm_device *dev = connector->base.dev; struct intel_panel *panel = &connector->panel; bool try_intel_interface = false, try_vesa_interface = false; + int enable_dpcd_backlight; + + /* + * Some devices have a broken VBT in some Panels, but not others, + * advertising INTEL_BACKLIGHT_DISPLAY_DDI, but actually failing to use + * the Intel interface. Adding a fallback to the Vesa interface gives + * these devices a working backlight for both the broken and non broken + * panel VBTs. + * + * This fallback can not be made the general default as this has broken + * other devices in the past whose VBT correctly reports + * INTEL_BACKLIGHT_DISPLAY_DDI and whose PWM path is the actual + * backlight control, but whose DPCD optimistically advertises + * DP_EDP_BACKLIGHT_AUX_ENABLE_CAP / _BRIGHTNESS_AUX_SET_CAP. + */ + enable_dpcd_backlight = display->params.enable_dpcd_backlight; + if (enable_dpcd_backlight == INTEL_DP_AUX_BACKLIGHT_AUTO && + intel_has_dpcd_quirk(intel_dp, QUIRK_ENABLE_DPCD_BACKLIGHT)) + enable_dpcd_backlight = INTEL_DP_AUX_BACKLIGHT_ON; /* Check the VBT and user's module parameters to figure out which * interfaces to probe */ - switch (display->params.enable_dpcd_backlight) { + switch (enable_dpcd_backlight) { case INTEL_DP_AUX_BACKLIGHT_OFF: return -ENODEV; case INTEL_DP_AUX_BACKLIGHT_AUTO: diff --git a/drivers/gpu/drm/i915/display/intel_plane.c b/drivers/gpu/drm/i915/display/intel_plane.c index d0f99a87c42e..c1c5a6128614 100644 --- a/drivers/gpu/drm/i915/display/intel_plane.c +++ b/drivers/gpu/drm/i915/display/intel_plane.c @@ -140,6 +140,7 @@ intel_plane_duplicate_state(struct drm_plane *plane) intel_state->dpt_vma = NULL; intel_state->fence_id = -1; intel_state->damage = DRM_RECT_INIT(0, 0, 0, 0); + intel_state->psr2_sel_fetch_area = DRM_RECT_INIT(0, 0, 0, 0); /* add reference to fb */ if (intel_state->hw.fb) @@ -1930,6 +1931,16 @@ static int intel_add_affected_planes(struct intel_atomic_state *state) ret = intel_joiner_add_affected_planes(state, intel_crtc_joined_pipe_mask(crtc_state)); if (ret) return ret; + + /* + * Add affected plane here to ensure UV and Y plane linking is + * in place if UV plane is added as a part of selective update. + */ + if (crtc_state->enable_psr2_sel_fetch) { + ret = intel_plane_add_affected(state, crtc); + if (ret) + return ret; + } } return 0; diff --git a/drivers/gpu/drm/i915/display/intel_psr.c b/drivers/gpu/drm/i915/display/intel_psr.c index c28bcb1a0184..7aaf5b4d3503 100644 --- a/drivers/gpu/drm/i915/display/intel_psr.c +++ b/drivers/gpu/drm/i915/display/intel_psr.c @@ -2702,23 +2702,25 @@ static u32 psr2_pipe_srcsz_early_tpt_calc(struct intel_crtc_state *crtc_state, } static void clip_area_update(struct drm_rect *overlap_damage_area, - struct drm_rect *damage_area, - struct drm_rect *display_area) + const struct drm_rect *damage_area, + const struct drm_rect *display_area) { - if (!drm_rect_intersect(damage_area, display_area)) + struct drm_rect inter = *damage_area; + + if (!drm_rect_intersect(&inter, display_area)) return; if (overlap_damage_area->y1 == -1) { - overlap_damage_area->y1 = damage_area->y1; - overlap_damage_area->y2 = damage_area->y2; + overlap_damage_area->y1 = inter.y1; + overlap_damage_area->y2 = inter.y2; return; } - if (damage_area->y1 < overlap_damage_area->y1) - overlap_damage_area->y1 = damage_area->y1; + if (inter.y1 < overlap_damage_area->y1) + overlap_damage_area->y1 = inter.y1; - if (damage_area->y2 > overlap_damage_area->y2) - overlap_damage_area->y2 = damage_area->y2; + if (inter.y2 > overlap_damage_area->y2) + overlap_damage_area->y2 = inter.y2; } static bool intel_psr2_sel_fetch_pipe_alignment(struct intel_crtc_state *crtc_state) @@ -2895,7 +2897,7 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state, .y2 = crtc_state->hw.adjusted_mode.crtc_vdisplay, }; bool full_update = false, su_area_changed; - int i, ret; + int i; /* * Selective fetch is not always usable, for instance it is dropped @@ -2913,16 +2915,16 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state, if (!crtc_state->enable_psr2_sel_fetch) return 0; - if (!psr2_sel_fetch_pipe_state_supported(crtc_state)) { - full_update = true; - goto skip_sel_fetch_set_loop; - } - crtc_state->psr2_su_area.x1 = 0; crtc_state->psr2_su_area.y1 = -1; crtc_state->psr2_su_area.x2 = drm_rect_width(&display_area); crtc_state->psr2_su_area.y2 = -1; + if (!psr2_sel_fetch_pipe_state_supported(crtc_state)) { + full_update = true; + goto skip_su_area_calc; + } + /* * Calculate minimal selective fetch area of each plane and calculate * the pipe damaged area. @@ -3006,23 +3008,19 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state, } if (full_update) - goto skip_sel_fetch_set_loop; + goto skip_su_area_calc; intel_psr_apply_su_area_workarounds(crtc_state); - ret = drm_atomic_add_affected_planes(&state->base, &crtc->base); - if (ret) - return ret; - do { bool cursor_in_su_area = false; /* * Adjust su area to cover cursor fully as necessary - * (early transport). This needs to be done after - * drm_atomic_add_affected_planes to ensure visible - * cursor is added into affected planes even when - * cursor is not updated by itself. + * (early transport). A visible cursor is in the state + * even when it is not updated by itself as all planes + * of the pipe are added into the state before the plane + * checks when selective fetch is enabled. */ intel_psr2_sel_fetch_et_alignment(state, crtc, &display_area, &cursor_in_su_area); @@ -3042,6 +3040,11 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state, break; } while (su_area_changed); +skip_su_area_calc: + if (full_update) + clip_area_update(&crtc_state->psr2_su_area, &display_area, + &display_area); + /* * Now that we have the pipe damaged area check if it intersect with * every plane, if it does set the plane selective fetch area. @@ -3057,28 +3060,14 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state, inter = crtc_state->psr2_su_area; sel_fetch_area = &new_plane_state->psr2_sel_fetch_area; - if (!drm_rect_intersect(&inter, &new_plane_state->uapi.dst)) { - sel_fetch_area->y1 = -1; - sel_fetch_area->y2 = -1; - /* - * if plane sel fetch was previously enabled -> - * disable it - */ - if (drm_rect_height(&old_plane_state->psr2_sel_fetch_area) > 0) - crtc_state->update_planes |= BIT(plane->id); - - continue; - } - - if (!psr2_sel_fetch_plane_state_supported(new_plane_state)) { - full_update = true; - break; + if (drm_rect_intersect(&inter, &new_plane_state->uapi.dst)) { + sel_fetch_area->y1 = inter.y1 - new_plane_state->uapi.dst.y1; + sel_fetch_area->y2 = inter.y2 - new_plane_state->uapi.dst.y1; } - sel_fetch_area = &new_plane_state->psr2_sel_fetch_area; - sel_fetch_area->y1 = inter.y1 - new_plane_state->uapi.dst.y1; - sel_fetch_area->y2 = inter.y2 - new_plane_state->uapi.dst.y1; - crtc_state->update_planes |= BIT(plane->id); + if (!drm_rect_equals(&old_plane_state->psr2_sel_fetch_area, + sel_fetch_area)) + crtc_state->update_planes |= BIT(plane->id); /* * Sel_fetch_area is calculated for UV plane. Use @@ -3086,24 +3075,25 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state, */ if (linked) { struct intel_plane_state *linked_new_plane_state; + struct intel_plane_state *linked_old_plane_state; struct drm_rect *linked_sel_fetch_area; - linked_new_plane_state = intel_atomic_get_plane_state(state, linked); - if (IS_ERR(linked_new_plane_state)) - return PTR_ERR(linked_new_plane_state); + linked_new_plane_state = intel_atomic_get_new_plane_state(state, linked); + if (!linked_new_plane_state) + return -EINVAL; + + linked_old_plane_state = intel_atomic_get_old_plane_state(state, linked); linked_sel_fetch_area = &linked_new_plane_state->psr2_sel_fetch_area; linked_sel_fetch_area->y1 = sel_fetch_area->y1; linked_sel_fetch_area->y2 = sel_fetch_area->y2; - crtc_state->update_planes |= BIT(linked->id); + + if (!drm_rect_equals(&linked_old_plane_state->psr2_sel_fetch_area, + linked_sel_fetch_area)) + crtc_state->update_planes |= BIT(linked->id); } } -skip_sel_fetch_set_loop: - if (full_update) - clip_area_update(&crtc_state->psr2_su_area, &display_area, - &display_area); - psr2_man_trk_ctl_calc(crtc_state, full_update); crtc_state->pipe_srcsz_early_tpt = psr2_pipe_srcsz_early_tpt_calc(crtc_state, full_update); diff --git a/drivers/gpu/drm/i915/display/intel_psr_regs.h b/drivers/gpu/drm/i915/display/intel_psr_regs.h index 16a9e3af198d..428f3da04500 100644 --- a/drivers/gpu/drm/i915/display/intel_psr_regs.h +++ b/drivers/gpu/drm/i915/display/intel_psr_regs.h @@ -254,12 +254,12 @@ #define _LNL_SFF_CTL_A 0x60918 #define _LNL_SFF_CTL_B 0x61918 #define LNL_SFF_CTL(tran) _MMIO_TRANS(tran, _LNL_SFF_CTL_A, _LNL_SFF_CTL_B) -#define LNL_SFF_CTL_SF_SINGLE_FULL_FRAME REG_BIT(1) +#define LNL_SFF_CTL_SF_SINGLE_FULL_FRAME REG_BIT(0) #define _LNL_CFF_CTL_A 0x6091c #define _LNL_CFF_CTL_B 0x6191c #define LNL_CFF_CTL(tran) _MMIO_TRANS(tran, _LNL_CFF_CTL_A, _LNL_CFF_CTL_B) -#define LNL_CFF_CTL_SF_CONTINUOUS_FULL_FRAME REG_BIT(1) +#define LNL_CFF_CTL_SF_CONTINUOUS_FULL_FRAME REG_BIT(0) /* PSR2 Early transport */ #define _PIPE_SRCSZ_ERLY_TPT_A 0x70074 diff --git a/drivers/gpu/drm/i915/display/intel_quirks.c b/drivers/gpu/drm/i915/display/intel_quirks.c index 7d7db774d8c7..96eb9a40df9b 100644 --- a/drivers/gpu/drm/i915/display/intel_quirks.c +++ b/drivers/gpu/drm/i915/display/intel_quirks.c @@ -100,6 +100,14 @@ static void quirk_disable_psr2(struct intel_display *display) drm_info(display->drm, "PSR2 support not currently available for this setup, applying disable PSR2 quirk\n"); } +static void quirk_enable_dpcd_backlight(struct intel_dp *intel_dp) +{ + struct intel_display *display = to_intel_display(intel_dp); + + intel_set_dpcd_quirk(intel_dp, QUIRK_ENABLE_DPCD_BACKLIGHT); + drm_info(display->drm, "Applying Enable DPCD Backlight quirk\n"); +} + struct intel_quirk { int device; int subsystem_vendor; @@ -289,6 +297,22 @@ static const struct intel_dpcd_quirk intel_dpcd_quirks[] = { .sink_oui = SINK_OUI(0x00, 0x22, 0xb9), .hook = quirk_disable_edp_panel_replay, }, + /* TUXEDO InsanityBook 15 v1 */ + { + .device = 0x591b, + .subsystem_vendor = 0x1558, + .subsystem_device = 0x9501, + .sink_oui = SINK_OUI(0x38, 0xec, 0x11), + .hook = quirk_enable_dpcd_backlight, + }, + /* TUXEDO DX1708 */ + { + .device = 0x3e9b, + .subsystem_vendor = 0x1558, + .subsystem_device = 0x8500, + .sink_oui = SINK_OUI(0x38, 0xec, 0x11), + .hook = quirk_enable_dpcd_backlight, + }, }; void intel_init_quirks(struct intel_display *display) diff --git a/drivers/gpu/drm/i915/display/intel_quirks.h b/drivers/gpu/drm/i915/display/intel_quirks.h index 970a4fe52faf..4996419ae76b 100644 --- a/drivers/gpu/drm/i915/display/intel_quirks.h +++ b/drivers/gpu/drm/i915/display/intel_quirks.h @@ -23,6 +23,7 @@ enum intel_quirk_id { QUIRK_EDP_LIMIT_RATE_HBR2, QUIRK_DISABLE_EDP_PANEL_REPLAY, QUIRK_DISABLE_PSR2, + QUIRK_ENABLE_DPCD_BACKLIGHT, }; void intel_init_quirks(struct intel_display *display); diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c index d5aadced7da0..7fdd41a190b4 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c @@ -2790,7 +2790,6 @@ add_timeline_fence_array(struct i915_execbuffer *eb, u64 __user *user_values; struct eb_fence *f; u64 nfences; - int err = 0; nfences = timeline_fences->fence_count; if (!nfences) @@ -2828,6 +2827,7 @@ add_timeline_fence_array(struct i915_execbuffer *eb, struct drm_syncobj *syncobj; struct dma_fence *fence = NULL; u64 point; + int err = 0; if (__copy_from_user(&user_fence, user_fences++, diff --git a/drivers/gpu/drm/loongson/lsdc_gem.c b/drivers/gpu/drm/loongson/lsdc_gem.c index 2fb03487c983..9eba4e1a5ba6 100644 --- a/drivers/gpu/drm/loongson/lsdc_gem.c +++ b/drivers/gpu/drm/loongson/lsdc_gem.c @@ -157,14 +157,18 @@ struct drm_gem_object *lsdc_gem_object_create(struct drm_device *ddev, return ERR_PTR(ret); } + gobj = &lbo->tbo.base; + gobj->funcs = &lsdc_gem_object_funcs; + if (!sg) { /* VRAM is filled with random data */ - lsdc_bo_clear(lbo); + ret = lsdc_bo_clear(lbo); + if (ret) { + drm_gem_object_put(gobj); + return ERR_PTR(ret); + } } - gobj = &lbo->tbo.base; - gobj->funcs = &lsdc_gem_object_funcs; - /* tracking the BOs we created */ mutex_lock(&ldev->gem.mutex); list_add_tail(&lbo->list, &ldev->gem.objects); diff --git a/drivers/gpu/drm/loongson/lsdc_plane.c b/drivers/gpu/drm/loongson/lsdc_plane.c index bcc0ffa17bdf..7b9c65c30a9d 100644 --- a/drivers/gpu/drm/loongson/lsdc_plane.c +++ b/drivers/gpu/drm/loongson/lsdc_plane.c @@ -230,7 +230,7 @@ static void lsdc_cursor_plane_atomic_async_update(struct drm_plane *plane, new_state = drm_atomic_get_new_plane_state(state, plane); - new_fb = plane->state->fb; + new_fb = new_state->fb; plane->state->crtc_x = new_state->crtc_x; plane->state->crtc_y = new_state->crtc_y; diff --git a/drivers/gpu/drm/loongson/lsdc_ttm.c b/drivers/gpu/drm/loongson/lsdc_ttm.c index d7441d96a0dc..7b30dea039ee 100644 --- a/drivers/gpu/drm/loongson/lsdc_ttm.c +++ b/drivers/gpu/drm/loongson/lsdc_ttm.c @@ -388,9 +388,13 @@ void lsdc_bo_kunmap(struct lsdc_bo *lbo) ttm_bo_kunmap(&lbo->kmap); } -void lsdc_bo_clear(struct lsdc_bo *lbo) +int lsdc_bo_clear(struct lsdc_bo *lbo) { - lsdc_bo_kmap(lbo); + int ret; + + ret = lsdc_bo_kmap(lbo); + if (ret) + return ret; if (lbo->is_iomem) memset_io((void __iomem *)lbo->kptr, 0, lbo->size); @@ -398,6 +402,8 @@ void lsdc_bo_clear(struct lsdc_bo *lbo) memset(lbo->kptr, 0, lbo->size); lsdc_bo_kunmap(lbo); + + return 0; } int lsdc_bo_evict_vram(struct drm_device *ddev) @@ -475,10 +481,8 @@ struct lsdc_bo *lsdc_bo_create(struct drm_device *ddev, ret = ttm_bo_init_validate(bdev, tbo, bo_type, &lbo->placement, 0, false, sg, resv, lsdc_bo_destroy); - if (ret) { - kfree(lbo); + if (ret) return ERR_PTR(ret); - } return lbo; } diff --git a/drivers/gpu/drm/loongson/lsdc_ttm.h b/drivers/gpu/drm/loongson/lsdc_ttm.h index 843e1475064e..df47d9bb0d03 100644 --- a/drivers/gpu/drm/loongson/lsdc_ttm.h +++ b/drivers/gpu/drm/loongson/lsdc_ttm.h @@ -89,7 +89,7 @@ size_t lsdc_bo_size(struct lsdc_bo *lbo); int lsdc_bo_kmap(struct lsdc_bo *lbo); void lsdc_bo_kunmap(struct lsdc_bo *lbo); -void lsdc_bo_clear(struct lsdc_bo *lbo); +int lsdc_bo_clear(struct lsdc_bo *lbo); int lsdc_bo_evict_vram(struct drm_device *ddev); diff --git a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c index aab5a73f467a..ff6858f5e5d3 100644 --- a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c +++ b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c @@ -157,6 +157,7 @@ nvkm_subdev_init_(struct nvkm_subdev *subdev) ret = subdev->func->init(subdev); if (ret) { nvkm_error(subdev, "init failed, %d\n", ret); + subdev->use.enabled = false; return ret; } } diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c index 5368e705e7fd..186e09ea0cb1 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c @@ -27,6 +27,7 @@ #include #include #include +#include int gf100_devinit_pll_set(struct nvkm_devinit *init, u32 type, u32 freq) @@ -99,6 +100,25 @@ gf100_devinit_preinit(struct nvkm_devinit *base) * can use it as a reliable way to know whether we should run devinit. */ base->post = ((nvkm_rd32(device, 0x2240c) & BIT(1)) == 0); + + /* + * However some VBIOS init sequences miss this bit. So fall back to + * the nv50 method of checking if the bit is not set. Running the + * VBIOS multiple times may have detrimental effects. + */ + if (base->post) { + u64 disable = nvkm_devinit_disable(base); + /* magic to detect whether or not x86 vbios code has + * executed the devinit scripts to initialise the + * board. only works if there's a display engine. + */ + if (!(disable & (1ULL << NVKM_ENGINE_DISP))) { + if (nvkm_rdvgac(device, 0, 0x00) || + nvkm_rdvgac(device, 0, 0x1a)) { + base->post = false; + } + } + } } static const struct nvkm_devinit_func diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index 673ca9cbf362..0206bbad2ce0 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -130,10 +130,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, INIT_LIST_HEAD(&entity->list); entity->guilty = guilty; - entity->priority = priority; entity->last_user = current->group_leader; - entity->rq_priority = drm_sched_policy == DRM_SCHED_POLICY_FAIR ? - DRM_SCHED_PRIORITY_KERNEL : priority; entity->num_sched_list = num_sched_list; entity->sched_list = num_sched_list > 1 ? sched_list : NULL; RCU_INIT_POINTER(entity->last_scheduled, NULL); @@ -145,26 +142,26 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, * not to do this and to fix their DRM calling order. */ pr_warn("%s: called with uninitialized scheduler\n", __func__); - } else { - enum drm_sched_priority p = entity->priority; - + } else if (priority >= sched_list[0]->num_user_rqs) { /* * The "priority" of an entity cannot exceed the number of * run-queues of a scheduler. Protect against num_rqs being 0, * by converting to signed. Choose the lowest priority * available. */ - if (p >= sched_list[0]->num_user_rqs) { - dev_err(sched_list[0]->dev, "entity with out-of-bounds priority:%u num_user_rqs:%u\n", - p, sched_list[0]->num_user_rqs); - p = max_t(s32, - (s32)sched_list[0]->num_user_rqs - 1, - (s32)DRM_SCHED_PRIORITY_KERNEL); - entity->priority = p; - } - entity->rq = sched_list[0]->sched_rq[entity->rq_priority]; + dev_err(sched_list[0]->dev, "entity with out-of-bounds priority:%u num_user_rqs:%u\n", + priority, sched_list[0]->num_user_rqs); + priority = max_t(s32, + (s32)sched_list[0]->num_user_rqs - 1, + (s32)DRM_SCHED_PRIORITY_KERNEL); } + entity->priority = priority; + entity->rq_priority = drm_sched_policy == DRM_SCHED_POLICY_FAIR ? + DRM_SCHED_PRIORITY_KERNEL : priority; + if (sched_list[0]->sched_rq) + entity->rq = sched_list[0]->sched_rq[entity->rq_priority]; + init_completion(&entity->entity_idle); /* We start in an idle state. */ @@ -457,6 +454,8 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity, { spin_lock(&entity->lock); entity->priority = priority; + if (drm_sched_policy != DRM_SCHED_POLICY_FAIR) + entity->rq_priority = priority; spin_unlock(&entity->lock); } EXPORT_SYMBOL(drm_sched_entity_set_priority); diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c index bc88ebdcdbea..cf3df526e773 100644 --- a/drivers/gpu/drm/v3d/v3d_gem.c +++ b/drivers/gpu/drm/v3d/v3d_gem.c @@ -217,6 +217,8 @@ v3d_clean_caches(struct v3d_dev *v3d) struct drm_device *dev = &v3d->drm; int core = 0; + guard(mutex)(&v3d->cache_clean_lock); + trace_v3d_cache_clean_begin(dev); /* GFXH-1897: Ensure pending flushes complete before writing L2TCACTL */ @@ -233,7 +235,6 @@ v3d_clean_caches(struct v3d_dev *v3d) drm_err(dev, "Timeout waiting for TMU write combiner flush\n"); } - mutex_lock(&v3d->cache_clean_lock); V3D_CORE_WRITE(core, V3D_CTL_L2TCACTL, V3D_L2TCACTL_L2TFLS | V3D_SET_FIELD(V3D_L2TCACTL_FLM_CLEAN, V3D_L2TCACTL_FLM)); @@ -243,8 +244,6 @@ v3d_clean_caches(struct v3d_dev *v3d) drm_err(dev, "Timeout waiting for L2T clean\n"); } - mutex_unlock(&v3d->cache_clean_lock); - trace_v3d_cache_clean_end(dev); } diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c index e6283a75bcdf..002116efc874 100644 --- a/drivers/gpu/drm/v3d/v3d_sched.c +++ b/drivers/gpu/drm/v3d/v3d_sched.c @@ -360,6 +360,8 @@ v3d_csd_job_run(struct drm_sched_job *sched_job) dma_fence_put(job->base.irq_fence); job->base.irq_fence = dma_fence_get(fence); + job->timedout_batches = job->args.cfg[4]; + trace_v3d_submit_csd(dev, to_v3d_fence(fence)->seqno); v3d_job_start_stats(&job->base); diff --git a/drivers/gpu/drm/vc4/vc4_gem.c b/drivers/gpu/drm/vc4/vc4_gem.c index 6ee4bd77d433..0fef9b48d3c6 100644 --- a/drivers/gpu/drm/vc4/vc4_gem.c +++ b/drivers/gpu/drm/vc4/vc4_gem.c @@ -639,13 +639,14 @@ vc4_queue_submit(struct drm_device *dev, struct vc4_exec_info *exec, fence->seqno = exec->seqno; exec->fence = &fence->base; + /* The job can complete and drop exec->fence as soon as job_lock is + * released, so hold our own reference. + */ if (out_sync) - drm_syncobj_replace_fence(out_sync, exec->fence); + dma_fence_get(&fence->base); vc4_attach_fences(exec); - drm_exec_fini(exec_ctx); - list_add_tail(&exec->head, &vc4->bin_job_list); /* If no bin job was executing and if the render job (if any) has the @@ -662,6 +663,13 @@ vc4_queue_submit(struct drm_device *dev, struct vc4_exec_info *exec, spin_unlock_irqrestore(&vc4->job_lock, irqflags); + drm_exec_fini(exec_ctx); + + if (out_sync) { + drm_syncobj_replace_fence(out_sync, &fence->base); + dma_fence_put(&fence->base); + } + return 0; } diff --git a/drivers/gpu/drm/vc4/vc4_hdmi.c b/drivers/gpu/drm/vc4/vc4_hdmi.c index d4672aa81d85..f2043f907dc5 100644 --- a/drivers/gpu/drm/vc4/vc4_hdmi.c +++ b/drivers/gpu/drm/vc4/vc4_hdmi.c @@ -842,8 +842,7 @@ static void vc4_hdmi_disable_scrambling(struct drm_encoder *encoder) vc4_hdmi->scdc_enabled = false; - if (delayed_work_pending(&vc4_hdmi->scrambling_work)) - cancel_delayed_work_sync(&vc4_hdmi->scrambling_work); + cancel_delayed_work_sync(&vc4_hdmi->scrambling_work); if (!drm_dev_enter(drm, &idx)) return; @@ -918,8 +917,6 @@ static void vc4_hdmi_encoder_post_crtc_disable(struct drm_encoder *encoder, spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags); } - vc4_hdmi_disable_scrambling(encoder); - drm_dev_exit(idx); out: @@ -947,6 +944,8 @@ static void vc4_hdmi_encoder_post_crtc_powerdown(struct drm_encoder *encoder, if (vc4_hdmi->variant->phy_disable) vc4_hdmi->variant->phy_disable(vc4_hdmi); + vc4_hdmi_disable_scrambling(encoder); + clk_disable_unprepare(vc4_hdmi->pixel_bvb_clock); clk_disable_unprepare(vc4_hdmi->pixel_clock); @@ -1554,6 +1553,8 @@ static void vc4_hdmi_encoder_pre_crtc_configure(struct drm_encoder *encoder, goto err_disable_pixel_clock; } + vc4_hdmi_enable_scrambling(encoder); + if (vc4_hdmi->variant->phy_init) vc4_hdmi->variant->phy_init(vc4_hdmi, conn_state); @@ -1688,7 +1689,6 @@ static void vc4_hdmi_encoder_post_crtc_enable(struct drm_encoder *encoder, } vc4_hdmi_recenter_fifo(vc4_hdmi); - vc4_hdmi_enable_scrambling(encoder); drm_dev_exit(idx); diff --git a/drivers/gpu/drm/vc4/vc4_hdmi_phy.c b/drivers/gpu/drm/vc4/vc4_hdmi_phy.c index 56e6a35da357..5040f014bf7e 100644 --- a/drivers/gpu/drm/vc4/vc4_hdmi_phy.c +++ b/drivers/gpu/drm/vc4/vc4_hdmi_phy.c @@ -1197,4 +1197,9 @@ void vc6_hdmi_phy_init(struct vc4_hdmi *vc4_hdmi, void vc6_hdmi_phy_disable(struct vc4_hdmi *vc4_hdmi) { + unsigned long flags; + + spin_lock_irqsave(&vc4_hdmi->hw_lock, flags); + vc6_hdmi_reset_phy(vc4_hdmi); + spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags); } diff --git a/drivers/hwmon/aquacomputer_d5next.c b/drivers/hwmon/aquacomputer_d5next.c index 1ca70e726298..3d3786c82931 100644 --- a/drivers/hwmon/aquacomputer_d5next.c +++ b/drivers/hwmon/aquacomputer_d5next.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -132,6 +133,7 @@ static u8 aquaero_secondary_ctrl_report[] = { #define AQUAERO_FAN_CURRENT_OFFSET 0x06 #define AQUAERO_FAN_POWER_OFFSET 0x08 #define AQUAERO_FAN_SPEED_OFFSET 0x00 +#define AQUAERO_MIN_STATUS_REPORT_SIZE (0x18B + AQUAERO_FAN_POWER_OFFSET + sizeof(u16)) static u16 aquaero_sensor_fan_offsets[] = { 0x167, 0x173, 0x17f, 0x18B }; /* Control report offsets for the Aquaero fan controllers */ @@ -155,6 +157,8 @@ static u16 aquaero_ctrl_fan_offsets[] = { 0x20c, 0x220, 0x234, 0x248 }; #define D5NEXT_5V_VOLTAGE 0x39 #define D5NEXT_12V_VOLTAGE 0x37 #define D5NEXT_VIRTUAL_SENSORS_START 0x3f +#define D5NEXT_MIN_STATUS_REPORT_SIZE (D5NEXT_PUMP_OFFSET + AQC_FAN_SPEED_OFFSET + \ + sizeof(u16)) static u16 d5next_sensor_fan_offsets[] = { D5NEXT_PUMP_OFFSET, D5NEXT_FAN_OFFSET }; /* Control report offsets for the D5 Next pump */ @@ -179,11 +183,14 @@ static u16 d5next_ctrl_fan_offsets[] = { 0x97, 0x42 }; /* Pump and fan speed (fr #define AQUASTREAMULT_FAN_CURRENT_OFFSET 0x00 #define AQUASTREAMULT_FAN_POWER_OFFSET 0x04 #define AQUASTREAMULT_FAN_SPEED_OFFSET 0x06 +#define AQUASTREAMULT_MIN_STATUS_REPORT_SIZE (AQUASTREAMULT_PRESSURE_OFFSET + sizeof(u16)) static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET }; /* Spec and sensor report offset for the Farbwerk RGB controller */ #define FARBWERK_NUM_SENSORS 4 #define FARBWERK_SENSOR_START 0x2f +#define FARBWERK_MIN_STATUS_REPORT_SIZE (FARBWERK_SENSOR_START + \ + FARBWERK_NUM_SENSORS * AQC_SENSOR_SIZE) /* Specs of the Farbwerk 360 RGB controller */ #define FARBWERK360_NUM_SENSORS 4 @@ -193,6 +200,8 @@ static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET }; /* Sensor report offsets for the Farbwerk 360 */ #define FARBWERK360_SENSOR_START 0x32 #define FARBWERK360_VIRTUAL_SENSORS_START 0x3a +#define FARBWERK360_MIN_STATUS_REPORT_SIZE (FARBWERK360_VIRTUAL_SENSORS_START + \ + FARBWERK360_NUM_VIRTUAL_SENSORS * AQC_SENSOR_SIZE) /* Control report offsets for the Farbwerk 360 */ #define FARBWERK360_TEMP_CTRL_OFFSET 0x8 @@ -209,6 +218,7 @@ static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET }; #define OCTO_SENSOR_START 0x3D #define OCTO_VIRTUAL_SENSORS_START 0x45 #define OCTO_FLOW_SENSOR_OFFSET 0x7B +#define OCTO_MIN_STATUS_REPORT_SIZE (0xD8 + AQC_FAN_SPEED_OFFSET + sizeof(u16)) static u16 octo_sensor_fan_offsets[] = { 0x7D, 0x8A, 0x97, 0xA4, 0xB1, 0xBE, 0xCB, 0xD8 }; /* Control report offsets for the Octo */ @@ -229,6 +239,7 @@ static u16 octo_ctrl_fan_offsets[] = { 0x5B, 0xB0, 0x105, 0x15A, 0x1AF, 0x204, 0 #define QUADRO_SENSOR_START 0x34 #define QUADRO_VIRTUAL_SENSORS_START 0x3c #define QUADRO_FLOW_SENSOR_OFFSET 0x6e +#define QUADRO_MIN_STATUS_REPORT_SIZE (0x97 + AQC_FAN_SPEED_OFFSET + sizeof(u16)) static u16 quadro_sensor_fan_offsets[] = { 0x70, 0x7D, 0x8A, 0x97 }; /* Control report offsets for the Quadro */ @@ -248,6 +259,7 @@ static u16 quadro_ctrl_fan_offsets[] = { 0x37, 0x8c, 0xe1, 0x136 }; /* Fan speed #define HIGHFLOWNEXT_CONDUCTIVITY 95 #define HIGHFLOWNEXT_5V_VOLTAGE 97 #define HIGHFLOWNEXT_5V_VOLTAGE_USB 99 +#define HIGHFLOWNEXT_MIN_STATUS_REPORT_SIZE (HIGHFLOWNEXT_5V_VOLTAGE_USB + sizeof(u16)) /* Specs of the Leakshield */ #define LEAKSHIELD_NUM_SENSORS 2 @@ -263,6 +275,7 @@ static u16 quadro_ctrl_fan_offsets[] = { 0x37, 0x8c, 0xe1, 0x136 }; /* Fan speed #define LEAKSHIELD_FLOW_IN 111 #define LEAKSHIELD_RESERVOIR_VOLUME 313 #define LEAKSHIELD_RESERVOIR_FILLED 311 +#define LEAKSHIELD_MIN_STATUS_REPORT_SIZE (LEAKSHIELD_RESERVOIR_VOLUME + sizeof(u16)) /* Specs of the Aquastream XT pump */ #define AQUASTREAMXT_SERIAL_START 0x3a @@ -554,6 +567,7 @@ struct aqc_data { const char *name; int status_report_id; /* Used for legacy devices, report is stored in buffer */ + u16 min_report_size; int ctrl_report_id; int secondary_ctrl_report_id; int secondary_ctrl_report_size; @@ -599,9 +613,9 @@ struct aqc_data { u32 speed_input_min[1]; u32 speed_input_target[1]; u32 speed_input_max[1]; - u32 power_input[8]; + long power_input[8]; u16 voltage_input[8]; - u16 current_input[8]; + u32 current_input[8]; /* Label values */ const char *const *temp_label; @@ -976,7 +990,7 @@ static int aqc_legacy_read(struct aqc_data *priv) /* Calculation derived from linear regression */ sensor_value = get_unaligned_le16(priv->buffer + AQUASTREAMXT_PUMP_CURR_OFFSET); - priv->current_input[0] = DIV_ROUND_CLOSEST(sensor_value * 176, 100) - 52; + priv->current_input[0] = max(DIV_ROUND_CLOSEST(sensor_value * 176, 100) - 52, 0); sensor_value = get_unaligned_le16(priv->buffer + AQUASTREAMXT_PUMP_VOLTAGE_OFFSET); priv->voltage_input[0] = DIV_ROUND_CLOSEST(sensor_value * 1000, 61); @@ -1006,6 +1020,7 @@ static int aqc_legacy_read(struct aqc_data *priv) static int aqc_read(struct device *dev, enum hwmon_sensor_types type, u32 attr, int channel, long *val) { + long power; int ret; struct aqc_data *priv = dev_get_drvdata(dev); @@ -1070,7 +1085,10 @@ static int aqc_read(struct device *dev, enum hwmon_sensor_types type, u32 attr, } break; case hwmon_power: - *val = priv->power_input[channel]; + power = READ_ONCE(priv->power_input[channel]); + if (power == -ENODATA) + return -ENODATA; + *val = power; break; case hwmon_pwm: switch (priv->kind) { @@ -1333,6 +1351,8 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8 return 0; priv = hid_get_drvdata(hdev); + if (!priv->min_report_size || size < priv->min_report_size) + return 0; /* Info provided with every report */ priv->serial_number[0] = get_unaligned_be16(data + priv->serial_number_start_offset); @@ -1368,9 +1388,9 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8 priv->speed_input[i] = get_unaligned_be16(data + priv->fan_sensor_offsets[i] + priv->fan_structure->speed); - priv->power_input[i] = - get_unaligned_be16(data + priv->fan_sensor_offsets[i] + - priv->fan_structure->power) * 10000; + WRITE_ONCE(priv->power_input[i], + get_unaligned_be16(data + priv->fan_sensor_offsets[i] + + priv->fan_structure->power) * 10000); priv->voltage_input[i] = get_unaligned_be16(data + priv->fan_sensor_offsets[i] + priv->fan_structure->voltage) * 10; @@ -1410,7 +1430,8 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8 priv->speed_input[2] = get_unaligned_be16(data + AQUASTREAMULT_PRESSURE_OFFSET); priv->speed_input[3] = get_unaligned_be16(data + AQUASTREAMULT_FLOW_SENSOR_OFFSET); - priv->power_input[1] = get_unaligned_be16(data + AQUASTREAMULT_PUMP_POWER) * 10000; + WRITE_ONCE(priv->power_input[1], + get_unaligned_be16(data + AQUASTREAMULT_PUMP_POWER) * 10000); priv->voltage_input[1] = get_unaligned_be16(data + AQUASTREAMULT_PUMP_VOLTAGE) * 10; @@ -1423,10 +1444,11 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8 case highflownext: /* If external temp sensor is not connected, its power reading is also N/A */ if (priv->temp_input[1] == -ENODATA) - priv->power_input[0] = -ENODATA; + WRITE_ONCE(priv->power_input[0], -ENODATA); else - priv->power_input[0] = - get_unaligned_be16(data + HIGHFLOWNEXT_POWER) * 1000000; + WRITE_ONCE(priv->power_input[0], + min_t(u64, get_unaligned_be16(data + HIGHFLOWNEXT_POWER) * + 1000000ULL, LONG_MAX)); priv->voltage_input[0] = get_unaligned_be16(data + HIGHFLOWNEXT_5V_VOLTAGE) * 10; priv->voltage_input[1] = @@ -1518,6 +1540,7 @@ static void aqc_debugfs_init(struct aqc_data *priv) static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) { struct aqc_data *priv; + u16 min_report_size = 0; int ret; priv = devm_kzalloc(&hdev->dev, sizeof(*priv), GFP_KERNEL); @@ -1554,7 +1577,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) * they present. The two other devices have the type of the second element in * their respective collections set to 1, while the real device has it set to 0. */ - if (hdev->collection[1].type != 0) { + if (hdev->maxcollection < 2 || hdev->collection[1].type != 0) { ret = -ENODEV; goto fail_and_close; } @@ -1585,6 +1608,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->power_label = label_fan_power; priv->voltage_label = label_fan_voltage; priv->current_label = label_fan_current; + min_report_size = AQUAERO_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_D5NEXT: priv->kind = d5next; @@ -1610,6 +1634,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->power_label = label_d5next_power; priv->voltage_label = label_d5next_voltages; priv->current_label = label_d5next_current; + min_report_size = D5NEXT_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_FARBWERK: priv->kind = farbwerk; @@ -1620,6 +1645,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->temp_sensor_start_offset = FARBWERK_SENSOR_START; priv->temp_label = label_temp_sensors; + min_report_size = FARBWERK_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_FARBWERK360: priv->kind = farbwerk360; @@ -1636,6 +1662,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->temp_label = label_temp_sensors; priv->virtual_temp_label = label_virtual_temp_sensors; + min_report_size = FARBWERK360_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_OCTO: priv->kind = octo; @@ -1665,6 +1692,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->power_label = label_fan_power; priv->voltage_label = label_fan_voltage; priv->current_label = label_fan_current; + min_report_size = OCTO_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_QUADRO: priv->kind = quadro; @@ -1694,6 +1722,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->power_label = label_fan_power; priv->voltage_label = label_fan_voltage; priv->current_label = label_fan_current; + min_report_size = QUADRO_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_HIGHFLOWNEXT: priv->kind = highflownext; @@ -1711,6 +1740,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->speed_label = label_highflownext_fan_speed; priv->power_label = label_highflownext_power; priv->voltage_label = label_highflownext_voltage; + min_report_size = HIGHFLOWNEXT_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_LEAKSHIELD: /* @@ -1730,6 +1760,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->temp_label = label_leakshield_temp_sensors; priv->speed_label = label_leakshield_fan_speed; + min_report_size = LEAKSHIELD_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_AQUASTREAMXT: priv->kind = aquastreamxt; @@ -1760,6 +1791,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->power_label = label_aquastreamult_power; priv->voltage_label = label_aquastreamult_voltages; priv->current_label = label_aquastreamult_current; + min_report_size = AQUASTREAMULT_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_POWERADJUST3: priv->kind = poweradjust3; @@ -1833,6 +1865,8 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) break; } + priv->min_report_size = min_report_size; + if (priv->buffer_size != 0) { priv->checksum_start = 0x01; priv->checksum_length = priv->buffer_size - 3; diff --git a/drivers/hwmon/asus_rog_ryujin.c b/drivers/hwmon/asus_rog_ryujin.c index e297557ca346..a400e523d2e9 100644 --- a/drivers/hwmon/asus_rog_ryujin.c +++ b/drivers/hwmon/asus_rog_ryujin.c @@ -49,6 +49,7 @@ static const struct rog_ryujin_device_info rog_ryujin_iii_info = { #define MAX_REPORT_LENGTH 65 /* Cooler duty report offsets */ +#define RYUJIN_PUMP_SOURCE 3 #define RYUJIN_PUMP_DUTY 4 #define RYUJIN_INTERNAL_FAN_DUTY 5 @@ -69,10 +70,15 @@ static const u8 get_cooler_duty_cmd[] = { RYUJIN_CMD_PREFIX, 0x9A }; static const u8 get_controller_speed_cmd[] = { RYUJIN_CMD_PREFIX, 0xA0 }; static const u8 get_controller_duty_cmd[] = { RYUJIN_CMD_PREFIX, 0xA1 }; +#define RYUJIN_SET_COOLER_CHANNEL_OFFSET 2 #define RYUJIN_SET_COOLER_PUMP_DUTY_OFFSET 3 #define RYUJIN_SET_COOLER_FAN_DUTY_OFFSET 4 static const u8 set_cooler_duty_cmd[] = { RYUJIN_CMD_PREFIX, 0x1A, 0x00, 0x00, 0x00 }; +/* Pump control sources in the set cooler duty command and the cooler duty report */ +#define RYUJIN_SOURCE_HEADER 0 +#define RYUJIN_SOURCE_USB 1 + #define RYUJIN_SET_CONTROLLER_FAN_DUTY_OFFSET 4 static const u8 set_controller_duty_cmd[] = { RYUJIN_CMD_PREFIX, 0x21, 0x00, 0x00, 0x00 }; @@ -116,6 +122,7 @@ struct rog_ryujin_data { s32 temp_input[1]; u16 speed_input[6]; /* Pump, internal fan and four controller fan speeds in RPM */ u8 duty_input[3]; /* Pump, internal fan and controller fan duty in PWM */ + u8 pump_source; /* Pump control source from the cooler duty report */ u8 *buffer; unsigned long updated; /* jiffies */ @@ -305,6 +312,44 @@ static int rog_ryujin_read_string(struct device *dev, enum hwmon_sensor_types ty return 0; } +/* + * On devices with a non-zero duty_channel (the RYUJIN III), the channel byte of + * the set cooler duty command selects the pump control source. + * RYUJIN_SOURCE_HEADER hands the pump to its internal controller, which follows + * the PWM of the header its 4-pin cable is plugged into. RYUJIN_SOURCE_USB + * makes the pump follow the pump duty in the command. The cooler duty report + * returns the current source. + * + * A fan duty write sends the reported source back, so that it leaves the pump + * control source as it is. A pump duty write selects USB. The cooler does not + * pass on a pump duty equal to the one it reports, and after power-on it + * reports 40% while a pump handed over at that duty runs at full speed. So + * when a pump duty write asks for the reported duty, write a duty 1% away + * first. + */ +static int rog_ryujin_select_pump_source(struct rog_ryujin_data *priv, u8 *set_cmd, + int channel) +{ + u8 duty = set_cmd[RYUJIN_SET_COOLER_PUMP_DUTY_OFFSET]; + int ret; + + if (channel == 1) { + set_cmd[RYUJIN_SET_COOLER_CHANNEL_OFFSET] = priv->pump_source; + return 0; + } + + set_cmd[RYUJIN_SET_COOLER_CHANNEL_OFFSET] = RYUJIN_SOURCE_USB; + + if (duty != rog_ryujin_pwm_to_percent(priv->duty_input[0])) + return 0; + + set_cmd[RYUJIN_SET_COOLER_PUMP_DUTY_OFFSET] = duty < 100 ? duty + 1 : duty - 1; + ret = rog_ryujin_execute_cmd(priv, set_cmd, SET_CMD_LENGTH, &priv->cooler_duty_set); + set_cmd[RYUJIN_SET_COOLER_PUMP_DUTY_OFFSET] = duty; + + return ret; +} + static int rog_ryujin_write_fixed_duty(struct rog_ryujin_data *priv, int channel, int val) { u8 set_cmd[SET_CMD_LENGTH]; @@ -322,7 +367,7 @@ static int rog_ryujin_write_fixed_duty(struct rog_ryujin_data *priv, int channel return ret; memcpy(set_cmd, set_cooler_duty_cmd, SET_CMD_LENGTH); - set_cmd[2] = priv->info->duty_channel; + set_cmd[RYUJIN_SET_COOLER_CHANNEL_OFFSET] = priv->info->duty_channel; /* Cooler duties are set as 0-100% */ val = rog_ryujin_pwm_to_percent(val); @@ -339,6 +384,12 @@ static int rog_ryujin_write_fixed_duty(struct rog_ryujin_data *priv, int channel set_cmd[RYUJIN_SET_COOLER_FAN_DUTY_OFFSET] = val; } + if (priv->info->duty_channel) { + ret = rog_ryujin_select_pump_source(priv, set_cmd, channel); + if (ret < 0) + return ret; + } + return rog_ryujin_execute_cmd(priv, set_cmd, SET_CMD_LENGTH, &priv->cooler_duty_set); } else { /* @@ -481,6 +532,7 @@ static int rog_ryujin_raw_event(struct hid_device *hdev, struct hid_report *repo goto unlock; } read_cooler_duty: + priv->pump_source = data[RYUJIN_PUMP_SOURCE]; priv->duty_input[0] = rog_ryujin_percent_to_pwm(data[RYUJIN_PUMP_DUTY]); priv->duty_input[1] = rog_ryujin_percent_to_pwm(data[RYUJIN_INTERNAL_FAN_DUTY]); diff --git a/drivers/hwmon/asus_wmi_sensors.c b/drivers/hwmon/asus_wmi_sensors.c index c2dd7ff882f2..502996e97cf1 100644 --- a/drivers/hwmon/asus_wmi_sensors.c +++ b/drivers/hwmon/asus_wmi_sensors.c @@ -164,6 +164,7 @@ struct asus_wmi_wmi_info { struct asus_wmi_sensors { struct asus_wmi_wmi_info wmi; + struct hwmon_chip_info chip_info; /* lock access to internal cache */ struct mutex lock; }; @@ -496,11 +497,6 @@ static const struct hwmon_ops asus_wmi_hwmon_ops = { .read_string = asus_wmi_hwmon_read_string, }; -static struct hwmon_chip_info asus_wmi_chip_info = { - .ops = &asus_wmi_hwmon_ops, - .info = NULL, -}; - static int asus_wmi_configure_sensor_setup(struct device *dev, struct asus_wmi_sensors *sensor_data) { @@ -508,7 +504,7 @@ static int asus_wmi_configure_sensor_setup(struct device *dev, struct hwmon_channel_info *asus_wmi_hwmon_chan; int nr_count[hwmon_max] = {}, nr_types = 0; struct asus_wmi_sensor_info *temp_sensor; - const struct hwmon_chip_info *chip_info; + struct hwmon_chip_info *chip_info; enum hwmon_sensor_types type; struct device *hwdev; int i, idx; @@ -549,8 +545,9 @@ static int asus_wmi_configure_sensor_setup(struct device *dev, if (!ptr_asus_wmi_ci) return -ENOMEM; - asus_wmi_chip_info.info = ptr_asus_wmi_ci; - chip_info = &asus_wmi_chip_info; + chip_info = &sensor_data->chip_info; + chip_info->ops = &asus_wmi_hwmon_ops; + chip_info->info = ptr_asus_wmi_ci; sensor_data->wmi.info_by_id = devm_kcalloc(dev, sensor_data->wmi.sensor_count, sizeof(*sensor_data->wmi.info_by_id), diff --git a/drivers/hwmon/chipcap2.c b/drivers/hwmon/chipcap2.c index 9bef767b589e..00f62a33d463 100644 --- a/drivers/hwmon/chipcap2.c +++ b/drivers/hwmon/chipcap2.c @@ -20,7 +20,9 @@ #include #include #include +#include #include +#include #include #include #include @@ -90,6 +92,9 @@ struct cc2_data { int irq_low; int irq_high; bool process_irqs; + /* protects process_irqs and the alarm indicators */ + struct mutex irq_lock; + bool regulator_enabled; }; /* %RH as a per cent mille from a register value */ @@ -118,20 +123,33 @@ static int cc2_enable(struct cc2_data *data) { int ret; + if (data->irq_ready > 0) { + scoped_guard(mutex, &data->irq_lock) + data->process_irqs = false; + + synchronize_irq(data->irq_ready); + + scoped_guard(mutex, &data->irq_lock) + reinit_completion(&data->complete); + } + /* exclusive regulator, check in case a disable failed */ - if (regulator_is_enabled(data->regulator)) + if (data->regulator_enabled) { + scoped_guard(mutex, &data->irq_lock) + data->process_irqs = true; return 0; - - /* clear any pending completion */ - try_wait_for_completion(&data->complete); + } ret = regulator_enable(data->regulator); if (ret < 0) return ret; + data->regulator_enabled = true; + usleep_range(CC2_STARTUP_TIME_US, CC2_STARTUP_TIME_US + 125); - data->process_irqs = true; + scoped_guard(mutex, &data->irq_lock) + data->process_irqs = true; return 0; } @@ -140,15 +158,36 @@ static void cc2_disable(struct cc2_data *data) { int err; + /* let the alarms already raised by the last measurement be processed */ + if (data->irq_low > 0) + synchronize_irq(data->irq_low); + + if (data->irq_high > 0) + synchronize_irq(data->irq_high); + /* ignore alarms triggered by voltage toggling when powering up */ - data->process_irqs = false; + scoped_guard(mutex, &data->irq_lock) + data->process_irqs = false; + + if (data->irq_ready > 0) + synchronize_irq(data->irq_ready); /* exclusive regulator, check in case an enable failed */ - if (regulator_is_enabled(data->regulator)) { - err = regulator_disable(data->regulator); - if (err) - dev_dbg(&data->client->dev, "Failed to disable device"); + if (!data->regulator_enabled) + return; + + err = regulator_disable(data->regulator); + if (err) { + dev_dbg(&data->client->dev, "Failed to disable device"); + return; } + + data->regulator_enabled = false; +} + +static void cc2_disable_action(void *data) +{ + cc2_disable(data); } static int cc2_cmd_response_diagnostic(struct device *dev, u8 status) @@ -167,13 +206,13 @@ static int cc2_cmd_response_diagnostic(struct device *dev, u8 status) case CC2_RESPONSE_BUSY: return -EBUSY; case CC2_RESPONSE_NACK: - if (resp & CC2_ERR_CORR_EEPROM) + if (status & CC2_ERR_CORR_EEPROM) dev_dbg(dev, "Command failed: corrected EEPROM\n"); - if (resp & CC2_ERR_UNCORR_EEPROM) + if (status & CC2_ERR_UNCORR_EEPROM) dev_dbg(dev, "Command failed: uncorrected EEPROM\n"); - if (resp & CC2_ERR_RAM_PARITY) + if (status & CC2_ERR_RAM_PARITY) dev_dbg(dev, "Command failed: RAM parity\n"); - if (resp & CC2_ERR_RAM_PARITY) + if (status & CC2_ERR_CONFIG_LOAD) dev_dbg(dev, "Command failed: configuration error\n"); return -ENODATA; default: @@ -226,7 +265,7 @@ static int cc2_command_mode_start(struct cc2_data *data) 2 * CC2_RESP_START_CM_US); } ret = cc2_read_command_status(data->client); - if (ret != -ETIMEDOUT || i == CC2_CM_RETRIES) + if (ret != -ETIMEDOUT || i == CC2_CM_RETRIES - 1) break; /* command window missed, prepare for a retry */ @@ -260,8 +299,7 @@ static int cc2_write_reg(struct cc2_data *data, u8 reg, u16 val) if (ret < 0) goto disable; - cpu_to_be16s(&val); - ret = i2c_smbus_write_word_data(data->client, reg, val); + ret = i2c_smbus_write_word_swapped(data->client, reg, val); if (ret < 0) goto disable; @@ -311,7 +349,7 @@ static int cc2_read_reg(struct cc2_data *data, u8 reg, u16 *val) if (ret != CC2_EEPROM_DATA_LEN) return ret < 0 ? ret : -EIO; - *val = be16_to_cpup((__be16 *)&buf[1]); + *val = get_unaligned_be16(&buf[1]); return cc2_read_command_status(data->client); } @@ -351,10 +389,10 @@ static int cc2_data_fetch(struct i2c_client *client, switch (type) { case hwmon_humidity: - *val = cc2_rh_convert(be16_to_cpup((__be16 *)&data[0])); + *val = cc2_rh_convert(get_unaligned_be16(&data[0])); break; case hwmon_temp: - *val = cc2_temp_convert(be16_to_cpup((__be16 *)&data[2])); + *val = cc2_temp_convert(get_unaligned_be16(&data[2])); break; default: return -EINVAL; @@ -482,8 +520,10 @@ static irqreturn_t cc2_ready_interrupt(int irq, void *data) { struct cc2_data *cc2 = data; - if (cc2->process_irqs) - complete(&cc2->complete); + scoped_guard(mutex, &cc2->irq_lock) { + if (cc2->process_irqs) + complete(&cc2->complete); + } return IRQ_HANDLED; } @@ -492,10 +532,12 @@ static irqreturn_t cc2_low_interrupt(int irq, void *data) { struct cc2_data *cc2 = data; - if (cc2->process_irqs) { - hwmon_notify_event(cc2->hwmon, hwmon_humidity, - hwmon_humidity_min_alarm, 0); - cc2->rh_alarm.low_alarm = true; + scoped_guard(mutex, &cc2->irq_lock) { + if (cc2->process_irqs) { + hwmon_notify_event(cc2->hwmon, hwmon_humidity, + hwmon_humidity_min_alarm, 0); + cc2->rh_alarm.low_alarm = true; + } } return IRQ_HANDLED; @@ -505,10 +547,12 @@ static irqreturn_t cc2_high_interrupt(int irq, void *data) { struct cc2_data *cc2 = data; - if (cc2->process_irqs) { - hwmon_notify_event(cc2->hwmon, hwmon_humidity, - hwmon_humidity_max_alarm, 0); - cc2->rh_alarm.high_alarm = true; + scoped_guard(mutex, &cc2->irq_lock) { + if (cc2->process_irqs) { + hwmon_notify_event(cc2->hwmon, hwmon_humidity, + hwmon_humidity_max_alarm, 0); + cc2->rh_alarm.high_alarm = true; + } } return IRQ_HANDLED; @@ -524,11 +568,13 @@ static int cc2_humidity_min_alarm_status(struct cc2_data *data, long *val) if (ret < 0) return ret; - if (data->rh_alarm.low_alarm) { - *val = (measurement < min_hyst) ? 1 : 0; - data->rh_alarm.low_alarm = *val; - } else { - *val = 0; + scoped_guard(mutex, &data->irq_lock) { + if (data->rh_alarm.low_alarm) { + *val = (measurement < min_hyst) ? 1 : 0; + data->rh_alarm.low_alarm = *val; + } else { + *val = 0; + } } return 0; @@ -544,11 +590,13 @@ static int cc2_humidity_max_alarm_status(struct cc2_data *data, long *val) if (ret < 0) return ret; - if (data->rh_alarm.high_alarm) { - *val = (measurement > max_hyst) ? 1 : 0; - data->rh_alarm.high_alarm = *val; - } else { - *val = 0; + scoped_guard(mutex, &data->irq_lock) { + if (data->rh_alarm.high_alarm) { + *val = (measurement > max_hyst) ? 1 : 0; + data->rh_alarm.high_alarm = *val; + } else { + *val = 0; + } } return 0; @@ -631,28 +679,47 @@ static int cc2_write(struct device *dev, enum hwmon_sensor_types type, u32 attr, } } -static int cc2_request_ready_irq(struct cc2_data *data, struct device *dev) +static int cc2_get_irqs(struct cc2_data *data, struct device *dev) { - int ret = 0; - data->irq_ready = fwnode_irq_get_byname(dev_fwnode(dev), "ready"); - if (data->irq_ready > 0) { + if (data->irq_ready == -EPROBE_DEFER) + return data->irq_ready; + if (data->irq_ready > 0) init_completion(&data->complete); + + data->irq_low = fwnode_irq_get_byname(dev_fwnode(dev), "low"); + if (data->irq_low == -EPROBE_DEFER) + return data->irq_low; + if (data->irq_low > 0) + data->rh_alarm.low_alarm_visible = true; + + data->irq_high = fwnode_irq_get_byname(dev_fwnode(dev), "high"); + if (data->irq_high == -EPROBE_DEFER) + return data->irq_high; + if (data->irq_high > 0) + data->rh_alarm.high_alarm_visible = true; + + return 0; +} + +static int cc2_request_data_ready(struct cc2_data *data, struct device *dev) +{ + int ret = 0; + + if (data->irq_ready > 0) ret = devm_request_threaded_irq(dev, data->irq_ready, NULL, cc2_ready_interrupt, IRQF_ONESHOT | IRQF_TRIGGER_RISING, dev_name(dev), data); - } return ret; } -static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev) +static int cc2_request_alarms(struct cc2_data *data, struct device *dev) { int ret = 0; - data->irq_low = fwnode_irq_get_byname(dev_fwnode(dev), "low"); if (data->irq_low > 0) { ret = devm_request_threaded_irq(dev, data->irq_low, NULL, cc2_low_interrupt, @@ -661,11 +728,8 @@ static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev) dev_name(dev), data); if (ret) return ret; - - data->rh_alarm.low_alarm_visible = true; } - data->irq_high = fwnode_irq_get_byname(dev_fwnode(dev), "high"); if (data->irq_high > 0) { ret = devm_request_threaded_irq(dev, data->irq_high, NULL, cc2_high_interrupt, @@ -674,8 +738,6 @@ static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev) dev_name(dev), data); if (ret) return ret; - - data->rh_alarm.high_alarm_visible = true; } return ret; @@ -717,6 +779,8 @@ static int cc2_probe(struct i2c_client *client) i2c_set_clientdata(client, data); + mutex_init(&data->irq_lock); + data->client = client; data->regulator = devm_regulator_get_exclusive(dev, "vdd"); @@ -726,13 +790,17 @@ static int cc2_probe(struct i2c_client *client) device_property_read_string(dev, "label", &data->label); - ret = cc2_request_ready_irq(data, dev); + ret = cc2_get_irqs(data, dev); if (ret) - return dev_err_probe(dev, ret, "Failed to request ready irq\n"); + return dev_err_probe(dev, ret, "Failed to get IRQs\n"); - ret = cc2_request_alarm_irqs(data, dev); + ret = devm_add_action_or_reset(dev, cc2_disable_action, data); if (ret) - return dev_err_probe(dev, ret, "Failed to request alarm irqs\n"); + return ret; + + ret = cc2_request_data_ready(data, dev); + if (ret) + return dev_err_probe(dev, ret, "Failed to request ready irq\n"); data->hwmon = devm_hwmon_device_register_with_info(dev, client->name, data, &cc2_chip_info, @@ -741,14 +809,11 @@ static int cc2_probe(struct i2c_client *client) return dev_err_probe(dev, PTR_ERR(data->hwmon), "Failed to register hwmon device\n"); - return 0; -} - -static void cc2_remove(struct i2c_client *client) -{ - struct cc2_data *data = i2c_get_clientdata(client); + ret = cc2_request_alarms(data, dev); + if (ret) + return dev_err_probe(dev, ret, "Failed to request alarm irqs\n"); - cc2_disable(data); + return 0; } static const struct i2c_device_id cc2_id[] = { @@ -783,7 +848,6 @@ static struct i2c_driver cc2_driver = { .of_match_table = cc2_of_match, }, .probe = cc2_probe, - .remove = cc2_remove, .id_table = cc2_id, }; module_i2c_driver(cc2_driver); diff --git a/drivers/hwmon/coretemp.c b/drivers/hwmon/coretemp.c index 9ed21c8c093a..2d7487b2f80a 100644 --- a/drivers/hwmon/coretemp.c +++ b/drivers/hwmon/coretemp.c @@ -491,11 +491,12 @@ init_temp_data(struct platform_data *pdata, unsigned int cpu, int pkg_flag) struct temp_data *tdata; if (!pdata->core_data) { - pdata->nr_cores = topology_num_cores_per_package(); - pdata->core_data = kzalloc_objs(struct temp_data *, - pdata->nr_cores); + int nr_cores = topology_num_cores_per_package(); + + pdata->core_data = kzalloc_objs(struct temp_data *, nr_cores); if (!pdata->core_data) return NULL; + pdata->nr_cores = nr_cores; } tdata = kzalloc_obj(struct temp_data); @@ -695,6 +696,9 @@ static int coretemp_cpu_online(unsigned int cpu) if (!housekeeping_cpu(cpu, HK_TYPE_MISC)) return 0; + if (!pdev) + return 0; + pdata = platform_get_drvdata(pdev); if (!pdata->hwmon_dev) { struct device *hwmon; @@ -746,6 +750,9 @@ static int coretemp_cpu_offline(unsigned int cpu) return 0; /* If the physical CPU device does not exist, just return */ + if (!pdev) + return 0; + pd = platform_get_drvdata(pdev); if (!pd->hwmon_dev) return 0; diff --git a/drivers/hwmon/corsair-cpro.c b/drivers/hwmon/corsair-cpro.c index e6d31d367698..6ce5da37de03 100644 --- a/drivers/hwmon/corsair-cpro.c +++ b/drivers/hwmon/corsair-cpro.c @@ -215,10 +215,11 @@ static int set_target(struct ccp_device *ccp, int channel, long val) int ret; val = clamp_val(val, 0, 0xFFFF); - ccp->target[channel] = val; mutex_lock(&ccp->mutex); ret = send_usb_cmd(ccp, CTL_SET_FAN_TARGET, channel, val >> 8, val); + if (!ret) + ccp->target[channel] = val; mutex_unlock(&ccp->mutex); return ret; diff --git a/drivers/hwmon/gpd-fan.c b/drivers/hwmon/gpd-fan.c index ed212615b2fe..b3b75f99f588 100644 --- a/drivers/hwmon/gpd-fan.c +++ b/drivers/hwmon/gpd-fan.c @@ -444,6 +444,7 @@ static void gpd_duo_set_pwm_enable(struct gpd_fan_data *data, enum FAN_PWM_ENABL break; case AUTOMATIC: gpd_ecram_write(data->drvdata, data->drvdata->pwm_write, 0); + gpd_ecram_write(data->drvdata, data->drvdata->pwm_write + 1, 0); break; } } diff --git a/drivers/hwmon/gpio-fan.c b/drivers/hwmon/gpio-fan.c index 3f78375eeb44..08b58b49be5b 100644 --- a/drivers/hwmon/gpio-fan.c +++ b/drivers/hwmon/gpio-fan.c @@ -150,7 +150,10 @@ static int set_fan_speed(struct gpio_fan_data *fan_data, int speed_index) __set_fan_ctrl(fan_data, fan_data->speed[speed_index].ctrl_val); - if (fan_data->speed_index > 0 && speed_index == 0) { + /* The GPIOs have changed, and a PM put drops the reference on error. */ + fan_data->speed_index = speed_index; + + if (speed_index == 0) { int ret; ret = pm_runtime_put_sync(fan_data->dev); @@ -158,8 +161,6 @@ static int set_fan_speed(struct gpio_fan_data *fan_data, int speed_index) return ret; } - fan_data->speed_index = speed_index; - return 0; } diff --git a/drivers/hwmon/hp-wmi-sensors.c b/drivers/hwmon/hp-wmi-sensors.c index cfa0d6ad0fb3..12950666c981 100644 --- a/drivers/hwmon/hp-wmi-sensors.c +++ b/drivers/hwmon/hp-wmi-sensors.c @@ -390,6 +390,7 @@ struct hp_wmi_info { */ struct hp_wmi_sensors { struct wmi_device *wdev; + struct hwmon_chip_info chip_info; struct hp_wmi_info **info_map[hwmon_max]; u8 channel_count[hwmon_max]; bool has_intrusion; @@ -738,15 +739,21 @@ static long scale_numeric_sensor(const struct hp_wmi_numeric_sensor *nsensor) val = current_reading; - for (; unit_modifier < target_modifier; unit_modifier++) + for (; val && unit_modifier < target_modifier; unit_modifier++) val = DIV_ROUND_CLOSEST(val, 10); - for (; unit_modifier > target_modifier; unit_modifier--) { - if (val > LONG_MAX / 10) { - val = LONG_MAX; - break; + if (val) { + for (; unit_modifier > target_modifier; unit_modifier--) { + if (val > LONG_MAX / 10) { + val = LONG_MAX; + break; + } + if (val < LONG_MIN / 10) { + val = LONG_MIN; + break; + } + val *= 10; } - val *= 10; } if (sensor_type == HP_WMI_TYPE_TEMPERATURE) { @@ -1531,11 +1538,6 @@ static const struct hwmon_ops hp_wmi_hwmon_ops = { .write = hp_wmi_hwmon_write, }; -static struct hwmon_chip_info hp_wmi_chip_info = { - .ops = &hp_wmi_hwmon_ops, - .info = NULL, -}; - static struct hp_wmi_info *match_fan_event(struct hp_wmi_sensors *state, const char *event_description) { @@ -1962,7 +1964,8 @@ static int make_chip_info(struct hp_wmi_sensors *state, bool has_events) if (!ptr_channel_info) return -ENOMEM; - hp_wmi_chip_info.info = ptr_channel_info; + state->chip_info.ops = &hp_wmi_hwmon_ops; + state->chip_info.info = ptr_channel_info; for (type = hwmon_chip; type < hwmon_max; type++) { count = channel_count[type]; @@ -2053,7 +2056,7 @@ static int hp_wmi_sensors_init(struct hp_wmi_sensors *state) return err; hwdev = devm_hwmon_device_register_with_info(dev, "hp_wmi_sensors", - state, &hp_wmi_chip_info, + state, &state->chip_info, NULL); return PTR_ERR_OR_ZERO(hwdev); } diff --git a/drivers/hwmon/lm75.c b/drivers/hwmon/lm75.c index 2d2d752aeac9..917ac250a648 100644 --- a/drivers/hwmon/lm75.c +++ b/drivers/hwmon/lm75.c @@ -123,8 +123,7 @@ struct lm75_data { static const u8 lm75_sample_set_masks[] = { 0 << 5, 1 << 5, 2 << 5, 3 << 5 }; -#define LM75_ALERT_POLARITY_HIGH_8_BIT (BIT(2)) -#define LM75_ALERT_POLARITY_HIGH_16_BIT (BIT(2) << 8) +#define LM75_ALERT_POLARITY_HIGH_BIT BIT(2) #define LM75_SAMPLE_CLEAR_MASK (3 << 5) /* The structure below stores the configuration values of the supported devices. @@ -770,11 +769,7 @@ static int lm75_generic_probe(struct device *dev, const char *name, data->orig_conf = status; /* Enforce polarity active-low (default) or active-high (devicetree) */ - if (!data->params->config_reg_16bits) - pol_mask = LM75_ALERT_POLARITY_HIGH_8_BIT; - else - pol_mask = LM75_ALERT_POLARITY_HIGH_16_BIT; - + pol_mask = LM75_ALERT_POLARITY_HIGH_BIT; clr_mask = data->params->clr_mask | pol_mask; set_mask = data->params->set_mask & ~pol_mask; if (device_property_read_bool(dev, "ti,alert-polarity-active-high")) diff --git a/drivers/hwmon/lm87.c b/drivers/hwmon/lm87.c index c09074f44708..bb4e15b66355 100644 --- a/drivers/hwmon/lm87.c +++ b/drivers/hwmon/lm87.c @@ -606,13 +606,16 @@ static DEVICE_ATTR_RO(cpu0_vid); static ssize_t vrm_show(struct device *dev, struct device_attribute *attr, char *buf) { - struct lm87_data *data = dev_get_drvdata(dev); + struct i2c_client *client = dev_get_drvdata(dev); + struct lm87_data *data = i2c_get_clientdata(client); + return sprintf(buf, "%d\n", data->vrm); } static ssize_t vrm_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t count) { - struct lm87_data *data = dev_get_drvdata(dev); + struct i2c_client *client = dev_get_drvdata(dev); + struct lm87_data *data = i2c_get_clientdata(client); unsigned long val; int err; diff --git a/drivers/hwmon/nct6683.c b/drivers/hwmon/nct6683.c index e1c36c95affb..f34cb62305be 100644 --- a/drivers/hwmon/nct6683.c +++ b/drivers/hwmon/nct6683.c @@ -920,7 +920,7 @@ static ssize_t show_pwm(struct device *dev, struct device_attribute *attr, char *buf) { struct nct6683_data *data = nct6683_update_device(dev); - struct sensor_device_attribute_2 *sattr = to_sensor_dev_attr_2(attr); + struct sensor_device_attribute *sattr = to_sensor_dev_attr(attr); int index = sattr->index; return sprintf(buf, "%d\n", data->pwm[index]); @@ -930,7 +930,7 @@ static ssize_t store_pwm(struct device *dev, struct device_attribute *attr, const char *buf, size_t count) { - struct sensor_device_attribute_2 *sattr = to_sensor_dev_attr_2(attr); + struct sensor_device_attribute *sattr = to_sensor_dev_attr(attr); struct nct6683_data *data = dev_get_drvdata(dev); int index = sattr->index; unsigned long val; diff --git a/drivers/hwmon/peci/dimmtemp.c b/drivers/hwmon/peci/dimmtemp.c index bd3e8715dfec..617a4cc0f65c 100644 --- a/drivers/hwmon/peci/dimmtemp.c +++ b/drivers/hwmon/peci/dimmtemp.c @@ -4,7 +4,6 @@ #include #include #include -#include #include #include #include @@ -326,11 +325,13 @@ static int create_dimm_temp_info(struct peci_dimmtemp *priv) if (ret == -ENODEV) { dev_dbg(priv->dev, "No DIMMs found\n"); return 0; - } else if (ret) { + } + if (ret == -EAGAIN) { schedule_delayed_work(&priv->detect_work, DIMM_MASK_CHECK_DELAY_JIFFIES); dev_dbg(priv->dev, "Deferred populating DIMM temp info\n"); - return ret; } + if (ret) + return ret; channels = priv->gen_info->chan_rank_max * priv->gen_info->dimm_idx_max; @@ -398,10 +399,8 @@ static int peci_dimmtemp_probe(struct auxiliary_device *adev, const struct auxil "Unexpected PECI revision %#x, some features may be unavailable\n", peci_dev->info.peci_revision); - ret = devm_delayed_work_autocancel(priv->dev, &priv->detect_work, - create_dimm_temp_info_delayed); - if (ret) - return ret; + auxiliary_set_drvdata(adev, priv); + INIT_DELAYED_WORK(&priv->detect_work, create_dimm_temp_info_delayed); ret = create_dimm_temp_info(priv); if (ret && ret != -EAGAIN) { @@ -412,6 +411,13 @@ static int peci_dimmtemp_probe(struct auxiliary_device *adev, const struct auxil return 0; } +static void peci_dimmtemp_remove(struct auxiliary_device *adev) +{ + struct peci_dimmtemp *priv = auxiliary_get_drvdata(adev); + + cancel_delayed_work_sync(&priv->detect_work); +} + static int read_thresholds_hsx(struct peci_dimmtemp *priv, int dimm_order, int chan_rank, u32 *data) { @@ -665,6 +671,7 @@ MODULE_DEVICE_TABLE(auxiliary, peci_dimmtemp_ids); static struct auxiliary_driver peci_dimmtemp_driver = { .probe = peci_dimmtemp_probe, + .remove = peci_dimmtemp_remove, .id_table = peci_dimmtemp_ids, }; diff --git a/drivers/hwmon/pmbus/pmbus_core.c b/drivers/hwmon/pmbus/pmbus_core.c index 5f69c1420b4e..b033bd3b1527 100644 --- a/drivers/hwmon/pmbus/pmbus_core.c +++ b/drivers/hwmon/pmbus/pmbus_core.c @@ -1507,11 +1507,14 @@ static int pmbus_thermal_add_sensor(struct pmbus_data *pmbus_data, tzd = devm_thermal_of_zone_register(dev, index, tdata, &pmbus_thermal_ops); /* - * If CONFIG_THERMAL_OF is disabled, this returns -ENODEV, - * so ignore that error but forward any other error. + * If CONFIG_THERMAL_OF is disabled, devm_thermal_of_zone_register() + * returns -ENOTSUPP. If no thermal zone is configured for the sensor, + * it returns -ENODEV. Ignore those errors but forward any other error. */ - if (IS_ERR(tzd) && (PTR_ERR(tzd) != -ENODEV)) + if (IS_ERR(tzd) && PTR_ERR(tzd) != -ENODEV && PTR_ERR(tzd) != -ENOTSUPP) { + devm_kfree(dev, tdata); return PTR_ERR(tzd); + } return 0; } @@ -1526,10 +1529,11 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data, { struct sensor_device_attribute *a; struct pmbus_sensor *sensor; + int err; sensor = devm_kzalloc(data->dev, sizeof(*sensor), GFP_KERNEL); if (!sensor) - return NULL; + return ERR_PTR(-ENOMEM); a = &sensor->attribute; if (type) @@ -1554,14 +1558,17 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data, pmbus_set_sensor, -1); if (pmbus_add_attribute(data, &a->dev_attr.attr)) - return NULL; + return ERR_PTR(-ENOMEM); sensor->next = data->sensors; data->sensors = sensor; /* temperature sensors with _input values are registered with thermal */ - if (class == PSC_TEMPERATURE && strcmp(type, "input") == 0) - pmbus_thermal_add_sensor(data, sensor, seq); + if (class == PSC_TEMPERATURE && strcmp(type, "input") == 0) { + err = pmbus_thermal_add_sensor(data, sensor, seq); + if (err) + return ERR_PTR(err); + } return sensor; } @@ -1662,8 +1669,8 @@ static int pmbus_add_limit_attrs(struct i2c_client *client, page, 0xff, l->reg, attr->class, attr->update || l->update, l->readonly, l->writeonly, true); - if (!curr) - return -ENOMEM; + if (IS_ERR(curr)) + return PTR_ERR(curr); if (l->sbit && (info->func[page] & attr->sfunc)) { ret = pmbus_add_boolean(data, name, l->alarm, index, @@ -1702,8 +1709,8 @@ static int pmbus_add_sensor_attrs_one(struct i2c_client *client, } base = pmbus_add_sensor(data, name, "input", index, page, phase, attr->reg, attr->class, true, true, false, true); - if (!base) - return -ENOMEM; + if (IS_ERR(base)) + return PTR_ERR(base); /* No limit and alarm attributes for phase specific sensors */ if (attr->sfunc && phase == 0xff) { ret = pmbus_add_limit_attrs(client, data, info, name, @@ -2376,8 +2383,8 @@ static int pmbus_add_fan_ctrl(struct i2c_client *client, 0xff, PMBUS_VIRT_FAN_TARGET_1 + id, PSC_FAN, false, false, false, true); - if (!sensor) - return -ENOMEM; + if (IS_ERR(sensor)) + return PTR_ERR(sensor); if (!((data->info->func[page] & PMBUS_HAVE_PWM12) || (data->info->func[page] & PMBUS_HAVE_PWM34))) @@ -2387,15 +2394,15 @@ static int pmbus_add_fan_ctrl(struct i2c_client *client, 0xff, PMBUS_VIRT_PWM_1 + id, PSC_PWM, false, false, false, true); - if (!sensor) - return -ENOMEM; + if (IS_ERR(sensor)) + return PTR_ERR(sensor); sensor = pmbus_add_sensor(data, "pwm", "enable", index, page, 0xff, PMBUS_VIRT_PWM_ENABLE_1 + id, PSC_PWM, true, false, false, false); - if (!sensor) - return -ENOMEM; + if (IS_ERR(sensor)) + return PTR_ERR(sensor); return 0; } @@ -2412,6 +2419,7 @@ static int pmbus_add_fan_attributes(struct i2c_client *client, int f; for (f = 0; f < ARRAY_SIZE(pmbus_fan_registers); f++) { + struct pmbus_sensor *sensor; int regval; if (!(info->func[page] & pmbus_fan_flags[f])) @@ -2432,10 +2440,11 @@ static int pmbus_add_fan_attributes(struct i2c_client *client, (!(regval & (PB_FAN_1_INSTALLED >> ((f & 1) * 4))))) continue; - if (pmbus_add_sensor(data, "fan", "input", index, - page, 0xff, pmbus_fan_registers[f], - PSC_FAN, true, true, false, true) == NULL) - return -ENOMEM; + sensor = pmbus_add_sensor(data, "fan", "input", index, + page, 0xff, pmbus_fan_registers[f], + PSC_FAN, true, true, false, true); + if (IS_ERR(sensor)) + return PTR_ERR(sensor); /* Fan control */ if (pmbus_check_word_register(client, page, diff --git a/drivers/hwmon/pmbus/q54sj108a2.c b/drivers/hwmon/pmbus/q54sj108a2.c index ecac405d7536..b018a611d9b6 100644 --- a/drivers/hwmon/pmbus/q54sj108a2.c +++ b/drivers/hwmon/pmbus/q54sj108a2.c @@ -351,6 +351,8 @@ static int q54sj108a2_probe(struct i2c_client *client) psu->client = client; debugfs = pmbus_get_debugfs_dir(client); + if (!debugfs) + return 0; q54sj108a2_dir = debugfs_create_dir(client->name, debugfs); diff --git a/drivers/hwmon/sht15.c b/drivers/hwmon/sht15.c index 7bb2105d1ae2..48d93c61345c 100644 --- a/drivers/hwmon/sht15.c +++ b/drivers/hwmon/sht15.c @@ -31,6 +31,7 @@ #include #include #include +#include /* Commands */ #define SHT15_MEASURE_TEMP 0x03 @@ -976,25 +977,22 @@ static int sht15_probe(struct platform_device *pdev) goto err_release_reg; } - ret = devm_request_irq(&pdev->dev, gpiod_to_irq(data->data), - sht15_interrupt_fired, - IRQF_TRIGGER_FALLING, - "sht15 data", - data); + ret = request_irq(gpiod_to_irq(data->data), sht15_interrupt_fired, + IRQF_TRIGGER_FALLING, "sht15 data", data); if (ret) goto err_release_reg; disable_irq_nosync(gpiod_to_irq(data->data)); ret = sht15_connection_reset(data); if (ret) - goto err_release_reg; + goto err_free_irq; ret = sht15_soft_reset(data); if (ret) - goto err_release_reg; + goto err_free_irq; ret = sysfs_create_group(&pdev->dev.kobj, &sht15_attr_group); if (ret) { dev_err(&pdev->dev, "sysfs create failed\n"); - goto err_release_reg; + goto err_free_irq; } data->hwmon_dev = hwmon_device_register(data->dev); @@ -1007,9 +1005,13 @@ static int sht15_probe(struct platform_device *pdev) err_release_sysfs_group: sysfs_remove_group(&pdev->dev.kobj, &sht15_attr_group); +err_free_irq: + disable_work_sync(&data->read_work); + free_irq(gpiod_to_irq(data->data), data); err_release_reg: if (!IS_ERR(data->reg)) { regulator_unregister_notifier(data->reg, &data->nb); + cancel_work_sync(&data->update_supply_work); regulator_disable(data->reg); } return ret; @@ -1023,12 +1025,16 @@ static void sht15_remove(struct platform_device *pdev) hwmon_device_unregister(data->hwmon_dev); sysfs_remove_group(&pdev->dev.kobj, &sht15_attr_group); + disable_work_sync(&data->read_work); + free_irq(gpiod_to_irq(data->data), data); + ret = sht15_soft_reset(data); if (ret) dev_err(&pdev->dev, "Failed to reset device (%pe)\n", ERR_PTR(ret)); if (!IS_ERR(data->reg)) { regulator_unregister_notifier(data->reg, &data->nb); + cancel_work_sync(&data->update_supply_work); regulator_disable(data->reg); } } diff --git a/drivers/hwmon/sht4x.c b/drivers/hwmon/sht4x.c index 0bb9227674a8..f38c106b7090 100644 --- a/drivers/hwmon/sht4x.c +++ b/drivers/hwmon/sht4x.c @@ -99,7 +99,9 @@ static int sht4x_read_values(struct sht4x_data *data) msleep(jiffies_to_msecs(data->heating_complete - curr_jiffies)); if (data->data_pending && - time_before64(get_jiffies_64(), data->heating_complete + data->update_interval)) { + time_before64(get_jiffies_64(), + data->heating_complete + + msecs_to_jiffies(data->update_interval))) { data->data_pending = false; } else { next_update = data->last_updated + diff --git a/drivers/hwmon/tmp108.c b/drivers/hwmon/tmp108.c index 91d20a1ce112..862b944d3e60 100644 --- a/drivers/hwmon/tmp108.c +++ b/drivers/hwmon/tmp108.c @@ -13,7 +13,7 @@ #include #include #include -#include +#include #include #include #include @@ -78,7 +78,7 @@ struct tmp108 { struct regmap *regmap; u16 orig_config; - u64 ready_time; + ktime_t ready_time; const struct tmp108_params *params; }; @@ -121,6 +121,7 @@ static int tmp108_read(struct device *dev, enum hwmon_sensor_types type, struct tmp108 *tmp108 = dev_get_drvdata(dev); unsigned int regval; int err, hyst; + s64 delay; if (type == hwmon_chip) { if (attr == hwmon_chip_update_interval) { @@ -137,12 +138,10 @@ static int tmp108_read(struct device *dev, enum hwmon_sensor_types type, switch (attr) { case hwmon_temp_input: - /* Is it too early to return a conversion ? */ - if (time_before64(get_jiffies_64(), tmp108->ready_time)) { - dev_dbg(dev, "%s: Conversion not ready yet..\n", - __func__); - return -EAGAIN; - } + /* Wait if it is too early to return a conversion */ + delay = ktime_us_delta(tmp108->ready_time, ktime_get()); + if (delay > 0) + usleep_range(delay, delay + 1000); err = regmap_read(tmp108->regmap, TMP108_REG_TEMP, ®val); if (err < 0) return err; @@ -477,11 +476,8 @@ static int tmp108_common_probe(struct device *dev, struct regmap *regmap, char * return err; } - tmp108->ready_time = get_jiffies_64(); - if ((tmp108->orig_config & TMP108_CONF_MODE_MASK) == - TMP108_MODE_SHUTDOWN) - tmp108->ready_time += - msecs_to_jiffies(TMP108_CONVERSION_TIME_MS); + tmp108->ready_time = ktime_add_ms(ktime_get(), + TMP108_CONVERSION_TIME_MS); err = devm_add_action_or_reset(dev, tmp108_restore_config, tmp108); if (err) { @@ -528,8 +524,8 @@ static int tmp108_resume(struct device *dev) err = regmap_update_bits(tmp108->regmap, TMP108_REG_CONF, TMP108_CONF_MODE_MASK, TMP108_MODE_CONTINUOUS); - tmp108->ready_time = get_jiffies_64() + - msecs_to_jiffies(TMP108_CONVERSION_TIME_MS); + tmp108->ready_time = ktime_add_ms(ktime_get(), + TMP108_CONVERSION_TIME_MS); return err; } diff --git a/drivers/idle/intel_idle.c b/drivers/idle/intel_idle.c index 651408df9c24..ec951c148d4a 100644 --- a/drivers/idle/intel_idle.c +++ b/drivers/idle/intel_idle.c @@ -1905,8 +1905,13 @@ static void __init intel_idle_init_cstates_acpi_lpi(struct cpuidle_driver *drv) state = &drv->states[drv->state_count++]; - scnprintf(state->name, CPUIDLE_NAME_LEN, "C%d_LPI", index + 1); - strscpy(state->desc, lpi_state->desc, CPUIDLE_DESC_LEN); + if (strlen(lpi_state->desc) > 0) + strscpy(state->name, lpi_state->desc); + else + scnprintf(state->name, CPUIDLE_NAME_LEN, "C%d_LPI", index + 1); + + scnprintf(state->desc, CPUIDLE_DESC_LEN, "ACPI LPI MWAIT 0x%02llx", + lpi_state->address); state->exit_latency = lpi_state->wake_latency; state->target_residency = lpi_state->min_residency; state->flags = MWAIT2flg(lpi_state->address); @@ -1956,7 +1961,7 @@ static void __init intel_idle_init_cstates_acpi_cst(struct cpuidle_driver *drv) state = &drv->states[drv->state_count++]; snprintf(state->name, CPUIDLE_NAME_LEN, "C%d_ACPI", cstate); - strscpy(state->desc, cx->desc, CPUIDLE_DESC_LEN); + strscpy(state->desc, cx->desc); state->exit_latency = cx->latency; /* * For C1-type C-states use the same number for both the exit diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 0506617b27c2..cfc061a4247a 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -1720,6 +1720,21 @@ static int __init init_iommu_from_acpi(struct amd_iommu *iommu, return 0; } +/* + * Apply workaround for Zen1 to Zen5(BRH) generation of processors. + * References: CVE-2024-21963 + */ +static bool __init amd_iommu_force_max_dev_table(void) +{ + if (boot_cpu_data.x86 >= 0x17 && boot_cpu_data.x86 <= 0x1a) { + if (cpu_feature_enabled(X86_FEATURE_ZEN6)) + return false; + else + return true; + } + return false; +} + /* Allocate PCI segment data structure */ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id, struct acpi_table_header *ivrs_base) @@ -1742,9 +1757,15 @@ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id, pci_seg->last_bdf = last_bdf; DUMP_printk("PCI segment : 0x%0x, last bdf : 0x%04x\n", id, last_bdf); - pci_seg->dev_table_size = - max(roundup_pow_of_two((last_bdf + 1) * DEV_TABLE_ENTRY_SIZE), - SZ_4K); + if (amd_iommu_force_max_dev_table()) { + pr_info("Force device table to maximum size\n"); + pci_seg->dev_table_size = SZ_2M; + } else { + pci_seg->dev_table_size = + max(roundup_pow_of_two((last_bdf + 1) * + DEV_TABLE_ENTRY_SIZE), + SZ_4K); + } pci_seg->id = id; init_llist_head(&pci_seg->dev_data_list); @@ -2708,6 +2729,15 @@ static int __init init_unity_map_range(struct ivmd_header *m, return 0; } +static bool ivrs_block_type_ivmd(u8 type) +{ + if (type == ACPI_IVMD_TYPE || + type == ACPI_IVMD_TYPE_ALL || + type == ACPI_IVMD_TYPE_RANGE) + return true; + return false; +} + /* iterates over all memory definitions we find in the ACPI table */ static int __init init_memory_definitions(struct acpi_table_header *table) { @@ -2719,10 +2749,11 @@ static int __init init_memory_definitions(struct acpi_table_header *table) while (p < end) { m = (struct ivmd_header *)p; + p += m->length; + if (!ivrs_block_type_ivmd(m->type)) + continue; if (m->flags & (IVMD_FLAG_UNITY_MAP | IVMD_FLAG_EXCL_RANGE)) init_unity_map_range(m, table); - - p += m->length; } return 0; diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 893cd2be11a2..b19496f8c1b0 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -489,6 +489,10 @@ static void setup_aliases(struct amd_iommu *iommu, struct device *dev) clone_aliases(iommu, dev); } +/* + * We keep dev_data around for unplugged devices and reuse it when the + * device is re-plugged - not doing so would introduce a ton of races. + */ static struct iommu_dev_data *find_dev_data(struct amd_iommu *iommu, u16 devid) { struct iommu_dev_data *dev_data; @@ -2548,18 +2552,6 @@ static struct iommu_device *amd_iommu_probe_device(struct device *dev) return iommu_dev; } -static void amd_iommu_release_device(struct device *dev) -{ - struct iommu_dev_data *dev_data = dev_iommu_priv_get(dev); - - WARN_ON(dev_data->domain); - - /* - * We keep dev_data around for unplugged devices and reuse it when the - * device is re-plugged - not doing so would introduce a ton of races. - */ -} - static struct iommu_group *amd_iommu_device_group(struct device *dev) { if (dev_is_pci(dev)) @@ -3205,7 +3197,6 @@ const struct iommu_ops amd_iommu_ops = { .domain_alloc_paging_flags = amd_iommu_domain_alloc_paging_flags, .domain_alloc_sva = amd_iommu_domain_alloc_sva, .probe_device = amd_iommu_probe_device, - .release_device = amd_iommu_release_device, .device_group = amd_iommu_device_group, .get_resv_regions = amd_iommu_get_resv_regions, .is_attach_deferred = amd_iommu_is_attach_deferred, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 0a429c64fbf3..3f50298a1c14 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -215,6 +215,13 @@ bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) if (system_supports_haft()) feat_mask |= ARM_SMMU_FEAT_HAFT; + /* + * The workaround for ARM_SMMU_OPT_FULL_CONT_RANGE_INV requires range + * invalidation support. + */ + if (smmu->options & ARM_SMMU_OPT_FULL_CONT_RANGE_INV) + feat_mask |= ARM_SMMU_FEAT_RANGE_INV; + if ((smmu->features & feat_mask) != feat_mask) return false; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 81baafaffc41..b510dd47f231 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2538,6 +2538,37 @@ static void arm_smmu_cmdq_batch_add_range(struct arm_smmu_device *smmu, } } +/* + * Generate a range invalidation for ARM_SMMU_OPT_FULL_CONT_RANGE_INV by + * ensuring the entire SVA requested range is covered with a single range + * invalidation command. The scale is adjusted so that the range invalidation + * may extend past the end of the requested range. This ensures that any CONT + * the MM is invalidating is covered by a single range invalidation. TTL and + * LEAF are always 0 because this is only used by SVA. + */ +static bool arm_smmu_cmdq_batch_add_range_inv(struct arm_smmu_device *smmu, + struct arm_smmu_cmdq_batch *cmds, + struct arm_smmu_cmd *cmd, + unsigned long iova, size_t size, + u8 tgsz_lg2) +{ + u64 cur_tg = iova >> tgsz_lg2; + u64 num_tg = ((iova + size - 1) >> tgsz_lg2) - cur_tg + 1; + unsigned int scale = fls64((num_tg - 1) / 32); + + if (scale > 31) + return false; + + cmd->data[0] |= + FIELD_PREP(CMDQ_TLBI_0_NUM, + DIV_ROUND_UP_ULL(num_tg, 1ULL << scale) - 1) | + FIELD_PREP(CMDQ_TLBI_0_SCALE, scale); + cmd->data[1] = FIELD_PREP(CMDQ_TLBI_1_TG, (tgsz_lg2 - 10) / 2) | + (cur_tg << tgsz_lg2); + arm_smmu_cmdq_batch_add_cmd_p(smmu, cmds, cmd); + return true; +} + static bool arm_smmu_inv_size_too_big(struct arm_smmu_device *smmu, size_t size, size_t granule) { @@ -2565,21 +2596,30 @@ static bool arm_smmu_inv_size_too_big(struct arm_smmu_device *smmu, size_t size, static void arm_smmu_inv_to_cmdq_batch(struct arm_smmu_inv *inv, struct arm_smmu_cmdq_batch *cmds, struct arm_smmu_cmd *cmd, - bool leaf, + bool single_range_inv, bool leaf, unsigned long iova, size_t size, unsigned int granule) { - if (arm_smmu_inv_size_too_big(inv->smmu, size, granule)) { - struct arm_smmu_cmd nsize_cmd = *cmd; + struct arm_smmu_cmd nsize_cmd; + + if (arm_smmu_inv_size_too_big(inv->smmu, size, granule)) + goto full_inv; - u64p_replace_bits(&nsize_cmd.data[0], inv->nsize_opcode, - CMDQ_0_OP); - arm_smmu_cmdq_batch_add_cmd_p(inv->smmu, cmds, &nsize_cmd); + if (single_range_inv && size > granule) { + if (!arm_smmu_cmdq_batch_add_range_inv(inv->smmu, cmds, cmd, + iova, size, inv->pgsize)) + goto full_inv; return; } - arm_smmu_cmdq_batch_add_range(inv->smmu, cmds, cmd, leaf, - iova, size, granule, inv->pgsize); + arm_smmu_cmdq_batch_add_range(inv->smmu, cmds, cmd, leaf, iova, size, + granule, inv->pgsize); + return; + +full_inv: + nsize_cmd = *cmd; + u64p_replace_bits(&nsize_cmd.data[0], inv->nsize_opcode, CMDQ_0_OP); + arm_smmu_cmdq_batch_add_cmd_p(inv->smmu, cmds, &nsize_cmd); } static inline bool arm_smmu_invs_end_batch(struct arm_smmu_inv *cur, @@ -2600,7 +2640,8 @@ static inline bool arm_smmu_invs_end_batch(struct arm_smmu_inv *cur, static void __arm_smmu_domain_inv_range(struct arm_smmu_invs *invs, unsigned long iova, size_t size, - unsigned int granule, bool leaf) + unsigned int granule, + bool single_range_inv, bool leaf) { struct arm_smmu_cmdq_batch cmds = {}; struct arm_smmu_inv *cur; @@ -2630,14 +2671,16 @@ static void __arm_smmu_domain_inv_range(struct arm_smmu_invs *invs, case INV_TYPE_S1_ASID: cmd = arm_smmu_make_cmd_tlbi(cur->size_opcode, cur->id, 0); - arm_smmu_inv_to_cmdq_batch(cur, &cmds, &cmd, leaf, - iova, size, granule); + arm_smmu_inv_to_cmdq_batch(cur, &cmds, &cmd, + single_range_inv, leaf, iova, + size, granule); break; case INV_TYPE_S2_VMID: cmd = arm_smmu_make_cmd_tlbi(cur->size_opcode, 0, cur->id); - arm_smmu_inv_to_cmdq_batch(cur, &cmds, &cmd, leaf, - iova, size, granule); + arm_smmu_inv_to_cmdq_batch(cur, &cmds, &cmd, + single_range_inv, leaf, iova, + size, granule); break; case INV_TYPE_S2_VMID_S1_CLEAR: /* CMDQ_OP_TLBI_S12_VMALL already flushed S1 entries */ @@ -2684,6 +2727,9 @@ void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain, unsigned int granule, bool leaf) { struct arm_smmu_invs *invs; + bool single_range_inv = + smmu_domain->stage == ARM_SMMU_DOMAIN_SVA && + (smmu_domain->smmu->options & ARM_SMMU_OPT_FULL_CONT_RANGE_INV); /* * An invalidation request must follow some IOPTE change and then load @@ -2723,10 +2769,12 @@ void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain, unsigned long flags; read_lock_irqsave(&invs->rwlock, flags); - __arm_smmu_domain_inv_range(invs, iova, size, granule, leaf); + __arm_smmu_domain_inv_range(invs, iova, size, granule, + single_range_inv, leaf); read_unlock_irqrestore(&invs->rwlock, flags); } else { - __arm_smmu_domain_inv_range(invs, iova, size, granule, leaf); + __arm_smmu_domain_inv_range(invs, iova, size, granule, + single_range_inv, leaf); } rcu_read_unlock(); @@ -5009,11 +5057,20 @@ static void arm_smmu_device_iidr_probe(struct arm_smmu_device *smmu) /* Arm errata 2268618, 2812531 */ smmu->features &= ~ARM_SMMU_FEAT_NESTING; } + /* Arm errata 3777127 */ + smmu->options |= ARM_SMMU_OPT_FULL_CONT_RANGE_INV; break; case IIDR_PRODUCTID_ARM_MMU_L1: + /* Arm errata 3878312 */ + smmu->features &= ~ARM_SMMU_FEAT_BTM; + break; case IIDR_PRODUCTID_ARM_MMU_S3: - /* Arm errata 3878312/3995052 */ + /* Arm errata 3995052 */ smmu->features &= ~ARM_SMMU_FEAT_BTM; + /* Arm errata 3673557 */ + if (variant < 1 || (variant == 1 && revision < 1)) + smmu->options |= + ARM_SMMU_OPT_FULL_CONT_RANGE_INV; break; } break; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index dd2fee2f560e..094ce28c862f 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -934,6 +934,11 @@ struct arm_smmu_device { #define ARM_SMMU_OPT_MSIPOLL (1 << 2) #define ARM_SMMU_OPT_CMDQ_FORCE_SYNC (1 << 3) #define ARM_SMMU_OPT_TEGRA241_CMDQV (1 << 4) +/* + * Range invalidation is mandatory and one range invalidation must fully span an + * invalidated CONT + */ +#define ARM_SMMU_OPT_FULL_CONT_RANGE_INV (1 << 6) u32 options; struct arm_smmu_cmdq cmdq; diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c index 283ab372e8da..5ac736cf5bb5 100644 --- a/drivers/iommu/iommufd/io_pagetable.c +++ b/drivers/iommu/iommufd/io_pagetable.c @@ -589,7 +589,7 @@ iommu_read_and_clear_dirty(struct iommu_domain *domain, arg.iopt = iopt; arg.domain = domain; arg.dirty = &dirty; - iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty); + ret = iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty); if (!(flags & IOMMU_DIRTY_NO_CLEAR)) iommu_iotlb_sync(domain, &gather); diff --git a/drivers/iommu/iommufd/iova_bitmap.c b/drivers/iommu/iommufd/iova_bitmap.c index dac3e657d498..908c70f5362e 100644 --- a/drivers/iommu/iommufd/iova_bitmap.c +++ b/drivers/iommu/iommufd/iova_bitmap.c @@ -116,6 +116,9 @@ struct iova_bitmap { /* length of the IOVA range for the whole bitmap */ size_t length; + + /* first error from recording an IOVA range */ + int err; }; /* @@ -384,7 +387,9 @@ static int iova_bitmap_advance_to(struct iova_bitmap *bitmap, int iova_bitmap_for_each(struct iova_bitmap *bitmap, void *opaque, iova_bitmap_fn_t fn) { - return fn(bitmap, bitmap->iova, bitmap->length, opaque); + int ret = fn(bitmap, bitmap->iova, bitmap->length, opaque); + + return ret ? ret : bitmap->err; } EXPORT_SYMBOL_NS_GPL(iova_bitmap_for_each, "IOMMUFD"); @@ -404,6 +409,18 @@ void iova_bitmap_set(struct iova_bitmap *bitmap, struct iova_bitmap_map *mapped = &bitmap->mapped; unsigned long cur_bit, last_bit, last_page_idx; + /* + * If the range cannot be recorded then the system call returns failure. + * This is to prevent loosing dirties if userspace is expecting to step + * the bitmap through the IOVA space. + */ + if (!length || length > bitmap->length || iova < bitmap->iova || + iova - bitmap->iova > bitmap->length - length) { + if (!bitmap->err) + bitmap->err = -EPROTO; + return; + } + update_indexes: if (unlikely(!iova_bitmap_mapped_range(mapped, iova, length))) { /* diff --git a/drivers/iommu/virtio-iommu.c b/drivers/iommu/virtio-iommu.c index 587fc13197f1..fa72ae23b8af 100644 --- a/drivers/iommu/virtio-iommu.c +++ b/drivers/iommu/virtio-iommu.c @@ -1227,12 +1227,12 @@ static int viommu_probe(struct virtio_device *vdev) /* Populate the event queue with buffers */ ret = viommu_fill_evtq(viommu); if (ret) - goto err_free_vqs; + goto err_reset_vdev; ret = iommu_device_sysfs_add(&viommu->iommu, dev, NULL, "%s", virtio_bus_name(vdev)); if (ret) - goto err_free_vqs; + goto err_reset_vdev; vdev->priv = viommu; @@ -1244,6 +1244,8 @@ static int viommu_probe(struct virtio_device *vdev) return 0; +err_reset_vdev: + virtio_reset_device(vdev); err_free_vqs: vdev->config->del_vqs(vdev); diff --git a/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c b/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c index 62ed92ff1d30..9f5bd9c4ddf9 100644 --- a/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c +++ b/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c @@ -32,7 +32,7 @@ static int handle_proxy_response(struct ipu6_isys *isys, unsigned int req_id) struct ipu6_fw_isys_proxy_resp_info_abi *resp; int ret; - resp = ipu6_recv_get_token(isys->fwcom, IPU6_BASE_PROXY_RECV_QUEUES); + resp = ipu6_recv_get_token(isys->fwctx, IPU6_BASE_PROXY_RECV_QUEUES); if (!resp) return 1; @@ -42,7 +42,7 @@ static int handle_proxy_response(struct ipu6_isys *isys, unsigned int req_id) ret = req_id == resp->request_id ? 0 : -EIO; - ipu6_recv_put_token(isys->fwcom, IPU6_BASE_PROXY_RECV_QUEUES); + ipu6_recv_put_token(isys->fwctx, IPU6_BASE_PROXY_RECV_QUEUES); return ret; } @@ -52,7 +52,7 @@ int ipu6_fw_isys_send_proxy_token(struct ipu6_isys *isys, unsigned int index, unsigned int offset, u32 value) { - struct ipu6_fw_com_context *ctx = isys->fwcom; + struct ipu6_fw_com_context *ctx = isys->fwctx; struct device *dev = &isys->adev->auxdev.dev; struct ipu6_fw_proxy_send_queue_token *token; unsigned int timeout = 1000; @@ -96,7 +96,7 @@ int ipu6_fw_isys_complex_cmd(struct ipu6_isys *isys, dma_addr_t dma_mapped_buf, size_t size, u16 send_type) { - struct ipu6_fw_com_context *ctx = isys->fwcom; + struct ipu6_fw_com_context *ctx = isys->fwctx; struct device *dev = &isys->adev->auxdev.dev; struct ipu6_fw_send_queue_token *token; @@ -138,7 +138,7 @@ int ipu6_fw_isys_close(struct ipu6_isys *isys) struct device *dev = &isys->adev->auxdev.dev; int retry = IPU6_ISYS_CLOSE_RETRY; unsigned long flags; - void *fwcom; + void *fwctx; int ret; /* @@ -148,9 +148,9 @@ int ipu6_fw_isys_close(struct ipu6_isys *isys) * spinlock to wait the interrupt handler to be finished */ spin_lock_irqsave(&isys->power_lock, flags); - ret = ipu6_fw_com_close(isys->fwcom); - fwcom = isys->fwcom; - isys->fwcom = NULL; + ret = ipu6_fw_com_close(isys->fwctx); + fwctx = isys->fwctx; + isys->fwctx = NULL; spin_unlock_irqrestore(&isys->power_lock, flags); if (ret) dev_err(dev, "Device close failure: %d\n", ret); @@ -158,14 +158,14 @@ int ipu6_fw_isys_close(struct ipu6_isys *isys) /* release probably fails if the close failed. Let's try still */ do { usleep_range(400, 500); - ret = ipu6_fw_com_release(fwcom, 0); + ret = ipu6_fw_com_release(fwctx, 0); retry--; } while (ret && retry); if (ret) { dev_err(dev, "Device release time out %d\n", ret); spin_lock_irqsave(&isys->power_lock, flags); - isys->fwcom = fwcom; + isys->fwctx = fwctx; spin_unlock_irqrestore(&isys->power_lock, flags); } @@ -176,11 +176,11 @@ void ipu6_fw_isys_cleanup(struct ipu6_isys *isys) { int ret; - ret = ipu6_fw_com_release(isys->fwcom, 1); + ret = ipu6_fw_com_release(isys->fwctx, 1); if (ret < 0) dev_warn(&isys->adev->auxdev.dev, "Device busy, fw_com release failed."); - isys->fwcom = NULL; + isys->fwctx = NULL; } static void start_sp(struct ipu6_bus_device *adev) @@ -212,13 +212,12 @@ static int query_sp(struct ipu6_bus_device *adev) } static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys, - struct ipu6_fw_com_cfg *fwcom, + struct ipu6_fw_com_cfg *fwcom_cfg, unsigned int num_streams) { unsigned int max_send_queues, max_sram_blocks, max_devq_size; struct ipu6_fw_syscom_queue_config *input_queue_cfg; struct ipu6_fw_syscom_queue_config *output_queue_cfg; - struct device *dev = &isys->adev->auxdev.dev; int type_proxy = IPU6_FW_ISYS_QUEUE_TYPE_PROXY; int type_dev = IPU6_FW_ISYS_QUEUE_TYPE_DEV; int type_msg = IPU6_FW_ISYS_QUEUE_TYPE_MSG; @@ -228,7 +227,6 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys, struct ipu6_fw_isys_fw_config *isys_fw_cfg; u32 num_in_message_queues; unsigned int max_streams; - unsigned int size; unsigned int i; max_streams = isys->pdata->ipdata->max_streams; @@ -236,7 +234,7 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys, max_sram_blocks = isys->pdata->ipdata->max_sram_blocks; max_devq_size = isys->pdata->ipdata->max_devq_size; num_in_message_queues = clamp(num_streams, 1U, max_streams); - isys_fw_cfg = devm_kzalloc(dev, sizeof(*isys_fw_cfg), GFP_KERNEL); + isys_fw_cfg = kzalloc_obj(*isys_fw_cfg); if (!isys_fw_cfg) return -ENOMEM; @@ -248,24 +246,25 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys, isys_fw_cfg->num_recv_queues[type_dev] = 0; isys_fw_cfg->num_recv_queues[type_msg] = 1; - size = sizeof(*input_queue_cfg) * max_send_queues; - input_queue_cfg = devm_kzalloc(dev, size, GFP_KERNEL); + input_queue_cfg = kzalloc_objs(*input_queue_cfg, max_send_queues); if (!input_queue_cfg) - return -ENOMEM; + goto err_free_fw_cfg; - size = sizeof(*output_queue_cfg) * IPU6_N_MAX_RECV_QUEUES; - output_queue_cfg = devm_kzalloc(dev, size, GFP_KERNEL); + output_queue_cfg = kzalloc_objs(*output_queue_cfg, + IPU6_N_MAX_RECV_QUEUES); if (!output_queue_cfg) - return -ENOMEM; + goto err_free_input_queue_cfg; - fwcom->input = input_queue_cfg; - fwcom->output = output_queue_cfg; + fwcom_cfg->input = input_queue_cfg; + fwcom_cfg->output = output_queue_cfg; - fwcom->num_input_queues = isys_fw_cfg->num_send_queues[type_proxy] + + fwcom_cfg->num_input_queues = + isys_fw_cfg->num_send_queues[type_proxy] + isys_fw_cfg->num_send_queues[type_dev] + isys_fw_cfg->num_send_queues[type_msg]; - fwcom->num_output_queues = isys_fw_cfg->num_recv_queues[type_proxy] + + fwcom_cfg->num_output_queues = + isys_fw_cfg->num_recv_queues[type_proxy] + isys_fw_cfg->num_recv_queues[type_dev] + isys_fw_cfg->num_recv_queues[type_msg]; @@ -280,7 +279,7 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys, isys_fw_cfg->buffer_partition.num_gda_pages[i] = 0; } - /* FW assumes proxy interface at fwcom queue 0 */ + /* FW assumes proxy interface at fwcom_cfg queue 0 */ for (i = 0; i < isys_fw_cfg->num_send_queues[type_proxy]; i++) { input_queue_cfg[i].token_size = sizeof(struct ipu6_fw_proxy_send_queue_token); @@ -314,34 +313,51 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys, IPU6_ISYS_SIZE_RECV_QUEUE; } - fwcom->dmem_addr = isys->pdata->ipdata->hw_variant.dmem_offset; - fwcom->specific_addr = isys_fw_cfg; - fwcom->specific_size = sizeof(*isys_fw_cfg); + fwcom_cfg->dmem_addr = isys->pdata->ipdata->hw_variant.dmem_offset; + fwcom_cfg->specific_addr = isys_fw_cfg; + fwcom_cfg->specific_size = sizeof(*isys_fw_cfg); return 0; + +err_free_input_queue_cfg: + kfree(input_queue_cfg); +err_free_fw_cfg: + kfree(isys_fw_cfg); + + return -ENOMEM; +} + +static void ipu6_isys_fwcom_cfg_free(struct ipu6_fw_com_cfg *fwcom_cfg) +{ + kfree(fwcom_cfg->specific_addr); + kfree(fwcom_cfg->output); + kfree(fwcom_cfg->input); } int ipu6_fw_isys_init(struct ipu6_isys *isys, unsigned int num_streams) { struct device *dev = &isys->adev->auxdev.dev; int retry = IPU6_ISYS_OPEN_RETRY; - struct ipu6_fw_com_cfg fwcom = { + struct ipu6_fw_com_cfg fwcom_cfg = { .cell_start = start_sp, .cell_ready = query_sp, .buttress_boot_offset = SYSCOM_BUTTRESS_FW_PARAMS_ISYS_OFFSET, }; int ret; - ipu6_isys_fwcom_cfg_init(isys, &fwcom, num_streams); + ret = ipu6_isys_fwcom_cfg_init(isys, &fwcom_cfg, num_streams); + if (ret) + return ret; - isys->fwcom = ipu6_fw_com_prepare(&fwcom, isys->adev, + isys->fwctx = ipu6_fw_com_prepare(&fwcom_cfg, isys->adev, isys->pdata->base); - if (!isys->fwcom) { + ipu6_isys_fwcom_cfg_free(&fwcom_cfg); + if (!isys->fwctx) { dev_err(dev, "isys fw com prepare failed\n"); return -EIO; } - ret = ipu6_fw_com_open(isys->fwcom); + ret = ipu6_fw_com_open(isys->fwctx); if (ret) { dev_err(dev, "isys fw com open failed %d\n", ret); return ret; @@ -349,7 +365,7 @@ int ipu6_fw_isys_init(struct ipu6_isys *isys, unsigned int num_streams) do { usleep_range(400, 500); - if (ipu6_fw_com_ready(isys->fwcom)) + if (ipu6_fw_com_ready(isys->fwctx)) break; retry--; } while (retry > 0); diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c index 3ac48d2076da..89eb265737d7 100644 --- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c @@ -1114,7 +1114,7 @@ int ipu6_isys_fw_open(struct ipu6_isys *isys) */ ipu6_cleanup_fw_msg_bufs(isys); - if (isys->fwcom) { + if (isys->fwctx) { /* * Something went wrong in previous shutdown. As we are now * restarting isys we can safely delete old context. @@ -1147,7 +1147,7 @@ void ipu6_isys_fw_close(struct ipu6_isys *isys) isys->ref_count--; if (!isys->ref_count) { ipu6_fw_isys_close(isys); - if (isys->fwcom) { + if (isys->fwctx) { isys->need_reset = true; dev_warn(&isys->adev->auxdev.dev, "failed to close fw isys\n"); diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys.c b/drivers/media/pci/intel/ipu6/ipu6-isys.c index 24db2763de54..e13fbbae8c98 100644 --- a/drivers/media/pci/intel/ipu6/ipu6-isys.c +++ b/drivers/media/pci/intel/ipu6/ipu6-isys.c @@ -1212,10 +1212,10 @@ static int isys_isr_one(struct ipu6_bus_device *adev) u32 index; u64 ts; - if (!isys->fwcom) + if (!isys->fwctx) return 1; - resp = ipu6_fw_isys_get_resp(isys->fwcom, IPU6_BASE_MSG_RECV_QUEUES); + resp = ipu6_fw_isys_get_resp(isys->fwctx, IPU6_BASE_MSG_RECV_QUEUES); if (!resp) return 1; @@ -1324,7 +1324,7 @@ static int isys_isr_one(struct ipu6_bus_device *adev) ipu6_isys_put_stream(stream); leave: - ipu6_fw_isys_put_resp(isys->fwcom, IPU6_BASE_MSG_RECV_QUEUES); + ipu6_fw_isys_put_resp(isys->fwctx, IPU6_BASE_MSG_RECV_QUEUES); return 0; } diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys.h b/drivers/media/pci/intel/ipu6/ipu6-isys.h index 7fb8cb820912..0b139ab24431 100644 --- a/drivers/media/pci/intel/ipu6/ipu6-isys.h +++ b/drivers/media/pci/intel/ipu6/ipu6-isys.h @@ -114,8 +114,7 @@ struct sensor_async_sd { * @csi2_rx_ctrl_cached: cached shared value between all CSI2 receivers * @streams_lock: serialise access to streams * @streams: streams per firmware stream ID - * @fwcom: fw communication layer private pointer - * or optional external library private pointer + * @fwctx: fw communication layer context pointer * @phy_termcal_val: the termination calibration value, only used for DWC PHY * @need_reset: Isys requires d0i0->i3 transition * @ref_count: total number of callers fw open @@ -136,7 +135,7 @@ struct ipu6_isys { spinlock_t streams_lock; struct ipu6_isys_stream streams[IPU6_ISYS_MAX_STREAMS]; int streams_ref_count[IPU6_ISYS_MAX_STREAMS]; - void *fwcom; + void *fwctx; u32 phy_termcal_val; bool need_reset; bool icache_prefetch; diff --git a/drivers/media/platform/microchip/microchip-isc-base.c b/drivers/media/platform/microchip/microchip-isc-base.c index 21d108eb93c7..963d5fd53b58 100644 --- a/drivers/media/platform/microchip/microchip-isc-base.c +++ b/drivers/media/platform/microchip/microchip-isc-base.c @@ -8,6 +8,7 @@ * Author: Eugen Hristev * */ +#include #include #include #include @@ -62,17 +63,17 @@ static inline void isc_update_awb_ctrls(struct isc_device *isc) /* In here we set our actual hw pipeline config */ regmap_write(isc->regmap, ISC_WB_O_RGR, - ((ctrls->offset[ISC_HIS_CFG_MODE_R])) | - ((ctrls->offset[ISC_HIS_CFG_MODE_GR]) << 16)); + FIELD_PREP(ISC_WB_O_LO, ctrls->offset[ISC_HIS_CFG_MODE_R]) | + FIELD_PREP(ISC_WB_O_HI, ctrls->offset[ISC_HIS_CFG_MODE_GR])); regmap_write(isc->regmap, ISC_WB_O_BGB, - ((ctrls->offset[ISC_HIS_CFG_MODE_B])) | - ((ctrls->offset[ISC_HIS_CFG_MODE_GB]) << 16)); + FIELD_PREP(ISC_WB_O_LO, ctrls->offset[ISC_HIS_CFG_MODE_B]) | + FIELD_PREP(ISC_WB_O_HI, ctrls->offset[ISC_HIS_CFG_MODE_GB])); regmap_write(isc->regmap, ISC_WB_G_RGR, - ctrls->gain[ISC_HIS_CFG_MODE_R] | - (ctrls->gain[ISC_HIS_CFG_MODE_GR] << 16)); + FIELD_PREP(ISC_WB_G_LO, ctrls->gain[ISC_HIS_CFG_MODE_R]) | + FIELD_PREP(ISC_WB_G_HI, ctrls->gain[ISC_HIS_CFG_MODE_GR])); regmap_write(isc->regmap, ISC_WB_G_BGB, - ctrls->gain[ISC_HIS_CFG_MODE_B] | - (ctrls->gain[ISC_HIS_CFG_MODE_GB] << 16)); + FIELD_PREP(ISC_WB_G_LO, ctrls->gain[ISC_HIS_CFG_MODE_B]) | + FIELD_PREP(ISC_WB_G_HI, ctrls->gain[ISC_HIS_CFG_MODE_GB])); } static inline void isc_reset_awb_ctrls(struct isc_device *isc) @@ -291,7 +292,8 @@ static int isc_configure(struct isc_device *isc) struct isc_subdev_entity *subdev = isc->current_subdev; int ret; - pfe_cfg0 = isc->config.sd_format->pfe_cfg0_bps; + pfe_cfg0 = FIELD_PREP(ISC_PFE_CFG0_BPS_MASK, + isc->config.sd_format->pfe_cfg0_bps); pipeline = isc->config.bits_pipeline; dcfg = isc->config.dcfg_imode | isc->dcfg; diff --git a/drivers/media/platform/microchip/microchip-isc-regs.h b/drivers/media/platform/microchip/microchip-isc-regs.h index e77e1d9a1db8..fe145b142b82 100644 --- a/drivers/media/platform/microchip/microchip-isc-regs.h +++ b/drivers/media/platform/microchip/microchip-isc-regs.h @@ -31,11 +31,11 @@ #define ISC_PFE_CFG0_MODE_PROGRESSIVE (0x0 << 4) #define ISC_PFE_CFG0_MODE_MASK GENMASK(6, 4) -#define ISC_PFE_CFG0_BPS_EIGHT (0x4 << 28) -#define ISC_PFG_CFG0_BPS_NINE (0x3 << 28) -#define ISC_PFG_CFG0_BPS_TEN (0x2 << 28) -#define ISC_PFG_CFG0_BPS_ELEVEN (0x1 << 28) -#define ISC_PFG_CFG0_BPS_TWELVE (0x0 << 28) +#define ISC_PFE_CFG0_BPS_EIGHT 0x4 +#define ISC_PFE_CFG0_BPS_NINE 0x3 +#define ISC_PFE_CFG0_BPS_TEN 0x2 +#define ISC_PFE_CFG0_BPS_ELEVEN 0x1 +#define ISC_PFE_CFG0_BPS_TWELVE 0x0 #define ISC_PFE_CFG0_BPS_MASK GENMASK(30, 28) #define ISC_PFE_CFG0_COLEN BIT(12) @@ -149,6 +149,12 @@ /* ISC White Balance Gain for B, GB Register */ #define ISC_WB_G_BGB 0x0000006c +/* Each WB offset/gain register packs two 13-bit fields, low and high */ +#define ISC_WB_O_LO GENMASK(12, 0) /* R or B offset [12:0] */ +#define ISC_WB_O_HI GENMASK(28, 16) /* GR or GB offset [28:16] */ +#define ISC_WB_G_LO GENMASK(12, 0) /* R or B gain [12:0] */ +#define ISC_WB_G_HI GENMASK(28, 16) /* GR or GB gain [28:16] */ + /* ISC Color Filter Array Control Register */ #define ISC_CFA_CTRL 0x00000070 diff --git a/drivers/media/platform/microchip/microchip-isc.h b/drivers/media/platform/microchip/microchip-isc.h index f5e322c2e36b..d7bcd74efff9 100644 --- a/drivers/media/platform/microchip/microchip-isc.h +++ b/drivers/media/platform/microchip/microchip-isc.h @@ -62,7 +62,9 @@ struct isc_subdev_entity { * @mbus_code: V4L2 media bus format code. * @cfa_baycfg: If this format is RAW BAYER, indicate the type of bayer. this is either BGBG, RGRG, etc. - * @pfe_cfg0_bps: Number of hardware data lines connected to the ISC + * @pfe_cfg0_bps: ISC_PFE_CFG0 BPS field value (e.g. ISC_PFE_CFG0_BPS_EIGHT), + written into PFE_CFG0 with FIELD_PREP(ISC_PFE_CFG0_BPS_MASK) + at configure time. * @raw: If the format is raw bayer. */ diff --git a/drivers/media/platform/microchip/microchip-sama5d2-isc.c b/drivers/media/platform/microchip/microchip-sama5d2-isc.c index 5bd109e93a8f..5e41eee45dbd 100644 --- a/drivers/media/platform/microchip/microchip-sama5d2-isc.c +++ b/drivers/media/platform/microchip/microchip-sama5d2-isc.c @@ -147,49 +147,49 @@ static struct isc_format sama5d2_formats_list[] = { { .fourcc = V4L2_PIX_FMT_SBGGR10, .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, - .cfa_baycfg = ISC_BAY_CFG_RGRG, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, + .cfa_baycfg = ISC_BAY_CFG_BGBG, }, { .fourcc = V4L2_PIX_FMT_SGBRG10, .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, .cfa_baycfg = ISC_BAY_CFG_GBGB, }, { .fourcc = V4L2_PIX_FMT_SGRBG10, .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, .cfa_baycfg = ISC_BAY_CFG_GRGR, }, { .fourcc = V4L2_PIX_FMT_SRGGB10, .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, .cfa_baycfg = ISC_BAY_CFG_RGRG, }, { .fourcc = V4L2_PIX_FMT_SBGGR12, .mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_BGBG, }, { .fourcc = V4L2_PIX_FMT_SGBRG12, .mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_GBGB, }, { .fourcc = V4L2_PIX_FMT_SGRBG12, .mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_GRGR, }, { .fourcc = V4L2_PIX_FMT_SRGGB12, .mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_RGRG, }, { @@ -210,7 +210,7 @@ static struct isc_format sama5d2_formats_list[] = { { .fourcc = V4L2_PIX_FMT_Y10, .mbus_code = MEDIA_BUS_FMT_Y10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, }, }; diff --git a/drivers/media/platform/microchip/microchip-sama7g5-isc.c b/drivers/media/platform/microchip/microchip-sama7g5-isc.c index d978ce448a4d..9e78ea07b013 100644 --- a/drivers/media/platform/microchip/microchip-sama7g5-isc.c +++ b/drivers/media/platform/microchip/microchip-sama7g5-isc.c @@ -156,49 +156,49 @@ static struct isc_format sama7g5_formats_list[] = { { .fourcc = V4L2_PIX_FMT_SBGGR10, .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, - .cfa_baycfg = ISC_BAY_CFG_RGRG, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, + .cfa_baycfg = ISC_BAY_CFG_BGBG, }, { .fourcc = V4L2_PIX_FMT_SGBRG10, .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, .cfa_baycfg = ISC_BAY_CFG_GBGB, }, { .fourcc = V4L2_PIX_FMT_SGRBG10, .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, .cfa_baycfg = ISC_BAY_CFG_GRGR, }, { .fourcc = V4L2_PIX_FMT_SRGGB10, .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, .cfa_baycfg = ISC_BAY_CFG_RGRG, }, { .fourcc = V4L2_PIX_FMT_SBGGR12, .mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_BGBG, }, { .fourcc = V4L2_PIX_FMT_SGBRG12, .mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_GBGB, }, { .fourcc = V4L2_PIX_FMT_SGRBG12, .mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_GRGR, }, { .fourcc = V4L2_PIX_FMT_SRGGB12, .mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_RGRG, }, { @@ -224,7 +224,7 @@ static struct isc_format sama7g5_formats_list[] = { { .fourcc = V4L2_PIX_FMT_Y10, .mbus_code = MEDIA_BUS_FMT_Y10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, }, }; diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c index e289cc71ba98..73b97ec578ec 100644 --- a/drivers/media/usb/uvc/uvc_driver.c +++ b/drivers/media/usb/uvc/uvc_driver.c @@ -2851,6 +2851,26 @@ static const struct usb_device_id uvc_ids[] = { .bInterfaceSubClass = 1, .bInterfaceProtocol = 0, .driver_info = (kernel_ulong_t)&uvc_quirk_fix_bandwidth }, + /* Avermedia GC515 */ + { .match_flags = USB_DEVICE_ID_MATCH_DEVICE + | USB_DEVICE_ID_MATCH_INT_INFO, + .idVendor = 0x07ca, + .idProduct = 0x1515, + .bInterfaceClass = USB_CLASS_VIDEO, + .bInterfaceSubClass = 1, + .bInterfaceProtocol = UVC_PC_PROTOCOL_15, + .driver_info = (kernel_ulong_t)&(const struct uvc_device_info){ + .uvc_version = 0x0110, + } }, + /* AVerMedia Live Gamer HD 2 capture card */ + { .match_flags = USB_DEVICE_ID_MATCH_DEVICE + | USB_DEVICE_ID_MATCH_INT_INFO, + .idVendor = 0x07ca, + .idProduct = 0x1570, + .bInterfaceClass = USB_CLASS_VIDEO, + .bInterfaceSubClass = 1, + .bInterfaceProtocol = 0, + .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_DISABLE_AUTOSUSPEND) }, /* ViMicro Vega */ { .match_flags = USB_DEVICE_ID_MATCH_DEVICE | USB_DEVICE_ID_MATCH_INT_INFO, diff --git a/drivers/mtd/spi-nor/macronix.c b/drivers/mtd/spi-nor/macronix.c index e97f5cbd9aad..c854a1163a47 100644 --- a/drivers/mtd/spi-nor/macronix.c +++ b/drivers/mtd/spi-nor/macronix.c @@ -132,7 +132,9 @@ static const struct flash_info macronix_nor_parts[] = { }, { /* MX25L12805D */ .id = SNOR_ID(0xc2, 0x20, 0x18), + .size = SZ_16M, .flags = SPI_NOR_HAS_LOCK | SPI_NOR_4BIT_BP, + .no_sfdp_flags = SECT_4K, }, { /* MX25L25635E, MX25L25645G */ .id = SNOR_ID(0xc2, 0x20, 0x19), diff --git a/drivers/net/can/dev/netlink.c b/drivers/net/can/dev/netlink.c index 766d455950f5..b958c04d53db 100644 --- a/drivers/net/can/dev/netlink.c +++ b/drivers/net/can/dev/netlink.c @@ -812,7 +812,7 @@ static size_t can_get_size(const struct net_device *dev) size += nla_total_size(sizeof(u32)); /* IFLA_CAN_STATE */ size += nla_total_size(sizeof(struct can_ctrlmode)); /* IFLA_CAN_CTRLMODE */ size += nla_total_size(sizeof(u32)); /* IFLA_CAN_RESTART_MS */ - if (priv->do_get_berr_counter) /* IFLA_CAN_BERR_COUNTER */ + if (priv->do_get_berr_counter && netif_running(dev)) /* IFLA_CAN_BERR_COUNTER */ size += nla_total_size(sizeof(struct can_berr_counter)); if (priv->termination_const) { size += nla_total_size(sizeof(priv->termination)); /* IFLA_CAN_TERMINATION */ @@ -1006,7 +1006,7 @@ static int can_fill_info(struct sk_buff *skb, const struct net_device *dev) nla_put(skb, IFLA_CAN_CTRLMODE, sizeof(cm), &cm) || nla_put_u32(skb, IFLA_CAN_RESTART_MS, priv->restart_ms) || - (priv->do_get_berr_counter && + (priv->do_get_berr_counter && netif_running(dev) && !priv->do_get_berr_counter(dev, &bec) && nla_put(skb, IFLA_CAN_BERR_COUNTER, sizeof(bec), &bec)) || diff --git a/drivers/net/can/dev/skb.c b/drivers/net/can/dev/skb.c index 14edec5afb57..9e274147ffe5 100644 --- a/drivers/net/can/dev/skb.c +++ b/drivers/net/can/dev/skb.c @@ -4,6 +4,7 @@ * Copyright (C) 2008-2009 Wolfgang Grandegger */ +#include #include #include #include @@ -54,6 +55,7 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev, if (idx >= priv->echo_skb_max) { netdev_err(dev, "%s: BUG! Trying to access can_priv::echo_skb out of bounds (%u/max %u)\n", __func__, idx, priv->echo_skb_max); + dev_kfree_skb_any(skb); return -EINVAL; } @@ -62,7 +64,7 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev, (skb->protocol != htons(ETH_P_CAN) && skb->protocol != htons(ETH_P_CANFD) && skb->protocol != htons(ETH_P_CANXL))) { - kfree_skb(skb); + dev_kfree_skb_any(skb); return 0; } @@ -90,7 +92,7 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev, } else { /* locking problem with netif_stop_queue() ?? */ netdev_err(dev, "%s: BUG! echo_skb %d is occupied!\n", __func__, idx); - kfree_skb(skb); + dev_kfree_skb_any(skb); return -EBUSY; } @@ -388,7 +390,7 @@ bool can_dropped_invalid_skb(struct net_device *dev, struct sk_buff *skb) break; case ETH_P_CANXL: - if (!can_is_canxl_skb(skb)) + if (!can_cap_enabled(dev, CAN_CAP_XL) || !can_is_canxl_skb(skb)) goto inval_skb; break; @@ -402,7 +404,7 @@ bool can_dropped_invalid_skb(struct net_device *dev, struct sk_buff *skb) return false; inval_skb: - kfree_skb(skb); + dev_kfree_skb_any(skb); dev->stats.tx_dropped++; return true; } diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c index 0d46f4ce5dca..39f0740421c3 100644 --- a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c +++ b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c @@ -755,7 +755,7 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) u16 rx_msg_type, rx_msg_size; size_t rx_msg_min_size; - if (msg_end - msg_ptr < sizeof(*rx_msg)) { + if (msg_end - msg_ptr < sizeof(rx_msg->size)) { err = -EBADMSG; break; } @@ -766,6 +766,11 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) break; } + if (msg_end - msg_ptr < sizeof(*rx_msg)) { + err = -EBADMSG; + break; + } + rx_msg_size = le16_to_cpu(rx_msg->size); rx_msg_type = le16_to_cpu(rx_msg->type); diff --git a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c index e58f1d2c26bd..eddd8406867f 100644 --- a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c +++ b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c @@ -1934,6 +1934,39 @@ static int atl1c_clean_rx(struct napi_struct *napi, int budget) break; } atl1c_clean_rfd(rfd_ring, rrs, rfd_num); + + /* A link partner whose l2mtu is larger than ours can hand us a + * frame that does not fit the buffer the hardware was given: + * the descriptor reports the untruncated length while at most + * buffer_info->length bytes were written. skb_put() of such a + * length would pull in whatever the tail of the buffer holds + * and, past the end of the skb, trips skb_over_panic and takes + * the host down. Bound it by what the device actually wrote + * rather than by skb_tailroom(): the skb carries 64 bytes of + * slack for the DMA erratum, so the two are not the same. + * + * Frames shorter than an Ethernet header plus its FCS are + * rejected by the same test; before this, a length below + * ETH_FCS_LEN underflowed in skb_put(). + */ + if (unlikely(length < ETH_HLEN + ETH_FCS_LEN || + length > buffer_info->length)) { + dev_core_stats_rx_dropped_inc(netdev); + if (netif_msg_rx_err(adapter)) + dev_warn_ratelimited(&pdev->dev, + "dropped rx frame: %u bytes, buffer %u\n", + length, buffer_info->length); + kfree_skb_reason(skb, SKB_DROP_REASON_PKT_TOO_BIG); + /* atl1c_clean_rfd() has already recycled the RFD, so + * count the drop as work: the refill at the end of the + * poll only runs when work_done is non-zero, and + * sustained oversized traffic would otherwise drain the + * ring and stop RX for good. + */ + work_done++; + continue; + } + skb_put(skb, length - ETH_FCS_LEN); skb->protocol = eth_type_trans(skb, netdev); atl1c_rx_checksum(adapter, skb, rrs); diff --git a/drivers/net/ethernet/jme.c b/drivers/net/ethernet/jme.c index 25ec5f757c68..acb4d41f56e1 100644 --- a/drivers/net/ethernet/jme.c +++ b/drivers/net/ethernet/jme.c @@ -1502,7 +1502,8 @@ jme_intr_msi(struct jme_adapter *jme, u32 intrstat) * all other events are ignored */ jwrite32(jme, JME_IEVE, intrstat); - schedule_work(&jme->linkch_task); + if (!test_bit(JME_FLAG_WOL_QUIESCE, &jme->flags)) + schedule_work(&jme->linkch_task); goto out_reenable; } @@ -1546,7 +1547,8 @@ jme_intr_msi(struct jme_adapter *jme, u32 intrstat) /* * Re-enable interrupt */ - jwrite32f(jme, JME_IENS, INTR_ENABLE); + if (!test_bit(JME_FLAG_WOL_QUIESCE, &jme->flags)) + jwrite32f(jme, JME_IENS, INTR_ENABLE); } static irqreturn_t @@ -1892,6 +1894,7 @@ jme_powersave_phy(struct jme_adapter *jme) jme_set_100m_half(jme); if (jme->reg_pmcs & (PMCS_LFEN | PMCS_LREN)) jme_wait_link(jme); + jme_mac_rxclk_on(jme); jme_clear_pm_enable_wol(jme); } else { jme_phy_off(jme); @@ -3158,6 +3161,21 @@ jme_shutdown(struct pci_dev *pdev) struct net_device *netdev = pci_get_drvdata(pdev); struct jme_adapter *jme = netdev_priv(netdev); + if (netif_running(netdev)) { + set_bit(JME_FLAG_WOL_QUIESCE, &jme->flags); + netif_device_detach(netdev); + netif_stop_queue(netdev); + jme_stop_irq(jme); + synchronize_irq(pdev->irq); + jme_stop_irq(jme); + cancel_work_sync(&jme->linkch_task); + tasklet_disable(&jme->pcc_task); + napi_disable(&jme->napi); + tasklet_disable(&jme->txclean_task); + tasklet_disable(&jme->rxclean_task); + tasklet_disable(&jme->rxempty_task); + } + jme_powersave_phy(jme); pci_pme_active(pdev, true); } @@ -3172,11 +3190,16 @@ jme_suspend(struct device *dev) if (!netif_running(netdev)) return 0; - atomic_dec(&jme->link_changing); - + set_bit(JME_FLAG_WOL_QUIESCE, &jme->flags); netif_device_detach(netdev); netif_stop_queue(netdev); jme_stop_irq(jme); + synchronize_irq(jme->pdev->irq); + jme_stop_irq(jme); + cancel_work_sync(&jme->linkch_task); + tasklet_disable(&jme->pcc_task); + napi_disable(&jme->napi); + atomic_dec(&jme->link_changing); tasklet_disable(&jme->txclean_task); tasklet_disable(&jme->rxclean_task); @@ -3215,6 +3238,7 @@ jme_resume(struct device *dev) return 0; jme_clear_pm_disable_wol(jme); + jme_mac_rxclk_off(jme); jme_phy_on(jme); if (test_bit(JME_FLAG_SSET, &jme->flags)) jme_set_link_ksettings(netdev, &jme->old_cmd); @@ -3226,8 +3250,11 @@ jme_resume(struct device *dev) atomic_inc(&jme->link_changing); - jme_reset_link(jme); + napi_enable(&jme->napi); + tasklet_enable(&jme->pcc_task); + clear_bit(JME_FLAG_WOL_QUIESCE, &jme->flags); + jme_reset_link(jme); jme_start_irq(jme); return 0; diff --git a/drivers/net/ethernet/jme.h b/drivers/net/ethernet/jme.h index 860494ff3714..ec5dfddf7322 100644 --- a/drivers/net/ethernet/jme.h +++ b/drivers/net/ethernet/jme.h @@ -450,6 +450,7 @@ enum jme_flags_bits { JME_FLAG_SSET = 2, JME_FLAG_POLL = 5, JME_FLAG_SHUTDOWN = 6, + JME_FLAG_WOL_QUIESCE = 7, }; #define TX_TIMEOUT (5 * HZ) diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.c b/drivers/net/ethernet/ti/am65-cpsw-nuss.c index 6330540979d7..f54cfcb7d153 100644 --- a/drivers/net/ethernet/ti/am65-cpsw-nuss.c +++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.c @@ -128,6 +128,7 @@ #define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_319 BIT(21) #define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_320 BIT(22) #define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO BIT(23) +#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_UNI_EN BIT(24) /* The PTP event messages - Sync, Delay_Req, Pdelay_Req, and Pdelay_Resp. */ #define AM65_CPSW_TS_EVENT_MSG_TYPE_BITS (BIT(0) | BIT(1) | BIT(2) | BIT(3)) @@ -1857,7 +1858,8 @@ static int am65_cpsw_nuss_hwtstamp_set(struct net_device *ndev, AM65_CPSW_PN_TS_CTL_LTYPE2_TS_132 | AM65_CPSW_PN_TS_CTL_LTYPE2_TS_319 | AM65_CPSW_PN_TS_CTL_LTYPE2_TS_320 | - AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO; + AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO | + AM65_CPSW_PN_TS_CTL_LTYPE2_TS_UNI_EN; ts_ctrl = AM65_CPSW_TS_EVENT_MSG_TYPE_BITS << AM65_CPSW_PN_TS_CTL_MSG_TYPE_EN_SHIFT; diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 7db389ae866d..c1859ffcdaf1 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -1372,9 +1372,10 @@ static int veth_set_channels(struct net_device *dev, * to identify the range we have to disable */ veth_disable_range_safe(dev, new_rx_count, old_rx_count); - netif_carrier_on(dev); - if (peer) + if (peer && (peer->flags & IFF_UP)) { + netif_carrier_on(dev); netif_carrier_on(peer); + } } /* update XDP supported features */ diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c index 3390e341d1e5..7ecd73a5a374 100644 --- a/drivers/net/vxlan/vxlan_core.c +++ b/drivers/net/vxlan/vxlan_core.c @@ -996,12 +996,6 @@ static int vxlan_fdb_update_existing(struct vxlan_dev *vxlan, return -EOPNOTSUPP; } - if (rcu_access_pointer(f->nh) && - !(state & (NUD_PERMANENT | NUD_NOARP))) { - NL_SET_ERR_MSG(extack, "Cannot make a nexthop fdb dynamic"); - return -EOPNOTSUPP; - } - /* Do not allow an externally learned entry to take over an entry added * by the user. */ @@ -1263,11 +1257,6 @@ static int vxlan_fdb_add(struct ndmsg *ndm, struct nlattr *tb[], if (err) return err; - if (nhid && !(ndm->ndm_state & (NUD_PERMANENT | NUD_NOARP))) { - NL_SET_ERR_MSG(extack, "A nexthop fdb cannot be dynamic"); - return -EINVAL; - } - if (vxlan->default_dst.remote_ip.sa.sa_family != ip.sa.sa_family) return -EAFNOSUPPORT; @@ -2887,6 +2876,9 @@ static void vxlan_cleanup(struct timer_list *t) if (f->flags & NTF_EXT_LEARNED) continue; + if (rcu_access_pointer(f->nh)) + continue; + timeout = READ_ONCE(f->updated) + vxlan->cfg.age_interval * HZ; if (time_before_eq(timeout, jiffies)) { spin_lock(&vxlan->hash_lock); diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c index 872c48806d09..62d072b6369e 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c @@ -6504,6 +6504,7 @@ brcmf_bss_connect_done(struct brcmf_cfg80211_info *cfg, struct brcmf_cfg80211_profile *profile = &ifp->vif->profile; struct brcmf_cfg80211_connect_info *conn_info = cfg_to_conn(cfg); struct cfg80211_connect_resp_params conn_params; + bool authorized; brcmf_dbg(TRACE, "Enter\n"); @@ -6528,7 +6529,13 @@ brcmf_bss_connect_done(struct brcmf_cfg80211_info *cfg, conn_params.req_ie_len = conn_info->req_ie_len; conn_params.resp_ie = conn_info->resp_ie; conn_params.resp_ie_len = conn_info->resp_ie_len; + authorized = completed && + ((profile->use_fwsup == BRCMF_PROFILE_FWSUP_SAE) || + (profile->use_fwsup == BRCMF_PROFILE_FWSUP_PSK)); cfg80211_connect_done(ndev, &conn_params, GFP_KERNEL); + if (authorized) + cfg80211_port_authorized(ndev, profile->bssid, NULL, 0, + GFP_KERNEL); brcmf_dbg(CONN, "Report connect result - connection %s\n", completed ? "succeeded" : "failed"); } diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c index 069ba7016654..ff47d4827722 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c @@ -1639,7 +1639,7 @@ int brcmf_proto_msgbuf_attach(struct brcmf_pub *drvr) if (!msgbuf->tx_pktids) goto fail; msgbuf->rx_pktids = brcmf_msgbuf_init_pktids(NR_RX_PKTIDS, - DMA_FROM_DEVICE); + DMA_BIDIRECTIONAL); if (!msgbuf->rx_pktids) goto fail; diff --git a/drivers/net/wireless/realtek/rtw88/mac80211.c b/drivers/net/wireless/realtek/rtw88/mac80211.c index 0b622d7663b6..2a9b09fa76e7 100644 --- a/drivers/net/wireless/realtek/rtw88/mac80211.c +++ b/drivers/net/wireless/realtek/rtw88/mac80211.c @@ -474,6 +474,7 @@ static int rtw_ops_start_ap(struct ieee80211_hw *hw, mutex_lock(&rtwdev->mutex); rtw_write32_set(rtwdev, REG_TCR, BIT_TCR_UPDATE_HGQMD); + rtw_write16(rtwdev, REG_ATIMWND, ATIMWND_AP); rtwdev->ap_active = true; rtw_store_op_chan(rtwdev, true); chip->ops->phy_calibration(rtwdev); @@ -490,6 +491,7 @@ static void rtw_ops_stop_ap(struct ieee80211_hw *hw, mutex_lock(&rtwdev->mutex); rtw_write32_clr(rtwdev, REG_TCR, BIT_TCR_UPDATE_HGQMD); + rtw_write16(rtwdev, REG_ATIMWND, ATIMWND_DEFAULT); rtwdev->ap_active = false; if (!rtw_core_check_sta_active(rtwdev)) rtw_clear_op_chan(rtwdev); diff --git a/drivers/net/wireless/realtek/rtw88/reg.h b/drivers/net/wireless/realtek/rtw88/reg.h index 08e9494977e0..0b9ba6f37680 100644 --- a/drivers/net/wireless/realtek/rtw88/reg.h +++ b/drivers/net/wireless/realtek/rtw88/reg.h @@ -483,6 +483,8 @@ #define REG_DRVERLYINT 0x0558 #define REG_BCNDMATIM 0x0559 #define REG_ATIMWND 0x055A +#define ATIMWND_DEFAULT 0x2 +#define ATIMWND_AP 0xa #define REG_USTIME_TSF 0x055C #define REG_BCN_MAX_ERR 0x055D #define REG_RXTSF_OFFSET_CCK 0x055E diff --git a/drivers/net/wireless/realtek/rtw88/tx.c b/drivers/net/wireless/realtek/rtw88/tx.c index 8786bbb421c2..701ed7c4f3f7 100644 --- a/drivers/net/wireless/realtek/rtw88/tx.c +++ b/drivers/net/wireless/realtek/rtw88/tx.c @@ -196,7 +196,7 @@ void rtw_tx_report_purge_timer(struct timer_list *t) void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn) { struct rtw_tx_report *tx_report = &rtwdev->tx_report; - unsigned long timeout = RTW_TX_PROBE_TIMEOUT; + unsigned long timeout; unsigned long flags; u8 *drv_data; @@ -208,9 +208,10 @@ void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn) __skb_queue_tail(&tx_report->queue, skb); spin_unlock_irqrestore(&tx_report->q_lock, flags); - if (rtwdev->chip->id == RTW_CHIP_TYPE_8723D && - rtwdev->hci.type == RTW_HCI_TYPE_USB) - timeout = msecs_to_jiffies(2500); + if (rtwdev->hci.type == RTW_HCI_TYPE_PCIE) + timeout = RTW_TX_PROBE_TIMEOUT_SHORT; + else + timeout = RTW_TX_PROBE_TIMEOUT_LONG; mod_timer(&tx_report->purge_timer, jiffies + timeout); } diff --git a/drivers/net/wireless/realtek/rtw88/tx.h b/drivers/net/wireless/realtek/rtw88/tx.h index d34cdeca16f1..168f48e87853 100644 --- a/drivers/net/wireless/realtek/rtw88/tx.h +++ b/drivers/net/wireless/realtek/rtw88/tx.h @@ -7,7 +7,8 @@ #define RTK_TX_MAX_AGG_NUM_MASK 0x1f -#define RTW_TX_PROBE_TIMEOUT msecs_to_jiffies(500) +#define RTW_TX_PROBE_TIMEOUT_SHORT msecs_to_jiffies(500) +#define RTW_TX_PROBE_TIMEOUT_LONG msecs_to_jiffies(2500) struct rtw_tx_desc { __le32 w0; diff --git a/drivers/net/wireless/realtek/rtw88/usb.c b/drivers/net/wireless/realtek/rtw88/usb.c index 64e1c3420e0a..5c766701e632 100644 --- a/drivers/net/wireless/realtek/rtw88/usb.c +++ b/drivers/net/wireless/realtek/rtw88/usb.c @@ -2,9 +2,11 @@ /* Copyright(c) 2018-2019 Realtek Corporation */ +#include #include -#include #include +#include +#include #include "main.h" #include "debug.h" #include "mac.h" @@ -557,9 +559,61 @@ static int rtw_usb_write_data_h2c(struct rtw_dev *rtwdev, u8 *buf, u32 size) return rtw_usb_write_data(rtwdev, &pkt_info, buf); } -static u8 rtw_usb_tx_queue_mapping_to_qsel(struct sk_buff *skb) +static bool rtw_usb_bmc_needs_dtim(struct sk_buff *skb) +{ + struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb); + struct udphdr *udphdr; + + if (info->control.flags & IEEE80211_TX_CTRL_PORT_CTRL_PROTO) + return true; + + if (skb->protocol == htons(ETH_P_ARP)) + return true; + + if (skb->protocol == htons(ETH_P_IP) && + ip_hdr(skb)->protocol == IPPROTO_UDP) { + udphdr = udp_hdr(skb); + + return udphdr->dest == htons(67) || udphdr->dest == htons(68); + } + + return false; +} + +#define RTW_USB_HIQ_REFILL_INTERVAL (HZ / 10) /* one unit of budget per 100 ms */ +#define RTW_USB_HIQ_BUDGET_MAX 16 + +static bool rtw_usb_hiq_take_budget(struct rtw_usb *rtwusb) +{ + unsigned long flags, elapsed, add; + bool ok; + + spin_lock_irqsave(&rtwusb->hiq_lock, flags); + + elapsed = jiffies - rtwusb->hiq_refill; + add = elapsed / RTW_USB_HIQ_REFILL_INTERVAL; + if (add) { + rtwusb->hiq_budget = min_t(unsigned long, rtwusb->hiq_budget + add, + RTW_USB_HIQ_BUDGET_MAX); + + /* keep the unfinished part of the interval for the next unit */ + rtwusb->hiq_refill = jiffies - elapsed % RTW_USB_HIQ_REFILL_INTERVAL; + } + + ok = rtwusb->hiq_budget > 0; + if (ok) + rtwusb->hiq_budget--; + + spin_unlock_irqrestore(&rtwusb->hiq_lock, flags); + + return ok; +} + +static u8 rtw_usb_tx_queue_mapping_to_qsel(struct rtw_usb *rtwusb, + struct sk_buff *skb) { struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data; + struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb); __le16 fc = hdr->frame_control; u8 qsel; @@ -567,7 +621,9 @@ static u8 rtw_usb_tx_queue_mapping_to_qsel(struct sk_buff *skb) qsel = TX_DESC_QSEL_MGMT; else if (is_broadcast_ether_addr(hdr->addr1) || is_multicast_ether_addr(hdr->addr1)) - qsel = TX_DESC_QSEL_HIGH; + qsel = (info->flags & IEEE80211_TX_CTL_SEND_AFTER_DTIM) && + rtw_usb_bmc_needs_dtim(skb) && rtw_usb_hiq_take_budget(rtwusb) ? + TX_DESC_QSEL_HIGH : skb->priority; else if (skb_get_queue_mapping(skb) <= IEEE80211_AC_BK) qsel = skb->priority; else @@ -586,7 +642,7 @@ static int rtw_usb_tx_write(struct rtw_dev *rtwdev, u8 *pkt_desc; int ep; - pkt_info->qsel = rtw_usb_tx_queue_mapping_to_qsel(skb); + pkt_info->qsel = rtw_usb_tx_queue_mapping_to_qsel(rtwusb, skb); pkt_desc = skb_push(skb, chip->tx_pkt_desc_sz); memset(pkt_desc, 0, chip->tx_pkt_desc_sz); ep = qsel_to_ep(rtwusb, pkt_info->qsel); @@ -1027,6 +1083,10 @@ static int rtw_usb_init_tx(struct rtw_dev *rtwdev) struct rtw_usb *rtwusb = rtw_get_usb_priv(rtwdev); int i; + spin_lock_init(&rtwusb->hiq_lock); + rtwusb->hiq_budget = RTW_USB_HIQ_BUDGET_MAX; + rtwusb->hiq_refill = jiffies; + rtwusb->txwq = create_singlethread_workqueue("rtw88_usb: tx wq"); if (!rtwusb->txwq) { rtw_err(rtwdev, "failed to create TX work queue\n"); diff --git a/drivers/net/wireless/realtek/rtw88/usb.h b/drivers/net/wireless/realtek/rtw88/usb.h index 9b695b688b24..94ff21dd0846 100644 --- a/drivers/net/wireless/realtek/rtw88/usb.h +++ b/drivers/net/wireless/realtek/rtw88/usb.h @@ -75,6 +75,11 @@ struct rtw_usb { u8 out_ep[RTW_USB_EP_MAX]; int qsel_to_ep[TX_DESC_QSEL_MAX]; + /* protects hiq_budget and hiq_refill */ + spinlock_t hiq_lock; + u32 hiq_budget; + unsigned long hiq_refill; + struct workqueue_struct *txwq, *rxwq; struct sk_buff_head tx_queue[RTW_USB_EP_MAX]; diff --git a/drivers/net/wireless/rsi/rsi_91x_core.c b/drivers/net/wireless/rsi/rsi_91x_core.c index ead4d4e04328..3281fc94e107 100644 --- a/drivers/net/wireless/rsi/rsi_91x_core.c +++ b/drivers/net/wireless/rsi/rsi_91x_core.c @@ -458,7 +458,7 @@ void rsi_core_xmit(struct rsi_common *common, struct sk_buff *skb) tx_params->sta_id = 0; } - if (rsta) { + if (rsta && tid < IEEE80211_NUM_TIDS) { /* Start aggregation if not done for this tid */ if (!rsta->start_tx_aggr[tid]) { rsta->start_tx_aggr[tid] = true; diff --git a/drivers/nvme/common/keyring.c b/drivers/nvme/common/keyring.c index 32d16c53133b..87cb51e1f5a7 100644 --- a/drivers/nvme/common/keyring.c +++ b/drivers/nvme/common/keyring.c @@ -35,6 +35,7 @@ struct key *nvme_tls_key_lookup(key_serial_t key_id) } if (nvme_tls_psk_revoked(key)) { pr_err("key id %08x revoked\n", key_id); + key_put(key); return ERR_PTR(-EKEYREVOKED); } return key; diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index af71d02b8476..a0811b49710c 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -3941,10 +3941,16 @@ static int nvme_subsys_check_duplicate_ids(struct nvme_subsystem *subsys, return 0; } +/* The char device of a namespace rather than of a namespace head */ +static bool nvme_is_ns_cdev(struct device *dev) +{ + return dev->parent->class == &nvme_class; +} + static void nvme_cdev_rel(struct device *dev) { ida_free(&nvme_ns_chr_minor_ida, MINOR(dev->devt)); - if (dev->parent->class == &nvme_class) + if (nvme_is_ns_cdev(dev)) nvme_put_ns(container_of(dev, struct nvme_ns, cdev_device)); else nvme_put_ns_head(container_of(dev, struct nvme_ns_head, @@ -3978,6 +3984,14 @@ int nvme_cdev_add(struct cdev *cdev, struct device *cdev_device, device_initialize(cdev_device); cdev_init(cdev, fops); cdev->owner = owner; + /* Undone in nvme_cdev_rel() */ + if (nvme_is_ns_cdev(cdev_device)) + nvme_get_ns(container_of(cdev_device, struct nvme_ns, + cdev_device)); + else + nvme_get_ns_head(container_of(cdev_device, + struct nvme_ns_head, + cdev_device)); ret = cdev_device_add(cdev, cdev_device); if (ret) put_device(cdev_device); @@ -4010,13 +4024,11 @@ static void nvme_add_ns_cdev(struct nvme_ns *ns) { ns->cdev_device.parent = ns->ctrl->device; - nvme_get_ns(ns); /* Undone in nvme_cdev_rel() */ if (nvme_cdev_add(&ns->cdev, &ns->cdev_device, &nvme_ns_chr_fops, ns->ctrl->ops->module, ns->ctrl->instance, ns->head->instance)) { dev_err(ns->ctrl->device, "Unable to create the ng%dn%d device\n", ns->ctrl->instance, ns->head->instance); - nvme_put_ns(ns); return; } set_bit(NVME_NS_CDEV_LIVE, &ns->flags); @@ -4743,9 +4755,8 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl) nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING_NOIO); mutex_lock(&ctrl->namespaces_lock); - list_splice_init_rcu(&ctrl->namespaces, &ns_list, synchronize_rcu); + list_splice_init_srcu(&ctrl->namespaces, &ns_list, &ctrl->srcu); mutex_unlock(&ctrl->namespaces_lock); - synchronize_srcu(&ctrl->srcu); list_for_each_entry_safe(ns, next, &ns_list, list) nvme_ns_remove(ns); diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index 99ecc73393e5..0920fa9616de 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -648,14 +648,12 @@ static void nvme_add_ns_head_cdev(struct nvme_ns_head *head) { head->cdev_device.parent = &head->subsys->dev; - nvme_get_ns_head(head); /* Undone in nvme_cdev_rel() */ if (nvme_cdev_add(&head->cdev, &head->cdev_device, &nvme_ns_head_chr_fops, THIS_MODULE, head->subsys->instance, head->instance)) { dev_err(disk_to_dev(head->disk), "Unable to create the ng%dn%d device\n", head->subsys->instance, head->instance); - nvme_put_ns_head(head); return; } set_bit(NVME_NSHEAD_CDEV_LIVE, &head->flags); diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 77317e5d00f9..b7c5a18b899b 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -4198,6 +4198,8 @@ static const struct pci_device_id nvme_id_table[] = { .driver_data = NVME_QUIRK_BOGUS_NID, }, { PCI_DEVICE(0x1344, 0x6004), /* Micron 4100AT */ .driver_data = NVME_QUIRK_DMAPOOL_ALIGN_512, }, + { PCI_DEVICE(0x1e0f, 0x0032), /* KIOXIA XG10d */ + .driver_data = NVME_QUIRK_DMAPOOL_ALIGN_512, }, { PCI_DEVICE(0x1c5c, 0x1504), /* SK Hynix PC400 */ .driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, }, { PCI_DEVICE(0x1c5c, 0x174a), /* SK Hynix P31 SSD */ diff --git a/drivers/nvme/host/pr.c b/drivers/nvme/host/pr.c index fe7dbe264815..d8b6ac4d83bb 100644 --- a/drivers/nvme/host/pr.c +++ b/drivers/nvme/host/pr.c @@ -128,8 +128,12 @@ static int nvme_pr_register(struct block_device *bdev, u64 old_key, u64 new_key, data.crkey = cpu_to_le64(old_key); data.nrkey = cpu_to_le64(new_key); - cdw10 = old_key ? NVME_PR_REGISTER_ACT_REPLACE : - NVME_PR_REGISTER_ACT_REG; + if (!new_key) + cdw10 = NVME_PR_REGISTER_ACT_UNREG; + else if (old_key) + cdw10 = NVME_PR_REGISTER_ACT_REPLACE; + else + cdw10 = NVME_PR_REGISTER_ACT_REG; cdw10 |= (flags & PR_FL_IGNORE_KEY) ? NVME_PR_IGNORE_KEY : 0; cdw10 |= NVME_PR_CPTPL_PERSIST; diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c index 5b9b540693b5..ff7b9e379614 100644 --- a/drivers/nvme/host/rdma.c +++ b/drivers/nvme/host/rdma.c @@ -735,7 +735,6 @@ static int nvme_rdma_setup_io_queues(struct nvme_rdma_ctrl *ctrl, for (i = 0; i < nr_queues; i++) { struct nvme_rdma_queue *queue = &ctrl->queues[first + i]; - queue->ctrl = ctrl; queue->queue_size = queue_size; ctxs[i].queue = queue; @@ -798,7 +797,6 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl, bool pi_capable = false; int error; - ctrl->queues[0].ctrl = ctrl; ctrl->queues[0].queue_size = NVME_AQ_DEPTH; error = nvme_rdma_alloc_queue(&ctrl->queues[0]); if (error) @@ -2284,7 +2282,7 @@ static struct nvme_rdma_ctrl *nvme_rdma_alloc_ctrl(struct device *dev, struct nvmf_ctrl_options *opts) { struct nvme_rdma_ctrl *ctrl; - int ret; + int ret, i; ctrl = kzalloc_obj(*ctrl); if (!ctrl) @@ -2343,6 +2341,9 @@ static struct nvme_rdma_ctrl *nvme_rdma_alloc_ctrl(struct device *dev, if (!ctrl->queues) goto out_free_ctrl; + for (i = 0; i < ctrl->ctrl.queue_count; i++) + ctrl->queues[i].ctrl = ctrl; + ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops, 0 /* no quirks, we're perfect! */); if (ret) diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c index ce28c9b03817..50297ce8bdde 100644 --- a/drivers/nvme/host/tcp.c +++ b/drivers/nvme/host/tcp.c @@ -70,6 +70,12 @@ MODULE_PARM_DESC(tls_handshake_timeout, "nvme TLS handshake timeout in seconds (default 10)"); #endif +/* read a TLS control record in chunks of this size */ +#define NVME_TCP_TLS_REC_CHUNK 256 + +/* TLS records read while waiting for the ICResp, the ICResp included */ +#define NVME_TCP_ICRESP_MAX_RECORDS 8 + static atomic_t nvme_tcp_cpu_queues[NR_CPUS]; enum nvme_tcp_send_state { @@ -111,6 +117,9 @@ enum nvme_tcp_queue_flags { NVME_TCP_Q_IO_CPU_SET = 3, }; +#define NVME_TCP_OPT_NR_QUEUES(opts) ((opts)->nr_io_queues + \ + (opts)->nr_write_queues + (opts)->nr_poll_queues + 1) + enum nvme_tcp_recv_state { NVME_TCP_RECV_PDU = 0, NVME_TCP_RECV_DATA, @@ -1550,23 +1559,161 @@ static void nvme_tcp_free_queue(struct nvme_ctrl *nctrl, int qid) * nvme_reset_ctrl_work(), queue teardown is immediately followed by * re-allocation. fput() defers socket cleanup to delayed_fput_work * in workqueue context, which can race with new queue setup. + * + * Clear queue->sock under the queue lock so that a sysfs reader in + * nvme_tcp_get_address() either observes the NVME_TCP_Q_LIVE bit + * cleared or a still valid socket. The queue lock itself is owned + * by the controller and only destroyed when the controller is + * freed, so readers blocked on it cannot race its teardown. */ + mutex_lock(&queue->queue_lock); __fput_sync(queue->sock->file); queue->sock = NULL; + mutex_unlock(&queue->queue_lock); memalloc_noio_restore(noio_flag); kfree(queue->pdu); mutex_destroy(&queue->send_mutex); - mutex_destroy(&queue->queue_lock); mutex_destroy(&queue->pf_cache_lock); } +static int nvme_tcp_recv_tls_alert(struct nvme_tcp_queue *queue, + struct msghdr *msg, int len) +{ + struct sock *sk = queue->sock->sk; + u8 level, description; + + if (len < 2) { + dev_err(queue->ctrl->ctrl.device, + "queue %d: truncated TLS alert\n", + nvme_tcp_queue_id(queue)); + return -ENOTCONN; + } + + /* tls_alert_recv() reads through the iterator */ + iov_iter_revert(&msg->msg_iter, len); + tls_alert_recv(sk, msg, &level, &description); + + if (level == TLS_ALERT_LEVEL_FATAL) { + dev_err(queue->ctrl->ctrl.device, + "queue %d: fatal TLS alert %u\n", + nvme_tcp_queue_id(queue), description); + return -ENOTCONN; + } + + /* a peer shutting down normally is not worth a warning */ + if (description == TLS_ALERT_DESC_CLOSE_NOTIFY) + dev_dbg(queue->ctrl->ctrl.device, + "queue %d: TLS close_notify\n", + nvme_tcp_queue_id(queue)); + else + dev_warn(queue->ctrl->ctrl.device, + "queue %d: TLS alert %u\n", + nvme_tcp_queue_id(queue), description); + + return 0; +} + +/* + * Discard a TLS record that arrived in place of the ICResp. A peer may send + * a NewSessionTicket as soon as the handshake completes. + * + * Returns 0 when the record was discarded and the ICResp can be read, or a + * negative errno if the connection cannot continue. + */ +static int nvme_tcp_skip_tls_record(struct nvme_tcp_queue *queue, + struct msghdr *msg, u8 ctype, int len) +{ + u8 buf[NVME_TCP_TLS_REC_CHUNK]; + int ret; + + switch (ctype) { + case TLS_RECORD_TYPE_HANDSHAKE: + break; + case TLS_RECORD_TYPE_ALERT: + ret = nvme_tcp_recv_tls_alert(queue, msg, len); + return ret ? : -ENOTCONN; + default: + pr_err("queue %d: unhandled TLS record %u\n", + nvme_tcp_queue_id(queue), ctype); + return -ENOTCONN; + } + + /* the ICResp is in a later record, drain this one first */ + while (!(msg->msg_flags & MSG_EOR)) { + struct kvec iov = { + .iov_base = buf, + .iov_len = sizeof(buf), + }; + char cbuf[CMSG_LEN(sizeof(char))] = {}; + /* kTLS fails a control record read without a cmsg buffer */ + struct msghdr drain = { + .msg_control = cbuf, + .msg_controllen = sizeof(cbuf), + }; + + ret = kernel_recvmsg(queue->sock, &drain, &iov, 1, sizeof(buf), + 0); + if (ret <= 0) { + pr_warn("queue %d: failed to drain TLS record: %d\n", + nvme_tcp_queue_id(queue), ret); + return ret ? : -ECONNRESET; + } + msg->msg_flags = drain.msg_flags; + } + + return 0; +} + +/* + * Receive the ICResp. Over TLS, skip the records that a peer may send + * ahead of it. + * + * Returns the length received, or a negative errno. + */ +static int nvme_tcp_recv_icresp(struct nvme_tcp_queue *queue, + struct nvme_tcp_icresp_pdu *icresp) +{ + char cbuf[CMSG_LEN(sizeof(char))]; + struct msghdr msg; + struct kvec iov; + u8 ctype; + int i, ret; + + for (i = 0; i < NVME_TCP_ICRESP_MAX_RECORDS; i++) { + memset(&msg, 0, sizeof(msg)); + memset(cbuf, 0, sizeof(cbuf)); + iov.iov_base = icresp; + iov.iov_len = sizeof(*icresp); + if (nvme_tcp_queue_tls(queue)) { + msg.msg_control = cbuf; + msg.msg_controllen = sizeof(cbuf); + } + msg.msg_flags = MSG_WAITALL; + ret = kernel_recvmsg(queue->sock, &msg, &iov, 1, + iov.iov_len, msg.msg_flags); + if (ret <= 0 || !nvme_tcp_queue_tls(queue)) + return ret; + + ctype = tls_get_record_type(queue->sock->sk, + (struct cmsghdr *)cbuf); + if (ctype == TLS_RECORD_TYPE_DATA) + return ret; + + ret = nvme_tcp_skip_tls_record(queue, &msg, ctype, ret); + if (ret < 0) + return ret; + } + + pr_err("queue %d: too many TLS records before the ICResp\n", + nvme_tcp_queue_id(queue)); + return -EPROTO; +} + static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue) { struct nvme_tcp_icreq_pdu *icreq; struct nvme_tcp_icresp_pdu *icresp; - char cbuf[CMSG_LEN(sizeof(char))] = {}; - u8 ctype; struct msghdr msg = {}; struct kvec iov; bool ctrl_hdgst, ctrl_ddgst; @@ -1604,16 +1751,7 @@ static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue) goto free_icresp; } - memset(&msg, 0, sizeof(msg)); - iov.iov_base = icresp; - iov.iov_len = sizeof(*icresp); - if (nvme_tcp_queue_tls(queue)) { - msg.msg_control = cbuf; - msg.msg_controllen = sizeof(cbuf); - } - msg.msg_flags = MSG_WAITALL; - ret = kernel_recvmsg(queue->sock, &msg, &iov, 1, - iov.iov_len, msg.msg_flags); + ret = nvme_tcp_recv_icresp(queue, icresp); if (ret >= 0 && ret < sizeof(*icresp)) ret = -ECONNRESET; if (ret < 0) { @@ -1621,16 +1759,6 @@ static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue) nvme_tcp_queue_id(queue), ret); goto free_icresp; } - ret = -ENOTCONN; - if (nvme_tcp_queue_tls(queue)) { - ctype = tls_get_record_type(queue->sock->sk, - (struct cmsghdr *)cbuf); - if (ctype != TLS_RECORD_TYPE_DATA) { - pr_err("queue %d: unhandled TLS record %d\n", - nvme_tcp_queue_id(queue), ctype); - goto free_icresp; - } - } ret = -EINVAL; if (icresp->hdr.type != nvme_tcp_icresp) { pr_err("queue %d: bad type returned %d\n", @@ -1877,7 +2005,6 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid, int ret, rcv_pdu_size; struct file *sock_file; - mutex_init(&queue->queue_lock); queue->ctrl = ctrl; init_llist_head(&queue->req_list); INIT_LIST_HEAD(&queue->send_list); @@ -2015,7 +2142,6 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid, queue->sock = NULL; err_destroy_mutex: mutex_destroy(&queue->send_mutex); - mutex_destroy(&queue->queue_lock); mutex_destroy(&queue->pf_cache_lock); return ret; } @@ -2665,6 +2791,9 @@ static void nvme_tcp_stop_ctrl(struct nvme_ctrl *ctrl) static void nvme_tcp_free_ctrl(struct nvme_ctrl *nctrl) { struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl); + /* the same count used to size and initialize the queue array */ + unsigned int nr_queues = NVME_TCP_OPT_NR_QUEUES(nctrl->opts); + unsigned int i; if (list_empty_careful(&ctrl->list)) goto free_ctrl; @@ -2675,6 +2804,8 @@ static void nvme_tcp_free_ctrl(struct nvme_ctrl *nctrl) nvmf_free_options(nctrl->opts); free_ctrl: + for (i = 0; i < nr_queues; i++) + mutex_destroy(&ctrl->queues[i].queue_lock); kfree(ctrl->queues); kfree(ctrl); } @@ -2927,17 +3058,18 @@ static int nvme_tcp_get_address(struct nvme_ctrl *ctrl, char *buf, int size) len = nvmf_get_address(ctrl, buf, size); - if (!test_bit(NVME_TCP_Q_LIVE, &queue->flags)) - return len; - mutex_lock(&queue->queue_lock); - ret = kernel_getsockname(queue->sock, (struct sockaddr *)&src_addr); - if (ret > 0) { - if (len > 0) - len--; /* strip trailing newline */ - len += scnprintf(buf + len, size - len, "%ssrc_addr=%pISc\n", - (len) ? "," : "", &src_addr); + if (test_bit(NVME_TCP_Q_LIVE, &queue->flags)) { + ret = kernel_getsockname(queue->sock, + (struct sockaddr *)&src_addr); + if (ret > 0) { + if (len > 0) + len--; /* strip trailing newline */ + len += scnprintf(buf + len, size - len, + "%ssrc_addr=%pISc\n", + (len) ? "," : "", &src_addr); + } } mutex_unlock(&queue->queue_lock); @@ -3003,7 +3135,7 @@ static struct nvme_tcp_ctrl *nvme_tcp_alloc_ctrl(struct device *dev, struct nvmf_ctrl_options *opts) { struct nvme_tcp_ctrl *ctrl; - int ret; + int ret, i; ctrl = kzalloc_obj(*ctrl); if (!ctrl) @@ -3014,8 +3146,7 @@ static struct nvme_tcp_ctrl *nvme_tcp_alloc_ctrl(struct device *dev, */ context_unsafe(INIT_LIST_HEAD(&ctrl->list)); ctrl->ctrl.opts = opts; - ctrl->ctrl.queue_count = opts->nr_io_queues + opts->nr_write_queues + - opts->nr_poll_queues + 1; + ctrl->ctrl.queue_count = NVME_TCP_OPT_NR_QUEUES(opts); ctrl->ctrl.sqsize = opts->queue_size - 1; ctrl->ctrl.kato = opts->kato; @@ -3073,6 +3204,14 @@ static struct nvme_tcp_ctrl *nvme_tcp_alloc_ctrl(struct device *dev, goto out_free_ctrl; } + /* + * The queue locks are owned by the controller, not the queues: + * they outlive the per-queue connect/disconnect cycles so that + * sysfs readers can never race their destruction. + */ + for (i = 0; i < ctrl->ctrl.queue_count; i++) + mutex_init(&ctrl->queues[i].queue_lock); + ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_tcp_ctrl_ops, 0); if (ret) goto out_kfree_queues; diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c index 7764a3c0195c..3704493dbec9 100644 --- a/drivers/nvme/target/admin-cmd.c +++ b/drivers/nvme/target/admin-cmd.c @@ -543,6 +543,15 @@ static void nvmet_execute_get_log_page_ana(struct nvmet_req *req) u16 ngrps = 0; u16 status; + /* + * The header and one descriptor per enabled group are the only bytes + * written below; zero the whole transfer first so the unwritten tail + * does not leak stale kernel memory to the host. + */ + status = nvmet_zero_sgl(req, 0, req->transfer_len); + if (status) + goto out; + status = NVME_SC_INTERNAL; desc = kmalloc_flex(*desc, nsids, NVMET_MAX_NAMESPACES); if (!desc) @@ -1225,8 +1234,13 @@ u16 nvmet_set_feat_kato(struct nvmet_req *req) { u32 val32 = le32_to_cpu(req->cmd->common.cdw11); + if (val32 > NVMET_MAX_KATO * MSEC_PER_SEC) { + req->error_loc = offsetof(struct nvme_common_command, cdw11); + return NVME_SC_KA_TIMEOUT_INVALID | NVME_STATUS_DNR; + } + nvmet_stop_keep_alive_timer(req->sq->ctrl); - req->sq->ctrl->kato = DIV_ROUND_UP(val32, 1000); + req->sq->ctrl->kato = DIV_ROUND_UP_ULL(val32, MSEC_PER_SEC); nvmet_start_keep_alive_timer(req->sq->ctrl); nvmet_set_result(req, req->sq->ctrl->kato); diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c index 83b86180c50f..61107bcfc895 100644 --- a/drivers/nvme/target/configfs.c +++ b/drivers/nvme/target/configfs.c @@ -150,7 +150,7 @@ static ssize_t nvmet_addr_traddr_store(struct config_item *item, if (nvmet_is_port_enabled(port, __func__)) return -EACCES; - if (sscanf(page, "%s\n", port->disc_addr.traddr) != 1) + if (sscanf(page, "%255s", port->disc_addr.traddr) != 1) return -EINVAL; return count; } @@ -242,7 +242,7 @@ static ssize_t nvmet_addr_trsvcid_store(struct config_item *item, if (nvmet_is_port_enabled(port, __func__)) return -EACCES; - if (sscanf(page, "%s\n", port->disc_addr.trsvcid) != 1) + if (sscanf(page, "%31s", port->disc_addr.trsvcid) != 1) return -EINVAL; return count; } diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c index 07a74c2e1f58..0f5e94bf12bd 100644 --- a/drivers/nvme/target/core.c +++ b/drivers/nvme/target/core.c @@ -1612,6 +1612,12 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args) u8 dhchap_status; int ret; + if (kato > NVMET_MAX_KATO * MSEC_PER_SEC) { + args->status = NVME_SC_KA_TIMEOUT_INVALID | NVME_STATUS_DNR; + args->error_loc = offsetof(struct nvmf_connect_command, kato); + return NULL; + } + args->status = NVME_SC_CONNECT_INVALID_PARAM | NVME_STATUS_DNR; subsys = nvmet_find_get_subsys(args->port, args->subsysnqn); if (!subsys) { @@ -1678,7 +1684,7 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args) kato = NVMET_DISC_KATO_MS; /* keep-alive timeout in seconds */ - ctrl->kato = DIV_ROUND_UP(kato, 1000); + ctrl->kato = DIV_ROUND_UP_ULL(kato, MSEC_PER_SEC); ctrl->err_counter = 0; spin_lock_init(&ctrl->error_lock); diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h index 2032db380f99..215315ebe42d 100644 --- a/drivers/nvme/target/nvmet.h +++ b/drivers/nvme/target/nvmet.h @@ -7,6 +7,7 @@ #define _NVMET_H #include +#include #include #include #include @@ -702,6 +703,13 @@ void nvmet_add_async_event(struct nvmet_ctrl *ctrl, u8 event_type, #define NVMET_KAS 10 #define NVMET_DISC_KATO_MS 120000 +/* + * Largest keep-alive timeout in seconds that can be reported back in + * milliseconds and converted to jiffies without overflowing. + */ +#define NVMET_MAX_KATO \ + min3(U32_MAX / MSEC_PER_SEC, U32_MAX / HZ, MAX_JIFFY_OFFSET / HZ) + int __init nvmet_init_configfs(void); void __exit nvmet_exit_configfs(void); diff --git a/drivers/pci/controller/cadence/pcie-cadence-plat.c b/drivers/pci/controller/cadence/pcie-cadence-plat.c index a1ea24fc3b63..4a24302cc5b2 100644 --- a/drivers/pci/controller/cadence/pcie-cadence-plat.c +++ b/drivers/pci/controller/cadence/pcie-cadence-plat.c @@ -14,14 +14,6 @@ #define CDNS_PLAT_CPU_TO_BUS_ADDR 0x0FFFFFFF -/** - * struct cdns_plat_pcie - private data for this PCIe platform driver - * @pcie: Cadence PCIe controller - */ -struct cdns_plat_pcie { - struct cdns_pcie *pcie; -}; - static const struct of_device_id cdns_plat_pcie_of_match[]; static u64 cdns_plat_cpu_addr_fixup(struct cdns_pcie *pcie, u64 cpu_addr) @@ -36,12 +28,11 @@ static const struct cdns_pcie_ops cdns_plat_ops = { static int cdns_plat_pcie_probe(struct platform_device *pdev) { const struct cdns_plat_pcie_of_data *data; - struct cdns_plat_pcie *cdns_plat_pcie; struct device *dev = &pdev->dev; struct pci_host_bridge *bridge; + struct cdns_pcie *pcie; struct cdns_pcie_ep *ep; struct cdns_pcie_rc *rc; - int phy_count; bool is_rc; int ret; @@ -52,11 +43,6 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev) is_rc = data->is_rc; pr_debug(" Started %s with is_rc: %d\n", __func__, is_rc); - cdns_plat_pcie = devm_kzalloc(dev, sizeof(*cdns_plat_pcie), GFP_KERNEL); - if (!cdns_plat_pcie) - return -ENOMEM; - - platform_set_drvdata(pdev, cdns_plat_pcie); if (is_rc) { if (!IS_ENABLED(CONFIG_PCIE_CADENCE_PLAT_HOST)) return -ENODEV; @@ -68,9 +54,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev) rc = pci_host_bridge_priv(bridge); rc->pcie.dev = dev; rc->pcie.ops = &cdns_plat_ops; - cdns_plat_pcie->pcie = &rc->pcie; + pcie = &rc->pcie; - ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie); + ret = cdns_pcie_init_phy(dev, pcie); if (ret) { dev_err(dev, "failed to init phy\n"); return ret; @@ -95,9 +81,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev) ep->pcie.dev = dev; ep->pcie.ops = &cdns_plat_ops; - cdns_plat_pcie->pcie = &ep->pcie; + pcie = &ep->pcie; - ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie); + ret = cdns_pcie_init_phy(dev, pcie); if (ret) { dev_err(dev, "failed to init phy\n"); return ret; @@ -115,18 +101,17 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev) goto err_init; } + platform_set_drvdata(pdev, pcie); + return 0; err_init: err_get_sync: pm_runtime_put_sync(dev); pm_runtime_disable(dev); - cdns_pcie_disable_phy(cdns_plat_pcie->pcie); - phy_count = cdns_plat_pcie->pcie->phy_count; - while (phy_count--) - device_link_del(cdns_plat_pcie->pcie->link[phy_count]); + cdns_pcie_disable_phy(pcie); - return 0; + return ret; } static void cdns_plat_pcie_shutdown(struct platform_device *pdev) diff --git a/drivers/pci/controller/cadence/pcie-cadence.c b/drivers/pci/controller/cadence/pcie-cadence.c index a1eada56edba..0ac980249941 100644 --- a/drivers/pci/controller/cadence/pcie-cadence.c +++ b/drivers/pci/controller/cadence/pcie-cadence.c @@ -222,7 +222,6 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie) struct device_node *np = dev->of_node; int phy_count; struct phy **phy; - struct device_link **link; int i; int ret; const char *name; @@ -238,10 +237,6 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie) if (!phy) return -ENOMEM; - link = devm_kcalloc(dev, phy_count, sizeof(*link), GFP_KERNEL); - if (!link) - return -ENOMEM; - for (i = 0; i < phy_count; i++) { of_property_read_string_index(np, "phy-names", i, &name); phy[i] = devm_phy_get(dev, name); @@ -249,17 +244,10 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie) ret = PTR_ERR(phy[i]); goto err_phy; } - link[i] = device_link_add(dev, &phy[i]->dev, DL_FLAG_STATELESS); - if (!link[i]) { - devm_phy_put(dev, phy[i]); - ret = -EINVAL; - goto err_phy; - } } pcie->phy_count = phy_count; pcie->phy = phy; - pcie->link = link; ret = cdns_pcie_enable_phy(pcie); if (ret) @@ -268,10 +256,8 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie) return 0; err_phy: - while (--i >= 0) { - device_link_del(link[i]); + while (--i >= 0) devm_phy_put(dev, phy[i]); - } return ret; } diff --git a/drivers/pci/controller/cadence/pcie-cadence.h b/drivers/pci/controller/cadence/pcie-cadence.h index b4d51adefea5..4a96e1869e53 100644 --- a/drivers/pci/controller/cadence/pcie-cadence.h +++ b/drivers/pci/controller/cadence/pcie-cadence.h @@ -260,7 +260,6 @@ struct cdns_plat_pcie_of_data { * @is_hpa: indicates if the architecture is HPA * @phy_count: number of supported PHY devices * @phy: list of pointers to specific PHY control blocks - * @link: list of pointers to corresponding device link representations * @ops: Platform-specific ops to control various inputs from Cadence PCIe * wrapper * @cdns_pcie_reg_offsets: Register bank offsets for different SoC @@ -276,7 +275,6 @@ struct cdns_pcie { bool is_hpa; int phy_count; struct phy **phy; - struct device_link **link; const struct cdns_pcie_ops *ops; const struct cdns_plat_pcie_of_data *cdns_pcie_reg_offsets; int max_link_speed; diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c index 593388f29bdd..9297d0e607e3 100644 --- a/drivers/pci/controller/dwc/pcie-designware.c +++ b/drivers/pci/controller/dwc/pcie-designware.c @@ -577,6 +577,8 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci, /* The data-less messages only for now */ val |= PCIE_ATU_INHIBIT_PAYLOAD | atu->code; } + if (dw_pcie_ver_is_ge(pci, 460A)) + val |= PCIE_ATU_DMA_BYPASS; dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL2, val); /* diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h index 0735ae940924..668a5ba1c6f5 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -201,6 +201,7 @@ #define PCIE_ATU_ENABLE BIT(31) #define PCIE_ATU_BAR_MODE_ENABLE BIT(30) #define PCIE_ATU_CFG_SHIFT_MODE_ENABLE BIT(28) +#define PCIE_ATU_DMA_BYPASS BIT(27) #define PCIE_ATU_INHIBIT_PAYLOAD BIT(22) #define PCIE_ATU_FUNC_NUM_MATCH_EN BIT(19) #define PCIE_ATU_LOWER_BASE 0x008 diff --git a/drivers/platform/chrome/cros_ec_ishtp.c b/drivers/platform/chrome/cros_ec_ishtp.c index 3766cef81fe8..b21a27ad3762 100644 --- a/drivers/platform/chrome/cros_ec_ishtp.c +++ b/drivers/platform/chrome/cros_ec_ishtp.c @@ -369,7 +369,7 @@ static void ish_event_cb(struct ishtp_cl_device *cl_device) * @cros_ish_cl: ISHTP client instance * @reset: true if called from reset handler * - * This function complete the initializtion of the client. + * This function complete the initialization of the client. * * Return: 0 for success, negative error code for failure. */ diff --git a/drivers/platform/chrome/cros_usbpd_notify.c b/drivers/platform/chrome/cros_usbpd_notify.c index d842c05f4db0..25da31ec1996 100644 --- a/drivers/platform/chrome/cros_usbpd_notify.c +++ b/drivers/platform/chrome/cros_usbpd_notify.c @@ -64,7 +64,7 @@ static void cros_usbpd_get_event_and_notify(struct device *dev, /* * We still send a 0 event out to older devices which don't - * have the updated device heirarchy. + * have the updated device hierarchy. */ if (!ec_dev) { dev_dbg(dev, @@ -110,26 +110,29 @@ static int cros_usbpd_notify_probe_acpi(struct platform_device *pdev) if (!pdnotify) return -ENOMEM; - /* Get the EC device pointer needed to talk to the EC. */ - ec_dev = dev_get_drvdata(dev->parent); - if (!ec_dev) { - /* - * We continue even for older devices which don't have the - * correct device heirarchy, namely, GOOG0003 is a child - * of GOOG0004. If GOOG0003 is a child of GOOG0004 and we - * can't get a pointer to the Chrome EC device, defer the - * probe function. - */ - parent_fwnode = fwnode_get_parent(dev->fwnode); - if (parent_fwnode) { - parent_adev = to_acpi_device_node(parent_fwnode); - if (parent_adev && - acpi_dev_hid_match(parent_adev, CREC_DRV_NAME)) { - return -EPROBE_DEFER; - } + /* + * Get the EC device pointer needed to talk to the EC. The parent's + * driver data is only a struct cros_ec_device when GOOG0003 is a + * child of GOOG0004. On older devices without that hierarchy the + * parent may be bound to an unrelated driver (e.g. the ACPI EC + * driver for PNP0C09), so don't touch its driver data and continue + * without an EC pointer. If GOOG0003 is a child of GOOG0004 and we + * can't get a pointer to the Chrome EC device yet, defer the probe. + */ + ec_dev = NULL; + parent_fwnode = fwnode_get_parent(dev->fwnode); + parent_adev = to_acpi_device_node(parent_fwnode); + if (parent_adev && acpi_dev_hid_match(parent_adev, CREC_DRV_NAME)) { + ec_dev = dev_get_drvdata(dev->parent); + if (!ec_dev) { + fwnode_handle_put(parent_fwnode); + return -EPROBE_DEFER; } - dev_warn(dev, "Couldn't get Chrome EC device pointer.\n"); } + fwnode_handle_put(parent_fwnode); + + if (!ec_dev) + dev_warn(dev, "Couldn't get Chrome EC device pointer.\n"); pdnotify->dev = dev; pdnotify->ec = ec_dev; diff --git a/drivers/ptp/ptp_sysfs.c b/drivers/ptp/ptp_sysfs.c index dc398c6b7528..7d09256aca72 100644 --- a/drivers/ptp/ptp_sysfs.c +++ b/drivers/ptp/ptp_sysfs.c @@ -80,13 +80,17 @@ static ssize_t extts_fifo_show(struct device *dev, size_t qcnt; int cnt = 0; + spin_lock_irqsave(&ptp->tsevqs_lock, flags); cnt = list_count_nodes(&ptp->tsevqs); - if (cnt <= 0) + if (cnt <= 0) { + spin_unlock_irqrestore(&ptp->tsevqs_lock, flags); goto out; + } /* The sysfs fifo will always draw from the fist queue */ queue = list_first_entry(&ptp->tsevqs, struct timestamp_event_queue, qlist); + spin_unlock_irqrestore(&ptp->tsevqs_lock, flags); memset(&event, 0, sizeof(event)); spin_lock_irqsave(&queue->lock, flags); diff --git a/drivers/scsi/aic7xxx/aic79xx_core.c b/drivers/scsi/aic7xxx/aic79xx_core.c index 3602a1fc40e4..4abc7948187f 100644 --- a/drivers/scsi/aic7xxx/aic79xx_core.c +++ b/drivers/scsi/aic7xxx/aic79xx_core.c @@ -8942,8 +8942,10 @@ ahd_handle_scsi_status(struct ahd_softc *ahd, struct scb *scb) } #endif - if (ahd_perform_autosense(scb) == 0) + if (ahd_perform_autosense(scb) == 0) { + ahd_done(ahd, scb); break; + } ahd_compile_devinfo(&devinfo, SCB_GET_OUR_ID(scb), SCB_GET_TARGET(ahd, scb), diff --git a/drivers/scsi/aic7xxx/aic79xx_osm.h b/drivers/scsi/aic7xxx/aic79xx_osm.h index 793fe19993a9..3bcfcfe18f08 100644 --- a/drivers/scsi/aic7xxx/aic79xx_osm.h +++ b/drivers/scsi/aic7xxx/aic79xx_osm.h @@ -499,6 +499,8 @@ int ahd_linux_show_info(struct seq_file *,struct Scsi_Host *); static inline void ahd_cmd_set_transaction_status(struct scsi_cmnd *cmd, uint32_t status) { + if (!cmd) + return; cmd->result &= ~(CAM_STATUS_MASK << 16); cmd->result |= status << 16; } @@ -512,6 +514,8 @@ void ahd_set_transaction_status(struct scb *scb, uint32_t status) static inline void ahd_cmd_set_scsi_status(struct scsi_cmnd *cmd, uint32_t status) { + if (!cmd) + return; cmd->result &= ~0xFFFF; cmd->result |= status; } @@ -525,6 +529,8 @@ void ahd_set_scsi_status(struct scb *scb, uint32_t status) static inline uint32_t ahd_cmd_get_transaction_status(struct scsi_cmnd *cmd) { + if (!cmd) + return CAM_REQ_INPROG; return ((cmd->result >> 16) & CAM_STATUS_MASK); } @@ -537,6 +543,8 @@ uint32_t ahd_get_transaction_status(struct scb *scb) static inline uint32_t ahd_cmd_get_scsi_status(struct scsi_cmnd *cmd) { + if (!cmd) + return SAM_STAT_GOOD; return (cmd->result & 0xFFFF); } @@ -564,13 +572,16 @@ u_long ahd_get_transfer_length(struct scb *scb) static inline int ahd_get_transfer_dir(struct scb *scb) { + if (!scb->io_ctx) + return DMA_NONE; return (scb->io_ctx->sc_data_direction); } static inline void ahd_set_residual(struct scb *scb, u_long resid) { - scsi_set_resid(scb->io_ctx, resid); + if (scb->io_ctx) + scsi_set_resid(scb->io_ctx, resid); } static inline @@ -582,6 +593,8 @@ void ahd_set_sense_residual(struct scb *scb, u_long resid) static inline u_long ahd_get_residual(struct scb *scb) { + if (!scb->io_ctx) + return 0; return scsi_get_resid(scb->io_ctx); } @@ -595,11 +608,12 @@ static inline int ahd_perform_autosense(struct scb *scb) { /* - * We always perform autosense in Linux. - * On other platforms this is set on a - * per-transaction basis. + * We always perform autosense in Linux, except for the device + * reset SCB, which has no command to return sense data to. */ - return (1); + if (!scb->io_ctx) + return 0; + return 1; } static inline uint32_t @@ -629,6 +643,8 @@ void ahd_platform_freeze_devq(struct ahd_softc *ahd, struct scb *scb); static inline void ahd_freeze_scb(struct scb *scb) { + if (!scb->io_ctx) + return; if ((scb->io_ctx->result & (CAM_DEV_QFRZN << 16)) == 0) { scb->io_ctx->result |= CAM_DEV_QFRZN << 16; scb->platform_data->dev->qfrozen++; diff --git a/drivers/scsi/aic7xxx/aic7xxx_core.c b/drivers/scsi/aic7xxx/aic7xxx_core.c index b9761f9f0bbc..8ad82171e925 100644 --- a/drivers/scsi/aic7xxx/aic7xxx_core.c +++ b/drivers/scsi/aic7xxx/aic7xxx_core.c @@ -5266,9 +5266,13 @@ ahc_init(struct ahc_softc *ahc) /* * Assume we have a board at this stage and it has been reset. + * PCI controllers keep the host ID ahc_pci_config() chose. */ - if ((ahc->flags & AHC_USEDEFAULTS) != 0) - ahc->our_id = ahc->our_id_b = 7; + if ((ahc->flags & AHC_USEDEFAULTS) != 0) { + if ((ahc->chip & AHC_PCI) == 0) + ahc->our_id = 7; + ahc->our_id_b = 7; + } /* * Default to allowing initiator operations. diff --git a/drivers/scsi/aic7xxx/aic7xxx_pci.c b/drivers/scsi/aic7xxx/aic7xxx_pci.c index 2d4c85426dc3..5a04c7d55e4b 100644 --- a/drivers/scsi/aic7xxx/aic7xxx_pci.c +++ b/drivers/scsi/aic7xxx/aic7xxx_pci.c @@ -701,6 +701,21 @@ ahc_find_pci_device(ahc_dev_softc_t pci) return (NULL); } +/* + * The SGI O2 firmware runs its onboard controllers, in slots 1 and 2 of the + * root bus, without the Adaptec BIOS and at host ID 0. It leaves no settings + * in scratch RAM, which may still hold power-on contents with bad parity. + */ +static bool +ahc_pci_sgi_o2_onboard(struct ahc_softc *ahc) +{ + int slot = ahc_get_pci_slot(ahc->dev_softc); + + return IS_ENABLED(CONFIG_SGI_IP32) && + ahc_get_pci_bus(ahc->dev_softc) == 0 && + (slot == 1 || slot == 2); +} + int ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry) { @@ -782,6 +797,9 @@ ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry) ahc->bus_intr = ahc_pci_intr; ahc->bus_chip_init = ahc_pci_chip_init; + if (ahc_pci_sgi_o2_onboard(ahc)) + ahc->flags |= AHC_NO_BIOS_INIT; + /* Remember how the card was setup in case there is no SEEPROM */ if ((ahc_inb(ahc, HCNTRL) & POWRDN) == 0) { ahc_pause(ahc); @@ -898,7 +916,7 @@ ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry) * Assume only one connector and always turn * on termination. */ - our_id = 0x07; + our_id = ahc_pci_sgi_o2_onboard(ahc) ? 0 : 0x07; sxfrctl1 = STPWEN; } ahc_outb(ahc, SCSICONF, our_id|ENSPCHK|RESET_SCSI); diff --git a/drivers/scsi/scsi_bsg.c b/drivers/scsi/scsi_bsg.c index 5eec248a77a6..e4a65d7a2374 100644 --- a/drivers/scsi/scsi_bsg.c +++ b/drivers/scsi/scsi_bsg.c @@ -41,7 +41,7 @@ static void scsi_bsg_uring_task_cb(struct io_tw_req tw_req, io_tw_token_t tw) u8 sense_len_wr = 0; if (pdu->bio) - blk_rq_unmap_user(pdu->bio); + ret = blk_rq_unmap_user(pdu->bio); if (scsi_status_is_check_condition(scmd->result)) { driver_status = DRIVER_SENSE; @@ -199,7 +199,7 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr, struct scsi_cmnd *scmd; struct request *rq; struct bio *bio; - int ret; + int ret, unmap_ret; if (hdr->protocol != BSG_PROTOCOL_SCSI || hdr->subprotocol != BSG_SUB_PROTOCOL_SCSI_CMD) @@ -273,7 +273,9 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr, else hdr->dout_resid = scmd->resid_len; - blk_rq_unmap_user(bio); + unmap_ret = blk_rq_unmap_user(bio); + if (!ret) + ret = unmap_ret; out_put_request: blk_mq_free_request(rq); diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index 9db2201aa553..c5d5d6a91fee 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -349,6 +349,19 @@ int core_enable_device_list_for_node( new->attach_count++; mutex_lock(&nacl->lun_entry_mutex); + /* + * Do not attach a new dev entry to an se_lun that is being torn down. + * lun_shutdown is set by core_tpg_remove_lun() before its + * core_clear_lun_from_tpg() sweep; checking it here under the same + * nacl->lun_entry_mutex the sweep takes guarantees that once the sweep + * has passed this ACL no dev entry can be left dangling on the + * se_lun->lun_deve_list that is about to be freed. + */ + if (lun->lun_shutdown) { + mutex_unlock(&nacl->lun_entry_mutex); + ret = -ENODEV; + goto free_stats; + } orig = target_nacl_find_deve(nacl, mapped_lun); if (orig && orig->se_lun) { struct se_lun *orig_lun = orig->se_lun; @@ -375,12 +388,12 @@ int core_enable_device_list_for_node( new->se_lun_acl = lun_acl; hlist_del_rcu(&orig->link); hlist_add_head_rcu(&new->link, &nacl->lun_entry_hlist); - mutex_unlock(&nacl->lun_entry_mutex); spin_lock(&lun->lun_deve_lock); list_del(&orig->lun_link); list_add_tail(&new->lun_link, &lun->lun_deve_list); spin_unlock(&lun->lun_deve_lock); + mutex_unlock(&nacl->lun_entry_mutex); kref_put(&orig->pr_kref, target_pr_kref_release); wait_for_completion(&orig->pr_comp); @@ -393,11 +406,10 @@ int core_enable_device_list_for_node( new->se_lun = lun; new->se_lun_acl = lun_acl; hlist_add_head_rcu(&new->link, &nacl->lun_entry_hlist); - mutex_unlock(&nacl->lun_entry_mutex); - spin_lock(&lun->lun_deve_lock); list_add_tail(&new->lun_link, &lun->lun_deve_list); spin_unlock(&lun->lun_deve_lock); + mutex_unlock(&nacl->lun_entry_mutex); target_luns_data_has_changed(nacl, new, true); return 0; diff --git a/drivers/target/target_core_fabric_lib.c b/drivers/target/target_core_fabric_lib.c index 2853b95b2c59..ad024009df56 100644 --- a/drivers/target/target_core_fabric_lib.c +++ b/drivers/target/target_core_fabric_lib.c @@ -371,6 +371,16 @@ static bool iscsi_parse_pr_out_transport_id( pr_err("Missing ISID in iSCSI Initiator port TransportID\n"); return false; } + /* + * The registration code reads the ISID as a C string, + * and its first 8 bytes with get_unaligned_be64(), so + * require 8 bytes after the separator and a NUL inside + * the rest of the descriptor. + */ + if (buf + tid_len - p < 8 || !memchr(p, '\0', buf + tid_len - p)) { + pr_err("Incomplete or unterminated ISID in iSCSI Initiator port TransportID\n"); + return false; + } *port_nexus_ptr = p; /* * Go ahead and do the lower case conversion of the received diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c index ccff9f6e4826..e615e5e30427 100644 --- a/drivers/target/target_core_tpg.c +++ b/drivers/target/target_core_tpg.c @@ -266,6 +266,15 @@ struct se_node_acl *core_tpg_check_initiator_node_acl( kref_get(&acl->acl_kref); acl->dynamic_node_acl = 1; + /* + * Publish the dynamic NodeACL on tpg->acl_node_list before creating + * its demo-mode MappedLUN dev entries below, so that a concurrent LUN + * teardown sweep (core_clear_lun_from_tpg(), which walks acl_node_list) + * can observe and disable those dev entries. Otherwise a dev entry can + * be left on an se_lun->lun_deve_list that is about to be freed. + */ + target_add_node_acl(acl); + /* * Here we only create demo-mode MappedLUNs from the active * TPG LUNs if the fabric is not explicitly asking for @@ -275,7 +284,6 @@ struct se_node_acl *core_tpg_check_initiator_node_acl( (tpg->se_tpg_tfo->tpg_check_demo_mode_login_only(tpg) != 1)) core_tpg_add_node_to_devs(acl, tpg, NULL); - target_add_node_acl(acl); return acl; } EXPORT_SYMBOL(core_tpg_check_initiator_node_acl); diff --git a/drivers/target/target_core_user.c b/drivers/target/target_core_user.c index edc2afd5f4ee..839d5b2e7250 100644 --- a/drivers/target/target_core_user.c +++ b/drivers/target/target_core_user.c @@ -2524,6 +2524,11 @@ static int tcmu_set_data_pages_per_blk(struct tcmu_dev *udev, substring_t *arg) return ret; } + if (val <= 0) { + pr_err("Invalid data_pages_per_blk %d.\n", val); + return -EINVAL; + } + if (val > TCMU_MBS_TO_PAGES(udev->data_area_mb)) { pr_err("Invalid data_pages_per_blk %d: greater than max_data_area_mb %d -> %zd pages).\n", val, udev->data_area_mb, diff --git a/drivers/thermal/intel/int340x_thermal/int3403_thermal.c b/drivers/thermal/intel/int340x_thermal/int3403_thermal.c index 264c9bc8e645..2262de1e14d9 100644 --- a/drivers/thermal/intel/int340x_thermal/int3403_thermal.c +++ b/drivers/thermal/intel/int340x_thermal/int3403_thermal.c @@ -217,8 +217,7 @@ static int int3403_add(struct platform_device *pdev) } - status = acpi_evaluate_integer(priv->adev->handle, "_TMP", - NULL, &tmp); + status = acpi_evaluate_integer_exclusive(priv->adev->handle, "_TMP", NULL, &tmp); if (ACPI_FAILURE(status)) { status = acpi_evaluate_integer(priv->adev->handle, "PTYP", NULL, &priv->type); diff --git a/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.c b/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.c index 965700ee6962..9451fea1f65c 100644 --- a/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.c +++ b/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.c @@ -18,7 +18,11 @@ static int int340x_thermal_get_zone_temp(struct thermal_zone_device *zone, unsigned long long tmp; acpi_status status; - status = acpi_evaluate_integer(d->adev->handle, "_TMP", NULL, &tmp); + /* + * On some ThinkPads _TMP switches the EC page selector without + * taking the mutex the battery methods use. + */ + status = acpi_evaluate_integer_exclusive(d->adev->handle, "_TMP", NULL, &tmp); if (ACPI_FAILURE(status)) return -EIO; diff --git a/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c b/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c index b0284c2e2e74..9bdb65f54b6a 100644 --- a/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c +++ b/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c @@ -287,7 +287,7 @@ int proc_thermal_add(struct device *dev, struct proc_thermal_device *proc_priv) if (ret) return ret; - status = acpi_evaluate_integer(adev->handle, "_TMP", NULL, &tmp); + status = acpi_evaluate_integer_exclusive(adev->handle, "_TMP", NULL, &tmp); if (ACPI_FAILURE(status)) { /* there is no _TMP method, add local method */ if (intel_tcc_get_tjmax(-1) > 0) diff --git a/drivers/vdpa/solidrun/snet_ctrl.c b/drivers/vdpa/solidrun/snet_ctrl.c index e284c3a06717..09afa0487fd6 100644 --- a/drivers/vdpa/solidrun/snet_ctrl.c +++ b/drivers/vdpa/solidrun/snet_ctrl.c @@ -291,7 +291,6 @@ static int snet_send_ctrl_msg(struct snet *snet, u16 opcode, u16 vq_idx) /* Check for errors */ val = snet_read_ctrl(regs); - ret = SNET_VAL_TO_ERR(val); /* Clear the chunk ready bit */ val &= ~SNET_CTRL_CHUNK_RDY_MASK; @@ -301,6 +300,8 @@ static int snet_send_ctrl_msg(struct snet *snet, u16 opcode, u16 vq_idx) if (ret) SNET_WARN(pdev, "Timeout waiting for DPU to complete a control command, err %d\n", ret); + else + ret = SNET_VAL_TO_ERR(val); exit: mutex_unlock(&snet->ctrl_lock); diff --git a/drivers/vdpa/vdpa_user/iova_domain.c b/drivers/vdpa/vdpa_user/iova_domain.c index b6c958224b7c..28a977248a0d 100644 --- a/drivers/vdpa/vdpa_user/iova_domain.c +++ b/drivers/vdpa/vdpa_user/iova_domain.c @@ -76,6 +76,10 @@ int vduse_domain_set_map(struct vduse_iova_domain *domain, for (map = vhost_iotlb_itree_first(iotlb, start, last); map; map = vhost_iotlb_itree_next(map, start, last)) { map_file = (struct vdpa_map_file *)map->opaque; + if (!map_file || map_file->file == domain->file) { + ret = -EINVAL; + goto err; + } ret = vduse_iotlb_add_range(domain, map->start, map->last, map->addr, map->perm, map_file->file, @@ -226,6 +230,7 @@ vduse_domain_get_coherent_page(struct vduse_iova_domain *domain, u64 iova) { u64 start = iova & PAGE_MASK; u64 last = start + PAGE_SIZE - 1; + struct vdpa_map_file *map_file; struct vhost_iotlb_map *map; struct page *page = NULL; @@ -234,6 +239,10 @@ vduse_domain_get_coherent_page(struct vduse_iova_domain *domain, u64 iova) if (!map) goto out; + map_file = (struct vdpa_map_file *)map->opaque; + if (!map_file || map_file->file != domain->file) + goto out; + page = pfn_to_page((map->addr + iova - map->start) >> PAGE_SHIFT); get_page(page); out: @@ -579,7 +588,7 @@ static int vduse_domain_mmap(struct file *file, struct vm_area_struct *vma) { struct vduse_iova_domain *domain = file->private_data; - vm_flags_set(vma, VM_DONTDUMP | VM_DONTEXPAND); + vm_flags_set(vma, VM_IO | VM_DONTDUMP | VM_DONTEXPAND); vma->vm_private_data = domain; vma->vm_ops = &vduse_domain_mmap_ops; diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 4f8c0260bc9e..ddc9bea190de 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -937,7 +937,12 @@ vhost_scsi_copy_iov_to_sgl(struct vhost_scsi_cmd *cmd, struct iov_iter *iter, } for_each_sgtable_sg(sg_table, sg, i) { - page = alloc_page(GFP_KERNEL); + /* + * Completion copies the full bounce buffer, including bytes the + * target did not write. This is already the slow fallback for + * misaligned I/O, so zeroing is acceptable here. + */ + page = alloc_page(GFP_KERNEL | __GFP_ZERO); if (!page) { ret = -ENOMEM; goto err; diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index a31786796d4c..c7a9702c2447 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -1151,7 +1151,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, unsigned long list_size = PAGE_SIZE / sizeof(struct page *); unsigned int gup_flags = FOLL_LONGTERM; unsigned long npages, cur_base, map_pfn, last_pfn = 0; - unsigned long lock_limit, sz2pin, nchunks, i; + unsigned long lock_limit, sz2pin, i; unsigned long page_offset; u64 start = iova; long pinned; @@ -1187,7 +1187,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, cur_base = uaddr & PAGE_MASK; iova &= PAGE_MASK; - nchunks = 0; while (npages) { sz2pin = min_t(unsigned long, npages, list_size); @@ -1202,7 +1201,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, } goto out; } - nchunks++; if (!last_pfn) map_pfn = page_to_pfn(page_list[0]); @@ -1233,7 +1231,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, map_pfn = this_pfn; iova += csize; - nchunks = 0; } last_pfn = this_pfn; @@ -1248,7 +1245,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, PFN_PHYS(map_pfn), perm, NULL); out: if (ret) { - if (nchunks) { + if (last_pfn) { unsigned long pfn; /* @@ -1260,7 +1257,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, * the corresponding unpinning will be handled by * vdpa_unmap(). */ - WARN_ON(!last_pfn); for (pfn = map_pfn; pfn <= last_pfn; pfn++) unpin_user_page(pfn_to_page(pfn)); } diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 44cac11b68d2..0226c4c59709 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -3172,8 +3172,7 @@ static int vhost_add_used_n_in_order(struct vhost_virtqueue *vq, } start += nheads[i]; new += nheads[i]; - if (start >= vq->num) - start -= vq->num; + start &= vq->num - 1; } if (unlikely(vq->log_used)) { diff --git a/drivers/vhost/vringh.c b/drivers/vhost/vringh.c index 9066f9f12ba1..f027f669460f 100644 --- a/drivers/vhost/vringh.c +++ b/drivers/vhost/vringh.c @@ -197,8 +197,9 @@ static int move_to_indirect(const struct vringh *vrh, } len = vringh32_to_cpu(vrh, desc->len); - if (unlikely(len % sizeof(struct vring_desc))) { - vringh_bad("Strange indirect len %u", desc->len); + if (unlikely(len < sizeof(struct vring_desc) || + len % sizeof(struct vring_desc))) { + vringh_bad("Invalid indirect len %u", desc->len); return -EINVAL; } @@ -333,6 +334,17 @@ __vringh_iov(struct vringh *vrh, u16 i, if (unlikely(err)) goto fail; + if (up_next == -1) + count++; + else + indirect_count++; + + if (count > vrh->vring.num || indirect_count > desc_max) { + vringh_bad("Descriptor loop in %p", descs); + err = -ELOOP; + goto fail; + } + if (unlikely(desc.flags & cpu_to_vringh16(vrh, VRING_DESC_F_INDIRECT))) { u64 a = vringh64_to_cpu(vrh, desc.addr); @@ -358,17 +370,6 @@ __vringh_iov(struct vringh *vrh, u16 i, continue; } - if (up_next == -1) - count++; - else - indirect_count++; - - if (count > vrh->vring.num || indirect_count > desc_max) { - vringh_bad("Descriptor loop in %p", descs); - err = -ELOOP; - goto fail; - } - if (desc.flags & cpu_to_vringh16(vrh, VRING_DESC_F_WRITE)) iov = wiov; else { diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c index abed1fbcf66c..35f75e23c749 100644 --- a/drivers/vhost/vsock.c +++ b/drivers/vhost/vsock.c @@ -364,7 +364,7 @@ static struct sk_buff * vhost_vsock_alloc_skb(struct vhost_virtqueue *vq, unsigned int out, unsigned int in) { - struct virtio_vsock_hdr *hdr; + struct virtio_vsock_hdr hdr; struct iov_iter iov_iter; struct sk_buff *skb; size_t payload_len; @@ -382,34 +382,32 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq, len > VIRTIO_VSOCK_MAX_PKT_BUF_SIZE + VIRTIO_VSOCK_SKB_HEADROOM) return NULL; - /* len contains both payload and hdr */ - skb = virtio_vsock_alloc_skb(len, GFP_KERNEL); - if (!skb) - return NULL; - iov_iter_init(&iov_iter, ITER_SOURCE, vq->iov, out, len); - hdr = virtio_vsock_hdr(skb); - nbytes = copy_from_iter(hdr, sizeof(*hdr), &iov_iter); - if (nbytes != sizeof(*hdr)) { + nbytes = copy_from_iter(&hdr, sizeof(hdr), &iov_iter); + if (nbytes != sizeof(hdr)) { vq_err(vq, "Expected %zu bytes for pkt->hdr, got %zu bytes\n", - sizeof(*hdr), nbytes); - kfree_skb(skb); + sizeof(hdr), nbytes); return NULL; } - payload_len = le32_to_cpu(hdr->len); + payload_len = le32_to_cpu(hdr.len); + + /* The pkt is too big or the length in the header is invalid */ + if (payload_len > len - sizeof(hdr)) + return NULL; + + /* Allocate only for the payload declared in the header. */ + skb = virtio_vsock_alloc_skb(payload_len + sizeof(hdr), GFP_KERNEL); + if (!skb) + return NULL; + + memcpy(virtio_vsock_hdr(skb), &hdr, sizeof(hdr)); /* No payload */ if (!payload_len) return skb; - /* The pkt is too big or the length in the header is invalid */ - if (payload_len + sizeof(*hdr) > len) { - kfree_skb(skb); - return NULL; - } - virtio_vsock_skb_put(skb, payload_len); if (skb_copy_datagram_from_iter(skb, 0, &iov_iter, payload_len)) { diff --git a/fs/9p/v9fs.c b/fs/9p/v9fs.c index 274c5157135d..b30ca5145fae 100644 --- a/fs/9p/v9fs.c +++ b/fs/9p/v9fs.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -406,7 +407,7 @@ int v9fs_parse_param(struct fs_context *fc, struct fs_parameter *param) break; case Opt_privport: fd_opts->privport = true; - rdma_opts->port = true; + rdma_opts->privport = true; break; } @@ -617,21 +618,14 @@ static ssize_t caches_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { - ssize_t n = 0, count = 0, limit = PAGE_SIZE; + ssize_t count = 0; struct v9fs_session_info *v9ses; spin_lock(&v9fs_sessionlist_lock); list_for_each_entry(v9ses, &v9fs_sessionlist, slist) { - if (v9ses->cachetag) { - n = snprintf(buf + count, limit, "%s\n", v9ses->cachetag); - if (n < 0) { - count = n; - break; - } - - count += n; - limit -= n; - } + if (v9ses->cachetag) + count += sysfs_emit_at(buf, count, "%s\n", + v9ses->cachetag); } spin_unlock(&v9fs_sessionlist_lock); diff --git a/fs/9p/vfs_inode.c b/fs/9p/vfs_inode.c index 3829554ca369..2d7e0c1a5353 100644 --- a/fs/9p/vfs_inode.c +++ b/fs/9p/vfs_inode.c @@ -495,6 +495,12 @@ static int v9fs_at_to_dotl_flags(int flags) * nothing and races with concurrent metadata fetches that may already * have observed the post-unlink value (nlink == 0). * + * The same applies to a regular file whose cached nlink has already been + * driven to zero by the server: in the cached modes the attributes come + * from GETATTR and are applied verbatim, so decrementing a link count the + * server does not maintain only trips the drop_nlink() WARN_ON and + * underflows i_nlink. + * * @inode: inode whose nlink is being dropped */ static void v9fs_dec_count(struct inode *inode) @@ -503,6 +509,8 @@ static void v9fs_dec_count(struct inode *inode) if (!(v9ses->cache & (CACHE_META | CACHE_LOOSE))) return; + if (inode->i_nlink == 0) + return; if (!S_ISDIR(inode->i_mode) || inode->i_nlink > 2) drop_nlink(inode); } diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 576cd76ca0bf..beae9dcb5a3e 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -375,6 +375,19 @@ int btrfs_fileattr_set(struct mnt_idmap *idmap, } } + /* + * Callers write back the whole flag set, so only touch the compression + * flags and the compression property if FS_COMPR_FL or FS_NOCOMP_FL + * changes. + */ + if (!((fsflags ^ old_fsflags) & (FS_COMPR_FL | FS_NOCOMP_FL))) { + /* 1 item for the inode */ + trans = btrfs_start_transaction(root, 1); + if (IS_ERR(trans)) + return PTR_ERR(trans); + goto update_flags; + } + /* * The COMPRESS flag can only be changed by users, while the NOCOMPRESS * flag may be changed automatically if compression code won't make @@ -2380,6 +2393,17 @@ static noinline int btrfs_ioctl_snap_destroy(struct file *file, inode = d_inode(dentry); dest = BTRFS_I(inode)->root; + + /* + * A concurrent rename may have changed which subvolume the name + * refers to. Verify that the resolved ID still matches the requested ID. + */ + if (destroy_v2 && (vol_args2->flags & BTRFS_SUBVOL_SPEC_BY_ID) && + btrfs_root_id(dest) != vol_args2->subvolid) { + ret = -EAGAIN; + goto out_end_removing; + } + if (!capable(CAP_SYS_ADMIN)) { /* * Regular user. Only allow this with a special mount diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c index 39cb01ee441a..71b1ec6c0643 100644 --- a/fs/btrfs/sysfs.c +++ b/fs/btrfs/sysfs.c @@ -1857,6 +1857,7 @@ void btrfs_sysfs_add_block_group_type(struct btrfs_block_group *cache) if (space_info->block_group_kobjs[index]) { spin_unlock(&space_info->lock); kobject_put(&rkobj->kobj); + memalloc_nofs_restore(nofs_flag); return; } else { space_info->block_group_kobjs[index] = &rkobj->kobj; diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index e07a7b1d3663..f32a533f4100 100644 --- a/fs/ceph/addr.c +++ b/fs/ceph/addr.c @@ -2220,6 +2220,12 @@ void ceph_fill_inline_data(struct inode *inode, struct page *locked_page, doutc(cl, "%p %llx.%llx len %zu locked_page %p\n", inode, ceph_vinop(inode), len, locked_page); + if (len > PAGE_SIZE) { + pr_warn_ratelimited_client(cl, "oversized inline data %zu\n", + len); + len = PAGE_SIZE; + } + if (len > 0) { void *kaddr = kmap_atomic(page); memcpy(kaddr, data, len); diff --git a/fs/erofs/fileio.c b/fs/erofs/fileio.c index ebb81a7ffd4c..ac846cfbc563 100644 --- a/fs/erofs/fileio.c +++ b/fs/erofs/fileio.c @@ -201,17 +201,29 @@ int erofs_read_meta_folio(struct file *file, struct folio *folio) .dev = { .m_pa = folio_pos(folio), }, }; struct inode *inode = folio_inode(folio); + size_t len = folio_size(folio); + loff_t pos, isize; int err; err = erofs_map_dev(inode->i_sb, &io.dev); if (err) return err; + pos = io.dev.m_dif->fsoff + io.dev.m_pa; + isize = i_size_read(file_inode(io.dev.m_dif->file)); + if (pos + folio_size(folio) > isize) { + len = isize > pos ? isize - pos : 0; + folio_zero_segment(folio, len, folio_size(folio)); + if (!len) { + folio_end_read(folio, true); + return 0; + } + } + io.rq = erofs_fileio_rq_alloc(&io.dev); - io.rq->bio.bi_iter.bi_sector = - (io.dev.m_dif->fsoff + io.dev.m_pa) >> 9; + io.rq->bio.bi_iter.bi_sector = pos >> 9; erofs_onlinefolio_init(folio); - bio_add_folio_nofail(&io.rq->bio, folio, folio_size(folio), 0); + bio_add_folio_nofail(&io.rq->bio, folio, len, 0); erofs_fileio_rq_submit(io.rq); return 0; } diff --git a/fs/ext2/file.c b/fs/ext2/file.c index b9020df7d89e..d4a0a528c342 100644 --- a/fs/ext2/file.c +++ b/fs/ext2/file.c @@ -138,7 +138,8 @@ static ssize_t ext2_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) pos = iocb->ki_pos; status = generic_perform_write(iocb, from); if (unlikely(status < 0)) { - ret = status; + if (!ret) + ret = status; goto out_unlock; } diff --git a/fs/ext2/inode.c b/fs/ext2/inode.c index a9245f0cda4d..53edda7217f3 100644 --- a/fs/ext2/inode.c +++ b/fs/ext2/inode.c @@ -801,6 +801,12 @@ static int ext2_iomap_begin(struct inode *inode, loff_t offset, loff_t length, (first_block << blkbits) < i_size_read(inode)) create = 0; + ret = ext2_get_blocks(inode, first_block, max_blocks, + &bno, &new, &boundary, create); + if (ret < 0) + return ret; + + iomap->flags = 0; /* * Writes that span EOF might trigger an IO size update on completion, * so consider them to be dirty for the purposes of O_DSYNC even if @@ -808,13 +814,6 @@ static int ext2_iomap_begin(struct inode *inode, loff_t offset, loff_t length, */ if ((flags & IOMAP_WRITE) && offset + length > i_size_read(inode)) iomap->flags |= IOMAP_F_DIRTY; - - ret = ext2_get_blocks(inode, first_block, max_blocks, - &bno, &new, &boundary, create); - if (ret < 0) - return ret; - - iomap->flags = 0; iomap->offset = (u64)first_block << blkbits; iomap->bdev = inode->i_sb->s_bdev; diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c index 4b59f30ef45d..a84517023e54 100644 --- a/fs/f2fs/checkpoint.c +++ b/fs/f2fs/checkpoint.c @@ -291,7 +291,7 @@ static struct folio *__get_meta_folio(struct f2fs_sb_info *sbi, pgoff_t index, f2fs_update_iostat(sbi, NULL, FS_META_READ_IO, F2FS_BLKSIZE); folio_lock(folio); - if (unlikely(!is_meta_folio(folio))) { + if (unlikely(!is_meta_folio(sbi, folio))) { f2fs_folio_put(folio, true); goto repeat; } @@ -610,7 +610,7 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, long nr_to_write, enum iostat_type io_type) { struct address_space *mapping = META_MAPPING(sbi); - pgoff_t index = 0, prev = ULONG_MAX; + pgoff_t index = 0, next = ULONG_MAX; struct folio_batch fbatch; long nwritten = 0; int nr_folios; @@ -629,16 +629,15 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, long nr_to_write, for (i = 0; i < nr_folios; i++) { struct folio *folio = fbatch.folios[i]; - if (nr_to_write != LONG_MAX && i != 0 && - folio->index != prev + - folio_nr_pages(fbatch.folios[i-1])) { + if (nr_to_write != LONG_MAX && next != ULONG_MAX && + folio->index != next) { folio_batch_release(&fbatch); goto stop; } folio_lock(folio); - if (unlikely(!is_meta_folio(folio))) { + if (unlikely(!is_meta_folio(sbi, folio))) { continue_unlock: folio_unlock(folio); continue; @@ -659,7 +658,7 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, long nr_to_write, break; } nwritten += folio_nr_pages(folio); - prev = folio->index; + next = folio->index + folio_nr_pages(folio); if (unlikely(nwritten >= nr_to_write)) break; } diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index cfd81bfc28a2..e0abc159b796 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -172,7 +172,8 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task) dec_page_count(F2FS_F_SB(folio), __read_io_type(folio)); if (bio->bi_status == BLK_STS_OK && - F2FS_F_SB(folio)->node_inode && is_node_folio(folio) && + F2FS_F_SB(folio)->node_inode && + is_node_folio(F2FS_F_SB(folio), folio) && f2fs_sanity_check_node_footer(F2FS_F_SB(folio), folio, folio->index, NODE_TYPE_REGULAR, true)) bio->bi_status = BLK_STS_IOERR; @@ -359,12 +360,12 @@ static void f2fs_write_end_bio(struct bio *bio) } } - if (is_node_folio(folio)) { + if (is_node_folio(sbi, folio)) { f2fs_sanity_check_node_footer(sbi, folio, folio->index, NODE_TYPE_REGULAR, true); f2fs_bug_on(sbi, folio->index != nid_of_node(folio)); } - if (f2fs_in_warm_node_list(folio)) + if (f2fs_in_warm_node_list(sbi, folio)) f2fs_del_fsync_node_entry(sbi, folio); dec_page_count(sbi, type); @@ -399,7 +400,8 @@ static void f2fs_write_end_io(struct bio *bio) sbi = bio->bi_private; - if (in_atomic() && bio->bi_iter.bi_size > sbi->max_atc_write_bio_size) { + if (bio_in_atomic() && + bio->bi_iter.bi_size > sbi->max_atc_write_bio_size) { struct work_struct *w; w = &container_of(bio, struct f2fs_bio, bio)->work; @@ -500,6 +502,10 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages) struct bio *bio; bdev = f2fs_target_device(sbi, fio->new_blkaddr, §or); + /* avoid reversed IO if bio is split due to exceeding max_segments */ + if (npages > 1 && !is_read_io(fio->op) && + f2fs_is_sequential_zone_area(sbi, fio->new_blkaddr)) + npages = min_t(int, npages, bdev_max_segments(bdev)); bio = bio_alloc_bioset(bdev, npages, fio->op | fio->op_flags | f2fs_io_flags(fio), GFP_NOIO, &f2fs_bioset); @@ -790,7 +796,19 @@ static bool page_is_mergeable(struct f2fs_sb_info *sbi, struct bio *bio, return false; if (last_blkaddr + 1 != cur_blkaddr) return false; - return bio->bi_bdev == f2fs_target_device(sbi, cur_blkaddr, NULL); + if (bio->bi_bdev != f2fs_target_device(sbi, cur_blkaddr, NULL)) + return false; + /* avoid reversed IO if bio is split due to exceeding max_segments/sectors */ + if (f2fs_is_sequential_zone_area(sbi, cur_blkaddr)) { + struct request_queue *q = bdev_get_queue(bio->bi_bdev); + + if (bio->bi_vcnt >= queue_max_segments(q)) + return false; + if (bio_sectors(bio) + (F2FS_BLKSIZE >> SECTOR_SHIFT) > + queue_max_sectors(q)) + return false; + } + return true; } static bool io_type_is_mergeable(struct f2fs_bio_info *io, diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index 0c85d90307f7..f0024647153d 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -2316,14 +2316,14 @@ static inline struct address_space *NODE_MAPPING(struct f2fs_sb_info *sbi) return sbi->node_inode->i_mapping; } -static inline bool is_meta_folio(struct folio *folio) +static inline bool is_meta_folio(struct f2fs_sb_info *sbi, struct folio *folio) { - return folio->mapping == META_MAPPING(F2FS_F_SB(folio)); + return folio->mapping == META_MAPPING(sbi); } -static inline bool is_node_folio(struct folio *folio) +static inline bool is_node_folio(struct f2fs_sb_info *sbi, struct folio *folio) { - return folio->mapping == NODE_MAPPING(F2FS_F_SB(folio)); + return folio->mapping == NODE_MAPPING(sbi); } static inline bool is_sbi_flag_set(struct f2fs_sb_info *sbi, unsigned int type) @@ -3994,7 +3994,7 @@ enum node_type; int f2fs_check_nid_range(struct f2fs_sb_info *sbi, nid_t nid); bool f2fs_available_free_memory(struct f2fs_sb_info *sbi, int type); -bool f2fs_in_warm_node_list(struct folio *folio); +bool f2fs_in_warm_node_list(struct f2fs_sb_info *sbi, struct folio *folio); void f2fs_init_fsync_node_info(struct f2fs_sb_info *sbi); void f2fs_del_fsync_node_entry(struct f2fs_sb_info *sbi, struct folio *folio); void f2fs_reset_fsync_node_info(struct f2fs_sb_info *sbi); diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c index 0a00180c21dc..6220d55709e9 100644 --- a/fs/f2fs/gc.c +++ b/fs/f2fs/gc.c @@ -1429,7 +1429,7 @@ static int move_data_block(struct inode *inode, block_t bidx, F2FS_BLKSIZE); folio_lock(mfolio); - if (unlikely(!is_meta_folio(mfolio) || + if (unlikely(!is_meta_folio(fio.sbi, mfolio) || !folio_test_uptodate(mfolio))) { err = -EIO; f2fs_folio_put(mfolio, true); @@ -1857,6 +1857,9 @@ static int do_garbage_collect(struct f2fs_sb_info *sbi, if (block_end_segno > end_segno) block_end_segno = end_segno; + if (unlikely(freezing(current))) + goto next_block; + if (is_cursec(sbi, GET_SEC_FROM_SEG(sbi, segno))) { f2fs_err(sbi, "%s: segment %u is used by log", __func__, segno); @@ -1924,18 +1927,12 @@ static int do_garbage_collect(struct f2fs_sb_info *sbi, sbi->next_victim_seg[gc_type] = (cur_segno + 1 < sec_end_segno) ? cur_segno + 1 : NULL_SEGNO; - - if (unlikely(freezing(current))) { - folio_put_refs(sum_folio, 2); - goto stop; - } } next_block: folio_put_refs(sum_folio, 2); segno = block_end_segno; } -stop: if (submitted) f2fs_submit_merged_write(sbi, data_type); diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c index 86c2e67e43b6..e166c5a5d170 100644 --- a/fs/f2fs/node.c +++ b/fs/f2fs/node.c @@ -330,9 +330,9 @@ static unsigned int __gang_lookup_nat_set(struct f2fs_nm_info *nm_i, start, nr); } -bool f2fs_in_warm_node_list(struct folio *folio) +bool f2fs_in_warm_node_list(struct f2fs_sb_info *sbi, struct folio *folio) { - return is_node_folio(folio) && IS_DNODE(folio) && is_cold_node(folio); + return is_node_folio(sbi, folio) && IS_DNODE(folio) && is_cold_node(folio); } void f2fs_init_fsync_node_info(struct f2fs_sb_info *sbi) @@ -835,8 +835,20 @@ int f2fs_get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode) if (!dn->inode_folio) { nfolio[0] = f2fs_get_inode_folio(sbi, nids[0]); - if (IS_ERR(nfolio[0])) - return PTR_ERR(nfolio[0]); + if (IS_ERR(nfolio[0])) { + err = PTR_ERR(nfolio[0]); + /* + * cp_error drops dirty node caches, so the inode block + * of an inode that was never written reads as NEW_ADDR + * and returns -ENOENT. It is not a hole; the inode is + * gone. Callers treat -ENOENT as a hole and keep + * looking block by block (e.g. fiemap up to maxbytes). + */ + if (err == -ENOENT && (f2fs_cp_error(sbi) || + is_sbi_flag_set(sbi, SBI_IS_SHUTDOWN))) + err = -EIO; + return err; + } } else { nfolio[0] = dn->inode_folio; } @@ -1290,7 +1302,7 @@ int f2fs_truncate_inode_blocks(struct inode *inode, pgoff_t from) goto fail; if (offset[1] == 0 && get_nid(folio, offset[0], true)) { folio_lock(folio); - BUG_ON(!is_node_folio(folio)); + BUG_ON(!is_node_folio(sbi, folio)); set_nid(folio, offset[0], 0, true); folio_unlock(folio); } @@ -1605,7 +1617,7 @@ static struct folio *__get_node_folio(struct f2fs_sb_info *sbi, pgoff_t nid, folio_lock(folio); - if (unlikely(!is_node_folio(folio))) { + if (unlikely(!is_node_folio(sbi, folio))) { f2fs_folio_put(folio, true); goto repeat; } @@ -1724,7 +1736,7 @@ static struct folio *last_fsync_dnode(struct f2fs_sb_info *sbi, nid_t ino) folio_lock(folio); - if (unlikely(!is_node_folio(folio))) { + if (unlikely(!is_node_folio(sbi, folio))) { continue_unlock: folio_unlock(folio); continue; @@ -1834,7 +1846,7 @@ static bool __write_node_folio(struct folio *folio, bool atomic, bool do_fsync, f2fs_need_dentry_mark(sbi, ino_of_node(folio))); /* should add to global list before clearing PAGECACHE status */ - if (f2fs_in_warm_node_list(folio)) { + if (f2fs_in_warm_node_list(sbi, folio)) { seq = f2fs_add_fsync_node_entry(sbi, folio); if (seq_id) *seq_id = seq; @@ -1958,7 +1970,7 @@ int f2fs_fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode, folio_lock(folio); - if (unlikely(!is_node_folio(folio))) { + if (unlikely(!is_node_folio(sbi, folio))) { continue_unlock: folio_unlock(folio); continue; @@ -2016,7 +2028,7 @@ int f2fs_fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode, f2fs_debug(sbi, "Retry to write fsync mark: ino=%u, idx=%lx", ino, last_folio->index); folio_lock(last_folio); - if (unlikely(!is_node_folio(last_folio))) { + if (unlikely(!is_node_folio(sbi, last_folio))) { f2fs_folio_put(last_folio, true); ret = -EAGAIN; goto out; @@ -2094,7 +2106,7 @@ void f2fs_flush_inline_data(struct f2fs_sb_info *sbi) folio_lock(folio); - if (unlikely(!is_node_folio(folio))) + if (unlikely(!is_node_folio(sbi, folio))) goto unlock; if (!folio_test_dirty(folio)) goto unlock; @@ -2166,7 +2178,7 @@ int f2fs_sync_node_pages(struct f2fs_sb_info *sbi, else if (!folio_trylock(folio)) continue; - if (unlikely(!is_node_folio(folio))) { + if (unlikely(!is_node_folio(sbi, folio))) { continue_unlock: folio_unlock(folio); continue; @@ -2871,6 +2883,45 @@ int f2fs_recover_inline_xattr(struct inode *inode, struct folio *folio) f2fs_folio_wait_writeback(ifolio, NODE, true, true); memcpy(dst_addr, src_addr, inline_size); update_inode: + /* + * The xattr node of the checkpoint was removed before fsync, e.g. the + * xattrs became small enough to fit inline. Release it as well, or + * the inode keeps pointing to the stale xattrs. + */ + if (!ri->i_xattr_nid && F2FS_I(inode)->i_xattr_nid) { + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); + struct node_info ni; + int err; + + err = f2fs_get_node_info(sbi, F2FS_I(inode)->i_xattr_nid, + &ni, false); + if (err) { + f2fs_folio_put(ifolio, true); + return err; + } + + if (ni.blk_addr != NEW_ADDR && + !f2fs_is_valid_blkaddr(sbi, ni.blk_addr, + DATA_GENERIC_ENHANCE)) { + f2fs_err_ratelimited(sbi, + "nat entry is corrupted, run fsck to fix it, ino:%u, nid:%u, blkaddr:%u", + ni.ino, ni.nid, ni.blk_addr); + set_sbi_flag(sbi, SBI_NEED_FSCK); + f2fs_handle_error(sbi, ERROR_INCONSISTENT_NAT); + f2fs_folio_put(ifolio, true); + return -EFSCORRUPTED; + } + + f2fs_invalidate_blocks(sbi, ni.blk_addr, 1); + dec_valid_node_count(sbi, inode, false); + set_node_addr(sbi, &ni, NULL_ADDR, false); + /* drop the cached xattr node, which may be dirty */ + truncate_inode_pages_range(NODE_MAPPING(sbi), + F2FS_BLK_TO_BYTES((loff_t)ni.nid), + F2FS_BLK_END_BYTES((loff_t)ni.nid)); + f2fs_i_xnid_write(inode, 0); + } + f2fs_update_inode(inode, ifolio); f2fs_folio_put(ifolio, true); return 0; diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index f3a73b6c24c3..75f78ed7eb90 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -3234,6 +3234,7 @@ static int change_curseg(struct f2fs_sb_info *sbi, int type) if (IS_ERR(sum_folio)) { /* GC won't be able to use stale summary pages by cp_error */ memset(curseg->sum_blk, 0, sbi->sum_entry_size); + reset_curseg_fields(curseg); return PTR_ERR(sum_folio); } sum_node = SUM_BLK_PAGE_ADDR(sbi, sum_folio, new_segno); @@ -4158,7 +4159,7 @@ static void do_write_page(struct f2fs_summary *sum, struct f2fs_io_info *fio) __func__, fio->ino, folio->index, type, fio->old_blkaddr, fio->new_blkaddr, err); folio_end_writeback(folio); - if (f2fs_in_warm_node_list(folio)) + if (f2fs_in_warm_node_list(fio->sbi, folio)) f2fs_del_fsync_node_entry(fio->sbi, folio); f2fs_bug_on(fio->sbi, !is_set_ckpt_flags(fio->sbi, CP_ERROR_FLAG)); diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index b9adad85e18c..b18ea09bcbe2 100644 --- a/fs/f2fs/super.c +++ b/fs/f2fs/super.c @@ -4255,6 +4255,7 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi) block_t avail_node_count, valid_node_count; unsigned int nat_blocks, nat_bits_bytes, nat_bits_blocks; unsigned int sit_blk_cnt; + unsigned long long ckpt_size, sit_end, nat_end; int i, j; total = le32_to_cpu(raw_super->segment_count); @@ -4386,6 +4387,29 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi) return 1; } + /* + * Nothing above ties the bitmap lengths to sbi->ckpt, which is + * (1 + cp_payload) blocks. Bound both ends the way __bitmap_ptr() + * lays them out. + */ + ckpt_size = (unsigned long long)(cp_payload + 1) * F2FS_BLKSIZE; + if (__is_set_ckpt_flags(ckpt, CP_LARGE_NAT_BITMAP_FLAG)) { + nat_end = CP_MIN_CHKSUM_OFFSET + sizeof(__le32) + nat_bitmap_size; + sit_end = nat_end + sit_bitmap_size; + } else if (cp_payload > 0) { + nat_end = CP_MIN_CHKSUM_OFFSET + nat_bitmap_size; + sit_end = F2FS_BLKSIZE + sit_bitmap_size; + } else { + sit_end = CP_MIN_CHKSUM_OFFSET + sit_bitmap_size; + nat_end = sit_end + nat_bitmap_size; + } + + if (sit_end > ckpt_size || nat_end > ckpt_size) { + f2fs_err(sbi, "Bitmaps do not fit the checkpoint pack: sit ends at %llu, nat at %llu, pack is %llu bytes", + sit_end, nat_end, ckpt_size); + return 1; + } + if (__is_set_ckpt_flags(ckpt, CP_LARGE_NAT_BITMAP_FLAG) && le32_to_cpu(ckpt->checksum_offset) != CP_MIN_CHKSUM_OFFSET) { f2fs_warn(sbi, "using deprecated layout of large_nat_bitmap, " diff --git a/fs/f2fs/sysfs.c b/fs/f2fs/sysfs.c index 811e350a1430..5ee12547171c 100644 --- a/fs/f2fs/sysfs.c +++ b/fs/f2fs/sysfs.c @@ -1404,6 +1404,7 @@ F2FS_FEATURE_RO_ATTR(linear_lookup); #endif F2FS_FEATURE_RO_ATTR(packed_ssa); F2FS_FEATURE_RO_ATTR(fserror); +F2FS_FEATURE_RO_ATTR(resizable_tail); #define ATTR_LIST(name) (&f2fs_attr_##name.attr) static struct attribute *f2fs_attrs[] = { @@ -1574,6 +1575,7 @@ static struct attribute *f2fs_feat_attrs[] = { #endif BASE_ATTR_LIST(packed_ssa), BASE_ATTR_LIST(fserror), + BASE_ATTR_LIST(resizable_tail), NULL, }; ATTRIBUTE_GROUPS(f2fs_feat); diff --git a/fs/fat/inode.c b/fs/fat/inode.c index f775a004cae1..065658642e7f 100644 --- a/fs/fat/inode.c +++ b/fs/fat/inode.c @@ -216,7 +216,9 @@ static void fat_write_failed(struct address_space *mapping, loff_t to) if (to > inode->i_size) { truncate_pagecache(inode, inode->i_size); + down_write(&MSDOS_I(inode)->truncate_lock); fat_truncate_blocks(inode, inode->i_size); + up_write(&MSDOS_I(inode)->truncate_lock); } } diff --git a/fs/nfsd/nfs2acl.c b/fs/nfsd/nfs2acl.c index 190f5a001900..113ec827ae06 100644 --- a/fs/nfsd/nfs2acl.c +++ b/fs/nfsd/nfs2acl.c @@ -15,6 +15,62 @@ #define NFSDDBG_FACILITY NFSDDBG_PROC +/* + * These maps are identical to the NFSv3 maps (nfs3proc.c). This enables + * the behavior of the two versions to diverge if needed. + */ +static const struct nfsd_access_map nfsd2_regaccess[] = { + { NFS3_ACCESS_READ, NFSD_MAY_READ }, + { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC }, + { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC }, + { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE }, + { 0, 0 } +}; + +static const struct nfsd_access_map nfsd2_diraccess[] = { + { NFS3_ACCESS_READ, NFSD_MAY_READ }, + { NFS3_ACCESS_LOOKUP, NFSD_MAY_EXEC }, + { NFS3_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC }, + { NFS3_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE }, + { NFS3_ACCESS_DELETE, NFSD_MAY_REMOVE }, + { 0, 0 } +}; + +/* + * Some clients - Solaris 2.6 at least, make an access call to the NFS + * server to check for access for things like /dev/null (which really, + * NFSD doesn't care about). So NFSD provides simple access checking + * for those objects, looking mainly at mode bits, ignoring read-only + * filesystem checks. + */ +static const struct nfsd_access_map nfsd2_otheraccess[] = { + { NFS3_ACCESS_READ, NFSD_MAY_READ }, + { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC }, + { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS }, + { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS }, + { 0, 0 } +}; + +static const struct nfsd_access_maps nfsd2_access_maps = { + .regular = nfsd2_regaccess, + .directory = nfsd2_diraccess, + .other = nfsd2_otheraccess, +}; + +static __be32 nfsacld_map_status(__be32 status) +{ + switch (status) { + case nfserr_nofilehandle: + case nfserr_badhandle: + status = nfserr_stale; + break; + case nfserr_wrongsec: + status = nfserr_acces; + break; + } + return status; +} + /* * NULL call. */ @@ -79,6 +135,7 @@ static __be32 nfsacld_proc_getacl(struct svc_rqst *rqstp) /* resp->acl_{access,default} are released in nfssvc_release_getacl. */ out: + resp->status = nfsacld_map_status(resp->status); return rpc_success; fail: @@ -137,6 +194,7 @@ static __be32 nfsacld_proc_setacl(struct svc_rqst *rqstp) out: /* argp->acl_{access,default} are released in nfsaclsvc_release_setacl. */ + resp->status = nfsacld_map_status(resp->status); return rpc_success; out_drop_lock: @@ -163,6 +221,7 @@ static __be32 nfsacld_proc_getattr(struct svc_rqst *rqstp) goto out; resp->status = fh_getattr(&resp->fh, &resp->stat); out: + resp->status = nfsacld_map_status(resp->status); return rpc_success; } @@ -180,11 +239,14 @@ static __be32 nfsacld_proc_access(struct svc_rqst *rqstp) fh_copy(&resp->fh, &argp->fh); resp->access = argp->access; - resp->status = nfsd_access(rqstp, &resp->fh, &resp->access, NULL); + + resp->status = nfsd_access(rqstp, &resp->fh, &nfsd2_access_maps, + &resp->access, NULL); if (resp->status != nfs_ok) goto out; resp->status = fh_getattr(&resp->fh, &resp->stat); out: + resp->status = nfsacld_map_status(resp->status); return rpc_success; } diff --git a/fs/nfsd/nfs3proc.c b/fs/nfsd/nfs3proc.c index 0904d953d10e..b3c634190701 100644 --- a/fs/nfsd/nfs3proc.c +++ b/fs/nfsd/nfs3proc.c @@ -48,6 +48,58 @@ static bool nfsd3_time_in_range(const struct iattr *iap) return true; } +static const struct nfsd_access_map nfsd3_regaccess[] = { + { NFS3_ACCESS_READ, NFSD_MAY_READ }, + { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC }, + { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC }, + { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE }, + { 0, 0 } +}; + +static const struct nfsd_access_map nfsd3_diraccess[] = { + { NFS3_ACCESS_READ, NFSD_MAY_READ }, + { NFS3_ACCESS_LOOKUP, NFSD_MAY_EXEC }, + { NFS3_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC }, + { NFS3_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE }, + { NFS3_ACCESS_DELETE, NFSD_MAY_REMOVE }, + { 0, 0 } +}; + +/* + * Some clients - Solaris 2.6 at least, make an access call to the NFS + * server to check for access for things like /dev/null (which really, + * NFSD doesn't care about). So NFSD provides simple access checking + * for those objects, looking mainly at mode bits, ignoring read-only + * filesystem checks. + */ +static const struct nfsd_access_map nfsd3_otheraccess[] = { + { NFS3_ACCESS_READ, NFSD_MAY_READ }, + { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC }, + { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS }, + { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS }, + { 0, 0 } +}; + +static const struct nfsd_access_maps nfsd3_access_maps = { + .regular = nfsd3_regaccess, + .directory = nfsd3_diraccess, + .other = nfsd3_otheraccess, +}; + +static int nfsd3_iocb_flags(enum nfs3_stable_how how) +{ + switch (how) { + case NFS_FILE_SYNC: + /* persist data and timestamps */ + return IOCB_DSYNC | IOCB_SYNC; + case NFS_DATA_SYNC: + /* persist data only */ + return IOCB_DSYNC; + default: + return 0; + } +} + static __be32 nfsd3_map_status(__be32 status) { switch (status) { @@ -171,7 +223,8 @@ nfsd3_proc_access(struct svc_rqst *rqstp) fh_copy(&resp->fh, &argp->fh); resp->access = argp->access; - resp->status = nfsd_access(rqstp, &resp->fh, &resp->access, NULL); + resp->status = nfsd_access(rqstp, &resp->fh, &nfsd3_access_maps, + &resp->access, NULL); resp->status = nfsd3_map_status(resp->status); return rpc_success; } @@ -260,7 +313,8 @@ nfsd3_proc_write(struct svc_rqst *rqstp) resp->committed = argp->stable; resp->status = nfsd_write(rqstp, &resp->fh, argp->offset, &argp->payload, &cnt, - resp->committed, resp->verf); + nfsd3_iocb_flags(resp->committed), + resp->verf); resp->count = cnt; resp->status = nfsd3_map_status(resp->status); return rpc_success; diff --git a/fs/nfsd/nfs3xdr.c b/fs/nfsd/nfs3xdr.c index e481804bb120..f418cab0bd61 100644 --- a/fs/nfsd/nfs3xdr.c +++ b/fs/nfsd/nfs3xdr.c @@ -556,6 +556,8 @@ nfs3svc_decode_writeargs(struct svc_rqst *rqstp, struct xdr_stream *xdr) return false; if (xdr_stream_decode_u32(xdr, &args->stable) < 0) return false; + if (args->stable > NFS_FILE_SYNC) + return false; /* opaque data */ if (xdr_stream_decode_u32(xdr, &args->len) < 0) diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c index f3f7f14a9368..f19d8ed80a9d 100644 --- a/fs/nfsd/nfs4proc.c +++ b/fs/nfsd/nfs4proc.c @@ -69,6 +69,57 @@ MODULE_PARM_DESC(nfsd4_ssc_umount_timeout, #define NFSDDBG_FACILITY NFSDDBG_PROC +static const struct nfsd_access_map nfsd4_regaccess[] = { + { NFS4_ACCESS_READ, NFSD_MAY_READ }, + { NFS4_ACCESS_EXECUTE, NFSD_MAY_EXEC }, + { NFS4_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC }, + { NFS4_ACCESS_EXTEND, NFSD_MAY_WRITE }, + { NFS4_ACCESS_XAREAD, NFSD_MAY_READ }, + { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE }, + { NFS4_ACCESS_XALIST, NFSD_MAY_READ }, + { 0, 0 } +}; + +static const struct nfsd_access_map nfsd4_diraccess[] = { + { NFS4_ACCESS_READ, NFSD_MAY_READ }, + { NFS4_ACCESS_LOOKUP, NFSD_MAY_EXEC }, + { NFS4_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC }, + { NFS4_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE }, + { NFS4_ACCESS_DELETE, NFSD_MAY_REMOVE }, + { NFS4_ACCESS_XAREAD, NFSD_MAY_READ }, + { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE }, + { NFS4_ACCESS_XALIST, NFSD_MAY_READ }, + { 0, 0 } +}; + +static const struct nfsd_access_map nfsd4_otheraccess[] = { + { NFS4_ACCESS_READ, NFSD_MAY_READ }, + { NFS4_ACCESS_EXECUTE, NFSD_MAY_EXEC }, + { NFS4_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS }, + { NFS4_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS }, + { 0, 0 } +}; + +static const struct nfsd_access_maps nfsd4_access_maps = { + .regular = nfsd4_regaccess, + .directory = nfsd4_diraccess, + .other = nfsd4_otheraccess, +}; + +static int nfsd4_iocb_flags(enum stable_how4 how) +{ + switch (how) { + case FILE_SYNC4: + /* persist data and timestamps */ + return IOCB_DSYNC | IOCB_SYNC; + case DATA_SYNC4: + /* persist data only */ + return IOCB_DSYNC; + default: + return 0; + } +} + static u32 nfsd_attrmask[] = { NFSD_WRITEABLE_ATTRS_WORD0, NFSD_WRITEABLE_ATTRS_WORD1, @@ -795,10 +846,9 @@ nfsd4_access(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, if (access->ac_req_access & ~access_full) return nfserr_inval; - access->ac_resp_access = access->ac_req_access; - return nfsd_access(rqstp, &cstate->current_fh, &access->ac_resp_access, - &access->ac_supported); + return nfsd_access(rqstp, &cstate->current_fh, &nfsd4_access_maps, + &access->ac_resp_access, &access->ac_supported); } static __be32 @@ -1377,7 +1427,7 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, write->wr_how_written = write->wr_stable_how; status = nfsd_vfs_write(rqstp, &cstate->current_fh, nf, write->wr_offset, &write->wr_payload, - &cnt, write->wr_how_written, + &cnt, nfsd4_iocb_flags(write->wr_how_written), (__be32 *)write->wr_verifier.data); nfsd_file_put(nf); @@ -1974,7 +2024,7 @@ static void nfsd4_init_copy_res(struct nfsd4_copy *copy, bool sync) { copy->cp_res.wr_stable_how = test_bit(NFSD4_COPY_F_COMMITTED, ©->cp_flags) ? - NFS_FILE_SYNC : NFS_UNSTABLE; + FILE_SYNC4 : UNSTABLE4; nfsd4_copy_set_sync(copy, sync); } diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c index 6441e54283a3..907a914b04cd 100644 --- a/fs/nfsd/nfs4recover.c +++ b/fs/nfsd/nfs4recover.c @@ -649,6 +649,7 @@ struct cld_upcall { struct completion cu_done; /* daemon has read the whole upcall; protected by cn_lock */ bool cu_inflight; + struct rpc_pipe_msg cu_pipe_msg; union { struct cld_msg_hdr cu_hdr; struct cld_msg cu_msg; @@ -660,22 +661,22 @@ static int __cld_pipe_upcall(struct rpc_pipe *pipe, void *cmsg, struct nfsd_net *nn) { int ret; - struct rpc_pipe_msg msg; struct cld_upcall *cup = container_of(cmsg, struct cld_upcall, cu_u); + struct rpc_pipe_msg *msg = &cup->cu_pipe_msg; - memset(&msg, 0, sizeof(msg)); - msg.data = cmsg; - msg.len = nn->client_tracking_ops->msglen; + memset(msg, 0, sizeof(*msg)); + msg->data = cmsg; + msg->len = nn->client_tracking_ops->msglen; - ret = rpc_queue_upcall(pipe, &msg); + ret = rpc_queue_upcall(pipe, msg); if (ret < 0) { goto out; } wait_for_completion(&cup->cu_done); - if (msg.errno < 0) - ret = msg.errno; + if (msg->errno < 0) + ret = msg->errno; out: return ret; } @@ -838,8 +839,11 @@ cld_pipe_downcall(struct file *filp, const char __user *src, size_t mlen) if (status == -EINPROGRESS) return __cld_pipe_inprogress_downcall(cmsg, nn); - if (copy_from_user(&cup->cu_u.cu_msg_v2, src, mlen) != 0) + if (copy_from_user(&cup->cu_u.cu_msg_v2, src, mlen) != 0) { + cup->cu_pipe_msg.errno = -EFAULT; + complete(&cup->cu_done); return -EFAULT; + } complete(&cup->cu_done); return mlen; @@ -848,9 +852,8 @@ cld_pipe_downcall(struct file *filp, const char __user *src, size_t mlen) static void cld_pipe_destroy_msg(struct rpc_pipe_msg *msg) { - struct cld_msg *cmsg = msg->data; - struct cld_upcall *cup = container_of(cmsg, struct cld_upcall, - cu_u.cu_msg); + struct cld_upcall *cup = container_of(msg, struct cld_upcall, + cu_pipe_msg); struct cld_net *cn = cup->cu_net; /* @@ -867,10 +870,34 @@ cld_pipe_destroy_msg(struct rpc_pipe_msg *msg) complete(&cup->cu_done); } +/* + * An upcall the daemon has consumed is off every pipe list, so the + * purge in rpc_pipe_release() and rpc_close_pipes() cannot reach it. + * Release its waiter here instead; no downcall can arrive now. + */ +static void +cld_release_pipe(struct inode *inode) +{ + struct nfsd_net *nn = net_generic(inode->i_sb->s_fs_info, nfsd_net_id); + struct cld_net *cn = nn->cld_net; + struct cld_upcall *cup; + + spin_lock(&cn->cn_lock); + list_for_each_entry(cup, &cn->cn_list, cu_list) { + if (!cup->cu_inflight) + continue; + cup->cu_inflight = false; + cup->cu_pipe_msg.errno = -EPIPE; + complete(&cup->cu_done); + } + spin_unlock(&cn->cn_lock); +} + static const struct rpc_pipe_ops cld_upcall_ops = { .upcall = rpc_pipe_generic_upcall, .downcall = cld_pipe_downcall, .destroy_msg = cld_pipe_destroy_msg, + .release_pipe = cld_release_pipe, }; static int diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 3bdde4c1bb8e..f0a795940137 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -1517,6 +1517,7 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp) dp->dl_stid.sc_type = SC_TYPE_DELEG; list_add(&dp->dl_perfile, &fp->fi_delegations); list_add(&dp->dl_perclnt, &clp->cl_delegations); + clp->cl_deleg_count++; return 0; } @@ -1548,6 +1549,7 @@ unhash_delegation_locked(struct nfs4_delegation *dp, unsigned short statusmask) ++dp->dl_time; spin_lock(&fp->fi_lock); list_del_init(&dp->dl_perclnt); + dp->dl_stid.sc_client->cl_deleg_count--; list_del_init(&dp->dl_recall_lru); list_del_init(&dp->dl_perfile); spin_unlock(&fp->fi_lock); @@ -7938,6 +7940,7 @@ deleg_reaper(struct nfsd_net *nn) { struct list_head *pos, *next; struct nfs4_client *clp; + unsigned int count; spin_lock(&nn->client_lock); list_for_each_safe(pos, next, &nn->client_lru) { @@ -7947,23 +7950,36 @@ deleg_reaper(struct nfsd_net *nn) continue; if (clp->cl_state != NFSD4_ACTIVE) continue; - if (list_empty(&clp->cl_delegations)) - continue; if (atomic_read(&clp->cl_delegs_in_recall)) continue; if (ktime_get_boottime_seconds() - clp->cl_ra_time < 5) continue; if (clp->cl_cb_state != NFSD4_CB_UP) continue; + /* + * This read races with hash_delegation_locked() and + * unhash_delegation_locked() on other CPUs. A stale + * count only skews the keep value; the next + * laundromat pass sees a more current one. + */ + count = data_race(READ_ONCE(clp->cl_deleg_count)); + if (!count) + continue; if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &clp->cl_ra->ra_cb.cb_flags)) continue; /* release in nfsd4_cb_recall_any_release */ kref_get(&clp->cl_nfsdfs.cl_ref); clp->cl_ra_time = ktime_get_boottime_seconds(); - clp->cl_ra->ra_keep = 0; + /* + * Ask for a single delegation. Recalling one before it + * is needed costs the client an OPEN when it next + * touches the file. + */ + clp->cl_ra->ra_keep = count - 1; clp->cl_ra->ra_bmval[0] = BIT(RCA4_TYPE_MASK_RDATA_DLG) | - BIT(RCA4_TYPE_MASK_WDATA_DLG); + BIT(RCA4_TYPE_MASK_WDATA_DLG) | + BIT(RCA4_TYPE_MASK_DIR_DLG); trace_nfsd_cb_recall_any(clp->cl_ra); nfsd4_run_cb(&clp->cl_ra->ra_cb); } diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c index ebd0dc8f00fc..f7d2884aba44 100644 --- a/fs/nfsd/nfs4xdr.c +++ b/fs/nfsd/nfs4xdr.c @@ -1606,7 +1606,7 @@ nfsd4_decode_write(struct nfsd4_compoundargs *argp, union nfsd4_op_u *u) return nfserr_bad_xdr; if (xdr_stream_decode_u32(argp->xdr, &write->wr_stable_how) < 0) return nfserr_bad_xdr; - if (write->wr_stable_how > NFS_FILE_SYNC) + if (write->wr_stable_how > FILE_SYNC4) return nfserr_bad_xdr; if (xdr_stream_decode_u32(argp->xdr, &write->wr_buflen) < 0) return nfserr_bad_xdr; diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c index 86aa8d202b4b..3a5d0f5cc0fc 100644 --- a/fs/nfsd/nfsctl.c +++ b/fs/nfsd/nfsctl.c @@ -2086,8 +2086,13 @@ static int nfsd_nl_validate_listeners(struct genl_info *info) int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info) { struct net *net = genl_info_net(info); + const struct nlattr *bad_attr = NULL; struct svc_xprt *xprt, *tmp; + const char *bad_xprt = NULL; + unsigned int rpcb_failures; const struct nlattr *attr; + bool skipped_rpcb = false; + bool bad_rpcb = false; struct svc_serv *serv; LIST_HEAD(permsocks); struct nfsd_net *nn; @@ -2177,13 +2182,16 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info) if (delete) svc_xprt_destroy_all(serv, net, false); + rpcb_failures = svc_rpcb_failure_count(serv); + /* walk list of addrs again, open any that still don't exist */ nlmsg_for_each_attr_type(attr, NFSD_A_SERVER_SOCK_ADDR, info->nlhdr, GENL_HDRLEN, rem) { struct nlattr *tb[NFSD_A_SOCK_MAX + 1]; const char *xcl_name; struct sockaddr *sa; - int ret; + bool hit_rpcb; + int flags, ret; /* validated up front in nfsd_nl_validate_listeners() */ if (nla_parse_nested(tb, NFSD_A_SOCK_MAX, attr, @@ -2202,11 +2210,53 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info) continue; } - ret = svc_xprt_create_from_sa(serv, xcl_name, net, sa, 0, + flags = skipped_rpcb ? SVC_SOCK_ANONYMOUS : 0; + ret = svc_xprt_create_from_sa(serv, xcl_name, net, sa, flags, current_cred()); + + hit_rpcb = false; + if (!skipped_rpcb && + svc_rpcb_failure_count(serv) != rpcb_failures) { + skipped_rpcb = true; + hit_rpcb = true; + if (ret < 0) + ret = svc_xprt_create_from_sa(serv, xcl_name, + net, sa, + SVC_SOCK_ANONYMOUS, + current_cred()); + } + /* always save the latest error */ - if (ret < 0) + if (ret < 0) { + bad_attr = attr; + bad_xprt = xcl_name; + bad_rpcb = hit_rpcb; err = ret; + } + } + + /* + * The ack carries the errno of the last entry that failed. Point at + * that entry as well, since several entries can share a transport + * name and the errno alone cannot tell them apart. + */ + if (err) { + NL_SET_BAD_ATTR(info->extack, bad_attr); + if (bad_rpcb) + NL_SET_ERR_MSG_FMT(info->extack, + "cannot create %s listener; rpcbind did not answer", + bad_xprt); + else if (skipped_rpcb) + NL_SET_ERR_MSG_FMT(info->extack, + "cannot create %s listener; rpcbind did not answer earlier", + bad_xprt); + else + NL_SET_ERR_MSG_FMT(info->extack, + "cannot create %s listener", + bad_xprt); + } else if (skipped_rpcb) { + NL_SET_ERR_MSG(info->extack, + "rpcbind did not answer, some listeners are not registered"); } if (!serv->sv_nrthreads && list_empty(&nn->nfsd_serv->sv_permsocks)) diff --git a/fs/nfsd/nfsproc.c b/fs/nfsd/nfsproc.c index aaeb222b62ba..48182f78d6be 100644 --- a/fs/nfsd/nfsproc.c +++ b/fs/nfsd/nfsproc.c @@ -280,7 +280,7 @@ nfsd_proc_write(struct svc_rqst *rqstp) fh_copy(&resp->fh, &argp->fh); resp->status = nfsd_write(rqstp, &resp->fh, argp->offset, - &argp->payload, &cnt, NFS_DATA_SYNC, NULL); + &argp->payload, &cnt, IOCB_DSYNC, NULL); if (resp->status == nfs_ok) resp->status = fh_getattr(&resp->fh, &resp->stat); else if (resp->status == nfserr_jukebox) diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h index 9661c08b879e..df35c35890d4 100644 --- a/fs/nfsd/state.h +++ b/fs/nfsd/state.h @@ -610,6 +610,8 @@ struct nfs4_client { unsigned int cl_state; atomic_t cl_delegs_in_recall; + /* Length of cl_delegations, updated under nn->deleg_lock */ + unsigned int cl_deleg_count; struct nfsd4_cb_recall_any *cl_ra; time64_t cl_ra_time; diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index 8923a9910a08..792c0b327755 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -773,64 +773,21 @@ __be32 nfsd4_vfs_fallocate(struct svc_rqst *rqstp, struct svc_fh *fhp, } #endif /* defined(CONFIG_NFSD_V4) */ -/* - * Check server access rights to a file system object +/** + * nfsd_access - Check caller's access rights to a file system object + * @rqstp: RPC transaction context + * @fhp: target NFS filehandle + * @maps: tables mapping on-the-wire access bits to NFSD_MAY flags + * @access: requested access bits on entry, permitted bits on return + * @supported: optional output of the access bits the server supports + * + * Return: nfs_ok on success, otherwise an nfserr status code */ -struct accessmap { - u32 access; - int how; -}; -static struct accessmap nfs3_regaccess[] = { - { NFS3_ACCESS_READ, NFSD_MAY_READ }, - { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC }, - { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC }, - { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE }, - -#ifdef CONFIG_NFSD_V4 - { NFS4_ACCESS_XAREAD, NFSD_MAY_READ }, - { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE }, - { NFS4_ACCESS_XALIST, NFSD_MAY_READ }, -#endif - - { 0, 0 } -}; - -static struct accessmap nfs3_diraccess[] = { - { NFS3_ACCESS_READ, NFSD_MAY_READ }, - { NFS3_ACCESS_LOOKUP, NFSD_MAY_EXEC }, - { NFS3_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC}, - { NFS3_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE }, - { NFS3_ACCESS_DELETE, NFSD_MAY_REMOVE }, - -#ifdef CONFIG_NFSD_V4 - { NFS4_ACCESS_XAREAD, NFSD_MAY_READ }, - { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE }, - { NFS4_ACCESS_XALIST, NFSD_MAY_READ }, -#endif - - { 0, 0 } -}; - -static struct accessmap nfs3_anyaccess[] = { - /* Some clients - Solaris 2.6 at least, make an access call - * to the server to check for access for things like /dev/null - * (which really, the server doesn't care about). So - * We provide simple access checking for them, looking - * mainly at mode bits, and we make sure to ignore read-only - * filesystem checks - */ - { NFS3_ACCESS_READ, NFSD_MAY_READ }, - { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC }, - { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS }, - { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS }, - - { 0, 0 } -}; - -__be32 -nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, u32 *access, u32 *supported) +__be32 nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, + const struct nfsd_access_maps *maps, + u32 *access, u32 *supported) { - struct accessmap *map; + const struct nfsd_access_map *map; struct svc_export *export; struct dentry *dentry; u32 query, result = 0, sresult = 0; @@ -844,12 +801,11 @@ nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, u32 *access, u32 *suppor dentry = fhp->fh_dentry; if (d_is_reg(dentry)) - map = nfs3_regaccess; + map = maps->regular; else if (d_is_dir(dentry)) - map = nfs3_diraccess; + map = maps->directory; else - map = nfs3_anyaccess; - + map = maps->other; query = *access; for (; map->access; map++) { @@ -859,7 +815,7 @@ nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, u32 *access, u32 *suppor sresult |= map->access; err2 = nfsd_permission(&rqstp->rq_cred, export, - dentry, map->how); + dentry, map->may); switch (err2) { case nfs_ok: result |= map->access; @@ -1424,7 +1380,7 @@ nfsd_direct_write(struct svc_rqst *rqstp, struct svc_fh *fhp, * @offset: Byte offset of start * @payload: xdr_buf containing the write payload * @cnt: IN: number of bytes to write, OUT: number of bytes actually written - * @stable: An NFS stable_how value + * @iocb_flags: VFS IOCB_* flags expressing the requested write stability * @verf: NFS WRITE verifier * * Upon return, caller must invoke fh_put on @fhp. @@ -1436,7 +1392,7 @@ __be32 nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, loff_t offset, const struct xdr_buf *payload, unsigned long *cnt, - int stable, __be32 *verf) + int iocb_flags, __be32 *verf) { struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id); struct file *file = nf->nf_file; @@ -1473,21 +1429,11 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, exp = fhp->fh_export; if (!EX_ISSYNC(exp)) - stable = NFS_UNSTABLE; + iocb_flags = 0; init_sync_kiocb(&kiocb, file); kiocb.ki_pos = offset; - if (likely(!fhp->fh_use_wgather)) { - switch (stable) { - case NFS_FILE_SYNC: - /* persist data and timestamps */ - kiocb.ki_flags |= IOCB_DSYNC | IOCB_SYNC; - break; - case NFS_DATA_SYNC: - /* persist data only */ - kiocb.ki_flags |= IOCB_DSYNC; - break; - } - } + if (likely(!fhp->fh_use_wgather)) + kiocb.ki_flags |= iocb_flags; nvecs = xdr_buf_to_bvec(rqstp->rq_bvec, rqstp->rq_maxpages, payload); if (nvecs < 0) { @@ -1528,7 +1474,7 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, goto out_nfserr; } - if (stable && fhp->fh_use_wgather) { + if (iocb_flags && fhp->fh_use_wgather) { host_err = wait_for_concurrent_writes(file); if (host_err < 0) commit_reset_write_verifier(nn, rqstp, host_err); @@ -1619,7 +1565,7 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp, * @offset: Byte offset of start * @payload: xdr_buf containing the write payload * @cnt: IN: number of bytes to write, OUT: number of bytes actually written - * @stable: An NFS stable_how value + * @iocb_flags: VFS IOCB_* flags expressing the requested write stability * @verf: NFS WRITE verifier * * Upon return, caller must invoke fh_put on @fhp. @@ -1629,8 +1575,8 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp, */ __be32 nfsd_write(struct svc_rqst *rqstp, struct svc_fh *fhp, loff_t offset, - const struct xdr_buf *payload, unsigned long *cnt, int stable, - __be32 *verf) + const struct xdr_buf *payload, unsigned long *cnt, + int iocb_flags, __be32 *verf) { struct nfsd_file *nf; __be32 err; @@ -1642,7 +1588,7 @@ nfsd_write(struct svc_rqst *rqstp, struct svc_fh *fhp, loff_t offset, goto out; err = nfsd_vfs_write(rqstp, fhp, nf, offset, payload, cnt, - stable, verf); + iocb_flags, verf); nfsd_file_put(nf); out: trace_nfsd_write_done(rqstp, fhp, offset, *cnt); diff --git a/fs/nfsd/vfs.h b/fs/nfsd/vfs.h index 4af2ff9e9dfe..bdb58422a3fb 100644 --- a/fs/nfsd/vfs.h +++ b/fs/nfsd/vfs.h @@ -37,6 +37,17 @@ #define NFSD_MAY_CREATE (NFSD_MAY_EXEC|NFSD_MAY_WRITE) #define NFSD_MAY_REMOVE (NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC) +struct nfsd_access_map { + u32 access; + int may; +}; + +struct nfsd_access_maps { + const struct nfsd_access_map *regular; + const struct nfsd_access_map *directory; + const struct nfsd_access_map *other; +}; + struct nfsd_file; /* @@ -100,7 +111,9 @@ __be32 nfsd_create_locked(struct svc_rqst *, struct svc_fh *, __be32 nfsd_create(struct svc_rqst *, struct svc_fh *, char *name, int len, struct nfsd_attrs *attrs, int type, dev_t rdev, struct svc_fh *res); -__be32 nfsd_access(struct svc_rqst *, struct svc_fh *, u32 *, u32 *); +__be32 nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, + const struct nfsd_access_maps *maps, + u32 *access, u32 *supported); __be32 nfsd_create_setattr(struct svc_rqst *rqstp, struct svc_fh *fhp, struct svc_fh *resfhp, struct nfsd_attrs *iap); __be32 nfsd_commit(struct svc_rqst *rqst, struct svc_fh *fhp, @@ -135,11 +148,13 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp, u32 *eof); __be32 nfsd_write(struct svc_rqst *rqstp, struct svc_fh *fhp, loff_t offset, const struct xdr_buf *payload, - unsigned long *cnt, int stable, __be32 *verf); + unsigned long *cnt, int iocb_flags, + __be32 *verf); __be32 nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, loff_t offset, const struct xdr_buf *payload, - unsigned long *cnt, int stable, __be32 *verf); + unsigned long *cnt, int iocb_flags, + __be32 *verf); __be32 nfsd_readlink(struct svc_rqst *, struct svc_fh *, char *, int *); __be32 nfsd_symlink(struct svc_rqst *, struct svc_fh *, diff --git a/fs/nfsd/xdr3.h b/fs/nfsd/xdr3.h index 344203874b4c..cad875d14231 100644 --- a/fs/nfsd/xdr3.h +++ b/fs/nfsd/xdr3.h @@ -39,7 +39,7 @@ struct nfsd3_writeargs { svc_fh fh; __u64 offset; __u32 count; - int stable; + __u32 stable; __u32 len; struct xdr_buf payload; }; diff --git a/fs/notify/inotify/inotify_fsnotify.c b/fs/notify/inotify/inotify_fsnotify.c index 7c326ec2e8a8..6e3af6452dcc 100644 --- a/fs/notify/inotify/inotify_fsnotify.c +++ b/fs/notify/inotify/inotify_fsnotify.c @@ -36,7 +36,7 @@ static bool event_compare(struct fsnotify_event *old_fsn, old = INOTIFY_E(old_fsn); new = INOTIFY_E(new_fsn); - if (old->mask & FS_IN_IGNORED) + if (old->mask & (FS_IN_IGNORED | FS_MOVE)) return false; if ((old->mask == new->mask) && (old->wd == new->wd) && diff --git a/fs/ntfs/ea.c b/fs/ntfs/ea.c index 6ac551fe8ea6..9e00588ff9b5 100644 --- a/fs/ntfs/ea.c +++ b/fs/ntfs/ea.c @@ -699,6 +699,11 @@ static int ntfs_new_attr_flags(struct ntfs_inode *ni, __le32 fattr) goto err_out; } + if ((new_aflags & ATTR_IS_COMPRESSED) && ni->vol->cluster_size > 4096) { + err = -EOPNOTSUPP; + goto err_out; + } + if (!a->non_resident) { if (!(new_aflags & (ATTR_IS_SPARSE | ATTR_IS_COMPRESSED))) { err = 0; diff --git a/fs/ntfs3/xattr.c b/fs/ntfs3/xattr.c index 594ef6860b93..fb7276d4a428 100644 --- a/fs/ntfs3/xattr.c +++ b/fs/ntfs3/xattr.c @@ -146,29 +146,32 @@ static int ntfs_read_ea(struct ntfs_inode *ni, struct EA_FULL **ea, for (off = 0; off < size; off += ea_size) { const struct EA_FULL *ef = Add2Ptr(ea_p, off); u32 bytes = size - off; - size_t need; - - /* Check if we can use field ea->size. */ - if (bytes < sizeof(ef->size)) - goto out1; + u32 packed_size; /* Check if we can use fields ef->name_len and ef->elength. */ if (bytes < offsetof(struct EA_FULL, name)) goto out1; - /* Size needed to hold this record's name and value. */ - need = struct_size(ef, name, - 1 + ef->name_len + le16_to_cpu(ef->elength)); + /* + * The name and the value have to fit in the entry whether or + * not ef->size is set, because ntfs_get_ea() copies elength + * bytes out of this buffer and checks them only against the + * caller supplied buffer size. + */ + packed_size = + ALIGN(struct_size(ef, name, + 1 + ef->name_len + + le16_to_cpu(ef->elength)), + 4); if (ef->size) { ea_size = le32_to_cpu(ef->size); - /* ef->size must fit the list and cover the record. */ - if (ea_size > bytes || ea_size < need) + if (ea_size > bytes || packed_size > ea_size) goto out1; continue; } - ea_size = ALIGN(need, 4); + ea_size = packed_size; if (ea_size > bytes) goto out1; } diff --git a/fs/ocfs2/cluster/tcp.c b/fs/ocfs2/cluster/tcp.c index 474fe1414cee..0ba26bd2eba1 100644 --- a/fs/ocfs2/cluster/tcp.c +++ b/fs/ocfs2/cluster/tcp.c @@ -391,6 +391,9 @@ static void sc_kref_release(struct kref *kref) sclog(sc, "releasing\n"); + /* Unlink before freeing anything the debugfs readers dereference. */ + o2net_debug_del_sc(sc); + if (sc->sc_sock) { sock_release(sc->sc_sock); sc->sc_sock = NULL; @@ -400,8 +403,6 @@ static void sc_kref_release(struct kref *kref) o2nm_node_put(sc->sc_node); sc->sc_node = NULL; - o2net_debug_del_sc(sc); - if (sc->sc_page) __free_page(sc->sc_page); kfree(sc); diff --git a/fs/ocfs2/dcache.h b/fs/ocfs2/dcache.h index 7f246c5692d8..046ab9a80769 100644 --- a/fs/ocfs2/dcache.h +++ b/fs/ocfs2/dcache.h @@ -17,9 +17,8 @@ struct ocfs2_dentry_lock { u64 dl_parent_blkno; /* - * The ocfs2_dentry_lock keeps an inode reference until - * dl_lockres has been destroyed. This is usually done in - * ->d_iput() anyway, so there should be minimal impact. + * The inode reference is dropped before dl_lockres is unlocked. + * Callbacks must not use it to find the mount context during teardown. */ struct inode *dl_inode; struct ocfs2_lock_res dl_lockres; diff --git a/fs/ocfs2/dlm/dlmmaster.c b/fs/ocfs2/dlm/dlmmaster.c index 612969867ff9..ea48c013f7fb 100644 --- a/fs/ocfs2/dlm/dlmmaster.c +++ b/fs/ocfs2/dlm/dlmmaster.c @@ -2525,6 +2525,52 @@ static int dlm_is_lockres_migratable(struct dlm_ctxt *dlm, */ +/* + * Drop the sender's references on a migration MLE after the attempt has + * failed. The finishing assert from the target may already have unlinked + * the MLE and consumed its base reference; decide under the locks the + * assert handler holds, and only drop the base reference while the MLE is + * still hashed. Returns true when the assert got there first: the target + * has become the master, so the caller must complete the migration. + */ +static bool dlm_put_migration_mle(struct dlm_ctxt *dlm, + struct dlm_master_list_entry *mle) +{ + bool asserted; + + spin_lock(&dlm->spinlock); + spin_lock(&dlm->master_lock); + __dlm_mle_detach_hb_events(dlm, mle); + asserted = hlist_unhashed(&mle->master_hash_node); + if (!asserted) { + __dlm_unlink_mle(dlm, mle); + __dlm_put_mle(mle); + } + spin_unlock(&dlm->master_lock); + spin_unlock(&dlm->spinlock); + + dlm_put_mle_inuse(mle); + return asserted; +} + +/* + * The finishing assert got there first: it switched the owner to the + * target and cleared DLM_LOCK_RES_MIGRATING. Drop the other nodes' locks + * as a completed migration does, unless recovery has taken the lockres + * over since, and let the usage be recalculated. + */ +static void dlm_migrate_lockres_asserted(struct dlm_ctxt *dlm, + struct dlm_lock_resource *res, + u8 target) +{ + spin_lock(&res->spinlock); + if (res->owner == target) + dlm_remove_nonlocal_locks(dlm, res); + spin_unlock(&res->spinlock); + + dlm_lockres_calc_usage(dlm, res); +} + static int dlm_migrate_lockres(struct dlm_ctxt *dlm, struct dlm_lock_resource *res, u8 target) { @@ -2609,9 +2655,10 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm, if (ret < 0) { if (mle_added) { - dlm_mle_detach_hb_events(dlm, mle); - dlm_put_mle(mle); - dlm_put_mle_inuse(mle); + if (dlm_put_migration_mle(dlm, mle)) { + dlm_migrate_lockres_asserted(dlm, res, target); + ret = 0; + } } else if (mle) { kmem_cache_free(dlm_mle_cache, mle); mle = NULL; @@ -2640,9 +2687,11 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm, mlog(0, "migration to node %u failed with %d\n", target, ret); /* migration failed, detach and clean up mle */ - dlm_mle_detach_hb_events(dlm, mle); - dlm_put_mle(mle); - dlm_put_mle_inuse(mle); + if (dlm_put_migration_mle(dlm, mle)) { + dlm_migrate_lockres_asserted(dlm, res, target); + ret = 0; + goto leave; + } spin_lock(&res->spinlock); res->state &= ~DLM_LOCK_RES_MIGRATING; wake = 1; @@ -2686,9 +2735,12 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm, res->lockname.name, target); ret = -EINVAL; /* migration failed, detach and clean up mle */ - dlm_mle_detach_hb_events(dlm, mle); - dlm_put_mle(mle); - dlm_put_mle_inuse(mle); + if (dlm_put_migration_mle(dlm, mle)) { + dlm_migrate_lockres_asserted(dlm, res, + target); + ret = 0; + goto leave; + } spin_lock(&res->spinlock); res->state &= ~DLM_LOCK_RES_MIGRATING; wake = 1; diff --git a/fs/ocfs2/dlmglue.c b/fs/ocfs2/dlmglue.c index 1a83241b6d41..a2a33c34fdf9 100644 --- a/fs/ocfs2/dlmglue.c +++ b/fs/ocfs2/dlmglue.c @@ -57,7 +57,6 @@ struct ocfs2_mask_waiter { #endif }; -static struct ocfs2_super *ocfs2_get_dentry_osb(struct ocfs2_lock_res *lockres); static struct ocfs2_super *ocfs2_get_inode_osb(struct ocfs2_lock_res *lockres); static struct ocfs2_super *ocfs2_get_file_osb(struct ocfs2_lock_res *lockres); static struct ocfs2_super *ocfs2_get_qinfo_osb(struct ocfs2_lock_res *lockres); @@ -257,7 +256,6 @@ static const struct ocfs2_lock_res_ops ocfs2_orphan_scan_lops = { }; static const struct ocfs2_lock_res_ops ocfs2_dentry_lops = { - .get_osb = ocfs2_get_dentry_osb, .post_unlock = ocfs2_dentry_post_unlock, .downconvert_worker = ocfs2_dentry_convert_worker, .flags = 0, @@ -308,7 +306,7 @@ static inline struct ocfs2_dentry_lock *ocfs2_lock_res_dl(struct ocfs2_lock_res { BUG_ON(lockres->l_type != OCFS2_LOCK_TYPE_DENTRY); - return (struct ocfs2_dentry_lock *)lockres->l_priv; + return container_of(lockres, struct ocfs2_dentry_lock, dl_lockres); } static inline struct ocfs2_mem_dqinfo *ocfs2_lock_res_qinfo(struct ocfs2_lock_res *lockres) @@ -609,13 +607,6 @@ static __u64 ocfs2_get_dentry_lock_ino(struct ocfs2_lock_res *lockres) return be64_to_cpu(inode_blkno_be); } -static struct ocfs2_super *ocfs2_get_dentry_osb(struct ocfs2_lock_res *lockres) -{ - struct ocfs2_dentry_lock *dl = lockres->l_priv; - - return OCFS2_SB(dl->dl_inode->i_sb); -} - void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl, u64 parent, struct inode *inode) { @@ -623,6 +614,7 @@ void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl, u64 inode_blkno = OCFS2_I(inode)->ip_blkno; __be64 inode_blkno_be = cpu_to_be64(inode_blkno); struct ocfs2_lock_res *lockres = &dl->dl_lockres; + struct ocfs2_super *osb = OCFS2_SB(inode->i_sb); ocfs2_lock_res_init_once(lockres); @@ -651,9 +643,9 @@ void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl, memcpy(&lockres->l_name[OCFS2_DENTRY_LOCK_INO_START], &inode_blkno_be, sizeof(__be64)); - ocfs2_lock_res_init_common(OCFS2_SB(inode->i_sb), lockres, + ocfs2_lock_res_init_common(osb, lockres, OCFS2_LOCK_TYPE_DENTRY, &ocfs2_dentry_lops, - dl); + osb); } static void ocfs2_super_lock_res_init(struct ocfs2_lock_res *res, @@ -3335,6 +3327,14 @@ static void ocfs2_dlm_shutdown_debug(struct ocfs2_super *osb) { struct ocfs2_dlm_debug *dlm_debug = osb->osb_dlm_debug; + /* + * Drain the readers before the cluster stack goes away: the + * locking_state reader resolves ocfs2_dlm_lvb() through the + * stack. Every caller removes the parent directory later. + */ + debugfs_lookup_and_remove("locking_state", osb->osb_debug_root); + debugfs_lookup_and_remove("locking_filter", osb->osb_debug_root); + if (dlm_debug) ocfs2_put_dlm_debug(dlm_debug); } @@ -3349,8 +3349,6 @@ int ocfs2_dlm_init(struct ocfs2_super *osb) goto local; } - ocfs2_dlm_init_debug(osb); - /* launch downconvert thread */ osb->dc_task = kthread_run(ocfs2_downconvert_thread, osb, "ocfs2dc-%s", osb->uuid_str); @@ -3390,9 +3388,10 @@ int ocfs2_dlm_init(struct ocfs2_super *osb) ocfs2_orphan_scan_lock_res_init(&osb->osb_orphan_scan.os_lockres, osb); osb->cconn = conn; + if (conn) + ocfs2_dlm_init_debug(osb); bail: if (status < 0) { - ocfs2_dlm_shutdown_debug(osb); if (osb->dc_task) kthread_stop(osb->dc_task); } @@ -3422,10 +3421,11 @@ void ocfs2_dlm_shutdown(struct ocfs2_super *osb, ocfs2_lock_res_free(&osb->osb_orphan_scan.os_lockres); if (osb->cconn) { + /* The disconnect below may release the selected stack. */ + ocfs2_dlm_shutdown_debug(osb); + ocfs2_cluster_disconnect(osb->cconn, hangup_pending); osb->cconn = NULL; - - ocfs2_dlm_shutdown_debug(osb); } } diff --git a/fs/ocfs2/filecheck.c b/fs/ocfs2/filecheck.c index 75334e4bd082..e3b79bc749f5 100644 --- a/fs/ocfs2/filecheck.c +++ b/fs/ocfs2/filecheck.c @@ -189,6 +189,8 @@ int ocfs2_filecheck_create_sysfs(struct ocfs2_super *osb) /* Mount failure still calls ocfs2_filecheck_remove_sysfs(). */ entry->fs_fcheck = NULL; kobject_put(&entry->fs_kobj); + /* Keep ocfs2_super alive until the kobject is released. */ + wait_for_completion(&entry->fs_kobj_unregister); kfree(fcheck); return ret; } diff --git a/fs/ocfs2/journal.c b/fs/ocfs2/journal.c index a3938a03e93b..674e70dfbb5c 100644 --- a/fs/ocfs2/journal.c +++ b/fs/ocfs2/journal.c @@ -227,13 +227,22 @@ void ocfs2_recovery_disable_quota(struct ocfs2_super *osb) ocfs2_recovery_disable(osb, OCFS2_REC_QUOTA_WANT_DISABLE); } +/* Stop producers and drain completion work without freeing recovery state. */ +void ocfs2_recovery_stop(struct ocfs2_super *osb) +{ + ocfs2_recovery_disable(osb, OCFS2_REC_WANT_DISABLE); +} + void ocfs2_recovery_exit(struct ocfs2_super *osb) { struct ocfs2_recovery_map *rm; - /* disable any new recovery threads and wait for any currently - * running ones to exit. Do this before setting the vol_state. */ - ocfs2_recovery_disable(osb, OCFS2_REC_WANT_DISABLE); + /* + * Disable producers and drain work before freeing recovery state. + * Do this before setting vol_state to VOLUME_DISMOUNTED: completion + * work may still be waiting in __ocfs2_wait_on_mount(). + */ + ocfs2_recovery_stop(osb); /* * Now that recovery is shut down, and the osb is about to be diff --git a/fs/ocfs2/journal.h b/fs/ocfs2/journal.h index 19fc920d26b1..4b52bf7b0938 100644 --- a/fs/ocfs2/journal.h +++ b/fs/ocfs2/journal.h @@ -147,6 +147,7 @@ void ocfs2_complete_recovery(struct work_struct *work); void ocfs2_wait_for_recovery(struct ocfs2_super *osb); int ocfs2_recovery_init(struct ocfs2_super *osb); +void ocfs2_recovery_stop(struct ocfs2_super *osb); void ocfs2_recovery_exit(struct ocfs2_super *osb); void ocfs2_recovery_disable_quota(struct ocfs2_super *osb); diff --git a/fs/ocfs2/move_extents.c b/fs/ocfs2/move_extents.c index ad1678ee7cc4..93b45531bf7d 100644 --- a/fs/ocfs2/move_extents.c +++ b/fs/ocfs2/move_extents.c @@ -907,6 +907,30 @@ static int __ocfs2_move_extents_range(struct buffer_head *di_bh, return ret; } +static int ocfs2_validate_and_adjust_move_range(struct inode *inode, + struct ocfs2_move_extents *range) +{ + u64 i_size = i_size_read(inode); + + if (range->me_start > i_size) + return -EINVAL; + + if (range->me_len > i_size - range->me_start) + range->me_len = i_size - range->me_start; + + /* + * The default threshold for defragmentation is 1 MiB, which + * matches the maximum OCFS2 cluster size. + */ + if (!range->me_threshold) + range->me_threshold = 1024 * 1024; + + if (range->me_threshold > i_size) + range->me_threshold = i_size; + + return 0; +} + static int ocfs2_move_extents(struct ocfs2_move_extents_context *context) { int status; @@ -936,6 +960,21 @@ static int ocfs2_move_extents(struct ocfs2_move_extents_context *context) goto out_rw_unlock; } + status = ocfs2_validate_and_adjust_move_range(inode, context->range); + if (status) { + mlog_errno(status); + goto out_inode_unlock; + } + + /* The goal check depends on the adjusted range length. */ + if (!context->auto_defrag) { + status = ocfs2_validate_and_adjust_move_goal(inode, context->range); + if (status) { + mlog_errno(status); + goto out_inode_unlock; + } + } + /* * remember ip_xattr_sem also needs to be held if necessary */ @@ -1028,27 +1067,8 @@ int ocfs2_ioctl_move_extents(struct file *filp, void __user *argp) goto out_free; } - if (range.me_start > i_size_read(inode)) { - status = -EINVAL; - goto out_free; - } - - if (range.me_start + range.me_len > i_size_read(inode)) - range.me_len = i_size_read(inode) - range.me_start; - context->range = ⦥ - /* - * ok, the default threshold for the defragmentation - * is 1M, since our maximum clustersize was 1M also. - * any thought? - */ - if (!range.me_threshold) - range.me_threshold = 1024 * 1024; - - if (range.me_threshold > i_size_read(inode)) - range.me_threshold = i_size_read(inode); - if (range.me_flags & ~(OCFS2_MOVE_EXT_FL_AUTO_DEFRAG | OCFS2_MOVE_EXT_FL_PART_DEFRAG)) { status = -EINVAL; @@ -1060,23 +1080,11 @@ int ocfs2_ioctl_move_extents(struct file *filp, void __user *argp) if (range.me_flags & OCFS2_MOVE_EXT_FL_PART_DEFRAG) context->partial = 1; - } else { - /* - * first best-effort attempt to validate and adjust the goal - * (physical address in block), while it can't guarantee later - * operation can succeed all the time since global_bitmap may - * change a bit over time. - */ - - status = ocfs2_validate_and_adjust_move_goal(inode, &range); - if (status) - goto out_copy; } status = ocfs2_move_extents(context); if (status) mlog_errno(status); -out_copy: /* * movement/defragmentation may end up being partially completed, * that's the reason why we need to return userspace the finished diff --git a/fs/ocfs2/super.c b/fs/ocfs2/super.c index 169c576f06bf..e5482c556552 100644 --- a/fs/ocfs2/super.c +++ b/fs/ocfs2/super.c @@ -1828,6 +1828,9 @@ static void ocfs2_dismount_volume(struct super_block *sb, int mnt_err) /* Wait for worker to be done with the work structure in osb */ cancel_work_sync(&osb->dquot_drop_work); + /* Stop recovery producers before releasing their local allocators. */ + ocfs2_recovery_stop(osb); + ocfs2_shutdown_local_alloc(osb); ocfs2_truncate_log_shutdown(osb); diff --git a/fs/ocfs2/xattr.c b/fs/ocfs2/xattr.c index 18ebe3cc64ec..4d00a26ece3c 100644 --- a/fs/ocfs2/xattr.c +++ b/fs/ocfs2/xattr.c @@ -260,6 +260,7 @@ static int ocfs2_xattr_tree_list_index_block(struct inode *inode, size_t buffer_size); static int ocfs2_xattr_create_index_block(struct inode *inode, + struct ocfs2_xattr_info *xi, struct ocfs2_xattr_search *xs, struct ocfs2_xattr_set_ctxt *ctxt); @@ -3251,7 +3252,7 @@ static int ocfs2_xattr_block_set(struct inode *inode, else if ((ret != -ENOSPC) || ctxt->set_abort) goto end; else { - ret = ocfs2_xattr_create_index_block(inode, xs, ctxt); + ret = ocfs2_xattr_create_index_block(inode, xi, xs, ctxt); if (ret) goto end; } @@ -4533,6 +4534,21 @@ static void ocfs2_cp_xattr_block_to_bucket(struct inode *inode, for (i = 0; i < blks; i++) memset(bucket_block(bucket, i), 0, blocksize); + /* + * An empty block has no names or values to carry over, and the + * xh_entries[count - 1] below would be xh_entries[-1]. Leave the + * bucket empty, but carry over the hash ocfs2_xa_remove_entry() left + * in the block's xh_entries[0] so that the bucket still holds the key + * it is looked up by. + */ + if (!count) { + xh->xh_num_buckets = cpu_to_le16(1); + xh->xh_free_start = cpu_to_le16(OCFS2_XATTR_BUCKET_SIZE); + xh->xh_entries[0].xe_name_hash = + xb_xh->xh_entries[0].xe_name_hash; + return; + } + /* * Since the xe_name_offset is based on ocfs2_xattr_header, * there is a offset change corresponding to the change of @@ -4579,13 +4595,10 @@ static void ocfs2_cp_xattr_block_to_bucket(struct inode *inode, * real place of the xattr. */ static void ocfs2_xattr_update_xattr_search(struct inode *inode, - struct ocfs2_xattr_search *xs, - struct buffer_head *old_bh) + struct ocfs2_xattr_info *xi, + struct ocfs2_xattr_search *xs) { - char *buf = old_bh->b_data; - struct ocfs2_xattr_block *old_xb = (struct ocfs2_xattr_block *)buf; - struct ocfs2_xattr_header *old_xh = &old_xb->xb_attrs.xb_header; - int i; + int i, count; xs->header = bucket_xh(xs->bucket); xs->base = bucket_block(xs->bucket, 0); @@ -4594,11 +4607,47 @@ static void ocfs2_xattr_update_xattr_search(struct inode *inode, if (xs->not_found) return; - i = xs->here - old_xh->xh_entries; - xs->here = &xs->header->xh_entries[i]; + /* + * A block holds its entries in insertion order while a bucket holds + * them by name hash, so ocfs2_cp_xattr_block_to_bucket() sorted the + * copies and the index this entry had in the block no longer says + * where it is. It copied every entry and every name unchanged, and + * (type, name) pairs are unique within a block, so find it again by + * name. + * + * Search for the name being set, not for the name xs->here carries. + * A failed set may have removed that entry, and ocfs2_xa_remove_entry() + * then memmoves its successor into the slot. Comparing every entry + * rather than narrowing on the hash also finds an entry stored under + * the legacy signed hash, which ocfs2_find_xe_in_bucket() would skip. + */ + count = le16_to_cpu(xs->header->xh_count); + for (i = 0; i < count; i++) { + struct ocfs2_xattr_entry *xe = &xs->header->xh_entries[i]; + + if (ocfs2_xattr_get_type(xe) != xi->xi_name_index) + continue; + if (xe->xe_name_len != xi->xi_name_len) + continue; + if (!memcmp(xs->base + le16_to_cpu(xe->xe_name_offset), + xi->xi_name, xi->xi_name_len)) { + xs->here = xe; + return; + } + } + + /* + * The entry did not survive the failed set, so this has become an + * insert. Point xs->here back inside the bucket: it still refers to + * the block being converted, which is about to be rewritten as a tree + * root. + */ + xs->not_found = -ENODATA; + xs->here = xs->header->xh_entries; } static int ocfs2_xattr_create_index_block(struct inode *inode, + struct ocfs2_xattr_info *xi, struct ocfs2_xattr_search *xs, struct ocfs2_xattr_set_ctxt *ctxt) { @@ -4665,7 +4714,7 @@ static int ocfs2_xattr_create_index_block(struct inode *inode, ocfs2_cp_xattr_block_to_bucket(inode, xb_bh, xs->bucket); ocfs2_xattr_bucket_journal_dirty(handle, xs->bucket); - ocfs2_xattr_update_xattr_search(inode, xs, xb_bh); + ocfs2_xattr_update_xattr_search(inode, xi, xs); /* Change from ocfs2_xattr_header to ocfs2_xattr_tree_root */ memset(&xb->xb_attrs, 0, inode->i_sb->s_blocksize - @@ -5018,7 +5067,15 @@ static int ocfs2_divide_xattr_bucket(struct inode *inode, start = ocfs2_xattr_find_divide_pos(xh); if (start == count) { - xe = &xh->xh_entries[start-1]; + /* + * start == count means every entry shares one hash, so the + * first of them carries it too. An empty bucket has no + * entries[start - 1], but its entries[0] still carries the + * bucket key: either the hash ocfs2_xa_remove_entry() + * deliberately leaves behind, or the sentinel a previous + * divide wrote. + */ + xe = &xh->xh_entries[0]; /* * initialized a new empty bucket here. @@ -5848,11 +5905,20 @@ static int ocfs2_check_xattr_bucket_collision(struct ocfs2_xattr_bucket *bucket, u32 name_hash) { struct ocfs2_xattr_header *xh = bucket_xh(bucket); + u16 count = le16_to_cpu(xh->xh_count); + + /* + * An empty bucket keeps the hash of the entry that was removed in + * xh_entries[0] so that it can still be found, but it holds no + * entries to collide with. + */ + if (!count) + return 0; if (name_hash != le32_to_cpu(xh->xh_entries[0].xe_name_hash)) return 0; - if (xh->xh_entries[le16_to_cpu(xh->xh_count) - 1].xe_name_hash == + if (xh->xh_entries[count - 1].xe_name_hash == xh->xh_entries[0].xe_name_hash) { mlog(ML_ERROR, "Too much hash collision in xattr bucket %llu, " "hash = %u\n", @@ -5873,13 +5939,18 @@ static int ocfs2_xattr_set_entry_bucket(struct inode *inode, struct ocfs2_xattr_search *xs, struct ocfs2_xattr_set_ctxt *ctxt) { - int ret; + int ret, found = 0; + u16 xe_index; + u32 name_hash; struct ocfs2_xa_loc loc; trace_ocfs2_xattr_set_entry_bucket(xi->xi_name); ocfs2_init_xattr_bucket_xa_loc(&loc, xs->bucket, xs->not_found ? NULL : xs->here); + /* Defragmenting below may move this entry, so remember its hash. */ + name_hash = loc.xl_entry ? le32_to_cpu(loc.xl_entry->xe_name_hash) : 0; + ret = ocfs2_xa_set(&loc, xi, ctxt); if (!ret) { xs->here = loc.xl_entry; @@ -5898,6 +5969,29 @@ static int ocfs2_xattr_set_entry_bucket(struct inode *inode, goto out; } + /* + * ocfs2_defrag_xattr_bucket() sorts the entries by name hash and + * lib/sort.c is not stable, so entries sharing a hash may have traded + * slots. Look the entry up again before writing through it. + */ + if (loc.xl_entry) { + ret = ocfs2_find_xe_in_bucket(inode, xs->bucket, + xi->xi_name_index, xi->xi_name, + name_hash, &xe_index, &found); + if (ret) { + mlog_errno(ret); + goto out; + } + if (!found) { + mlog(ML_ERROR, "xattr entry not found in bucket %llu after defrag\n", + (unsigned long long)bucket_blkno(xs->bucket)); + ret = -EFSCORRUPTED; + goto out; + } + loc.xl_entry = &bucket_xh(xs->bucket)->xh_entries[xe_index]; + xs->here = loc.xl_entry; + } + ret = ocfs2_xa_set(&loc, xi, ctxt); if (!ret) { xs->here = loc.xl_entry; diff --git a/fs/open.c b/fs/open.c index 6b1c14e684a9..d7d9fcc666ab 100644 --- a/fs/open.c +++ b/fs/open.c @@ -1147,6 +1147,7 @@ struct file *dentry_open_nonotify(const struct path *path, int flags, } return f; } +EXPORT_SYMBOL_GPL(dentry_open_nonotify); /** * kernel_file_open - open a file for kernel internal use diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c index f716ac13badf..c77af9a79d88 100644 --- a/fs/overlayfs/dir.c +++ b/fs/overlayfs/dir.c @@ -74,39 +74,34 @@ static struct dentry *ovl_start_creating_temp(struct ovl_fs *ofs, &QSTR(name)); } +/* + * The whiteout is the source of a rename, so it has to be hashed. + * ovl_create_temp() takes care of that for a casefolded workdir. + */ static struct dentry *ovl_whiteout(struct ovl_fs *ofs) { int err; struct dentry *whiteout, *link; struct dentry *workdir = ofs->workdir; - struct inode *wdir = workdir->d_inode; - char name[OVL_TEMPNAME_SIZE]; guard(mutex)(&ofs->whiteout_lock); if (!ofs->whiteout) { - whiteout = ovl_start_creating_temp(ofs, workdir, name); + whiteout = ovl_create_temp(ofs, workdir, &(struct ovl_cattr) { + .mode = S_IFCHR | WHITEOUT_MODE, + .rdev = WHITEOUT_DEV }); if (IS_ERR(whiteout)) return whiteout; - err = ovl_do_whiteout(ofs, wdir, whiteout); - if (!err) - ofs->whiteout = dget(whiteout); - end_creating(whiteout); - if (err) - return ERR_PTR(err); + ofs->whiteout = whiteout; } if (!ofs->no_shared_whiteout) { - link = ovl_start_creating_temp(ofs, workdir, name); - if (IS_ERR(link)) + link = ovl_create_temp(ofs, workdir, &(struct ovl_cattr) { + .hardlink = ofs->whiteout }); + if (!IS_ERR(link)) return link; - err = ovl_do_link(ofs, ofs->whiteout, wdir, link); - if (!err) - whiteout = dget(link); - end_creating(link); - if (!err) - return whiteout; + err = PTR_ERR(link); if (err != -EMLINK) { pr_warn("Failed to link whiteout - disabling whiteout inode sharing(nlink=%u, err=%u)\n", ofs->whiteout->d_inode->i_nlink, @@ -120,9 +115,10 @@ static struct dentry *ovl_whiteout(struct ovl_fs *ofs) } int ovl_cleanup_and_whiteout(struct ovl_fs *ofs, struct dentry *dir, - struct dentry *dentry) + struct dentry *dentry, const struct qstr *name) { struct dentry *whiteout; + struct qstr last = QSTR_LEN(name->name, name->len); struct renamedata rd = {}; int err; int flags = 0; @@ -139,9 +135,17 @@ int ovl_cleanup_and_whiteout(struct ovl_fs *ofs, struct dentry *dir, rd.old_parent = ofs->workdir; rd.new_parent = dir; rd.flags = flags; - err = start_renaming_two_dentries(&rd, whiteout, dentry); + /* + * The caller looked up @dentry by @name without a lock on @dir, and a + * casefolded directory leaves a negative dentry unhashed. Nothing tells + * if such a dentry is still valid, so look up @name again with @dir locked. + */ + err = start_renaming_dentry(&rd, 0, whiteout, &last); if (!err) { - err = ovl_do_rename_rd(&rd); + if (d_inode(rd.new_dentry) != d_inode(dentry)) + err = -ESTALE; + else + err = ovl_do_rename_rd(&rd); end_renaming(&rd); } if (err) @@ -847,7 +851,7 @@ static int ovl_remove_and_whiteout(struct dentry *dentry, goto out_dput_upper; } - err = ovl_cleanup_and_whiteout(ofs, upperdir, upper); + err = ovl_cleanup_and_whiteout(ofs, upperdir, upper, &dentry->d_name); if (!err) ovl_dir_modified(dentry->d_parent, true); diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c index 9aae20c59608..bd998a3d8b95 100644 --- a/fs/overlayfs/export.c +++ b/fs/overlayfs/export.c @@ -713,9 +713,9 @@ static struct dentry *ovl_lower_fh_to_d(struct super_block *sb, } } - /* Then lookup indexed upper/whiteout by origin fh */ + /* Then lookup indexed upper/whiteout by origin */ if (ovl_indexdir(sb)) { - index = ovl_get_index_fh(ofs, fh); + index = ovl_get_index_origin(ofs, origin.dentry); err = PTR_ERR(index); if (IS_ERR(index)) { index = NULL; diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c index 7e1ac70c5d97..e3aa0931bea9 100644 --- a/fs/overlayfs/inode.c +++ b/fs/overlayfs/inode.c @@ -658,6 +658,14 @@ int ovl_fileattr_set(struct mnt_idmap *idmap, unsigned int flags; int err; + /* + * Lookup refuses a directory that does not agree with the casefold + * state of the mount. Only FS_IOC_SETFLAGS can ask for a change. + */ + if (S_ISDIR(inode->i_mode) && fa->flags_valid && + !!(fa->flags & FS_CASEFOLD_FL) != OVL_FS(inode->i_sb)->casefold) + return -EOPNOTSUPP; + err = ovl_copy_up(dentry); if (!err) { ovl_path_real(dentry, &upperpath); diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c index c6127edb9be9..64d63b260709 100644 --- a/fs/overlayfs/namei.c +++ b/fs/overlayfs/namei.c @@ -781,14 +781,21 @@ int ovl_get_index_name(struct ovl_fs *ofs, struct dentry *origin, return err; } -/* Lookup index by file handle for NFS export */ -struct dentry *ovl_get_index_fh(struct ovl_fs *ofs, struct ovl_fh *fh) +/* + * Lookup index by decoded origin for NFS export. + * + * The name of an index is the hex of the file handle that copy up encoded. + * The bytes of the handle to decode cannot be used as the name, because a + * lower fs may decode many handles to the same inode. For example, it may + * ignore the fid type, extra fid bytes or a zero generation. + */ +struct dentry *ovl_get_index_origin(struct ovl_fs *ofs, struct dentry *origin) { struct dentry *index; struct qstr name; int err; - err = ovl_get_index_name_fh(fh, &name); + err = ovl_get_index_name(ofs, origin, &name); if (err) return ERR_PTR(err); diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h index 91cdd7e15696..a5aa22cc79a8 100644 --- a/fs/overlayfs/overlayfs.h +++ b/fs/overlayfs/overlayfs.h @@ -394,14 +394,6 @@ static inline int ovl_do_rename(struct ovl_fs *ofs, struct dentry *olddir, return ovl_do_rename_rd(&rd); } -static inline int ovl_do_whiteout(struct ovl_fs *ofs, - struct inode *dir, struct dentry *dentry) -{ - int err = vfs_whiteout(ovl_upper_mnt_idmap(ofs), dir, dentry); - pr_debug("whiteout(%pd2) = %i\n", dentry, err); - return err; -} - static inline struct file *ovl_do_tmpfile(struct ovl_fs *ofs, struct dentry *dentry, umode_t mode) { @@ -537,7 +529,14 @@ void ovl_dir_modified(struct dentry *dentry, bool impurity); u64 ovl_inode_version_get(struct inode *inode); bool ovl_is_whiteout(struct dentry *dentry); bool ovl_path_is_whiteout(struct ovl_fs *ofs, const struct path *path); -struct file *ovl_path_open(const struct path *path, int flags); +struct file *ovl_path_open_realfile(const struct file *user_file, + const struct path *path, int flags); + +static inline struct file *ovl_path_open(const struct path *path, int flags) +{ + return ovl_path_open_realfile(NULL, path, flags); +} + int ovl_copy_up_start(struct dentry *dentry, int flags); void ovl_copy_up_end(struct dentry *dentry); bool ovl_already_copied_up(struct dentry *dentry, int flags); @@ -749,7 +748,7 @@ int ovl_verify_index(struct ovl_fs *ofs, struct dentry *index); int ovl_get_index_name_fh(const struct ovl_fh *fh, struct qstr *name); int ovl_get_index_name(struct ovl_fs *ofs, struct dentry *origin, struct qstr *name); -struct dentry *ovl_get_index_fh(struct ovl_fs *ofs, struct ovl_fh *fh); +struct dentry *ovl_get_index_origin(struct ovl_fs *ofs, struct dentry *origin); struct dentry *ovl_lookup_index(struct ovl_fs *ofs, struct dentry *upper, struct dentry *origin, bool verify); int ovl_path_next(int idx, struct dentry *dentry, struct path *path, @@ -897,7 +896,7 @@ static inline void ovl_copyflags(struct inode *from, struct inode *to) /* dir.c */ extern const struct inode_operations ovl_dir_inode_operations; int ovl_cleanup_and_whiteout(struct ovl_fs *ofs, struct dentry *dir, - struct dentry *dentry); + struct dentry *dentry, const struct qstr *name); struct ovl_cattr { dev_t rdev; umode_t mode; diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c index 80366424b3dc..e23331212a29 100644 --- a/fs/overlayfs/readdir.c +++ b/fs/overlayfs/readdir.c @@ -973,7 +973,8 @@ static struct file *ovl_dir_open_realfile(const struct file *file, const struct path *realpath) { with_ovl_creds(file_inode(file)->i_sb) - return ovl_path_open(realpath, O_RDONLY | (file->f_flags & O_LARGEFILE)); + return ovl_path_open_realfile(file, realpath, + O_RDONLY | (file->f_flags & O_LARGEFILE)); } /* @@ -1322,7 +1323,8 @@ int ovl_indexdir_cleanup(struct ovl_fs *ofs) * Whiteout orphan index to block future open by * handle after overlay nlink dropped to zero. */ - err = ovl_cleanup_and_whiteout(ofs, indexdir, index); + err = ovl_cleanup_and_whiteout(ofs, indexdir, index, + &QSTR_LEN(p->name, p->len)); } else { /* Cleanup orphan index entries */ err = ovl_cleanup(ofs, indexdir, index); diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c index 861b0775f9dd..c06bfe9765e2 100644 --- a/fs/overlayfs/super.c +++ b/fs/overlayfs/super.c @@ -1366,7 +1366,6 @@ static struct dentry *ovl_get_root(struct super_block *sb, ovl_dentry_set_flag(OVL_E_CONNECTED, root); ovl_set_upperdata(d_inode(root)); ovl_inode_init(d_inode(root), &oip, ino, fsid); - WARN_ON(!!IS_CASEFOLDED(d_inode(root)) != ofs->casefold); ovl_dentry_init_flags(root, upperdentry, oe, DCACHE_OP_WEAK_REVALIDATE); /* root keeps a reference of upperdentry */ dget(upperdentry); @@ -1550,6 +1549,17 @@ static int ovl_fill_super_creds(struct fs_context *fc, struct super_block *sb) if (!sb->s_root) goto out_free_oe; + /* + * The casefold flag of an empty layer root may have changed since the + * mount options were parsed. The root inode owns oe by now. + */ + if (!!IS_CASEFOLDED(d_inode(sb->s_root)) != ofs->casefold) { + pr_err("casefold flag of the root directory has changed\n"); + dput(sb->s_root); + sb->s_root = NULL; + return -EINVAL; + } + return 0; out_free_oe: diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c index b41f4788e4f0..04103a6741ba 100644 --- a/fs/overlayfs/util.c +++ b/fs/overlayfs/util.c @@ -654,7 +654,8 @@ bool ovl_path_is_whiteout(struct ovl_fs *ofs, const struct path *path) ovl_path_check_xwhiteout_xattr(ofs, path); } -struct file *ovl_path_open(const struct path *path, int flags) +struct file *ovl_path_open_realfile(const struct file *user_file, + const struct path *path, int flags) { struct inode *inode = d_inode(path->dentry); struct mnt_idmap *real_idmap = mnt_idmap(path->mnt); @@ -682,6 +683,14 @@ struct file *ovl_path_open(const struct path *path, int flags) if (inode_owner_or_capable(real_idmap, inode)) flags |= O_NOATIME; + /* + * Directory opens use dentry_open() here, not backing_file_open(). + * Inherit FMODE_NONOTIFY from a fanotify permission fd so a listener + * that also watches the real filesystem does not deadlock. + */ + if (user_file && FMODE_FSNOTIFY_NONE(user_file->f_mode)) + return dentry_open_nonotify(path, flags, current_cred()); + return dentry_open(path, flags, current_cred()); } @@ -1118,7 +1127,7 @@ static void ovl_cleanup_index(struct dentry *dentry) } else if (ovl_index_all(dentry->d_sb)) { /* Whiteout orphan index to block future open by handle */ err = ovl_cleanup_and_whiteout(OVL_FS(dentry->d_sb), - indexdir, index); + indexdir, index, &name); } else { /* Cleanup orphan index entries */ err = ovl_cleanup(ofs, indexdir, index); diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index a2db957aa799..2eaa79ab1138 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -3314,7 +3314,7 @@ static long do_pagemap_cmd(struct file *file, unsigned int cmd, return do_pagemap_scan(mm, arg); default: - return -EINVAL; + return -ENOIOCTLCMD; } } diff --git a/fs/smb/server/mgmt/user_session.c b/fs/smb/server/mgmt/user_session.c index 1b30e432ebfd..d131809dd08a 100644 --- a/fs/smb/server/mgmt/user_session.c +++ b/fs/smb/server/mgmt/user_session.c @@ -89,6 +89,8 @@ static const char *session_state_string(struct ksmbd_session *session) static const char *session_user_name(struct ksmbd_session *session) { + if (!session->user) + return "(Unauthenticated)"; if (user_guest(session->user)) return "(Guest)"; else if (ksmbd_anonymous_user(session->user)) @@ -98,6 +100,8 @@ static const char *session_user_name(struct ksmbd_session *session) static const char *session_account_type(struct ksmbd_session *session) { + if (!session->user) + return "unauthenticated"; if (user_guest(session->user)) return "guest"; if (ksmbd_anonymous_user(session->user)) diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c index 3f1ab9a20307..1c8037c85de9 100644 --- a/fs/smb/server/smb2pdu.c +++ b/fs/smb/server/smb2pdu.c @@ -3097,9 +3097,10 @@ int smb2_tree_disconnect(struct ksmbd_work *work) goto err_out; } + err = ksmbd_tree_conn_disconnect(sess, tcon); + ksmbd_close_tree_conn_fds(work); - err = ksmbd_tree_conn_disconnect(sess, tcon); if (err) { rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED; goto err_out; @@ -3452,7 +3453,8 @@ static noinline int smb2_set_stream_name_xattr(const struct path *path, if (fp->cdoption == FILE_OPEN_LE) { if (!strcmp(stream_name, "AFP_AfpInfo") && test_share_config_flag(fp->tcon->share_conf, - KSMBD_SHARE_FLAG_TIME_MACHINE)) { + KSMBD_SHARE_FLAG_TIME_MACHINE) && + test_tree_conn_flag(fp->tcon, KSMBD_TREE_CONN_FLAG_WRITABLE)) { /* * Synthesize an empty AFP_AfpInfo xattr on first access. * type=0/creator=0 tells macOS to use the file extension @@ -3478,6 +3480,9 @@ static noinline int smb2_set_stream_name_xattr(const struct path *path, return -EBADF; } + if (!test_tree_conn_flag(fp->tcon, KSMBD_TREE_CONN_FLAG_WRITABLE)) + return -EACCES; + rc = ksmbd_vfs_setxattr(idmap, path, xattr_stream_name, NULL, 0, 0, false); if (rc < 0) pr_err("Failed to store XATTR stream name :%d\n", rc); @@ -4427,7 +4432,7 @@ int smb2_open(struct ksmbd_work *work) file_info = FILE_OPENED; rc = ksmbd_vfs_getattr(&fp->filp->f_path, &stat); if (rc) - goto err_out2; + goto err_out1; goto reconnected_fp; } @@ -4439,8 +4444,12 @@ int smb2_open(struct ksmbd_work *work) goto err_out2; rc = ksmbd_reopen_durable_fd(work, dh_info.fp); - if (rc) + if (rc) { + if (rc == -ENOTCONN) + rsp->hdr.Status = + STATUS_NETWORK_NAME_DELETED; goto err_out2; + } fp = dh_info.fp; @@ -4453,7 +4462,7 @@ int smb2_open(struct ksmbd_work *work) rc = ksmbd_vfs_getattr(&fp->filp->f_path, &stat); if (rc) - goto err_out2; + goto err_out1; goto reconnected_fp; } @@ -4503,6 +4512,14 @@ int smb2_open(struct ksmbd_work *work) goto err_out2; } } + + /* MS-FSA 2.1.5.1: delete-on-close requires DELETE access */ + if (req->CreateOptions & FILE_DELETE_ON_CLOSE_LE && + !(smb_map_generic_desired_access(req->DesiredAccess) & + FILE_DELETE_LE)) { + rc = -EINVAL; + goto err_out2; + } } if (le32_to_cpu(req->CreateDisposition) > @@ -4772,6 +4789,13 @@ int smb2_open(struct ksmbd_work *work) maximal_access = daccess; } + /* MAXIMUM_ALLOWED may not have granted the requested DELETE */ + if (req->CreateOptions & FILE_DELETE_ON_CLOSE_LE && + !(daccess & FILE_DELETE_LE)) { + rc = -EACCES; + goto err_out; + } + open_flags = smb2_create_open_flags(file_present, daccess, req->CreateDisposition, &may_flags, @@ -4874,6 +4898,8 @@ int smb2_open(struct ksmbd_work *work) if (IS_ERR(fp)) { fput(filp); rc = PTR_ERR(fp); + if (rc == -ENOTCONN) + rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED; fp = NULL; goto err_out; } @@ -5477,11 +5503,20 @@ int smb2_open(struct ksmbd_work *work) err_out2: if (!rc) { - if (!dh_info.replay) + if (!dh_info.replay) { + smb2_complete_request_open(work); rc = ksmbd_update_fstate(&work->sess->file_table, fp, FP_INITED); - if (!rc) + if (rc == -ENOENT) + rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED; + if (!rc) { + work->request_open = fp; + work->request_open_chseq = + smb3_hdr_channel_sequence(&req->hdr); + } + } else { rc = smb2_set_request_open(work, fp, &req->hdr, false, false); + } if (!rc) rc = ksmbd_iov_pin_rsp(work, (void *)rsp, iov_len); } @@ -5496,8 +5531,7 @@ int smb2_open(struct ksmbd_work *work) rsp->hdr.Status = STATUS_PRIVILEGE_NOT_HELD; else rsp->hdr.Status = STATUS_ACCESS_DENIED; - } - else if (rc == -ENOENT) + } else if (rc == -ENOENT && !rsp->hdr.Status) rsp->hdr.Status = STATUS_OBJECT_NAME_INVALID; else if (rc == -EPERM) rsp->hdr.Status = STATUS_SHARING_VIOLATION; @@ -7082,17 +7116,8 @@ static int get_file_stream_info(struct ksmbd_work *work, streamlen *= 2; kfree(stream_buf); file_info->StreamNameLength = cpu_to_le32(streamlen); - /* - * stream_name_len is the byte length of the xattr's *name*, - * not its value -- same class of bug ksmbd_stream_eof() - * (smb2pdu.c) already fixes for EndOfFile/AllocationSize on - * a stream handle; this enumeration path needs the same - * real xattr value length, not the name length reused as a - * size. - */ - slen = ksmbd_vfs_casexattr_len(file_mnt_idmap(fp->filp), - path->dentry, stream_name, - strlen(stream_name) + 1, NULL); + slen = ksmbd_vfs_xattr_len(file_mnt_idmap(fp->filp), + path->dentry, stream_name); ssize = slen < 0 ? 0 : (loff_t)slen; file_info->StreamSize = cpu_to_le64(ssize); file_info->StreamAllocationSize = cpu_to_le64(ssize); diff --git a/fs/smb/server/smbacl.c b/fs/smb/server/smbacl.c index f8bd8e2ed33e..cc3a450e4cb2 100644 --- a/fs/smb/server/smbacl.c +++ b/fs/smb/server/smbacl.c @@ -1470,6 +1470,7 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path, int i, found = 0; unsigned int access_bits = 0, denied = 0; struct smb_ace *others_ace = NULL; + bool or_ace = false; struct posix_acl_entry *pa_entry; unsigned int sid_type = SIDOWNER; unsigned short ace_size; @@ -1502,6 +1503,8 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path, is_owner = uid == from_kuid(&init_user_ns, vfsuid_into_kuid(vfsuid)); if (!pdacl->num_aces) { + unsigned int allowed = 0; + /* * An empty (present, zero-ACE) DACL grants no access to * anyone except the object owner's implicit READ_CONTROL @@ -1510,9 +1513,14 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path, * not let trailing bytes after a zero-ACE DACL become an * implicit grant. */ - if (!is_owner || - (*pdaccess & ~(FILE_READ_CONTROL_LE | FILE_WRITE_DAC_LE))) + if (is_owner) + allowed = READ_CONTROL | WRITE_DAC; + if (*pdaccess & FILE_MAXIMAL_ACCESS_LE) + requested |= allowed; + if (!allowed || requested & ~allowed) rc = -EACCES; + else + *pdaccess = cpu_to_le32(requested); goto err_out; } @@ -1597,6 +1605,9 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path, found = 1; break; } + if (!compare_sids(&sid_owner_rights, &ace->sid) && + !(ace->flags & INHERIT_ONLY_ACE)) + or_ace = true; if (!compare_sids(&sid_everyone, &ace->sid) || !compare_sids(&sid_authusers, &ace->sid)) others_ace = ace; @@ -1618,7 +1629,10 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path, else continue; - if (id == uid) { + /* A group entry grants only to the group's members */ + if (id == uid && + (pa_entry->e_tag == ACL_USER || + in_group_p(make_kgid(&init_user_ns, id)))) { mode_to_access_flags(pa_entry->e_perm, 0777, &access_bits); @@ -1638,8 +1652,23 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path, if (others_ace) { ace = others_ace; } else { - ksmbd_debug(SMB, "Can't find corresponding sid\n"); - rc = -EACCES; + unsigned int allowed = 0; + + /* + * No ACE names the owner, who may still read and + * change the DACL unless OWNER RIGHTS applies + * (MS-DTYP 2.5.3.2). + */ + if (is_owner && !or_ace) + allowed = READ_CONTROL | WRITE_DAC; + if (*pdaccess & FILE_MAXIMAL_ACCESS_LE) + requested |= allowed; + if (!allowed || requested & ~allowed) { + ksmbd_debug(SMB, "Can't find corresponding sid\n"); + rc = -EACCES; + goto err_out; + } + *pdaccess = cpu_to_le32(requested); goto err_out; } } @@ -1665,8 +1694,8 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path, } if (granted & ~access_bits) { - ksmbd_debug(SMB, "Access denied with winACL, granted : %x, access_req : %x\n", - granted, le32_to_cpu(ace->access_req)); + ksmbd_debug(SMB, "Access denied with winACL, requested : %x, allowed : %x\n", + granted, access_bits); rc = -EACCES; goto err_out; } diff --git a/fs/smb/server/vfs.c b/fs/smb/server/vfs.c index 7453b6e72bd4..674e4fcd80cb 100644 --- a/fs/smb/server/vfs.c +++ b/fs/smb/server/vfs.c @@ -902,8 +902,8 @@ ssize_t ksmbd_vfs_listxattr(struct dentry *dentry, char **list) return size; } -static ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap, - struct dentry *dentry, char *xattr_name) +ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap, + struct dentry *dentry, char *xattr_name) { return vfs_getxattr(idmap, dentry, xattr_name, NULL, 0); } diff --git a/fs/smb/server/vfs.h b/fs/smb/server/vfs.h index 53044549571a..7cdd49678667 100644 --- a/fs/smb/server/vfs.h +++ b/fs/smb/server/vfs.h @@ -109,6 +109,8 @@ ssize_t ksmbd_vfs_getxattr(struct mnt_idmap *idmap, struct dentry *dentry, char *xattr_name, char **xattr_buf); +ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap, + struct dentry *dentry, char *xattr_name); ssize_t ksmbd_vfs_casexattr_len(struct mnt_idmap *idmap, struct dentry *dentry, char *attr_name, int attr_name_len, char *actual_name); diff --git a/fs/smb/server/vfs_cache.c b/fs/smb/server/vfs_cache.c index 1efa9f814560..38196befa03a 100644 --- a/fs/smb/server/vfs_cache.c +++ b/fs/smb/server/vfs_cache.c @@ -698,16 +698,18 @@ static void __ksmbd_close_fd(struct ksmbd_file_table *ft, struct ksmbd_file *fp) } /* - * Drop fp's strong reference on conn (taken in ksmbd_open_fd() / - * ksmbd_reopen_durable_fd()). Durable fps that reached the - * scavenger have already had fp->conn cleared by session_fd_check(), - * in which case there is nothing to drop here. + * Drop fp's strong connection references. Durable fps that reached + * the scavenger have already had these cleared by session_fd_check(). */ if (fp->conn) { atomic_dec(&fp->conn->stats.open_files_count); ksmbd_conn_put(fp->conn); fp->conn = NULL; } + if (fp->tcon) { + ksmbd_tree_connect_put(fp->tcon); + fp->tcon = NULL; + } if (ksmbd_stream_fd(fp)) kfree(fp->stream.name); @@ -1331,14 +1333,14 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp) fp->filp = filp; /* - * fp owns a strong reference on fp->conn for as long as fp->conn is - * non-NULL, so session_fd_check() and __ksmbd_close_fd() never - * dereference a dangling pointer. Paired with ksmbd_conn_put() in - * session_fd_check() (durable preserve), in __ksmbd_close_fd() - * (final close), and on the error paths below. + * fp owns strong references on fp->conn and fp->tcon for as long as + * they are non-NULL. Paired with the puts in session_fd_check() + * (durable preserve), in __ksmbd_close_fd() (final close), and on the + * error paths below. */ fp->conn = ksmbd_conn_get(work->conn); fp->tcon = work->tcon; + atomic_inc(&fp->tcon->refcount); fp->share_conf = work->tcon->share_conf; atomic_inc(&fp->share_conf->refcount); fp->volatile_id = KSMBD_NO_FID; @@ -1351,7 +1353,13 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp) goto err_out; } - ret = __open_id(&work->sess->file_table, fp, OPEN_ID_TYPE_VOLATILE_ID); + down_read(&work->sess->tree_conns_lock); + if (work->tcon->t_state == TREE_CONNECTED) + ret = __open_id(&work->sess->file_table, fp, + OPEN_ID_TYPE_VOLATILE_ID); + else + ret = -ENOTCONN; + up_read(&work->sess->tree_conns_lock); if (ret) { ksmbd_inode_put(fp->f_ci); goto err_out; @@ -1361,7 +1369,8 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp) return fp; err_out: - /* fp->conn was set and refcounted before every branch here. */ + /* fp->conn and fp->tcon were refcounted before every branch here. */ + ksmbd_tree_connect_put(fp->tcon); ksmbd_conn_put(fp->conn); ksmbd_share_config_put(fp->share_conf); kmem_cache_free(filp_cache, fp); @@ -1376,8 +1385,9 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp) * * Return: 0 on success. The FP_NEW -> FP_INITED transition is special: * -ENOENT if teardown already unpublished @fp by advancing the state or - * clearing the volatile id. Other state updates preserve the historical - * fire-and-forget behavior. + * clearing the volatile id. On success it also takes a reference for the + * opening request, which must be released with ksmbd_fd_put(). Other state + * updates preserve the historical fire-and-forget behavior. */ int ksmbd_update_fstate(struct ksmbd_file_table *ft, struct ksmbd_file *fp, unsigned int state) @@ -1393,6 +1403,8 @@ int ksmbd_update_fstate(struct ksmbd_file_table *ft, struct ksmbd_file *fp, ret = -ENOENT; } else { fp->f_state = state; + if (state == FP_INITED) + atomic_inc(&fp->refcount); ret = 0; } write_unlock(&ft->lock); @@ -1813,6 +1825,7 @@ static bool session_fd_check(struct ksmbd_tree_connect *tcon, struct oplock_info *op; struct ksmbd_conn *conn; struct ksmbd_lock *smb_lock, *tmp_lock; + struct ksmbd_tree_connect *tree_conn; if (!is_reconnectable(fp)) return false; @@ -1834,6 +1847,7 @@ static bool session_fd_check(struct ksmbd_tree_connect *tcon, ci = fp->f_ci; down_write(&ci->m_lock); conn = fp->conn; + tree_conn = fp->tcon; list_for_each_entry_rcu(op, &ci->m_op_list, op_entry, lockdep_is_held(&ci->m_lock)) { if (op->o_fp != fp) @@ -1869,6 +1883,7 @@ static bool session_fd_check(struct ksmbd_tree_connect *tcon, /* Drop fp's own reference on conn. */ ksmbd_conn_put(conn); + ksmbd_tree_connect_put(tree_conn); return true; } @@ -1972,7 +1987,8 @@ int ksmbd_reopen_durable_fd(struct ksmbd_work *work, struct ksmbd_file *fp) * concurrent teardown that iterates the table can observe a valid * volatile_id with fp->conn == NULL and preserve a * partially-initialized fp. fp owns a strong reference on the new - * conn (see ksmbd_open_fd()); undo it on __open_id() failure. + * conn (see ksmbd_open_fd()); undo it on __open_id() failure. Serialize + * publication with tree disconnect just as ksmbd_open_fd() does. */ spin_lock(&fp->f_lock); old_f_state = fp->f_state; @@ -1982,12 +1998,19 @@ int ksmbd_reopen_durable_fd(struct ksmbd_work *work, struct ksmbd_file *fp) atomic_inc(&fp->share_conf->refcount); fp->conn = ksmbd_conn_get(conn); fp->tcon = work->tcon; + atomic_inc(&fp->tcon->refcount); spin_unlock(&fp->f_lock); atomic_inc(&conn->stats.open_files_count); write_unlock(&global_ft.lock); - __open_id(&work->sess->file_table, fp, OPEN_ID_TYPE_VOLATILE_ID); - if (!has_file_id(fp->volatile_id)) { + down_read(&work->sess->tree_conns_lock); + if (work->tcon->t_state == TREE_CONNECTED) + ret = __open_id(&work->sess->file_table, fp, + OPEN_ID_TYPE_VOLATILE_ID); + else + ret = -ENOTCONN; + up_read(&work->sess->tree_conns_lock); + if (ret) { write_lock(&global_ft.lock); spin_lock(&fp->f_lock); fp->conn = NULL; @@ -1998,7 +2021,10 @@ int ksmbd_reopen_durable_fd(struct ksmbd_work *work, struct ksmbd_file *fp) atomic_dec(&conn->stats.open_files_count); write_unlock(&global_ft.lock); ksmbd_conn_put(conn); + ksmbd_tree_connect_put(work->tcon); ksmbd_share_config_put(work->tcon->share_conf); + if (ret != -ENOTCONN) + ret = -EBADF; goto out; } diff --git a/fs/udf/balloc.c b/fs/udf/balloc.c index 30cec5600149..75124188e745 100644 --- a/fs/udf/balloc.c +++ b/fs/udf/balloc.c @@ -457,6 +457,7 @@ static void udf_table_free_blocks(struct super_block *sb, int adsize; + eloc.partitionReferenceNum = bloc->partitionReferenceNum; eloc.logicalBlockNum = start; elen = EXT_RECORDED_ALLOCATED | (count << sb->s_blocksize_bits); @@ -532,7 +533,7 @@ static int udf_table_prealloc_blocks(struct super_block *sb, eloc.logicalBlockNum, elen, first_block); } - if (first_block == eloc.logicalBlockNum) { + if (ret > 0 && first_block == eloc.logicalBlockNum) { epos.offset -= adsize; alloc_count = (elen >> sb->s_blocksize_bits); @@ -572,7 +573,7 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb, uint32_t elen, goal_elen = 0; struct kernel_lb_addr eloc, goal_eloc; struct extent_position epos, goal_epos; - int8_t etype; + int8_t etype, goal_etype = 0; struct udf_inode_info *iinfo = UDF_I(table); int ret = 0; @@ -623,7 +624,8 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb, goal_epos.block = epos.block; goal_epos.offset = epos.offset - adsize; goal_eloc = eloc; - goal_elen = (etype << 30) | elen; + goal_elen = elen; + goal_etype = etype; } } @@ -647,7 +649,8 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb, goal_elen -= sb->s_blocksize; if (goal_elen) - udf_write_aext(table, &goal_epos, &goal_eloc, goal_elen, 1); + udf_write_aext(table, &goal_epos, &goal_eloc, + (goal_etype << 30) | goal_elen, 1); else udf_delete_aext(table, goal_epos, &freed); brelse(goal_epos.bh); diff --git a/fs/udf/super.c b/fs/udf/super.c index 5351755aca3e..7570506ff2dd 100644 --- a/fs/udf/super.c +++ b/fs/udf/super.c @@ -1106,6 +1106,73 @@ static int check_partition_desc(struct super_block *sb, return 0; } +/* + * Check the Unallocated Space Table once when it is loaded: the allocator + * hands out blocks from the start of each extent and trusts that every + * extent covers at least one whole block inside the partition. + */ +static int udf_check_unalloc_table(struct super_block *sb, + struct inode *table, u32 partition_len) +{ + struct extent_position epos = { + .block = UDF_I(table)->i_location, + .offset = sizeof(struct unallocSpaceEntry), + }; + struct kernel_lb_addr eloc; + uint32_t elen, blocks; + struct kernel_lb_addr seen_block = {}; + uint32_t seen_offset = 0; + u64 steps = 0, period = 1; + int8_t etype; + int ret; + + /* The walk below assumes the layout of an Unallocated Space Entry */ + if (!UDF_I(table)->i_use) { + udf_err(sb, "unallocated space table is not an unallocated space entry\n"); + return -EFSCORRUPTED; + } + + while ((ret = udf_next_aext(table, &epos, &eloc, &elen, &etype, 1)) > 0) { + blocks = elen >> sb->s_blocksize_bits; + if (!blocks || (elen & (sb->s_blocksize - 1)) || + eloc.logicalBlockNum >= partition_len || + blocks > partition_len - eloc.logicalBlockNum) { + udf_err(sb, "invalid unallocated space table extent (block %u, length %u)\n", + eloc.logicalBlockNum, elen); + ret = -EFSCORRUPTED; + break; + } + /* + * A chain of allocation extents can lead back to itself, and + * then the walk returns the same extents forever. Remember a + * position at doubling intervals; a walk that comes back to it + * is in a loop. + */ + if (epos.block.logicalBlockNum == seen_block.logicalBlockNum && + epos.block.partitionReferenceNum == + seen_block.partitionReferenceNum && + epos.offset == seen_offset) { + udf_err(sb, "unallocated space table loops back on itself\n"); + ret = -EFSCORRUPTED; + break; + } + if (++steps == period) { + seen_block = epos.block; + seen_offset = epos.offset; + steps = 0; + period <<= 1; + } + if (fatal_signal_pending(current)) { + udf_err(sb, "interrupted while checking the unallocated space table\n"); + ret = -EINTR; + break; + } + cond_resched(); + } + brelse(epos.bh); + return ret; +} + static int udf_fill_partdesc_info(struct super_block *sb, struct partitionDesc *p, int p_index) { @@ -1166,6 +1233,16 @@ static int udf_fill_partdesc_info(struct super_block *sb, p_index); return PTR_ERR(inode); } + err = udf_check_unalloc_table(sb, inode, map->s_partition_len); + if (err) { + iput(inode); + if (err == -EINTR) + return err; + if (!sb_rdonly(sb)) + return -EACCES; + UDF_SET_FLAG(sb, UDF_FLAG_RW_INCOMPAT); + return 0; + } map->s_uspace.s_table = inode; map->s_partition_flags |= UDF_PART_FLAG_UNALLOC_TABLE; udf_debug("unallocSpaceTable (part %d) @ %llu\n", @@ -2233,8 +2310,9 @@ static int udf_fill_super(struct super_block *sb, struct fs_context *fc) /* * EACCES is special - we want to propagate to * upper layers that we cannot handle RW mount. + * EINTR means that a fatal signal is pending. */ - if (ret == -EACCES) + if (ret == -EACCES || ret == -EINTR) break; } else break; diff --git a/fs/udf/truncate.c b/fs/udf/truncate.c index 0990f94b8551..75151a59f0da 100644 --- a/fs/udf/truncate.c +++ b/fs/udf/truncate.c @@ -124,8 +124,8 @@ void udf_discard_prealloc(struct inode *inode) { struct extent_position epos = {}; struct extent_position prev_epos = {}; - struct kernel_lb_addr eloc; - uint32_t elen; + struct kernel_lb_addr eloc, tmpeloc; + uint32_t elen, tmpelen; uint64_t lbcount = 0; int8_t etype = -1; struct udf_inode_info *iinfo = UDF_I(inode); @@ -141,7 +141,8 @@ void udf_discard_prealloc(struct inode *inode) /* Find the last extent in the file */ while (1) { - ret = udf_next_aext(inode, &epos, &eloc, &elen, &tmpetype, 0); + ret = udf_next_aext(inode, &epos, &tmpeloc, &tmpelen, + &tmpetype, 0); if (ret < 0) goto out; if (ret == 0) diff --git a/fs/xfs/libxfs/xfs_attr_leaf.c b/fs/xfs/libxfs/xfs_attr_leaf.c index 02ca50442b46..5b1fd6271420 100644 --- a/fs/xfs/libxfs/xfs_attr_leaf.c +++ b/fs/xfs/libxfs/xfs_attr_leaf.c @@ -3042,7 +3042,14 @@ xfs_attr3_leaf_setflag( #endif entry = &xfs_attr3_leaf_entryp(leaf)[args->index]; - ASSERT((entry->flags & XFS_ATTR_INCOMPLETE) == 0); + if (entry->flags & XFS_ATTR_INCOMPLETE) { + if (!(args->op_flags & XFS_DA_OP_RECOVERY)) { + ASSERT(args->op_flags & XFS_DA_OP_RECOVERY); + return -EFSCORRUPTED; + } + return 0; + } + entry->flags |= XFS_ATTR_INCOMPLETE; xfs_trans_log_buf(args->trans, bp, XFS_DA_LOGRANGE(leaf, entry, sizeof(*entry))); diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c index d64defeda645..fce2705c67d2 100644 --- a/fs/xfs/libxfs/xfs_bmap.c +++ b/fs/xfs/libxfs/xfs_bmap.c @@ -4128,6 +4128,8 @@ xfs_bmapi_finish( * Returns a negative error code on failure, including -ENOSPC when it could not * allocate any blocks and -ENOSR when it did allocate blocks to convert a * delalloc range, but those blocks were before the passed in range. + * With XFS_BMAPI_CONVERT_ONLY, it stops at a hole or a delalloc extent instead + * of allocating blocks, and returns -ENOSR if it stops before mapping anything. */ int xfs_bmapi_write( @@ -4187,6 +4189,8 @@ xfs_bmapi_write( */ ASSERT((flags & (XFS_BMAPI_PREALLOC | XFS_BMAPI_ZERO)) != (XFS_BMAPI_PREALLOC | XFS_BMAPI_ZERO)); + ASSERT(!(flags & XFS_BMAPI_CONVERT_ONLY) || + (flags & XFS_BMAPI_CONVERT)); if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) || XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) { @@ -4235,6 +4239,9 @@ xfs_bmapi_write( * that we found, if any. */ if (need_alloc || wasdelay) { + if (flags & XFS_BMAPI_CONVERT_ONLY) + break; + bma.eof = eof; bma.conv = !!(flags & XFS_BMAPI_CONVERT); bma.wasdel = wasdelay; @@ -4330,10 +4337,13 @@ xfs_bmapi_write( * * To avoid a successful return with *nmap set to 0, return the magic * -ENOSR error code for this particular case so that the caller can - * handle it. + * handle it. The same goes for XFS_BMAPI_CONVERT_ONLY stopping at a + * hole or a delalloc extent at the start of the range, in which case + * nothing has been changed. */ if (!n) { - ASSERT(bma.nallocs >= *nmap); + ASSERT((flags & XFS_BMAPI_CONVERT_ONLY) ? bma.nallocs == 0 : + bma.nallocs >= *nmap); return -ENOSR; } *nmap = n; diff --git a/fs/xfs/libxfs/xfs_bmap.h b/fs/xfs/libxfs/xfs_bmap.h index d5f2729305fa..76d25e5d5d9c 100644 --- a/fs/xfs/libxfs/xfs_bmap.h +++ b/fs/xfs/libxfs/xfs_bmap.h @@ -90,6 +90,13 @@ struct xfs_bmalloca { /* Try to align allocations to the extent size hint */ #define XFS_BMAPI_EXTSZALIGN (1u << 11) +/* + * Use with XFS_BMAPI_CONVERT to convert unwritten extents without allocating + * blocks. Stop at a hole or a delalloc extent: return the mappings found so + * far, or -ENOSR if there are none. + */ +#define XFS_BMAPI_CONVERT_ONLY (1u << 12) + #define XFS_BMAPI_FLAGS \ { XFS_BMAPI_ENTIRE, "ENTIRE" }, \ { XFS_BMAPI_METADATA, "METADATA" }, \ @@ -102,7 +109,8 @@ struct xfs_bmalloca { { XFS_BMAPI_COWFORK, "COWFORK" }, \ { XFS_BMAPI_NODISCARD, "NODISCARD" }, \ { XFS_BMAPI_NORMAP, "NORMAP" },\ - { XFS_BMAPI_EXTSZALIGN, "EXTSZALIGN" } + { XFS_BMAPI_EXTSZALIGN, "EXTSZALIGN" }, \ + { XFS_BMAPI_CONVERT_ONLY, "CONVERT_ONLY" } static inline int xfs_bmapi_aflag(int w) diff --git a/fs/xfs/libxfs/xfs_btree.c b/fs/xfs/libxfs/xfs_btree.c index 60ef7f08b1d3..b978ddc13c15 100644 --- a/fs/xfs/libxfs/xfs_btree.c +++ b/fs/xfs/libxfs/xfs_btree.c @@ -2567,7 +2567,7 @@ xfs_btree_lshift( if (XFS_IS_CORRUPT(tcur->bc_mp, i != 1)) { xfs_btree_mark_sick(cur); error = -EFSCORRUPTED; - goto error0; + goto error1; } error = xfs_btree_decrement(tcur, level, &i); @@ -2737,7 +2737,7 @@ xfs_btree_rshift( if (XFS_IS_CORRUPT(tcur->bc_mp, i != 1)) { xfs_btree_mark_sick(cur); error = -EFSCORRUPTED; - goto error0; + goto error1; } error = xfs_btree_increment(tcur, level, &i); diff --git a/fs/xfs/scrub/attr.c b/fs/xfs/scrub/attr.c index 842d65834afb..f909e2b25158 100644 --- a/fs/xfs/scrub/attr.c +++ b/fs/xfs/scrub/attr.c @@ -293,6 +293,78 @@ xchk_xattr_set_map( return ret; } +static unsigned int +xchk_xattr_local_entry( + struct xchk_da_btree *ds, + int level, + char *buf_end, + struct xfs_attr_leafblock *leaf, + int idx) +{ + struct xfs_attr_leaf_name_local *lentry = + xfs_attr3_leaf_name_local(leaf, idx); + char *name_end; + unsigned int namesize; + + name_end = (char *)lentry + offsetof(typeof(*lentry), nameval[0]); + if (name_end > buf_end) { + xchk_da_set_corrupt(ds, level); + return 0; + } + + namesize = xfs_attr_leaf_entsize_local(lentry->namelen, + be16_to_cpu(lentry->valuelen)); + name_end = (char *)lentry + namesize; + if (lentry->namelen == 0) { + xchk_da_set_corrupt(ds, level); + return 0; + } + if (name_end > buf_end) { + xchk_da_set_corrupt(ds, level); + return 0; + } + + return namesize; +} + +static unsigned int +xchk_xattr_remote_entry( + struct xchk_da_btree *ds, + int level, + char *buf_end, + struct xfs_attr_leafblock *leaf, + int idx, + struct xfs_attr_leaf_entry *ent) +{ + struct xfs_attr_leaf_name_remote *rentry = + xfs_attr3_leaf_name_remote(leaf, idx); + char *name_end; + unsigned int namesize; + + name_end = (char *)rentry + offsetof(typeof(*rentry), name[0]); + if (name_end > buf_end) { + xchk_da_set_corrupt(ds, level); + return 0; + } + + namesize = xfs_attr_leaf_entsize_remote(rentry->namelen); + name_end = (char *)rentry + namesize; + if (rentry->namelen == 0) { + xchk_da_set_corrupt(ds, level); + return 0; + } + if (rentry->valueblk == 0 && !(ent->flags & XFS_ATTR_INCOMPLETE)) { + xchk_da_set_corrupt(ds, level); + return 0; + } + if (name_end > buf_end) { + xchk_da_set_corrupt(ds, level); + return 0; + } + + return namesize; +} + /* * Check this leaf entry's relations to everything else. * Returns the number of bytes used for the name/value data. @@ -311,9 +383,6 @@ xchk_xattr_entry( { struct xfs_mount *mp = ds->state->mp; struct xchk_xattr_buf *ab = ds->sc->buf; - char *name_end; - struct xfs_attr_leaf_name_local *lentry; - struct xfs_attr_leaf_name_remote *rentry; unsigned int nameidx; unsigned int namesize; @@ -333,25 +402,14 @@ xchk_xattr_entry( } /* Check the name information. */ - if (ent->flags & XFS_ATTR_LOCAL) { - lentry = xfs_attr3_leaf_name_local(leaf, idx); - namesize = xfs_attr_leaf_entsize_local(lentry->namelen, - be16_to_cpu(lentry->valuelen)); - name_end = (char *)lentry + namesize; - if (lentry->namelen == 0) - xchk_da_set_corrupt(ds, level); - } else { - rentry = xfs_attr3_leaf_name_remote(leaf, idx); - namesize = xfs_attr_leaf_entsize_remote(rentry->namelen); - name_end = (char *)rentry + namesize; - if (rentry->namelen == 0) - xchk_da_set_corrupt(ds, level); - if (rentry->valueblk == 0 && - !(ent->flags & XFS_ATTR_INCOMPLETE)) - xchk_da_set_corrupt(ds, level); - } - if (name_end > buf_end) - xchk_da_set_corrupt(ds, level); + if (ent->flags & XFS_ATTR_LOCAL) + namesize = xchk_xattr_local_entry(ds, level, buf_end, leaf, + idx); + else + namesize = xchk_xattr_remote_entry(ds, level, buf_end, leaf, + idx, ent); + if (!namesize) + return; if (!xchk_xattr_set_map(ds->sc, ab->usedmap, nameidx, namesize)) xchk_da_set_corrupt(ds, level); diff --git a/fs/xfs/scrub/attr_repair.c b/fs/xfs/scrub/attr_repair.c index c7695d4cc8e7..ad688f9f7ce9 100644 --- a/fs/xfs/scrub/attr_repair.c +++ b/fs/xfs/scrub/attr_repair.c @@ -285,6 +285,7 @@ xrep_xattr_salvage_local_attr( struct xfs_attr_leaf_name_local *lentry) { struct xchk_xattr_buf *ab = rx->sc->buf; + char *name_end; unsigned char *value; unsigned int valuelen; unsigned int namesize; @@ -293,10 +294,14 @@ xrep_xattr_salvage_local_attr( * Decode the leaf local entry format. If something seems wrong, we * junk the attribute. */ + name_end = (char *)lentry + offsetof(typeof(*lentry), nameval[0]); + if (name_end > buf_end) + return 0; value = &lentry->nameval[lentry->namelen]; valuelen = be16_to_cpu(lentry->valuelen); namesize = xfs_attr_leaf_entsize_local(lentry->namelen, valuelen); - if ((char *)lentry + namesize > buf_end) + name_end = (char *)lentry + namesize; + if (name_end > buf_end) return 0; if (!xrep_xattr_want_salvage(rx, ent->flags, lentry->nameval, lentry->namelen, value, valuelen)) @@ -333,8 +338,8 @@ xrep_xattr_salvage_remote_attr( .attr_filter = ent->flags & XFS_ATTR_NSP_ONDISK_MASK, .namelen = rentry->namelen, .name = rentry->name, - .valuelen = be32_to_cpu(rentry->valuelen), }; + char *name_end; unsigned int namesize; int error; @@ -343,9 +348,14 @@ xrep_xattr_salvage_remote_attr( * junk the attribute. Note that we should never find a zero-length * remote attribute value. */ + name_end = (char *)rentry + offsetof(typeof(*rentry), name[0]); + if (name_end > buf_end) + return 0; namesize = xfs_attr_leaf_entsize_remote(rentry->namelen); - if ((char *)rentry + namesize > buf_end) + name_end = (char *)rentry + namesize; + if (name_end > buf_end) return 0; + args.valuelen = be32_to_cpu(rentry->valuelen); if (args.valuelen == 0 || !xrep_xattr_want_salvage(rx, ent->flags, rentry->name, rentry->namelen, NULL, args.valuelen)) diff --git a/fs/xfs/scrub/dir.c b/fs/xfs/scrub/dir.c index 4ab7e6469248..05fe8bc71427 100644 --- a/fs/xfs/scrub/dir.c +++ b/fs/xfs/scrub/dir.c @@ -313,6 +313,74 @@ xchk_dir_actor( return error; } +/* + * Deal with an unused directory entry. Returns a positive number of bytes to + * advance in the directory block, zero to stop processing, or -1 if corruption + * was found. + */ +static int +xchk_dir_check_unused( + struct xchk_da_btree *ds, + struct xfs_buf *bp, + unsigned int iter_off, + unsigned int end, + xfs_dablk_t rec_bno) +{ + struct xfs_dir2_data_unused *dup = bp->b_addr + iter_off; + unsigned int advance; + + if (iter_off + sizeof(*dup) > end) { + xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); + return -1; + } + + advance = xfs_dir2_data_unusedsize(be16_to_cpu(dup->length)); + if (!advance || iter_off + advance > end) { + xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); + return -1; + } + + return advance; +} + +/* + * Check a directory entry. Returns a positive number of bytes to advance + * in the directory block, zero to stop processing, or -1 if corruption was + * found. + */ +static int +xchk_dir_check_entry( + struct xchk_da_btree *ds, + struct xfs_buf *bp, + unsigned int iter_off, + unsigned int end, + struct xfs_dir2_data_entry *dent, + xfs_dablk_t rec_bno) +{ + struct xfs_dir2_data_entry *dep = bp->b_addr + iter_off; + struct xfs_mount *mp = ds->state->mp; + unsigned int advance; + + /* must have namelen */ + advance = offsetof(typeof(*dep), name); + if (iter_off + advance >= end) { + xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); + return -1; + } + + /* Don't walk off the end of the block. */ + advance = xfs_dir2_data_entsize(mp, dep->namelen); + if (!advance || iter_off + advance > end) { + xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); + return -1; + } + + if (dep == dent) + return 0; + + return advance; +} + /* Scrub a directory btree record. */ STATIC int xchk_dir_rec( @@ -338,6 +406,7 @@ xchk_dir_rec( xfs_dahash_t hash; struct xfs_dir3_icleaf_hdr hdr; unsigned int tag; + bool foundit = false; int error; ASSERT(blk->magic == XFS_DIR2_LEAF1_MAGIC || @@ -382,28 +451,42 @@ xchk_dir_rec( dent = bp->b_addr + off; /* Make sure we got a real directory entry. */ - iter_off = geo->data_entry_offset; end = xfs_dir3_data_end_offset(geo, bp->b_addr); if (!end) { xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); goto out_relse; } - for (;;) { - struct xfs_dir2_data_entry *dep = bp->b_addr + iter_off; + iter_off = geo->data_entry_offset; + while (iter_off < end) { struct xfs_dir2_data_unused *dup = bp->b_addr + iter_off; + int advance; - if (iter_off >= end) { + /* must have freetag */ + advance = offsetof(typeof(*dup), length); + if (iter_off + advance >= end) { xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); goto out_relse; } - if (be16_to_cpu(dup->freetag) == XFS_DIR2_DATA_FREE_TAG) { - iter_off += be16_to_cpu(dup->length); - continue; - } - if (dep == dent) + if (dup->freetag == cpu_to_be16(XFS_DIR2_DATA_FREE_TAG)) + advance = xchk_dir_check_unused(ds, bp, iter_off, end, + rec_bno); + else + advance = xchk_dir_check_entry(ds, bp, iter_off, end, + dent, rec_bno); + if (advance < 0) + goto out_relse; + if (!advance) { + foundit = true; break; - iter_off += xfs_dir2_data_entsize(mp, dep->namelen); + } + iter_off += advance; + } + + /* Hash tree must point to the exact dirent */ + if (!foundit) { + xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); + goto out_relse; } /* Retrieve the entry, sanity check it, and compare hashes. */ @@ -417,6 +500,13 @@ xchk_dir_rec( goto out_relse; } + /* Name must not overflow end of block */ + if ((char *)dent->name + dent->namelen > + (char *)bp->b_addr + BBTOB(bp->b_length)) { + xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); + goto out_relse; + } + /* Does the directory hash match? */ dname.name = dent->name; dname.len = dent->namelen; diff --git a/fs/xfs/scrub/dir_repair.c b/fs/xfs/scrub/dir_repair.c index bb3f2f462bde..84fe3dda0d42 100644 --- a/fs/xfs/scrub/dir_repair.c +++ b/fs/xfs/scrub/dir_repair.c @@ -462,6 +462,63 @@ xrep_dir_salvage_data_entry( return xrep_dir_salvage_entry(rd, dep->name, dep->namelen, ino); } +/* + * Deal with an unused directory entry. Returns a positive number of bytes to + * advance in the directory block, or zero to stop processing. + */ +static int +xrep_dir_recover_unused( + struct xfs_buf *bp, + unsigned int offset, + unsigned int end) +{ + struct xfs_dir2_data_unused *dup = bp->b_addr + offset; + unsigned int advance; + + if (offset + sizeof(*dup) > end) + return 0; + + advance = xfs_dir2_data_unusedsize(be16_to_cpu(dup->length)); + if (!advance || offset + advance > end) + return 0; + + return advance; +} + +/* + * Salvage a directory entry. Returns a positive number of bytes to advance + * in the directory block, zero to stop processing, or a negative errno. + */ +static int +xrep_dir_recover_entry( + struct xrep_dir *rd, + struct xfs_buf *bp, + unsigned int offset, + unsigned int end) +{ + struct xfs_dir2_data_entry *dep = bp->b_addr + offset; + unsigned int advance; + int error; + + /* must have namelen */ + advance = offsetof(typeof(*dep), name); + if (offset + advance >= end) + return 0; + + /* Don't walk off the end of the block. */ + advance = xfs_dir2_data_entsize(rd->sc->mp, dep->namelen); + if (!advance || offset + advance > end) + return 0; + + /* Ok, let's save this entry. */ + error = xrep_dir_salvage_data_entry(rd, dep); + if (error < 0) + return error; + + return advance; +} + + /* Try to recover block/data format directory entries. */ STATIC int xrep_dir_recover_data( @@ -483,33 +540,25 @@ xrep_dir_recover_data( while (offset < end) { struct xfs_dir2_data_unused *dup = bp->b_addr + offset; - struct xfs_dir2_data_entry *dep = bp->b_addr + offset; - unsigned int advance; + int advance; if (xchk_should_terminate(rd->sc, &error)) return error; - /* Skip unused entries. */ - if (be16_to_cpu(dup->freetag) == XFS_DIR2_DATA_FREE_TAG) { - if (!dup->length) - break; - offset += be16_to_cpu(dup->length); - continue; - } + /* must have freetag */ + advance = offsetof(typeof(*dup), length); + if (offset + advance >= end) + break; - /* Don't walk off the end of the block. */ - advance = xfs_dir2_data_entsize(rd->sc->mp, dep->namelen); + if (dup->freetag == cpu_to_be16(XFS_DIR2_DATA_FREE_TAG)) + advance = xrep_dir_recover_unused(bp, offset, end); + else + advance = xrep_dir_recover_entry(rd, bp, offset, end); + if (advance < 0) + return advance; if (!advance) break; offset += advance; - if (offset > end) - break; - - /* Ok, let's save this entry. */ - error = xrep_dir_salvage_data_entry(rd, dep); - if (error) - return error; - } return 0; @@ -532,6 +581,19 @@ xrep_dir_recover_sf( hdr = ifp->if_data; end = (unsigned char *)ifp->if_data + ifp->if_bytes; + /* sf header must be big enough for count/i8count */ + if (!hdr || ifp->if_bytes < 2) + return 0; + + /* and large enough for the parent inumber */ + if (hdr->i8count) { + if (ifp->if_bytes < offsetof(struct xfs_dir2_sf_hdr, parent[8])) + return 0; + } else { + if (ifp->if_bytes < offsetof(struct xfs_dir2_sf_hdr, parent[4])) + return 0; + } + ino = xfs_dir2_sf_get_parent_ino(hdr); trace_xrep_dir_salvaged_parent(rd->sc->ip, ino); diff --git a/fs/xfs/scrub/quota_repair.c b/fs/xfs/scrub/quota_repair.c index b4f33f432b12..3be56d73dd66 100644 --- a/fs/xfs/scrub/quota_repair.c +++ b/fs/xfs/scrub/quota_repair.c @@ -297,7 +297,6 @@ xrep_quota_block( xfs_dqid_t id) { struct xfs_dqblk *dqblk; - struct xfs_disk_dquot *ddq; struct xfs_quotainfo *qi = sc->mp->m_quotainfo; struct xfs_def_quota *defq = xfs_get_defquota(qi, dqtype); struct xfs_buf *bp = NULL; @@ -319,14 +318,21 @@ xrep_quota_block( break; case 0: dqblk = bp->b_addr; - ddq = &dqblk[0].dd_diskdq; + error = 0; /* * If there's nothing that would impede a dqiterate, we're * done. */ - if ((ddq->d_type & XFS_DQTYPE_REC_MASK) == dqtype && - id == be32_to_cpu(ddq->d_id)) { + for (i = 0; i < qi->qi_dqperchunk; i++, dqblk++) { + struct xfs_disk_dquot *ddq = &dqblk->dd_diskdq; + + if ((ddq->d_type & XFS_DQTYPE_REC_MASK) != dqtype || + id + i != be32_to_cpu(ddq->d_id)) + error++; + } + + if (!error) { xfs_trans_brelse(sc->tp, bp); return 0; } @@ -339,7 +345,9 @@ xrep_quota_block( dqblk = bp->b_addr; bp->b_ops = &xfs_dquot_buf_ops; for (i = 0; i < qi->qi_dqperchunk; i++, dqblk++) { - ddq = &dqblk->dd_diskdq; + struct xfs_disk_dquot *ddq = &dqblk->dd_diskdq; + bool was_bigtime = + !!(ddq->d_type & XFS_DQTYPE_BIGTIME); trace_xrep_disk_dquot(sc->mp, dqtype, id + i); @@ -348,8 +356,21 @@ xrep_quota_block( ddq->d_type = dqtype; ddq->d_id = cpu_to_be32(id + i); - if (xfs_has_bigtime(sc->mp) && ddq->d_id) + if (xfs_has_bigtime(sc->mp) && ddq->d_id) { + /* + * If something was broken with this dquot and it was a + * non-bigtime dquot, we'll grant everyone a fresh + * grace period by zeroing the timer field rather than + * try to interpret what might be garbage. + */ + if (!was_bigtime) { + ddq->d_btimer = 0; + ddq->d_itimer = 0; + ddq->d_rtbtimer = 0; + } + ddq->d_type |= XFS_DQTYPE_BIGTIME; + } xrep_quota_fix_timer(sc->mp, ddq, ddq->d_blk_softlimit, ddq->d_bcount, &ddq->d_btimer, diff --git a/fs/xfs/xfs_discard.c b/fs/xfs/xfs_discard.c index 906221ffe9ba..cdf660344c28 100644 --- a/fs/xfs/xfs_discard.c +++ b/fs/xfs/xfs_discard.c @@ -155,6 +155,8 @@ struct xfs_trim_cur { xfs_agblock_t end; xfs_extlen_t minlen; bool by_bno; + /* restart point (with count) for the by-length walk */ + xfs_agblock_t restart_bno; }; static int @@ -198,14 +200,15 @@ xfs_trim_gather_extents( error = xfs_alloc_lookup_le(cur, tcur->start, 0, &i); if (!error && !i) error = xfs_alloc_lookup_ge(cur, tcur->start, 0, &i); - } else if (tcur->start == 0) { - /* first time through a by-len starts with max length */ - cur = xfs_cntbt_init_cursor(mp, tp, agbp, pag); - error = xfs_alloc_lookup_ge(cur, 0, tcur->count, &i); } else { - /* nth time through a by-len starts where we left off */ + /* + * A by-len walk starts at the last record of the longest + * length (restart_bno is NULLAGBLOCK until the first batch + * ends) and then where the previous batch stopped. + */ cur = xfs_cntbt_init_cursor(mp, tp, agbp, pag); - error = xfs_alloc_lookup_le(cur, tcur->start, tcur->count, &i); + error = xfs_alloc_lookup_le(cur, tcur->restart_bno, + tcur->count, &i); } if (error) goto out_del_cursor; @@ -235,9 +238,15 @@ xfs_trim_gather_extents( if (--batch <= 0) { /* * Update the cursor to point at this extent so we - * restart the next batch from this extent. + * restart the next batch from this extent. The + * by-length walk does not visit extents in block + * order, so it must keep the start of the range and + * restart from restart_bno instead. */ - tcur->start = fbno; + if (tcur->by_bno) + tcur->start = fbno; + else + tcur->restart_bno = fbno; tcur->count = flen; break; } @@ -340,6 +349,7 @@ xfs_trim_perag_extents( .start = start, .end = end, .minlen = minlen, + .restart_bno = NULLAGBLOCK, }; int error = 0; diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index d8202da15aca..59bd9a24b3ef 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -668,7 +668,7 @@ xfs_dio_write_end_io( * they are converted. */ if (flags & IOMAP_DIO_UNWRITTEN) { - error = xfs_iomap_write_unwritten(ip, offset, size, true); + error = xfs_iomap_write_unwritten(ip, offset, size, true, 0); goto out; } diff --git a/fs/xfs/xfs_healthmon.c b/fs/xfs/xfs_healthmon.c index 2bdd747f1ead..05d6f0494139 100644 --- a/fs/xfs/xfs_healthmon.c +++ b/fs/xfs/xfs_healthmon.c @@ -114,8 +114,8 @@ xfs_healthmon_attach( } refcount_inc(&hm->ref); - rcu_assign_pointer(mp->m_healthmon, hm); hm->mount_cookie = (uintptr_t)mp->m_super; + rcu_assign_pointer(mp->m_healthmon, hm); spin_unlock(&xfs_healthmon_lock); return 0; diff --git a/fs/xfs/xfs_ioctl32.c b/fs/xfs/xfs_ioctl32.c index c66e192448a8..e411f9453bfb 100644 --- a/fs/xfs/xfs_ioctl32.c +++ b/fs/xfs/xfs_ioctl32.c @@ -123,8 +123,8 @@ xfs_ioctl32_bstat_copyin( xfs_ioctl32_bstime_copyin(&bstat->bs_atime, &bstat32->bs_atime) || xfs_ioctl32_bstime_copyin(&bstat->bs_mtime, &bstat32->bs_mtime) || xfs_ioctl32_bstime_copyin(&bstat->bs_ctime, &bstat32->bs_ctime) || - get_user(bstat->bs_blocks, &bstat32->bs_size) || - get_user(bstat->bs_xflags, &bstat32->bs_size) || + get_user(bstat->bs_blocks, &bstat32->bs_blocks) || + get_user(bstat->bs_xflags, &bstat32->bs_xflags) || get_user(bstat->bs_extsize, &bstat32->bs_extsize) || get_user(bstat->bs_extents, &bstat32->bs_extents) || get_user(bstat->bs_gen, &bstat32->bs_gen) || diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c index 40695d18dac0..aced9a023db1 100644 --- a/fs/xfs/xfs_ioend.c +++ b/fs/xfs/xfs_ioend.c @@ -102,7 +102,7 @@ xfs_end_ioend_write( else if (ioend->io_flags & IOMAP_IOEND_SHARED) error = xfs_reflink_end_cow(ip, offset, size); else if (ioend->io_flags & IOMAP_IOEND_UNWRITTEN) - error = xfs_iomap_write_unwritten(ip, offset, size, false); + error = xfs_iomap_write_unwritten(ip, offset, size, false, 0); if (!error && !(ioend->io_flags & IOMAP_IOEND_DIRECT) && diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c index 7c6238fed61e..373ff0e64f14 100644 --- a/fs/xfs/xfs_iomap.c +++ b/fs/xfs/xfs_iomap.c @@ -285,6 +285,14 @@ xfs_iomap_write_direct( ASSERT(count_fsb > 0); + /* + * xfs_bmapi_write() maps at most XFS_MAX_BMBT_EXTLEN blocks in one + * mapping, so don't reserve blocks for more. A pNFS layout range + * comes from a client, and its block count may not even fit the + * 32-bit reservation. The caller maps the rest with a later call. + */ + count_fsb = min_t(xfs_filblks_t, count_fsb, XFS_MAX_BMBT_EXTLEN); + resaligned = xfs_aligned_fsb_count(offset_fsb, count_fsb, xfs_get_extsz_hint(ip)); if (unlikely(XFS_IS_REALTIME_INODE(ip))) { @@ -614,7 +622,8 @@ xfs_iomap_write_unwritten( xfs_inode_t *ip, xfs_off_t offset, xfs_off_t count, - bool update_isize) + bool update_isize, + uint32_t bmapi_flags) { xfs_mount_t *mp = ip->i_mount; xfs_fileoff_t offset_fsb; @@ -676,8 +685,21 @@ xfs_iomap_write_unwritten( */ nimaps = 1; error = xfs_bmapi_write(tp, ip, offset_fsb, count_fsb, - XFS_BMAPI_CONVERT, resblks, &imap, - &nimaps); + XFS_BMAPI_CONVERT | bmapi_flags, + resblks, &imap, &nimaps); + if (error == -ENOSR && (bmapi_flags & XFS_BMAPI_CONVERT_ONLY)) { + /* + * Nothing was mapped: the rest of the range starts + * with a hole or a delalloc extent. Nothing was + * changed either, but xfs_iext_count_extend() may + * have logged the inode, and cancelling a dirty + * transaction shuts the filesystem down. Commit it, + * as xfs_alloc_file_space() does on -ENOSR. + */ + error = xfs_trans_commit(tp); + xfs_iunlock(ip, XFS_ILOCK_EXCL); + return error ? error : -EIO; + } if (error) goto error_on_bmapi_transaction; diff --git a/fs/xfs/xfs_iomap.h b/fs/xfs/xfs_iomap.h index f2520a9b3a13..8fc0c6fc8bea 100644 --- a/fs/xfs/xfs_iomap.h +++ b/fs/xfs/xfs_iomap.h @@ -15,7 +15,8 @@ struct xfs_zone_alloc_ctx; int xfs_iomap_write_direct(struct xfs_inode *ip, xfs_fileoff_t offset_fsb, xfs_fileoff_t count_fsb, unsigned int flags, struct xfs_bmbt_irec *imap, u64 *sequence); -int xfs_iomap_write_unwritten(struct xfs_inode *, xfs_off_t, xfs_off_t, bool); +int xfs_iomap_write_unwritten(struct xfs_inode *ip, xfs_off_t offset, + xfs_off_t count, bool update_isize, uint32_t bmapi_flags); xfs_fileoff_t xfs_iomap_eof_align_last_fsb(struct xfs_inode *ip, xfs_fileoff_t end_fsb); diff --git a/fs/xfs/xfs_pnfs.c b/fs/xfs/xfs_pnfs.c index ab385617003a..7153e7a1e1de 100644 --- a/fs/xfs/xfs_pnfs.c +++ b/fs/xfs/xfs_pnfs.c @@ -110,7 +110,15 @@ xfs_fs_map_update_inode( ip->i_diflags |= XFS_DIFLAG_PREALLOC; xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); - return xfs_trans_commit(tp); + error = xfs_trans_commit(tp); + if (error) + return error; + + /* + * Force the log so that the blocks handed out to the client are + * guaranteed to be present even after a server crash. + */ + return xfs_log_force_inode(ip); } /* @@ -134,6 +142,7 @@ xfs_fs_map_blocks( uint lock_flags; int error = 0; u64 seq; + bool update_inode = false; if (xfs_is_shutdown(mp)) return -EIO; @@ -167,10 +176,9 @@ xfs_fs_map_blocks( if (!write) limit = max(limit, round_up(i_size_read(inode), inode->i_sb->s_blocksize)); - if (offset > limit) + if (offset < 0 || offset >= limit) goto out_unlock; - if (offset > limit - length) - length = limit - offset; + length = min_t(u64, length, limit - offset); error = filemap_write_and_wait(inode->i_mapping); if (error) @@ -221,20 +229,27 @@ xfs_fs_map_blocks( end_fsb - offset_fsb, 0, &imap, &seq); if (error) goto out_unlock; + update_inode = true; + } else { + xfs_iunlock(ip, lock_flags); /* - * Ensure the next transaction is committed synchronously so - * that the blocks allocated and handed out to the client are - * guaranteed to be present even after a server crash. + * Blocks past EOF that were allocated for buffered writes, such + * as speculative preallocation, can be freed by + * xfs_free_eofblocks() without breaking the layout unless the + * inode is marked as preallocated. Mark it before handing them + * out for writing, as for blocks allocated above. */ + if (write && !(ip->i_diflags & XFS_DIFLAG_PREALLOC) && + imap.br_startoff + imap.br_blockcount > + XFS_B_TO_FSB(mp, XFS_ISIZE(ip))) + update_inode = true; + } + + if (update_inode) { error = xfs_fs_map_update_inode(ip); - if (!error) - error = xfs_log_force_inode(ip); if (error) goto out_unlock; - - } else { - xfs_iunlock(ip, lock_flags); } xfs_iunlock(ip, XFS_IOLOCK_EXCL); @@ -312,9 +327,7 @@ xfs_fs_commit_blocks( if (start > size) continue; - end = start + maps[i].length; - if (end > size) - end = size; + end = start + min_t(u64, maps[i].length, size - start); length = end - start; if (!length) @@ -328,7 +341,14 @@ xfs_fs_commit_blocks( (end - 1) >> PAGE_SHIFT); WARN_ON_ONCE(error); - error = xfs_iomap_write_unwritten(ip, start, length, false); + /* + * A client may only write to blocks that a layout handed out, + * and xfs_fs_map_blocks() allocates blocks for any hole in a + * write layout, so refuse to allocate blocks for a hole or a + * delalloc extent in the committed range. + */ + error = xfs_iomap_write_unwritten(ip, start, length, false, + XFS_BMAPI_CONVERT_ONLY); if (error) goto out_drop_iolock; } diff --git a/include/acpi/acpi_bus.h b/include/acpi/acpi_bus.h index a10a591c18b2..8b1d329455f7 100644 --- a/include/acpi/acpi_bus.h +++ b/include/acpi/acpi_bus.h @@ -32,6 +32,12 @@ acpi_status acpi_evaluate_integer(acpi_handle handle, acpi_string pathname, struct acpi_object_list *arguments, unsigned long long *data); +acpi_status acpi_evaluate_object_exclusive(acpi_handle handle, acpi_string pathname, + struct acpi_object_list *arguments, + struct acpi_buffer *buffer); +acpi_status acpi_evaluate_integer_exclusive(acpi_handle handle, acpi_string pathname, + struct acpi_object_list *arguments, + unsigned long long *data); bool acpi_evaluate_reference(acpi_handle handle, acpi_string pathname, struct acpi_object_list *arguments, struct acpi_handle_list *list); @@ -48,6 +54,7 @@ acpi_evaluate_ost(acpi_handle handle, u32 source_event, u32 status_code, bool acpi_has_method(acpi_handle handle, char *name); acpi_status acpi_execute_simple_method(acpi_handle handle, char *method, u64 arg); +acpi_status acpi_execute_simple_method_exclusive(acpi_handle handle, char *method, u64 arg); acpi_status acpi_evaluate_ej0(acpi_handle handle); acpi_status acpi_evaluate_lck(acpi_handle handle, int lock); acpi_status acpi_evaluate_reg(acpi_handle handle, u8 space_id, u32 function); diff --git a/include/acpi/cppc_acpi.h b/include/acpi/cppc_acpi.h index 94a6277edab2..a32b09ec3394 100644 --- a/include/acpi/cppc_acpi.h +++ b/include/acpi/cppc_acpi.h @@ -188,6 +188,7 @@ extern int cppc_set_epp(int cpu, u64 epp_val); extern int cppc_get_auto_act_window(int cpu, u64 *auto_act_window); extern int cppc_set_auto_act_window(int cpu, u64 auto_act_window); extern int cppc_get_auto_sel(int cpu, bool *enable); +bool cppc_auto_sel_is_immutable(int cpu); extern int cppc_set_auto_sel(int cpu, bool enable); extern int cppc_get_perf_limited(int cpu, u64 *perf_limited); extern int cppc_set_perf_limited(int cpu, u64 bits_to_clear); @@ -289,6 +290,11 @@ static inline int cppc_get_auto_sel(int cpu, bool *enable) { return -EOPNOTSUPP; } +static inline bool cppc_auto_sel_is_immutable(int cpu) +{ + return false; +} + static inline int cppc_set_auto_sel(int cpu, bool enable) { return -EOPNOTSUPP; diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h index bdcc2778ac64..32ffffd9d7b9 100644 --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -398,8 +398,7 @@ struct mmu_gather { void tlb_flush_mmu(struct mmu_gather *tlb); static inline void __tlb_adjust_range(struct mmu_gather *tlb, - unsigned long address, - unsigned int range_size) + unsigned long address, unsigned long range_size) { tlb->start = min(tlb->start, address); tlb->end = max(tlb->end, address + range_size); diff --git a/include/crypto/mldsa.h b/include/crypto/mldsa.h index 3ef2676787c9..b156f166a141 100644 --- a/include/crypto/mldsa.h +++ b/include/crypto/mldsa.h @@ -48,11 +48,13 @@ enum mldsa_alg { * Return: * * 0 if the signature is valid * * -EBADMSG if the signature and/or public key is malformed + * * -EINVAL if @alg is not a valid value * * -EKEYREJECTED if the signature is invalid but otherwise well-formed * * -ENOMEM if out of memory so the validity of the signature is unknown */ -int mldsa_verify(enum mldsa_alg alg, const u8 *sig, size_t sig_len, - const u8 *msg, size_t msg_len, const u8 *pk, size_t pk_len); +int __must_check mldsa_verify(enum mldsa_alg alg, const u8 *sig, size_t sig_len, + const u8 *msg, size_t msg_len, const u8 *pk, + size_t pk_len); #if IS_ENABLED(CONFIG_CRYPTO_LIB_MLDSA_KUNIT_TEST) /* Internal function, exposed only for unit testing */ diff --git a/include/dt-bindings/clock/nuvoton,ma35d1-clk.h b/include/dt-bindings/clock/nuvoton,ma35d1-clk.h index ba2d70f776a6..764d837d6162 100644 --- a/include/dt-bindings/clock/nuvoton,ma35d1-clk.h +++ b/include/dt-bindings/clock/nuvoton,ma35d1-clk.h @@ -248,6 +248,7 @@ #define ADC_GATE 233 #define EADC_DIV 234 #define EADC_GATE 235 -#define CLK_MAX_IDX 236 +#define PCLK3_DIV4096 236 +#define PCLK4_DIV4096 237 #endif /* __DT_BINDINGS_CLOCK_NUVOTON_MA35D1_CLK_H */ diff --git a/include/linux/assoc_array_priv.h b/include/linux/assoc_array_priv.h index dca733ef6750..37a9f76e1505 100644 --- a/include/linux/assoc_array_priv.h +++ b/include/linux/assoc_array_priv.h @@ -80,7 +80,7 @@ struct assoc_array_edit { struct assoc_array_ptr **leaf_p; struct assoc_array_ptr *dead_leaf; struct assoc_array_ptr *new_meta[3]; - struct assoc_array_ptr *excised_meta[1]; + struct assoc_array_ptr *excised_meta[2]; struct assoc_array_ptr *excised_subtree; struct assoc_array_ptr **set_backpointers[ASSOC_ARRAY_FAN_OUT]; struct assoc_array_ptr *set_backpointers_to; diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 4f7905c3412b..3989bb0dd43b 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -675,7 +675,7 @@ enum { QUEUE_FLAG_HCTX_ACTIVE, /* at least one blk-mq hctx is active */ QUEUE_FLAG_SQ_SCHED, /* single queue style io dispatch */ QUEUE_FLAG_DISABLE_WBT_DEF, /* for sched to disable/enable wbt */ - QUEUE_FLAG_NO_ELV_SWITCH, /* can't switch elevator any more */ + QUEUE_FLAG_NO_ELV_SWITCH, /* a switch is running or disk is going away */ QUEUE_FLAG_QOS_ENABLED, /* qos is enabled */ QUEUE_FLAG_BIO_ISSUE_TIME, /* record bio->issue_time_ns */ QUEUE_FLAG_ZONED_QD1_WRITES, /* Limit zoned devices writes to QD=1 */ @@ -716,8 +716,6 @@ void blk_queue_flag_clear(unsigned int flag, struct request_queue *q); ((q)->limits.features & BLK_FEAT_SKIP_TAGSET_QUIESCE) #define blk_queue_disable_wbt(q) \ test_bit(QUEUE_FLAG_DISABLE_WBT_DEF, &(q)->queue_flags) -#define blk_queue_no_elv_switch(q) \ - test_bit(QUEUE_FLAG_NO_ELV_SWITCH, &(q)->queue_flags) #define blk_queue_zoned_qd1_writes(q) \ test_bit(QUEUE_FLAG_ZONED_QD1_WRITES, &(q)->queue_flags) diff --git a/include/linux/can/can-ml.h b/include/linux/can/can-ml.h index 1e99fda2b380..cfefa4a385be 100644 --- a/include/linux/can/can-ml.h +++ b/include/linux/can/can-ml.h @@ -45,6 +45,7 @@ #include #include #include +#include /* exposed CAN device capabilities for network layer */ #define CAN_CAP_CC BIT(0) /* CAN CC aka Classical CAN */ @@ -59,6 +60,7 @@ enum { RX_ERR, RX_ALL, RX_FIL, RX_INV, RX_MAX }; struct can_dev_rcv_lists { + spinlock_t lock; /* protects the receive lists and entries */ struct hlist_head rx[RX_MAX]; struct hlist_head rx_sff[CAN_SFF_RCV_ARRAY_SZ]; struct hlist_head rx_eff[CAN_EFF_RCV_ARRAY_SZ]; @@ -81,6 +83,7 @@ static inline struct can_ml_priv *can_get_ml_priv(struct net_device *dev) static inline void can_set_ml_priv(struct net_device *dev, struct can_ml_priv *ml_priv) { + spin_lock_init(&ml_priv->dev_rcv_lists.lock); netdev_set_ml_priv(dev, ml_priv, ML_PRIV_CAN); } diff --git a/include/linux/can/dev.h b/include/linux/can/dev.h index 6d0710d6f571..346faddee02e 100644 --- a/include/linux/can/dev.h +++ b/include/linux/can/dev.h @@ -176,7 +176,7 @@ static inline bool can_dev_dropped_skb(struct net_device *dev, struct sk_buff *s return can_dropped_invalid_skb(dev, skb); invalid_skb: - kfree_skb(skb); + dev_kfree_skb_any(skb); dev->stats.tx_dropped++; return true; } diff --git a/include/linux/can/skb.h b/include/linux/can/skb.h index 5d27843862fc..87ace09c0d8b 100644 --- a/include/linux/can/skb.h +++ b/include/linux/can/skb.h @@ -78,12 +78,12 @@ static inline struct sk_buff *can_create_echo_skb(struct sk_buff *skb) nskb = skb_clone(skb, GFP_ATOMIC); if (unlikely(!nskb)) { - kfree_skb(skb); + dev_kfree_skb_any(skb); return NULL; } can_skb_set_owner(nskb, skb->sk); - consume_skb(skb); + dev_consume_skb_any(skb); return nskb; } diff --git a/include/linux/damon.h b/include/linux/damon.h index 0c8b7ddef9ab..9772309b2b58 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -914,6 +914,16 @@ struct damon_ctx { /* @rnd_state: Per-ctx PRNG state for damon_rand(). */ struct rnd_state rnd_state; + /* + * number of regions after last merge. Read kdamond_split_regions() + * for more details. + */ + unsigned long last_nr_regions; + /* + * number of damon_split_some_regions() invocations for this context. + * Read damon_split_some_regions() for more details. + */ + unsigned long region_split_rotation; }; /* Get a random number in [@l, @r) using @ctx's lockless PRNG. */ diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index c745f7ad2298..fac05120ab77 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -12,9 +12,8 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma); bool huge_pmd_set_accessed(struct vm_fault *vmf); -int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, - pud_t *dst_pud, pud_t *src_pud, unsigned long addr, - struct vm_area_struct *vma); +int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, + pud_t *dst_pud, pud_t *src_pud, unsigned long addr); #ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud); diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 45ada75dc04e..77d62c930dc3 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -121,10 +121,9 @@ void hugepage_put_subpool(struct hugepage_subpool *spool); void hugetlb_dup_vma_private(struct vm_area_struct *vma); void clear_vma_resv_huge_pages(struct vm_area_struct *vma); -int move_hugetlb_page_tables(struct vm_area_struct *vma, - struct vm_area_struct *new_vma, - unsigned long old_addr, unsigned long new_addr, - unsigned long len); +unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma, + struct vm_area_struct *new_vma, unsigned long old_addr, + unsigned long new_addr, unsigned long len); int copy_hugetlb_page_range(struct mm_struct *, struct mm_struct *, struct vm_area_struct *, struct vm_area_struct *); void unmap_hugepage_range(struct vm_area_struct *, @@ -339,11 +338,9 @@ static inline int copy_hugetlb_page_range(struct mm_struct *dst, return 0; } -static inline int move_hugetlb_page_tables(struct vm_area_struct *vma, - struct vm_area_struct *new_vma, - unsigned long old_addr, - unsigned long new_addr, - unsigned long len) +static inline unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma, + struct vm_area_struct *new_vma, unsigned long old_addr, + unsigned long new_addr, unsigned long len) { BUG(); return 0; diff --git a/include/linux/ksm.h b/include/linux/ksm.h index d39d0d5483a2..a0d358a1feb6 100644 --- a/include/linux/ksm.h +++ b/include/linux/ksm.h @@ -91,8 +91,8 @@ static inline void ksm_exit(struct mm_struct *mm) * We'd like to make this conditional on vma->vm_flags & VM_MERGEABLE, * but what if the vma was unmerged while the page was swapped out? */ -struct folio *ksm_might_need_to_copy(struct folio *folio, - struct vm_area_struct *vma, unsigned long addr); +struct folio *ksm_might_need_to_copy(struct folio *folio, struct page *page, + struct vm_area_struct *vma, unsigned long addr); void rmap_walk_ksm(struct folio *folio, struct rmap_walk_control *rwc); void folio_migrate_ksm(struct folio *newfolio, struct folio *folio); @@ -145,7 +145,7 @@ static inline int ksm_madvise(struct vm_area_struct *vma, unsigned long start, } static inline struct folio *ksm_might_need_to_copy(struct folio *folio, - struct vm_area_struct *vma, unsigned long addr) + struct page *page, struct vm_area_struct *vma, unsigned long addr) { return folio; } diff --git a/include/linux/memfd.h b/include/linux/memfd.h index c159e40e3f34..272479317cf1 100644 --- a/include/linux/memfd.h +++ b/include/linux/memfd.h @@ -2,10 +2,19 @@ #ifndef __LINUX_MEMFD_H #define __LINUX_MEMFD_H +#include #include #define MEMFD_ANON_NAME "[memfd]" +static inline int memfd_check_seals_mode(unsigned int seals, umode_t old_mode, + umode_t new_mode) +{ + if ((seals & F_SEAL_EXEC) && ((old_mode ^ new_mode) & 0111)) + return -EPERM; + return 0; +} + #ifdef CONFIG_MEMFD_CREATE extern long memfd_fcntl(struct file *file, unsigned int cmd, unsigned int arg); struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx); diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index a11a44eef521..5db78e5761c5 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -483,7 +483,8 @@ mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) /* * This version of mmu_notifier_invalidate_range_start() avoids blocking, but it * can return an error if a notifier can't proceed without blocking, in which - * case you're not allowed to modify PTEs in the specified range. + * case you're not allowed to modify PTEs in the specified range or call + * mmu_notifier_invalidate_range_end(). * * This is mainly intended for OOM handling. */ diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 94f9c3ff5416..da8dd04fb462 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -492,6 +492,8 @@ enum lruvec_flags { * folio->flags, masked by LRU_REFS_MASK. */ #define MAX_NR_TIERS 4U +#define LRU_TIER_MIN 0U +#define LRU_TIER_MAX (MAX_NR_TIERS - 1) #ifndef __GENERATING_BOUNDS_H diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h index 1a3981c26b23..41b7cdcc674f 100644 --- a/include/linux/nfs4.h +++ b/include/linux/nfs4.h @@ -263,6 +263,12 @@ enum why_no_delegation4 { /* new to v4.1 */ WND4_IS_DIR = 8, }; +enum stable_how4 { + UNSTABLE4 = 0, + DATA_SYNC4 = 1, + FILE_SYNC4 = 2, +}; + enum lock_type4 { NFS4_UNLOCK_LT = 0, NFS4_READ_LT = 1, diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h index 5663dff53186..842cc0f96147 100644 --- a/include/linux/shmem_fs.h +++ b/include/linux/shmem_fs.h @@ -111,8 +111,14 @@ extern unsigned long shmem_get_unmapped_area(struct file *, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags); extern int shmem_lock(struct file *file, int lock, struct ucounts *ucounts); #ifdef CONFIG_SHMEM +bool shmem_file(struct file *file); bool shmem_mapping(const struct address_space *mapping); #else +static inline bool shmem_file(struct file *file) +{ + return false; +} + static inline bool shmem_mapping(const struct address_space *mapping) { return false; @@ -187,15 +193,6 @@ static inline struct page *shmem_read_mapping_page( mapping_gfp_mask(mapping)); } -static inline bool shmem_file(struct file *file) -{ - if (!IS_ENABLED(CONFIG_SHMEM)) - return false; - if (!file || !file->f_mapping) - return false; - return shmem_mapping(file->f_mapping); -} - /* Must be called with inode lock taken exclusive. */ static inline void shmem_freeze(struct inode *inode, bool freeze) { diff --git a/include/linux/srcu.h b/include/linux/srcu.h index 7d9bc06df98d..1c221861ba43 100644 --- a/include/linux/srcu.h +++ b/include/linux/srcu.h @@ -18,6 +18,7 @@ #include #include +#include #include #include @@ -103,6 +104,41 @@ void call_srcu(struct srcu_struct *ssp, struct rcu_head *head, void cleanup_srcu_struct(struct srcu_struct *ssp); void synchronize_srcu(struct srcu_struct *ssp); +/** + * list_splice_init_srcu - splice an SRCU-protected list into an existing list + * @list: the SRCU-protected list to splice + * @head: the place in the existing list to splice the first list into + * @ssp: the srcu_struct that readers of @list use + * + * Like list_splice_init_rcu(), but wait for the SRCU readers of @list + * instead of RCU ones, so that none of them can follow a next pointer + * onto @head. + * + * Note that this function blocks, and that the caller must prevent any + * other updates to @list. + */ +static inline void list_splice_init_srcu(struct list_head *list, + struct list_head *head, + struct srcu_struct *ssp) +{ + struct list_head *first = list->next; + struct list_head *last = list->prev; + struct list_head *next = head->next; + + if (list_empty(list)) + return; + + INIT_LIST_HEAD_RCU(list); + synchronize_srcu(ssp); + ASSERT_EXCLUSIVE_ACCESS(*first); + ASSERT_EXCLUSIVE_ACCESS(*last); + + last->next = next; + rcu_assign_pointer(list_next_rcu(head), first); + first->prev = head; + next->prev = last; +} + #define SRCU_GET_STATE_COMPLETED 0x1 /** diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h index 3c2b8c355ab3..30344c0d6a9d 100644 --- a/include/linux/sunrpc/clnt.h +++ b/include/linux/sunrpc/clnt.h @@ -199,7 +199,8 @@ struct rpc_xprt *rpc_task_get_xprt(struct rpc_clnt *clnt, int rpcb_create_local(struct net *); void rpcb_put_local(struct net *); -int rpcb_register(struct net *, u32, u32, int, unsigned short); +int rpcb_register(struct net *net, u32 prog, u32 vers, int prot, + unsigned short port); int rpcb_v4_register(struct net *net, const u32 program, const u32 version, const struct sockaddr *address, diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h index 2db1b9ec5658..5fa9417e034d 100644 --- a/include/linux/sunrpc/svc.h +++ b/include/linux/sunrpc/svc.h @@ -78,6 +78,7 @@ struct svc_serv { unsigned int sv_max_payload; /* datagram payload size */ unsigned int sv_max_mesg; /* max_payload + 1 page for overheads */ unsigned int sv_xdrsize; /* XDR buffer size */ + atomic_t sv_rpcb_failures; /* unanswered rpcbind calls */ struct list_head sv_permsocks; /* all permanent sockets */ struct list_head sv_tempsocks; /* all temporary sockets */ int sv_tmpcnt; /* count of temporary "valid" sockets */ @@ -451,6 +452,7 @@ int sunrpc_set_pool_mode(const char *val); int sunrpc_get_pool_mode(char *val, size_t size); void svc_rpcb_cleanup(struct svc_serv *serv, struct net *net); int svc_bind(struct svc_serv *serv, struct net *net); +unsigned int svc_rpcb_failure_count(struct svc_serv *serv); struct svc_serv *svc_create(struct svc_program *, unsigned int, int (*threadfn)(void *data)); bool svc_rqst_replace_page(struct svc_rqst *rqstp, @@ -471,8 +473,9 @@ unsigned int svc_serv_maxthreads(const struct svc_serv *serv); int svc_pool_stats_open(struct svc_info *si, struct file *file); void svc_process(struct svc_rqst *rqstp); void svc_process_bc(struct rpc_rqst *req, struct svc_rqst *rqstp); -int svc_register(const struct svc_serv *, struct net *, const int, - const unsigned short, const unsigned short); +int svc_register(struct svc_serv *serv, struct net *net, + const int family, const unsigned short proto, + const unsigned short port); void svc_wake_up(struct svc_serv *); void svc_reserve(struct svc_rqst *rqstp, int space); diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index a4351cffc60c..4c0924c73166 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -154,8 +154,6 @@ extern int mrwprotect_range(struct userfaultfd_ctx *ctx, unsigned long start, unsigned long len, bool enable_rwp); /* move_pages */ -void double_pt_lock(spinlock_t *ptl1, spinlock_t *ptl2); -void double_pt_unlock(spinlock_t *ptl1, spinlock_t *ptl2); int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pmd_t dst_pmdval, struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, diff --git a/include/net/espintcp.h b/include/net/espintcp.h index c70efd704b6d..7cad3c6822df 100644 --- a/include/net/espintcp.h +++ b/include/net/espintcp.h @@ -34,7 +34,8 @@ static inline struct espintcp_ctx *espintcp_getctx(const struct sock *sk) { const struct inet_connection_sock *icsk = inet_csk(sk); - /* RCU is only needed for diag */ - return (__force void *)icsk->icsk_ulp_data; + /* pairs with smp_wmb() in espintcp_init_sk() */ + smp_rmb(); + return (__force void *)READ_ONCE(icsk->icsk_ulp_data); } #endif diff --git a/include/net/inet_frag.h b/include/net/inet_frag.h index b02d2472b003..c1fb419a9cf0 100644 --- a/include/net/inet_frag.h +++ b/include/net/inet_frag.h @@ -79,6 +79,7 @@ struct frag_v6_compare_key { * @tstamp_type: stamp has a mono delivery time (EDT) * @flags: fragment queue flags * @max_size: maximum received fragment size + * @truesize: sum of the truesize of queued fragments * @fqdir: pointer to struct fqdir * @rcu: rcu head for freeing deferall */ @@ -100,6 +101,7 @@ struct inet_frag_queue { u8 tstamp_type; __u8 flags; u16 max_size; + unsigned int truesize; struct fqdir *fqdir; struct rcu_head rcu; }; @@ -172,6 +174,13 @@ extern const u8 ip_frag_ecn_table[16]; #define IPFRAG_DUP 1 #define IPFRAG_OVERLAP 2 #define IPFRAG_MIXED_SECPATH 3 +#define IPFRAG_TOOBIG 4 + +/* Max sum of the truesize of fragments in a queue. This keeps the sums + * done in inet_frag_reasm_prepare() and inet_frag_reasm_finish() well + * below INT_MAX. Legitimate traffic stays far below this limit. + */ +#define INET_FRAG_MAX_TRUESIZE (INT_MAX / 2) int inet_frag_queue_insert(struct inet_frag_queue *q, struct sk_buff *skb, int offset, int end); void *inet_frag_reasm_prepare(struct inet_frag_queue *q, struct sk_buff *skb, diff --git a/include/net/netns/can.h b/include/net/netns/can.h index 48b79f7e6236..54f23681e0d4 100644 --- a/include/net/netns/can.h +++ b/include/net/netns/can.h @@ -29,7 +29,7 @@ struct netns_can { /* receive filters subscribed for 'all' CAN devices */ struct can_dev_rcv_lists *rx_alldev_list; - spinlock_t rcvlists_lock; + spinlock_t rcvlists_lock; /* protects the rcv_entries(_max) stats */ struct timer_list stattimer; /* timer for statistics update */ struct can_pkg_stats *pkg_stats; struct can_rcv_lists_stats *rcv_lists_stats; diff --git a/include/net/sctp/sm.h b/include/net/sctp/sm.h index 3bfd261a53cc..76605d1ee839 100644 --- a/include/net/sctp/sm.h +++ b/include/net/sctp/sm.h @@ -252,6 +252,7 @@ struct sctp_chunk *sctp_make_fwdtsn(const struct sctp_association *asoc, struct sctp_fwdtsn_skip *skiplist); struct sctp_chunk *sctp_make_auth(const struct sctp_association *asoc, __u16 key_id); +void sctp_control_set_owner_w(struct sctp_chunk *chunk); struct sctp_chunk *sctp_make_strreset_req(const struct sctp_association *asoc, __u16 stream_num, __be16 *stream_list, bool out, bool in); diff --git a/include/sound/cs35l56.h b/include/sound/cs35l56.h index 747e10ec1a27..ce6383c2c60c 100644 --- a/include/sound/cs35l56.h +++ b/include/sound/cs35l56.h @@ -209,6 +209,9 @@ struct snd_ctl_elem_value; #define CS35L56_GPIO_FN_GPIO 0x00000001 +/* DSP1_FW_VER */ +#define CS35L56_FW_MAIN_VERSION_MASK GENMASK(19, 0) + /* Mixer input sources */ #define CS35L56_INPUT_SRC_NONE 0x00 #define CS35L56_INPUT_SRC_ASP1RX1 0x08 @@ -446,6 +449,7 @@ int cs35l56_cal_set_status_get(struct cs35l56_base *cs35l56_base, int cs35l56_read_prot_status(struct cs35l56_base *cs35l56_base, bool *fw_missing, unsigned int *fw_version); void cs35l56_warn_if_firmware_missing(struct cs35l56_base *cs35l56_base); +bool cs35l56_needs_wait_for_firmware_timer_expiry(struct cs35l56_base *cs35l56_base); void cs35l56_log_tuning(struct cs35l56_base *cs35l56_base, struct cs_dsp *cs_dsp); int cs35l56_hw_init(struct cs35l56_base *cs35l56_base); int cs35l56_get_speaker_id(struct cs35l56_base *cs35l56_base); diff --git a/include/sound/hwdep.h b/include/sound/hwdep.h index b0da633184cd..7357727f0660 100644 --- a/include/sound/hwdep.h +++ b/include/sound/hwdep.h @@ -57,6 +57,7 @@ struct snd_hwdep { struct mutex open_mutex; int used; /* reference counter */ + struct mutex dsp_mutex; /* serializes DSP load requests */ unsigned int dsp_loaded; /* bit fields of loaded dsp indices */ unsigned int exclusive:1; /* exclusive access mode */ }; diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 5d47d1421c1f..63a86354e822 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -871,6 +871,12 @@ enum iommufd_hwpt_get_dirty_bitmap_flags { * Walk the IOMMU pagetables for a given IOVA range to return a bitmap * with the dirty IOVAs. In doing so it will also by default clear any * dirty bit metadata set in the IOPTE. + * + * The IOVA range given must fully enclose any IOPTE. If the range starts or + * ends inside a dirty IOPTE then the ioctl will fail. This avoids partially + * reporting a dirty range. + * + * On failure the dirty state becomes unknown and the userspace must restart. */ struct iommu_hwpt_get_dirty_bitmap { __u32 size; diff --git a/io_uring/cancel.c b/io_uring/cancel.c index 75a21ecfe69c..6be224ab53dc 100644 --- a/io_uring/cancel.c +++ b/io_uring/cancel.c @@ -198,7 +198,8 @@ static int __io_async_cancel(struct io_cancel_data *cd, break; if (!all) return ret; - nr++; + /* poll, futex and waitid return how many they cancelled */ + nr += max(ret, 1); } while (1); /* slow path, try all io-wq's */ @@ -650,7 +651,16 @@ __cold void io_uring_cancel_generic(bool cancel_all, struct io_sq_data *sqd) } if (loop) { + /* + * A request that runs in io-wq and doesn't end makes + * each round come here. Don't hold sqd->lock for that + * long either. + */ + if (sqd) + mutex_unlock(&sqd->lock); cond_resched(); + if (sqd) + mutex_lock(&sqd->lock); continue; } @@ -678,8 +688,20 @@ __cold void io_uring_cancel_generic(bool cancel_all, struct io_sq_data *sqd) * avoids a race where a completion comes in before we did * prepare_to_wait(). */ - if (inflight == tctx_inflight(tctx, !cancel_all)) + if (inflight == tctx_inflight(tctx, !cancel_all)) { + /* + * A zero-copy notification can't be canceled, and a + * peer that doesn't read keeps it pending. Don't hold + * sqd->lock while waiting for it, or whoever wants the + * lock sleeps as long. No ring is used across this, + * the next round walks ->ctx_list anew. + */ + if (sqd) + mutex_unlock(&sqd->lock); schedule(); + if (sqd) + mutex_lock(&sqd->lock); + } end_wait: finish_wait(&tctx->wait, &wait); } while (1); diff --git a/io_uring/poll.c b/io_uring/poll.c index 4f04bda42f7f..fc1800c7097b 100644 --- a/io_uring/poll.c +++ b/io_uring/poll.c @@ -128,12 +128,24 @@ static void io_poll_req_insert(struct io_kiocb *req) hlist_add_head(&req->hash_node, &table->hbs[index].list); } +#define IO_POLL_UNMASK (EPOLLERR|EPOLLHUP|EPOLLNVAL) + +static __poll_t io_poll_unmask(__poll_t events) +{ + /* + * A peer that stopped sending is an event for a reader, but a write + * only poll must stay armed until the file is writable again. + */ + if (events & (EPOLLIN | EPOLLRDNORM | EPOLLRDBAND)) + return IO_POLL_UNMASK | EPOLLRDHUP; + return IO_POLL_UNMASK; +} + static void io_init_poll_iocb(struct io_poll *poll, __poll_t events) { poll->head = NULL; -#define IO_POLL_UNMASK (EPOLLERR|EPOLLHUP|EPOLLNVAL|EPOLLRDHUP) /* mask in events that we always want/need */ - poll->events = events | IO_POLL_UNMASK; + poll->events = events | io_poll_unmask(events); INIT_LIST_HEAD(&poll->wait.entry); init_waitqueue_func_entry(&poll->wait, io_poll_wake); } @@ -727,7 +739,7 @@ int io_arm_apoll(struct io_kiocb *req, unsigned issue_flags, __poll_t mask) int io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags) { const struct io_issue_def *def = &io_issue_defs[req->opcode]; - __poll_t mask = POLLPRI | POLLERR; + __poll_t mask = EPOLLERR; if (!def->pollin && !def->pollout) return IO_APOLL_ABORTED; @@ -735,7 +747,8 @@ int io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags) return IO_APOLL_ABORTED; if (def->pollin) { - mask |= EPOLLIN | EPOLLRDNORM; + /* urgent data is an event for a reader only */ + mask |= EPOLLIN | EPOLLRDNORM | EPOLLPRI; /* If reading from MSG_ERRQUEUE using recvmsg, ignore POLLIN */ if (req->flags & REQ_F_CLEAR_POLLIN) @@ -827,12 +840,47 @@ static int io_poll_disarm(struct io_kiocb *req) return 0; } +/* + * Cancel every request matching @cd in one pass. Cancelled requests stay + * hashed until their task work runs, so finding them one at a time would + * walk past all the earlier ones on each lookup. + */ +static int io_poll_cancel_all(struct io_ring_ctx *ctx, struct io_cancel_data *cd) +{ + unsigned int nr_buckets = 1U << ctx->cancel_table.hash_bits; + unsigned int i = 0; + struct hlist_node *tmp; + struct io_kiocb *req; + int nr = 0; + + if (!(cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP | + IORING_ASYNC_CANCEL_ANY))) { + /* keyed by user_data, so only one bucket can match */ + i = hash_long(cd->data, ctx->cancel_table.hash_bits); + nr_buckets = i + 1; + } + + for (; i < nr_buckets; i++) { + struct io_hash_bucket *hb = &ctx->cancel_table.hbs[i]; + + hlist_for_each_entry_safe(req, tmp, &hb->list, hash_node) { + if (!io_cancel_req_match(req, cd)) + continue; + io_poll_cancel_req(req); + nr++; + } + } + return nr ?: -ENOENT; +} + static int __io_poll_cancel(struct io_ring_ctx *ctx, struct io_cancel_data *cd) { struct io_kiocb *req; - if (cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP | - IORING_ASYNC_CANCEL_ANY)) + if (cd->flags & (IORING_ASYNC_CANCEL_ALL | IORING_ASYNC_CANCEL_ANY)) + return io_poll_cancel_all(ctx, cd); + + if (cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP)) req = io_poll_file_find(ctx, cd); else req = io_poll_find(ctx, false, cd); @@ -971,7 +1019,7 @@ int io_poll_remove(struct io_kiocb *req, unsigned int issue_flags) poll->events &= ~0xffff; poll->events |= poll_update->events & 0xffff; - poll->events |= IO_POLL_UNMASK; + poll->events |= io_poll_unmask(poll->events); } if (poll_update->update_user_data) preq->cqe.user_data = poll_update->new_user_data; diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index d69baa47c7e7..c1ab760c4591 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -521,11 +521,12 @@ static int io_files_update_with_index_alloc(struct io_kiocb *req, { struct io_rsrc_update *up = io_kiocb_to_cmd(req, struct io_rsrc_update); __s32 __user *fds = u64_to_user_ptr(up->arg); + struct io_ring_ctx *ctx = req->ctx; unsigned int done; struct file *file; int ret, fd; - if (!req->ctx->file_table.data.nr) + if (!ctx->file_table.data.nr) return -ENXIO; for (done = 0; done < up->nr_args; done++) { @@ -539,15 +540,25 @@ static int io_files_update_with_index_alloc(struct io_kiocb *req, ret = -EBADF; break; } - ret = io_fixed_fd_install(req, issue_flags, file, - IORING_FILE_INDEX_ALLOC); - if (ret < 0) + /* + * Hold the lock across the put_user(), if it's dropped the + * slot can be closed and reused before the error path runs. + */ + io_ring_submit_lock(ctx, issue_flags); + ret = __io_fixed_fd_install(ctx, file, + IORING_FILE_INDEX_ALLOC); + if (ret < 0) { + io_ring_submit_unlock(ctx, issue_flags); + fput(file); break; + } if (put_user(ret, &fds[done])) { - __io_close_fixed(req->ctx, issue_flags, ret); + io_fixed_fd_remove(ctx, ret); + io_ring_submit_unlock(ctx, issue_flags); ret = -EFAULT; break; } + io_ring_submit_unlock(ctx, issue_flags); } if (done) diff --git a/io_uring/sqpoll.c b/io_uring/sqpoll.c index 49c52284ecdd..d387488db2de 100644 --- a/io_uring/sqpoll.c +++ b/io_uring/sqpoll.c @@ -29,6 +29,7 @@ enum { IO_SQ_THREAD_SHOULD_STOP = 0, IO_SQ_THREAD_SHOULD_PARK, + IO_SQ_THREAD_EXITING, }; void io_sq_thread_unpark(struct io_sq_data *sqd) @@ -348,6 +349,18 @@ static int io_sq_thread(void *data) timeout = jiffies + sqd->sq_thread_idle; } + /* + * The SQPOLL thread never returns to userspace, so do the + * return-to-userspace work, like memory.high reclaim, here. + * It can sleep for a long time, so drop sqd->lock. + */ + if (test_thread_flag(TIF_NOTIFY_RESUME)) { + mutex_unlock(&sqd->lock); + resume_user_mode_work(NULL); + mutex_lock(&sqd->lock); + sqd->sq_cpu = raw_smp_processor_id(); + } + cap_entries = !list_is_singular(&sqd->ctx_list); list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) { int ret; @@ -433,6 +446,13 @@ static int io_sq_thread(void *data) if (io_sq_tw_pending()) io_sq_tw(UINT_MAX); + /* + * Cancelation drops sqd->lock while it waits, and a ring could attach + * to a thread that is on its way out. A fatal signal doesn't set the + * stop bit, so mark the thread as exiting here for + * io_sq_offload_create() to see. + */ + set_bit(IO_SQ_THREAD_EXITING, &sqd->state); io_uring_cancel_generic(true, sqd); rcu_assign_pointer(sqd->thread, NULL); put_task_struct(current); @@ -502,7 +522,11 @@ __cold int io_sq_offload_create(struct io_ring_ctx *ctx, list_add(&ctx->sqd_list, &sqd->ctx_list); io_sqd_update_thread_idle(sqd); /* don't attach to a dying SQPOLL thread, would be racy */ - ret = (attached && !sqd->thread) ? -ENXIO : 0; + ret = 0; + if (attached && (!sqd->thread || + test_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state) || + test_bit(IO_SQ_THREAD_EXITING, &sqd->state))) + ret = -ENXIO; io_sq_thread_unpark(sqd); if (ret < 0) diff --git a/kernel/events/core.c b/kernel/events/core.c index c4217c996f5a..9b90b6831de7 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7634,6 +7634,15 @@ static void perf_sigtrap(struct perf_event *event) if (event->ctx->task == TASK_TOMBSTONE) return; + /* + * The event was removed after this signal was queued, e.g. by exec() + * (remove_on_exec), which may also have closed its fd (close-on-exec). + * The new program has default signal handlers, so a SIGTRAP would + * kill it. + */ + if (READ_ONCE(event->state) <= PERF_EVENT_STATE_EXIT) + return; + /* * We'd expect this to only occur if the irq_work is delayed and either * ctx->task or current has changed in the meantime. This can be the diff --git a/kernel/fork.c b/kernel/fork.c index 8ad34646962d..48d7ca064404 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1535,12 +1535,19 @@ static struct mm_struct *dup_mm(struct task_struct *tsk, if (!mm) goto fail_nomem; + uprobe_start_dup_mmap(); + if (mmap_write_lock_killable(oldmm)) { + free_mm(mm); + goto fail_uprobe; + } + memcpy(mm, oldmm, sizeof(*mm)); - if (!mm_init(mm, tsk)) - goto fail_nomem; + if (!mm_init(mm, tsk)) { + mmap_write_unlock(oldmm); + goto fail_uprobe; + } - uprobe_start_dup_mmap(); err = dup_mmap(mm, oldmm); if (err) goto free_pt; @@ -1559,9 +1566,10 @@ static struct mm_struct *dup_mm(struct task_struct *tsk, mm->binfmt = NULL; mm_init_owner(mm, NULL); mmput(mm); - if (err) - uprobe_end_dup_mmap(); - + if (!err) + return NULL; +fail_uprobe: + uprobe_end_dup_mmap(); fail_nomem: return NULL; } diff --git a/kernel/panic.c b/kernel/panic.c index cec271601935..f08453f0d8b4 100644 --- a/kernel/panic.c +++ b/kernel/panic.c @@ -893,7 +893,7 @@ static __init int alloc_taint_buf(void) size += 1; /* For NULL terminator */ - buf = kmalloc(size, GFP_KERNEL); + buf = kzalloc(size, GFP_KERNEL); if (!buf) { panic("Failed to allocate taint string buffer"); diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b35e23f93bcd..913630c0b059 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7021,8 +7021,8 @@ static void dequeue_throttled_task(struct task_struct *p, int flags) WARN_ON_ONCE(p->se.on_rq); list_del_init(&p->throttle_node); - /* task blocked after throttled */ - if (flags & DEQUEUE_SLEEP) { + /* Task blocked after throttling or is leaving the fair class: */ + if (flags & (DEQUEUE_SLEEP | DEQUEUE_CLASS)) { p->throttled = false; return; } diff --git a/lib/assoc_array.c b/lib/assoc_array.c index b6c9723e12ce..3a7cddb1290b 100644 --- a/lib/assoc_array.c +++ b/lib/assoc_array.c @@ -470,33 +470,32 @@ static bool assoc_array_insert_in_empty_tree(struct assoc_array_edit *edit) } /* - * Handle insertion into a terminal node. + * The pointer that leads to a node or shortcut with the given back pointer + * and parent slot: the root, a slot in a node, or a shortcut's next node. */ -static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, - const struct assoc_array_ops *ops, - const void *index_key, - struct assoc_array_walk_result *result) +static struct assoc_array_ptr **assoc_array_parent_link(struct assoc_array *array, + struct assoc_array_ptr *back_pointer, + int parent_slot) { - struct assoc_array_shortcut *shortcut, *new_s0; - struct assoc_array_node *node, *new_n0, *new_n1, *side; - struct assoc_array_ptr *ptr; - unsigned long dissimilarity, base_seg, blank; - size_t keylen; - bool have_meta; - int level, diff; - int slot, next_slot, free_slot, i, j; - - node = result->terminal_node.node; - level = result->terminal_node.level; - edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = result->terminal_node.slot; + if (!back_pointer) + return &array->root; + if (assoc_array_ptr_is_node(back_pointer)) + return &assoc_array_ptr_to_node(back_pointer)->slots[parent_slot]; + return &assoc_array_ptr_to_shortcut(back_pointer)->next_node; +} - pr_devel("-->%s()\n", __func__); +/* + * Insert into a terminal node without changing its shape, if that can be + * done: in place of a leaf with the same index key, or in a free slot. + */ +static bool assoc_array_insert_in_node(struct assoc_array_edit *edit, + const struct assoc_array_ops *ops, + const void *index_key, + struct assoc_array_node *node) +{ + struct assoc_array_ptr *ptr; + int free_slot, i; - /* We arrived at a node which doesn't have an onward node or shortcut - * pointer that we have to follow. This means that (a) the leaf we - * want must go here (either by insertion or replacement) or (b) we - * need to split this node and insert in one of the fragments. - */ free_slot = -1; /* Firstly, we have to check the leaves in this node to see if there's @@ -514,7 +513,6 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, pr_devel("replace in slot %d\n", i); edit->leaf_p = &node->slots[i]; edit->dead_leaf = node->slots[i]; - pr_devel("<--%s() = ok [replace]\n", __func__); return true; } } @@ -526,28 +524,28 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, pr_devel("insert in free slot %d\n", free_slot); edit->leaf_p = &node->slots[free_slot]; edit->adjust_count_on = node; - pr_devel("<--%s() = ok [insert]\n", __func__); return true; } - /* The node has no spare slots - so we're either going to have to split - * it or insert another node before it. - * - * Whatever, we're going to need at least two new nodes - so allocate - * those now. We may also need a new shortcut, but we deal with that - * when we need it. - */ - new_n0 = kzalloc_obj(struct assoc_array_node); - if (!new_n0) - return false; - edit->new_meta[0] = assoc_array_node_to_ptr(new_n0); - new_n1 = kzalloc_obj(struct assoc_array_node); - if (!new_n1) - return false; - edit->new_meta[1] = assoc_array_node_to_ptr(new_n1); + return false; +} + +/* + * Fill in the edit script's segment cache for a full node: the segment at + * this level of each leaf's index key, or 0xff for a metadata pointer. + * The new key's segment, in the last entry, is left to the caller. Returns + * true if there are any metadata pointers. + */ +static bool assoc_array_fill_segment_cache(struct assoc_array_edit *edit, + const struct assoc_array_ops *ops, + const struct assoc_array_node *node, + int level) +{ + struct assoc_array_ptr *ptr; + unsigned long base_seg; + bool have_meta; + int i; - /* We need to find out how similar the leaves are. */ - pr_devel("no spare slots\n"); have_meta = false; for (i = 0; i < ASSOC_ARRAY_FAN_OUT; i++) { ptr = node->slots[i]; @@ -561,65 +559,42 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK; edit->segment_cache[i] = base_seg & ASSOC_ARRAY_FAN_MASK; } + return have_meta; +} - if (have_meta) { - pr_devel("have meta\n"); - goto split_node; - } - - /* The node contains only leaves */ - dissimilarity = 0; - base_seg = edit->segment_cache[0]; - for (i = 1; i < ASSOC_ARRAY_FAN_OUT; i++) - dissimilarity |= edit->segment_cache[i] ^ base_seg; - - pr_devel("only leaves; dissimilarity=%lx\n", dissimilarity); +/* + * Whether the leaves of a full node and the new key all share one segment, + * so that splitting the node cannot make room. + */ +static bool assoc_array_all_one_segment(const struct assoc_array_edit *edit) +{ + int i; - if ((dissimilarity & ASSOC_ARRAY_FAN_MASK) == 0) { - /* The old leaves all cluster in the same slot. We will need - * to insert a shortcut if the new node wants to cluster with them. - */ - if ((edit->segment_cache[ASSOC_ARRAY_FAN_OUT] ^ base_seg) == 0) - goto all_leaves_cluster_together; + for (i = 1; i < ASSOC_ARRAY_FAN_OUT + 1; i++) + if (edit->segment_cache[i] != edit->segment_cache[0]) + return false; + return true; +} - /* Otherwise all the old leaves cluster in the same slot, but - * the new leaf wants to go into a different slot - so we - * create a new node (n0) to hold the new leaf and a pointer to - * a new node (n1) holding all the old leaves. - * - * This can be done by falling through to the node splitting - * path. - */ - pr_devel("present leaves cluster but not new leaf\n"); - } +/* + * Split a full node into new_n0, which takes its place, and new_n1, below + * new_n0 in the slot of a segment that at least two of the leaves share, + * the new one included. The caller has filled in the segment cache and + * set new_n0's back pointer and parent slot. + */ +static void assoc_array_split_node(struct assoc_array_edit *edit, + struct assoc_array_node *node, + struct assoc_array_node *new_n0, + struct assoc_array_node *new_n1) +{ + struct assoc_array_shortcut *shortcut; + struct assoc_array_node *side; + struct assoc_array_ptr *ptr; + int slot, next_slot, free_slot, i, j; -split_node: pr_devel("split node\n"); - /* We need to split the current node. The node must contain anything - * from a single leaf (in the one leaf case, this leaf will cluster - * with the new leaf) and the rest meta-pointers, to all leaves, some - * of which may cluster. - * - * It won't contain the case in which all the current leaves plus the - * new leaves want to cluster in the same slot. - * - * We need to expel at least two leaves out of a set consisting of the - * leaves in the node and the new leaf. The current meta pointers can - * just be copied as they shouldn't cluster with any of the leaves. - * - * We need a new node (n0) to replace the current one and a new node to - * take the expelled nodes (n1). - */ - edit->set[0].to = assoc_array_node_to_ptr(new_n0); - new_n0->back_pointer = node->back_pointer; - new_n0->parent_slot = node->parent_slot; new_n1->back_pointer = assoc_array_node_to_ptr(new_n0); - new_n1->parent_slot = -1; /* Need to calculate this */ - -do_split_node: - pr_devel("do_split_node\n"); - new_n0->nr_leaves_on_branch = node->nr_leaves_on_branch; new_n1->nr_leaves_on_branch = 0; @@ -699,26 +674,29 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, } } } +} - ptr = node->back_pointer; - if (!ptr) - edit->set[0].ptr = &edit->array->root; - else if (assoc_array_ptr_is_node(ptr)) - edit->set[0].ptr = &assoc_array_ptr_to_node(ptr)->slots[node->parent_slot]; - else - edit->set[0].ptr = &assoc_array_ptr_to_shortcut(ptr)->next_node; - edit->excised_meta[0] = assoc_array_node_to_ptr(node); - pr_devel("<--%s() = ok [split node]\n", __func__); - return true; +/* + * All the leaves, new and old, want to cluster together in this node in the + * same slot, so we have to replace this node with a shortcut to skip over + * the identical parts of the key, leading to new_n0. If the node is the + * target of a shortcut, the new one replaces that shortcut as well. The + * caller then splits the leaves between new_n0 and new_n1 at the shortcut's + * far end, for which this fills in the segment cache. + */ +static struct assoc_array_shortcut * +assoc_array_insert_shortcut(struct assoc_array_edit *edit, + const struct assoc_array_ops *ops, + const void *index_key, + struct assoc_array_node *node, int level, + struct assoc_array_node *new_n0) +{ + struct assoc_array_shortcut *new_s0; + unsigned long base_seg, blank; + size_t keylen; + int diff, i; -all_leaves_cluster_together: - /* All the leaves, new and old, want to cluster together in this node - * in the same slot, so we have to replace this node with a shortcut to - * skip over the identical parts of the key and then place a pair of - * nodes, one inside the other, at the end of the shortcut and - * distribute the keys between them. - * - * Firstly we need to work out where the leaves start diverging as a + /* Firstly we need to work out where the leaves start diverging as a * bit position into their keys so that we know how big the shortcut * needs to be. * @@ -744,17 +722,28 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, new_s0 = kzalloc_flex(*new_s0, index_key, keylen); if (!new_s0) - return false; + return NULL; edit->new_meta[2] = assoc_array_shortcut_to_ptr(new_s0); - edit->set[0].to = assoc_array_shortcut_to_ptr(new_s0); new_s0->back_pointer = node->back_pointer; new_s0->parent_slot = node->parent_slot; + + /* A shortcut has to lead to a node, so a node at the end of a shortcut + * is replaced together with that shortcut. The new one holds the index + * key from the root, so it covers what the old one did. + */ + if (node->back_pointer && + assoc_array_ptr_is_shortcut(node->back_pointer)) { + struct assoc_array_shortcut *above = + assoc_array_ptr_to_shortcut(node->back_pointer); + + new_s0->back_pointer = above->back_pointer; + new_s0->parent_slot = above->parent_slot; + edit->excised_meta[1] = node->back_pointer; + } new_s0->next_node = assoc_array_node_to_ptr(new_n0); new_n0->back_pointer = assoc_array_shortcut_to_ptr(new_s0); new_n0->parent_slot = 0; - new_n1->back_pointer = assoc_array_node_to_ptr(new_n0); - new_n1->parent_slot = -1; /* Need to calculate this */ new_s0->skip_to_level = level = diff & ~ASSOC_ARRAY_LEVEL_STEP_MASK; pr_devel("skip_to_level = %d [diff %d]\n", level, diff); @@ -773,18 +762,104 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, /* This now reduces to a node splitting exercise for which we'll need * to regenerate the disparity table. */ - for (i = 0; i < ASSOC_ARRAY_FAN_OUT; i++) { - ptr = node->slots[i]; - base_seg = ops->get_object_key_chunk(assoc_array_ptr_to_leaf(ptr), - level); - base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK; - edit->segment_cache[i] = base_seg & ASSOC_ARRAY_FAN_MASK; - } - + assoc_array_fill_segment_cache(edit, ops, node, level); base_seg = ops->get_key_chunk(index_key, level); base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK; edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = base_seg & ASSOC_ARRAY_FAN_MASK; - goto do_split_node; + return new_s0; +} + +/* + * Handle insertion into a terminal node. + */ +static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, + const struct assoc_array_ops *ops, + const void *index_key, + struct assoc_array_walk_result *result) +{ + struct assoc_array_shortcut *new_s0; + struct assoc_array_node *node, *new_n0, *new_n1; + bool have_meta; + int level; + + node = result->terminal_node.node; + level = result->terminal_node.level; + + pr_devel("-->%s()\n", __func__); + + /* We arrived at a node which doesn't have an onward node or shortcut + * pointer that we have to follow. This means that (a) the leaf we + * want must go here (either by insertion or replacement) or (b) we + * need to split this node and insert in one of the fragments. + */ + if (assoc_array_insert_in_node(edit, ops, index_key, node)) { + pr_devel("<--%s() = ok [in node]\n", __func__); + return true; + } + + /* The node has no spare slots - so we're either going to have to split + * it or insert another node before it. + * + * Whatever, we're going to need at least two new nodes - so allocate + * those now. We may also need a new shortcut, but we deal with that + * when we need it. + */ + new_n0 = kzalloc_obj(struct assoc_array_node); + if (!new_n0) + return false; + edit->new_meta[0] = assoc_array_node_to_ptr(new_n0); + new_n1 = kzalloc_obj(struct assoc_array_node); + if (!new_n1) + return false; + edit->new_meta[1] = assoc_array_node_to_ptr(new_n1); + + /* We need to find out how similar the leaves are. */ + pr_devel("no spare slots\n"); + have_meta = assoc_array_fill_segment_cache(edit, ops, node, level); + edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = result->terminal_node.slot; + + if (!have_meta && assoc_array_all_one_segment(edit)) { + /* The node only contains leaves and all the leaves cluster in + * the same slot, so we need to insert a shortcut to skip the + * common portion of the key and split the node at the + * shortcut's far end. The shortcut goes wherever its back + * pointer says, which may be further up than the node. + */ + new_s0 = assoc_array_insert_shortcut(edit, ops, index_key, + node, level, new_n0); + if (!new_s0) + return false; + edit->set[0].ptr = assoc_array_parent_link(edit->array, + new_s0->back_pointer, + new_s0->parent_slot); + edit->set[0].to = assoc_array_shortcut_to_ptr(new_s0); + } else { + /* We need to split the current node. The node must contain + * anything from a single leaf (in the one leaf case, this leaf + * will cluster with the new leaf) and the rest meta-pointers, + * to all leaves, some of which may cluster. + * + * We need to expel at least two leaves out of a set consisting + * of the leaves in the node and the new leaf. The current meta + * pointers can just be copied as they shouldn't cluster with + * any of the leaves. If all the old leaves cluster in one slot + * and the new leaf does not, they all go and the new leaf stays. + * + * We need a new node (n0) to replace the current one and a new + * node to take the expelled nodes (n1). + */ + new_n0->back_pointer = node->back_pointer; + new_n0->parent_slot = node->parent_slot; + edit->set[0].ptr = assoc_array_parent_link(edit->array, + new_n0->back_pointer, + new_n0->parent_slot); + edit->set[0].to = assoc_array_node_to_ptr(new_n0); + } + + assoc_array_split_node(edit, node, new_n0, new_n1); + edit->excised_meta[0] = assoc_array_node_to_ptr(node); + pr_devel("<--%s() = ok [split node]\n", __func__); + return true; } /* @@ -843,7 +918,8 @@ static bool assoc_array_insert_mid_shortcut(struct assoc_array_edit *edit, * zero length - otherwise we just connect the new node directly to the * parent. */ - level += ASSOC_ARRAY_LEVEL_STEP; + if (shortcut->back_pointer) + level += ASSOC_ARRAY_LEVEL_STEP; if (diff > level) { pr_devel("pre-shortcut %d...%d\n", level, diff); keylen = round_up(diff, ASSOC_ARRAY_KEY_CHUNK_SIZE); @@ -865,9 +941,11 @@ static bool assoc_array_insert_mid_shortcut(struct assoc_array_edit *edit, memcpy(new_s0->index_key, shortcut->index_key, flex_array_size(new_s0, index_key, keylen)); - blank = ULONG_MAX << (diff & ASSOC_ARRAY_KEY_CHUNK_MASK); - pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, diff, blank); - new_s0->index_key[keylen - 1] &= ~blank; + if (diff & ASSOC_ARRAY_KEY_CHUNK_MASK) { + blank = ULONG_MAX << (diff & ASSOC_ARRAY_KEY_CHUNK_MASK); + pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, diff, blank); + new_s0->index_key[keylen - 1] &= ~blank; + } } else { pr_devel("no pre-shortcut\n"); edit->set[0].to = assoc_array_node_to_ptr(new_n0); @@ -1482,16 +1560,25 @@ int assoc_array_gc(struct assoc_array *array, cursor = array->root; descend: - /* If this point is a shortcut, then we need to duplicate it and - * advance the target cursor. + /* The node at this position is duplicated, and so is the shortcut in + * front of it if there is one. The node is allocated first: the copy + * of the shortcut starts out pointing at the old node, and must not be + * in the new tree if the node cannot be allocated, or the cleanup + * would follow it into the old tree. */ + new_n = kzalloc_obj(struct assoc_array_node); + if (!new_n) + goto enomem; + if (assoc_array_ptr_is_shortcut(cursor)) { shortcut = assoc_array_ptr_to_shortcut(cursor); keylen = round_up(shortcut->skip_to_level, ASSOC_ARRAY_KEY_CHUNK_SIZE); keylen >>= ASSOC_ARRAY_KEY_CHUNK_SHIFT; new_s = kmalloc_flex(*new_s, index_key, keylen); - if (!new_s) + if (!new_s) { + kfree(new_n); goto enomem; + } pr_devel("dup shortcut %p -> %p\n", shortcut, new_s); memcpy(new_s, shortcut, struct_size(new_s, index_key, keylen)); new_s->back_pointer = new_parent; @@ -1501,11 +1588,7 @@ int assoc_array_gc(struct assoc_array *array, cursor = shortcut->next_node; } - /* Duplicate the node at this position */ node = assoc_array_ptr_to_node(cursor); - new_n = kzalloc_obj(struct assoc_array_node); - if (!new_n) - goto enomem; pr_devel("dup node %p -> %p\n", node, new_n); new_n->back_pointer = new_parent; new_n->parent_slot = node->parent_slot; diff --git a/lib/base64.c b/lib/base64.c index 325c7332b049..e46be3a55585 100644 --- a/lib/base64.c +++ b/lib/base64.c @@ -52,11 +52,14 @@ static const char base64_tables[][65] = { INIT_32(0x60, ch_62, ch_63), \ [0x80 ... 0xff] = -1 } +__diag_push(); +__diag_ignore(clang, all, "-Wconstant-conversion", "https://github.com/llvm/llvm-project/issues/223923"); static const s8 base64_rev_maps[][256] = { [BASE64_STD] = BASE64_REV_INIT('+', '/'), [BASE64_URLSAFE] = BASE64_REV_INIT('-', '_'), [BASE64_IMAP] = BASE64_REV_INIT('+', ',') }; +__diag_pop(); #undef BASE64_REV_INIT #undef INIT_32 diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig index 56f308dd0273..83b4a211c15d 100644 --- a/lib/crypto/Kconfig +++ b/lib/crypto/Kconfig @@ -237,7 +237,7 @@ config CRYPTO_LIB_SHA1_ARCH depends on CRYPTO_LIB_SHA1 && !UML default y if ARM default y if ARM64 - default y if MIPS && CPU_CAVIUM_OCTEON + default y if MIPS && CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN default y if PPC default y if S390 default y if SPARC64 @@ -255,7 +255,7 @@ config CRYPTO_LIB_SHA256_ARCH depends on CRYPTO_LIB_SHA256 && !UML default y if ARM && !CPU_V7M default y if ARM64 - default y if MIPS && CPU_CAVIUM_OCTEON + default y if MIPS && CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN default y if PPC && SPE default y if RISCV && 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \ RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS @@ -275,7 +275,7 @@ config CRYPTO_LIB_SHA512_ARCH depends on CRYPTO_LIB_SHA512 && !UML default y if ARM && !CPU_V7M default y if ARM64 - default y if MIPS && CPU_CAVIUM_OCTEON + default y if MIPS && CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN default y if RISCV && 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \ RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS default y if S390 diff --git a/lib/crypto/mldsa.c b/lib/crypto/mldsa.c index c96fddc4e7dc..80e0e63b418a 100644 --- a/lib/crypto/mldsa.c +++ b/lib/crypto/mldsa.c @@ -306,7 +306,7 @@ static const u8 *decode_t1_elem(struct mldsa_ring_elem *out, * Reference: FIPS 204 Algorithm 27, sigDecode. * * This also validates that the coefficients of z are in range, corresponding - * the infinity norm check at the end of Algorithm 8, ML-DSA.Verify_internal. + * to the infinity norm check at the end of Algorithm 8, ML-DSA.Verify_internal. * * Finally, this also converts z to NTT form. */ @@ -528,15 +528,21 @@ static size_t encode_w1(u8 out[MAX_W1_ENCODED_LEN], int mldsa_verify(enum mldsa_alg alg, const u8 *sig, size_t sig_len, const u8 *msg, size_t msg_len, const u8 *pk, size_t pk_len) { - const struct mldsa_parameter_set *params = &mldsa_parameter_sets[alg]; - const int k = params->k, l = params->l; + const struct mldsa_parameter_set *params; + int k, l; /* For now this just does pure ML-DSA with an empty context string. */ static const u8 msg_prefix[2] = { /* dom_sep= */ 0, /* ctx_len= */ 0 }; const u8 *ctilde; /* The signer's commitment hash */ - const u8 *t1_encoded = &pk[RHO_LEN]; /* Next encoded element of t_1 */ + const u8 *t1_encoded; /* Next encoded element of t_1 */ u8 *h; /* The signer's hint vector, length k * N */ size_t w1_enc_len; + if (alg != MLDSA44 && alg != MLDSA65 && alg != MLDSA87) + return -EINVAL; + params = &mldsa_parameter_sets[alg]; + k = params->k; + l = params->l; + /* Validate the public key and signature lengths. */ if (pk_len != params->pk_len || sig_len != params->sig_len) return -EBADMSG; @@ -587,6 +593,7 @@ int mldsa_verify(enum mldsa_alg alg, const u8 *sig, size_t sig_len, * The computation is the same for each of the k rows. Just do each row * before moving on to the next, resulting in only one loop over k. */ + t1_encoded = &pk[RHO_LEN]; for (int i = 0; i < k; i++) { /* * tmp = NTT(A) * NTT(z) * 2^-32 diff --git a/lib/crypto/tests/mldsa_kunit.c b/lib/crypto/tests/mldsa_kunit.c index ec64e10c97f2..d0ee704a3694 100644 --- a/lib/crypto/tests/mldsa_kunit.c +++ b/lib/crypto/tests/mldsa_kunit.c @@ -372,6 +372,14 @@ static void test_mldsa_use_hint(struct kunit *test) } } +static void test_mldsa_invalid_alg(struct kunit *test) +{ + KUNIT_ASSERT_EQ(test, -EINVAL, + mldsa_verify(MLDSA44 - 1, NULL, 0, NULL, 0, NULL, 0)); + KUNIT_ASSERT_EQ(test, -EINVAL, + mldsa_verify(MLDSA87 + 1, NULL, 0, NULL, 0, NULL, 0)); +} + static void benchmark_mldsa(struct kunit *test, const struct mldsa_testvector *tv) { @@ -414,6 +422,7 @@ static struct kunit_case mldsa_kunit_cases[] = { KUNIT_CASE(test_mldsa65), KUNIT_CASE(test_mldsa87), KUNIT_CASE(test_mldsa_use_hint), + KUNIT_CASE(test_mldsa_invalid_alg), KUNIT_CASE(benchmark_mldsa44), KUNIT_CASE(benchmark_mldsa65), KUNIT_CASE(benchmark_mldsa87), diff --git a/mm/cma.c b/mm/cma.c index a10ea37a261d..9ec1d1fd5dff 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -381,6 +381,8 @@ static int __init cma_fixed_reserve(phys_addr_t base, phys_addr_t size) return -EBUSY; } + accept_memory(base, size); + return 0; } @@ -678,6 +680,8 @@ int __init cma_declare_contiguous_multi(phys_addr_t total_size, break; } + accept_memory(mlp->base, size); + pr_debug("created region %d: %016llx - %016llx\n", nr, (u64)mlp->base, (u64)mlp->base + size); cmrp = &cma->ranges[nr++]; @@ -1020,7 +1024,7 @@ bool cma_release(struct cma *cma, const struct page *pages, unsigned long count) { struct cma_memrange *cmr; - unsigned long ret = 0; + unsigned long leaked = 0; unsigned long i, pfn; cmr = find_cma_memrange(cma, pages, count); @@ -1029,9 +1033,13 @@ bool cma_release(struct cma *cma, const struct page *pages, pfn = page_to_pfn(pages); for (i = 0; i < count; i++, pfn++) - ret += !put_page_testzero(pfn_to_page(pfn)); + leaked += !page_ref_freeze(pfn_to_page(pfn), 1); - WARN(ret, "%lu pages are still in use!\n", ret); + if (leaked) { + pr_warn_ratelimited("%lu pages are still in use, not freeing CMA region!\n", + leaked); + return true; + } __cma_release_frozen(cma, cmr, pages, count); diff --git a/mm/damon/core.c b/mm/damon/core.c index e018635774be..85bcb6da9ad0 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -3200,6 +3200,26 @@ static void damos_trace_esz(struct damon_ctx *c, struct damos *s, trace_damos_esz(cidx, sidx, quota->esz); } +static void damos_fix_stale_charge_pointers(struct damon_ctx *c, + struct damos *s) +{ + struct damos_quota *quota; + struct damon_target *t; + + quota = &s->quota; + if (!quota->charge_target_from) + return; + damon_for_each_target(t, c) { + if (t == quota->charge_target_from) { + if (damon_nr_regions(t) > 0) + return; + break; + } + } + quota->charge_target_from = NULL; + quota->charge_addr_from = 0; +} + static void damos_adjust_quota(struct damon_ctx *c, struct damos *s) { struct damos_quota *quota = &s->quota; @@ -3235,6 +3255,8 @@ static void damos_adjust_quota(struct damon_ctx *c, struct damos *s) damos_trace_esz(c, s, quota); } + damos_fix_stale_charge_pointers(c, s); + if (!c->ops.get_scheme_score) return; @@ -3546,12 +3568,13 @@ static void damon_split_regions_of(struct damon_ctx *ctx, static void damon_split_some_regions(struct damon_ctx *ctx, unsigned long split_step) { - static unsigned long rotation; struct damon_target *t; struct damon_region *r, *next; - unsigned long offset = rotation++ % split_step; + unsigned long offset; unsigned long idx = 0; + offset = ctx->region_split_rotation++ % split_step; + damon_for_each_target(t, ctx) { damon_for_each_region_safe(r, next, t) { unsigned long sz_region, sz_sub; @@ -3588,7 +3611,6 @@ static void kdamond_split_regions(struct damon_ctx *ctx) struct damon_target *t; unsigned long nr_regions = 0; unsigned long max_nr_regions = ctx->attrs.max_nr_regions; - static unsigned long last_nr_regions; int nr_subregions = 2; damon_for_each_target(t, ctx) @@ -3604,7 +3626,7 @@ static void kdamond_split_regions(struct damon_ctx *ctx) } /* Maybe the middle of the region has different access frequency */ - if (last_nr_regions == nr_regions && + if (ctx->last_nr_regions == nr_regions && nr_regions < max_nr_regions / 3) nr_subregions = 3; @@ -3613,7 +3635,7 @@ static void kdamond_split_regions(struct damon_ctx *ctx) ctx->min_region_sz); done: - last_nr_regions = nr_regions; + ctx->last_nr_regions = nr_regions; } /* diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index 974c456a9e09..20eb8c242f2f 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -1300,7 +1300,10 @@ static ssize_t target_metric_store(struct kobject *kobj, metric_name = &damos_sysfs_qgoal_metric_names[i]; if (sysfs_streq(buf, metric_name->name)) { + if (!mutex_trylock(&damon_sysfs_lock)) + return -EBUSY; goal->metric = metric_name->metric; + mutex_unlock(&damon_sysfs_lock); return count; } } @@ -2831,12 +2834,11 @@ static int damon_sysfs_add_scheme_filters(struct damos *scheme, } else if (filter->type == DAMOS_FILTER_TYPE_TARGET) { filter->target_idx = sysfs_filter->target_idx; } else if (filter->type == DAMOS_FILTER_TYPE_HUGEPAGE_SIZE) { - if (sysfs_filter->sz_range.min > - sysfs_filter->sz_range.max) { + filter->sz_range = sysfs_filter->sz_range; + if (filter->sz_range.min > filter->sz_range.max) { damos_destroy_filter(filter); return -EINVAL; } - filter->sz_range = sysfs_filter->sz_range; } damos_add_filter(scheme, filter); diff --git a/mm/filemap.c b/mm/filemap.c index 00fd89cf6f55..73a26cd5b6d5 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -227,18 +227,6 @@ void __filemap_remove_folio(struct folio *folio, void *shadow) page_cache_delete(mapping, folio, shadow); } -static void filemap_free_folio(const struct address_space *mapping, - struct folio *folio) -{ - void (*free_folio)(struct folio *); - - free_folio = mapping->a_ops->free_folio; - if (free_folio) - free_folio(folio); - - folio_put_refs(folio, folio_nr_pages(folio)); -} - /** * filemap_remove_folio - Remove folio from page cache. * @folio: The folio. @@ -250,6 +238,7 @@ static void filemap_free_folio(const struct address_space *mapping, void filemap_remove_folio(struct folio *folio) { struct address_space *mapping = folio->mapping; + void (*free_folio)(struct folio *folio); BUG_ON(!folio_test_locked(folio)); spin_lock(&mapping->host->i_lock); @@ -258,9 +247,12 @@ void filemap_remove_folio(struct folio *folio) xa_unlock_irq(&mapping->i_pages); if (mapping_shrinkable(mapping)) inode_lru_list_add(mapping->host); + free_folio = mapping->a_ops->free_folio; spin_unlock(&mapping->host->i_lock); - filemap_free_folio(mapping, folio); + if (free_folio) + free_folio(folio); + folio_put_refs(folio, folio_nr_pages(folio)); } /* @@ -320,6 +312,7 @@ static void page_cache_delete_batch(struct address_space *mapping, void delete_from_page_cache_batch(struct address_space *mapping, struct folio_batch *fbatch) { + void (*free_folio)(struct folio *folio); int i; if (!folio_batch_count(fbatch)) @@ -337,10 +330,16 @@ void delete_from_page_cache_batch(struct address_space *mapping, xa_unlock_irq(&mapping->i_pages); if (mapping_shrinkable(mapping)) inode_lru_list_add(mapping->host); + free_folio = mapping->a_ops->free_folio; spin_unlock(&mapping->host->i_lock); - for (i = 0; i < folio_batch_count(fbatch); i++) - filemap_free_folio(mapping, fbatch->folios[i]); + for (i = 0; i < folio_batch_count(fbatch); i++) { + struct folio *folio = fbatch->folios[i]; + + if (free_folio) + free_folio(folio); + folio_put_refs(folio, folio_nr_pages(folio)); + } } int filemap_check_errors(struct address_space *mapping) @@ -1375,10 +1374,22 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, * return value based on that state without races. * * Also note that WQ_FLAG_WOKEN is sufficient for a non-exclusive - * waiter, but an exclusive one requires WQ_FLAG_DONE. + * waiter, but an exclusive one must also acquire the bit. */ - if (behavior == EXCLUSIVE) - return wait->flags & WQ_FLAG_DONE ? 0 : -EINTR; + if (behavior == EXCLUSIVE) { + if (wait->flags & WQ_FLAG_DONE) + return 0; + /* + * We might have just been woken up and the waker will not + * wake up anybody else. If we ignore that now, the remaining + * waiters may not get woken up: take the bit now if it is + * still free. + */ + if ((wait->flags & WQ_FLAG_WOKEN) && + !test_and_set_bit(bit_nr, folio_flags(folio, 0))) + return 0; + return -EINTR; + } return wait->flags & WQ_FLAG_WOKEN ? 0 : -EINTR; } @@ -3279,11 +3290,11 @@ loff_t mapping_seek_hole_data(struct address_space *mapping, loff_t start, * @folio - the folio to lock. * @fpin - the pointer to the file we may pin (or is already pinned). * - * This works similar to lock_folio_or_retry in that it can drop the - * mmap_lock. It differs in that it actually returns the folio locked - * if it returns 1 and 0 if it couldn't lock the folio. If we did have - * to drop the mmap_lock then fpin will point to the pinned file and - * needs to be fput()'ed at a later point. + * Returns 1 with the folio locked, or 0 if locking failed. + * + * On both entry and return, a non-NULL *fpin means the fault lock has + * already been dropped and the caller must eventually fput() the file. + * Otherwise the fault lock is still held, including on failure. */ static int lock_folio_maybe_drop_mmap(struct vm_fault *vmf, struct folio *folio, struct file **fpin) @@ -3300,21 +3311,9 @@ static int lock_folio_maybe_drop_mmap(struct vm_fault *vmf, struct folio *folio, return 0; *fpin = maybe_unlock_mmap_for_io(vmf, *fpin); - if (vmf->flags & FAULT_FLAG_KILLABLE) { - if (__folio_lock_killable(folio)) { - /* - * We didn't have the right flags to drop the - * fault lock, but all fault_handlers only check - * for fatal signals if we return VM_FAULT_RETRY, - * so we need to drop the fault lock here and - * return 0 if we don't have a fpin. - */ - if (*fpin == NULL) - release_fault_lock(vmf); - return 0; - } - } else - __folio_lock(folio); + if (vmf->flags & FAULT_FLAG_KILLABLE) + return !__folio_lock_killable(folio); + __folio_lock(folio); return 1; } @@ -3543,7 +3542,7 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf) * vma->vm_mm->mmap_lock must be held on entry. * * If our return value has VM_FAULT_RETRY set, it's because the mmap_lock - * may be dropped before doing I/O or by lock_folio_maybe_drop_mmap(). + * may be dropped before doing I/O or when aborting an interrupted wait. * * If our return value does not have VM_FAULT_RETRY set, the mmap_lock * has not been released. @@ -3699,9 +3698,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) out_retry: /* - * We dropped the mmap_lock, we need to return to the fault handler to - * re-find the vma and come back and find our hopefully still populated - * page. + * With fpin set, the fault lock was already dropped. Otherwise it is + * still held. Release invalidate_lock before dropping either protection. */ if (!IS_ERR(folio)) folio_put(folio); @@ -3709,6 +3707,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) filemap_invalidate_unlock_shared(mapping); if (fpin) fput(fpin); + else if (!(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) + release_fault_lock(vmf); return ret | VM_FAULT_RETRY; } EXPORT_SYMBOL(filemap_fault); diff --git a/mm/gup.c b/mm/gup.c index eb898ea1ee22..4f58ad22a756 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -658,9 +658,6 @@ static struct page *follow_huge_pud(struct vm_area_struct *vma, assert_spin_locked(pud_lockptr(mm, pudp)); - if (!pud_present(pud)) - return NULL; - if ((flags & FOLL_WRITE) && !can_follow_write_pud(pud, pfn_to_page(pfn), vma, flags)) return NULL; @@ -896,15 +893,17 @@ static struct page *follow_page_pte(struct vm_area_struct *vma, } static struct page *follow_pmd_mask(struct vm_area_struct *vma, - unsigned long address, pud_t *pudp, - unsigned int flags, - unsigned long *page_mask) + unsigned long address, pud_t *pudp, pud_t pud, + unsigned int flags, unsigned long *page_mask) { pmd_t *pmd, pmdval; spinlock_t *ptl; struct page *page; struct mm_struct *mm = vma->vm_mm; + if (unlikely(pud_bad(pud))) + return no_page_table(vma, flags, address); + pmd = pmd_offset(pudp, address); pmdval = pmdp_get_lockless(pmd); if (pmd_none(pmdval)) @@ -953,18 +952,23 @@ static struct page *follow_pud_mask(struct vm_area_struct *vma, pud = pudp_get(pudp); if (!pud_present(pud)) return no_page_table(vma, flags, address); - if (pud_leaf(pud)) { - ptl = pud_lock(mm, pudp); - page = follow_huge_pud(vma, address, pudp, flags, page_mask); + if (!pud_leaf(pud)) + return follow_pmd_mask(vma, address, pudp, pud, flags, page_mask); + + ptl = pud_lock(mm, pudp); + pud = *pudp; + if (unlikely(!pud_present(pud))) { spin_unlock(ptl); - if (page) - return page; return no_page_table(vma, flags, address); } - if (unlikely(pud_bad(pud))) - return no_page_table(vma, flags, address); + if (unlikely(!pud_leaf(pud))) { + spin_unlock(ptl); + return follow_pmd_mask(vma, address, pudp, pud, flags, page_mask); + } - return follow_pmd_mask(vma, address, pudp, flags, page_mask); + page = follow_huge_pud(vma, address, pudp, flags, page_mask); + spin_unlock(ptl); + return page; } static struct page *follow_p4d_mask(struct vm_area_struct *vma, @@ -3186,6 +3190,9 @@ static int gup_fast_fallback(unsigned long start, unsigned long nr_pages, FOLL_PCI_P2PDMA | FOLL_HONOR_NUMA_FAULT))) return -EINVAL; + if (unlikely(!nr_pages)) + return 0; + if (gup_flags & FOLL_PIN) mm_set_has_pinned_flag(current->mm); @@ -3193,8 +3200,8 @@ static int gup_fast_fallback(unsigned long start, unsigned long nr_pages, might_lock_read(¤t->mm->mmap_lock); start = untagged_addr(start) & PAGE_MASK; - len = nr_pages << PAGE_SHIFT; - if (check_add_overflow(start, len, &end)) + if (check_shl_overflow(nr_pages, PAGE_SHIFT, &len) || + check_add_overflow(start, len, &end)) return -EOVERFLOW; if (end > TASK_SIZE_MAX) return -EFAULT; @@ -3439,7 +3446,7 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end, { unsigned int flags, nr_folios, nr_found; unsigned int i, pgshift = PAGE_SHIFT; - pgoff_t start_idx, end_idx; + pgoff_t start_idx, end_idx, retry_idx; struct folio *folio = NULL; struct folio_batch fbatch; struct hstate *h; @@ -3480,6 +3487,7 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end, * reason, we try to allocate them and add them to the * page cache. */ + retry_idx = start_idx; nr_found = filemap_get_folios_contig(memfd->f_mapping, &start_idx, end_idx, @@ -3492,12 +3500,38 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end, for (i = 0; i < nr_found; i++) { folio = fbatch.folios[i]; + /* + * The folio might have been removed from the + * page cache (e.g., truncated or collapsed) + * after the lookup. Recheck under the folio + * lock. + */ + folio_lock(folio); + if (unlikely(folio->mapping != memfd->f_mapping)) { + folio_unlock(folio); + start_idx = retry_idx; + break; + } + if (try_grab_folio(folio, 1, FOLL_PIN)) { + folio_unlock(folio); folio_batch_release(&fbatch); ret = -EINVAL; goto err; } + /* + * fallocate() can leave folios in the + * page cache that were never cleared. + */ + if (!folio_test_uptodate(folio)) { + folio_zero_range(folio, 0, folio_size(folio)); + flush_dcache_folio(folio); + folio_mark_uptodate(folio); + } + retry_idx = folio_next_index(folio); + folio_unlock(folio); + if (nr_folios == 0) *offset = offset_in_folio(folio, start); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 779c02e0bf6c..5f7e3320d4d0 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2073,11 +2073,13 @@ void touch_pud(struct vm_area_struct *vma, unsigned long addr, update_mmu_cache_pud(vma, addr, pud); } -int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, - pud_t *dst_pud, pud_t *src_pud, unsigned long addr, - struct vm_area_struct *vma) +int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, + pud_t *dst_pud, pud_t *src_pud, unsigned long addr) { + struct mm_struct *dst_mm = dst_vma->vm_mm; + struct mm_struct *src_mm = src_vma->vm_mm; spinlock_t *dst_ptl, *src_ptl; + struct page *page; pud_t pud; int ret; @@ -2090,11 +2092,20 @@ int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, if (unlikely(!pud_trans_huge(pud))) goto out_unlock; - /* - * TODO: once we support anonymous pages, use - * folio_try_dup_anon_rmap_*() and split if duplicating fails. - */ - if (vma_is_cow_mapping(vma) && pud_write(pud)) { + page = vm_normal_page_pud(src_vma, addr, pud); + if (page) { + struct folio *folio = page_folio(page); + + folio_get(folio); + /* + * TODO: once we support anonymous pages, use + * folio_try_dup_anon_rmap_*() and split if duplicating fails. + */ + folio_add_file_rmap_pud(folio, page, dst_vma); + add_mm_counter(dst_mm, mm_counter_file(folio), HPAGE_PUD_NR); + } + + if (vma_is_cow_mapping(src_vma) && pud_write(pud)) { pudp_set_wrprotect(src_mm, addr, src_pud); pud = pud_wrprotect(pud); } @@ -2483,11 +2494,7 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma, pmd_t pmdval, struct folio *folio, bool is_present) { - const bool is_device_private = folio_is_device_private(folio); - - /* Present and device private folios are rmappable. */ - if (is_present || is_device_private) - folio_remove_rmap_pmd(folio, &folio->page, vma); + const bool is_device_private = pmd_is_device_private_entry(pmdval); if (folio_test_anon(folio)) { add_mm_counter(mm, MM_ANONPAGES, -HPAGE_PMD_NR); @@ -2502,6 +2509,13 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma, folio_mark_accessed(folio); } + /* + * Present and device private folios are rmappable. Remove the + * rmap after transferring young+dirty to the folio. + */ + if (is_present || is_device_private) + folio_remove_rmap_pmd(folio, &folio->page, vma); + /* Device private folios are pinned. */ if (is_device_private) folio_put(folio); @@ -2645,15 +2659,11 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr, return false; } - /* - * We don't have to worry about the ordering of src and dst - * ptlocks because exclusive mmap_lock prevents deadlock. - */ - old_ptl = __pmd_trans_huge_lock(old_pmd, vma); - if (old_ptl) { - new_ptl = pmd_lockptr(mm, new_pmd); - if (new_ptl != old_ptl) - spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING); + old_ptl = pmd_lockptr(mm, old_pmd); + new_ptl = pmd_lockptr(mm, new_pmd); + + double_pt_lock(old_ptl, new_ptl); + if (likely(pmd_is_huge(*old_pmd))) { pmd = pmdp_huge_get_and_clear(mm, old_addr, old_pmd); if (pmd_present(pmd)) force_flush = true; @@ -2681,11 +2691,10 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr, set_pmd_at(mm, new_addr, new_pmd, pmd); if (force_flush) flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE); - if (new_ptl != old_ptl) - spin_unlock(new_ptl); - spin_unlock(old_ptl); + double_pt_unlock(old_ptl, new_ptl); return true; } + double_pt_unlock(old_ptl, new_ptl); return false; } @@ -3052,6 +3061,8 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma, spinlock_t *ptl; pud_t orig_pud; + tlb_change_page_size(tlb, HPAGE_PUD_SIZE); + ptl = __pud_trans_huge_lock(pud, vma); if (!ptl) return 0; @@ -3507,16 +3518,66 @@ void split_huge_pmd_address(struct vm_area_struct *vma, unsigned long address, __split_huge_pmd(vma, pmd, address, freeze); } -static inline void split_huge_pmd_if_needed(struct vm_area_struct *vma, unsigned long address) +static bool pud_split_required(const struct vm_area_struct *vma, + unsigned long address) +{ + if (!IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)) + return false; + if (IS_ALIGNED(address, HPAGE_PUD_SIZE)) + return false; + /* + * Only selected VMAs support mapping PUDs for now. Lift the + * restriction once this changes. + */ + if (!vma_is_dax(vma) && !vma_test_any(vma, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT)) + return false; + return range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PUD_SIZE), + ALIGN(address, HPAGE_PUD_SIZE)); +} + +static bool pmd_split_required(const struct vm_area_struct *vma, + unsigned long address) { + if (IS_ALIGNED(address, HPAGE_PMD_SIZE)) + return false; + + return range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PMD_SIZE), + ALIGN(address, HPAGE_PMD_SIZE)); +} + +static void split_huge_leaf(struct vm_area_struct *vma, + unsigned long address) +{ + const bool split_pud = pud_split_required(vma, address); + const bool split_pmd = pmd_split_required(vma, address); + pgd_t *pgd; + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + /* - * If the new address isn't hpage aligned and it could previously - * contain an hugepage: check if we need to split an huge pmd. + * mmap, VMA and rmap write locks protect the walk; the split helpers + * take the page table locks. */ - if (!IS_ALIGNED(address, HPAGE_PMD_SIZE) && - range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PMD_SIZE), - ALIGN(address, HPAGE_PMD_SIZE))) - split_huge_pmd_address(vma, address, false); + if (!split_pud && !split_pmd) + return; + + pgd = pgd_offset(vma->vm_mm, address); + if (!pgd_present(*pgd)) + return; + + p4d = p4d_offset(pgd, address); + if (!p4d_present(*p4d)) + return; + + pud = pud_offset(p4d, address); + if (pud_leaf(*pud)) + __split_huge_pud(vma, pud, address); + if (!split_pmd || !pud_present(*pud)) + return; + + pmd = pmd_offset(pud, address); + __split_huge_pmd(vma, pmd, address, false); } void vma_adjust_trans_huge(struct vm_area_struct *vma, @@ -3525,14 +3586,14 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma, struct vm_area_struct *next) { /* Check if we need to split start first. */ - split_huge_pmd_if_needed(vma, start); + split_huge_leaf(vma, start); /* Check if we need to split end next. */ - split_huge_pmd_if_needed(vma, end); + split_huge_leaf(vma, end); /* If we're incrementing next->vm_start, we might need to split it. */ if (next) - split_huge_pmd_if_needed(next, end); + split_huge_leaf(next, end); } static void unmap_folio(struct folio *folio) @@ -4511,8 +4572,7 @@ int split_folio_to_list(struct folio *folio, struct list_head *list) * * It is unsafe to call folio_unqueue_deferred_split() until folio refcount is * zero: because even when the list_lru lock is held, a non-empty - * _deferred_list might be in use on deferred_split_scan()'s unlocked - * on-stack list. + * _deferred_list might be in use on deferred_split_scan()'s on-stack list. * * The list_lru sublist is determined by folio's memcg: it is therefore * important to unqueue deferred split before changing folio memcg. @@ -4658,10 +4718,22 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, deferred_split_isolate, &dispose); list_for_each_entry_safe(folio, next, &dispose, _deferred_list) { + struct mem_cgroup *memcg; + struct list_lru_one *lru; bool did_split = false; bool underused = false; + /* + * Sync with deferred_split_folio(), which can requeue the + * folio once list_empty() is true, and sets PG_partially_mapped. + */ + rcu_read_lock(); + memcg = folio_memcg(folio); + lru = list_lru_lock_irq(&deferred_split_lru, folio_nid(folio), + &memcg); list_del_init(&folio->_deferred_list); + list_lru_unlock_irq(lru); + rcu_read_unlock(); if (!folio_test_partially_mapped(folio)) { /* diff --git a/mm/hugetlb.c b/mm/hugetlb.c index e71e0ab116bf..860c3c6b7673 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -125,6 +125,10 @@ static int __huge_pmd_unshare(struct mmu_gather *tlb, bool check_locks); static void hugetlb_unshare_pmds(struct vm_area_struct *vma, unsigned long start, unsigned long end, bool take_locks); +static void hugetlb_unmap_range(struct mmu_gather *tlb, + struct vm_area_struct *vma, unsigned long start, + unsigned long end, struct folio *folio, + zap_flags_t zap_flags); static struct resv_map *vma_resv_map(struct vm_area_struct *vma); static inline bool subpool_is_free(struct hugepage_subpool *spool) @@ -4935,10 +4939,11 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, raw_write_seqcount_begin(&src->write_protect_seq); } else { /* - * For shared mappings the vma lock must be held before - * calling hugetlb_walk() in the src vma. Otherwise, the - * returned ptep could go away if part of a shared pmd and - * another thread calls huge_pmd_unshare. + * Shared mappings only need copying when we could have + * information stored in page tables (e.g., userfaultfd PTE + * bits or markers), which is incompatible with PMD sharing. + * Still take the hugetlb VMA lock for now to satisfy + * hugetlb_walk()'s locking requirements. */ hugetlb_vma_lock_read(src_vma); } @@ -4946,6 +4951,13 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, last_addr_mask = hugetlb_mask_last_page(h); for (addr = src_vma->vm_start; addr < src_vma->vm_end; addr += sz) { spinlock_t *src_ptl, *dst_ptl; + + if (WARN_ON_ONCE(want_pmd_share(src_vma, addr) || + want_pmd_share(dst_vma, addr))) { + ret = -EINVAL; + break; + } + src_pte = hugetlb_walk(src_vma, addr, sz); if (!src_pte) { addr |= last_addr_mask; @@ -4957,14 +4969,6 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, break; } -#ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING - /* If the pagetables are shared, there is nothing to do */ - if (ptdesc_pmd_is_shared(virt_to_ptdesc(dst_pte))) { - addr |= last_addr_mask; - continue; - } -#endif - dst_ptl = huge_pte_lock(h, dst, dst_pte); src_ptl = huge_pte_lockptr(h, src, src_pte); spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING); @@ -5115,15 +5119,9 @@ static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr, spinlock_t *src_ptl, *dst_ptl; pte_t pte; - dst_ptl = huge_pte_lock(h, mm, dst_pte); + dst_ptl = huge_pte_lockptr(h, mm, dst_pte); src_ptl = huge_pte_lockptr(h, mm, src_pte); - - /* - * We don't have to worry about the ordering of src and dst ptlocks - * because exclusive mmap_lock (or the i_mmap_lock) prevents deadlock. - */ - if (src_ptl != dst_ptl) - spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING); + double_pt_lock(dst_ptl, src_ptl); pte = huge_ptep_get_and_clear(mm, old_addr, src_pte, sz); @@ -5152,15 +5150,12 @@ static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr, set_huge_pte_at(mm, new_addr, dst_pte, pte, sz); } - if (src_ptl != dst_ptl) - spin_unlock(src_ptl); - spin_unlock(dst_ptl); + double_pt_unlock(dst_ptl, src_ptl); } -int move_hugetlb_page_tables(struct vm_area_struct *vma, - struct vm_area_struct *new_vma, - unsigned long old_addr, unsigned long new_addr, - unsigned long len) +unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma, + struct vm_area_struct *new_vma, unsigned long old_addr, + unsigned long new_addr, unsigned long len) { struct hstate *h = hstate_vma(vma); struct address_space *mapping = vma->vm_file->f_mapping; @@ -5190,6 +5185,8 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma, for (; old_addr < old_end; old_addr += sz, new_addr += sz) { const unsigned long offset_to_last_entry = (old_addr | last_addr_mask) - old_addr; + const unsigned long offset_to_last_dst_entry = + (new_addr | last_addr_mask) - new_addr; src_pte = hugetlb_walk(vma, old_addr, sz); if (!src_pte) { @@ -5206,6 +5203,24 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma, continue; } + /* + * Avoid entering the destination PMD sharing path while holding + * i_mmap_rwsem for write. Leave destination page table + * population to the fault path, which can share the PMD lazily. + */ + if (sz == PMD_SIZE && want_pmd_share(new_vma, new_addr)) { + const unsigned long unmap_size = + min(old_end - old_addr, + offset_to_last_dst_entry + sz); + + hugetlb_unmap_range(&tlb, vma, old_addr, + old_addr + unmap_size, NULL, + ZAP_FLAG_DROP_MARKER); + old_addr += unmap_size - sz; + new_addr += unmap_size - sz; + continue; + } + dst_pte = huge_pte_alloc(mm, new_vma, new_addr, sz); if (!dst_pte) break; @@ -5225,9 +5240,10 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma, return len + old_addr - old_end; } -void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, - unsigned long start, unsigned long end, - struct folio *folio, zap_flags_t zap_flags) +static void hugetlb_unmap_range(struct mmu_gather *tlb, + struct vm_area_struct *vma, unsigned long start, + unsigned long end, struct folio *folio, + zap_flags_t zap_flags) { struct mm_struct *mm = vma->vm_mm; const bool folio_provided = !!folio; @@ -5245,13 +5261,6 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, BUG_ON(start & ~huge_page_mask(h)); BUG_ON(end & ~huge_page_mask(h)); - /* - * This is a hugetlb vma, all the pte entries should point - * to huge page. - */ - tlb_change_page_size(tlb, sz); - tlb_start_vma(tlb, vma); - last_addr_mask = hugetlb_mask_last_page(h); address = start; for (; address < end; address += sz) { @@ -5380,6 +5389,15 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, if (folio_provided) break; } +} + +void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, + unsigned long start, unsigned long end, + struct folio *folio, zap_flags_t zap_flags) +{ + tlb_change_page_size(tlb, huge_page_size(hstate_vma(vma))); + tlb_start_vma(tlb, vma); + hugetlb_unmap_range(tlb, vma, start, end, folio, zap_flags); tlb_end_vma(tlb, vma); huge_pmd_unshare_flush(tlb, vma); diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 917db0984143..8e5981888937 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -222,10 +222,12 @@ static void vmemmap_remap_pte(pte_t *pte, unsigned long addr, VM_WARN_ON_ONCE(!PageTail((const struct page *)addr)); /* - * Remap the tail pages as read-only to catch illegal write - * operation to the tail pages. + * Map the shared tail page writable. A lockless get that read + * a non-zero refcount before the remap can still do its + * cmpxchg after it. The refcount is zero, so the cmpxchg + * fails and leaves the page unchanged. */ - entry = mk_pte(walk->vmemmap_tail, PAGE_KERNEL_RO); + entry = mk_pte(walk->vmemmap_tail, PAGE_KERNEL); } list_add(&page->lru, walk->vmemmap_pages); diff --git a/mm/internal.h b/mm/internal.h index 38b1165212c9..526ad50e3c1d 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -584,14 +584,16 @@ void free_pgtables(struct mmu_gather *tlb, struct unmap_desc *desc); void pmd_install(struct mm_struct *mm, pmd_t *pmd, pgtable_t *pte); /** - * sync_with_folio_pmd_zap - sync with concurrent zapping of a folio PMD + * sync_with_folio_pmd_zap - sync with concurrent zapping of folios * @mm: The mm_struct. * @pmdp: Pointer to the pmd that was found to be pmd_none(). * * When we find a pmd_none() while unmapping a folio without holding the PTL, - * zap_huge_pmd() may have cleared the PMD but not yet modified the folio to - * indicate that it's unmapped. Skipping the PMD without synchronization could - * make folio unmapping code assume that unmapping failed. + * zap_huge_pmd() may have cleared the PMD but not yet updated the folio + * mapcount, and zap_empty_pte_table() clears the PMD only after the rmaps + * for the table have been removed, but nothing orders the pmd_none() read + * against later mapcount reads. Skipping the PMD without synchronization + * could make folio unmapping code assume that unmapping failed. * * Wait for concurrent zapping to complete by grabbing the PTL. */ @@ -619,6 +621,34 @@ static inline void force_page_cache_readahead(struct address_space *mapping, force_page_cache_ra(&ractl, nr_to_read); } +/* + * Take two page table locks at the same level in address order such that + * we cannot deadlock. + */ +static inline void double_pt_lock(spinlock_t *ptl1, spinlock_t *ptl2) + __acquires(ptl1) + __acquires(ptl2) +{ + if (ptl1 > ptl2) + swap(ptl1, ptl2); + spin_lock(ptl1); + if (ptl1 != ptl2) + spin_lock_nested(ptl2, SINGLE_DEPTH_NESTING); + else + __acquire(ptl2); +} + +static inline void double_pt_unlock(spinlock_t *ptl1, spinlock_t *ptl2) + __releases(ptl1) + __releases(ptl2) +{ + spin_unlock(ptl1); + if (ptl1 != ptl2) + spin_unlock(ptl2); + else + __release(ptl2); +} + unsigned find_lock_entries(struct address_space *mapping, pgoff_t *start, pgoff_t end, struct folio_batch *fbatch, pgoff_t *indices); unsigned find_get_entries(struct address_space *mapping, pgoff_t *start, diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 792166950bba..47ed64f51748 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1928,7 +1928,7 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign result = SCAN_FAIL; start_pte = pte_offset_map_lock(mm, pmd, haddr, &ptl); - if (!start_pte) /* mmap_lock + page lock should prevent this */ + if (!start_pte) goto drop_folio; /* step 1: check all mapped PTEs are to the right huge page */ @@ -1959,6 +1959,7 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign } pte_unmap_unlock(start_pte, ptl); + ptl = NULL; mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, mm, haddr, haddr + HPAGE_PMD_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -1976,7 +1977,7 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign pml = pmd_lock(mm, pmd); start_pte = pte_offset_map_rw_nolock(mm, pmd, haddr, &pgt_pmd, &ptl); - if (!start_pte) /* mmap_lock + page lock should prevent this */ + if (!start_pte) goto abort; if (!pml) spin_lock(ptl); @@ -2018,9 +2019,10 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign /* * Must clear entry, or a racing truncate may re-remove it. * TLB flush can be left until pmdp_collapse_flush() does it. - * PTE dirty? Shmem page is already dirty; file is read-only. */ - clear_ptes(mm, addr, pte, nr_batch_ptes); + ptent = get_and_clear_ptes(mm, addr, pte, nr_batch_ptes); + if (pte_dirty(ptent)) + folio_mark_dirty(folio); folio_remove_rmap_ptes(folio, page, nr_batch_ptes, vma); nr_mapped_ptes += nr_batch_ptes; } @@ -2037,12 +2039,11 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign /* step 4: remove empty page table */ if (!pml) { pml = pmd_lock(mm, pmd); - if (ptl != pml) { + if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); - if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) { - flush_tlb_mm(mm); - goto unlock; - } + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) { + flush_tlb_mm(mm); + goto unlock; } } pgt_pmd = pmdp_collapse_flush(vma, haddr, pmd); @@ -2251,6 +2252,7 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr, struct file *file, pgoff_t start, struct collapse_control *cc) { struct address_space *mapping = file->f_mapping; + const bool is_shmem = shmem_mapping(mapping); struct page *dst; struct folio *folio, *tmp, *new_folio; unsigned long new_pfn = -1; @@ -2259,7 +2261,8 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr, XA_STATE_ORDER(xas, &mapping->i_pages, start, HPAGE_PMD_ORDER); enum scan_result result = SCAN_SUCCEED; int nr_none = 0; - bool is_shmem = shmem_file(file); + bool do_readahead = false; + bool readahead_done = false; /* * MADV_COLLAPSE ignores shmem huge config, so do not check shmem @@ -2277,6 +2280,10 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr, mapping_set_update(&xas, mapping); +retry: + result = SCAN_SUCCEED; + do_readahead = false; + xas_set_order(&xas, start, HPAGE_PMD_ORDER); __folio_set_locked(new_folio); if (is_shmem) __folio_set_swapbacked(new_folio); @@ -2341,17 +2348,9 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr, } } else { /* !is_shmem */ if (!folio || xa_is_value(folio)) { - xas_unlock_irq(&xas); - page_cache_sync_readahead(mapping, &file->f_ra, - file, index, - end - index); - /* drain lru cache to help folio_isolate_lru() */ - lru_add_drain(); - folio = filemap_lock_folio(mapping, index); - if (IS_ERR(folio)) { - result = SCAN_FAIL; - goto xa_unlocked; - } + do_readahead = !readahead_done; + result = SCAN_FAIL; + goto xa_locked; } else if (folio_test_dirty(folio)) { /* * This page is dirty because it hasn't @@ -2488,6 +2487,11 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr, index += folio_nr_pages(folio); continue; out_unlock: + /* + * The folio may have been unmapped with TTU_BATCH_FLUSH. + * Flush before releasing the lock and our last reference. + */ + try_to_unmap_flush(); folio_unlock(folio); folio_put(folio); goto xa_unlocked; @@ -2498,9 +2502,8 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr, xa_unlocked: /* - * If collapse is successful, flush must be done now before copying. - * If collapse is unsuccessful, does flush actually need to be done? - * Do it anyway, to clear the state. + * Flush before copying the folios, or releasing them in rollback. + * This is a no-op if out_unlock already flushed the batch. */ try_to_unmap_flush(); @@ -2674,6 +2677,30 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr, new_folio->mapping = NULL; folio_unlock(new_folio); + + /* Readahead may take invalidate_lock; source folios are unlocked. */ + if (do_readahead) { + for (index = start; index < end;) { + folio = filemap_lock_folio(mapping, index); + if (IS_ERR(folio)) { + page_cache_sync_readahead(mapping, &file->f_ra, + file, index, end - index); + folio = filemap_lock_folio(mapping, index); + if (IS_ERR(folio)) { + result = SCAN_FAIL; + goto out_put; + } + } + index = folio_next_index(folio); + folio_unlock(folio); + folio_put(folio); + } + /* Drain the local LRU batch before retrying isolation. */ + lru_add_drain(); + readahead_done = true; + goto retry; + } +out_put: folio_put(new_folio); out: VM_BUG_ON(!list_empty(&pagelist)); diff --git a/mm/ksm.c b/mm/ksm.c index fe42c17490b8..a554e5c0ed07 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include #include @@ -475,6 +476,7 @@ static void advisor_stop_scan(void) #ifdef CONFIG_NUMA /* Zeroed when merging across nodes is not allowed */ static unsigned int ksm_merge_across_nodes = 1; +/* Number of allocated stable and unstable tree roots. */ static int ksm_nr_node_ids = 1; #else #define ksm_merge_across_nodes 1U @@ -1462,6 +1464,15 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, dec_mm_counter(mm, MM_ANONPAGES); } + /* Preserve soft-dirty and uffd protection when replacing the page. */ + if (pte_soft_dirty(orig_pte)) + newpte = pte_mksoft_dirty(newpte); + if (pte_uffd(orig_pte)) { + newpte = pte_mkuffd(newpte); + if (userfaultfd_rwp(vma)) + newpte = pte_modify(newpte, PAGE_NONE); + } + flush_cache_page(vma, addr, pte_pfn(ptep_get(ptep))); /* * No need to notify as we are replacing a read only page with another @@ -3135,24 +3146,52 @@ void __ksm_exit(struct mm_struct *mm) trace_ksm_exit(mm); } -struct folio *ksm_might_need_to_copy(struct folio *folio, - struct vm_area_struct *vma, unsigned long addr) +static bool folio_anon_rmap_compatible(const struct folio *folio, + const struct page *page, struct vm_area_struct *vma, + unsigned long addr) { - struct page *page = folio_page(folio, 0); - struct anon_vma *anon_vma = folio_anon_vma(folio); - struct folio *new_folio; + const struct anon_vma *anon_vma = folio_anon_vma(folio); + struct anon_vma_chain *avc; - if (folio_test_large(folio)) - return folio; + VM_WARN_ON_ONCE_FOLIO(!folio_test_locked(folio), folio); + VM_WARN_ON_ONCE_FOLIO(page_folio(page) != folio, folio); + VM_WARN_ON_ONCE_FOLIO(folio_test_ksm(folio), folio); + vma_assert_stabilised(vma); + + if (!anon_vma) + return true; + if (page_pgoff(folio, page) != linear_anon_page_index(vma, addr)) + return false; + + /* + * Careful: anon_vma might have been freed, so we cannot safely + * dereference it. Add a fast path before we walk the anon_vma chain. + */ + if (anon_vma == vma->anon_vma || anon_vma == vma->anon_vma->root) + return true; + list_for_each_entry(avc, &vma->anon_vma_chain, same_vma) { + if (avc->anon_vma == anon_vma) + return true; + } + return false; +} + +struct folio *ksm_might_need_to_copy(struct folio *folio, struct page *page, + struct vm_area_struct *vma, unsigned long addr) +{ + struct folio *new_folio; + + /* + * A KSM folio is small, but the slot that it left behind can be swapped + * in as a part of a large folio by one of the processes that shared it. + * So test, and copy, the page of the slot. + */ if (folio_test_ksm(folio)) { if (folio_stable_node(folio) && !(ksm_run & KSM_RUN_UNMERGE)) return folio; /* no need to copy it */ - } else if (!anon_vma) { - return folio; /* no need to copy it */ - } else if (folio->index == linear_anon_page_index(vma, addr) && - anon_vma->root == vma->anon_vma->root) { + } else if (folio_anon_rmap_compatible(folio, page, vma, addr)) { return folio; /* still no need to copy it */ } if (PageHWPoison(page)) @@ -3635,14 +3674,13 @@ static ssize_t merge_across_nodes_store(struct kobject *kobj, else { root_stable_tree = buf; root_unstable_tree = buf + nr_node_ids; + ksm_nr_node_ids = nr_node_ids; /* Stable tree is empty but not the unstable */ root_unstable_tree[0] = one_unstable_tree[0]; } } - if (!err) { + if (!err) ksm_merge_across_nodes = knob; - ksm_nr_node_ids = knob ? 1 : nr_node_ids; - } } mutex_unlock(&ksm_thread_mutex); diff --git a/mm/madvise.c b/mm/madvise.c index 54d8397059ba..222c10eb61eb 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -755,6 +755,15 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, if (folio_test_swapcache(folio) || folio_test_dirty(folio)) { if (!folio_trylock(folio)) continue; + /* + * KSM folios can gain new mappings at any time, so we must + * not clear their dirty flag. + */ + if (folio_test_ksm(folio)) { + folio_unlock(folio); + continue; + } + /* * If we have a large folio at this point, we know it is * fully mapped so if its mapcount is the same as its diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b5137e86bed7..b303bfc6b46e 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2419,7 +2419,10 @@ static void high_work_func(struct work_struct *work) struct mem_cgroup *memcg; memcg = container_of(work, struct mem_cgroup, high_work); + if (!css_tryget_online(&memcg->css)) + return; reclaim_high(memcg, MEMCG_CHARGE_BATCH, GFP_KERNEL); + css_put(&memcg->css); } /* @@ -4369,6 +4372,8 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css) page_counter_set_min(&memcg->memory, 0); page_counter_set_low(&memcg->memory, 0); + page_counter_set_high(&memcg->memory, PAGE_COUNTER_MAX); + page_counter_set_high(&memcg->swap, PAGE_COUNTER_MAX); zswap_memcg_offline_cleanup(memcg); diff --git a/mm/memfd.c b/mm/memfd.c index 0f6fff004f5e..0e3ff3eebfa8 100644 --- a/mm/memfd.c +++ b/mm/memfd.c @@ -482,6 +482,8 @@ struct file *memfd_alloc_file(const char *name, unsigned int flags) } file->f_mode |= FMODE_LSEEK | FMODE_PREAD | FMODE_PWRITE; + /* a regular file like any other: POSIX.1-2008/SUSv4 Section XSI 2.9.7 */ + file->f_mode |= FMODE_ATOMIC_POS; file->f_flags |= O_LARGEFILE; if (flags & MFD_NOEXEC_SEAL) { diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c index 54851d8a195b..ddc135f55eb8 100644 --- a/mm/memory-tiers.c +++ b/mm/memory-tiers.c @@ -571,21 +571,14 @@ static struct memory_tier *set_node_memory_tier(int node) return memtier; } -static void destroy_memory_tier(struct memory_tier *memtier) +static struct memory_tier *clear_node_memory_tier(int node) { - list_del(&memtier->list); - device_unregister(&memtier->dev); -} - -static bool clear_node_memory_tier(int node) -{ - bool cleared = false; pg_data_t *pgdat; struct memory_tier *memtier; pgdat = NODE_DATA(node); if (!pgdat) - return false; + return NULL; /* * Make sure that anybody looking at NODE_DATA who finds @@ -603,14 +596,10 @@ static bool clear_node_memory_tier(int node) synchronize_rcu(); memtype = node_memory_types[node].memtype; node_clear(node, memtype->nodes); - if (nodes_empty(memtype->nodes)) { + if (nodes_empty(memtype->nodes)) list_del_init(&memtype->tier_sibling); - if (list_empty(&memtier->memory_types)) - destroy_memory_tier(memtier); - } - cleared = true; } - return cleared; + return memtier; } static void release_memtype(struct kref *kref) @@ -883,13 +872,21 @@ static int __meminit memtier_hotplug_callback(struct notifier_block *self, { struct memory_tier *memtier; struct node_notify *nn = _arg; + bool empty = false; switch (action) { case NODE_REMOVED_LAST_MEMORY: mutex_lock(&memory_tier_lock); - if (clear_node_memory_tier(nn->nid)) + memtier = clear_node_memory_tier(nn->nid); + if (memtier) { + empty = list_empty(&memtier->memory_types); + if (empty) + list_del(&memtier->list); establish_demotion_targets(); + } mutex_unlock(&memory_tier_lock); + if (empty) + device_unregister(&memtier->dev); break; case NODE_ADDED_FIRST_MEMORY: mutex_lock(&memory_tier_lock); diff --git a/mm/memory.c b/mm/memory.c index 9e395ad33ffd..77e0a9bf6321 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1500,7 +1500,6 @@ copy_pud_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, unsigned long end) { struct mm_struct *dst_mm = dst_vma->vm_mm; - struct mm_struct *src_mm = src_vma->vm_mm; pud_t *src_pud, *dst_pud; unsigned long next; @@ -1514,8 +1513,7 @@ copy_pud_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, int err; VM_BUG_ON_VMA(next-addr != HPAGE_PUD_SIZE, src_vma); - err = copy_huge_pud(dst_mm, src_mm, - dst_pud, src_pud, addr, src_vma); + err = copy_huge_pud(dst_vma, src_vma, dst_pud, src_pud, addr); if (err == -ENOMEM) return -ENOMEM; if (!err) @@ -2073,6 +2071,15 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, } } while (pte += nr, addr += PAGE_SIZE * nr, addr != end); + add_mm_rss_vec(mm, rss); + lazy_mmu_mode_disable(); + + /* Do the actual TLB flush before dropping ptl */ + if (force_flush) { + tlb_flush_mmu_tlbonly(tlb); + tlb_flush_rmaps(tlb, vma); + } + /* * Fast path: try to hold the pmd lock and unmap the PTE page. * @@ -2080,18 +2087,12 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, * to hold the pmd lock failed, then we need to recheck all pte entries * to ensure they are still none, thereby preventing the pte entries * from being repopulated by another thread. + * + * Clear the PMD after the rmaps for the table have been removed, in case + * an incorrect mapcount is observed after unmapping the folio. */ if (can_reclaim_pt && direct_reclaim && addr == end) direct_reclaim = zap_empty_pte_table(mm, pmd, ptl, &pmdval); - - add_mm_rss_vec(mm, rss); - lazy_mmu_mode_disable(); - - /* Do the actual TLB flush before dropping ptl */ - if (force_flush) { - tlb_flush_mmu_tlbonly(tlb); - tlb_flush_rmaps(tlb, vma); - } pte_unmap_unlock(start_pte, ptl); /* @@ -2138,13 +2139,15 @@ static inline unsigned long zap_pmd_range(struct mmu_gather *tlb, addr = next; continue; } + /* We already synchronized with concurrent zapping. */ + if (pmd_none(*pmd)) { + addr = next; + continue; + } /* fall through */ - } else if (details && details->single_folio && - folio_test_pmd_mappable(details->single_folio) && - next - addr == HPAGE_PMD_SIZE && pmd_none(*pmd)) { - sync_with_folio_pmd_zap(tlb->mm, pmd); - } - if (pmd_none(*pmd)) { + } else if (pmd_none(*pmd)) { + if (details && details->single_folio) + sync_with_folio_pmd_zap(tlb->mm, pmd); addr = next; continue; } @@ -3700,6 +3703,13 @@ static inline int __wp_page_copy_user(struct page *dst, struct page *src, return 0; } + /* + * __copy_from_user_inatomic() only works as expected for local faults. + * Reject remote faults. + */ + if (unlikely(mm != current->mm)) + return -EFAULT; + /* * If the source page was a PFN mapping, we don't have * a "struct page" for it. We do a best-effort copy by @@ -4025,6 +4035,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) folio_put(old_folio); delayacct_wpcopy_end(); + if (err == -EFAULT) + return VM_FAULT_SIGBUS; return err == -EHWPOISON ? VM_FAULT_HWPOISON : 0; } kmsan_copy_page_meta(&new_folio->page, vmf->page); @@ -4057,8 +4069,11 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) if (unlikely(unshare)) { if (pte_soft_dirty(vmf->orig_pte)) entry = pte_mksoft_dirty(entry); - if (pte_uffd(vmf->orig_pte)) + if (pte_uffd(vmf->orig_pte)) { entry = pte_mkuffd(entry); + if (userfaultfd_rwp(vma)) + entry = pte_modify(entry, PAGE_NONE); + } } else { entry = maybe_mkwrite(pte_mkdirty(entry), vma); } @@ -4579,8 +4594,8 @@ EXPORT_SYMBOL_GPL(unmap_mapping_pages); void unmap_mapping_range(struct address_space *mapping, loff_t const holebegin, loff_t const holelen, int even_cows) { - pgoff_t hba = (pgoff_t)(holebegin) >> PAGE_SHIFT; - pgoff_t hlen = ((pgoff_t)(holelen) + PAGE_SIZE - 1) >> PAGE_SHIFT; + pgoff_t hba = (u64)holebegin >> PAGE_SHIFT; + pgoff_t hlen = ((u64)holelen + PAGE_SIZE - 1) >> PAGE_SHIFT; /* Check for overflow. */ if (sizeof(holelen) > sizeof(hlen)) { @@ -4870,6 +4885,34 @@ static void check_swap_exclusive(struct folio *folio, swp_entry_t entry, } while (--nr_pages); } +/* + * Get the address range in which a folio might be mapped, assuming its pages + * are mapped consecutively and the folio page at the given idx is mapped at + * the given address. + */ +static void folio_pte_range(const struct folio *folio, + const struct vm_area_struct *vma, unsigned long addr, + unsigned long idx, unsigned long *start, unsigned long *end) +{ + /* Stay within the page table and VMA boundaries. */ + const unsigned long pmd_start = addr & PMD_MASK; + const unsigned long pmd_end = pmd_start + PMD_SIZE; + const unsigned long lower = max(pmd_start, vma->vm_start); + /* + * We cannot use pmd_addr_end(), as it does not clamp reliably. To + * avoid wraparounds, compute using the inclusive end. + */ + const unsigned long upper = min(pmd_end - 1, vma->vm_end - 1) + 1; + const unsigned long offset = idx << PAGE_SHIFT; + + /* + * Avoid start/end addresses wrapping by not adding/subtracting more + * than the lower/upper limits would allow. + */ + *start = addr - min(addr - lower, offset); + *end = addr + min(upper - addr, folio_size(folio) - offset); +} + /* * We enter with either the VMA lock or the mmap_lock held (see * FAULT_FLAG_VMA_LOCK), and pte mapped but not yet locked. @@ -5020,7 +5063,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) * folio->index of non-ksm folios would be nonlinear inside the * anon VMA -- the ksm flag is lost on actual swapout. */ - folio = ksm_might_need_to_copy(folio, vma, vmf->address); + folio = ksm_might_need_to_copy(folio, page, vma, vmf->address); if (unlikely(!folio)) { ret = VM_FAULT_OOM; folio = swapcache; @@ -5052,16 +5095,15 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) address = vmf->address; ptep = vmf->pte; if (folio_test_large(folio) && folio_test_swapcache(folio)) { - int nr = folio_nr_pages(folio); - unsigned long idx = folio_page_idx(folio, page); - unsigned long folio_start = address - idx * PAGE_SIZE; - unsigned long folio_end = folio_start + nr * PAGE_SIZE; + const int nr = folio_nr_pages(folio); + const unsigned long idx = folio_page_idx(folio, page); + unsigned long folio_start, folio_end; pte_t *folio_ptep; pte_t folio_pte; - if (unlikely(folio_start < max(address & PMD_MASK, vma->vm_start))) - goto check_folio; - if (unlikely(folio_end > pmd_addr_end(address, vma->vm_end))) + folio_pte_range(folio, vma, address, idx, &folio_start, + &folio_end); + if (unlikely((folio_end - folio_start) >> PAGE_SHIFT != nr)) goto check_folio; folio_ptep = vmf->pte - idx; @@ -5202,7 +5244,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (unlikely(folio != swapcache)) { folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE); folio_add_lru_vma(folio, vma); - folio_put_swap(swapcache, NULL); + folio_put_swap(swapcache, + folio_file_page(swapcache, swp_offset(entry))); } else if (!folio_test_anon(folio)) { /* * We currently only expect !anon folios that are fully @@ -6211,16 +6254,11 @@ static void numa_rebuild_large_mapping(struct vm_fault *vmf, struct vm_area_stru struct folio *folio, pte_t fault_pte, bool ignore_writable, bool pte_write_upgrade) { - int nr = pte_pfn(fault_pte) - folio_pfn(folio); + const unsigned long idx = pte_pfn(fault_pte) - folio_pfn(folio); unsigned long start, end, addr = vmf->address; - unsigned long addr_start = addr - (nr << PAGE_SHIFT); - unsigned long pt_start = ALIGN_DOWN(addr, PMD_SIZE); pte_t *start_ptep; - /* Stay within the VMA and within the page table. */ - start = max3(addr_start, pt_start, vma->vm_start); - end = min3(addr_start + folio_size(folio), pt_start + PMD_SIZE, - vma->vm_end); + folio_pte_range(folio, vma, addr, idx, &start, &end); start_ptep = vmf->pte - ((addr - start) >> PAGE_SHIFT); /* Restore all PTEs' mapping of the large folio */ @@ -6566,9 +6604,12 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) * VM_UFFD_RWP VMA, a protnone PTE without the uffd bit is * NUMA hinting and must still fall through to do_numa_page(). */ - if (userfaultfd_pte_rwp(vmf->vma, vmf->orig_pte)) - return do_uffd_rwp(vmf); - return do_numa_page(vmf); + if (userfaultfd_pte_rwp(vmf->vma, vmf->orig_pte)) { + if (!(vmf->flags & FAULT_FLAG_UNSHARE)) + return do_uffd_rwp(vmf); + } else { + return do_numa_page(vmf); + } } spin_lock(vmf->ptl); @@ -6883,8 +6924,12 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, lru_gen_exit_fault(); - /* If the mapping is droppable, then errors due to OOM aren't fatal. */ - if (is_droppable) + /* + * If the mapping is droppable, then errors due to OOM aren't fatal: the + * task returns to user mode, handles its signals and tries again. A + * fault from kernel mode has no such return, so it must see the error. + */ + if (is_droppable && (flags & FAULT_FLAG_USER)) ret &= ~VM_FAULT_OOM; if (flags & FAULT_FLAG_USER) { diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 839c3fe985a0..d9de3b5d5ebe 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -3346,11 +3346,11 @@ void __init numa_policy_init(void) policy_cache = kmem_cache_create("numa_policy", sizeof(struct mempolicy), - 0, SLAB_PANIC, NULL); + 0, SLAB_PANIC | SLAB_ACCOUNT, NULL); sn_cache = kmem_cache_create("shared_policy_node", sizeof(struct sp_node), - 0, SLAB_PANIC, NULL); + 0, SLAB_PANIC | SLAB_ACCOUNT, NULL); for_each_node(nid) { preferred_node_policy[nid] = (struct mempolicy) { diff --git a/mm/migrate.c b/mm/migrate.c index 15b45832bcfa..ee78577177d6 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2462,7 +2462,7 @@ static void do_pages_stat_array(struct mm_struct *mm, unsigned long nr_pages, mmap_read_lock(mm); for (i = 0; i < nr_pages; i++) { - unsigned long addr = (unsigned long)(*pages); + unsigned long addr = (unsigned long)untagged_addr_remote(mm, *pages); struct vm_area_struct *vma; struct folio_walk fw; struct folio *folio; @@ -2757,6 +2757,7 @@ int migrate_misplaced_folio(struct folio *folio, int node) pg_data_t *pgdat = NODE_DATA(node); int nr_remaining; unsigned int nr_succeeded; + int src_node = folio_nid(folio); LIST_HEAD(migratepages); struct mem_cgroup *memcg = get_mem_cgroup_from_folio(folio); struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); @@ -2771,7 +2772,7 @@ int migrate_misplaced_folio(struct folio *folio, int node) count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded); count_memcg_events(memcg, NUMA_PAGE_MIGRATE, nr_succeeded); if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) - && !node_is_toptier(folio_nid(folio)) + && !node_is_toptier(src_node) && node_is_toptier(node)) mod_lruvec_state(lruvec, PGPROMOTE_SUCCESS, nr_succeeded); } diff --git a/mm/mm_init.c b/mm/mm_init.c index 1533aebafb68..36ec1ef0712c 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -2422,8 +2422,6 @@ void __init memblock_free_pages(unsigned long pfn, unsigned int order) return; } - /* pages were reserved and not allocated */ - clear_page_tag_ref(page); __free_pages_core(page, order, MEMINIT_EARLY); } diff --git a/mm/mmap.c b/mm/mmap.c index 4bf26b0f1e6e..6f22cd76be4f 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1713,8 +1713,7 @@ __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm) LIST_HEAD(uf); VMA_ITERATOR(vmi, mm, 0); - if (mmap_write_lock_killable(oldmm)) - return -EINTR; + mmap_assert_write_locked(oldmm); flush_cache_dup_mm(oldmm); uprobe_dup_mmap(oldmm, mm); /* @@ -1725,11 +1724,6 @@ __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm) /* No ordering required: file already has been exposed. */ dup_mm_exe_file(mm, oldmm); - mm->total_vm = oldmm->total_vm; - mm->data_vm = oldmm->data_vm; - mm->exec_vm = oldmm->exec_vm; - mm->stack_vm = oldmm->stack_vm; - /* Use __mt_dup() to efficiently build an identical maple tree. */ retval = __mt_dup(&oldmm->mm_mt, &mm->mm_mt, GFP_KERNEL); if (unlikely(retval)) diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index df69ba6e797f..a7bae5de5fed 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -569,16 +569,25 @@ int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) { struct mmu_notifier_subscriptions *subscriptions = range->mm->notifier_subscriptions; - int ret; + const bool has_itree = subscriptions->has_itree; + int ret = 0; - if (subscriptions->has_itree) { + if (has_itree) { ret = mn_itree_invalidate(subscriptions, range); if (ret) return ret; } if (!hlist_empty(&subscriptions->list)) - return mn_hlist_invalidate_range_start(subscriptions, range); - return 0; + ret = mn_hlist_invalidate_range_start(subscriptions, range); + + /* + * On error, the non-blocking caller is not allowed to call + * invalidate_range_end(), as in mn_itree_invalidate(). + */ + if (ret && has_itree && !mmu_notifier_range_blockable(range)) + mn_itree_inv_end(subscriptions); + + return ret; } static void @@ -938,7 +947,6 @@ static int __mmu_interval_notifier_insert( struct mmu_notifier_subscriptions *subscriptions, unsigned long start, unsigned long length, const struct mmu_interval_notifier_ops *ops) { - interval_sub->mm = mm; interval_sub->ops = ops; RB_CLEAR_NODE(&interval_sub->interval_tree.rb); interval_sub->interval_tree.start = start; @@ -955,8 +963,12 @@ static int __mmu_interval_notifier_insert( if (WARN_ON(atomic_read(&mm->mm_users) <= 0)) return -EINVAL; - /* pairs with mmdrop in mmu_interval_notifier_remove() */ + /* + * Pairs with mmdrop in mmu_interval_notifier_remove(). Drivers take a + * non-NULL ->mm for "registered", so set it only with the reference. + */ mmgrab(mm); + interval_sub->mm = mm; /* * If some invalidate_range_start/end region is going on in parallel diff --git a/mm/mremap.c b/mm/mremap.c index d404e4b1b6cb..23498d0783c4 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -87,7 +87,7 @@ static pud_t *get_old_pud(struct mm_struct *mm, unsigned long addr) return NULL; pud = pud_offset(p4d, addr); - if (pud_none_or_clear_bad(pud)) + if (!pud_trans_huge(*pud) && pud_none_or_clear_bad(pud)) return NULL; return pud; @@ -202,7 +202,7 @@ static int move_ptes(struct pagetable_move_control *pmc, struct mm_struct *mm = vma->vm_mm; pte_t *old_ptep, *new_ptep; pte_t old_pte, pte; - pmd_t dummy_pmdval; + pmd_t old_pmdval, new_pmdval; spinlock_t *old_ptl, *new_ptl; bool force_flush = false; unsigned long old_addr = pmc->old_addr; @@ -234,31 +234,30 @@ static int move_ptes(struct pagetable_move_control *pmc, if (pmc->need_rmap_locks) take_rmap_locks(vma); - /* - * We don't have to worry about the ordering of src and dst - * pte locks because exclusive mmap_lock prevents deadlock. - */ - old_ptep = pte_offset_map_lock(mm, old_pmd, old_addr, &old_ptl); + old_ptep = pte_offset_map_rw_nolock(mm, old_pmd, old_addr, &old_pmdval, + &old_ptl); if (!old_ptep) { err = -EAGAIN; goto out; } - /* - * Now new_pte is none, so collapse_scan_file() path can not find - * this by traversing file->f_mapping, so there is no concurrency with - * retract_page_tables(). In addition, we already hold the exclusive - * mmap_lock, so this new_pte page is stable, so there is no need to get - * pmdval and do pmd_same() check. - */ - new_ptep = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &dummy_pmdval, - &new_ptl); + new_ptep = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &new_pmdval, + &new_ptl); if (!new_ptep) { - pte_unmap_unlock(old_ptep, old_ptl); + pte_unmap(old_ptep); + err = -EAGAIN; + goto out; + } + + double_pt_lock(old_ptl, new_ptl); + if (unlikely(!pmd_same(old_pmdval, pmdp_get_lockless(old_pmd)) || + !pmd_same(new_pmdval, pmdp_get_lockless(new_pmd)))) { + double_pt_unlock(old_ptl, new_ptl); + pte_unmap(new_ptep); + pte_unmap(old_ptep); err = -EAGAIN; goto out; } - if (new_ptl != old_ptl) - spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING); + flush_tlb_batched_pending(vma->vm_mm); lazy_mmu_mode_enable(); @@ -317,10 +316,9 @@ static int move_ptes(struct pagetable_move_control *pmc, lazy_mmu_mode_disable(); if (force_flush) flush_tlb_range(vma, old_end - len, old_end); - if (new_ptl != old_ptl) - spin_unlock(new_ptl); + double_pt_unlock(old_ptl, new_ptl); pte_unmap(new_ptep - 1); - pte_unmap_unlock(old_ptep - 1, old_ptl); + pte_unmap(old_ptep - 1); out: if (pmc->need_rmap_locks) drop_rmap_locks(vma); @@ -395,14 +393,9 @@ static bool move_normal_pmd(struct pagetable_move_control *pmc, if (WARN_ON_ONCE(!pmd_none(*new_pmd))) return false; - /* - * We don't have to worry about the ordering of src and dst - * ptlocks because exclusive mmap_lock prevents deadlock. - */ - old_ptl = pmd_lock(mm, old_pmd); + old_ptl = pmd_lockptr(mm, old_pmd); new_ptl = pmd_lockptr(mm, new_pmd); - if (new_ptl != old_ptl) - spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING); + double_pt_lock(old_ptl, new_ptl); pmd = *old_pmd; @@ -418,9 +411,7 @@ static bool move_normal_pmd(struct pagetable_move_control *pmc, pmd_populate(mm, new_pmd, pmd_pgtable(pmd)); flush_tlb_range(vma, pmc->old_addr, pmc->old_addr + PMD_SIZE); out_unlock: - if (new_ptl != old_ptl) - spin_unlock(new_ptl); - spin_unlock(old_ptl); + double_pt_unlock(old_ptl, new_ptl); return res; } @@ -452,26 +443,24 @@ static bool move_normal_pud(struct pagetable_move_control *pmc, if (WARN_ON_ONCE(!pud_none(*new_pud))) return false; - /* - * We don't have to worry about the ordering of src and dst - * ptlocks because exclusive mmap_lock prevents deadlock. - */ - old_ptl = pud_lock(mm, old_pud); + old_ptl = pud_lockptr(mm, old_pud); new_ptl = pud_lockptr(mm, new_pud); - if (new_ptl != old_ptl) - spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING); - /* Clear the pud */ + double_pt_lock(old_ptl, new_ptl); + pud = *old_pud; + if (unlikely(pud_none(pud))) + goto out_unlock; + + /* Clear the pud */ pud_clear(old_pud); VM_BUG_ON(!pud_none(*new_pud)); pud_populate(mm, new_pud, pud_pgtable(pud)); flush_tlb_range(vma, pmc->old_addr, pmc->old_addr + PUD_SIZE); - if (new_ptl != old_ptl) - spin_unlock(new_ptl); - spin_unlock(old_ptl); +out_unlock: + double_pt_unlock(old_ptl, new_ptl); return true; } @@ -499,18 +488,17 @@ static bool move_huge_pud(struct pagetable_move_control *pmc, if (WARN_ON_ONCE(!pud_none(*new_pud))) return false; - /* - * We don't have to worry about the ordering of src and dst - * ptlocks because exclusive mmap_lock prevents deadlock. - */ - old_ptl = pud_lock(mm, old_pud); + old_ptl = pud_lockptr(mm, old_pud); new_ptl = pud_lockptr(mm, new_pud); - if (new_ptl != old_ptl) - spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING); + + double_pt_lock(old_ptl, new_ptl); + if (unlikely(!pud_trans_huge(*old_pud))) { + double_pt_unlock(old_ptl, new_ptl); + return false; + } /* Clear the pud */ - pud = *old_pud; - pud_clear(old_pud); + pud = pudp_huge_get_and_clear(mm, pmc->old_addr, old_pud); VM_BUG_ON(!pud_none(*new_pud)); @@ -518,9 +506,7 @@ static bool move_huge_pud(struct pagetable_move_control *pmc, /* mark soft_ditry when we add pud level soft dirty support */ set_pud_at(mm, pmc->new_addr, new_pud, pud); flush_pud_tlb_range(vma, pmc->old_addr, pmc->old_addr + HPAGE_PUD_SIZE); - if (new_ptl != old_ptl) - spin_unlock(new_ptl); - spin_unlock(old_ptl); + double_pt_unlock(old_ptl, new_ptl); return true; } @@ -842,11 +828,10 @@ unsigned long move_page_tables(struct pagetable_move_control *pmc) if (!new_pud) break; if (pud_trans_huge(*old_pud)) { - if (extent == HPAGE_PUD_SIZE) { - move_pgt_entry(pmc, HPAGE_PUD, old_pud, new_pud); - /* We ignore and continue on error? */ + if (extent == HPAGE_PUD_SIZE && + move_pgt_entry(pmc, HPAGE_PUD, old_pud, new_pud)) continue; - } + split_huge_pud(pmc->old, old_pud, pmc->old_addr); } else if (IS_ENABLED(CONFIG_HAVE_MOVE_PUD) && extent == PUD_SIZE) { if (move_pgt_entry(pmc, NORMAL_PUD, old_pud, new_pud)) continue; diff --git a/mm/nommu.c b/mm/nommu.c index ed44510e3770..6e0f9c7da236 100644 --- a/mm/nommu.c +++ b/mm/nommu.c @@ -1926,7 +1926,7 @@ subsys_initcall(init_admin_reserve); int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm) { - mmap_write_lock(oldmm); + mmap_assert_write_locked(oldmm); dup_mm_exe_file(mm, oldmm); mmap_write_unlock(oldmm); return 0; diff --git a/mm/page-writeback.c b/mm/page-writeback.c index eeab25d6ce36..1d6aab8781a5 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -883,6 +883,7 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc, { struct wb_domain *dom = dtc_dom(dtc); struct bdi_writeback *wb = dtc->wb; + u64 threshold = thresh; u64 wb_thresh; u64 wb_max_thresh; unsigned long numerator, denominator; @@ -894,13 +895,14 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc, fprop_fraction_percpu(&dom->completions, dtc->wb_completions, &numerator, &denominator); - wb_thresh = (thresh * (100 * BDI_RATIO_SCALE - bdi_min_ratio)) / (100 * BDI_RATIO_SCALE); + wb_thresh = div_u64(threshold * (100 * BDI_RATIO_SCALE - bdi_min_ratio), + 100 * BDI_RATIO_SCALE); wb_thresh *= numerator; wb_thresh = div64_ul(wb_thresh, denominator); wb_min_max_ratio(wb, &wb_min_ratio, &wb_max_ratio); - wb_thresh += (thresh * wb_min_ratio) / (100 * BDI_RATIO_SCALE); + wb_thresh += div_u64(threshold * wb_min_ratio, 100 * BDI_RATIO_SCALE); /* * It's very possible that wb_thresh is close to 0 not because the @@ -916,7 +918,7 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc, wb_thresh = max(wb_thresh, (thresh - dtc->dirty) / 8); } - wb_max_thresh = thresh * wb_max_ratio / (100 * BDI_RATIO_SCALE); + wb_max_thresh = div_u64(threshold * wb_max_ratio, 100 * BDI_RATIO_SCALE); if (wb_thresh > wb_max_thresh) wb_thresh = wb_max_thresh; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 8ca2289d0491..8714425ffe84 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1634,6 +1634,9 @@ void __meminit __free_pages_core(struct page *page, unsigned int order, atomic_long_add(nr_pages, &page_zone(page)->managed_pages); } + /* pages were never allocated */ + clear_page_tag_ref(page); + if (page_contains_unaccepted(page, order)) { if (order == MAX_PAGE_ORDER && __free_unaccepted(page)) return; @@ -2947,7 +2950,7 @@ static bool free_frozen_page_commit(struct zone *zone, * kswapd work again by resetting kswapd_failures. */ if (kswapd_test_hopeless(pgdat) && - next_memory_node(pgdat->node_id) < MAX_NUMNODES) + num_node_state(N_MEMORY) > 1) kswapd_clear_hopeless(pgdat, KSWAPD_CLEAR_HOPELESS_PCP); } return ret; @@ -3256,7 +3259,9 @@ struct page *rmqueue_buddy(struct zone *preferred_zone, struct zone *zone, * reserves as failing now is worse than failing a * high-order atomic allocation in the future. */ - if (!page && (alloc_flags & (ALLOC_OOM|ALLOC_NON_BLOCK))) + if (!page && + ((alloc_flags & ALLOC_OOM) || + (alloc_flags & ALLOC_MASK_ATOMIC) == ALLOC_MASK_ATOMIC)) page = __rmqueue_smallest(zone, order, MIGRATE_HIGHATOMIC); if (!page) { @@ -4786,10 +4791,10 @@ static inline struct page * __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, struct alloc_context *ac) { - bool can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM; - bool can_compact = can_direct_reclaim && gfp_compaction_allowed(gfp_mask); - bool nofail = gfp_mask & __GFP_NOFAIL; const bool costly_order = order > PAGE_ALLOC_COSTLY_ORDER; + bool can_direct_reclaim; + bool can_compact; + bool nofail; struct page *page = NULL; unsigned int alloc_flags; unsigned long did_some_progress; @@ -4804,6 +4809,18 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, bool can_retry_reserves = true; unsigned long alloc_start_time = jiffies; + /* + * Costly __GFP_NORETRY callers have a cheap fallback, so don't stall + * them in reclaim or compaction. __GFP_THISNODE callers are exempt. + */ + if (costly_order && (gfp_mask & __GFP_NORETRY) && + !(gfp_mask & __GFP_THISNODE)) + gfp_mask &= ~__GFP_DIRECT_RECLAIM; + + can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM; + can_compact = can_direct_reclaim && gfp_compaction_allowed(gfp_mask); + nofail = gfp_mask & __GFP_NOFAIL; + if (unlikely(nofail)) { /* * Also we don't support __GFP_NOFAIL without __GFP_DIRECT_RECLAIM, @@ -4920,8 +4937,11 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, * Reclaim/compaction cannot run, so defrag_mode's strategy * of enforcing ALLOC_NOFRAGMENT cannot be fulfilled. Allow * fallbacks rather than failing the allocation outright. + * Not for __GFP_NORETRY: those have a cheap lower order + * fallback, so failing beats fragmenting. */ if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT) && + !(gfp_mask & __GFP_NORETRY) && (gfp_mask & __GFP_KSWAPD_RECLAIM)) { alloc_flags &= ~ALLOC_NOFRAGMENT; goto retry; diff --git a/mm/page_alloc.h b/mm/page_alloc.h index 2b1017a91928..a9b72cb9520e 100644 --- a/mm/page_alloc.h +++ b/mm/page_alloc.h @@ -32,12 +32,11 @@ #define ALLOC_OOM ALLOC_NO_WATERMARKS #endif -#define ALLOC_NON_BLOCK 0x10 /* Caller cannot block. Allow access - * to 25% of the min watermark or - * 62.5% if __GFP_HIGH is set. - */ +#define ALLOC_NON_BLOCK 0x10 /* Caller cannot block. */ #define ALLOC_MIN_RESERVE 0x20 /* __GFP_HIGH set. Allow access to 50% - * of the min watermark. + * of the min watermark, or 62.5% if + * the caller cannot block either + * (ALLOC_NON_BLOCK). */ #define ALLOC_CPUSET 0x40 /* check for correct cpuset */ #define ALLOC_CMA 0x80 /* allow allocations from CMA areas */ @@ -58,7 +57,10 @@ #define ALLOC_NO_CODETAG 0x1000 /* Flags that allow allocations below the min watermark. */ -#define ALLOC_RESERVES (ALLOC_NON_BLOCK|ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM) +#define ALLOC_RESERVES (ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM) + +/* Flags that mean GFP_ATOMIC */ +#define ALLOC_MASK_ATOMIC (ALLOC_NON_BLOCK|ALLOC_MIN_RESERVE) /* * Structure for holding the mostly immutable allocation parameters passed diff --git a/mm/rmap.c b/mm/rmap.c index 6b3480330b92..9ffd3e833e08 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -926,6 +926,7 @@ static bool folio_referenced_one(struct folio *folio, DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, address, 0); int ptes = 0, referenced = 0; unsigned int nr; + pte_t pteval; while (page_vma_mapped_walk(&pvmw)) { address = pvmw.address; @@ -970,10 +971,12 @@ static bool folio_referenced_one(struct folio *folio, return false; } - if (pvmw.pte && folio_test_large(folio)) { + if (pvmw.pte) + pteval = ptep_get(pvmw.pte); + + if (pvmw.pte && pte_present(pteval) && folio_test_large(folio)) { const unsigned long end_addr = pmd_addr_end(address, vma->vm_end); const unsigned int max_nr = (end_addr - address) >> PAGE_SHIFT; - pte_t pteval = ptep_get(pvmw.pte); nr = folio_pte_batch(folio, pvmw.pte, pteval, max_nr); } @@ -983,11 +986,23 @@ static bool folio_referenced_one(struct folio *folio, * are. —they could be on active/inactive lists or on MGLRU. So the * simplest approach is to disable this look-around optimization. */ - if (lru_gen_enabled() && !lru_gen_switching() && pvmw.pte) { + if (lru_gen_enabled() && !lru_gen_switching() && pvmw.pte && + pte_present(pteval)) { if (lru_gen_look_around(&pvmw, nr)) referenced++; } else if (pvmw.pte) { - if (clear_flush_young_ptes_notify(vma, address, pvmw.pte, nr)) + bool young = false; + + /* + * PFN swap PTEs, such as device-exclusive ones, that + * actually map pages are "old" from a CPU perspective. + * The MMU notifier takes care of any device aspects. + */ + if (pte_present(pteval)) + young = clear_flush_young_ptes(vma, address, pvmw.pte, nr); + young |= mmu_notifier_clear_flush_young(vma->vm_mm, address, + address + nr * PAGE_SIZE); + if (young) referenced++; } else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) { if (pmdp_clear_flush_young_notify(vma, address, diff --git a/mm/shmem.c b/mm/shmem.c index de2362209cc3..597bbc2e7321 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -77,6 +77,7 @@ static struct vfsmount *shm_mnt __ro_after_init; #include #include #include +#include #include #include #include @@ -275,6 +276,11 @@ static const struct vm_operations_struct shmem_vm_ops; static const struct vm_operations_struct shmem_anon_vm_ops; static struct file_system_type shmem_fs_type; +bool shmem_file(struct file *file) +{ + return file && file_inode(file)->i_fop == &shmem_file_operations; +} + bool shmem_mapping(const struct address_space *mapping) { return mapping->a_ops == &shmem_aops; @@ -968,12 +974,11 @@ static void shmem_delete_from_page_cache(struct folio *folio, void *radswap) } /* - * Remove swap entry from page cache, free the swap and its page cache. Returns - * the number of pages being freed. 0 means entry not found in XArray (0 pages - * being freed). + * Remove a swap entry or swapin-error marker wholly within [index, end]. + * Return whether it was removed, counting freed swap pages in nr_swaps_freed. */ -static long shmem_free_swap(struct address_space *mapping, - pgoff_t index, pgoff_t end, void *radswap) +static bool shmem_free_swap(struct address_space *mapping, + pgoff_t index, pgoff_t end, void *radswap, long *nr_swaps_freed) { XA_STATE(xas, &mapping->i_pages, index); const softleaf_t swp = radix_to_swp_entry(radswap); @@ -994,8 +999,10 @@ static long shmem_free_swap(struct address_space *mapping, xas_unlock_irq(&xas); /* A swapin-error marker holds no swap slot, so just drop it. */ - if (nr_pages && softleaf_is_swap(swp)) + if (nr_pages && softleaf_is_swap(swp)) { swap_put_entries_direct(swp, nr_pages); + *nr_swaps_freed += nr_pages; + } return nr_pages; } @@ -1089,12 +1096,10 @@ void shmem_unlock_mapping(struct address_space *mapping) static struct folio *shmem_get_partial_folio(struct inode *inode, pgoff_t index) { + const gfp_t gfp = mapping_gfp_mask(inode->i_mapping); struct folio *folio; - /* - * At first avoid shmem_get_folio(,,,SGP_READ): that fails - * beyond i_size, and reports fallocated folios as holes. - */ +retry: folio = filemap_get_entry(inode->i_mapping, index); if (!folio) return folio; @@ -1105,13 +1110,14 @@ static struct folio *shmem_get_partial_folio(struct inode *inode, pgoff_t index) /* The folio has been swapped out */ folio_unlock(folio); folio_put(folio); + goto retry; } /* - * But read a folio back from swap if any of it is within i_size - * (although in some cases this is just a waste of time). + * Truncation can look up a large swap entry at or beyond i_size. + * Swap it in directly to bypass the size check in shmem_get_folio(). */ - folio = NULL; - shmem_get_folio(inode, index, 0, &folio, SGP_READ); + if (shmem_swapin_folio(inode, index, &folio, SGP_READ, gfp, NULL, NULL) == -EEXIST) + goto retry; return folio; } @@ -1150,8 +1156,8 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend, if (xa_is_value(folio)) { if (unfalloc) continue; - nr_swaps_freed += shmem_free_swap(mapping, indices[i], - end - 1, folio); + shmem_free_swap(mapping, indices[i], end - 1, + folio, &nr_swaps_freed); continue; } @@ -1218,32 +1224,31 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend, if (xa_is_value(folio)) { int order; - long swaps_freed; if (unfalloc) continue; - swaps_freed = shmem_free_swap(mapping, indices[i], - end - 1, folio); - if (!swaps_freed) { + if (!shmem_free_swap(mapping, indices[i], end - 1, + folio, &nr_swaps_freed)) { pgoff_t base = indices[i]; order = shmem_confirm_swap(mapping, indices[i], radix_to_swp_entry(folio)); /* - * If found a large swap entry cross the end or start - * border, skip it as the truncate_inode_partial_folio - * above should have at least zerod its content once. + * Skip boundary entries when restarting + * the scan to avoid endless loops after + * swapin failure. */ if (order > 0) { base = round_down(base, 1 << order); - if (base < start || base + (1 << order) > end) + if (base < start || base + (1 << order) > end) { + start = max(start, base + (1 << order)); continue; + } } /* Swap was replaced by page or extended, retry */ index = base; break; } - nr_swaps_freed += swaps_freed; continue; } @@ -1259,22 +1264,34 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend, VM_BUG_ON_FOLIO(folio_test_writeback(folio), folio); - if (!folio_test_large(folio)) { + /* + * A shared mapping can have dirty PTEs while + * the folio is still clean. Mark the folio + * dirty so a failed split cannot discard data + * outside the range. + */ + if (!unfalloc && folio_test_large(folio)) + folio_mark_dirty(folio); + + if (unfalloc || !folio_test_large(folio)) { truncate_inode_folio(mapping, folio); } else if (truncate_inode_partial_folio(folio, lstart, lend)) { /* - * If we split a page, reset the loop so - * that we pick up the new sub pages. - * Otherwise the THP was entirely - * dropped or the target range was - * zeroed, so just continue the loop as - * is. + * The folio was split. Retry the scan + * to find the split folios. */ - if (!folio_test_large(folio)) { - folio_unlock(folio); - index = start; - break; - } + folio_unlock(folio); + index = start; + break; + } else if (folio->mapping) { + /* + * Splitting failed, but the part in the + * range is already zeroed. The folio + * was not dropped, so skip it when + * restarting the scan to avoid retrying + * it forever. + */ + start = max(start, folio_next_index(folio)); } } folio_unlock(folio); @@ -1341,10 +1358,11 @@ static int shmem_setattr(struct mnt_idmap *idmap, if (error) return error; - if ((info->seals & F_SEAL_EXEC) && (attr->ia_valid & ATTR_MODE)) { - if ((inode->i_mode ^ attr->ia_mode) & 0111) { - return -EPERM; - } + if (attr->ia_valid & ATTR_MODE) { + error = memfd_check_seals_mode(info->seals, inode->i_mode, + attr->ia_mode); + if (error) + return error; } if (S_ISREG(inode->i_mode) && (attr->ia_valid & ATTR_SIZE)) { @@ -1373,7 +1391,8 @@ static int shmem_setattr(struct mnt_idmap *idmap, if (oldsize > holebegin) unmap_mapping_range(inode->i_mapping, holebegin, 0, 1); - if (info->alloced) + /* Swapin-error markers are not counted in alloced. */ + if (info->alloced || !mapping_empty(inode->i_mapping)) shmem_truncate_range(inode, newsize, (loff_t)-1); /* unmap again to remove racily COWed private pages */ @@ -1544,6 +1563,10 @@ static int shmem_unuse_inode(struct inode *inode, unsigned int type) if (ret < 0) break; + if (signal_pending(current)) { + ret = -ERESTARTSYS; + break; + } start = indices[folio_batch_count(&fbatch) - 1]; } while (true); @@ -2171,11 +2194,10 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, folio_put_swap(folio, NULL); swap_cache_del_folio(folio); /* - * Don't treat swapin error folio as alloced. Otherwise inode->i_blocks - * won't be 0 when inode is released and thus trigger WARN_ON(i_blocks) - * in shmem_evict_inode(). + * Let shmem_recalc_inode() reduce alloced and release the block charge + * for the freed swap pages. */ - shmem_recalc_inode(inode, -nr_pages, -nr_pages); + shmem_recalc_inode(inode, 0, -nr_pages); } static int shmem_split_large_entry(struct inode *inode, pgoff_t index, @@ -2316,6 +2338,8 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, else error = -ENOMEM; folio = NULL; + /* Make sure to validate the original value. */ + swap = index_entry; goto failed; } if (fault_type) { @@ -5198,12 +5222,35 @@ static const struct file_operations shmem_file_operations = { #endif }; +#ifdef CONFIG_TMPFS_POSIX_ACL +static int shmem_set_acl(struct mnt_idmap *idmap, struct dentry *dentry, + struct posix_acl *acl, int type) +{ + struct inode *inode = d_inode(dentry); + umode_t mode = inode->i_mode; + int error; + + if (type == ACL_TYPE_ACCESS) { + error = posix_acl_equiv_mode(acl, &mode); + if (error < 0) + return error; + error = memfd_check_seals_mode(SHMEM_I(inode)->seals, + inode->i_mode, mode); + if (error) + return error; + } + return simple_set_acl(idmap, dentry, acl, type); +} +#else +#define shmem_set_acl simple_set_acl +#endif + static const struct inode_operations shmem_inode_operations = { .getattr = shmem_getattr, .setattr = shmem_setattr, #ifdef CONFIG_TMPFS_XATTR .listxattr = shmem_listxattr, - .set_acl = simple_set_acl, + .set_acl = shmem_set_acl, .fileattr_get = shmem_fileattr_get, .fileattr_set = shmem_fileattr_set, #endif diff --git a/mm/shmem_quota.c b/mm/shmem_quota.c index d0b92d6da50f..eeca9efd448a 100644 --- a/mm/shmem_quota.c +++ b/mm/shmem_quota.c @@ -234,7 +234,7 @@ static int shmem_acquire_dquot(struct dquot *dquot) return ret; } -static bool shmem_is_empty_dquot(struct dquot *dquot) +static bool shmem_is_default_dquot(struct dquot *dquot) { struct shmem_sb_info *sbinfo = dquot->dq_sb->s_fs_info; qsize_t bhardlimit; @@ -248,19 +248,21 @@ static bool shmem_is_empty_dquot(struct dquot *dquot) ihardlimit = sbinfo->qlimits.grpquota_ihardlimit; } - if (test_bit(DQ_FAKE_B, &dquot->dq_flags) || - (dquot->dq_dqb.dqb_curspace == 0 && - dquot->dq_dqb.dqb_curinodes == 0 && - dquot->dq_dqb.dqb_bhardlimit == bhardlimit && - dquot->dq_dqb.dqb_ihardlimit == ihardlimit)) - return true; - - return false; + /* + * Defaults are zero usage and soft limits, and the mount's hard limits. + * DQ_FAKE_B only means that *all* limits are zero, which need not match + * the mount defaults. + */ + return dquot->dq_dqb.dqb_curspace == 0 && + dquot->dq_dqb.dqb_curinodes == 0 && + dquot->dq_dqb.dqb_bsoftlimit == 0 && + dquot->dq_dqb.dqb_isoftlimit == 0 && + dquot->dq_dqb.dqb_bhardlimit == bhardlimit && + dquot->dq_dqb.dqb_ihardlimit == ihardlimit; } /* - * Store limits from dquot in the tree unless it's fake. If it is fake - * remove the id from the tree since there is no useful information in - * there. + * Remove the ID from the tree if it has no usage and its limits match + * the mount defaults. Otherwise, save its limits for the next acquire. */ static int shmem_release_dquot(struct dquot *dquot) { @@ -295,7 +297,7 @@ static int shmem_release_dquot(struct dquot *dquot) return -ENOENT; found: - if (shmem_is_empty_dquot(dquot)) { + if (shmem_is_default_dquot(dquot)) { /* Remove entry from the tree */ rb_erase(&entry->node, info->dqi_priv); kfree(entry); diff --git a/mm/swapfile.c b/mm/swapfile.c index 601979b97f95..278060e24113 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -2434,7 +2434,8 @@ static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd, return 0; swapcache = folio; - folio = ksm_might_need_to_copy(folio, vma, addr); + folio = ksm_might_need_to_copy(folio, + folio_file_page(folio, swp_offset(entry)), vma, addr); if (unlikely(!folio)) return -ENOMEM; else if (unlikely(folio == ERR_PTR(-EHWPOISON))) { @@ -2609,6 +2610,8 @@ static inline int unuse_pmd_range(struct vm_area_struct *vma, pud_t *pud, pmd = pmd_offset(pud, addr); do { cond_resched(); + if (signal_pending(current)) + return -ERESTARTSYS; next = pmd_addr_end(addr, end); ret = unuse_pte_range(vma, pmd, addr, next, type); if (ret) @@ -2823,7 +2826,7 @@ static int try_to_unuse(unsigned int type) if (swap_usage_in_pages(si)) { if (!signal_pending(current)) goto retry; - return -EINTR; + return -ERESTARTSYS; } success: diff --git a/mm/truncate.c b/mm/truncate.c index 29782aced8ba..b7c5d3384748 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -436,8 +436,11 @@ void truncate_inode_pages_range(struct address_space *mapping, } } + if (start >= end) + return; + index = start; - while (index < end) { + while (true) { cond_resched(); if (!find_get_entries(mapping, &index, end - 1, &fbatch, indices)) { @@ -460,6 +463,7 @@ void truncate_inode_pages_range(struct address_space *mapping, folio_lock(folio); VM_BUG_ON_FOLIO(!folio_contains(folio, indices[i]), folio); folio_wait_writeback(folio); + /* Do not handle errors, we'll do another lookup */ truncate_inode_folio(mapping, folio); folio_unlock(folio); } @@ -721,7 +725,12 @@ int invalidate_inode_pages2_range(struct address_space *mapping, folio_lock(folio); if (unlikely(folio->mapping != mapping)) { folio_unlock(folio); - continue; + /* + * Folio might have been replaced: look up + * again. + */ + index = indices[i]; + break; } VM_BUG_ON_FOLIO(!folio_contains(folio, indices[i]), folio); folio_wait_writeback(folio); diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 500962be1162..f8e2b7f2f61c 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -115,8 +115,7 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm, vma = vma_lookup(mm, addr); if (!vma) vma = ERR_PTR(-ENOENT); - else if (!(vma->vm_flags & VM_SHARED) && - unlikely(anon_vma_prepare(vma))) + else if (vma_is_cow_mapping(vma) && unlikely(anon_vma_prepare(vma))) vma = ERR_PTR(-ENOMEM); return vma; @@ -144,7 +143,7 @@ static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm, * We know we're going to need to use anon_vma, so check * that early. */ - if (!(vma->vm_flags & VM_SHARED) && unlikely(!vma->anon_vma)) + if (vma_is_cow_mapping(vma) && unlikely(!vma->anon_vma)) vma_end_read(vma); else return vma; @@ -1234,33 +1233,6 @@ int mrwprotect_range(struct userfaultfd_ctx *ctx, unsigned long start, return 0; } -void double_pt_lock(spinlock_t *ptl1, - spinlock_t *ptl2) - __acquires(ptl1) - __acquires(ptl2) -{ - if (ptl1 > ptl2) - swap(ptl1, ptl2); - /* lock in virtual address order to avoid lock inversion */ - spin_lock(ptl1); - if (ptl1 != ptl2) - spin_lock_nested(ptl2, SINGLE_DEPTH_NESTING); - else - __acquire(ptl2); -} - -void double_pt_unlock(spinlock_t *ptl1, - spinlock_t *ptl2) - __releases(ptl1) - __releases(ptl2) -{ - spin_unlock(ptl1); - if (ptl1 != ptl2) - spin_unlock(ptl2); - else - __release(ptl2); -} - static inline bool is_pte_pages_stable(pte_t *dst_pte, pte_t *src_pte, pte_t orig_dst_pte, pte_t orig_src_pte, pmd_t *dst_pmd, pmd_t dst_pmdval) @@ -4021,6 +3993,8 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx, ret = 0; for_each_vma_range(vmi, vma, end) { + struct userfaultfd_wake_range range; + cond_resched(); /* VMA not registered with userfaultfd. */ @@ -4035,18 +4009,17 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx, start = vma->vm_start; vma_end = min(end, vma->vm_end); - if (userfaultfd_missing(vma)) { - /* - * Wake any concurrent pending userfault while - * we unregister, so they will not hang - * permanently and it avoids userland to call - * UFFDIO_WAKE explicitly. - */ - struct userfaultfd_wake_range range; - range.start = start; - range.len = vma_end - start; - wake_userfault(vma->vm_userfaultfd_ctx.ctx, &range); - } + /* Prevent new userfaults from being queued after the wakeup. */ + vma_start_write(vma); + + /* + * Wake any concurrent pending userfault while we unregister, + * so they will not hang permanently and it avoids userland to + * call UFFDIO_WAKE explicitly. + */ + range.start = start; + range.len = vma_end - start; + wake_userfault(ctx, &range); vma = userfaultfd_clear_vma(&vmi, prev, vma, start, vma_end); diff --git a/mm/util.c b/mm/util.c index bf0513d1d3d0..e31eb565445c 100644 --- a/mm/util.c +++ b/mm/util.c @@ -697,6 +697,19 @@ void *vcalloc_noprof(size_t n, size_t size) } EXPORT_SYMBOL(vcalloc_noprof); +/** + * folio_anon_vma - Get a folio's raw anon_vma pointer. + * @folio: The folio. + * + * The returned pointer may be stale for an unmapped folio. The caller + * must ensure the anon_vma stays alive before dereferencing it. A folio + * reference or lock alone is insufficient; use folio_get_anon_vma() when + * the anon_vma's lifetime is not otherwise protected. Note that + * folio_get_anon_vma() is only useful for folios that are still mapped. + * + * Return: The raw anon_vma pointer for an anonymous, non-KSM folio, or + * %NULL otherwise. + */ struct anon_vma *folio_anon_vma(const struct folio *folio) { unsigned long mapping = (unsigned long)folio->mapping; diff --git a/mm/vma.c b/mm/vma.c index 3d63f73e723b..e382fd32f0ff 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -535,6 +535,15 @@ void unmap_region(struct unmap_desc *unmap) tlb_finish_mmu(&tlb); } +static int check_split(struct vm_area_struct *vma, unsigned long addr) +{ + if (!vma->vm_ops) + return 0; + if (!vma->vm_ops->may_split) + return 0; + return vma->vm_ops->may_split(vma, addr); +} + /* * __split_vma() bypasses sysctl_max_map_count checking. We use this where it * has already been checked or doesn't make sense to fail. @@ -551,12 +560,11 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, WARN_ON(vma->vm_start >= addr); WARN_ON(vma->vm_end <= addr); - if (vma->vm_ops && vma->vm_ops->may_split) { - err = vma->vm_ops->may_split(vma, addr); - if (err) - return err; - } + err = check_split(vma, addr); + if (err) + return err; + vma_start_write(vma); new = vm_area_dup(vma); if (!new) return -ENOMEM; @@ -587,7 +595,6 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, if (new->vm_ops && new->vm_ops->open) new->vm_ops->open(new); - vma_start_write(vma); vma_start_write(new); init_vma_prep(&vp, vma); @@ -964,6 +971,10 @@ static __must_check struct vm_area_struct *vma_merge_existing_range( if (vmg->__remove_middle && !can_merge_remove_vma(middle)) return NULL; + /* If middle isn't removed then it's resized, i.e. splits. */ + if (!vmg->__remove_middle && check_split(middle, left_side ? end : start)) + return NULL; + /* * If we merge both VMAs, then next is also deleted. This implies * merge_will_delete_vma also. @@ -1143,6 +1154,7 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg) struct vm_area_struct *next = vmg->next; unsigned long end = vmg->end; bool can_merge_left, can_merge_right; + bool move_iter_back = false; mmap_assert_write_locked(vmg->mm); VM_WARN_ON_VMG(vmg->middle, vmg); @@ -1182,23 +1194,30 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg) vmg->end = end; /* In expand-only case we are already positioned at prev. */ - if (!vmg->just_expand) { - /* Equivalent to going to the previous range. */ - vma_prev(vmg->vmi); - } + if (!vmg->just_expand) + move_iter_back = true; } + if (!can_merge_left && !can_merge_right) + return NULL; + + if (move_iter_back) + vma_prev(vmg->vmi); + /* * Now try to expand adjacent VMA(s). This takes care of removing the * following VMA if we have VMAs on both sides. */ - if (vmg->target && !vma_expand(vmg)) { - khugepaged_enter_vma(vmg->target, vmg->vm_flags); - vmg->state = VMA_MERGE_SUCCESS; - return vmg->target; + if (vma_expand(vmg)) { + /* Error occurred, restore iterator. */ + if (move_iter_back) + vma_iter_next_range(vmg->vmi); + return NULL; } - return NULL; + khugepaged_enter_vma(vmg->target, vmg->vm_flags); + vmg->state = VMA_MERGE_SUCCESS; + return vmg->target; } /* @@ -1293,7 +1312,7 @@ int vma_expand(struct vma_merge_struct *vmg) if (!ret && vmg->copied_from) ret = dup_anon_vma(target, vmg->copied_from, &anon_dup); if (ret) - return ret; + goto nomem; if (remove_next) { vma_flags_t next_sticky; @@ -3241,7 +3260,8 @@ static int acct_stack_growth(struct vm_area_struct *vma, * Overcommit.. This must be the final test, as it will * update security statistics. */ - if (security_vm_enough_memory_mm(mm, grow)) + if (vma_test(vma, VMA_ACCOUNT_BIT) && + security_vm_enough_memory_mm(mm, grow)) return -ENOMEM; return 0; diff --git a/mm/vma_init.c b/mm/vma_init.c index baa7e82f47e3..d661f229a734 100644 --- a/mm/vma_init.c +++ b/mm/vma_init.c @@ -127,6 +127,9 @@ struct vm_area_struct *vm_area_dup(struct vm_area_struct *orig) { struct vm_area_struct *new = kmem_cache_alloc(vm_area_cachep, GFP_KERNEL); + /* Must hold a write lock to stabilise atomic VMA flags. */ + vma_assert_write_locked(orig); + if (!new) return NULL; diff --git a/mm/vmscan.c b/mm/vmscan.c index 6a53762d5544..88fd31d13585 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3194,8 +3194,11 @@ struct ctrl_pos { int gain; }; -static void read_ctrl_pos(struct lruvec *lruvec, int type, int tier, int gain, - struct ctrl_pos *pos) +/* + * __noipa works around gcc-16 warning for uninitialized use of pos->refaulted + */ +static void __noipa read_ctrl_pos(struct lruvec *lruvec, int type, int tier_min, + int tier_max, int gain, struct ctrl_pos *pos) { int i; struct lru_gen_folio *lrugen = &lruvec->lrugen; @@ -3204,7 +3207,7 @@ static void read_ctrl_pos(struct lruvec *lruvec, int type, int tier, int gain, pos->gain = gain; pos->refaulted = pos->total = 0; - for (i = tier % MAX_NR_TIERS; i <= min(tier, MAX_NR_TIERS - 1); i++) { + for (i = tier_min; i <= tier_max; i++) { pos->refaulted += lrugen->avg_refaulted[type][i] + atomic_long_read(&lrugen->refaulted[hist][type][i]); pos->total += lrugen->avg_total[type][i] + @@ -3771,7 +3774,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end, walk->mm_stats[MM_NONLEAF_FOUND]++; - if (!walk_pte_range(&val, addr, next, args)) + if (!walk_pte_range(pmd + i, addr, next, args)) continue; walk->mm_stats[MM_NONLEAF_ADDED]++; @@ -3807,7 +3810,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long start, unsigned long end, if (!pud_present(val) || WARN_ON_ONCE(pud_leaf(val))) continue; - walk_pmd_range(&val, addr, next, args); + walk_pmd_range(pud + i, addr, next, args); if (need_resched() || walk->batched >= MAX_LRU_BATCH) { end = (addr | ~PUD_MASK) + 1; @@ -4801,16 +4804,16 @@ static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec, static int get_tier_idx(struct lruvec *lruvec, int type) { int tier; - struct ctrl_pos sp, pv = {}; + struct ctrl_pos sp, pv; /* * To leave a margin for fluctuations, use a larger gain factor (2:3). * This value is chosen because any other tier would have at least twice * as many refaults as the first tier. */ - read_ctrl_pos(lruvec, type, 0, 2, &sp); - for (tier = 1; tier < MAX_NR_TIERS; tier++) { - read_ctrl_pos(lruvec, type, tier, 3, &pv); + read_ctrl_pos(lruvec, type, LRU_TIER_MIN, LRU_TIER_MIN, 2, &sp); + for (tier = LRU_TIER_MIN + 1; tier <= LRU_TIER_MAX; tier++) { + read_ctrl_pos(lruvec, type, tier, tier, 3, &pv); if (!positive_ctrl_err(&sp, &pv)) break; } @@ -4820,7 +4823,7 @@ static int get_tier_idx(struct lruvec *lruvec, int type) static int get_type_to_scan(struct lruvec *lruvec, int swappiness) { - struct ctrl_pos sp, pv = {}; + struct ctrl_pos sp, pv; if (swappiness <= MIN_SWAPPINESS + 1) return LRU_GEN_FILE; @@ -4831,8 +4834,10 @@ static int get_type_to_scan(struct lruvec *lruvec, int swappiness) * Compare the sum of all tiers of anon with that of file to determine * which type to scan. */ - read_ctrl_pos(lruvec, LRU_GEN_ANON, MAX_NR_TIERS, swappiness, &sp); - read_ctrl_pos(lruvec, LRU_GEN_FILE, MAX_NR_TIERS, MAX_SWAPPINESS - swappiness, &pv); + read_ctrl_pos(lruvec, LRU_GEN_ANON, LRU_TIER_MIN, LRU_TIER_MAX, + swappiness, &sp); + read_ctrl_pos(lruvec, LRU_GEN_FILE, LRU_TIER_MIN, LRU_TIER_MAX, + MAX_SWAPPINESS - swappiness, &pv); return positive_ctrl_err(&sp, &pv); } diff --git a/net/9p/trans_usbg.c b/net/9p/trans_usbg.c index 419cda13a7b5..7d600e1288a3 100644 --- a/net/9p/trans_usbg.c +++ b/net/9p/trans_usbg.c @@ -149,7 +149,8 @@ static void usb9pfs_tx_complete(struct usb_ep *ep, struct usb_request *req) { struct f_usb9pfs *usb9pfs = ep->driver_data; struct usb_composite_dev *cdev = usb9pfs->function.config->cdev; - struct p9_req_t *p9_tx_req = req->context; + struct p9_client *client; + struct p9_req_t *p9_tx_req; unsigned long flags; /* reset zero packages */ @@ -165,18 +166,25 @@ static void usb9pfs_tx_complete(struct usb_ep *ep, struct usb_request *req) ep->name, req->status, req->actual, req->length); spin_lock_irqsave(&usb9pfs->lock, flags); - WRITE_ONCE(p9_tx_req->status, REQ_STATUS_SENT); + client = usb9pfs->client; + p9_tx_req = req->context; + req->context = NULL; - p9_req_put(usb9pfs->client, p9_tx_req); + if (!client || !p9_tx_req) + goto unlock_complete; - req->context = NULL; + WRITE_ONCE(p9_tx_req->status, REQ_STATUS_SENT); + + p9_req_put(client, p9_tx_req); +unlock_complete: spin_unlock_irqrestore(&usb9pfs->lock, flags); complete(&usb9pfs->send); } -static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf) +static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, + struct p9_client *client, void *buf) { struct p9_req_t *p9_rx_req; struct p9_fcall rc; @@ -202,7 +210,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf) "mux %p pkt: size: %d bytes tag: %d\n", usb9pfs, rc.size, rc.tag); - p9_rx_req = p9_tag_lookup(usb9pfs->client, rc.tag); + p9_rx_req = p9_tag_lookup(client, rc.tag); if (!p9_rx_req || p9_rx_req->status != REQ_STATUS_SENT) { p9_debug(P9_DEBUG_ERROR, "Unexpected packet tag %d\n", rc.tag); return NULL; @@ -212,7 +220,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf) p9_debug(P9_DEBUG_ERROR, "requested packet size too big: %d for tag %d with capacity %zd\n", rc.size, rc.tag, p9_rx_req->rc.capacity); - p9_req_put(usb9pfs->client, p9_rx_req); + p9_req_put(client, p9_rx_req); return NULL; } @@ -220,7 +228,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf) p9_debug(P9_DEBUG_ERROR, "No recv fcall for tag %d (req %p), disconnecting!\n", rc.tag, p9_rx_req); - p9_req_put(usb9pfs->client, p9_rx_req); + p9_req_put(client, p9_rx_req); return NULL; } @@ -231,8 +239,10 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req) { struct f_usb9pfs *usb9pfs = ep->driver_data; struct usb_composite_dev *cdev = usb9pfs->function.config->cdev; + struct p9_client *client; struct p9_req_t *p9_rx_req; unsigned int req_size = req->actual; + unsigned long flags; int status = REQ_STATUS_RCVD; if (req->status) { @@ -241,9 +251,16 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req) return; } - p9_rx_req = usb9pfs_rx_header(usb9pfs, req->buf); - if (!p9_rx_req) + spin_lock_irqsave(&usb9pfs->lock, flags); + client = usb9pfs->client; + if (!client) { + spin_unlock_irqrestore(&usb9pfs->lock, flags); return; + } + + p9_rx_req = usb9pfs_rx_header(usb9pfs, client, req->buf); + if (!p9_rx_req) + goto out_unlock; if (req_size > p9_rx_req->rc.capacity) { dev_err(&cdev->gadget->dev, @@ -257,8 +274,11 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req) p9_rx_req->rc.size = req_size; - p9_client_cb(usb9pfs->client, p9_rx_req, status); - p9_req_put(usb9pfs->client, p9_rx_req); + p9_client_cb(client, p9_rx_req, status); + p9_req_put(client, p9_rx_req); + +out_unlock: + spin_unlock_irqrestore(&usb9pfs->lock, flags); complete(&usb9pfs->received); } @@ -278,18 +298,18 @@ static void disable_usb9pfs(struct f_usb9pfs *usb9pfs) struct usb_composite_dev *cdev = usb9pfs->function.config->cdev; + disable_ep(cdev, usb9pfs->in_ep); + disable_ep(cdev, usb9pfs->out_ep); + if (usb9pfs->in_req) { usb_ep_free_request(usb9pfs->in_ep, usb9pfs->in_req); usb9pfs->in_req = NULL; } if (usb9pfs->out_req) { - usb_ep_free_request(usb9pfs->out_ep, usb9pfs->out_req); + free_ep_req(usb9pfs->out_ep, usb9pfs->out_req); usb9pfs->out_req = NULL; } - - disable_ep(cdev, usb9pfs->in_ep); - disable_ep(cdev, usb9pfs->out_ep); dev_dbg(&cdev->gadget->dev, "%s disabled\n", usb9pfs->function.name); } @@ -314,10 +334,6 @@ static int alloc_requests(struct usb_composite_dev *cdev, usb9pfs->in_req->complete = usb9pfs_tx_complete; usb9pfs->out_req->complete = usb9pfs_rx_complete; - /* length will be set in complete routine */ - usb9pfs->in_req->context = usb9pfs; - usb9pfs->out_req->context = usb9pfs; - return 0; fail_in: @@ -391,7 +407,7 @@ static int p9_usbg_create(struct p9_client *client, struct fs_context *fc) guard(mutex)(&usb9pfs_lock); list_for_each_entry(dev, &usbg_instance_list, usb9pfs_instance) { - if (!strncmp(devname, dev->tag, strlen(devname))) { + if (!strcmp(devname, dev->tag)) { if (!dev->inuse) { dev->inuse = true; found = 1; @@ -416,7 +432,9 @@ static int p9_usbg_create(struct p9_client *client, struct fs_context *fc) client->status = Disconnected; else client->status = Connected; + spin_lock_irq(&usb9pfs->lock); usb9pfs->client = client; + spin_unlock_irq(&usb9pfs->lock); client->trans_mod->maxsize = usb9pfs->buflen; @@ -427,18 +445,26 @@ static int p9_usbg_create(struct p9_client *client, struct fs_context *fc) static void usb9pfs_clear_tx(struct f_usb9pfs *usb9pfs) { + struct p9_client *client; struct p9_req_t *req; + unsigned long flags; - guard(spinlock_irqsave)(&usb9pfs->lock); + spin_lock_irqsave(&usb9pfs->lock, flags); + client = usb9pfs->client; + usb9pfs->client = NULL; + req = usb9pfs->in_req ? usb9pfs->in_req->context : NULL; + if (usb9pfs->in_req) + usb9pfs->in_req->context = NULL; + spin_unlock_irqrestore(&usb9pfs->lock, flags); - req = usb9pfs->in_req->context; - if (!req) + if (!req || !client) return; if (!req->t_err) req->t_err = -ECONNRESET; - p9_client_cb(usb9pfs->client, req, REQ_STATUS_ERROR); + p9_client_cb(client, req, REQ_STATUS_ERROR); + usb9pfs->in_req->context = NULL; } static void p9_usbg_close(struct p9_client *client) @@ -466,6 +492,9 @@ static void p9_usbg_close(struct p9_client *client) mutex_lock(&usb9pfs_lock); dev->inuse = false; mutex_unlock(&usb9pfs_lock); + + disable_usb9pfs(usb9pfs); + reinit_completion(&usb9pfs->send); } static int p9_usbg_request(struct p9_client *client, struct p9_req_t *p9_req) @@ -748,8 +777,18 @@ static int usb9pfs_set_alt(struct usb_function *f, static void usb9pfs_disable(struct usb_function *f) { struct f_usb9pfs *usb9pfs = func_to_usb9pfs(f); + unsigned long flags; + spin_lock_irqsave(&usb9pfs->lock, flags); + if (usb9pfs->client) + usb9pfs->client->status = Disconnected; + spin_unlock_irqrestore(&usb9pfs->lock, flags); usb9pfs_clear_tx(usb9pfs); + + if (usb9pfs->in_ep->enabled) + disable_usb9pfs(usb9pfs); + + reinit_completion(&usb9pfs->send); } static struct usb_function *usb9pfs_alloc(struct usb_function_instance *fi) diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c index b0d0094ec8e2..9fa63e3ebf67 100644 --- a/net/9p/trans_virtio.c +++ b/net/9p/trans_virtio.c @@ -633,6 +633,11 @@ static int p9_virtio_probe(struct virtio_device *vdev) err = -EINVAL; goto out_free_vq; } + if (tag_len > NAME_MAX) { + dev_err(&vdev->dev, "mount tag too long (%u bytes)\n", tag_len); + err = -EINVAL; + goto out_free_vq; + } tag = kzalloc(tag_len + 1, GFP_KERNEL); if (!tag) { err = -ENOMEM; diff --git a/net/9p/trans_xen.c b/net/9p/trans_xen.c index f9fb2db7a066..71a4d276321e 100644 --- a/net/9p/trans_xen.c +++ b/net/9p/trans_xen.c @@ -200,9 +200,19 @@ static void p9_xen_response(struct work_struct *work) masked_prod, &masked_cons, XEN_9PFS_RING_SIZE(ring)); + if (h.size < sizeof(h) || + h.size > xen_9pfs_queued(prod, cons, + XEN_9PFS_RING_SIZE(ring))) { + dev_warn_ratelimited(&priv->dev->dev, + "bad response size %u from backend\n", h.size); + break; + } + req = p9_tag_lookup(priv->client, h.tag); if (!req || req->status != REQ_STATUS_SENT) { dev_warn(&priv->dev->dev, "Wrong req tag=%x\n", h.tag); + if (req) + p9_req_put(priv->client, req); cons += h.size; virt_mb(); ring->intf->in_cons = cons; diff --git a/net/batman-adv/bat_iv_ogm.c b/net/batman-adv/bat_iv_ogm.c index 53fbdbbe8f4f..1c2ab79adabc 100644 --- a/net/batman-adv/bat_iv_ogm.c +++ b/net/batman-adv/bat_iv_ogm.c @@ -1909,6 +1909,11 @@ static void batadv_iv_send_outstanding_bat_ogm_packet(struct work_struct *work) goto out; } + /* Mark aggregate as full before forcing emit. */ + spin_lock_bh(&bat_priv->forw_bat_list_lock); + forw_packet->num_packets = BATADV_MAX_AGGREGATION_PACKETS; + spin_unlock_bh(&bat_priv->forw_bat_list_lock); + batadv_iv_ogm_emit(forw_packet); /* we have to have at least one packet in the queue to determine the diff --git a/net/batman-adv/distributed-arp-table.c b/net/batman-adv/distributed-arp-table.c index 011cfdc47fab..86791ddc148f 100644 --- a/net/batman-adv/distributed-arp-table.c +++ b/net/batman-adv/distributed-arp-table.c @@ -653,6 +653,7 @@ static void batadv_choose_next_candidate(struct batadv_priv *bat_priv, * NOT_FOUND */ cands[select].type = BATADV_DAT_CANDIDATE_NOT_FOUND; + cands[select].orig_node = NULL; /* iterate over the originator list and find the node with the closest * dat_address which has not been selected yet diff --git a/net/batman-adv/main.c b/net/batman-adv/main.c index d89d44706269..5d5a0be323e6 100644 --- a/net/batman-adv/main.c +++ b/net/batman-adv/main.c @@ -294,6 +294,11 @@ void batadv_mesh_free(struct net_device *mesh_iface) WRITE_ONCE(bat_priv->mesh_state, BATADV_MESH_DEACTIVATING); + /* the originator purge worker accesses (among others) the TT tables + * and must therefore be stopped before any of them are freed + */ + disable_delayed_work_sync(&bat_priv->orig_work); + batadv_purge_outstanding_packets(bat_priv, NULL); batadv_tp_stop_all(bat_priv); @@ -483,8 +488,10 @@ int batadv_batman_skb_recv(struct sk_buff *skb, struct net_device *dev, * shut down otherwise the packet may trigger de-reference errors * further down in the receive path. */ - if (!kref_get_unless_zero(&hard_iface->refcount)) - goto err_out; + if (!kref_get_unless_zero(&hard_iface->refcount)) { + kfree_skb(skb); + return NET_RX_DROP; + } skb = skb_share_check(skb, GFP_ATOMIC); @@ -540,7 +547,6 @@ int batadv_batman_skb_recv(struct sk_buff *skb, struct net_device *dev, kfree_skb(skb); err_put: batadv_hardif_put(hard_iface); -err_out: return NET_RX_DROP; } diff --git a/net/batman-adv/multicast_forw.c b/net/batman-adv/multicast_forw.c index ac97cd8b81dc..9ffed1f237b6 100644 --- a/net/batman-adv/multicast_forw.c +++ b/net/batman-adv/multicast_forw.c @@ -950,8 +950,11 @@ static int batadv_mcast_forw_packet(struct batadv_priv *bat_priv, dest = (u8 *)mcast_tracker + sizeof(*mcast_tracker); - /* check if full tracker tvlv is within skb length */ + /* check if full tracker tvlv, including padding, is within skb length */ tvlv_len = sizeof(*mcast_tracker) + ETH_ALEN * num_dests; + if (!(num_dests % 2)) + tvlv_len += 2; + if (tvlv_len > skb_network_header_len(skb)) return -EINVAL; diff --git a/net/batman-adv/tp_meter.c b/net/batman-adv/tp_meter.c index 022eb13e8ec4..56feb02d2b7e 100644 --- a/net/batman-adv/tp_meter.c +++ b/net/batman-adv/tp_meter.c @@ -221,7 +221,7 @@ static void batadv_tp_batctl_notify(enum batadv_tp_meter_reason reason, unsigned long start_time, u64 total_sent, u32 cookie) { - u32 total_bytes; + u64 total_bytes; u32 test_time; u8 result; diff --git a/net/batman-adv/translation-table.c b/net/batman-adv/translation-table.c index 68f79853032e..ed8dcb8a66b1 100644 --- a/net/batman-adv/translation-table.c +++ b/net/batman-adv/translation-table.c @@ -418,6 +418,9 @@ batadv_tt_orig_list_entry_put(struct batadv_tt_orig_list_entry *orig_entry) * @bat_priv: the bat priv with all the mesh interface information * @tt_local_entry: the TT entry involved in the event * @event_flags: flags to store in the event structure + * + * Only a single change per client is queued. It describes the latest state of + * the client and a new event therefore replaces an already queued one. */ static void batadv_tt_local_event(struct batadv_priv *bat_priv, struct batadv_tt_local_entry *tt_local_entry, @@ -429,14 +432,12 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv, struct batadv_tt_change_node *entry; struct batadv_tt_change_node *safe; bool del_op_requested; - bool del_op_entry; size_t changes; tt_change_node = kmem_cache_alloc(batadv_tt_change_cache, GFP_ATOMIC); if (!tt_change_node) return; - tt_change_node->change.flags = flags; memset(tt_change_node->change.reserved, 0, sizeof(tt_change_node->change.reserved)); ether_addr_copy(tt_change_node->change.addr, common->addr); @@ -444,6 +445,15 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv, del_op_requested = flags & BATADV_TT_CLIENT_DEL; + /* the ROAM flag is only meaningful for the receiver in combination + * with a DEL. An ADD with ROAM would be interpreted as a roaming + * advertisement + */ + if (!del_op_requested) + flags &= ~BATADV_TT_CLIENT_ROAM; + + tt_change_node->change.flags = flags; + /* check for ADD+DEL, DEL+ADD, ADD+ADD or DEL+DEL events */ spin_lock_bh(&bat_priv->tt.changes_list_lock); changes = READ_ONCE(bat_priv->tt.local_changes); @@ -455,25 +465,12 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv, if (entry->change.vid != tt_change_node->change.vid) continue; - del_op_entry = entry->change.flags & BATADV_TT_CLIENT_DEL; - if (del_op_requested != del_op_entry) { - /* DEL+ADD in the same orig interval have no effect and - * can be removed to avoid silly behaviour on the - * receiver side. The other way around (ADD+DEL) can - * happen in case of roaming of a client still in the - * NEW state. Roaming of NEW clients is now possible due - * to automatically recognition of "temporary" clients - */ - list_del(&entry->list); - kmem_cache_free(batadv_tt_change_cache, entry); - changes--; - } else { - /* this is a second add or del in the same originator - * interval. It could mean that flags have been changed - * (e.g. double add): update them - */ - entry->change.flags = flags; - } + /* the other nodes may know the client with different + * flags. A DEL+ADD must therefore still announce the + * current flags and an ADD+DEL must still remove the + * client. ADD+ADD and DEL+DEL only update the flags + */ + entry->change.flags = flags; kmem_cache_free(batadv_tt_change_cache, tt_change_node); goto update_changes; @@ -624,6 +621,7 @@ bool batadv_tt_local_add(struct net_device *mesh_iface, const u8 *addr, bool iif_is_wifi = false; struct hlist_head *head; int packet_size_max; + bool added = false; bool ret = false; u8 remote_flags; int hash_added; @@ -656,7 +654,7 @@ bool batadv_tt_local_add(struct net_device *mesh_iface, const u8 *addr, * flag can be reset like it was never enqueued */ tt_local->common.flags &= ~BATADV_TT_CLIENT_PENDING; - goto add_event; + added = true; } if (tt_local->common.flags & BATADV_TT_CLIENT_ROAM) { @@ -737,14 +735,17 @@ bool batadv_tt_local_add(struct net_device *mesh_iface, const u8 *addr, goto out; } -add_event: - batadv_tt_local_event(bat_priv, tt_local, BATADV_NO_FLAGS); + added = true; check_roaming: + if (added) + batadv_tt_local_event(bat_priv, tt_local, BATADV_NO_FLAGS); + /* Check whether it is a roaming, but don't do anything if the roaming * process has already been handled */ - if (tt_global && !(tt_global->common.flags & BATADV_TT_CLIENT_ROAM)) { + if (tt_global && + (roamed_back || !(tt_global->common.flags & BATADV_TT_CLIENT_ROAM))) { /* These node are probably going to update their tt table */ head = &tt_global->orig_list; rcu_read_lock(); @@ -1006,22 +1007,24 @@ batadv_tt_prepare_tvlv_local_data(struct batadv_priv *bat_priv, * batadv_tt_tvlv_container_update() - update the translation table tvlv * container after local tt changes have been committed * @bat_priv: the bat priv with all the mesh interface information + * @changes_list: list of committed changes which will be consumed + * @tt_diff_entries_num: number of entries in @changes_list */ -static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv) +static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv, + struct list_head *changes_list, + int tt_diff_entries_num) { struct batadv_tvlv_tt_change *tt_change; struct batadv_tt_change_node *entry; struct batadv_tvlv_tt_data *tt_data; struct batadv_tt_change_node *safe; int tt_diff_entries_count = 0; - int tt_diff_entries_num = 0; bool drop_changes = false; size_t tt_extra_len = 0; int tt_change_len = 0; int tt_diff_len; u16 tvlv_len; - tt_diff_entries_num = READ_ONCE(bat_priv->tt.local_changes); tt_diff_len = batadv_tt_len(tt_diff_entries_num); /* if we have too many changes for one packet don't send any @@ -1039,19 +1042,39 @@ static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv) tvlv_len = batadv_tt_prepare_tvlv_local_data(bat_priv, &tt_data, &tt_change, &tt_diff_len); - if (!tvlv_len) + if (!tvlv_len) { + /* the changes are lost and the other nodes have to request the + * full table. Don't answer TT requests for the new TTVN with + * the stale changeset of the previous TTVN. Without changes, the + * TTVN was not incremented and the last changeset is still valid + */ + if (drop_changes || tt_diff_entries_num > 0) { + list_for_each_entry_safe(entry, safe, changes_list, + list) { + list_del(&entry->list); + kmem_cache_free(batadv_tt_change_cache, entry); + } + + spin_lock_bh(&bat_priv->tt.last_changeset_lock); + kfree(bat_priv->tt.last_changeset); + bat_priv->tt.last_changeset = NULL; + bat_priv->tt.last_changeset_len = 0; + spin_unlock_bh(&bat_priv->tt.last_changeset_lock); + } + + /* announce the already committed TTVN in the next OGM interval + * instead of waiting for BATADV_TT_OGM_APPEND_MAX idle rounds + */ + atomic_set(&bat_priv->tt.ogm_append_cnt, 0); return; + } tt_data->flags = BATADV_TT_OGM_DIFF; if (!drop_changes && tt_diff_len == 0) goto container_register; - spin_lock_bh(&bat_priv->tt.changes_list_lock); - WRITE_ONCE(bat_priv->tt.local_changes, 0); - - list_for_each_entry_safe(entry, safe, &bat_priv->tt.changes_list, - list) { + list_for_each_entry_safe(entry, safe, changes_list, list) { if (tt_diff_entries_count < tt_diff_entries_num) { memcpy(tt_change + tt_diff_entries_count, &entry->change, @@ -1061,7 +1084,6 @@ static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv) list_del(&entry->list); kmem_cache_free(batadv_tt_change_cache, entry); } - spin_unlock_bh(&bat_priv->tt.changes_list_lock); tt_extra_len = batadv_tt_len(tt_diff_entries_num - tt_diff_entries_count); @@ -2830,8 +2852,12 @@ static bool batadv_tt_local_valid(const void *entry_ptr, if (tt_common_entry->flags & BATADV_TT_CLIENT_NEW) return false; + /* the entry is (still) announced. A ROAM flag of an entry which is + * pending to be removed must not be transmitted. Otherwise, the + * receiver would handle it like a roaming advertisement + */ if (flags) - *flags = tt_common_entry->flags; + *flags = tt_common_entry->flags & ~BATADV_TT_CLIENT_ROAM; return true; } @@ -3408,21 +3434,30 @@ static void _batadv_tt_update_changes(struct batadv_priv *bat_priv, u16 tt_num_changes, u8 ttvn) { int roams; + u8 flags; int i; for (i = 0; i < tt_num_changes; i++) { - if ((tt_change + i)->flags & BATADV_TT_CLIENT_DEL) { - roams = (tt_change + i)->flags & BATADV_TT_CLIENT_ROAM; + flags = (tt_change + i)->flags; + + if (flags & BATADV_TT_CLIENT_DEL) { + roams = flags & BATADV_TT_CLIENT_ROAM; batadv_tt_global_del(bat_priv, orig_node, (tt_change + i)->addr, ntohs((tt_change + i)->vid), "tt removed by changes", roams); } else { + /* an announced client cannot have roamed away from + * orig_node. Only a ROAMING_ADV is allowed to add a + * global entry with the ROAM flag + */ + flags &= ~BATADV_TT_CLIENT_ROAM; + if (!batadv_tt_global_add(bat_priv, orig_node, (tt_change + i)->addr, ntohs((tt_change + i)->vid), - (tt_change + i)->flags, ttvn)) + flags, ttvn)) /* In case of problem while storing a * global_entry, we stop the updating * procedure without committing the @@ -3883,11 +3918,25 @@ static void batadv_tt_local_purge_pending_clients(struct batadv_priv *bat_priv) */ static void batadv_tt_local_commit_changes_nolock(struct batadv_priv *bat_priv) { + LIST_HEAD(changes_list); + int changes_num; + lockdep_assert_held(&bat_priv->tt.commit_lock); - if (READ_ONCE(bat_priv->tt.local_changes) == 0) { + /* atomically get the list of changes. The TTVN decision and the + * announced changeset must cover the same events. Events queued + * afterwards stay in the tt.changes_list for the next TT round + */ + spin_lock_bh(&bat_priv->tt.changes_list_lock); + list_splice_init(&bat_priv->tt.changes_list, &changes_list); + changes_num = READ_ONCE(bat_priv->tt.local_changes); + WRITE_ONCE(bat_priv->tt.local_changes, 0); + spin_unlock_bh(&bat_priv->tt.changes_list_lock); + + if (changes_num == 0) { if (!batadv_atomic_dec_not_zero(&bat_priv->tt.ogm_append_cnt)) - batadv_tt_tvlv_container_update(bat_priv); + batadv_tt_tvlv_container_update(bat_priv, &changes_list, + changes_num); return; } @@ -3904,7 +3953,7 @@ static void batadv_tt_local_commit_changes_nolock(struct batadv_priv *bat_priv) /* reset the sending counter */ atomic_set(&bat_priv->tt.ogm_append_cnt, BATADV_TT_OGM_APPEND_MAX); - batadv_tt_tvlv_container_update(bat_priv); + batadv_tt_tvlv_container_update(bat_priv, &changes_list, changes_num); } /** diff --git a/net/batman-adv/types.h b/net/batman-adv/types.h index 9bdc5a3e799e..db0d4f680e65 100644 --- a/net/batman-adv/types.h +++ b/net/batman-adv/types.h @@ -2019,7 +2019,13 @@ struct batadv_forw_packet { /** @direct_link_flags: direct link flags for aggregated OGM packets */ DECLARE_BITMAP(direct_link_flags, BATADV_MAX_AGGREGATION_PACKETS); - /** @num_packets: counter for aggregated OGMv1 packets */ + /** + * @num_packets: counter for aggregated OGMv1 packets. Set to + * BATADV_MAX_AGGREGATION_PACKETS (independent of the real number of + * aggregated packets) by the send worker to prevent further + * aggregation while the packet is emitted. It must therefore not be + * used to iterate over the aggregated packets + */ u8 num_packets; /** @delayed_work: work queue callback item for packet sending */ diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c index a92a81846e9d..5b99da7028b9 100644 --- a/net/bluetooth/hci_sync.c +++ b/net/bluetooth/hci_sync.c @@ -2687,7 +2687,7 @@ static int hci_pause_advertising_sync(struct hci_dev *hdev) static int hci_resume_advertising_sync(struct hci_dev *hdev) { struct adv_info *adv, *tmp; - int err; + int err = 0; /* If advertising has not been paused there is nothing to do. */ if (!hdev->advertising_paused) @@ -2720,13 +2720,19 @@ static int hci_resume_advertising_sync(struct hci_dev *hdev) */ if (hci_dev_test_and_clear_flag(hdev, HCI_LE_ADV_0)) err = hci_enable_ext_advertising_sync(hdev, 0x00); - } else { + } else if (hdev->cur_adv_instance) { /* Schedule for most recent instance to be restarted and begin * the software rotation loop */ err = hci_schedule_adv_instance_sync(hdev, hdev->cur_adv_instance, true); + } else if (hci_dev_test_flag(hdev, HCI_ADVERTISING) && + !hci_dev_test_flag(hdev, HCI_LE_ADV)) { + /* hci_schedule_adv_instance_sync() rejects instance 0x00 while + * HCI_ADVERTISING is set, so enable it directly. + */ + err = hci_start_adv_sync(hdev, 0x00); } hdev->advertising_paused = false; @@ -6272,6 +6278,7 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval) u8 filter_policy = 0x00; /* Default is to enable duplicates filter */ u8 filter_dup = LE_SCAN_FILTER_DUP_ENABLE; + bool paused; int err; bt_dev_dbg(hdev, ""); @@ -6295,6 +6302,12 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval) if (err) goto failed; + /* LE Set Random Address is disallowed while advertising is enabled. */ + paused = !hdev->advertising_paused; + err = hci_pause_advertising_sync(hdev); + if (err) + goto failed; + /* All active scans will be done with either a resolvable private * address (when privacy feature has been enabled) or non-resolvable * private address. @@ -6304,6 +6317,9 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval) if (err < 0) own_addr_type = ADDR_LE_DEV_PUBLIC; + if (paused) + hci_resume_advertising_sync(hdev); + if (hci_is_adv_monitoring(hdev) || (hci_test_quirk(hdev, HCI_QUIRK_STRICT_DUPLICATE_FILTER) && hdev->discovery.result_filtering)) { @@ -6327,9 +6343,8 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval) return err; failed: - /* Resume advertising if it was paused */ - if (ll_privacy_capable(hdev)) - hci_resume_advertising_sync(hdev); + /* No-op when advertising was not paused. */ + hci_resume_advertising_sync(hdev); /* Resume passive scanning */ hci_update_passive_scan_sync(hdev); diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c index b86999899f66..47f0449588ea 100644 --- a/net/bluetooth/l2cap_core.c +++ b/net/bluetooth/l2cap_core.c @@ -158,6 +158,28 @@ static struct l2cap_chan *__l2cap_get_chan_by_ident(struct l2cap_conn *conn, return NULL; } +/* Our connection request has been sent and is still outstanding */ +static bool l2cap_chan_connect_pending(struct l2cap_chan *chan) +{ + return chan->state == BT_CONNECT && + test_bit(CONF_CONNECT_PEND, &chan->conf_state); +} + +/* Other channels may hold the same ident for a response they still have + * to send, so only look at channels waiting for a connection response. + */ +static struct l2cap_chan *__l2cap_get_connect_chan_by_ident(struct l2cap_conn *conn, + u8 ident) +{ + struct l2cap_chan *c; + + list_for_each_entry(c, &conn->chan_l, list) { + if (c->ident == ident && l2cap_chan_connect_pending(c)) + return c; + } + return NULL; +} + static struct l2cap_chan *__l2cap_global_chan_by_addr(__le16 psm, bdaddr_t *src, u8 src_type) { @@ -4298,7 +4320,7 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn, if (!chan) return -EBADSLT; } else { - chan = __l2cap_get_chan_by_ident(conn, cmd->ident); + chan = __l2cap_get_connect_chan_by_ident(conn, cmd->ident); if (!chan) return -EBADSLT; } @@ -4311,6 +4333,13 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn, l2cap_chan_lock(chan); + /* Only accept a response matching a connection request we sent and + * that is still outstanding. A channel waiting in BT_CONNECT for + * security or the feature mask has not sent one yet. + */ + if (!l2cap_chan_connect_pending(chan) || chan->ident != cmd->ident) + goto done; + switch (result) { case L2CAP_CR_SUCCESS: if (__l2cap_get_chan_by_dcid(conn, dcid)) { @@ -4340,6 +4369,7 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn, break; } +done: l2cap_chan_unlock(chan); l2cap_chan_put(chan); @@ -4472,7 +4502,13 @@ static inline int l2cap_config_req(struct l2cap_conn *conn, goto unlock; } - if (!test_and_set_bit(CONF_REQ_SENT, &chan->conf_state)) { + /* A channel in BT_CONNECT2 has not passed security or userspace + * accept yet. Answer the peer's request, but hold our own until + * the channel reaches BT_CONFIG: the CONNECT2 -> CONFIG sites send + * it. Connected channels may also need to reconfigure. + */ + if ((chan->state == BT_CONFIG || chan->state == BT_CONNECTED) && + !test_and_set_bit(CONF_REQ_SENT, &chan->conf_state)) { u8 buf[64]; l2cap_send_cmd(conn, l2cap_get_ident(conn), L2CAP_CONF_REQ, l2cap_build_conf_req(chan, buf, sizeof(buf)), buf); @@ -4520,6 +4556,10 @@ static inline int l2cap_config_rsp(struct l2cap_conn *conn, if (!chan) return 0; + /* Connected channels may also receive reconfiguration responses. */ + if (chan->state != BT_CONFIG && chan->state != BT_CONNECTED) + goto done; + switch (result) { case L2CAP_CONF_SUCCESS: l2cap_conf_rfc_get(chan, rsp->data, len); @@ -4587,6 +4627,9 @@ static inline int l2cap_config_rsp(struct l2cap_conn *conn, if (test_bit(CONF_OUTPUT_DONE, &chan->conf_state)) { set_default_fcs(chan); + if (chan->state != BT_CONFIG) + goto done; + if (chan->mode == L2CAP_MODE_ERTM || chan->mode == L2CAP_MODE_STREAMING) err = l2cap_ertm_init(chan); diff --git a/net/can/af_can.c b/net/can/af_can.c index 34fe3b28d576..167b2dc40eeb 100644 --- a/net/can/af_can.c +++ b/net/can/af_can.c @@ -226,7 +226,7 @@ int can_send(struct sk_buff *skb, int loop) goto inval_skb; } - if (unlikely(skb->dev->type != ARPHRD_CAN)) { + if (unlikely(!can_get_ml_priv(skb->dev))) { err = -EPERM; goto inval_skb; } @@ -452,7 +452,7 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id, /* insert new receiver (dev,canid,mask) -> (func,data) */ - if (dev && (dev->type != ARPHRD_CAN || !can_get_ml_priv(dev))) + if (dev && !can_get_ml_priv(dev)) return -ENODEV; if (dev && !net_eq(net, dev_net(dev))) @@ -462,9 +462,10 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id, if (!rcv) return -ENOMEM; - spin_lock_bh(&net->can.rcvlists_lock); - dev_rcv_lists = can_dev_rcv_lists_find(net, dev); + + spin_lock_bh(&dev_rcv_lists->lock); + rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists); rcv->can_id = can_id; @@ -478,6 +479,9 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id, hlist_add_head_rcu(&rcv->list, rcv_list); dev_rcv_lists->entries++; + spin_unlock_bh(&dev_rcv_lists->lock); + + spin_lock_bh(&net->can.rcvlists_lock); rcv_lists_stats->rcv_entries++; rcv_lists_stats->rcv_entries_max = max(rcv_lists_stats->rcv_entries_max, rcv_lists_stats->rcv_entries); @@ -509,6 +513,14 @@ static void can_rx_delete_receiver(struct rcu_head *rp) * * Description: * Removes subscription entry depending on given (subscription) values. + * The receive lists of a CAN device are protected by a per-device lock, + * so the subscription is found even when the device has been moved to + * another network namespace after can_rx_register(). The caller has to + * ensure that @dev stays valid, e.g. by holding a reference to @dev or + * RTNL. @net should be the namespace that has been used for + * can_rx_register(). It selects the 'any' CAN device lists for + * @dev == NULL and is otherwise only used for the receive list + * statistics, which become imprecise when another namespace is passed. */ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id, canid_t mask, void (*func)(struct sk_buff *, void *), @@ -519,15 +531,13 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id, struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats; struct can_dev_rcv_lists *dev_rcv_lists; - if (dev && dev->type != ARPHRD_CAN) + if (dev && !can_get_ml_priv(dev)) return; - if (dev && !net_eq(net, dev_net(dev))) - return; + dev_rcv_lists = can_dev_rcv_lists_find(net, dev); - spin_lock_bh(&net->can.rcvlists_lock); + spin_lock_bh(&dev_rcv_lists->lock); - dev_rcv_lists = can_dev_rcv_lists_find(net, dev); rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists); /* Search the receiver list for the item to delete. This should @@ -547,26 +557,26 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id, * a warning here. */ if (!rcv) { + spin_unlock_bh(&dev_rcv_lists->lock); pr_warn("can: receive list entry not found for dev %s, id %03X, mask %03X\n", DNAME(dev), can_id, mask); - goto out; + return; } hlist_del_rcu(&rcv->list); dev_rcv_lists->entries--; + spin_unlock_bh(&dev_rcv_lists->lock); + + spin_lock_bh(&net->can.rcvlists_lock); if (rcv_lists_stats->rcv_entries > 0) rcv_lists_stats->rcv_entries--; - - out: spin_unlock_bh(&net->can.rcvlists_lock); /* schedule the receiver item for deletion */ - if (rcv) { - if (rcv->sk) - sock_hold(rcv->sk); - call_rcu(&rcv->rcu, can_rx_delete_receiver); - } + if (rcv->sk) + sock_hold(rcv->sk); + call_rcu(&rcv->rcu, can_rx_delete_receiver); } EXPORT_SYMBOL(can_rx_unregister); @@ -684,8 +694,7 @@ static int can_rcv(struct sk_buff *skb, struct net_device *dev, { struct can_skb_ext *csx = can_skb_ext_find(skb); - if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || - !csx || !can_is_can_skb(skb))) { + if (unlikely(!can_get_ml_priv(dev) || !csx || !can_is_can_skb(skb))) { pr_warn_once("PF_CAN: dropped non conform CAN skbuff: dev type %d, len %d\n", dev->type, skb->len); @@ -710,8 +719,7 @@ static int canfd_rcv(struct sk_buff *skb, struct net_device *dev, { struct can_skb_ext *csx = can_skb_ext_find(skb); - if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || - !csx || !can_is_canfd_skb(skb))) { + if (unlikely(!can_get_ml_priv(dev) || !csx || !can_is_canfd_skb(skb))) { pr_warn_once("PF_CAN: dropped non conform CAN FD skbuff: dev type %d, len %d\n", dev->type, skb->len); @@ -736,8 +744,7 @@ static int canxl_rcv(struct sk_buff *skb, struct net_device *dev, { struct can_skb_ext *csx = can_skb_ext_find(skb); - if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || - !csx || !can_is_canxl_skb(skb))) { + if (unlikely(!can_get_ml_priv(dev) || !csx || !can_is_canxl_skb(skb))) { pr_warn_once("PF_CAN: dropped non conform CAN XL skbuff: dev type %d, len %d\n", dev->type, skb->len); @@ -826,6 +833,7 @@ static int can_pernet_init(struct net *net) net->can.rx_alldev_list = kzalloc_obj(*net->can.rx_alldev_list); if (!net->can.rx_alldev_list) goto out; + spin_lock_init(&net->can.rx_alldev_list->lock); net->can.pkg_stats = kzalloc_obj(*net->can.pkg_stats); if (!net->can.pkg_stats) goto out_free_rx_alldev_list; diff --git a/net/can/bcm.c b/net/can/bcm.c index 3d637a1e0ac1..9288b96724da 100644 --- a/net/can/bcm.c +++ b/net/can/bcm.c @@ -54,6 +54,7 @@ #include #include #include +#include #include #include #include @@ -963,7 +964,7 @@ static void bcm_remove_op(struct bcm_op *op) static void bcm_rx_unreg(struct net_device *dev, struct bcm_op *op) { if (op->rx_reg_dev == dev) { - can_rx_unregister(dev_net(dev), dev, op->can_id, + can_rx_unregister(sock_net(op->sk), dev, op->can_id, REGMASK(op->can_id), bcm_rx_handler, op); /* mark as removed subscription */ @@ -989,21 +990,10 @@ static int bcm_delete_rx_op(struct list_head *ops, struct bcm_msg_head *mh, /* disable automatic timer on frame reception */ op->flags |= RX_NO_AUTOTIMER; - /* - * Don't care if we're bound or not (due to netdev - * problems) can_rx_unregister() is always a save - * thing to do here. - */ if (op->ifindex) { - /* - * Only remove subscriptions that had not - * been removed due to NETDEV_UNREGISTER - * in bcm_notifier() - * - * op->rx_reg_dev is a tracked reference taken - * when the subscription was registered, so it - * stays valid here even if a concurrent - * NETDEV_UNREGISTER already unlisted the dev. + /* Only remove subscriptions that had not been + * removed in bcm_notify(). op->rx_reg_dev keeps + * the device's receive lists accessible. */ if (op->rx_reg_dev) bcm_rx_unreg(op->rx_reg_dev, op); @@ -1719,7 +1709,7 @@ static int bcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) goto out_release; } - if (dev->type != ARPHRD_CAN) { + if (!can_get_ml_priv(dev)) { dev_put(dev); ret = -ENODEV; goto out_release; @@ -1867,7 +1857,7 @@ static int bcm_notifier(struct notifier_block *nb, unsigned long msg, { struct net_device *dev = netdev_notifier_info_to_dev(ptr); - if (dev->type != ARPHRD_CAN) + if (!can_get_ml_priv(dev)) return NOTIFY_DONE; if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN) return NOTIFY_DONE; @@ -1935,6 +1925,7 @@ static int bcm_release(struct socket *sock) list_del(&bo->notifier); spin_unlock(&bcm_notifier_lock); + /* bcm_notify() can not access this socket anymore */ lock_sock(sk); #if IS_ENABLED(CONFIG_PROC_FS) @@ -1948,29 +1939,20 @@ static int bcm_release(struct socket *sock) bcm_remove_op(op); } - list_for_each_entry_safe(op, next, &bo->rx_ops, list) { - /* - * Don't care if we're bound or not (due to netdev problems) - * can_rx_unregister() is always a save thing to do here. - */ + list_for_each_entry(op, &bo->rx_ops, list) { if (op->ifindex) { - /* - * Only remove subscriptions that had not - * been removed due to NETDEV_UNREGISTER - * in bcm_notifier() - * - * op->rx_reg_dev is a tracked reference taken - * when the subscription was registered, so it - * stays valid here even if a concurrent - * NETDEV_UNREGISTER already unlisted the device. + /* Only remove subscriptions that had not been removed + * in bcm_notify(). op->rx_reg_dev keeps the device's + * receive lists accessible. */ if (op->rx_reg_dev) bcm_rx_unreg(op->rx_reg_dev, op); - } else + } else { can_rx_unregister(net, NULL, op->can_id, REGMASK(op->can_id), bcm_rx_handler, op); - + } + cond_resched(); } synchronize_rcu(); @@ -2024,7 +2006,7 @@ static int bcm_connect(struct socket *sock, struct sockaddr_unsized *uaddr, int ret = -ENODEV; goto fail; } - if (dev->type != ARPHRD_CAN) { + if (!can_get_ml_priv(dev)) { dev_put(dev); ret = -ENODEV; goto fail; diff --git a/net/can/gw.c b/net/can/gw.c index 5793cb2420ed..54bb5bd3242a 100644 --- a/net/can/gw.c +++ b/net/can/gw.c @@ -52,6 +52,7 @@ #include #include #include +#include #include #include #include @@ -612,7 +613,7 @@ static int cgw_notifier(struct notifier_block *nb, struct net_device *dev = netdev_notifier_info_to_dev(ptr); struct net *net = dev_net(dev); - if (dev->type != ARPHRD_CAN) + if (!can_get_ml_priv(dev)) return NOTIFY_DONE; if (msg == NETDEV_UNREGISTER) { @@ -1163,7 +1164,7 @@ static int cgw_create_job(struct sk_buff *skb, struct nlmsghdr *nlh, if (!gwj->src.dev) goto out; - if (gwj->src.dev->type != ARPHRD_CAN) + if (!can_get_ml_priv(gwj->src.dev)) goto out; gwj->dst.dev = __dev_get_by_index(net, gwj->ccgw.dst_idx); @@ -1171,7 +1172,7 @@ static int cgw_create_job(struct sk_buff *skb, struct nlmsghdr *nlh, if (!gwj->dst.dev) goto out; - if (gwj->dst.dev->type != ARPHRD_CAN) + if (!can_get_ml_priv(gwj->dst.dev)) goto out; /* is sending the skb back to the incoming interface intended? */ diff --git a/net/can/isotp.c b/net/can/isotp.c index f5dc9d04bd68..5e2de051fc5e 100644 --- a/net/can/isotp.c +++ b/net/can/isotp.c @@ -65,6 +65,7 @@ #include #include #include +#include #include #include #include @@ -755,6 +756,14 @@ static void isotp_rcv(struct sk_buff *skb, void *data) if (skb->len != so->ll.mtu) return; + /* check for correct CAN CC/FD frame content */ + if (so->ll.mtu == CAN_MTU) { + if (!can_is_can_skb(skb)) + return; + } else if (!can_is_canfd_skb(skb)) { + return; + } + cf = (struct canfd_frame *)skb->data; /* if enabled: check reception of my configured extended address */ @@ -1123,7 +1132,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) int off; int err; - if (!so->bound || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN) + if (!READ_ONCE(so->bound) || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN) return -EADDRNOTAVAIL; /* claim the socket under so->rx_lock: this serializes the claim @@ -1168,7 +1177,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) /* so->bound is only checked once above - a wakeup may have * unbound/rebound the socket meanwhile => recheck */ - if (!so->bound) { + if (!READ_ONCE(so->bound)) { err = -EADDRNOTAVAIL; goto err_out_drop; } @@ -1414,7 +1423,7 @@ static int isotp_recvmsg(struct socket *sock, struct msghdr *msg, size_t size, if (flags & ~(MSG_DONTWAIT | MSG_TRUNC | MSG_PEEK | MSG_CMSG_COMPAT)) return -EINVAL; - if (!so->bound) + if (!READ_ONCE(so->bound)) return -EADDRNOTAVAIL; skb = skb_recv_datagram(sk, flags, &ret); @@ -1488,11 +1497,11 @@ static int isotp_release(struct socket *sock) list_del(&so->notifier); spin_unlock(&isotp_notifier_lock); - rtnl_lock(); lock_sock(sk); /* remove current filters & unregister - * tracked reference so->dev is taken at bind() time with rtnl_lock + * the tracked reference so->dev keeps the device's receive lists + * accessible, even after a namespace change */ if (so->bound && so->dev) { if (isotp_register_rxid(so)) @@ -1507,11 +1516,9 @@ static int isotp_release(struct socket *sock) } so->ifindex = 0; - so->bound = 0; + WRITE_ONCE(so->bound, 0); so->dev = NULL; - rtnl_unlock(); - /* Always wait for a grace period before touching the timers below. * A concurrent NETDEV_UNREGISTER may have already unregistered our * filters and cleared so->bound in isotp_notify() without waiting @@ -1541,7 +1548,6 @@ static int isotp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int l struct sock *sk = sock->sk; struct isotp_sock *so = isotp_sk(sk); struct net *net = sock_net(sk); - int ifindex; struct net_device *dev; canid_t tx_id = addr->can_addr.tp.tx_id; canid_t rx_id = addr->can_addr.tp.rx_id; @@ -1579,7 +1585,7 @@ static int isotp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int l if (!addr->can_ifindex) return -ENODEV; - rtnl_lock(); + /* lock_sock() serializes the binding with isotp_notify() */ lock_sock(sk); if (so->bound) { @@ -1605,61 +1611,83 @@ static int isotp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int l goto out; } - dev = dev_get_by_index(net, addr->can_ifindex); - if (!dev) { - err = -ENODEV; - goto out; - } - if (dev->type != ARPHRD_CAN) { + /* Publish the device inside the RCU section of the lookup: When it + * is unlisted concurrently, either isotp_notify() sees so->dev after + * synchronize_net() or the lookup fails. Check the device before + * publishing it. + */ + rcu_read_lock(); + dev = dev_get_by_index_rcu(net, addr->can_ifindex); + if (!dev || !can_get_ml_priv(dev)) err = -ENODEV; - goto out_put_dev; - } - if (READ_ONCE(dev->mtu) < so->ll.mtu) { + else if (READ_ONCE(dev->mtu) < so->ll.mtu) err = -EINVAL; - goto out_put_dev; + if (!err) { + dev_hold(dev); + WRITE_ONCE(so->dev, dev); } - if (!(dev->flags & IFF_UP)) - notify_enetdown = 1; - - ifindex = dev->ifindex; + rcu_read_unlock(); + if (err) + goto out; - if (isotp_register_rxid(so)) - can_rx_register(net, dev, rx_id, SINGLE_MASK(rx_id), - isotp_rcv, sk, "isotp", sk); + /* switch to new settings */ + so->ifindex = dev->ifindex; + so->rxid = rx_id; + so->txid = tx_id; /* no consecutive frame echo skb in flight */ WRITE_ONCE(so->cfecho, 0); - /* register for echo skb's */ - can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id), - isotp_rcv_echo, sk, "isotpe", sk); + /* register for echo skb's first (no echo traffic at this point) */ + err = can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id), + isotp_rcv_echo, sk, "isotpe", sk); + if (err) + goto out_put_dev; - /* switch to new settings */ - so->ifindex = ifindex; - so->rxid = rx_id; - so->txid = tx_id; - so->bound = 1; + if (isotp_register_rxid(so)) { + err = can_rx_register(net, dev, rx_id, SINGLE_MASK(rx_id), + isotp_rcv, sk, "isotp", sk); + if (err) { + /* unregister echo skb subscription */ + can_rx_unregister(net, dev, tx_id, SINGLE_MASK(tx_id), + isotp_rcv_echo, sk); + goto out_put_dev; + } + } + + WRITE_ONCE(so->bound, 1); /* bind() ok -> hold a reference for so->dev so that isotp_release() * can safely reach the device later, even if a concurrent * NETDEV_UNREGISTER has already unlisted it by ifindex. */ - so->dev = dev; netdev_hold(so->dev, &so->dev_tracker, GFP_KERNEL); + /* Either isotp_notify() sees so->dev on NETDEV_DOWN or we see the + * cleared IFF_UP, pairs with isotp_notify() + */ + smp_mb(); + if (!(READ_ONCE(dev->flags) & IFF_UP)) + notify_enetdown = 1; + out_put_dev: - /* remove potential reference from dev_get_by_index() */ + /* revert so->ifindex and so->dev on failed bind() */ + if (err) { + so->ifindex = 0; + WRITE_ONCE(so->dev, NULL); + } + + /* remove reference from the device lookup */ dev_put(dev); out: - release_sock(sk); - rtnl_unlock(); - if (notify_enetdown) { sk->sk_err = ENETDOWN; if (!sock_flag(sk, SOCK_DEAD)) sk_error_report(sk); } + release_sock(sk); + return err; } @@ -1681,8 +1709,8 @@ static int isotp_getname(struct socket *sock, struct sockaddr *uaddr, int peer) return ISOTP_MIN_NAMELEN; } -static int isotp_setsockopt_locked(struct socket *sock, int level, int optname, - sockptr_t optval, unsigned int optlen) +static int isotp_setsockopt_locked(struct socket *sock, int optname, + const void *val) { struct sock *sk = sock->sk; struct isotp_sock *so = isotp_sk(sk); @@ -1693,11 +1721,7 @@ static int isotp_setsockopt_locked(struct socket *sock, int level, int optname, switch (optname) { case CAN_ISOTP_OPTS: - if (optlen != sizeof(struct can_isotp_options)) - return -EINVAL; - - if (copy_from_sockptr(&so->opt, optval, optlen)) - return -EFAULT; + memcpy(&so->opt, val, sizeof(so->opt)); /* no separate rx_ext_address is given => use ext_address */ if (!(so->opt.flags & CAN_ISOTP_RX_EXT_ADDR)) @@ -1722,55 +1746,37 @@ static int isotp_setsockopt_locked(struct socket *sock, int level, int optname, break; case CAN_ISOTP_RECV_FC: - if (optlen != sizeof(struct can_isotp_fc_options)) - return -EINVAL; - - if (copy_from_sockptr(&so->rxfc, optval, optlen)) - return -EFAULT; + memcpy(&so->rxfc, val, sizeof(so->rxfc)); break; case CAN_ISOTP_TX_STMIN: - if (optlen != sizeof(u32)) - return -EINVAL; - - if (copy_from_sockptr(&so->force_tx_stmin, optval, optlen)) - return -EFAULT; + so->force_tx_stmin = *(const u32 *)val; break; case CAN_ISOTP_RX_STMIN: - if (optlen != sizeof(u32)) - return -EINVAL; - - if (copy_from_sockptr(&so->force_rx_stmin, optval, optlen)) - return -EFAULT; + so->force_rx_stmin = *(const u32 *)val; break; - case CAN_ISOTP_LL_OPTS: - if (optlen == sizeof(struct can_isotp_ll_options)) { - struct can_isotp_ll_options ll; - - if (copy_from_sockptr(&ll, optval, optlen)) - return -EFAULT; + case CAN_ISOTP_LL_OPTS: { + const struct can_isotp_ll_options *ll = val; - /* check for correct ISO 11898-1 DLC data length */ - if (ll.tx_dl != padlen(ll.tx_dl)) - return -EINVAL; + /* check for correct ISO 11898-1 DLC data length */ + if (ll->tx_dl != padlen(ll->tx_dl)) + return -EINVAL; - if (ll.mtu != CAN_MTU && ll.mtu != CANFD_MTU) - return -EINVAL; + if (ll->mtu != CAN_MTU && ll->mtu != CANFD_MTU) + return -EINVAL; - if (ll.mtu == CAN_MTU && - (ll.tx_dl > CAN_MAX_DLEN || ll.tx_flags != 0)) - return -EINVAL; + if (ll->mtu == CAN_MTU && + (ll->tx_dl > CAN_MAX_DLEN || ll->tx_flags != 0)) + return -EINVAL; - memcpy(&so->ll, &ll, sizeof(ll)); + memcpy(&so->ll, ll, sizeof(*ll)); - /* set ll_dl for tx path to similar place as for rx */ - so->tx.ll_dl = ll.tx_dl; - } else { - return -EINVAL; - } + /* set ll_dl for tx path to similar place as for rx */ + so->tx.ll_dl = ll->tx_dl; break; + } default: ret = -ENOPROTOOPT; @@ -1784,13 +1790,49 @@ static int isotp_setsockopt(struct socket *sock, int level, int optname, { struct sock *sk = sock->sk; + union { + struct can_isotp_options opt; + struct can_isotp_fc_options rxfc; + struct can_isotp_ll_options ll; + u32 stmin; + } val; + unsigned int size; int ret; if (level != SOL_CAN_ISOTP) return -EINVAL; + /* copy the option values from user space before lock_sock() */ + switch (optname) { + case CAN_ISOTP_OPTS: + size = sizeof(val.opt); + break; + + case CAN_ISOTP_RECV_FC: + size = sizeof(val.rxfc); + break; + + case CAN_ISOTP_TX_STMIN: + case CAN_ISOTP_RX_STMIN: + size = sizeof(val.stmin); + break; + + case CAN_ISOTP_LL_OPTS: + size = sizeof(val.ll); + break; + + default: + return -ENOPROTOOPT; + } + + if (optlen != size) + return -EINVAL; + + if (copy_from_sockptr(&val, optval, size)) + return -EFAULT; + lock_sock(sk); - ret = isotp_setsockopt_locked(sock, level, optname, optval, optlen); + ret = isotp_setsockopt_locked(sock, optname, &val); release_sock(sk); return ret; } @@ -1855,12 +1897,21 @@ static void isotp_notify(struct isotp_sock *so, unsigned long msg, if (!net_eq(dev_net(dev), sock_net(sk))) return; - if (so->dev != dev) + if (msg == NETDEV_DOWN) + smp_mb(); /* pairs with isotp_bind(), see raw_notify() */ + + /* lockless fast path, see raw_notify() */ + if (READ_ONCE(so->dev) != dev) return; switch (msg) { case NETDEV_UNREGISTER: lock_sock(sk); + if (so->dev != dev) { + release_sock(sk); + break; + } + /* remove current filters & unregister */ if (so->bound) { if (isotp_register_rxid(so)) @@ -1875,16 +1926,19 @@ static void isotp_notify(struct isotp_sock *so, unsigned long msg, } so->ifindex = 0; - so->bound = 0; - so->dev = NULL; - release_sock(sk); + WRITE_ONCE(so->bound, 0); + WRITE_ONCE(so->dev, NULL); sk->sk_err = ENODEV; if (!sock_flag(sk, SOCK_DEAD)) sk_error_report(sk); + release_sock(sk); break; case NETDEV_DOWN: + if (!READ_ONCE(so->bound)) + break; + sk->sk_err = ENETDOWN; if (!sock_flag(sk, SOCK_DEAD)) sk_error_report(sk); @@ -1897,7 +1951,7 @@ static int isotp_notifier(struct notifier_block *nb, unsigned long msg, { struct net_device *dev = netdev_notifier_info_to_dev(ptr); - if (dev->type != ARPHRD_CAN) + if (!can_get_ml_priv(dev)) return NOTIFY_DONE; if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN) return NOTIFY_DONE; diff --git a/net/can/proc.c b/net/can/proc.c index de4d05ae3459..fb144d2adc35 100644 --- a/net/can/proc.c +++ b/net/can/proc.c @@ -76,16 +76,39 @@ static const char rx_list_name[][8] = { * af_can statistics stuff */ +static void can_reset_pkg_stats(struct can_pkg_stats *pkg_stats) +{ + atomic_long_set(&pkg_stats->rx_frames, 0); + atomic_long_set(&pkg_stats->tx_frames, 0); + atomic_long_set(&pkg_stats->matches, 0); + + pkg_stats->total_rx_rate = 0; + pkg_stats->total_tx_rate = 0; + pkg_stats->total_rx_match_ratio = 0; + + pkg_stats->current_rx_rate = 0; + pkg_stats->current_tx_rate = 0; + pkg_stats->current_rx_match_ratio = 0; + + pkg_stats->max_rx_rate = 0; + pkg_stats->max_tx_rate = 0; + pkg_stats->max_rx_match_ratio = 0; + + atomic_long_set(&pkg_stats->rx_frames_delta, 0); + atomic_long_set(&pkg_stats->tx_frames_delta, 0); + atomic_long_set(&pkg_stats->matches_delta, 0); +} + static void can_init_stats(struct net *net) { struct can_pkg_stats *pkg_stats = net->can.pkg_stats; struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats; /* - * This memset function is called from a timer context (when + * This stats reset is called from a timer context (when * can_stattimer is active which is the default) OR in a process * context (reading the proc_fs when can_stattimer is disabled). */ - memset(pkg_stats, 0, sizeof(struct can_pkg_stats)); + can_reset_pkg_stats(pkg_stats); pkg_stats->jiffies_init = jiffies; rcv_lists_stats->stats_reset++; @@ -121,29 +144,23 @@ void can_stat_update(struct timer_list *t) long rx_frames = atomic_long_read(&pkg_stats->rx_frames); long tx_frames = atomic_long_read(&pkg_stats->tx_frames); long matches = atomic_long_read(&pkg_stats->matches); - long rx_frames_delta = atomic_long_read(&pkg_stats->rx_frames_delta); - long tx_frames_delta = atomic_long_read(&pkg_stats->tx_frames_delta); - long matches_delta = atomic_long_read(&pkg_stats->matches_delta); + /* fetch and clear values for 'current rate' calculation */ + long rx_frames_delta = atomic_long_xchg(&pkg_stats->rx_frames_delta, 0); + long tx_frames_delta = atomic_long_xchg(&pkg_stats->tx_frames_delta, 0); + long matches_delta = atomic_long_xchg(&pkg_stats->matches_delta, 0); - /* restart counting in timer context on user request */ - if (user_reset) - can_init_stats(net); - - /* restart counting on jiffies overflow */ - if (j < pkg_stats->jiffies_init) - can_init_stats(net); - - /* prevent overflow in calc_rate() */ - if (rx_frames > (LONG_MAX / HZ)) - can_init_stats(net); - - /* prevent overflow in calc_rate() */ - if (tx_frames > (LONG_MAX / HZ)) - can_init_stats(net); - - /* matches overflow - very improbable */ - if (matches > (LONG_MAX / 100)) + /* + * Restart counting in timer context on user request, on jiffies + * overflow or to prevent overflows in calc_rate() and in the + * match ratio calculation. The local snapshots are stale after + * the reset, so skip the calculation for this interval. + */ + if (user_reset || j < pkg_stats->jiffies_init || + rx_frames > (LONG_MAX / HZ) || tx_frames > (LONG_MAX / HZ) || + matches > (LONG_MAX / 100)) { can_init_stats(net); + goto out; + } /* calc total values */ if (rx_frames) @@ -172,11 +189,7 @@ void can_stat_update(struct timer_list *t) if (pkg_stats->max_rx_match_ratio < pkg_stats->current_rx_match_ratio) pkg_stats->max_rx_match_ratio = pkg_stats->current_rx_match_ratio; - /* clear values for 'current rate' calculation */ - atomic_long_set(&pkg_stats->tx_frames_delta, 0); - atomic_long_set(&pkg_stats->rx_frames_delta, 0); - atomic_long_set(&pkg_stats->matches_delta, 0); - +out: /* restart timer (one second) */ mod_timer(&net->can.stattimer, round_jiffies(jiffies + HZ)); } diff --git a/net/can/raw.c b/net/can/raw.c index 0acd4f6c6dd6..0926d5686e41 100644 --- a/net/can/raw.c +++ b/net/can/raw.c @@ -85,6 +85,7 @@ struct raw_sock { struct sock sk; struct net_device *dev; netdevice_tracker dev_tracker; + struct net_device *bind_dev; /* hint for raw_notify(), no reference */ struct list_head notifier; int ifindex; unsigned int bound:1; @@ -224,6 +225,7 @@ static int raw_enable_filters(struct net *net, struct net_device *dev, int i; for (i = 0; i < count; i++) { + cond_resched(); err = can_rx_register(net, dev, filter[i].can_id, filter[i].can_mask, raw_rcv, sk, "raw", sk); @@ -258,9 +260,11 @@ static void raw_disable_filters(struct net *net, struct net_device *dev, { int i; - for (i = 0; i < count; i++) + for (i = 0; i < count; i++) { can_rx_unregister(net, dev, filter[i].can_id, filter[i].can_mask, raw_rcv, sk); + cond_resched(); + } } static inline void raw_disable_errfilter(struct net *net, @@ -305,16 +309,31 @@ static void raw_notify(struct raw_sock *ro, unsigned long msg, struct net_device *dev) { struct sock *sk = &ro->sk; + struct net_device *bind_dev; if (!net_eq(dev_net(dev), sock_net(sk))) return; - if (ro->dev != dev) + if (msg == NETDEV_DOWN) + smp_mb(); /* IFF_UP is cleared before, pairs with raw_bind() */ + + /* Lockless fast path, ro->bind_dev covers a concurrent bind(). + * Read it before ro->dev, pairs with smp_store_release() in + * raw_bind(). A stale ro->bind_dev only costs a lock_sock(). + */ + bind_dev = smp_load_acquire(&ro->bind_dev); + if (READ_ONCE(ro->dev) != dev && + (msg != NETDEV_UNREGISTER || bind_dev != dev)) return; switch (msg) { case NETDEV_UNREGISTER: lock_sock(sk); + if (ro->dev != dev) { + release_sock(sk); + break; + } + /* remove current filters & unregister */ if (ro->bound) { raw_disable_allfilters(dev_net(dev), dev, sk); @@ -326,13 +345,13 @@ static void raw_notify(struct raw_sock *ro, unsigned long msg, ro->ifindex = 0; ro->bound = 0; - ro->dev = NULL; + WRITE_ONCE(ro->dev, NULL); ro->count = 0; - release_sock(sk); sk->sk_err = ENODEV; if (!sock_flag(sk, SOCK_DEAD)) sk_error_report(sk); + release_sock(sk); break; case NETDEV_DOWN: @@ -348,7 +367,7 @@ static int raw_notifier(struct notifier_block *nb, unsigned long msg, { struct net_device *dev = netdev_notifier_info_to_dev(ptr); - if (dev->type != ARPHRD_CAN) + if (!can_get_ml_priv(dev)) return NOTIFY_DONE; if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN) return NOTIFY_DONE; @@ -381,6 +400,7 @@ static int raw_init(struct sock *sk) ro->bound = 0; ro->ifindex = 0; ro->dev = NULL; + ro->bind_dev = NULL; /* set default filter to single entry dfilter */ ro->dfilter.can_id = 0; @@ -431,13 +451,15 @@ static int raw_release(struct socket *sock) list_del(&ro->notifier); spin_unlock(&raw_notifier_lock); - rtnl_lock(); lock_sock(sk); - /* remove current filters & unregister */ + /* remove current filters & unregister + * the tracked reference ro->dev keeps the device's receive lists + * accessible, even after a namespace change + */ if (ro->bound) { if (ro->dev) { - raw_disable_allfilters(dev_net(ro->dev), ro->dev, sk); + raw_disable_allfilters(net, ro->dev, sk); netdev_put(ro->dev, &ro->dev_tracker); } else { raw_disable_allfilters(net, NULL, sk); @@ -456,7 +478,6 @@ static int raw_release(struct socket *sock) sock->sk = NULL; release_sock(sk); - rtnl_unlock(); sock_prot_inuse_add(net, sk->sk_prot, -1); sock_put(sk); @@ -479,26 +500,34 @@ static int raw_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int len if (addr->can_family != AF_CAN) return -EINVAL; - rtnl_lock(); + /* lock_sock() serializes the binding with raw_notify() */ lock_sock(sk); if (ro->bound && addr->can_ifindex == ro->ifindex) goto out; if (addr->can_ifindex) { - dev = dev_get_by_index(sock_net(sk), addr->can_ifindex); + /* Publish the device inside the RCU section of the lookup: + * When it is unlisted concurrently, either raw_notify() sees + * ro->bind_dev after synchronize_net() or the lookup fails. + */ + rcu_read_lock(); + dev = dev_get_by_index_rcu(sock_net(sk), addr->can_ifindex); + if (dev) { + dev_hold(dev); + /* pairs with smp_load_acquire() in raw_notify() */ + smp_store_release(&ro->bind_dev, dev); + } + rcu_read_unlock(); if (!dev) { err = -ENODEV; goto out; } - if (dev->type != ARPHRD_CAN) { + if (!can_get_ml_priv(dev)) { err = -ENODEV; goto out_put_dev; } - if (!(dev->flags & IFF_UP)) - notify_enetdown = 1; - ifindex = dev->ifindex; /* filters set by default/setsockopt */ @@ -517,7 +546,7 @@ static int raw_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int len if (ro->bound) { /* unregister old filters */ if (ro->dev) { - raw_disable_allfilters(dev_net(ro->dev), + raw_disable_allfilters(sock_net(sk), ro->dev, sk); /* drop reference to old ro->dev */ netdev_put(ro->dev, &ro->dev_tracker); @@ -528,24 +557,32 @@ static int raw_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int len ro->ifindex = ifindex; ro->bound = 1; /* bind() ok -> hold a reference for new ro->dev */ - ro->dev = dev; + WRITE_ONCE(ro->dev, dev); if (ro->dev) netdev_hold(ro->dev, &ro->dev_tracker, GFP_KERNEL); + + if (dev) { + /* Either raw_notify() sees ro->dev on NETDEV_DOWN or + * we see the cleared IFF_UP, pairs with raw_notify() + */ + smp_mb(); + if (!(READ_ONCE(dev->flags) & IFF_UP)) + notify_enetdown = 1; + } } out_put_dev: - /* remove potential reference from dev_get_by_index() */ + /* remove reference from the device lookup */ dev_put(dev); out: - release_sock(sk); - rtnl_unlock(); - if (notify_enetdown) { sk->sk_err = ENETDOWN; if (!sock_flag(sk, SOCK_DEAD)) sk_error_report(sk); } + release_sock(sk); + return err; } @@ -567,58 +604,36 @@ static int raw_getname(struct socket *sock, struct sockaddr *uaddr, } static int raw_setsockopt_locked(struct socket *sock, int optname, - sockptr_t optval, unsigned int optlen) + const void *val, struct can_filter *filter, + int count) { struct sock *sk = sock->sk; struct raw_sock *ro = raw_sk(sk); - struct can_filter *filter = NULL; /* dyn. alloc'ed filters */ - struct can_filter sfilter; /* single filter */ struct net_device *dev = NULL; - can_err_mask_t err_mask = 0; - int count = 0; + can_err_mask_t err_mask; int flag; int err = 0; switch (optname) { case CAN_RAW_FILTER: - if (optlen % sizeof(struct can_filter) != 0) - return -EINVAL; - - if (optlen > CAN_RAW_FILTER_MAX * sizeof(struct can_filter)) - return -EINVAL; - - count = optlen / sizeof(struct can_filter); - - if (count > 1) { - /* filter does not fit into dfilter => alloc space */ - filter = memdup_sockptr(optval, optlen); - if (IS_ERR(filter)) - return PTR_ERR(filter); - } else if (count == 1) { - if (copy_from_sockptr(&sfilter, optval, sizeof(sfilter))) - return -EFAULT; - } - + /* @filter has been copied by raw_setsockopt(). It is linked + * to the socket on success and freed by the caller on error. + * + * The reg_state check is only a shortcut: On a concurrent + * unregistration or namespace change raw_notify() waits for + * lock_sock() and removes the filters set here. + */ dev = ro->dev; - if (ro->bound && dev && dev->reg_state != NETREG_REGISTERED) { - if (count > 1) - kfree(filter); + if (ro->bound && dev && + READ_ONCE(dev->reg_state) != NETREG_REGISTERED) return -ENODEV; - } if (ro->bound) { /* (try to) register the new filters */ - if (count == 1) - err = raw_enable_filters(sock_net(sk), dev, sk, - &sfilter, 1); - else - err = raw_enable_filters(sock_net(sk), dev, sk, - filter, count); - if (err) { - if (count > 1) - kfree(filter); + err = raw_enable_filters(sock_net(sk), dev, sk, filter, + count); + if (err) return err; - } /* remove old filter registrations */ raw_disable_filters(sock_net(sk), dev, sk, ro->filter, @@ -632,7 +647,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname, /* link new filters to the socket */ if (count == 1) { /* copy filter data for single filter */ - ro->dfilter = sfilter; + ro->dfilter = *filter; filter = &ro->dfilter; } ro->filter = filter; @@ -640,16 +655,12 @@ static int raw_setsockopt_locked(struct socket *sock, int optname, break; case CAN_RAW_ERR_FILTER: - if (optlen != sizeof(err_mask)) - return -EINVAL; - - if (copy_from_sockptr(&err_mask, optval, optlen)) - return -EFAULT; - - err_mask &= CAN_ERR_MASK; + err_mask = *(const can_err_mask_t *)val & CAN_ERR_MASK; + /* lockless shortcut, see CAN_RAW_FILTER */ dev = ro->dev; - if (ro->bound && dev && dev->reg_state != NETREG_REGISTERED) + if (ro->bound && dev && + READ_ONCE(dev->reg_state) != NETREG_REGISTERED) return -ENODEV; /* remove current error mask */ @@ -671,31 +682,15 @@ static int raw_setsockopt_locked(struct socket *sock, int optname, break; case CAN_RAW_LOOPBACK: - if (optlen != sizeof(flag)) - return -EINVAL; - - if (copy_from_sockptr(&flag, optval, optlen)) - return -EFAULT; - - ro->loopback = !!flag; + ro->loopback = !!*(const int *)val; break; case CAN_RAW_RECV_OWN_MSGS: - if (optlen != sizeof(flag)) - return -EINVAL; - - if (copy_from_sockptr(&flag, optval, optlen)) - return -EFAULT; - - ro->recv_own_msgs = !!flag; + ro->recv_own_msgs = !!*(const int *)val; break; case CAN_RAW_FD_FRAMES: - if (optlen != sizeof(flag)) - return -EINVAL; - - if (copy_from_sockptr(&flag, optval, optlen)) - return -EFAULT; + flag = *(const int *)val; /* Enabling CAN XL includes CAN FD */ if (ro->xl_frames && !flag) @@ -705,13 +700,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname, break; case CAN_RAW_XL_FRAMES: - if (optlen != sizeof(flag)) - return -EINVAL; - - if (copy_from_sockptr(&flag, optval, optlen)) - return -EFAULT; - - ro->xl_frames = !!flag; + ro->xl_frames = !!*(const int *)val; /* Enabling CAN XL includes CAN FD */ if (ro->xl_frames) @@ -719,11 +708,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname, break; case CAN_RAW_XL_VCID_OPTS: - if (optlen != sizeof(ro->raw_vcid_opts)) - return -EINVAL; - - if (copy_from_sockptr(&ro->raw_vcid_opts, optval, optlen)) - return -EFAULT; + memcpy(&ro->raw_vcid_opts, val, sizeof(ro->raw_vcid_opts)); /* prepare 32 bit values for handling in hot path */ ro->tx_vcid_shifted = ro->raw_vcid_opts.tx_vcid << CANXL_VCID_OFFSET; @@ -732,13 +717,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname, break; case CAN_RAW_JOIN_FILTERS: - if (optlen != sizeof(flag)) - return -EINVAL; - - if (copy_from_sockptr(&flag, optval, optlen)) - return -EFAULT; - - ro->join_filters = !!flag; + ro->join_filters = !!*(const int *)val; break; default: @@ -751,18 +730,83 @@ static int raw_setsockopt(struct socket *sock, int level, int optname, sockptr_t optval, unsigned int optlen) { struct sock *sk = sock->sk; + struct can_filter *filter = NULL; /* dyn. alloc'ed filters */ + struct can_filter sfilter; /* single filter */ + union { + can_err_mask_t err_mask; + int flag; + struct can_raw_vcid_options vcid_opts; + } val; + unsigned int size = 0; + int count = 0; int err; if (level != SOL_CAN_RAW) return -EINVAL; - rtnl_lock(); + /* Copy the option values from user space before lock_sock(), so a + * stalled copy can not block raw_notify() waiting for lock_sock() + * under RTNL. + */ + switch (optname) { + case CAN_RAW_FILTER: + if (optlen % sizeof(struct can_filter) != 0) + return -EINVAL; + + if (optlen > CAN_RAW_FILTER_MAX * sizeof(struct can_filter)) + return -EINVAL; + + count = optlen / sizeof(struct can_filter); + + if (count > 1) { + /* filter does not fit into dfilter => alloc space */ + filter = memdup_sockptr(optval, optlen); + if (IS_ERR(filter)) + return PTR_ERR(filter); + } else if (count == 1) { + if (copy_from_sockptr(&sfilter, optval, sizeof(sfilter))) + return -EFAULT; + filter = &sfilter; + } + break; + + case CAN_RAW_ERR_FILTER: + size = sizeof(val.err_mask); + break; + + case CAN_RAW_XL_VCID_OPTS: + size = sizeof(val.vcid_opts); + break; + + case CAN_RAW_LOOPBACK: + case CAN_RAW_RECV_OWN_MSGS: + case CAN_RAW_FD_FRAMES: + case CAN_RAW_XL_FRAMES: + case CAN_RAW_JOIN_FILTERS: + size = sizeof(val.flag); + break; + + default: + return -ENOPROTOOPT; + } + + if (size) { + if (optlen != size) + return -EINVAL; + + if (copy_from_sockptr(&val, optval, size)) + return -EFAULT; + } + lock_sock(sk); - err = raw_setsockopt_locked(sock, optname, optval, optlen); + err = raw_setsockopt_locked(sock, optname, &val, filter, count); release_sock(sk); - rtnl_unlock(); + + /* on error free the filters allocated for CAN_RAW_FILTER above */ + if (err && count > 1) + kfree(filter); return err; } @@ -784,6 +828,7 @@ static int raw_getsockopt(struct socket *sock, int level, int optname, switch (optname) { case CAN_RAW_FILTER: { + struct can_filter *filter = NULL; int err = 0; lock_sock(sk); @@ -798,15 +843,22 @@ static int raw_getsockopt(struct socket *sock, int level, int optname, } else { if (len > fsize) len = fsize; - if (copy_to_iter(ro->filter, len, - &opt->iter_out) != len) - err = -EFAULT; + /* copy to user space after release_sock() */ + filter = kmemdup(ro->filter, len, GFP_KERNEL); + if (!filter) + err = -ENOMEM; } } else { len = 0; } release_sock(sk); + if (filter) { + if (copy_to_iter(filter, len, &opt->iter_out) != len) + err = -EFAULT; + kfree(filter); + } + if (!err) opt->optlen = len; return err; diff --git a/net/core/dev.c b/net/core/dev.c index cc7b32ab5608..f803d8ba866f 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4204,7 +4204,9 @@ static enum skb_drop_reason qdisc_pkt_len_segs_init(struct sk_buff *skb) if (likely(shinfo->gso_type & (SKB_GSO_TCPV4 | SKB_GSO_TCPV6))) { const struct tcphdr *th; - if (!pskb_may_pull(skb, hdr_len + sizeof(struct tcphdr))) + /* A header in front of skb->data gives a negative offset. */ + if ((int)hdr_len < 0 || + !pskb_may_pull(skb, hdr_len + sizeof(struct tcphdr))) return SKB_DROP_REASON_SKB_BAD_GSO; th = (const struct tcphdr *)(skb->data + hdr_len); diff --git a/net/core/filter.c b/net/core/filter.c index 70dc621672f2..d3e5a2843d5e 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -2271,9 +2271,13 @@ static int bpf_out_neigh_v6(struct net *net, struct sk_buff *skb, static int __bpf_redirect_neigh_v6(struct sk_buff *skb, struct net_device *dev, struct bpf_nh_params *nh) { - const struct ipv6hdr *ip6h = ipv6_hdr(skb); struct net *net = dev_net(dev); int err, ret = NET_XMIT_DROP; + const struct ipv6hdr *ip6h; + + if (unlikely(!pskb_may_pull(skb, sizeof(*ip6h)))) + goto out_drop; + ip6h = ipv6_hdr(skb); if (!nh) { struct dst_entry *dst; @@ -2380,9 +2384,13 @@ static int bpf_out_neigh_v4(struct net *net, struct sk_buff *skb, static int __bpf_redirect_neigh_v4(struct sk_buff *skb, struct net_device *dev, struct bpf_nh_params *nh) { - const struct iphdr *ip4h = ip_hdr(skb); struct net *net = dev_net(dev); int err, ret = NET_XMIT_DROP; + const struct iphdr *ip4h; + + if (unlikely(!pskb_may_pull(skb, sizeof(*ip4h)))) + goto out_drop; + ip4h = ip_hdr(skb); if (!nh) { struct flowi4 fl4 = { @@ -2432,15 +2440,14 @@ static int __bpf_redirect_neigh_v4(struct sk_buff *skb, struct net_device *dev, static int __bpf_redirect_neigh(struct sk_buff *skb, struct net_device *dev, struct bpf_nh_params *nh) { - struct ethhdr *ethh = eth_hdr(skb); - - if (unlikely(skb->mac_header >= skb->network_header)) + if (unlikely(skb_network_offset(skb) < ETH_HLEN || + !pskb_may_pull(skb, ETH_HLEN))) goto out; bpf_push_mac_rcsum(skb); - if (is_multicast_ether_addr(ethh->h_dest)) + if (is_multicast_ether_addr(eth_hdr(skb)->h_dest)) goto out; - skb_pull(skb, sizeof(*ethh)); + skb_pull(skb, ETH_HLEN); skb_unset_mac_header(skb); skb_reset_network_header(skb); @@ -7989,7 +7996,7 @@ BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags) if (sk_unhashed(sk)) return -EOPNOTSUPP; if (sk_is_refcounted(sk) && - unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt))) return -ENOENT; skb_orphan(skb); diff --git a/net/core/sock.c b/net/core/sock.c index e8551df8330f..1108543f9fec 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -3262,7 +3262,7 @@ void __release_sock(struct sock *sk) spin_unlock_bh(&sk->sk_lock.slock); - while (1) { + do { next = skb->next; prefetch(next); DEBUG_NET_WARN_ON_ONCE(skb_dst_is_noref(skb)); @@ -3270,12 +3270,10 @@ void __release_sock(struct sock *sk) sk_backlog_rcv(sk, skb); skb = next; - if (!skb) - break; if (!(++nb & 15)) cond_resched(); - } + } while (skb); spin_lock_bh(&sk->sk_lock.slock); } diff --git a/net/ipv4/cipso_ipv4.c b/net/ipv4/cipso_ipv4.c index 1aacbbeffbc6..e65a3f29d2ca 100644 --- a/net/ipv4/cipso_ipv4.c +++ b/net/ipv4/cipso_ipv4.c @@ -2013,6 +2013,10 @@ static int cipso_v4_delopt(struct ip_options_rcu __rcu **opt_ptr) cipso_ptr = &opt->opt.__data[cipso_off]; cipso_len = cipso_ptr[1]; + optlen_new = cipso_v4_get_actual_opt_len(opt->opt.__data, + opt->opt.optlen); + optlen_new -= cipso_len; + if (opt->opt.srr > opt->opt.cipso) opt->opt.srr -= cipso_len; if (opt->opt.rr > opt->opt.cipso) @@ -2026,11 +2030,12 @@ static int cipso_v4_delopt(struct ip_options_rcu __rcu **opt_ptr) memmove(cipso_ptr, cipso_ptr + cipso_len, opt->opt.optlen - cipso_off - cipso_len); - optlen_new = cipso_v4_get_actual_opt_len(opt->opt.__data, - opt->opt.optlen); hdr_delta = opt->opt.optlen; opt->opt.optlen = (optlen_new + 3) & ~3; hdr_delta -= opt->opt.optlen; + + memset(&opt->opt.__data[optlen_new], IPOPT_END, + opt->opt.optlen - optlen_new); } else { /* only the cipso option was present on the socket so we can * remove the entire option struct */ diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c index 8e891c8141f8..a1fe2531cb59 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -1060,8 +1060,8 @@ static void reqsk_timer_handler(struct timer_list *t) * hold another count to prevent use-after-free and * call reqsk_put() just before return. */ - refcount_set(&nreq->rsk_refcnt, 2 + 1); timer_setup(&nreq->rsk_timer, reqsk_timer_handler, TIMER_PINNED); + refcount_set_release(&nreq->rsk_refcnt, 2 + 1); reqsk_queue_migrated(&inet_csk(nsk)->icsk_accept_queue, req); req = nreq; @@ -1445,7 +1445,7 @@ struct sock *inet_csk_complete_hashdance(struct sock *sk, struct sock *child, goto child_put; } - refcount_set(&nreq->rsk_refcnt, 1); + refcount_set_release(&nreq->rsk_refcnt, 1); if (inet_csk_reqsk_queue_add(sk, nreq, child)) { __NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMIGRATEREQSUCCESS); reqsk_migrate_reset(req); @@ -1498,7 +1498,7 @@ void inet_csk_listen_stop(struct sock *sk) if (nsk) { nreq = inet_reqsk_clone(req, nsk); if (nreq) { - refcount_set(&nreq->rsk_refcnt, 1); + refcount_set_release(&nreq->rsk_refcnt, 1); rcu_read_lock(); if (inet_csk_reqsk_queue_add(nsk, nreq, child)) { diff --git a/net/ipv4/inet_fragment.c b/net/ipv4/inet_fragment.c index c18da91c1f03..0b8d22ffafca 100644 --- a/net/ipv4/inet_fragment.c +++ b/net/ipv4/inet_fragment.c @@ -336,6 +336,7 @@ void inet_frag_queue_flush(struct inet_frag_queue *q, q->rb_fragments = RB_ROOT; q->fragments_tail = NULL; q->last_run_head = NULL; + q->truesize = 0; } EXPORT_SYMBOL(inet_frag_queue_flush); @@ -446,6 +447,9 @@ int inet_frag_queue_insert(struct inet_frag_queue *q, struct sk_buff *skb, if (last && !secpath_equal(last, skb)) return IPFRAG_MIXED_SECPATH; + if (skb->truesize > INET_FRAG_MAX_TRUESIZE - q->truesize) + return IPFRAG_TOOBIG; + /* An IP fragment is never a GSO packet, but an untrusted source * (virtio_net_hdr) may have attached GSO metadata to it. Do not let * that reach the reassembled skb, whose head keeps the first @@ -510,6 +514,7 @@ int inet_frag_queue_insert(struct inet_frag_queue *q, struct sk_buff *skb, if (offset) nf_reset_ct(skb); + q->truesize += skb->truesize; return IPFRAG_OK; } EXPORT_SYMBOL(inet_frag_queue_insert); diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c index 58d54965036a..7ffa9171d6de 100644 --- a/net/ipv4/inet_hashtables.c +++ b/net/ipv4/inet_hashtables.c @@ -550,7 +550,7 @@ struct sock *__inet_lookup_established(const struct net *net, if (sk->sk_hash != hash) continue; if (likely(inet_match(net, sk, acookie, ports, dif, sdif))) { - if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt))) goto out; if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) || !inet_match(net, sk, acookie, ports, dif, diff --git a/net/ipv4/inet_timewait_sock.c b/net/ipv4/inet_timewait_sock.c index d4c781a0667f..74ff40d42e56 100644 --- a/net/ipv4/inet_timewait_sock.c +++ b/net/ipv4/inet_timewait_sock.c @@ -141,7 +141,7 @@ void inet_twsk_hashdance_schedule(struct inet_timewait_sock *tw, * Also note that after this point, we lost our implicit reference * so we are not allowed to use tw anymore. */ - refcount_set(&tw->tw_refcnt, 3); + refcount_set_release(&tw->tw_refcnt, 3); /* Ensure tw_refcnt has been set before tw is published. * smp_wmb() provides the necessary memory barrier to enforce this @@ -328,7 +328,7 @@ void inet_twsk_purge(struct inet_hashinfo *hashinfo) if (check_net(sock_net(sk))) continue; - if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt))) continue; if (check_net(sock_net(sk))) { diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c index 644030561537..4b2d2713653b 100644 --- a/net/ipv4/syncookies.c +++ b/net/ipv4/syncookies.c @@ -205,7 +205,7 @@ struct sock *tcp_get_cookie_sock(struct sock *sk, struct sk_buff *skb, child = icsk->icsk_af_ops->syn_recv_sock(sk, skb, req, dst, NULL, &own_req, NULL); if (child) { - refcount_set(&req->rsk_refcnt, 1); + refcount_set_release(&req->rsk_refcnt, 1); sock_rps_save_rxhash(child, skb); if (rsk_drop_req(req)) { diff --git a/net/ipv4/tcp_fastopen.c b/net/ipv4/tcp_fastopen.c index 471c78be5513..e40477b6b554 100644 --- a/net/ipv4/tcp_fastopen.c +++ b/net/ipv4/tcp_fastopen.c @@ -364,7 +364,7 @@ static struct sock *tcp_fastopen_create_child(struct sock *sk, tcp_reset_xmit_timer(child, ICSK_TIME_RETRANS, req->timeout, false); - refcount_set(&req->rsk_refcnt, 2); + refcount_set_release(&req->rsk_refcnt, 2); sk_mark_napi_id_set(child, skb); diff --git a/net/ipv4/xfrm4_output.c b/net/ipv4/xfrm4_output.c index 0ae67d537499..fffa3c4f456e 100644 --- a/net/ipv4/xfrm4_output.c +++ b/net/ipv4/xfrm4_output.c @@ -38,9 +38,12 @@ int xfrm4_output(struct net *net, struct sock *sk, struct sk_buff *skb) void xfrm4_local_error(struct sk_buff *skb, u32 mtu) { + struct sock *sk = skb_to_full_sk(skb); struct iphdr *hdr; + if (!sk) + return; + hdr = skb->encapsulation ? inner_ip_hdr(skb) : ip_hdr(skb); - ip_local_error(skb->sk, EMSGSIZE, hdr->daddr, - inet_sk(skb->sk)->inet_dport, mtu); + ip_local_error(sk, EMSGSIZE, hdr->daddr, inet_sk(sk)->inet_dport, mtu); } diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c index a2d3fb187f76..3350330c3f1c 100644 --- a/net/ipv6/addrconf.c +++ b/net/ipv6/addrconf.c @@ -1484,7 +1484,8 @@ static int ipv6_create_tempaddr(struct inet6_ifaddr *ifp, bool block) age = (now - tmp_tstamp + ADDRCONF_TIMER_FUZZ_MINUS) / HZ; if (cfg.preferred_lft <= regen_advance + age) { cfg.preferred_lft = regen_advance + age + 1; - if (cfg.preferred_lft > cfg.valid_lft || + if (cfg.preferred_lft <= regen_advance + age || + cfg.preferred_lft > cfg.valid_lft || cfg.preferred_lft > if_public_preferred_lft) { in6_ifa_put(ifp); in6_dev_put(idev); diff --git a/net/ipv6/datagram.c b/net/ipv6/datagram.c index 38d7b4845281..9672ac979f47 100644 --- a/net/ipv6/datagram.c +++ b/net/ipv6/datagram.c @@ -149,6 +149,7 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr_unsized *uaddr, __be32 fl6_flowlabel = 0; __be32 old_fl6_flowlabel; __be16 old_dport; + int old_bound_dev_if; int addr_type; int err; @@ -219,6 +220,8 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr_unsized *uaddr, goto out; } + old_bound_dev_if = sk->sk_bound_dev_if; + if (__ipv6_addr_needs_scope_id(addr_type)) { if (addr_len >= sizeof(struct sockaddr_in6) && usin->sin6_scope_id) { @@ -261,6 +264,7 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr_unsized *uaddr, sk->sk_v6_daddr = old_daddr; np->flow_label = old_fl6_flowlabel; inet->inet_dport = old_dport; + WRITE_ONCE(sk->sk_bound_dev_if, old_bound_dev_if); goto out; } diff --git a/net/ipv6/inet6_hashtables.c b/net/ipv6/inet6_hashtables.c index aa5fa1dd062f..1eea584d80df 100644 --- a/net/ipv6/inet6_hashtables.c +++ b/net/ipv6/inet6_hashtables.c @@ -105,7 +105,7 @@ struct sock *__inet6_lookup_established(const struct net *net, continue; if (!inet6_match(net, sk, saddr, daddr, ports, dif, sdif)) continue; - if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt))) goto out; if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) || diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c index b965258cf9e5..bdea825434ea 100644 --- a/net/ipv6/raw.c +++ b/net/ipv6/raw.c @@ -654,6 +654,9 @@ static int rawv6_send_hdrinc(struct sock *sk, struct msghdr *msg, int length, if (unlikely(!skb)) return 0; + /* l3mdev_ip6_out() may have replaced the skb dst. */ + rt = dst_rt6_info(skb_dst(skb)); + /* Acquire rcu_read_lock() in case we need to use rt->rt6i_idev * in the error path. Since skb has been freed, the dst could * have been queued for deletion. diff --git a/net/l2tp/l2tp_core.c b/net/l2tp/l2tp_core.c index 7e042a3e648c..0cd3e38bf662 100644 --- a/net/l2tp/l2tp_core.c +++ b/net/l2tp/l2tp_core.c @@ -315,10 +315,19 @@ EXPORT_SYMBOL_GPL(l2tp_v2_session_get); struct l2tp_session *l2tp_session_get(const struct net *net, struct sock *sk, int pver, u32 tunnel_id, u32 session_id) { + struct l2tp_session *session; + if (pver == L2TP_HDR_VER_2) return l2tp_v2_session_get(net, tunnel_id, session_id); - else - return l2tp_v3_session_get(net, sk, session_id); + + /* l2tp_v3_session_get() doesn't check sk if the ID hasn't collided */ + session = l2tp_v3_session_get(net, sk, session_id); + if (session && session->tunnel->sock != sk) { + l2tp_session_put(session); + return NULL; + } + + return session; } EXPORT_SYMBOL_GPL(l2tp_session_get); diff --git a/net/mac80211/tdls.c b/net/mac80211/tdls.c index f663d28d9209..71db07aa301b 100644 --- a/net/mac80211/tdls.c +++ b/net/mac80211/tdls.c @@ -2021,7 +2021,7 @@ void ieee80211_process_tdls_channel_switch(struct ieee80211_sub_if_data *sdata, struct sk_buff *skb) { - struct ieee80211_tdls_data *tf = (void *)skb->data; + struct ieee80211_tdls_data *tf; struct wiphy *wiphy = sdata->local->hw.wiphy; lockdep_assert_wiphy(wiphy); @@ -2037,6 +2037,7 @@ ieee80211_process_tdls_channel_switch(struct ieee80211_sub_if_data *sdata, * The packet/size was already validated by mac80211 Rx path, only look * at the action type. */ + tf = (void *)skb->data; switch (tf->action_code) { case WLAN_TDLS_CHANNEL_SWITCH_REQUEST: ieee80211_process_tdls_channel_switch_req(sdata, skb); diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c index 5714634c8103..27bae2a4fa86 100644 --- a/net/mac80211/tx.c +++ b/net/mac80211/tx.c @@ -1130,6 +1130,7 @@ static bool ieee80211_tx_prep_agg(struct ieee80211_tx_data *tx, struct sk_buff *purge_skb = NULL; if (test_bit(HT_AGG_STATE_OPERATIONAL, &tid_tx->state)) { + info->flags |= IEEE80211_TX_CTL_AMPDU; reset_agg_timer = true; } else if (test_bit(HT_AGG_STATE_WANT_START, &tid_tx->state)) { /* @@ -1161,6 +1162,7 @@ static bool ieee80211_tx_prep_agg(struct ieee80211_tx_data *tx, if (!tid_tx) { /* do nothing, let packet pass through */ } else if (test_bit(HT_AGG_STATE_OPERATIONAL, &tid_tx->state)) { + info->flags |= IEEE80211_TX_CTL_AMPDU; reset_agg_timer = true; } else { queued = true; diff --git a/net/mptcp/pm_kernel.c b/net/mptcp/pm_kernel.c index 1a7750813235..f7396628490e 100644 --- a/net/mptcp/pm_kernel.c +++ b/net/mptcp/pm_kernel.c @@ -906,6 +906,7 @@ static int mptcp_pm_nl_create_listen_socket(struct sock *sk, * old status is known (TCP_CLOSE). */ inet_sk_state_store(newsk, TCP_LISTEN); + sock_set_flag(newsk, SOCK_RCU_FREE); lock_sock(ssk); WRITE_ONCE(mptcp_subflow_ctx(ssk)->pm_listener, true); err = __inet_listen_sk(ssk, backlog); diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c index 2c72cb50fbf3..0b67b46ef872 100644 --- a/net/mptcp/subflow.c +++ b/net/mptcp/subflow.c @@ -379,6 +379,10 @@ static void subflow_prep_synack(const struct sock *sk, struct request_sock *req, struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk); struct inet_request_sock *ireq = inet_rsk(req); + /* MP_JOIN cannot use TFO, do not send a cookie in the SYN/ACK */ + if (foc && mptcp_subflow_rsk(req)->mp_join) + foc->len = -1; + /* clear tstamp_ok, as needed depending on cookie */ if (foc && foc->len > -1) ireq->tstamp_ok = 0; @@ -887,6 +891,13 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk, fallback = true; } else if (subflow_req->mp_join) { + /* a SYN skb here comes from TFO, which MP_JOIN cannot use: just + * fall back to the regular path. + */ + if ((TCP_SKB_CB(skb)->tcp_flags & (TCPHDR_SYN | TCPHDR_ACK)) == + TCPHDR_SYN) + return NULL; + mptcp_get_options(skb, &mp_opt); if (!(mp_opt.suboptions & OPTION_MPTCP_MPJ_ACK)) fallback = true; diff --git a/net/qrtr/af_qrtr.c b/net/qrtr/af_qrtr.c index e7b3647424b8..45cb8e8c36ca 100644 --- a/net/qrtr/af_qrtr.c +++ b/net/qrtr/af_qrtr.c @@ -122,6 +122,7 @@ static DEFINE_XARRAY_ALLOC(qrtr_ports); * @qrtr_tx_flow: xarray of qrtr_tx_flow, keyed by node << 32 | port * @qrtr_tx_lock: lock for qrtr_tx_flow inserts * @hello_sent: hello packet send successful + * @hello_rcvd: remote has sent a hello packet * @rx_queue: receive queue * @item: list item for broadcast list * @say_hello: delayed work for sending hello packet @@ -136,6 +137,7 @@ struct qrtr_node { struct mutex qrtr_tx_lock; /* for qrtr_tx_flow */ bool hello_sent; + bool hello_rcvd; struct sk_buff_head rx_queue; struct list_head item; @@ -541,6 +543,11 @@ int qrtr_endpoint_post(struct qrtr_endpoint *ep, const void *data, size_t len) qrtr_node_assign(node, le32_to_cpu(pkt->server.node)); } + if (cb->type == QRTR_TYPE_HELLO) { + /* Pairs with smp_load_acquire() in qrtr_hello_work() */ + smp_store_release(&node->hello_rcvd, true); + } + if (cb->type == QRTR_TYPE_RESUME_TX) { qrtr_tx_resume(node, skb); } else { @@ -599,6 +606,8 @@ static void qrtr_hello_work(struct work_struct *work) struct qrtr_node *node; struct qrtr_sock *ctrl; struct sk_buff *skb; + bool hello_sent; + int rc = 0; node = container_of(to_delayed_work(work), struct qrtr_node, say_hello); @@ -609,17 +618,45 @@ static void qrtr_hello_work(struct work_struct *work) return; } - skb = qrtr_alloc_ctrl_packet(&pkt, GFP_KERNEL); - if (!skb) { - qrtr_port_put(ctrl); - schedule_delayed_work(&node->say_hello, msecs_to_jiffies(100)); - return; + mutex_lock(&node->ep_lock); + hello_sent = node->hello_sent; + mutex_unlock(&node->ep_lock); + + from.sq_node = qrtr_local_nid; + if (!hello_sent) { + skb = qrtr_alloc_ctrl_packet(&pkt, GFP_KERNEL); + if (!skb) + goto retry; + + pkt->cmd = cpu_to_le32(QRTR_TYPE_HELLO); + to.sq_node = node->nid; + if (qrtr_node_enqueue(node, skb, QRTR_TYPE_HELLO, &from, &to)) + goto out; } + /* Pairs with smp_store_release() in qrtr_endpoint_post() */ + if (!smp_load_acquire(&node->hello_rcvd)) + goto out; + + skb = qrtr_alloc_ctrl_packet(&pkt, GFP_KERNEL); + if (!skb) + goto retry; + pkt->cmd = cpu_to_le32(QRTR_TYPE_HELLO); - from.sq_node = qrtr_local_nid; - to.sq_node = node->nid; - qrtr_node_enqueue(node, skb, QRTR_TYPE_HELLO, &from, &to); + to.sq_node = READ_ONCE(node->nid); + + mutex_lock(&node->ep_lock); + if (node->ep) + rc = qrtr_local_enqueue(NULL, skb, QRTR_TYPE_HELLO, &to, &from); + else + kfree_skb(skb); + mutex_unlock(&node->ep_lock); + if (!rc) + goto out; + +retry: + schedule_delayed_work(&node->say_hello, msecs_to_jiffies(100)); +out: qrtr_port_put(ctrl); } diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c index 3e8ee4a691c3..d2e0d20f3ba1 100644 --- a/net/sched/sch_generic.c +++ b/net/sched/sch_generic.c @@ -96,9 +96,34 @@ static void qdisc_maybe_clear_missed(struct Qdisc *q, #define SKB_XOFF_MAGIC ((struct sk_buff *)1UL) +/* Re-cap a stale queue_mapping; an ONETXQUEUE qdisc pins q->dev_queue. */ +static u16 qdisc_cap_skb_tx_queue(struct Qdisc *q, struct sk_buff *skb) +{ + u16 queue = skb_get_queue_mapping(skb); + struct net_device *dev = qdisc_dev(q); + u16 capped; + + if (q->flags & TCQ_F_ONETXQUEUE) + capped = q->dev_queue - netdev_get_tx_queue(dev, 0); + else + capped = netdev_cap_txqueue(dev, queue); + + if (unlikely(capped != queue)) + skb_set_queue_mapping(skb, capped); + + return capped; +} + +static struct netdev_queue *skb_cap_tx_queue(struct Qdisc *q, + struct sk_buff *skb) +{ + return netdev_get_tx_queue(qdisc_dev(q), + qdisc_cap_skb_tx_queue(q, skb)); +} + static inline struct sk_buff *__skb_dequeue_bad_txq(struct Qdisc *q) { - const struct netdev_queue *txq = q->dev_queue; + const struct netdev_queue *txq; spinlock_t *lock = NULL; struct sk_buff *skb; @@ -110,7 +135,8 @@ static inline struct sk_buff *__skb_dequeue_bad_txq(struct Qdisc *q) skb = skb_peek(&q->skb_bad_txq); if (skb) { /* check the reason of requeuing without tx lock first */ - txq = skb_get_tx_queue(txq->dev, skb); + /* re-cap: real_num_tx_queues may have shrunk */ + txq = skb_cap_tx_queue(q, skb); if (!netif_xmit_frozen_or_stopped(txq)) { skb = __skb_dequeue(&q->skb_bad_txq); if (qdisc_is_percpu_stats(q)) { @@ -224,6 +250,8 @@ static void try_bulk_dequeue_skb(struct Qdisc *q, if (!nskb) break; + /* Every element of an ONETXQUEUE qdisc is pinned to q->dev_queue. */ + qdisc_cap_skb_tx_queue(q, nskb); bytelimit -= nskb->len; /* covers GSO len */ skb->next = nskb; skb = nskb; @@ -241,7 +269,6 @@ static void try_bulk_dequeue_skb_slow(struct Qdisc *q, struct sk_buff *skb, int *packets) { - int mapping = skb_get_queue_mapping(skb); struct sk_buff *nskb; int cnt = 0; @@ -249,7 +276,9 @@ static void try_bulk_dequeue_skb_slow(struct Qdisc *q, nskb = q->dequeue(q); if (!nskb) break; - if (unlikely(skb_get_queue_mapping(nskb) != mapping)) { + /* A tc BPF store may have changed the mapping; cap it. */ + if (unlikely(qdisc_cap_skb_tx_queue(q, nskb) != + skb_get_queue_mapping(skb))) { qdisc_enqueue_skb_bad_txq(q, nskb); break; } @@ -294,7 +323,7 @@ static struct sk_buff *dequeue_skb(struct Qdisc *q, bool *validate, if (xfrm_offload(skb)) *validate = true; /* check the reason of requeuing without tx lock first */ - txq = skb_get_tx_queue(txq->dev, skb); + txq = skb_cap_tx_queue(q, skb); if (!netif_xmit_frozen_or_stopped(txq)) { skb = __skb_dequeue(&q->gso_skb); if (qdisc_is_percpu_stats(q)) { @@ -330,6 +359,8 @@ static struct sk_buff *dequeue_skb(struct Qdisc *q, bool *validate, skb = q->dequeue(q); if (skb) { bulk: + /* cap the head; the bulk helpers cap every appended element */ + qdisc_cap_skb_tx_queue(q, skb); if (qdisc_may_bulk(q)) try_bulk_dequeue_skb(q, skb, txq, packets, budget); else diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c index 6f3b7273cb16..e900890e9c54 100644 --- a/net/sched/sch_qfq.c +++ b/net/sched/sch_qfq.c @@ -517,11 +517,14 @@ static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid, sch_tree_lock(sch); qfq_init_agg(q, new_agg, lmax, weight); } + if (existing && new_agg == cl->agg) + goto unlock; if (existing) qfq_deact_rm_from_agg(q, cl); else qdisc_class_hash_insert(&q->clhash, &cl->common); qfq_add_to_agg(q, new_agg, cl); +unlock: sch_tree_unlock(sch); qdisc_class_hash_grow(sch, &q->clhash); diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c index 39ac5b97aa3a..0eb0803047a1 100644 --- a/net/sched/sch_taprio.c +++ b/net/sched/sch_taprio.c @@ -632,6 +632,11 @@ static int taprio_enqueue(struct sk_buff *skb, struct Qdisc *sch, int queue; queue = skb_get_queue_mapping(skb); + if (unlikely(queue >= qdisc_dev(sch)->real_num_tx_queues)) { + /* A tc BPF program may have stored an out-of-range mapping. */ + queue = netdev_cap_txqueue(qdisc_dev(sch), queue); + skb_set_queue_mapping(skb, queue); + } child = q->qdiscs[queue]; if (unlikely(child == &noop_qdisc)) diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c index 84a4c97d0f75..0896ec6c75ce 100644 --- a/net/sctp/sm_make_chunk.c +++ b/net/sctp/sm_make_chunk.c @@ -94,7 +94,7 @@ static void sctp_control_release_owner(struct sk_buff *skb) } } -static void sctp_control_set_owner_w(struct sctp_chunk *chunk) +void sctp_control_set_owner_w(struct sctp_chunk *chunk) { struct sctp_association *asoc = chunk->asoc; struct sk_buff *skb = chunk->skb; @@ -107,7 +107,8 @@ static void sctp_control_set_owner_w(struct sctp_chunk *chunk) * For now don't do anything for now. */ if (chunk->auth) { - chunk->shkey = asoc->shkey; + if (!chunk->shkey) + chunk->shkey = asoc->shkey; sctp_auth_shkey_hold(chunk->shkey); } skb->sk = asoc ? asoc->base.sk : NULL; @@ -136,6 +137,7 @@ int sctp_init_cause(struct sctp_chunk *chunk, __be16 cause_code, size_t paylen) { struct sctp_errhdr err; + int chunklen, padlen; __u16 len; /* Cause code constants are now defined in network order. */ @@ -143,7 +145,13 @@ int sctp_init_cause(struct sctp_chunk *chunk, __be16 cause_code, len = sizeof(err) + paylen; err.length = htons(len); - if (skb_tailroom(chunk->skb) < len) + /* sctp_addto_chunk() pads the chunk up to a 4-byte boundary before + * appending the cause, so account for that padding here as well. + */ + chunklen = ntohs(chunk->chunk_hdr->length); + padlen = SCTP_PAD4(chunklen) - chunklen; + + if (skb_tailroom(chunk->skb) < padlen + len) return -ENOSPC; chunk->subh.err_hdr = sctp_addto_chunk(chunk, sizeof(err), &err); diff --git a/net/sctp/socket.c b/net/sctp/socket.c index c616f1609d0c..b5a57f61ee6d 100644 --- a/net/sctp/socket.c +++ b/net/sctp/socket.c @@ -148,25 +148,47 @@ static inline void sctp_set_owner_w(struct sctp_chunk *chunk) static void sctp_clear_owner_w(struct sctp_chunk *chunk) { + /* Keep the shkey alive until the new owner takes its reference. */ + if (chunk->shkey) + sctp_auth_shkey_hold(chunk->shkey); skb_orphan(chunk->skb); } +static void sctp_set_owner_w_migrate(struct sctp_chunk *chunk) +{ + struct sctp_shared_key *shkey = chunk->shkey; + + if (chunk->msg) + sctp_set_owner_w(chunk); + else + sctp_control_set_owner_w(chunk); + + if (shkey) + sctp_auth_shkey_release(shkey); +} + +static void sctp_process_tx_chunk(struct sctp_association *asoc, + struct sctp_chunk *chunk, bool clear, + void (*cb)(struct sctp_chunk *)) +{ + if ((clear && asoc->base.sk == chunk->skb->sk) || + (!clear && asoc->base.sk != chunk->skb->sk)) + cb(chunk); +} + #define traverse_and_process() \ do { \ msg = chunk->msg; \ if (msg == prev_msg) \ continue; \ - list_for_each_entry(c, &msg->chunks, frag_list) { \ - if ((clear && asoc->base.sk == c->skb->sk) || \ - (!clear && asoc->base.sk != c->skb->sk)) \ - cb(c); \ - } \ + list_for_each_entry(c, &msg->chunks, frag_list) \ + sctp_process_tx_chunk(asoc, c, clear, cb); \ prev_msg = msg; \ } while (0) -static void sctp_for_each_tx_datachunk(struct sctp_association *asoc, - bool clear, - void (*cb)(struct sctp_chunk *)) +static void sctp_for_each_tx_chunk(struct sctp_association *asoc, + bool clear, + void (*cb)(struct sctp_chunk *)) { struct sctp_datamsg *msg, *prev_msg = NULL; @@ -189,6 +211,21 @@ static void sctp_for_each_tx_datachunk(struct sctp_association *asoc, list_for_each_entry(chunk, &q->out_chunk_list, list) traverse_and_process(); + + list_for_each_entry(chunk, &q->control_chunk_list, list) + sctp_process_tx_chunk(asoc, chunk, clear, cb); + + list_for_each_entry(chunk, &asoc->asconf_ack_list, transmitted_list) + sctp_process_tx_chunk(asoc, chunk, clear, cb); + + list_for_each_entry(chunk, &asoc->addip_chunk_list, list) + sctp_process_tx_chunk(asoc, chunk, clear, cb); + + if (asoc->strreset_chunk) + sctp_process_tx_chunk(asoc, asoc->strreset_chunk, clear, cb); + + if (asoc->addip_last_asconf) + sctp_process_tx_chunk(asoc, asoc->addip_last_asconf, clear, cb); } static void sctp_for_each_rx_skb(struct sctp_association *asoc, struct sock *sk, @@ -9647,9 +9684,9 @@ static int sctp_sock_migrate(struct sock *oldsk, struct sock *newsk, * paths won't try to lock it and then oldsk. */ lock_sock_nested(newsk, SINGLE_DEPTH_NESTING); - sctp_for_each_tx_datachunk(assoc, true, sctp_clear_owner_w); + sctp_for_each_tx_chunk(assoc, true, sctp_clear_owner_w); sctp_assoc_migrate(assoc, newsk); - sctp_for_each_tx_datachunk(assoc, false, sctp_set_owner_w); + sctp_for_each_tx_chunk(assoc, false, sctp_set_owner_w_migrate); /* If the association on the newsk is already closed before accept() * is called, set RCV_SHUTDOWN flag. diff --git a/net/smc/smc_rx.c b/net/smc/smc_rx.c index 197fddc6271f..63c178c10bf1 100644 --- a/net/smc/smc_rx.c +++ b/net/smc/smc_rx.c @@ -286,6 +286,27 @@ int smc_rx_wait(struct smc_sock *smc, long *timeo, size_t peeked, return rc; } +/* Wait for the pipe to release the spliced data. Unlike smc_rx_wait() + * it does not end on shutdown or sk_err, the data is already in the + * RMB; smc_conn_kill() wakes it and the caller checks conn->killed. + */ +static void smc_rx_wait_splice(struct smc_sock *smc, long *timeo) +{ + DEFINE_WAIT_FUNC(wait, woken_wake_function); + struct smc_connection *conn = &smc->conn; + struct sock *sk = &smc->sk; + + sk_set_bit(SOCKWQ_ASYNC_WAITDATA, sk); + add_wait_queue(sk_sleep(sk), &wait); + /* pairs with the smp_mb() in wq_has_sleeper(): the release drops + * splice_pending, then looks for a sleeper + */ + smp_mb(); + sk_wait_event(sk, timeo, !atomic_read(&conn->splice_pending), &wait); + remove_wait_queue(sk_sleep(sk), &wait); + sk_clear_bit(SOCKWQ_ASYNC_WAITDATA, sk); +} + static int smc_rx_recv_urg(struct smc_sock *smc, struct msghdr *msg, int len, int flags) { @@ -443,7 +464,23 @@ int smc_rx_recvmsg(struct smc_sock *smc, struct msghdr *msg, /* could be just 1 byte, even after waiting on data above */ readable = smc_rx_data_available(conn, peeked_bytes); splbytes = atomic_read(&conn->splice_pending); - if (!readable || (msg && splbytes)) { + if (msg && splbytes) { + /* a pipe holds bytes ahead of these; the copy path + * skips the timeout and signal checks + */ + if (!read_remaining) + break; + if (!timeo || signal_pending(current)) { + if (!read_done) + read_done = timeo ? + sock_intr_errno(timeo) : + -EAGAIN; + break; + } + smc_rx_wait_splice(smc, &timeo); + continue; + } + if (!readable) { if (splbytes) func = smc_rx_data_available_and_no_splice_pend; else diff --git a/net/sunrpc/rpcb_clnt.c b/net/sunrpc/rpcb_clnt.c index 4c0b7fefee4e..84c06f99e1c7 100644 --- a/net/sunrpc/rpcb_clnt.c +++ b/net/sunrpc/rpcb_clnt.c @@ -400,7 +400,8 @@ static struct rpc_clnt *rpcb_create(struct net *net, const char *nodename, return rpc_create(&args); } -static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, struct rpc_message *msg, bool is_set) +static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, + struct rpc_message *msg, bool is_set) { int flags = RPC_TASK_NOCONNECT; int error, result = 0; @@ -410,8 +411,23 @@ static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, stru msg->rpc_resp = &result; error = rpc_call_sync(clnt, msg, flags); - if (error < 0) - return error; + if (error < 0) { + switch (error) { + /* rpcbind answered; the reply itself carries the error */ + case -EPROTONOSUPPORT: + case -EPFNOSUPPORT: + case -EOPNOTSUPP: + case -EACCES: + /* the call never made it onto the wire */ + case -ENOMEM: + case -EMSGSIZE: + /* the caller is going away; this says nothing about rpcbind */ + case -ERESTARTSYS: + return error; + } + /* anything else, we assume that rpcbind isn't functional */ + return -EIO; + } if (!result) return -EACCES; diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index 4f402bbf97ba..ca6f90653327 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -1179,10 +1179,40 @@ int svc_generic_rpcbind_set(struct net *net, error = svc_rpcbind_set_version(net, progp, version, family, proto, port); + /* -EIO means no answer, not a refusal, so vs_rpcb_optnl must keep it. */ + if (error == -EIO) + return error; + return (vers->vs_rpcb_optnl) ? 0 : error; } EXPORT_SYMBOL_GPL(svc_generic_rpcbind_set); +/** + * svc_rpcb_failure_count - local rpcbind calls for @serv that got no answer + * @serv: RPC service to query + * + * svc_register() adds one for each of its calls that got no answer. A reply + * that refuses one entry does not count, because rpcbind answered and the + * next entry may still succeed. + * + * The count is kept per serv rather than per net. The local rpcbind client + * is per-net and lockd shares it, but a count that another service can move + * says nothing about this serv's own calls. + * + * This is for callers that cannot see the svc_register() return, because a + * transport class sits in between. Such a caller reads the count before it + * starts and compares as it goes, so there is no state to reset between + * operations. The count never resets, and callers must not attach meaning + * to the value itself. + * + * Return: the number of unanswered calls since this serv was created. + */ +unsigned int svc_rpcb_failure_count(struct svc_serv *serv) +{ + return atomic_read(&serv->sv_rpcb_failures); +} +EXPORT_SYMBOL_GPL(svc_rpcb_failure_count); + /** * svc_register - register an RPC service with the local portmapper * @serv: svc_serv struct for the service to register @@ -1193,10 +1223,11 @@ EXPORT_SYMBOL_GPL(svc_generic_rpcbind_set); * * Service is registered for any address in the passed-in protocol family */ -int svc_register(const struct svc_serv *serv, struct net *net, +int svc_register(struct svc_serv *serv, struct net *net, const int family, const unsigned short proto, const unsigned short port) { + bool noanswer = false; unsigned int p, i; int error = 0; @@ -1208,10 +1239,16 @@ int svc_register(const struct svc_serv *serv, struct net *net, struct svc_program *progp = &serv->sv_programs[p]; for (i = 0; i < progp->pg_nvers; i++) { + const struct svc_version *vers = progp->pg_vers[i]; int ret; ret = progp->pg_rpcbind_set(net, progp, i, family, proto, port); + if (ret == -EIO) { + noanswer = true; + if (vers && vers->vs_rpcb_optnl) + ret = 0; + } if (ret < 0) { printk(KERN_WARNING "svc: failed to register " "%sv%u RPC service (errno %d).\n", @@ -1223,6 +1260,9 @@ int svc_register(const struct svc_serv *serv, struct net *net, } } + if (noanswer) + atomic_inc(&serv->sv_rpcb_failures); + return error; } diff --git a/net/wireless/reg.c b/net/wireless/reg.c index 11665e0a7efc..44226687349e 100644 --- a/net/wireless/reg.c +++ b/net/wireless/reg.c @@ -679,6 +679,9 @@ static bool valid_rule(const u8 *data, unsigned int size, u16 rule_ptr) /* mandatory fields */ if (rule->len < offsetofend(struct fwdb_rule, max_bw)) return false; + /* the rule must lie entirely within the database */ + if ((u8 *)rule + rule->len > data + size) + return false; if (rule->len >= offsetofend(struct fwdb_rule, wmm_ptr)) { u32 wmm_ptr = be16_to_cpu(rule->wmm_ptr) << 2; struct fwdb_wmm_rule *wmm; diff --git a/net/xfrm/espintcp.c b/net/xfrm/espintcp.c index 3e72b9f067b9..4052ae5bbb2e 100644 --- a/net/xfrm/espintcp.c +++ b/net/xfrm/espintcp.c @@ -434,7 +434,9 @@ static void espintcp_destruct(struct sock *sk) bool tcp_is_ulp_esp(struct sock *sk) { - return sk->sk_prot == &espintcp_prot || sk->sk_prot == &espintcp6_prot; + const struct proto *prot = READ_ONCE(sk->sk_prot); + + return prot == &espintcp_prot || prot == &espintcp6_prot; } EXPORT_SYMBOL_GPL(tcp_is_ulp_esp); @@ -466,34 +468,41 @@ static int espintcp_init_sk(struct sock *sk) __sk_dst_reset(sk); - strp_check_rcv(&ctx->strp); skb_queue_head_init(&ctx->ike_queue); skb_queue_head_init(&ctx->out_queue); + INIT_WORK(&ctx->work, espintcp_tx_work); + + ctx->saved_data_ready = sk->sk_data_ready; + ctx->saved_write_space = sk->sk_write_space; + ctx->saved_destruct = sk->sk_destruct; + + /* avoid using task_frag */ + sk->sk_allocation = GFP_ATOMIC; + sk->sk_use_task_frag = false; + + rcu_assign_pointer(icsk->icsk_ulp_data, ctx); + + /* pairs with smp_rmb() in espintcp_getctx() */ + smp_wmb(); + + WRITE_ONCE(sk->sk_data_ready, espintcp_data_ready); + WRITE_ONCE(sk->sk_write_space, espintcp_write_space); + sk->sk_destruct = espintcp_destruct; if (sk->sk_family == AF_INET) { - sk->sk_prot = &espintcp_prot; - sk->sk_socket->ops = &espintcp_ops; + WRITE_ONCE(sk->sk_prot, &espintcp_prot); + WRITE_ONCE(sk->sk_socket->ops, &espintcp_ops); } else { mutex_lock(&tcpv6_prot_mutex); if (!espintcp6_prot.recvmsg) build_protos(&espintcp6_prot, &espintcp6_ops, sk->sk_prot, sk->sk_socket->ops); mutex_unlock(&tcpv6_prot_mutex); - sk->sk_prot = &espintcp6_prot; - sk->sk_socket->ops = &espintcp6_ops; + WRITE_ONCE(sk->sk_prot, &espintcp6_prot); + WRITE_ONCE(sk->sk_socket->ops, &espintcp6_ops); } - ctx->saved_data_ready = sk->sk_data_ready; - ctx->saved_write_space = sk->sk_write_space; - ctx->saved_destruct = sk->sk_destruct; - sk->sk_data_ready = espintcp_data_ready; - sk->sk_write_space = espintcp_write_space; - sk->sk_destruct = espintcp_destruct; - rcu_assign_pointer(icsk->icsk_ulp_data, ctx); - INIT_WORK(&ctx->work, espintcp_tx_work); - /* avoid using task_frag */ - sk->sk_allocation = GFP_ATOMIC; - sk->sk_use_task_frag = false; + strp_check_rcv(&ctx->strp); return 0; diff --git a/net/xfrm/xfrm_interface_core.c b/net/xfrm/xfrm_interface_core.c index 688306bf62c5..9835fbe63ccb 100644 --- a/net/xfrm/xfrm_interface_core.c +++ b/net/xfrm/xfrm_interface_core.c @@ -531,8 +531,13 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev) memset(&fl, 0, sizeof(fl)); + if (!pskb_inet_may_pull(skb)) + goto tx_err; + switch (skb->protocol) { case htons(ETH_P_IPV6): + if (ipv6_hdr(skb)->version != 6) + goto tx_err; memset(IP6CB(skb), 0, sizeof(*IP6CB(skb))); xfrm_decode_session(dev_net(dev), skb, &fl, AF_INET6); if (!dst) { @@ -548,6 +553,9 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev) } break; case htons(ETH_P_IP): + if (ip_hdr(skb)->version != 4 || ip_hdr(skb)->ihl < 5 || + !pskb_network_may_pull(skb, ip_hdr(skb)->ihl * 4)) + goto tx_err; memset(IPCB(skb), 0, sizeof(*IPCB(skb))); xfrm_decode_session(dev_net(dev), skb, &fl, AF_INET); if (!dst) { diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c index 513c9f228334..f6f40ba713d5 100644 --- a/net/xfrm/xfrm_policy.c +++ b/net/xfrm/xfrm_policy.c @@ -2157,6 +2157,7 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type, struct xfrm_pol_inexact_bin *bin; struct xfrm_policy *pol, *ret; struct hlist_head *chain; + unsigned int inexact_sequence; unsigned int sequence; int err; @@ -2191,12 +2192,18 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type, goto skip_inexact; bin = xfrm_policy_inexact_lookup_rcu(net, type, family, dir, if_id); - if (!bin || !xfrm_policy_find_inexact_candidates(&cand, bin, saddr, - daddr)) + if (!bin) + goto skip_inexact; + + inexact_sequence = read_seqcount_begin(&bin->count); + if (!xfrm_policy_find_inexact_candidates(&cand, bin, saddr, daddr)) goto skip_inexact; pol = xfrm_policy_eval_candidates(&cand, ret, fl, type, family, if_id); + if (read_seqcount_retry(&bin->count, inexact_sequence)) + goto retry; + if (pol) { ret = pol; if (IS_ERR(pol)) diff --git a/rust/kernel/cpufreq.rs b/rust/kernel/cpufreq.rs index b75d692846a2..f31671129c98 100644 --- a/rust/kernel/cpufreq.rs +++ b/rust/kernel/cpufreq.rs @@ -696,8 +696,8 @@ fn clear_data(&mut self) -> Option { impl<'a> PolicyCpu<'a> { fn from_cpu(cpu: CpuId) -> Result { // SAFETY: It is safe to call `cpufreq_cpu_get` for any valid CPU. - let ptr = - NonNull::new(unsafe { bindings::cpufreq_cpu_get(u32::from(cpu)) }).ok_or(ENODEV)?; + let ptr = unsafe { bindings::cpufreq_cpu_get(u32::from(cpu)) }; + let ptr = NonNull::new(ptr).ok_or(ENODEV)?; Ok(Self( // SAFETY: `ptr` is non-NULL and `cpufreq_cpu_get()` took a reference on it, so it is diff --git a/sound/core/hwdep.c b/sound/core/hwdep.c index c6de5345e946..fb9034897c56 100644 --- a/sound/core/hwdep.c +++ b/sound/core/hwdep.c @@ -212,6 +212,7 @@ static int snd_hwdep_dsp_load(struct snd_hwdep *hw, return -ENXIO; if (info->index >= 32) return -EINVAL; + guard(mutex)(&hw->dsp_mutex); /* check whether the dsp was already loaded */ if (hw->dsp_loaded & (1u << info->index)) return -EBUSY; @@ -380,6 +381,7 @@ int snd_hwdep_new(struct snd_card *card, char *id, int device, init_waitqueue_head(&hwdep->open_wait); mutex_init(&hwdep->open_mutex); + mutex_init(&hwdep->dsp_mutex); hwdep->card = card; hwdep->device = device; if (id) diff --git a/sound/core/ump_convert.c b/sound/core/ump_convert.c index 0a7972d91ec5..ecbd894f4c83 100644 --- a/sound/core/ump_convert.c +++ b/sound/core/ump_convert.c @@ -69,7 +69,7 @@ static u32 upscale_14_to_32bit(u16 src) static int cvt_ump_system_to_legacy(u32 data, unsigned char *buf) { buf[0] = ump_message_status_channel(data); - switch (ump_message_status_code(data)) { + switch (buf[0]) { case UMP_SYSTEM_STATUS_MIDI_TIME_CODE: case UMP_SYSTEM_STATUS_SONG_SELECT: buf[1] = (data >> 8) & 0x7f; diff --git a/sound/hda/codecs/realtek/alc269.c b/sound/hda/codecs/realtek/alc269.c index 68650004ce0f..49c4d01759b0 100644 --- a/sound/hda/codecs/realtek/alc269.c +++ b/sound/hda/codecs/realtek/alc269.c @@ -3537,6 +3537,27 @@ static void alc233_fixup_no_audio_jack(struct hda_codec *codec, alc_process_coef_fw(codec, alc233_fixup_no_audio_jack_coefs); } +static const struct coef_fw alc256_asus_speaker_coefs[] = { + WRITE_COEF(0x10, 0x7f20), WRITE_COEF(0x16, 0x0c50), WRITE_COEF(0x35, 0x8d6a), + WRITE_COEF(0x37, 0xfe06), WRITE_COEF(0x57, 0x7f7f), + {} +}; + +static void alc256_fixup_asus_speaker_coefs(struct hda_codec *codec, + const struct hda_fixup *fix, + int action) +{ + /* + * The internal speakers of the ASUS ExpertBook PM3606CHA stay silent + * with the default COEF values. These are the values set by the + * Windows driver. They are applied on every init so that they are + * restored after resume, too. + */ + if (action != HDA_FIXUP_ACT_INIT) + return; + alc_process_coef_fw(codec, alc256_asus_speaker_coefs); +} + static void alc256_fixup_mic_no_presence_and_resume(struct hda_codec *codec, const struct hda_fixup *fix, int action) @@ -4391,6 +4412,8 @@ enum { ALC285_LENOVO_DAC_RENAME, ALC287_FIXUP_YOGA9_SPEAKER2_TO_DAC1, ALC256_FIXUP_IPASON_SMARTBOOK_S1, + ALC256_FIXUP_ASUS_SPEAKER_COEFS, + ALC245_FIXUP_MINISFORUM_V3_MIC_NO_PRESENCE, }; /* A special fixup for Lenovo C940 and Yoga Duet 7; @@ -7208,6 +7231,21 @@ static const struct hda_fixup alc269_fixups[] = { { } }, }, + [ALC256_FIXUP_ASUS_SPEAKER_COEFS] = { + .type = HDA_FIXUP_FUNC, + .v.func = alc256_fixup_asus_speaker_coefs, + .chained = true, + .chain_id = ALC256_FIXUP_ASUS_MIC_NO_PRESENCE, + }, + [ALC245_FIXUP_MINISFORUM_V3_MIC_NO_PRESENCE] = { + .type = HDA_FIXUP_PINS, + .v.pins = (const struct hda_pintbl[]) { + { 0x19, 0x04a19150 }, /* headset mic jack, no presence detect */ + { } + }, + .chained = true, + .chain_id = ALC245_FIXUP_BASS_HP_DAC + }, }; static const struct hda_quirk alc269_fixup_tbl[] = { @@ -7335,6 +7373,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x1028, 0x097e, "Dell Precision", ALC289_FIXUP_DUAL_SPK), SND_PCI_QUIRK(0x1028, 0x098d, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE), SND_PCI_QUIRK(0x1028, 0x09bf, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE), + SND_PCI_QUIRK(0x1028, 0x09d1, "Alienware m17 R3", ALC289_FIXUP_DUAL_SPK), SND_PCI_QUIRK(0x1028, 0x0a2e, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC), SND_PCI_QUIRK(0x1028, 0x0a30, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC), SND_PCI_QUIRK(0x1028, 0x0a38, "Dell Latitude 7520", ALC269_FIXUP_DELL4_MIC_NO_PRESENCE_QUIET), @@ -7605,7 +7644,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x103c, 0x8a2b, "HP Envy 15", ALC287_FIXUP_CS35L41_I2C_2), SND_PCI_QUIRK(0x103c, 0x8a2c, "HP Envy 16", ALC287_FIXUP_CS35L41_I2C_2), SND_PCI_QUIRK(0x103c, 0x8a2d, "HP Envy 16", ALC287_FIXUP_CS35L41_I2C_2), - SND_PCI_QUIRK(0x103c, 0x8a2e, "HP Envy 16", ALC287_FIXUP_CS35L41_I2C_2), + SND_PCI_QUIRK(0x103c, 0x8a2e, "HP ENVY Laptop 17-cr0xxx", ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS), SND_PCI_QUIRK(0x103c, 0x8a30, "HP Envy 17", ALC287_FIXUP_CS35L41_I2C_2), SND_PCI_QUIRK(0x103c, 0x8a31, "HP ENVY x360 2-in-1 Laptop 15-ey0xxx", ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS), SND_PCI_QUIRK(0x103c, 0x8a34, "HP Pavilion x360 2-in-1 Laptop 14-ek0xxx", ALC245_FIXUP_HP_MUTE_LED_COEFBIT), @@ -7990,6 +8029,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x1043, 0x31e1, "ASUS B5605CCA", ALC294_FIXUP_ASUS_CS35L41_SPI_2), SND_PCI_QUIRK(0x1043, 0x31f1, "ASUS B3605CCA", ALC294_FIXUP_ASUS_CS35L41_SPI_2), SND_PCI_QUIRK(0x1043, 0x3391, "ASUS PM3606CKA", ALC287_FIXUP_CS35L41_I2C_2), + SND_PCI_QUIRK(0x1043, 0x3501, "ASUS PM3606CHA", ALC256_FIXUP_ASUS_SPEAKER_COEFS), SND_PCI_QUIRK(0x1043, 0x3601, "ASUS PM5406CGA", ALC287_FIXUP_CS35L41_I2C_2), SND_PCI_QUIRK(0x1043, 0x3611, "ASUS PM5606CGA", ALC287_FIXUP_CS35L41_I2C_2), SND_PCI_QUIRK(0x1043, 0x3701, "ASUS P5406CCA", ALC245_FIXUP_CS35L41_SPI_2), @@ -8445,7 +8485,8 @@ static const struct hda_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x1ee7, 0x2081, "HONOR MRB-XXX M1020", ALC256_FIXUP_HONOR_MRB_XXX_M1020_AUDIO), SND_PCI_QUIRK(0x1f4c, 0xb020, "Minisforum AI X1 Pro", ALC245_FIXUP_MINISFORUM_JACK_DETECT), - SND_PCI_QUIRK(0x1f4c, 0xe001, "Minisforum V3 (SE)", ALC245_FIXUP_BASS_HP_DAC), + SND_PCI_QUIRK(0x1f4c, 0xe001, "Minisforum V3 (SE)", + ALC245_FIXUP_MINISFORUM_V3_MIC_NO_PRESENCE), SND_PCI_QUIRK(0x1f66, 0x0105, "Ayaneo Portable Game Player", ALC287_FIXUP_CS35L41_I2C_2), SND_PCI_QUIRK(0x2014, 0x800a, "Positivo ARN50", ALC269_FIXUP_LIMIT_INT_MIC_BOOST), SND_PCI_QUIRK(0x2039, 0x0001, "Inspur S14-G1", ALC295_FIXUP_CHROME_BOOK), diff --git a/sound/hda/codecs/side-codecs/tas2781_hda.c b/sound/hda/codecs/side-codecs/tas2781_hda.c index e0da299df780..9162173ccb10 100644 --- a/sound/hda/codecs/side-codecs/tas2781_hda.c +++ b/sound/hda/codecs/side-codecs/tas2781_hda.c @@ -50,6 +50,31 @@ static void cali_cnv(unsigned char *data, unsigned int base, int offset) (u32 *)®_data, TASDEV_CALIB_N); } +/* + * V1 data without CRC is only trusted if every expected device has a + * populated, self-consistent entry: R0 (Q27) * 1/R0 (Q31) must be + * close to 1.0 in Q58. + */ +static bool tas2781_v1_nocrc_valid(const unsigned int *v, int ndev) +{ + const u64 one = 1ULL << 58; + int i; + + if (ndev <= 0 || ndev > 4) + return false; + + for (i = 0; i < ndev; i++) { + const unsigned int *e = &v[i * 5]; + u64 prod = (u64)e[0] * e[1]; + + if (!e[0] || !e[1] || + prod < one - one / 10 || prod > one + one / 10) + return false; + } + + return true; +} + static void tas2781_apply_calib(struct tasdevice_priv *p) { struct calidata *cali_data = &p->cali_data; @@ -145,7 +170,7 @@ static void tas2781_apply_calib(struct tasdevice_priv *p) * but leave TimeStamp and CRC zeroed. */ if (!tmp_val[20] && !tmp_val[21] && - tmp_val[0] && tmp_val[1]) { + tas2781_v1_nocrc_valid(tmp_val, p->ndev)) { dev_warn(p->dev, "%s: V1 data without CRC, accepting\n", __func__); diff --git a/sound/pci/cs46xx/dsp_spos.c b/sound/pci/cs46xx/dsp_spos.c index a77e6f5a14ad..57b0a6321e0b 100644 --- a/sound/pci/cs46xx/dsp_spos.c +++ b/sound/pci/cs46xx/dsp_spos.c @@ -1104,6 +1104,8 @@ int cs46xx_dsp_scb_and_task_init (struct snd_cs46xx *chip) null_scb.entry_point = null_algorithm->address; ins->the_null_scb = cs46xx_dsp_create_scb(chip, "nullSCB", (u32 *)&null_scb, NULL_SCB_ADDR); + if (!ins->the_null_scb) + return -ENOMEM; ins->the_null_scb->task_entry = null_algorithm; ins->the_null_scb->sub_list_ptr = ins->the_null_scb; ins->the_null_scb->next_scb_ptr = ins->the_null_scb; diff --git a/sound/pci/cs46xx/dsp_spos_scb_lib.c b/sound/pci/cs46xx/dsp_spos_scb_lib.c index fd19365026b4..c5208e42cde9 100644 --- a/sound/pci/cs46xx/dsp_spos_scb_lib.c +++ b/sound/pci/cs46xx/dsp_spos_scb_lib.c @@ -274,7 +274,8 @@ _dsp_create_generic_scb (struct snd_cs46xx *chip, char * name, u32 * scb_data, u dev_dbg(chip->card->dev, "dsp_spos: creating SCB <%s>\n", name); scb = cs46xx_dsp_create_scb(chip,name,scb_data,dest); - + if (!scb) + return NULL; scb->sub_list_ptr = ins->the_null_scb; scb->next_scb_ptr = ins->the_null_scb; diff --git a/sound/soc/amd/yc/acp6x-mach.c b/sound/soc/amd/yc/acp6x-mach.c index 1be8504571ad..96b0bfeddc06 100644 --- a/sound/soc/amd/yc/acp6x-mach.c +++ b/sound/soc/amd/yc/acp6x-mach.c @@ -745,6 +745,13 @@ static const struct dmi_system_id yc_acp_quirk_table[] = { DMI_MATCH(DMI_BOARD_NAME, "8BD6"), } }, + { + .driver_data = &acp6x_card, + .matches = { + DMI_MATCH(DMI_BOARD_VENDOR, "HP"), + DMI_MATCH(DMI_BOARD_NAME, "8EE3"), + } + }, { .driver_data = &acp6x_card, .matches = { diff --git a/sound/soc/codecs/cs35l56-sdw.c b/sound/soc/codecs/cs35l56-sdw.c index e10f04d677f4..a49ad69644b5 100644 --- a/sound/soc/codecs/cs35l56-sdw.c +++ b/sound/soc/codecs/cs35l56-sdw.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -36,6 +37,9 @@ #define CS35L56_LATE_READ_POLL_US 10 #define CS35L56_LATE_READ_TIMEOUT_US 1000 +static DEFINE_MUTEX(cs35l56_fw_idle_wait_lock); +static bool cs35l56_fw_idle_wait_pending; + static int cs35l56_sdw_poll_mem_status(struct sdw_slave *peripheral, unsigned int mask, unsigned int match) @@ -387,14 +391,34 @@ static int __maybe_unused cs35l56_sdw_runtime_resume(struct device *dev) return 0; } +static int cs35l56_sdw_system_suspend_prepare(struct device *dev) +{ + cs35l56_fw_idle_wait_pending = true; + + return 0; +} + static int __maybe_unused cs35l56_sdw_system_suspend(struct device *dev) { struct cs35l56_private *cs35l56 = dev_get_drvdata(dev); + int ret; if (cs35l56->sdw_attached) cs35l56_mask_soundwire_interrupts(cs35l56); - return cs35l56_system_suspend(dev); + ret = cs35l56_system_suspend(dev); + if (ret < 0) + return ret; + + scoped_guard(mutex, &cs35l56_fw_idle_wait_lock) { + if (cs35l56_fw_idle_wait_pending && cs35l56->needs_wait_for_fw_idle) { + dev_dbg(cs35l56->base.dev, "Wait for FW timer expiry\n"); + msleep(CS35L56_FW_REQ_ACTIVE_TIMEOUT_MS + 10); + cs35l56_fw_idle_wait_pending = false; + } + } + + return 0; } static int cs35l56_sdw_probe(struct sdw_slave *peripheral, const struct sdw_device_id *id) @@ -460,6 +484,7 @@ static void cs35l56_sdw_remove(struct sdw_slave *peripheral) } static const struct dev_pm_ops cs35l56_sdw_pm = { + .prepare = cs35l56_sdw_system_suspend_prepare, SET_RUNTIME_PM_OPS(cs35l56_sdw_runtime_suspend, cs35l56_sdw_runtime_resume, NULL) SYSTEM_SLEEP_PM_OPS(cs35l56_sdw_system_suspend, cs35l56_system_resume) LATE_SYSTEM_SLEEP_PM_OPS(cs35l56_system_suspend_late, cs35l56_system_resume_early) diff --git a/sound/soc/codecs/cs35l56-shared.c b/sound/soc/codecs/cs35l56-shared.c index 7b3e37d462d6..b37f65cd4d33 100644 --- a/sound/soc/codecs/cs35l56-shared.c +++ b/sound/soc/codecs/cs35l56-shared.c @@ -1379,6 +1379,44 @@ void cs35l56_warn_if_firmware_missing(struct cs35l56_base *cs35l56_base) } EXPORT_SYMBOL_NS_GPL(cs35l56_warn_if_firmware_missing, "SND_SOC_CS35L56_SHARED"); +bool cs35l56_needs_wait_for_firmware_timer_expiry(struct cs35l56_base *cs35l56_base) +{ + unsigned int fw_ver; + bool prot_sts; + int ret; + + switch (cs35l56_base->type) { + case 0x54: + case 0x56: + case 0x57: + switch (cs35l56_base->rev) { + case 0xb0: + ret = cs35l56_read_prot_status(cs35l56_base, &prot_sts, &fw_ver); + if (ret) + return true; + + if ((fw_ver & CS35L56_FW_MAIN_VERSION_MASK) < 0x30d07) + return true; + + return false; + case 0xb2: + ret = cs35l56_read_prot_status(cs35l56_base, &prot_sts, &fw_ver); + if (ret) + return true; + + if ((fw_ver & CS35L56_FW_MAIN_VERSION_MASK) < 0x40710) + return true; + + return false; + default: + return false; + } + default: + return false; + } +} +EXPORT_SYMBOL_NS_GPL(cs35l56_needs_wait_for_firmware_timer_expiry, "SND_SOC_CS35L56_SHARED"); + void cs35l56_log_tuning(struct cs35l56_base *cs35l56_base, struct cs_dsp *cs_dsp) { __be32 pid, sid, tid; diff --git a/sound/soc/codecs/cs35l56.c b/sound/soc/codecs/cs35l56.c index 279db3f13f1e..573cc71a1910 100644 --- a/sound/soc/codecs/cs35l56.c +++ b/sound/soc/codecs/cs35l56.c @@ -1693,6 +1693,10 @@ int cs35l56_system_resume(struct device *dev) dev_dbg(dev, "system_resume\n"); + /* Assume the firmware reset to default until we can check it */ + if (cs35l56->base.init_done) + cs35l56->needs_wait_for_fw_idle = true; + /* * We might have done a hard reset or the CS35L56 was power-cycled * so wait for control port to be ready. @@ -1707,6 +1711,16 @@ int cs35l56_system_resume(struct device *dev) if (ret) return ret; + if (cs35l56->sdw_peripheral && cs35l56->base.init_done) { + ret = pm_runtime_resume_and_get(cs35l56->base.dev); + if (ret < 0) + return ret; + + cs35l56->needs_wait_for_fw_idle = + cs35l56_needs_wait_for_firmware_timer_expiry(&cs35l56->base); + pm_runtime_put_autosuspend(cs35l56->base.dev); + } + /* Firmware won't have been loaded if the component hasn't probed */ if (!cs35l56->component) return 0; @@ -2231,6 +2245,9 @@ int cs35l56_init(struct cs35l56_private *cs35l56) if (ret) return dev_err_probe(cs35l56->base.dev, ret, "Failed to write ASP1_CONTROL3\n"); + cs35l56->needs_wait_for_fw_idle = + cs35l56_needs_wait_for_firmware_timer_expiry(&cs35l56->base); + cs35l56->base.init_done = true; complete_all(&cs35l56->init_completion); diff --git a/sound/soc/codecs/cs35l56.h b/sound/soc/codecs/cs35l56.h index f7cf8aa653e2..b46f410f0e09 100644 --- a/sound/soc/codecs/cs35l56.h +++ b/sound/soc/codecs/cs35l56.h @@ -43,6 +43,7 @@ struct cs35l56_private { bool soft_resetting; bool sdw_attached; bool component_registered; + bool needs_wait_for_fw_idle; struct completion init_completion; int speaker_id; diff --git a/sound/soc/codecs/hdac_hda.c b/sound/soc/codecs/hdac_hda.c index 40d602f79314..9e45b20940f2 100644 --- a/sound/soc/codecs/hdac_hda.c +++ b/sound/soc/codecs/hdac_hda.c @@ -633,12 +633,35 @@ static const struct snd_soc_component_driver hdac_hda_hdmi_codec = { .endianness = 1, }; +static void hdac_hda_restore_dev_type(void *data) +{ + struct hdac_device *hdev = data; + + hdev->type = HDA_DEV_LEGACY; +} + static int hdac_hda_dev_probe(struct hdac_device *hdev) { - struct hdac_hda_priv *hda_pvt = dev_get_drvdata(&hdev->dev); + struct hdac_hda_priv *hda_pvt; struct hdac_ext_link *hlink; int ret; + hda_pvt = devm_kzalloc(&hdev->dev, sizeof(*hda_pvt), GFP_KERNEL); + if (!hda_pvt) + return -ENOMEM; + + hda_pvt->codec = dev_to_hda_codec(&hdev->dev); + hda_pvt->dev_index = hdev->addr; + /* Intel iDisp (HDMI/DP) codec */ + if (((hdev->vendor_id >> 16) & 0xFFFF) == 0x8086) + hda_pvt->need_display_power = true; + dev_set_drvdata(&hdev->dev, hda_pvt); + + /* Allow the legacy codec driver to match again after unbind */ + ret = devm_add_action(&hdev->dev, hdac_hda_restore_dev_type, hdev); + if (ret) + return ret; + /* hold the ref while we probe */ hlink = snd_hdac_ext_bus_get_hlink_by_name(hdev->bus, dev_name(&hdev->dev)); if (!hlink) { diff --git a/sound/soc/codecs/tas2770.c b/sound/soc/codecs/tas2770.c index e79754edac5c..355fcbfd01c7 100644 --- a/sound/soc/codecs/tas2770.c +++ b/sound/soc/codecs/tas2770.c @@ -513,7 +513,7 @@ static int tas2770_set_dai_tdm_idle(struct snd_soc_dai *dai, ret = snd_soc_component_update_bits(component, TAS2770_DIN_PD, TAS2770_DIN_PD_SDOUT, TAS2770_DIN_PD_SDOUT); - if (ret) + if (ret < 0) return ret; break; @@ -521,12 +521,12 @@ static int tas2770_set_dai_tdm_idle(struct snd_soc_dai *dai, ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4, TAS2770_TDM_CFG_REG4_TX_KEEPER, TAS2770_TDM_CFG_REG4_TX_KEEPER); - if (ret) + if (ret < 0) return ret; ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4, TAS2770_TDM_CFG_REG4_TX_FILL, 0); - if (ret) + if (ret < 0) return ret; break; @@ -534,25 +534,25 @@ static int tas2770_set_dai_tdm_idle(struct snd_soc_dai *dai, ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4, TAS2770_TDM_CFG_REG4_TX_KEEPER, TAS2770_TDM_CFG_REG4_TX_KEEPER); - if (ret) + if (ret < 0) return ret; ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4, TAS2770_TDM_CFG_REG4_TX_FILL, TAS2770_TDM_CFG_REG4_TX_FILL); - if (ret) + if (ret < 0) return ret; break; case SND_SOC_DAI_TDM_IDLE_OFF: ret = snd_soc_component_update_bits(component, TAS2770_DIN_PD, TAS2770_DIN_PD_SDOUT, 0); - if (ret) + if (ret < 0) return ret; ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4, TAS2770_TDM_CFG_REG4_TX_KEEPER, 0); - if (ret) + if (ret < 0) return ret; break; diff --git a/sound/soc/qcom/qdsp6/q6apm.c b/sound/soc/qcom/qdsp6/q6apm.c index a79bbb34c777..8a756249e588 100644 --- a/sound/soc/qcom/qdsp6/q6apm.c +++ b/sound/soc/qcom/qdsp6/q6apm.c @@ -884,8 +884,6 @@ static int apm_probe(gpr_device_t *gdev) if (!apm) return -ENOMEM; - dev_set_drvdata(dev, apm); - mutex_init(&apm->lock); apm->dev = dev; apm->gdev = gdev; @@ -899,6 +897,7 @@ static int apm_probe(gpr_device_t *gdev) idr_init(&apm->modules_idr); + dev_set_drvdata(dev, apm); g_apm = apm; q6apm_get_apm_state(apm); @@ -942,6 +941,9 @@ static int apm_callback(const struct gpr_resp_pkt *data, void *priv, int op) const struct gpr_hdr *hdr = &data->hdr; int graph_id, is_pos_buf; + if (!apm) + return 0; + result = data->payload; switch (hdr->opcode) { diff --git a/sound/soc/qcom/qdsp6/q6prm.c b/sound/soc/qcom/qdsp6/q6prm.c index 1f3ce4cc0837..bcc73127d54d 100644 --- a/sound/soc/qcom/qdsp6/q6prm.c +++ b/sound/soc/qcom/qdsp6/q6prm.c @@ -192,6 +192,9 @@ static int prm_callback(const struct gpr_resp_pkt *data, void *priv, int op) const struct gpr_ibasic_rsp_result_t *result; const struct gpr_hdr *hdr = &data->hdr; + if (!prm) + return 0; + switch (hdr->opcode) { case PRM_CMD_RSP_REQUEST_HW_RSC: case PRM_CMD_RSP_RELEASE_HW_RSC: diff --git a/sound/soc/sof/intel/hda-codec.c b/sound/soc/sof/intel/hda-codec.c index fd371850b0d6..32fcc1cd3f92 100644 --- a/sound/soc/sof/intel/hda-codec.c +++ b/sound/soc/sof/intel/hda-codec.c @@ -15,8 +15,6 @@ #include "hda.h" #if IS_ENABLED(CONFIG_SND_SOC_SOF_HDA_AUDIO_CODEC) -#include "../../codecs/hdac_hda.h" - #define CODEC_PROBE_RETRIES 3 #define IDISP_VID_INTEL 0x80860000 @@ -148,7 +146,6 @@ static struct hda_codec *hda_codec_device_init(struct hdac_bus *bus, int addr, i /* probe individual codec */ static int hda_codec_probe(struct snd_sof_dev *sdev, int address) { - struct hdac_hda_priv *hda_priv; struct hda_bus *hbus = sof_to_hbus(sdev); struct hda_codec *codec; u32 hda_cmd = (address << 28) | (AC_NODE_ROOT << 20) | @@ -168,27 +165,15 @@ static int hda_codec_probe(struct snd_sof_dev *sdev, int address) dev_dbg(sdev->dev, "HDA codec #%d probed OK: response: %x\n", address, resp); - hda_priv = devm_kzalloc(sdev->dev, sizeof(*hda_priv), GFP_KERNEL); - if (!hda_priv) - return -ENOMEM; - codec = hda_codec_device_init(&hbus->core, address, HDA_DEV_LEGACY); ret = PTR_ERR_OR_ZERO(codec); if (ret < 0) return ret; - hda_priv->codec = codec; - hda_priv->dev_index = address; - dev_set_drvdata(&codec->core.dev, hda_priv); - - if ((resp & 0xFFFF0000) == IDISP_VID_INTEL) { - if (!hbus->core.audio_component) { - dev_dbg(sdev->dev, - "iDisp hw present but no driver\n"); - ret = -ENOENT; - goto out; - } - hda_priv->need_display_power = true; + if ((resp & 0xFFFF0000) == IDISP_VID_INTEL && !hbus->core.audio_component) { + dev_dbg(sdev->dev, "iDisp hw present but no driver\n"); + ret = -ENOENT; + goto out; } if (is_generic_config(hbus)) diff --git a/sound/soc/sof/intel/hda-sdw-bpt.c b/sound/soc/sof/intel/hda-sdw-bpt.c index ba9fbd662b85..9d97d8676251 100644 --- a/sound/soc/sof/intel/hda-sdw-bpt.c +++ b/sound/soc/sof/intel/hda-sdw-bpt.c @@ -456,6 +456,12 @@ int hda_sdw_bpt_close(struct device *dev, int link_id, struct hdac_ext_stream *b int ret; int ret1; + if (bpt_rx_stream && hdac_stream(bpt_rx_stream)->running) + hda_sdw_bpt_dma_disable(dev, bpt_rx_stream); + + if (bpt_tx_stream && hdac_stream(bpt_tx_stream)->running) + hda_sdw_bpt_dma_disable(dev, bpt_tx_stream); + /* * In the case of SoundWire we need to reset the PCMSyCM registers. * Need to continue depreparing the DMA buffers even if this fails. diff --git a/sound/usb/bcd2000/bcd2000.c b/sound/usb/bcd2000/bcd2000.c index 2bd49bf82748..0fd086cac955 100644 --- a/sound/usb/bcd2000/bcd2000.c +++ b/sound/usb/bcd2000/bcd2000.c @@ -183,10 +183,24 @@ static int bcd2000_midi_output_open(struct snd_rawmidi_substream *substream) static int bcd2000_midi_output_close(struct snd_rawmidi_substream *substream) { struct bcd2000 *bcd2k = substream->rmidi->private_data; + struct urb *urb = NULL; - if (bcd2k->midi_out_active && bcd2k->midi_out_urb) { - usb_kill_urb(bcd2k->midi_out_urb); - bcd2k->midi_out_active = 0; + /* + * Reference the URB under the lock so that it stays valid until + * usb_kill_urb() returns, even if a disconnect frees it in the + * meantime. usb_kill_urb() cannot run under the lock, as the + * completion handler takes the same lock. + */ + scoped_guard(spinlock_irqsave, &bcd2k->midi_lock) { + if (bcd2k->midi_out_active && bcd2k->midi_out_urb) { + urb = usb_get_urb(bcd2k->midi_out_urb); + bcd2k->midi_out_active = 0; + } + } + + if (urb) { + usb_kill_urb(urb); + usb_put_urb(urb); } return 0; diff --git a/sound/usb/caiaq/input.c b/sound/usb/caiaq/input.c index 8d924330c54c..309a361ca9f7 100644 --- a/sound/usb/caiaq/input.c +++ b/sound/usb/caiaq/input.c @@ -840,6 +840,8 @@ int snd_usb_caiaq_input_init(struct snd_usb_caiaqdev *cdev) return 0; exit_free_idev: + usb_free_urb(cdev->ep4_in_urb); + cdev->ep4_in_urb = NULL; input_free_device(input); cdev->input_dev = NULL; return ret; diff --git a/sound/usb/line6/capture.c b/sound/usb/line6/capture.c index ceee51eeda0d..32922b7f2ffc 100644 --- a/sound/usb/line6/capture.c +++ b/sound/usb/line6/capture.c @@ -94,6 +94,18 @@ void line6_capture_copy(struct snd_line6_pcm *line6pcm, char *fbuf, int fsize) if (runtime == NULL) return; + /* + * snd_line6_capture_open() constrains the buffer to hold at least + * two packets, so this cannot happen for a conforming device. Drop + * the packet rather than overrun the single wrap below. + */ + if (unlikely(frames > runtime->buffer_size)) { + dev_err_ratelimited(line6pcm->line6->ifcdev, + "packet of %d frames exceeds %lu-frame buffer, dropping\n", + frames, runtime->buffer_size); + return; + } + if (line6pcm->in.pos_done + frames > runtime->buffer_size) { /* The transferred area goes over buffer boundary, @@ -231,6 +243,23 @@ static int snd_line6_capture_open(struct snd_pcm_substream *substream) if (err < 0) return err; + /* + * line6_capture_copy() wraps the ring buffer once per packet and + * advances pos_done by the packet, which only works when the buffer + * holds more than one packet: with exactly one, pos_done gains a + * buffer and loses it again on every packet. The packet size comes + * from the device, so bound the buffer by two packets here. The + * impulse-response path copies a playback URB into this ring, so + * take the larger of the two endpoints. + */ + err = snd_pcm_hw_constraint_minmax(runtime, + SNDRV_PCM_HW_PARAM_BUFFER_BYTES, + 2 * max(line6pcm->max_packet_size_in, + line6pcm->max_packet_size_out), + UINT_MAX); + if (err < 0) + return err; + line6_pcm_acquire(line6pcm, LINE6_STREAM_CAPTURE_HELPER, false); runtime->hw = line6pcm->properties->capture_hw; diff --git a/sound/usb/midi.c b/sound/usb/midi.c index 7e5b1b13360f..1099e5b81411 100644 --- a/sound/usb/midi.c +++ b/sound/usb/midi.c @@ -113,6 +113,7 @@ struct snd_usb_midi { unsigned int opened[2]; unsigned char disconnected; unsigned char input_running; + unsigned char keep_input_running; struct snd_kcontrol *roland_load_ctl; }; @@ -1149,41 +1150,42 @@ static int substream_open(struct snd_rawmidi_substream *substream, int dir, int open) { struct snd_usb_midi *umidi = substream->rmidi->private_data; - struct snd_kcontrol *ctl; + struct snd_ctl_elem_id ctl_id; + bool activate_ctl = false; + bool active; guard(rwsem_read)(&umidi->disc_rwsem); if (umidi->disconnected) return open ? -ENODEV : 0; - guard(mutex)(&umidi->mutex); - if (open) { - if (!umidi->opened[0] && !umidi->opened[1]) { - if (umidi->roland_load_ctl) { - ctl = umidi->roland_load_ctl; - ctl->vd[0].access |= - SNDRV_CTL_ELEM_ACCESS_INACTIVE; - snd_ctl_notify(umidi->card, - SNDRV_CTL_EVENT_MASK_INFO, &ctl->id); - update_roland_altsetting(umidi); + scoped_guard(mutex, &umidi->mutex) { + if (open) { + if (!umidi->opened[0] && !umidi->opened[1]) { + if (umidi->roland_load_ctl) { + ctl_id = umidi->roland_load_ctl->id; + activate_ctl = true; + active = false; + update_roland_altsetting(umidi); + } } - } - umidi->opened[dir]++; - if (umidi->opened[1]) - snd_usbmidi_input_start(&umidi->list); - } else { - umidi->opened[dir]--; - if (!umidi->opened[1]) - snd_usbmidi_input_stop(&umidi->list); - if (!umidi->opened[0] && !umidi->opened[1]) { - if (umidi->roland_load_ctl) { - ctl = umidi->roland_load_ctl; - ctl->vd[0].access &= - ~SNDRV_CTL_ELEM_ACCESS_INACTIVE; - snd_ctl_notify(umidi->card, - SNDRV_CTL_EVENT_MASK_INFO, &ctl->id); + umidi->opened[dir]++; + if (umidi->opened[1]) + snd_usbmidi_input_start(&umidi->list); + } else { + umidi->opened[dir]--; + if (!umidi->opened[1] && !umidi->keep_input_running) + snd_usbmidi_input_stop(&umidi->list); + if (!umidi->opened[0] && !umidi->opened[1]) { + if (umidi->roland_load_ctl) { + ctl_id = umidi->roland_load_ctl->id; + activate_ctl = true; + active = true; + } } } } + if (activate_ctl) + snd_ctl_activate_id(umidi->card, &ctl_id, active); return 0; } @@ -2262,6 +2264,10 @@ static int snd_usbmidi_detect_roland(struct snd_usb_midi *umidi, struct usb_host_interface *hostif; u8 *cs_desc; + /* TD-11 becomes unusable if input isn't serviced continuously */ + if (umidi->usb_id == USB_ID(0x0582, 0x0151)) + umidi->keep_input_running = 1; + intf = umidi->iface; if (!intf) return -ENOENT; @@ -2469,7 +2475,7 @@ void snd_usbmidi_input_start(struct list_head *p) int i; umidi = list_entry(p, struct snd_usb_midi, list); - if (umidi->input_running || !umidi->opened[1]) + if (umidi->input_running || (!umidi->opened[1] && !umidi->keep_input_running)) return; for (i = 0; i < MIDI_MAX_ENDPOINTS; ++i) snd_usbmidi_input_start_ep(umidi, umidi->endpoints[i].in); @@ -2644,6 +2650,10 @@ int __snd_usbmidi_create(struct snd_card *card, if (err < 0) goto exit; + /* start input for devices that need continuous servicing */ + if (umidi->keep_input_running) + snd_usbmidi_input_start(&umidi->list); + usb_autopm_get_interface_no_resume(umidi->iface); list_add_tail(&umidi->list, midi_list); diff --git a/sound/usb/qcom/qc_audio_offload.c b/sound/usb/qcom/qc_audio_offload.c index e4bfd43a2488..5ee568e0454c 100644 --- a/sound/usb/qcom/qc_audio_offload.c +++ b/sound/usb/qcom/qc_audio_offload.c @@ -1969,8 +1969,13 @@ static int qc_usb_audio_probe(struct auxiliary_device *auxdev, QMI_UAUDIO_STREAM_REQ_MSG_V01_MAX_MSG_LEN, &uaudio_svc_ops_options, &uaudio_stream_req_handlers); + if (ret) + goto free_qmi_hdl; + ret = qmi_add_server(svc->uaudio_svc_hdl, UAUDIO_STREAM_SERVICE_ID_V01, UAUDIO_STREAM_SERVICE_VERS_V01, 0); + if (ret) + goto release_qmi_hdl; uaudio_svc = svc; @@ -1987,7 +1992,9 @@ static int qc_usb_audio_probe(struct auxiliary_device *auxdev, release_qmi: qc_usb_audio_cleanup_qmi_dev(); +release_qmi_hdl: qmi_handle_release(svc->uaudio_svc_hdl); +free_qmi_hdl: kfree(svc->uaudio_svc_hdl); free_svc: kfree(svc); diff --git a/sound/usb/usx2y/us122l.c b/sound/usb/usx2y/us122l.c index a5db0d044ef9..9b4ec9bdc138 100644 --- a/sound/usb/usx2y/us122l.c +++ b/sound/usb/usx2y/us122l.c @@ -129,19 +129,26 @@ static const struct vm_operations_struct usb_stream_hwdep_vm_ops = { static int usb_stream_hwdep_open(struct snd_hwdep *hw, struct file *file) { struct us122l *us122l = hw->private_data; - struct usb_interface *iface; + struct usb_interface *iface, *iface0 = NULL; if (hw->used >= 2) return -EBUSY; - if (!us122l->first) - us122l->first = file; - if (us122l->is_us144) { - iface = usb_ifnum_to_if(us122l->dev, 0); - usb_autopm_get_interface(iface); + iface0 = usb_ifnum_to_if(us122l->dev, 0); + if (!iface0) + return -ENODEV; } + iface = usb_ifnum_to_if(us122l->dev, 1); + if (!iface) + return -ENODEV; + + if (!us122l->first) + us122l->first = file; + + if (iface0) + usb_autopm_get_interface(iface0); usb_autopm_get_interface(iface); return 0; } @@ -153,10 +160,12 @@ static int usb_stream_hwdep_release(struct snd_hwdep *hw, struct file *file) if (us122l->is_us144) { iface = usb_ifnum_to_if(us122l->dev, 0); - usb_autopm_put_interface(iface); + if (iface) + usb_autopm_put_interface(iface); } iface = usb_ifnum_to_if(us122l->dev, 1); - usb_autopm_put_interface(iface); + if (iface) + usb_autopm_put_interface(iface); if (us122l->first == file) us122l->first = NULL; guard(mutex)(&us122l->mutex); @@ -179,6 +188,8 @@ static int usb_stream_hwdep_mmap(struct snd_hwdep *hw, offset = area->vm_pgoff << PAGE_SHIFT; guard(mutex)(&us122l->mutex); s = us122l->sk.s; + if (!s) + return -ENODEV; read = offset < s->read_size; if (read) { if (area->vm_flags & VM_WRITE) diff --git a/sound/usb/usx2y/usX2Yhwdep.c b/sound/usb/usx2y/usX2Yhwdep.c index 7f153a2fcdd8..3800c5b5887c 100644 --- a/sound/usb/usx2y/usX2Yhwdep.c +++ b/sound/usb/usx2y/usX2Yhwdep.c @@ -179,6 +179,9 @@ static int snd_usx2y_hwdep_dsp_load(struct snd_hwdep *hw, int lret, err; char *buf; + if (priv->chip_status & USX2Y_STAT_CHIP_HUP) + return -ENODEV; + buf = memdup_user(dsp->image, dsp->length); if (IS_ERR(buf)) return PTR_ERR(buf); @@ -201,11 +204,13 @@ static int snd_usx2y_hwdep_dsp_load(struct snd_hwdep *hw, err = usx2y_in04_init(priv); if (err) { dev_err(&dev->dev, "usx2y_in04_init error\n"); + priv->chip_status |= USX2Y_STAT_CHIP_HUP; return err; } err = usx2y_create_alsa_devices(hw->card); if (err) { dev_err(&dev->dev, "usx2y_create_alsa_devices error %i\n", err); + priv->chip_status |= USX2Y_STAT_CHIP_HUP; return err; } priv->chip_status |= USX2Y_STAT_CHIP_INIT; diff --git a/sound/usb/usx2y/usb_stream.c b/sound/usb/usx2y/usb_stream.c index 3122cf653273..7d621280fba3 100644 --- a/sound/usb/usx2y/usb_stream.c +++ b/sound/usb/usx2y/usb_stream.c @@ -682,6 +682,10 @@ int usb_stream_start(struct usb_stream_kernel *sk) s->sync_packet = -1; s->inpacket_head = -1; sk->iso_frame_balance = 0; + sk->idle_inurb = sk->inurb[USB_STREAM_NURBS - 2]; + sk->idle_outurb = sk->outurb[USB_STREAM_NURBS - 2]; + sk->completed_inurb = sk->inurb[USB_STREAM_NURBS - 1]; + sk->completed_outurb = sk->outurb[USB_STREAM_NURBS - 1]; ++try; for (u = 0; u < 2; u++) { struct urb *inurb = sk->inurb[u]; @@ -739,11 +743,6 @@ int usb_stream_start(struct usb_stream_kernel *sk) return -EFAULT; } - sk->idle_inurb = sk->inurb[USB_STREAM_NURBS - 2]; - sk->idle_outurb = sk->outurb[USB_STREAM_NURBS - 2]; - sk->completed_inurb = sk->inurb[USB_STREAM_NURBS - 1]; - sk->completed_outurb = sk->outurb[USB_STREAM_NURBS - 1]; - /* wait, check */ { int wait_ms = 3000; diff --git a/sound/usb/usx2y/usbusx2yaudio.c b/sound/usb/usx2y/usbusx2yaudio.c index a3a0bc13632b..798c9bb5386a 100644 --- a/sound/usb/usx2y/usbusx2yaudio.c +++ b/sound/usb/usx2y/usbusx2yaudio.c @@ -51,6 +51,18 @@ static int usx2y_urb_capt_retire(struct snd_usx2y_substream *subs) dev_dbg(&usx2y->dev->dev, "%s: 0 == len ERROR!\n", __func__); continue; } + /* + * snd_usx2y_pcm_open() constrains the buffer to hold at least + * two URBs, so this cannot happen for a device that keeps its + * packets at 50 frames. Drop the packet rather than exceed the + * single wrap below. + */ + if (unlikely(len > runtime->buffer_size)) { + dev_err_ratelimited(&usx2y->dev->dev, + "%s: packet of %i frames exceeds %lu-frame buffer, dropping\n", + __func__, len, runtime->buffer_size); + continue; + } /* copy a data chunk */ if ((hwptr_done + len) > runtime->buffer_size) { @@ -106,14 +118,31 @@ static int usx2y_urb_play_prepare(struct snd_usx2y_substream *subs, __func__, counts); return -EPIPE; } + if (counts * usx2y->stride > subs->maxpacksize) { + dev_err_ratelimited(&usx2y->dev->dev, + "%s: packet of %i bytes exceeds maxpacksize %u\n", + __func__, counts * usx2y->stride, + subs->maxpacksize); + return -EPIPE; + } /* set up descriptor */ urb->iso_frame_desc[pack].offset = pack ? urb->iso_frame_desc[pack - 1].offset + urb->iso_frame_desc[pack - 1].length : 0; - urb->iso_frame_desc[pack].length = cap_urb->iso_frame_desc[pack].actual_length; + urb->iso_frame_desc[pack].length = counts * usx2y->stride; } if (atomic_read(&subs->state) >= STATE_PRERUNNING) { + /* + * The copy below wraps once, which needs count <= buffer_size. + * snd_usx2y_pcm_open() guarantees it; guard anyway. + */ + if (unlikely(count > runtime->buffer_size)) { + dev_err_ratelimited(&usx2y->dev->dev, + "%s: URB of %i frames exceeds %lu-frame buffer, stopping\n", + __func__, count, runtime->buffer_size); + return -EPIPE; + } if (subs->hwptr + count > runtime->buffer_size) { /* err, the transferred area goes over buffer boundary. * copy the data to the temp buffer. @@ -888,6 +917,18 @@ static int snd_usx2y_pcm_open(struct snd_pcm_substream *substream) runtime->private_data = subs; subs->pcm_substream = substream; snd_pcm_hw_constraint_minmax(runtime, SNDRV_PCM_HW_PARAM_PERIOD_TIME, 1000, 200000); + /* + * Capture wraps the ring buffer once per packet and playback once + * per URB of nr_of_packs() packets of at most 50 frames (see + * usx2y_urb_play_prepare()). Playback needs the buffer to hold more + * than one URB and capture more than one packet; use the URB bound + * for both, and two URBs rather than one, because the position only + * advances when a URB is less than the whole buffer. The bound is in + * frames and does not depend on the altsetting, which is selected + * in prepare. + */ + snd_pcm_hw_constraint_minmax(runtime, SNDRV_PCM_HW_PARAM_BUFFER_SIZE, + 2 * nr_of_packs() * 50, UINT_MAX); return 0; } diff --git a/tools/perf/tests/tool_pmu.c b/tools/perf/tests/tool_pmu.c index c6c5ebf0e935..667160f9e2cd 100644 --- a/tools/perf/tests/tool_pmu.c +++ b/tools/perf/tests/tool_pmu.c @@ -1,8 +1,16 @@ // SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +#include +#include +#include +#include +#include +#include "cpumap.h" #include "debug.h" #include "evlist.h" +#include "evsel.h" #include "parse-events.h" #include "tests.h" +#include "thread_map.h" #include "tool_pmu.h" static int do_test(enum tool_pmu_event ev, bool with_pmu) @@ -100,9 +108,80 @@ static int test__tool_pmu_with_pmu(struct test_suite *test __maybe_unused, return TEST_OK; } +static int test__tool_pmu_incremental_open_unwind(struct test_suite *test __maybe_unused, + int subtest __maybe_unused) +{ + struct evlist *evlist = evlist__new(); + struct parse_events_error err; + struct perf_cpu_map *cpus = NULL; + struct perf_thread_map *threads = NULL; + struct evsel *evsel; + int ret = TEST_FAIL, fd0 = -1; + + if (!evlist) + return TEST_FAIL; + + parse_events_error__init(&err); + if (parse_events(evlist, "tool/user_time/", &err)) { + parse_events_error__exit(&err); + evlist__put(evlist); + return TEST_FAIL; + } + parse_events_error__exit(&err); + + evsel = evlist__first(evlist); + cpus = perf_cpu_map__new("0,1"); + threads = thread_map__new_by_tid(getpid()); + if (!cpus || !threads) + goto out; + + /* Step 1: Open CPU index 0 successfully */ + if (evsel__open_per_cpu_and_thread(evsel, cpus, 0, threads) < 0) { + pr_debug("Failed to open CPU index 0\n"); + goto out; + } + + fd0 = (*(int *)xyarray__entry(evsel->core.fd, 0, 0)); + if (fd0 < 0 || fcntl(fd0, F_GETFD) < 0) { + pr_debug("CPU index 0 FD is invalid (%d)\n", fd0); + goto out; + } + + /* Step 2: Intentionally trigger failure on CPU index 1 */ + evsel->core.attr.sample_period = 1; /* Not supported for tool PMU -> -EINVAL */ + if (evsel__open_per_cpu_and_thread(evsel, cpus, 1, threads) >= 0) { + pr_debug("Unexpected success opening CPU index 1 with sample_period=1\n"); + goto out; + } + + /* + * Step 3: Check that CPU index 0's FD was NOT destroyed by CPU index 1's unwind. + */ + if ((*(int *)xyarray__entry(evsel->core.fd, 0, 0)) != fd0) { + pr_debug("FAILED: CPU 0 FD overwritten: FD(evsel, 0, 0)=%d, expected %d\n", + (*(int *)xyarray__entry(evsel->core.fd, 0, 0)), fd0); + goto out; + } + + if (fcntl(fd0, F_GETFD) < 0) { + pr_debug("FAILED: CPU 0 FD %d was closed by error unwind! errno=%d (%s)\n", + fd0, errno, strerror(errno)); + goto out; + } + + ret = TEST_OK; +out: + evsel__close(evsel); + perf_cpu_map__put(cpus); + perf_thread_map__put(threads); + evlist__put(evlist); + return ret; +} + static struct test_case tests__tool_pmu[] = { TEST_CASE("Parsing without PMU name", tool_pmu_without_pmu), TEST_CASE("Parsing with PMU name", tool_pmu_with_pmu), + TEST_CASE("Incremental open error unwind boundary", tool_pmu_incremental_open_unwind), { .name = NULL, } }; diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c index d4cb455f4a7d..5db43520f25e 100644 --- a/tools/perf/util/evsel.c +++ b/tools/perf/util/evsel.c @@ -2974,6 +2974,19 @@ static bool evsel__detect_missing_features(struct evsel *evsel, struct perf_cpu return false; } +static void evsel__close_fds(struct evsel *evsel, int start_cpu_map_idx, + int cpu_map_idx, int thread, int nthreads) +{ + do { + while (--thread >= 0) { + if (FD(evsel, cpu_map_idx, thread) >= 0) + close(FD(evsel, cpu_map_idx, thread)); + FD(evsel, cpu_map_idx, thread) = -1; + } + thread = nthreads; + } while (--cpu_map_idx >= start_cpu_map_idx); +} + static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus, struct perf_thread_map *threads, int start_cpu_map_idx, int end_cpu_map_idx) @@ -3109,8 +3122,14 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus, if (err == -EMFILE && rlimit__increase_nofile(&set_rlimit)) goto retry_open; - if (err == -EINVAL && evsel__detect_missing_features(evsel, cpu)) + if (err == -EINVAL && evsel__detect_missing_features(evsel, cpu)) { + /* + * The restart opens the events again, so close what this + * attempt opened before its entries are overwritten. + */ + evsel__close_fds(evsel, start_cpu_map_idx, idx, thread, nthreads); goto fallback_missing_features; + } if (evsel__precise_ip_fallback(evsel)) goto retry_open; @@ -3120,14 +3139,7 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus, threads->err_thread = thread; old_errno = errno; - do { - while (--thread >= 0) { - if (FD(evsel, idx, thread) >= 0) - close(FD(evsel, idx, thread)); - FD(evsel, idx, thread) = -1; - } - thread = nthreads; - } while (--idx >= 0); + evsel__close_fds(evsel, start_cpu_map_idx, idx, thread, nthreads); errno = old_errno; out: if (err) diff --git a/tools/perf/util/hwmon_pmu.c b/tools/perf/util/hwmon_pmu.c index ed544dca70c3..96aeb379e169 100644 --- a/tools/perf/util/hwmon_pmu.c +++ b/tools/perf/util/hwmon_pmu.c @@ -803,7 +803,7 @@ int evsel__hwmon_pmu_open(struct evsel *evsel, FD(evsel, idx, thread) = -1; } thread = nthreads; - } while (--idx >= 0); + } while (--idx >= start_cpu_map_idx); close(dir); return err; } diff --git a/tools/perf/util/tool_pmu.c b/tools/perf/util/tool_pmu.c index 5c30854b4644..d2f7591747d9 100644 --- a/tools/perf/util/tool_pmu.c +++ b/tools/perf/util/tool_pmu.c @@ -349,7 +349,7 @@ int evsel__tool_pmu_open(struct evsel *evsel, FD(evsel, idx, thread) = -1; } thread = nthreads; - } while (--idx >= 0); + } while (--idx >= start_cpu_map_idx); errno = old_errno; return err; } diff --git a/tools/testing/selftests/filesystems/fscontext_ns/fscontext_ns_test.c b/tools/testing/selftests/filesystems/fscontext_ns/fscontext_ns_test.c index 0f30a3e6e197..535bf1f4e1a3 100644 --- a/tools/testing/selftests/filesystems/fscontext_ns/fscontext_ns_test.c +++ b/tools/testing/selftests/filesystems/fscontext_ns/fscontext_ns_test.c @@ -193,7 +193,7 @@ FIXTURE_VARIANT_ADD(fscontext_ns, binfmt_misc) { FIXTURE_VARIANT_ADD(fscontext_ns, overlay) { .fsname = "overlay", - .expected_errno = EIO, + .expected_errno = EINVAL, }; TEST_F(fscontext_ns, create_from_descendant_userns) diff --git a/tools/testing/selftests/net/fib_nexthops.sh b/tools/testing/selftests/net/fib_nexthops.sh index 431d7bed7622..adbec51917eb 100755 --- a/tools/testing/selftests/net/fib_nexthops.sh +++ b/tools/testing/selftests/net/fib_nexthops.sh @@ -425,6 +425,14 @@ stop_ip_monitor() return $rc } +fdb_entry_aged() +{ + local mac=$1 + local dev=$2 + + ! $BRIDGE fdb get to $mac dev $dev self &> /dev/null +} + check_nexthop_fdb_support() { $IP nexthop help 2>&1 | grep -q fdb @@ -538,14 +546,18 @@ ipv6_fdb_grp_fcnal() log_test $? 0 "Fdb mac add with nexthop group and static state" run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic" - log_test $? 255 "Fdb mac add with nexthop group and dynamic state" + log_test $? 0 "Fdb mac add with nexthop group and dynamic state" + + run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self dynamic extern_learn" + log_test $? 0 "Fdb mac add with nexthop group and extern_learn" - run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self" - run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic" - log_test $? 255 "Fdb mac replace with nexthop group and dynamic state" + run_cmd "$IP link set dev vx10 up type vxlan ageing 1" + run_cmd "$BRIDGE fdb add 02:02:00:00:00:18 dev vx10 dst 2001:db8:91::11 self dynamic" + slowwait 20 fdb_entry_aged 02:02:00:00:00:18 vx10 + log_test $? 0 "Fdb mac without nexthop group aged out" - run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic" - log_test $? 255 "Fdb mac append with nexthop group and dynamic state" + run_cmd "$BRIDGE fdb get to 02:02:00:00:00:16 dev vx10 self" + log_test $? 0 "Fdb mac with nexthop group not aged out" run_cmd "$IP -6 ro add 2001:db8:101::1/128 nhid 66" log_test $? 2 "Route add with fdb nexthop" @@ -688,14 +700,18 @@ ipv4_fdb_grp_fcnal() log_test $? 0 "Fdb mac add with nexthop group and static state" run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic" - log_test $? 255 "Fdb mac add with nexthop group and dynamic state" + log_test $? 0 "Fdb mac add with nexthop group and dynamic state" + + run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self dynamic extern_learn" + log_test $? 0 "Fdb mac add with nexthop group and extern_learn" - run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self" - run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic" - log_test $? 255 "Fdb mac replace with nexthop group and dynamic state" + run_cmd "$IP link set dev vx10 up type vxlan ageing 1" + run_cmd "$BRIDGE fdb add 02:02:00:00:00:18 dev vx10 dst 10.0.0.3 self dynamic" + slowwait 20 fdb_entry_aged 02:02:00:00:00:18 vx10 + log_test $? 0 "Fdb mac without nexthop group aged out" - run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic" - log_test $? 255 "Fdb mac append with nexthop group and dynamic state" + run_cmd "$BRIDGE fdb get to 02:02:00:00:00:16 dev vx10 self" + log_test $? 0 "Fdb mac with nexthop group not aged out" run_cmd "$IP ro add 172.16.0.0/22 nhid 16" log_test $? 2 "Route add with fdb nexthop"