From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org, akpm@linux-foundation.org,
torvalds@linux-foundation.org, stable@vger.kernel.org
Cc: lwn@lwn.net, jslaby@suse.cz,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Subject: Re: Linux 7.3.5
Date: Tue, 17 Nov 2026 11:45:30 +0100 [thread overview]
Message-ID: <2026111729-armory-overjoyed-70fd@gregkh> (raw)
In-Reply-To: <2026111729-brim-nest-fa62@gregkh>
diff --git a/Documentation/ABI/testing/sysfs-fs-f2fs b/Documentation/ABI/testing/sysfs-fs-f2fs
index 0cebc89799dd..4f3915948176 100644
--- a/Documentation/ABI/testing/sysfs-fs-f2fs
+++ b/Documentation/ABI/testing/sysfs-fs-f2fs
@@ -271,7 +271,7 @@ Description: Shows all enabled kernel features.
inode_crtime, lost_found, verity, sb_checksum,
casefold, readonly, compression, test_dummy_encryption_v2,
atomic_write, pin_file, encrypted_casefold, linear_lookup,
- fserror.
+ fserror, resizable_tail.
What: /sys/fs/f2fs/<disk>/inject_rate
Date: May 2016
diff --git a/Documentation/admin-guide/pm/intel_idle.rst b/Documentation/admin-guide/pm/intel_idle.rst
index 996e5bd83c42..d1a6251790b8 100644
--- a/Documentation/admin-guide/pm/intel_idle.rst
+++ b/Documentation/admin-guide/pm/intel_idle.rst
@@ -145,11 +145,13 @@ for the other idle state types (C2 and C3) the target residency value is 3 times
the exit latency (again, that is because it reflects the target residency to
exit latency ratio in the majority of cases for the processor models recognized
by ``intel_idle``). The name of the idle state (to be returned by the ``name``
-idle state attribute in ``sysfs``) is either "Cx_LPI" (if it comes from ``_LPI``
-processing) or "Cx_ACPI", where x is the index of that idle state in the final
-list (note that the minimum value of x is 1, because 0 is reserved for the
-"polling" state), and its target residency is based on the exit latency value.
-All of the idle states in the final list are enabled by default in this case.
+idle state attribute in ``sysfs``) either is equal to the state name value that
+comes from ``_LPI`` processing (with the possible fallback to "Cx_LPI" if the
+state name is not provided) or it is "Cx_ACPI", where x is the index of that
+idle state in the final list (note that the minimum value of x is 1, because 0
+is reserved for the "polling" state), and its target residency is based on the
+exit latency value. All of the idle states in the final list are enabled by
+default in this case.
.. _intel-idle-initialization:
diff --git a/Documentation/arch/arm64/silicon-errata.rst b/Documentation/arch/arm64/silicon-errata.rst
index 99a1eb4b833d..95faec78b7f7 100644
--- a/Documentation/arch/arm64/silicon-errata.rst
+++ b/Documentation/arch/arm64/silicon-errata.rst
@@ -273,7 +273,8 @@ stable kernels.
+----------------+-----------------+-----------------+-----------------------------+
| ARM | MMU L1 | #3878312 | N/A |
+----------------+-----------------+-----------------+-----------------------------+
-| ARM | MMU S3 | #3995052 | N/A |
+| ARM | MMU S3 | #3995052, | N/A |
+| | | #3673557 | |
+----------------+-----------------+-----------------+-----------------------------+
| ARM | GIC-700 | #2941627 | ARM64_ERRATUM_2941627 |
+----------------+-----------------+-----------------+-----------------------------+
diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst
index 16c35b491dad..4c241d220bc2 100644
--- a/Documentation/filesystems/overlayfs.rst
+++ b/Documentation/filesystems/overlayfs.rst
@@ -731,7 +731,8 @@ When decoding an overlay file handle, the following steps are followed:
1. Find underlying layer by UUID and path type information.
2. Decode the underlying filesystem file handle to underlying dentry.
- 3. For a lower file handle, lookup the handle in index directory by name.
+ 3. For a lower file handle, encode the decoded underlying inode again and
+ lookup that handle in index directory by name.
4. If a whiteout is found in index, return ESTALE. This represents an
overlay object that was deleted after its file handle was encoded.
5. For a non-directory, instantiate a disconnected overlay dentry from the
diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index e4b66af55ffb..6b641c3cee64 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -351,12 +351,12 @@ with the following files:
"mbm_assign_mode":
The supported counter assignment modes. The enclosed brackets indicate which mode
is enabled. The MBM events associated with counters may reset when "mbm_assign_mode"
- is changed.
+ is changed. See architecture specific notes below.
::
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
- [mbm_event]
- default
+ [default]
+ mbm_event
"mbm_event":
@@ -374,20 +374,17 @@ with the following files:
Moving to mbm_event counter assignment mode requires users to assign the counters
to the events. Otherwise, the MBM event counters will return 'Unassigned' when read.
- The mode is beneficial for AMD platforms that support more CTRL_MON
- and MON groups than available hardware counters. By default, this
- feature is enabled on AMD platforms with the ABMC (Assignable Bandwidth
- Monitoring Counters) capability, ensuring counters remain assigned even
- when the corresponding RMID is not actively used by any processor.
+ It is intended for deployments that need to manage counter assignment on
+ platforms where the number of monitoring groups exceeds the available
+ hardware counters. Hardware counters are scarce on these platforms, so
+ mbm_event mode suits workflows that monitor a subset of groups at a time and
+ rotate assignments as needed. The mbm_event mode ensures counters remain
+ assigned even when the corresponding RMID is not actively used.
"default":
In default mode, resctrl assumes there is a hardware counter for each
- event within every CTRL_MON and MON group. On AMD platforms, it is
- recommended to use the mbm_event mode, if supported, to prevent reset of MBM
- events between reads resulting from hardware re-allocating counters. This can
- result in misleading values or display "Unavailable" if no counter is assigned
- to the event.
+ event within every CTRL_MON and MON group.
* To enable "mbm_event" counter assignment mode:
::
@@ -399,6 +396,24 @@ with the following files:
# echo "default" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
+ In default mode on AMD platforms that support more CTRL_MON and MON groups
+ than hardware counters, hardware dynamically shares a smaller pool of
+ counters among RMIDs. One counter from this pool is used per monitoring
+ group and counts every MBM event of that group's RMID. The size of the pool
+ is not enumerated to software (unlike "num_mbm_cntrs"), and "num_rmids" may
+ be much larger. For example, 64 counters in this pool monitor 64 groups.
+ Newer hardware may provide a larger pool.
+
+ On such platforms, while the number of monitoring groups does not exceed
+ that pool, a counter stays attached to each RMID and readings remain
+ accurate. Creating more groups than the pool can cause hardware to
+ re-allocate those counters between successive reads of an event. Bandwidth
+ values may then be misleading, or a read may return "Unavailable" if no
+ counter is allocated to the RMID. There is no user-visible indication when
+ this begins. Users who need stable readings beyond that pool should switch
+ to mbm_event mode, if supported, and assign counters to the groups of
+ interest (rotating assignments as needed).
+
"num_mbm_cntrs":
The maximum number of counters (total of available and assigned counters) in
each domain when the system supports mbm_event mode.
@@ -1783,17 +1798,19 @@ View the llc occupancy snapshot::
Examples on working with mbm_assign_mode
========================================
-a. Check if MBM counter assignment mode is supported.
+a. Check if MBM counter assignment mode is supported. Enable "mbm_event"
+mode. Switching the mbm_assign_mode may reset all the MBM counters (and
+thus all MBM events) of all the resctrl groups.
::
- # mount -t resctrl resctrl /sys/fs/resctrl/
-
+ # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
+ [default]
+ mbm_event
+ # echo "mbm_event" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
[mbm_event]
default
-The "mbm_event" mode is detected and enabled.
-
b. Check how many assignable counters are supported.
::
@@ -1804,36 +1821,39 @@ c. Check how many assignable counters are available for assignment in each domai
::
# cat /sys/fs/resctrl/info/L3_MON/available_mbm_cntrs
- 0=30;1=30
+ 0=32;1=32
-d. To list the default group's assign states.
+d. To list the default group's assign states. Counters are unassigned when the mode
+is switched to "mbm_event".
::
# cat /sys/fs/resctrl/mbm_L3_assignments
- mbm_total_bytes:0=e;1=e
- mbm_local_bytes:0=e;1=e
+ mbm_total_bytes:0=_;1=_
+ mbm_local_bytes:0=_;1=_
-e. To unassign the counter associated with the mbm_total_bytes event on domain 0.
+e. To assign the counter associated with the mbm_total_bytes event on domain 0
+in exclusive mode.
::
- # echo "mbm_total_bytes:0=_" > /sys/fs/resctrl/mbm_L3_assignments
+ # echo "mbm_total_bytes:0=e" > /sys/fs/resctrl/mbm_L3_assignments
# cat /sys/fs/resctrl/mbm_L3_assignments
- mbm_total_bytes:0=_;1=e
- mbm_local_bytes:0=e;1=e
+ mbm_total_bytes:0=e;1=_
+ mbm_local_bytes:0=_;1=_
f. To unassign the counter associated with the mbm_total_bytes event on all domains.
::
# echo "mbm_total_bytes:*=_" > /sys/fs/resctrl/mbm_L3_assignments
- # cat /sys/fs/resctrl/mbm_L3_assignment
+ # cat /sys/fs/resctrl/mbm_L3_assignments
mbm_total_bytes:0=_;1=_
- mbm_local_bytes:0=e;1=e
+ mbm_local_bytes:0=_;1=_
-g. To assign a counter associated with the mbm_total_bytes event on all domains in
-exclusive mode.
+g. To assign a counter associated with the mbm_total_bytes and mbm_local_bytes
+events on all domains in exclusive mode.
::
# echo "mbm_total_bytes:*=e" > /sys/fs/resctrl/mbm_L3_assignments
+ # echo "mbm_local_bytes:*=e" > /sys/fs/resctrl/mbm_L3_assignments
# cat /sys/fs/resctrl/mbm_L3_assignments
mbm_total_bytes:0=e;1=e
mbm_local_bytes:0=e;1=e
@@ -1890,13 +1910,8 @@ the MBM counters (and thus all MBM events) of all the resctrl groups.
# echo "default" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
- mbm_event
[default]
-
-m. Unmount the resctrl filesystem.
-::
-
- # umount /sys/fs/resctrl/
+ mbm_event
Intel RDT Errata
================
diff --git a/Documentation/hwmon/asus_rog_ryujin.rst b/Documentation/hwmon/asus_rog_ryujin.rst
index b0d7ce8dd921..24dccef2d2df 100644
--- a/Documentation/hwmon/asus_rog_ryujin.rst
+++ b/Documentation/hwmon/asus_rog_ryujin.rst
@@ -33,6 +33,12 @@ Usage notes
As these are USB HIDs, the driver can be loaded automatically by the kernel and
supports hot swapping.
+The pump of the RYUJIN III also has a 4-pin cable for a motherboard fan header.
+The cooler either hands the pump to its internal controller, which follows the
+PWM of the header its 4-pin cable is plugged into, or runs it at the pump duty
+set over USB. Writing pwm1 selects the duty set over USB. Writing pwm2 sets
+only the internal fan duty and leaves the pump control as it is.
+
Sysfs entries
-------------
diff --git a/Documentation/mm/vmemmap_dedup.rst b/Documentation/mm/vmemmap_dedup.rst
index 9fa8642ded48..66d58fbea5b1 100644
--- a/Documentation/mm/vmemmap_dedup.rst
+++ b/Documentation/mm/vmemmap_dedup.rst
@@ -131,8 +131,19 @@ pages of ``struct page`` (page 1 to page 7) are tail pages.
The optimization is only applied when the size of the struct page is a power
of 2. In this case, all tail pages of the same order are identical. See
compound_head(). This allows us to remap the tail pages of the vmemmap to a
-shared, read-only page. The head page is also remapped to a new page. This
-allows the original vmemmap pages to be freed.
+shared page. The head page is also remapped to a new page. This allows the
+original vmemmap pages to be freed.
+
+The shared page is mapped writable. A lockless get of a page reference can
+read a non-zero refcount before the remap and attempt its atomic update
+after it. The refcount of a tail page is zero, so the update fails without
+changing the page. Nothing else may write to the shared page: it is shared
+by all HugeTLB pages of the same size in the zone, and a stray write would
+corrupt all of them.
+
+HugeTLB pages whose vmemmap is optimized when the memmap is first set up at
+boot keep a read-only mapping of the shared page. Nothing can do a lockless
+get on them before they are handed to HugeTLB.
Here is how things look after remapping::
diff --git a/Makefile b/Makefile
index 05c5b682ad50..51e278ccea9c 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
VERSION = 7
PATCHLEVEL = 3
-SUBLEVEL = 4
+SUBLEVEL = 5
EXTRAVERSION =
NAME = Baby Opossum Posse
diff --git a/arch/arm64/include/asm/sdei.h b/arch/arm64/include/asm/sdei.h
index b2248bd3cb58..a78a5337ff85 100644
--- a/arch/arm64/include/asm/sdei.h
+++ b/arch/arm64/include/asm/sdei.h
@@ -44,7 +44,8 @@ asmlinkage unsigned long __sdei_handler(struct pt_regs *regs,
struct sdei_registered_event *arg);
unsigned long do_sdei_event(struct pt_regs *regs,
- struct sdei_registered_event *arg);
+ struct sdei_registered_event *arg,
+ unsigned long vbar);
unsigned long sdei_arch_get_entry_point(int conduit);
#define sdei_arch_get_entry_point(x) sdei_arch_get_entry_point(x)
diff --git a/arch/arm64/include/asm/suspend.h b/arch/arm64/include/asm/suspend.h
index e9ce68d50ba4..a5abf0615594 100644
--- a/arch/arm64/include/asm/suspend.h
+++ b/arch/arm64/include/asm/suspend.h
@@ -2,7 +2,7 @@
#ifndef __ASM_SUSPEND_H
#define __ASM_SUSPEND_H
-#define NR_CTX_REGS 14
+#define NR_CTX_REGS 15
#define NR_CALLEE_SAVED_REGS 12
/*
diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
index 72c03ccea59f..92b2fe0b2784 100644
--- a/arch/arm64/kernel/entry-common.c
+++ b/arch/arm64/kernel/entry-common.c
@@ -30,6 +30,7 @@
#include <asm/stacktrace.h>
#include <asm/sysreg.h>
#include <asm/system_misc.h>
+#include <asm/vectors.h>
/*
* Handle IRQ/context state management when entering from kernel mode.
@@ -1002,6 +1003,7 @@ asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs)
asmlinkage noinstr unsigned long
__sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
{
+ unsigned long elr, spsr, esr, far, vbar;
irqentry_state_t state;
unsigned long ret;
@@ -1027,10 +1029,46 @@ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
else if (cpu_has_pan())
set_pstate_pan(0);
+ /*
+ * Nothing saved the interrupted context's exception registers on the
+ * way in, and it may be in the middle of exception entry or return
+ * (SDEI ignores DAIF). Save them before anything can take an exception
+ * and overwrite them.
+ */
+ elr = read_sysreg(elr_el1);
+ spsr = read_sysreg(spsr_el1);
+ esr = read_sysreg(esr_el1);
+ far = read_sysreg(far_el1);
+
+ /*
+ * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to
+ * EL0 until the next exception from EL0 maps the kernel. They only
+ * handle exceptions from EL0: an exception taken here would run their
+ * empty current-EL half and never come back. Switch to the kernel
+ * vectors. The IRQ-like resume keeps using the interrupted VBAR, so it
+ * still goes through the trampoline.
+ */
+ vbar = read_sysreg(vbar_el1);
+ if (arm64_kernel_unmapped_at_el0() &&
+ vbar == (unsigned long)raw_cpu_read(this_cpu_vector)) {
+ write_sysreg((unsigned long)vectors, vbar_el1);
+ isb();
+ }
+
state = irqentry_nmi_enter(regs);
- ret = do_sdei_event(regs, arg);
+ ret = do_sdei_event(regs, arg, vbar);
irqentry_nmi_exit(regs, state);
+ /*
+ * Put them back unconditionally. An unchanged ELR does not rule out a
+ * nested exception: one may have returned to the same address.
+ */
+ write_sysreg(elr, elr_el1);
+ write_sysreg(spsr, spsr_el1);
+ write_sysreg(esr, esr_el1);
+ write_sysreg(far, far_el1);
+ write_sysreg(vbar, vbar_el1);
+
return ret;
}
#endif /* CONFIG_ARM_SDE_INTERFACE */
diff --git a/arch/arm64/kernel/machine_kexec_file.c b/arch/arm64/kernel/machine_kexec_file.c
index 854d872dfd0f..14f453dd0946 100644
--- a/arch/arm64/kernel/machine_kexec_file.c
+++ b/arch/arm64/kernel/machine_kexec_file.c
@@ -112,6 +112,8 @@ int load_other_segments(struct kimage *image,
vfree(headers);
goto out_err;
}
+
+ vfree(image->elf_headers);
image->elf_headers = headers;
image->elf_load_addr = kbuf.mem;
image->elf_headers_sz = headers_sz;
diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c
index 778f2a1faac8..1b902da8ba43 100644
--- a/arch/arm64/kernel/sdei.c
+++ b/arch/arm64/kernel/sdei.c
@@ -195,21 +195,49 @@ unsigned long sdei_arch_get_entry_point(int conduit)
return 0;
}
+/*
+ * Exceptions an SDEI handler may take without harm once __sdei_handler() has
+ * restored the interrupted context's exception registers. Their handlers are
+ * safe in NMI context:
+ * - EL1 data aborts on translation, permission and access-flag faults:
+ * faulthandler_disabled() is true in NMI context, so do_page_fault() never
+ * takes the mmap lock. This covers the _nofault() accessors used by the
+ * generic dump_stack()/show_regs() code (e.g. print_worker_info()).
+ * - BRK: kprobes, WARN/BUG, kCFI and UBSAN traps, all NMI-capable.
+ * Anything else taken at EL1 is fatal on its own, or unexpected here.
+ */
+static bool sdei_nested_exception_ok(unsigned long esr)
+{
+ switch (ESR_ELx_EC(esr)) {
+ case ESR_ELx_EC_DABT_CUR:
+ return esr_fsc_is_translation_fault(esr) ||
+ esr_fsc_is_permission_fault(esr) ||
+ esr_fsc_is_access_flag_fault(esr);
+ case ESR_ELx_EC_BRK64:
+ return true;
+ default:
+ return false;
+ }
+}
+
/*
* do_sdei_event() returns one of:
* SDEI_EV_HANDLED - success, return to the interrupted context.
* SDEI_EV_FAILED - failure, return this error code to firmware.
* virtual-address - success, return to this address.
+ *
+ * vbar is the interrupted context's VBAR_EL1, for the IRQ-like resume.
*/
unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
- struct sdei_registered_event *arg)
+ struct sdei_registered_event *arg,
+ unsigned long vbar)
{
u32 mode;
int i, err = 0;
int clobbered_registers = 4;
u64 elr = read_sysreg(elr_el1);
+ u64 nested_elr, nested_esr, nested_far;
u32 kernel_mode = read_sysreg(CurrentEL) | 1; /* +SPSel */
- unsigned long vbar = read_sysreg(vbar_el1);
if (arm64_kernel_unmapped_at_el0())
clobbered_registers++;
@@ -221,18 +249,27 @@ unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
}
err = sdei_event_handler(regs, arg);
- if (err)
- return SDEI_EV_FAILED;
- if (elr != read_sysreg(elr_el1)) {
- /*
- * We took a synchronous exception from the SDEI handler.
- * This could deadlock, and if you interrupt KVM it will
- * hyp-panic instead.
- */
- pr_warn("unsafe: exception during handler\n");
+ /*
+ * A synchronous exception taken by the handler has overwritten the
+ * exception registers; __sdei_handler() puts them back. Only the last
+ * one's syndrome survives; report it if it is not one known to be
+ * harmless.
+ *
+ * If the event interrupted a KVM world switch, VBAR points at the KVM
+ * hyp vectors and any such exception is a hyp panic instead.
+ */
+ nested_elr = read_sysreg(elr_el1);
+ nested_esr = read_sysreg(esr_el1);
+ nested_far = read_sysreg(far_el1);
+ if (nested_elr != elr && !sdei_nested_exception_ok(nested_esr)) {
+ pr_warn_once("unexpected exception during handler: ESR %#llx FAR %#llx near %pS\n",
+ nested_esr, nested_far, (void *)nested_elr);
}
+ if (err)
+ return SDEI_EV_FAILED;
+
mode = regs->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK);
/*
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index b8baba281166..44305b997a02 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -470,6 +470,12 @@ static void amu_read_core_const_ctrs(void *val)
cpu_read_corecnt(&ctrs->corecnt);
}
+static bool cpc_ffh_reg_valid(const struct cpc_reg *reg)
+{
+ return reg->bit_width && reg->bit_width <= 64 &&
+ reg->bit_offset <= 64 - reg->bit_width;
+}
+
static u64 cpc_ffh_extract_bits(const struct cpc_reg *reg, u64 val)
{
val &= GENMASK_ULL(reg->bit_offset + reg->bit_width - 1,
@@ -506,7 +512,8 @@ int cpc_read_ffh_fb_ctrs(int cpu, struct cpc_reg *reg1, u64 *val1,
struct amu_ffh_ctrs ctrs;
int ret;
- if (!is_amu_ctr_reg(reg1) || !is_amu_ctr_reg(reg2))
+ if (!is_amu_ctr_reg(reg1) || !is_amu_ctr_reg(reg2) ||
+ !cpc_ffh_reg_valid(reg1) || !cpc_ffh_reg_valid(reg2))
return -EINVAL;
ret = counters_read_on_cpu(cpu, amu_read_core_const_ctrs, &ctrs);
@@ -530,6 +537,9 @@ int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val)
{
int ret = -EOPNOTSUPP;
+ if (!cpc_ffh_reg_valid(reg))
+ return -EINVAL;
+
switch ((u64)reg->address) {
case CPC_FFH_CTR_CORE:
ret = counters_read_on_cpu(cpu, cpu_read_corecnt, val);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index bf6d6526639f..cae133805c76 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1362,21 +1362,32 @@ static void kvm_send_hwpoison_signal(unsigned long address, short lsb)
send_sig_mceerr(BUS_MCEERR_AR, (void __user *)address, lsb, current);
}
-static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot,
- unsigned long hva,
+struct kvm_s2_fault_desc {
+ struct kvm_vcpu *vcpu;
+ phys_addr_t fault_ipa;
+ struct kvm_s2_trans *nested;
+ struct kvm_memory_slot *memslot;
+ unsigned long hva;
+};
+
+static bool fault_supports_stage2_huge_mapping(const struct kvm_s2_fault_desc *s2fd,
unsigned long map_size)
{
- gpa_t gpa_start;
+ struct kvm_memory_slot *memslot = s2fd->memslot;
+ unsigned long hva = s2fd->hva;
hva_t uaddr_start, uaddr_end;
+ gpa_t gpa_start;
size_t size;
/* The memslot and the VMA are guaranteed to be aligned to PAGE_SIZE */
if (map_size == PAGE_SIZE)
return true;
- /* pKVM only supports PMD_SIZE huge-mappings */
- if (is_protected_kvm_enabled() && map_size != PMD_SIZE)
- return false;
+ /* pKVM only supports PMD_SIZE huge-mappings for non-protected VMs */
+ if (is_protected_kvm_enabled()) {
+ if (vcpu_is_protected(s2fd->vcpu) || map_size != PMD_SIZE)
+ return false;
+ }
size = memslot->npages * PAGE_SIZE;
@@ -1436,9 +1447,9 @@ static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot,
* Returns the size of the mapping.
*/
static long
-transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot,
- unsigned long hva, kvm_pfn_t *pfnp, gfn_t *gfnp)
+transparent_hugepage_adjust(const struct kvm_s2_fault_desc *s2fd, kvm_pfn_t *pfnp, gfn_t *gfnp)
{
+ struct kvm *kvm = s2fd->vcpu->kvm;
kvm_pfn_t pfn = *pfnp;
gfn_t gfn = *gfnp;
@@ -1447,8 +1458,8 @@ transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot,
* sure that the HVA and IPA are sufficiently aligned and that the
* block map is contained within the memslot.
*/
- if (fault_supports_stage2_huge_mapping(memslot, hva, PMD_SIZE)) {
- int sz = get_user_mapping_size(kvm, hva);
+ if (fault_supports_stage2_huge_mapping(s2fd, PMD_SIZE)) {
+ int sz = get_user_mapping_size(kvm, s2fd->hva);
if (sz < 0)
return sz;
@@ -1585,14 +1596,27 @@ static enum kvm_pgtable_prot adjust_nested_exec_perms(struct kvm *kvm,
return prot;
}
-struct kvm_s2_fault_desc {
- struct kvm_vcpu *vcpu;
- phys_addr_t fault_ipa;
- struct kvm_s2_trans *nested;
- struct kvm_memory_slot *memslot;
- unsigned long hva;
+struct kvm_s2_fault_vma_info {
+ unsigned long mmu_seq;
+ long vma_pagesize;
+ vm_flags_t vm_flags;
+ unsigned long max_map_size;
+ struct page *page;
+ kvm_pfn_t pfn;
+ gfn_t gfn;
+ bool device;
+ bool mte_allowed;
+ bool is_vma_cacheable;
+ bool map_writable;
+ bool map_non_cacheable;
};
+static int kvm_s2_fault_get_vma_info(const struct kvm_s2_fault_desc *s2fd,
+ struct kvm_s2_fault_vma_info *s2vi);
+
+static gfn_t get_canonical_gfn(const struct kvm_s2_fault_desc *s2fd,
+ const struct kvm_s2_fault_vma_info *s2vi);
+
static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
{
bool write_fault, exec_fault;
@@ -1600,12 +1624,10 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_SHARED;
enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
struct kvm_pgtable *pgt = s2fd->vcpu->arch.hw_mmu->pgt;
- unsigned long mmu_seq;
- struct page *page;
+ struct kvm_s2_fault_vma_info s2vi = {};
struct kvm *kvm = s2fd->vcpu->kvm;
void *memcache = NULL;
- kvm_pfn_t pfn;
- gfn_t gfn;
+ gfn_t canonical_gfn;
int ret;
if (!perm_fault) {
@@ -1615,24 +1637,23 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
return ret;
}
- if (s2fd->nested)
- gfn = kvm_s2_trans_output(s2fd->nested) >> PAGE_SHIFT;
- else
- gfn = s2fd->fault_ipa >> PAGE_SHIFT;
+ s2vi.vma_pagesize = PAGE_SIZE;
+ s2vi.gfn = ALIGN_DOWN(s2fd->fault_ipa, s2vi.vma_pagesize) >> PAGE_SHIFT;
+ canonical_gfn = get_canonical_gfn(s2fd, &s2vi);
write_fault = kvm_is_write_fault(s2fd->vcpu);
exec_fault = kvm_vcpu_trap_is_exec_fault(s2fd->vcpu);
VM_WARN_ON_ONCE(write_fault && exec_fault);
- mmu_seq = kvm->mmu_invalidate_seq;
+ s2vi.mmu_seq = kvm->mmu_invalidate_seq;
/* Pairs with the smp_wmb() in kvm_mmu_invalidate_end(). */
smp_rmb();
- ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, gfn, &pfn, &page, NULL);
+ ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, canonical_gfn, &s2vi.pfn, &s2vi.page, NULL);
if (ret) {
- kvm_prepare_memory_fault_exit(s2fd->vcpu, s2fd->fault_ipa, PAGE_SIZE,
- write_fault, exec_fault, false);
+ kvm_prepare_memory_fault_exit(s2fd->vcpu, gfn_to_gpa(canonical_gfn),
+ s2vi.vma_pagesize, write_fault, exec_fault, false);
return ret;
}
@@ -1649,7 +1670,7 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
prot = adjust_nested_exec_perms(kvm, s2fd->nested, prot);
kvm_fault_lock(kvm);
- if (mmu_invalidate_retry(kvm, mmu_seq)) {
+ if (mmu_invalidate_retry(kvm, s2vi.mmu_seq)) {
ret = -EAGAIN;
goto out_unlock;
}
@@ -1660,48 +1681,34 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
* PTE, which will be preserved.
*/
prot &= ~KVM_NV_GUEST_MAP_SZ;
- ret = KVM_PGT_FN(kvm_pgtable_stage2_relax_perms)(pgt, s2fd->fault_ipa,
+ ret = KVM_PGT_FN(kvm_pgtable_stage2_relax_perms)(pgt, gfn_to_gpa(s2vi.gfn),
prot, flags);
} else {
- ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, s2fd->fault_ipa, PAGE_SIZE,
- __pfn_to_phys(pfn), prot,
+ ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, gfn_to_gpa(s2vi.gfn),
+ s2vi.vma_pagesize,
+ __pfn_to_phys(s2vi.pfn), prot,
memcache, flags);
}
out_unlock:
- kvm_release_faultin_page(kvm, page, !!ret, prot & KVM_PGTABLE_PROT_W);
+ kvm_release_faultin_page(kvm, s2vi.page, !!ret, prot & KVM_PGTABLE_PROT_W);
kvm_fault_unlock(kvm);
if ((prot & KVM_PGTABLE_PROT_W) && !ret)
- mark_page_dirty_in_slot(kvm, s2fd->memslot, gfn);
+ mark_page_dirty_in_slot(kvm, s2fd->memslot, canonical_gfn);
return ret != -EAGAIN ? ret : 0;
}
-struct kvm_s2_fault_vma_info {
- unsigned long mmu_seq;
- long vma_pagesize;
- vm_flags_t vm_flags;
- unsigned long max_map_size;
- struct page *page;
- kvm_pfn_t pfn;
- gfn_t gfn;
- bool device;
- bool mte_allowed;
- bool is_vma_cacheable;
- bool map_writable;
- bool map_non_cacheable;
-};
-
static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd)
{
unsigned int flags = FOLL_HWPOISON | FOLL_LONGTERM | FOLL_WRITE;
+ struct kvm_s2_fault_vma_info s2vi = {};
struct kvm_vcpu *vcpu = s2fd->vcpu;
struct kvm_pgtable *pgt = vcpu->arch.hw_mmu->pgt;
struct mm_struct *mm = current->mm;
struct kvm *kvm = vcpu->kvm;
void *hyp_memcache;
- struct page *page;
int ret;
hyp_memcache = get_mmu_memcache(vcpu);
@@ -1709,12 +1716,16 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd)
if (ret)
return -ENOMEM;
+ ret = kvm_s2_fault_get_vma_info(s2fd, &s2vi);
+ if (ret)
+ return ret;
+
ret = account_locked_vm(mm, 1, true);
if (ret)
return ret;
mmap_read_lock(mm);
- ret = pin_user_pages(s2fd->hva, 1, flags, &page);
+ ret = pin_user_pages(s2fd->hva, 1, flags, &s2vi.page);
mmap_read_unlock(mm);
if (ret == -EHWPOISON) {
@@ -1724,7 +1735,7 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd)
} else if (ret != 1) {
ret = -EFAULT;
goto dec_account;
- } else if (!folio_test_swapbacked(page_folio(page))) {
+ } else if (!folio_test_swapbacked(page_folio(s2vi.page))) {
/*
* We really can't deal with page-cache pages returned by GUP
* because (a) we may trigger writeback of a page for which we
@@ -1744,8 +1755,8 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd)
}
write_lock(&kvm->mmu_lock);
- ret = pkvm_pgtable_stage2_map(pgt, s2fd->fault_ipa, PAGE_SIZE,
- page_to_phys(page), KVM_PGTABLE_PROT_RWX,
+ ret = pkvm_pgtable_stage2_map(pgt, gfn_to_gpa(s2vi.gfn), PAGE_SIZE,
+ page_to_phys(s2vi.page), KVM_PGTABLE_PROT_RWX,
hyp_memcache, 0);
write_unlock(&kvm->mmu_lock);
if (ret) {
@@ -1756,7 +1767,7 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd)
return 0;
unpin:
- unpin_user_pages(&page, 1);
+ unpin_user_page(s2vi.page);
dec_account:
account_locked_vm(mm, 1, false);
return ret;
@@ -1779,7 +1790,7 @@ static short kvm_s2_resolve_vma_size(const struct kvm_s2_fault_desc *s2fd,
switch (vma_shift) {
#ifndef __PAGETABLE_PMD_FOLDED
case PUD_SHIFT:
- if (fault_supports_stage2_huge_mapping(s2fd->memslot, s2fd->hva, PUD_SIZE))
+ if (fault_supports_stage2_huge_mapping(s2fd, PUD_SIZE))
break;
fallthrough;
#endif
@@ -1787,7 +1798,7 @@ static short kvm_s2_resolve_vma_size(const struct kvm_s2_fault_desc *s2fd,
vma_shift = PMD_SHIFT;
fallthrough;
case PMD_SHIFT:
- if (fault_supports_stage2_huge_mapping(s2fd->memslot, s2fd->hva, PMD_SIZE))
+ if (fault_supports_stage2_huge_mapping(s2fd, PMD_SIZE))
break;
fallthrough;
case CONT_PTE_SHIFT:
@@ -2027,9 +2038,7 @@ static int kvm_s2_fault_map(const struct kvm_s2_fault_desc *s2fd,
if (perm_fault_granule > PAGE_SIZE) {
mapping_size = perm_fault_granule;
} else {
- mapping_size = transparent_hugepage_adjust(kvm, s2fd->memslot,
- s2fd->hva, &pfn,
- &gfn);
+ mapping_size = transparent_hugepage_adjust(s2fd, &pfn, &gfn);
if (mapping_size < 0) {
ret = mapping_size;
goto out_unlock;
diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
index 8e4c6e4bec12..c298a5b9b12a 100644
--- a/arch/arm64/kvm/pkvm.c
+++ b/arch/arm64/kvm/pkvm.c
@@ -414,6 +414,9 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,
u64 end = addr + size;
int ret;
+ if (WARN_ON_ONCE(!PAGE_ALIGNED(addr | size)))
+ return -EINVAL;
+
lockdep_assert_held_write(&kvm->mmu_lock);
mapping = pkvm_mapping_iter_first(&pgt->pkvm_mappings, addr, end - 1);
diff --git a/arch/arm64/kvm/vgic/vgic-mmio-v3.c b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
index 5ab789accc9d..9e6200c5adce 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
@@ -303,9 +303,19 @@ static void vgic_mmio_write_v3r_ctlr(struct kvm_vcpu *vcpu,
if (ctlr != GICR_CTLR_ENABLE_LPIS)
return;
+ /*
+ * Yes, disabling LPIs is painful, since it can be done from
+ * a *remote* vcpu! So let's not take any chance, and make
+ * sure that the target vCPU has written their LRs back to the
+ * irq structures, and release any reference they would have.
+ *
+ * If it hurts, don't do it.
+ */
+ kvm_arm_halt_guest(vcpu->kvm);
vgic_flush_pending_lpis(vcpu);
vgic_its_invalidate_all_caches(vcpu->kvm);
atomic_set_release(&vgic_cpu->ctlr, 0);
+ kvm_arm_resume_guest(vcpu->kvm);
} else {
ctlr = atomic_cmpxchg_acquire(&vgic_cpu->ctlr, 0,
GICR_CTLR_ENABLE_LPIS);
diff --git a/arch/arm64/mm/mmap.c b/arch/arm64/mm/mmap.c
index 32e0771d6477..e212270d4e09 100644
--- a/arch/arm64/mm/mmap.c
+++ b/arch/arm64/mm/mmap.c
@@ -61,7 +61,9 @@ int valid_phys_addr_range(phys_addr_t addr, size_t size)
*/
int valid_mmap_phys_addr_range(unsigned long pfn, size_t size)
{
- return !(((pfn << PAGE_SHIFT) + size) & ~PHYS_MASK);
+ const phys_addr_t limit = PHYS_MASK + 1;
+
+ return size <= limit && pfn <= (limit - size) >> PAGE_SHIFT;
}
static int __init adjust_protection_map(void)
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 12aacc74f764..b1516630904d 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -113,6 +113,10 @@ SYM_FUNC_START(cpu_do_suspend)
alternative_if ARM64_HAS_TCR2
mrs x2, REG_TCR2_EL1
str x2, [x0, #104]
+alternative_else_nop_endif
+alternative_if ARM64_SME
+ mrs_s x2, SYS_TPIDR2_EL0
+ str x2, [x0, #112]
alternative_else_nop_endif
ret
SYM_FUNC_END(cpu_do_suspend)
@@ -152,6 +156,10 @@ alternative_if ARM64_HAS_TCR2
ldr x2, [x0, #104]
msr REG_TCR2_EL1, x2
alternative_else_nop_endif
+alternative_if ARM64_SME
+ ldr x2, [x0, #112]
+ msr_s SYS_TPIDR2_EL0, x2
+alternative_else_nop_endif
msr sctlr_el1, x12
set_this_cpu_offset x13
diff --git a/arch/mips/mm/cex-gen.S b/arch/mips/mm/cex-gen.S
index e528583d1331..6989fe274aab 100644
--- a/arch/mips/mm/cex-gen.S
+++ b/arch/mips/mm/cex-gen.S
@@ -8,6 +8,7 @@
*
* Cache error handler
*/
+#include <asm/addrspace.h>
#include <asm/asm.h>
#include <asm/regdef.h>
#include <asm/mipsregs.h>
@@ -37,6 +38,21 @@
nop
nop
+#if defined(CONFIG_64BIT) && VMLINUX_LOAD_ADDRESS >= XKPHYS && \
+ VMLINUX_LOAD_ADDRESS < XKSEG
+ /*
+ * An XKPHYS kernel may lie outside the 256MB region a j from the
+ * vector reaches, so jump to the handler's uncached XKPHYS address.
+ */
+ PTR_LA k0, cache_parity_error
+ PTR_LI k1, TO_PHYS_MASK
+ and k0, k0, k1
+ PTR_LI k1, UNCAC_BASE
+ or k0, k0, k1
+ jr k0
+ nop
+#else
j cache_parity_error
nop
+#endif
END(except_vec2_generic)
diff --git a/arch/mips/pci/ops-mace.c b/arch/mips/pci/ops-mace.c
index 951d8070fb48..8827e5f2ea46 100644
--- a/arch/mips/pci/ops-mace.c
+++ b/arch/mips/pci/ops-mace.c
@@ -35,15 +35,27 @@ static inline int mkaddr(struct pci_bus *bus, unsigned int devfn,
(reg & 0xfc);
}
+/*
+ * On CRIME 1.1, a MACE PCI register read followed by two register writes
+ * can hang the machine under DMA load, so every register write in this
+ * file is followed by a read. Config cycles to empty slots end in a
+ * master abort; ack it with interrupts still off so the bridge error
+ * interrupt does not report it.
+ */
+static void mace_pci_ack_abort(void)
+{
+ u32 error = mace->pci.error;
+
+ if (error & MACEPCI_ERROR_MASTER_ABORT) {
+ mace->pci.error = error & ~MACEPCI_ERROR_MASTER_ABORT;
+ mace->pci.error;
+ }
+}
static int
mace_pci_read_config(struct pci_bus *bus, unsigned int devfn,
int reg, int size, u32 *val)
{
- u32 control = mace->pci.control;
-
- /* disable master aborts interrupts during config read */
- mace->pci.control = control & ~MACEPCI_CONTROL_MAR_INT;
mace->pci.config_addr = mkaddr(bus, devfn, reg);
switch (size) {
case 1:
@@ -56,9 +68,7 @@ mace_pci_read_config(struct pci_bus *bus, unsigned int devfn,
*val = mace->pci.config_data.l;
break;
}
- /* ack possible master abort */
- mace->pci.error &= ~MACEPCI_ERROR_MASTER_ABORT;
- mace->pci.control = control;
+ mace_pci_ack_abort();
/*
* someone forgot to set the ultra bit for the onboard
* scsi chips; we fake it here
@@ -77,6 +87,7 @@ mace_pci_write_config(struct pci_bus *bus, unsigned int devfn,
int reg, int size, u32 val)
{
mace->pci.config_addr = mkaddr(bus, devfn, reg);
+ mace->pci.error;
switch (size) {
case 1:
mace->pci.config_data.b[(reg & 3) ^ 3] = val;
@@ -88,6 +99,7 @@ mace_pci_write_config(struct pci_bus *bus, unsigned int devfn,
mace->pci.config_data.l = val;
break;
}
+ mace_pci_ack_abort();
DPRINTK("write%d: reg=%08x,val=%02x\n", size * 8, reg, val);
diff --git a/arch/mips/pci/pci-ip32.c b/arch/mips/pci/pci-ip32.c
index 7ae89d0c7099..41ec8453c865 100644
--- a/arch/mips/pci/pci-ip32.c
+++ b/arch/mips/pci/pci-ip32.c
@@ -77,6 +77,8 @@ static irqreturn_t macepci_error(int irq, void *dev)
}
mace->pci.error = flags;
+ /* follow the write with a read, see mace_pci_ack_abort() */
+ mace->pci.error;
return IRQ_HANDLED;
}
@@ -125,10 +127,13 @@ static int __init mace_init(void)
{
PCIBIOS_MIN_IO = 0x1000;
- /* Clear any outstanding errors and enable interrupts */
+ /*
+ * Clear any outstanding errors and enable interrupts. PCI interrupts
+ * are masked in CRIME, so the per-slot enables stay on.
+ */
mace->pci.error_addr = 0;
mace->pci.error = 0;
- mace->pci.control = 0xff008500;
+ mace->pci.control = 0xff0085ff;
printk("MACE PCI rev %d\n", mace->pci.rev);
diff --git a/arch/mips/sgi-ip32/ip32-irq.c b/arch/mips/sgi-ip32/ip32-irq.c
index 29d04468a06b..b9c72af481ed 100644
--- a/arch/mips/sgi-ip32/ip32-irq.c
+++ b/arch/mips/sgi-ip32/ip32-irq.c
@@ -160,38 +160,6 @@ static struct irq_chip crime_edge_interrupt = {
.irq_unmask = crime_enable_irq,
};
-/*
- * This is for MACE PCI interrupts. We can decrease bus traffic by masking
- * as close to the source as possible. This also means we can take the
- * next chunk of the CRIME register in one piece.
- */
-
-static unsigned long macepci_mask;
-
-static void enable_macepci_irq(struct irq_data *d)
-{
- macepci_mask |= MACEPCI_CONTROL_INT(d->irq - MACEPCI_SCSI0_IRQ);
- mace->pci.control = macepci_mask;
- crime_mask |= 1 << (d->irq - CRIME_IRQ_BASE);
- crime->imask = crime_mask;
-}
-
-static void disable_macepci_irq(struct irq_data *d)
-{
- crime_mask &= ~(1 << (d->irq - CRIME_IRQ_BASE));
- crime->imask = crime_mask;
- flush_crime_bus();
- macepci_mask &= ~MACEPCI_CONTROL_INT(d->irq - MACEPCI_SCSI0_IRQ);
- mace->pci.control = macepci_mask;
- flush_mace_bus();
-}
-
-static struct irq_chip ip32_macepci_interrupt = {
- .name = "IP32 MACE PCI",
- .irq_mask = disable_macepci_irq,
- .irq_unmask = enable_macepci_irq,
-};
-
/* This is used for MACE ISA interrupts. That means bits 4-6 in the
* CRIME register.
*/
@@ -299,8 +267,8 @@ static struct irq_chip ip32_maceisa_edge_interrupt = {
.irq_unmask = enable_maceisa_irq,
};
-/* This is used for regular non-ISA, non-PCI MACE interrupts. That means
- * bits 0-3 and 7 in the CRIME register.
+/* This is used for regular non-ISA MACE interrupts. That means bits 0-3
+ * and 7-15 in the CRIME register.
*/
static void enable_mace_irq(struct irq_data *d)
@@ -436,20 +404,13 @@ void __init arch_init_irq(void)
mips_cpu_irq_init();
for (irq = CRIME_IRQ_BASE; irq <= IP32_IRQ_MAX; irq++) {
switch (irq) {
- case MACE_VID_IN1_IRQ ... MACE_PCI_BRIDGE_IRQ:
+ case MACE_VID_IN1_IRQ ... MACEPCI_SHARED2_IRQ:
irq_set_chip_and_handler_name(irq,
&ip32_mace_interrupt,
handle_level_irq,
"level");
break;
- case MACEPCI_SCSI0_IRQ ... MACEPCI_SHARED2_IRQ:
- irq_set_chip_and_handler_name(irq,
- &ip32_macepci_interrupt,
- handle_level_irq,
- "level");
- break;
-
case CRIME_CPUERR_IRQ:
case CRIME_MEMERR_IRQ:
irq_set_chip_and_handler_name(irq,
diff --git a/arch/openrisc/kernel/or32_ksyms.c b/arch/openrisc/kernel/or32_ksyms.c
index 84a937a64e2a..5aabef30ca09 100644
--- a/arch/openrisc/kernel/or32_ksyms.c
+++ b/arch/openrisc/kernel/or32_ksyms.c
@@ -35,6 +35,7 @@ DECLARE_EXPORT(__divsi3);
DECLARE_EXPORT(__umodsi3);
DECLARE_EXPORT(__modsi3);
DECLARE_EXPORT(__muldi3);
+DECLARE_EXPORT(__mulsi3);
DECLARE_EXPORT(__ashrdi3);
DECLARE_EXPORT(__ashldi3);
DECLARE_EXPORT(__lshrdi3);
diff --git a/arch/x86/boot/startup/sme.c b/arch/x86/boot/startup/sme.c
index c07a2c381ed1..2d5cb3f7eee6 100644
--- a/arch/x86/boot/startup/sme.c
+++ b/arch/x86/boot/startup/sme.c
@@ -99,7 +99,7 @@ static void __init sme_clear_pgd(struct sme_populate_pgd_data *ppd)
pgd_t *pgd_p;
pgd_start = ppd->vaddr & PGDIR_MASK;
- pgd_end = ppd->vaddr_end & PGDIR_MASK;
+ pgd_end = (ppd->vaddr_end - 1) & PGDIR_MASK;
pgd_size = (((pgd_end - pgd_start) / PGDIR_SIZE) + 1) * sizeof(pgd_t);
diff --git a/arch/x86/entry/syscall_32.c b/arch/x86/entry/syscall_32.c
index 91123a90710c..4c36e8db0e34 100644
--- a/arch/x86/entry/syscall_32.c
+++ b/arch/x86/entry/syscall_32.c
@@ -77,6 +77,10 @@ static __always_inline void do_syscall_32_irqs_on(struct pt_regs *regs, unsigned
if (likely(nr < IA32_NR_syscalls)) {
nr = array_index_nospec(nr, IA32_NR_syscalls);
regs->ax = ia32_sys_call(regs, (unsigned int)nr);
+ } else {
+ /* See comment in do_syscall_64() */
+ if (nr != -1UL)
+ regs->ax = -ENOSYS;
}
}
diff --git a/arch/x86/entry/syscall_64.c b/arch/x86/entry/syscall_64.c
index 966d3d1b7586..bb95ad0ab786 100644
--- a/arch/x86/entry/syscall_64.c
+++ b/arch/x86/entry/syscall_64.c
@@ -64,15 +64,20 @@ static __always_inline bool do_syscall_x64(struct pt_regs *regs, unsigned long n
return false;
}
-static __always_inline void do_syscall_x32(struct pt_regs *regs, unsigned long nr)
+static __always_inline bool do_syscall_x32(struct pt_regs *regs, unsigned long nr)
{
+ if (!IS_ENABLED(CONFIG_X86_X32_ABI))
+ return false;
+
/* Adjust the starting offset of the table */
nr -= __X32_SYSCALL_BIT;
- if (IS_ENABLED(CONFIG_X86_X32_ABI) && likely(nr < X32_NR_syscalls)) {
+ if (likely(nr < X32_NR_syscalls)) {
nr = array_index_nospec(nr, X32_NR_syscalls);
regs->ax = x32_sys_call(regs, (unsigned int)nr);
+ return true;
}
+ return false;
}
/* Returns true to return using SYSRET, or false to use IRET */
@@ -81,8 +86,21 @@ __visible noinstr bool do_syscall_64(struct pt_regs *regs, long nr)
if (likely(syscall_enter_from_user_mode_randomize_stack(regs, &nr))) {
instrumentation_begin();
- if (!do_syscall_x64(regs, nr))
- do_syscall_x32(regs, nr);
+ if (unlikely(!do_syscall_x64(regs, nr))) {
+ /*
+ * The special treatment of syscall number -1 is a
+ * historical bogosity as the original code used -1 to
+ * indicate syscall abort, which is not distinguishable
+ * from a syscall which is invoked with syscall number
+ * -1. The historic behavior is to keep the return value
+ * in regs->ax unmodified in that case, which means it's
+ * either the preset value -ENOSYS or a value which was
+ * written by a tracer. If the syscall number is invalid
+ * but not -1 then overwrite any tracer modification.
+ */
+ if (!do_syscall_x32(regs, nr) && nr != -1L)
+ regs->ax = -ENOSYS;
+ }
instrumentation_end();
}
diff --git a/arch/x86/include/asm/checksum_32.h b/arch/x86/include/asm/checksum_32.h
index 17da95387997..f74873a5e351 100644
--- a/arch/x86/include/asm/checksum_32.h
+++ b/arch/x86/include/asm/checksum_32.h
@@ -159,7 +159,7 @@ static inline __sum16 csum_ipv6_magic(const struct in6_addr *saddr,
"adcl $0, %0 ;\n"
: "=&r" (sum)
: "r" (saddr), "r" (daddr),
- "r" (htonl(len)), "r" (htonl(proto)), "0" (sum)
+ "g" (htonl(len)), "g" (htonl(proto)), "0" (sum)
: "memory");
return csum_fold(sum);
diff --git a/arch/x86/include/asm/vmware.h b/arch/x86/include/asm/vmware.h
index 4220dae14a2d..fb011cef01e7 100644
--- a/arch/x86/include/asm/vmware.h
+++ b/arch/x86/include/asm/vmware.h
@@ -276,20 +276,22 @@ unsigned long vmware_hypercall_hb_out(unsigned long cmd, unsigned long in2,
asm_inline volatile (
UNWIND_HINT_SAVE
+ "mov %[in6], %%" _ASM_AX "\n\t"
"push %%" _ASM_BP "\n\t"
UNWIND_HINT_UNDEFINED
- "mov %[in6], %%" _ASM_BP "\n\t"
+ "mov %%" _ASM_AX ", %%" _ASM_BP "\n\t"
+ "mov %[magic], %%eax\n\t"
"rep outsb\n\t"
"pop %%" _ASM_BP "\n\t"
UNWIND_HINT_RESTORE
- : "=a" (out0), "=b" (*out1)
- : "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (cmd),
+ : "=&a" (out0), "=b" (*out1)
+ : "b" (cmd),
"c" (in2),
"d" (in3 | VMWARE_HYPERVISOR_PORT_HB),
"S" (in4),
"D" (in5),
- [in6] VMW_BP_CONSTRAINT (in6)
+ [in6] VMW_BP_CONSTRAINT(in6),
+ [magic] "i" (VMWARE_HYPERVISOR_MAGIC)
: "cc", "memory");
return out0;
}
@@ -304,20 +306,22 @@ unsigned long vmware_hypercall_hb_in(unsigned long cmd, unsigned long in2,
asm_inline volatile (
UNWIND_HINT_SAVE
+ "mov %[in6], %%" _ASM_AX "\n\t"
"push %%" _ASM_BP "\n\t"
UNWIND_HINT_UNDEFINED
- "mov %[in6], %%" _ASM_BP "\n\t"
+ "mov %%" _ASM_AX ", %%" _ASM_BP "\n\t"
+ "mov %[magic], %%eax\n\t"
"rep insb\n\t"
"pop %%" _ASM_BP "\n\t"
UNWIND_HINT_RESTORE
- : "=a" (out0), "=b" (*out1)
- : "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (cmd),
+ : "=&a" (out0), "=b" (*out1)
+ : "b" (cmd),
"c" (in2),
"d" (in3 | VMWARE_HYPERVISOR_PORT_HB),
"S" (in4),
"D" (in5),
- [in6] VMW_BP_CONSTRAINT (in6)
+ [in6] VMW_BP_CONSTRAINT(in6),
+ [magic] "i" (VMWARE_HYPERVISOR_MAGIC)
: "cc", "memory");
return out0;
}
diff --git a/arch/x86/kernel/acpi/cppc.c b/arch/x86/kernel/acpi/cppc.c
index bbade0da5130..174b734e8261 100644
--- a/arch/x86/kernel/acpi/cppc.c
+++ b/arch/x86/kernel/acpi/cppc.c
@@ -6,6 +6,8 @@
#include <linux/bitfield.h>
+#include <linux/limits.h>
+
#include <acpi/cppc_acpi.h>
#include <asm/msr.h>
#include <asm/processor.h>
@@ -45,10 +47,20 @@ bool cpc_ffh_supported(void)
return true;
}
+static bool cpc_ffh_reg_valid(const struct cpc_reg *reg)
+{
+ return reg->address <= U32_MAX && reg->bit_width &&
+ reg->bit_width <= 64 &&
+ reg->bit_offset <= 64 - reg->bit_width;
+}
+
int cpc_read_ffh(int cpunum, struct cpc_reg *reg, u64 *val)
{
int err;
+ if (!cpc_ffh_reg_valid(reg))
+ return -EINVAL;
+
err = rdmsrq_safe_on_cpu(cpunum, reg->address, val);
if (!err) {
u64 mask = GENMASK_ULL(reg->bit_offset + reg->bit_width - 1,
@@ -65,6 +77,9 @@ int cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val)
u64 rd_val;
int err;
+ if (!cpc_ffh_reg_valid(reg))
+ return -EINVAL;
+
err = rdmsrq_safe_on_cpu(cpunum, reg->address, &rd_val);
if (!err) {
u64 mask = GENMASK_ULL(reg->bit_offset + reg->bit_width - 1,
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 000ebee2d247..4ced10c6ba0d 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -2088,14 +2088,12 @@ static void __init bhi_select_mitigation(void)
if (bhi_mitigation != BHI_MITIGATION_AUTO)
return;
- if (cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST)) {
- if (cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL))
- bhi_mitigation = BHI_MITIGATION_ON;
- else
- bhi_mitigation = BHI_MITIGATION_VMEXIT_ONLY;
- } else {
+ if (cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL))
+ bhi_mitigation = BHI_MITIGATION_ON;
+ else if (cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST))
+ bhi_mitigation = BHI_MITIGATION_VMEXIT_ONLY;
+ else
bhi_mitigation = BHI_MITIGATION_OFF;
- }
}
static void __init bhi_update_mitigation(void)
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index e3cfa0c10e92..ffd74a68671b 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -214,8 +214,8 @@ union l3_qos_abmc_cfg {
bw_src :12,
reserved1: 3,
is_clos : 1,
- cntr_id : 5,
- reserved : 9,
+ cntr_id :12,
+ reserved : 2,
cntr_en : 1,
cfg_en : 1;
} split;
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 3838e0a13d36..a6a9090c4808 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -470,8 +470,8 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)
r->mon.mbm_cntr_assignable = true;
r->mon.mbm_cntr_configurable = true;
cpuid_count(0x80000020, 5, &eax, &ebx, &ecx, &edx);
- r->mon.num_mbm_cntrs = (ebx & GENMASK(15, 0)) + 1;
- hw_res->mbm_cntr_assign_enabled = true;
+ /* cntr_id is 12 bits and can only encode 4096 counters. */
+ r->mon.num_mbm_cntrs = min((ebx & GENMASK(15, 0)) + 1, BIT(12));
}
r->mon_capable = true;
diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c
index 3c9f60d6ca5a..cb46da482c85 100644
--- a/arch/x86/kernel/nmi.c
+++ b/arch/x86/kernel/nmi.c
@@ -115,6 +115,14 @@ static void nmi_check_duration(struct nmiaction *action, u64 duration)
{
int remainder_ns, decimal_msecs;
+ /* sched_clock() went backwards? */
+ if ((s64)duration < 0) {
+ pr_warn_once("NMI handler (%ps): sched_clock() went backwards by %lld ns on CPU%d\n",
+ action->handler, -(s64)duration,
+ smp_processor_id());
+ return;
+ }
+
if (duration < nmi_longest_ns || duration < action->max_duration)
return;
diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c
index 252c5827d063..3b9b41820c84 100644
--- a/arch/x86/kernel/x86_init.c
+++ b/arch/x86/kernel/x86_init.c
@@ -37,7 +37,7 @@ static void iommu_shutdown_noop(void) { }
bool __init bool_x86_init_noop(void) { return false; }
void x86_op_int_noop(int cpu) { }
int set_rtc_noop(const struct timespec64 *now) { return -EINVAL; }
-void get_rtc_noop(struct timespec64 *now) { }
+void get_rtc_noop(struct timespec64 *now) { *now = (struct timespec64){0}; }
static __initconst const struct of_device_id of_cmos_match[] = {
{ .compatible = "motorola,mc146818" },
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index e56497089b59..2c2ff20f8ad0 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -2310,6 +2310,16 @@ static struct kvm_mmu_page *kvm_mmu_find_shadow_page(struct kvm *kvm,
continue;
}
+ /*
+ * Note! Unlike fully direct MMUs, direct SPs in indirect MMUs
+ * can have an arbitrary number of parents, e.g if the guest is
+ * mapping a huge PTE at many points in its page tables.
+ */
+ if (pte_list_count(&sp->parent_ptes) >= RMAP_RECYCLE_THRESHOLD) {
+ kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list);
+ break;
+ }
+
/* unsync and write-flooding only apply to indirect SPs. */
if (sp->role.direct)
goto out;
diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h
index 6e0153f23136..7454dac59b3a 100644
--- a/arch/x86/kvm/mmu/paging_tmpl.h
+++ b/arch/x86/kvm/mmu/paging_tmpl.h
@@ -339,6 +339,13 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,
gpa_t real_gpa;
gfn_t gfn;
+ /*
+ * KVM might sleep/resched when retrying the walk if the updating A/D
+ * fails because the gPTE changed between reading the gPTE and trying
+ * the CMPXHG to set A/D bits.
+ */
+ might_sleep();
+
trace_kvm_mmu_pagetable_walk(addr, access);
retry_walk:
walker->level = w->cpu_role.base.level;
@@ -479,8 +486,10 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,
addr, write_fault);
if (unlikely(ret < 0))
goto error;
- else if (ret)
+ else if (ret) {
+ cond_resched();
goto retry_walk;
+ }
}
return 1;
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 5165275128fc..a1450c55e946 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -5059,6 +5059,7 @@ static void nested_vmx_restore_host_state(struct kvm_vcpu *vcpu)
__func__, j, h.index, h.value);
goto vmabort;
}
+ break;
}
}
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index c76498d1a162..56a19a7260d3 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -6515,10 +6515,12 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
} else if (vcpu->arch.complete_userspace_io) {
writeback = false;
r = 0;
- } else if (r == EMULATION_RESTART)
+ } else if (r == EMULATION_RESTART) {
+ cond_resched();
goto restart;
- else
+ } else {
r = 1;
+ }
writeback:
if (writeback) {
@@ -9661,18 +9663,23 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
kvm_x86_call(set_efer)(vcpu, 0);
kvm_x86_call(update_exception_bitmap)(vcpu);
+ if (old_cr0 & X86_CR0_PG)
+ kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu);
+
/*
- * On the standard CR0/CR4/EFER modification paths, there are several
- * complex conditions determining whether the MMU has to be reset and/or
- * which PCIDs have to be flushed. However, CR0.WP and the paging-related
- * bits in CR4 and EFER are irrelevant if CR0.PG was '0'; and a reset+flush
- * is needed anyway if CR0.PG was '1' (which can only happen for INIT, as
- * CR0 will be '0' prior to RESET). So we only need to check CR0.PG here.
+ * Unconditionally reset the MMU, even though super strictly speaking,
+ * a MMU reset is only necessary if and only one of the page role bits
+ * is changing. All of the CR0/CR4/EFER bits effectively get covered
+ * by checking CR0.PG=1, as CR0.WP and the paging-related bits in CR4
+ * and EFER are irrelevant if CR0.PG was '0'. Subtly, if the vCPU was
+ * in guest mode (L2) or SMM (although emulating INIT in SMM _should_
+ * be impossible), an MMU reset is also necessary as "guest mode" and
+ * "smm" are both encoded in the shadow page role. As a result, the
+ * only scenarios where an MMU reset is *not* needed is if the vCPU is
+ * in Real Mode or Unpaged Protected Mode, and micro-optimizing those
+ * paths are not worth the risk of running with a stale MMU context.
*/
- if (old_cr0 & X86_CR0_PG) {
- kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu);
- kvm_mmu_reset_context(vcpu);
- }
+ kvm_mmu_reset_context(vcpu);
/*
* Intel's SDM states that all TLB entries are flushed on INIT. AMD's
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 0c4da320831c..53dcab39566f 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -320,13 +320,19 @@ static u16 allocate_global_asid(void)
lockdep_assert_held(&global_asid_lock);
- /* The previous allocation hit the edge of available address space */
- if (last_global_asid >= MAX_ASID_AVAILABLE - 1)
- reset_global_asid_space();
-
asid = find_next_zero_bit(global_asid_used, MAX_ASID_AVAILABLE, last_global_asid);
- if (asid >= MAX_ASID_AVAILABLE && !global_asid_available) {
+ /*
+ * Nothing is free between last_global_asid and MAX_ASID_AVAILABLE.
+ * Flush lazy-freed ASIDs and reset the search space.
+ */
+ if (asid >= MAX_ASID_AVAILABLE) {
+ reset_global_asid_space();
+ asid = find_next_zero_bit(global_asid_used, MAX_ASID_AVAILABLE,
+ last_global_asid);
+ }
+
+ if (asid >= MAX_ASID_AVAILABLE) {
/* This should never happen. */
VM_WARN_ONCE(1, "Unable to allocate global ASID despite %d available\n",
global_asid_available);
diff --git a/block/bio.c b/block/bio.c
index f95b63c0604a..90104e6abbcf 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1285,6 +1285,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter,
unsigned mem_align_mask, unsigned len_align_mask)
{
iov_iter_extraction_t flags = 0;
+ struct iov_iter saved_iter;
if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED)))
return -EIO;
@@ -1303,6 +1304,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter,
if (bio->bi_bdev && blk_queue_pci_p2pdma(bio->bi_bdev->bd_disk->queue))
flags |= ITER_ALLOW_P2PDMA;
+ saved_iter = *iter;
do {
ssize_t ret;
@@ -1320,6 +1322,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter,
bio_release_pages(bio, false);
bio_clear_flag(bio, BIO_PAGE_PINNED);
bio->bi_vcnt = 0;
+ *iter = saved_iter;
}
if (!bio->bi_vcnt)
return ret;
diff --git a/block/blk-iocost.c b/block/blk-iocost.c
index 7b8af094ab37..23445d701433 100644
--- a/block/blk-iocost.c
+++ b/block/blk-iocost.c
@@ -2851,6 +2851,9 @@ static void ioc_rqos_done(struct rq_qos *rqos, struct request *rq)
if (!ioc->enabled || !rq->alloc_time_ns || !rq->start_time_ns)
return;
+ if (unlikely(rq->rq_flags & RQF_FLUSH_SEQ))
+ return;
+
switch (req_op(rq)) {
case REQ_OP_READ:
pidx = QOS_RLAT;
diff --git a/block/blk-mq.c b/block/blk-mq.c
index 5a5dc66fc3b1..e21d69767fd4 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -3193,16 +3193,15 @@ void blk_mq_submit_bio(struct bio *bio)
blk_mq_bio_to_request(rq, bio, nr_segs);
+ if (bio_zone_write_plugging(bio))
+ blk_zone_write_plug_init_request(rq);
+
ret = blk_crypto_rq_get_keyslot(rq);
if (ret != BLK_STS_OK) {
- bio_endio_status(bio, ret);
- blk_mq_free_request(rq);
+ blk_mq_end_request(rq, ret);
return;
}
- if (bio_zone_write_plugging(bio))
- blk_zone_write_plug_init_request(rq);
-
if (op_is_flush(bio->bi_opf) && blk_insert_flush(rq))
return;
@@ -3485,8 +3484,10 @@ void blk_mq_free_rqs(struct blk_mq_tag_set *set, struct blk_mq_tags *tags,
if (blk_mq_is_shared_tags(set->flags))
drv_tags = set->shared_tags;
- else
+ else if (hctx_idx < set->nr_hw_queues)
drv_tags = set->tags[hctx_idx];
+ else
+ drv_tags = NULL;
if (tags->static_rqs && set->ops->exit_request) {
int i;
diff --git a/block/elevator.c b/block/elevator.c
index 2161b6eea680..66edf1a5768a 100644
--- a/block/elevator.c
+++ b/block/elevator.c
@@ -812,25 +812,29 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf,
* reference during concurrent disk deletion:
* update_nr_hwq_lock -> kn->active (via del_gendisk -> kobject_del)
* kn->active -> update_nr_hwq_lock (via this sysfs write path)
- *
- * Use the writer lock instead of the reader lock of update_nr_hwq_lock
- * to serialize the two-stage elevator switch steps in
- * elevator_change(): the core switch step under the elevator lock and
- * the elevator_change_done() step outside the elevator lock.
*/
- if (!down_write_trylock(&set->update_nr_hwq_lock)) {
+ if (!down_read_trylock(&set->update_nr_hwq_lock)) {
ret = -EBUSY;
goto out;
}
- if (!blk_queue_no_elv_switch(q)) {
- ret = elevator_change(q, &ctx);
- if (!ret)
- ret = count;
- } else {
- ret = -ENOENT;
+
+ /*
+ * The reader lock is shared, so use QUEUE_FLAG_NO_ELV_SWITCH to allow
+ * only one switch of this queue at a time, and none once the disk is
+ * going away.
+ */
+ if (test_and_set_bit(QUEUE_FLAG_NO_ELV_SWITCH, &q->queue_flags)) {
+ ret = -EBUSY;
+ goto out_unlock;
}
- up_write(&set->update_nr_hwq_lock);
+ ret = elevator_change(q, &ctx);
+ if (!ret)
+ ret = count;
+
+ blk_queue_flag_clear(QUEUE_FLAG_NO_ELV_SWITCH, q);
+out_unlock:
+ up_read(&set->update_nr_hwq_lock);
out:
if (ctx.type)
elevator_put(ctx.type);
diff --git a/block/ioctl.c b/block/ioctl.c
index 64b4e6c0f696..fb2f4b4d2505 100644
--- a/block/ioctl.c
+++ b/block/ioctl.c
@@ -835,8 +835,8 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg)
return compat_put_ulong(argp, bdev_nr_sectors(bdev));
/* The data is compatible, but the command number is different */
- case BLKBSZGET_32: /* get the logical block size (cf. BLKSSZGET) */
- return put_int(argp, bdev_logical_block_size(bdev));
+ case BLKBSZGET_32: /* get the soft block size (cf. BLKSSZGET) */
+ return put_int(argp, block_size(bdev));
case BLKBSZSET_32:
return blkdev_bszset(file, mode, argp);
case BLKGETSIZE64_32:
diff --git a/drivers/acpi/battery.c b/drivers/acpi/battery.c
index 670853ec3a4d..7169ac7e83f3 100644
--- a/drivers/acpi/battery.c
+++ b/drivers/acpi/battery.c
@@ -583,9 +583,8 @@ static int acpi_battery_get_info(struct acpi_battery *battery)
struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL };
acpi_status status = AE_ERROR;
- status = acpi_evaluate_object(battery->device->handle,
- use_bix ? "_BIX":"_BIF",
- NULL, &buffer);
+ status = acpi_evaluate_object_exclusive(battery->device->handle,
+ use_bix ? "_BIX" : "_BIF", NULL, &buffer);
if (ACPI_FAILURE(status)) {
acpi_handle_info(battery->device->handle,
@@ -624,8 +623,7 @@ static int acpi_battery_get_state(struct acpi_battery *battery)
msecs_to_jiffies(cache_time)))
return 0;
- status = acpi_evaluate_object(battery->device->handle, "_BST",
- NULL, &buffer);
+ status = acpi_evaluate_object_exclusive(battery->device->handle, "_BST", NULL, &buffer);
if (ACPI_FAILURE(status)) {
acpi_handle_info(battery->device->handle,
"_BST evaluation failed: %s",
@@ -675,8 +673,8 @@ static int acpi_battery_set_alarm(struct acpi_battery *battery)
!test_bit(ACPI_BATTERY_ALARM_PRESENT, &battery->flags))
return -ENODEV;
- status = acpi_execute_simple_method(battery->device->handle, "_BTP",
- battery->alarm);
+ status = acpi_execute_simple_method_exclusive(battery->device->handle, "_BTP",
+ battery->alarm);
if (ACPI_FAILURE(status))
return -ENODEV;
diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c
index fef54fcd00b7..8aa0ce556863 100644
--- a/drivers/acpi/cppc_acpi.c
+++ b/drivers/acpi/cppc_acpi.c
@@ -36,6 +36,7 @@
#include <linux/delay.h>
#include <linux/iopoll.h>
#include <linux/ktime.h>
+#include <linux/mutex.h>
#include <linux/rwsem.h>
#include <linux/wait.h>
#include <linux/topology.h>
@@ -93,6 +94,16 @@ static DEFINE_PER_CPU(int, cpu_pcc_subspace_idx);
*/
static DEFINE_PER_CPU(struct cpc_desc *, cpc_desc_ptr);
+/* Protect immutable capability queries against descriptor removal. */
+static DEFINE_MUTEX(cpc_desc_lock);
+
+static void cpc_set_desc(unsigned int cpu, struct cpc_desc *desc)
+{
+ mutex_lock(&cpc_desc_lock);
+ per_cpu(cpc_desc_ptr, cpu) = desc;
+ mutex_unlock(&cpc_desc_lock);
+}
+
/* pcc mapped address + header size + offset within PCC subspace */
#define GET_PCC_VADDR(offs, pcc_ss_id) (pcc_data[pcc_ss_id]->pcc_channel->shmem + \
0x8 + (offs))
@@ -999,12 +1010,12 @@ int acpi_cppc_processor_probe(struct acpi_processor *pr)
}
/* Plug PSD data into this CPU's CPC descriptor. */
- per_cpu(cpc_desc_ptr, pr->id) = cpc_ptr;
+ cpc_set_desc(pr->id, cpc_ptr);
ret = kobject_init_and_add(&cpc_ptr->kobj, &cppc_ktype, &cpu_dev->kobj,
"acpi_cppc");
if (ret) {
- per_cpu(cpc_desc_ptr, pr->id) = NULL;
+ cpc_set_desc(pr->id, NULL);
kobject_put(&cpc_ptr->kobj);
goto out_free;
}
@@ -1056,6 +1067,8 @@ void acpi_cppc_processor_exit(struct acpi_processor *pr)
if (!cpc_ptr)
return;
+ cpc_set_desc(pr->id, NULL);
+
/* Free all the mapped sys mem areas for this CPU */
for (i = 2; i < cpc_ptr->num_entries; i++) {
addr = cpc_ptr->cpc_regs[i-2].sys_mem_vaddr;
@@ -1133,6 +1146,9 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val)
}
*val = 0;
+ if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
+ return cpc_read_ffh(cpu, reg, val);
+
size = GET_BIT_WIDTH(reg);
if (IS_ENABLED(CONFIG_HAS_IOPORT) &&
@@ -1160,8 +1176,6 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val)
}
else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY)
vaddr = reg_res->sys_mem_vaddr;
- else if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
- return cpc_read_ffh(cpu, reg, val);
else
return acpi_os_read_memory((acpi_physical_address)reg->address,
val, size);
@@ -1208,6 +1222,9 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val)
unsigned long flags;
bool locked = false;
+ if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
+ return cpc_write_ffh(cpu, reg, val);
+
size = GET_BIT_WIDTH(reg);
if (IS_ENABLED(CONFIG_HAS_IOPORT) &&
@@ -1233,8 +1250,6 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val)
}
else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY)
vaddr = reg_res->sys_mem_vaddr;
- else if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
- return cpc_write_ffh(cpu, reg, val);
else
return acpi_os_write_memory((acpi_physical_address)reg->address,
val, size);
@@ -1400,6 +1415,11 @@ static int cppc_set_reg_val(int cpu, enum cppc_regs reg_idx, u64 val)
reg = &cpc_desc->cpc_regs[reg_idx];
+ /* Integer 1 describes autonomous selection that is always enabled. */
+ if (reg_idx == AUTO_SEL_ENABLE && reg->type == ACPI_TYPE_INTEGER &&
+ reg->cpc_entry.int_value == 1)
+ return val == 1 ? 0 : -EOPNOTSUPP;
+
/* if a register is writeable, it must be a buffer and not null */
if ((reg->type != ACPI_TYPE_BUFFER) || IS_NULL_REG(®->cpc_entry.reg)) {
pr_debug("CPC register is not supported\n");
@@ -1613,6 +1633,9 @@ bool cppc_perf_ctrs_in_pcc_cpu(unsigned int cpu)
{
struct cpc_desc *cpc_desc = per_cpu(cpc_desc_ptr, cpu);
+ if (!cpc_desc)
+ return false;
+
return CPC_IN_PCC(&cpc_desc->cpc_regs[DELIVERED_CTR]) ||
CPC_IN_PCC(&cpc_desc->cpc_regs[REFERENCE_CTR]) ||
CPC_IN_PCC(&cpc_desc->cpc_regs[CTR_WRAP_TIME]);
@@ -1924,6 +1947,32 @@ int cppc_get_auto_sel(int cpu, bool *enable)
}
EXPORT_SYMBOL_GPL(cppc_get_auto_sel);
+/**
+ * cppc_auto_sel_is_immutable - Check for always-enabled autonomous selection.
+ * @cpu: CPU whose _CPC descriptor to check.
+ *
+ * Context: Process context.
+ * Return: true for Integer 1, false for a register or an absent descriptor.
+ */
+bool cppc_auto_sel_is_immutable(int cpu)
+{
+ struct cpc_desc *cpc_desc;
+ struct cpc_register_resource *reg;
+ bool immutable = false;
+
+ mutex_lock(&cpc_desc_lock);
+ cpc_desc = per_cpu(cpc_desc_ptr, cpu);
+ if (cpc_desc) {
+ reg = &cpc_desc->cpc_regs[AUTO_SEL_ENABLE];
+ immutable = reg->type == ACPI_TYPE_INTEGER &&
+ reg->cpc_entry.int_value == 1;
+ }
+ mutex_unlock(&cpc_desc_lock);
+
+ return immutable;
+}
+EXPORT_SYMBOL_GPL(cppc_auto_sel_is_immutable);
+
/**
* cppc_set_auto_sel - Write autonomous selection register.
* @cpu : CPU to which to write register.
diff --git a/drivers/acpi/utils.c b/drivers/acpi/utils.c
index d499b72574ab..ea8cb876c12d 100644
--- a/drivers/acpi/utils.c
+++ b/drivers/acpi/utils.c
@@ -10,6 +10,7 @@
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/slab.h>
#include <linux/init.h>
#include <linux/types.h>
@@ -243,6 +244,28 @@ acpi_extract_package(union acpi_object *package,
EXPORT_SYMBOL(acpi_extract_package);
+static DEFINE_MUTEX(acpi_exclusive_eval_lock);
+
+/**
+ * acpi_evaluate_object_exclusive - Evaluate an object without interleaving
+ * @handle: ACPI handle of the object or its scope.
+ * @pathname: Path of the object relative to @handle.
+ * @arguments: Arguments to pass, or NULL.
+ * @buffer: Buffer for the return value, or NULL.
+ *
+ * Same as acpi_evaluate_object(), but serialized with other exclusive
+ * evaluations. Use it only for methods known to break when interleaved.
+ */
+acpi_status acpi_evaluate_object_exclusive(acpi_handle handle, acpi_string pathname,
+ struct acpi_object_list *arguments,
+ struct acpi_buffer *buffer)
+{
+ guard(mutex)(&acpi_exclusive_eval_lock);
+
+ return acpi_evaluate_object(handle, pathname, arguments, buffer);
+}
+EXPORT_SYMBOL_GPL(acpi_evaluate_object_exclusive);
+
acpi_status
acpi_evaluate_integer(acpi_handle handle,
acpi_string pathname,
@@ -277,6 +300,16 @@ acpi_evaluate_integer(acpi_handle handle,
EXPORT_SYMBOL(acpi_evaluate_integer);
+acpi_status acpi_evaluate_integer_exclusive(acpi_handle handle, acpi_string pathname,
+ struct acpi_object_list *arguments,
+ unsigned long long *data)
+{
+ guard(mutex)(&acpi_exclusive_eval_lock);
+
+ return acpi_evaluate_integer(handle, pathname, arguments, data);
+}
+EXPORT_SYMBOL_GPL(acpi_evaluate_integer_exclusive);
+
int acpi_get_local_u64_address(acpi_handle handle, u64 *addr)
{
acpi_status status;
@@ -685,6 +718,14 @@ acpi_status acpi_execute_simple_method(acpi_handle handle, char *method,
}
EXPORT_SYMBOL(acpi_execute_simple_method);
+acpi_status acpi_execute_simple_method_exclusive(acpi_handle handle, char *method, u64 arg)
+{
+ guard(mutex)(&acpi_exclusive_eval_lock);
+
+ return acpi_execute_simple_method(handle, method, arg);
+}
+EXPORT_SYMBOL_GPL(acpi_execute_simple_method_exclusive);
+
/**
* acpi_evaluate_ej0: Evaluate _EJ0 method for hotplug operations
* @handle: ACPI device handle
diff --git a/drivers/ata/libata-pmp.c b/drivers/ata/libata-pmp.c
index 904f418387d1..43991e54eb08 100644
--- a/drivers/ata/libata-pmp.c
+++ b/drivers/ata/libata-pmp.c
@@ -482,6 +482,20 @@ static void sata_pmp_quirks(struct ata_port *ap)
if (link->pmp == 4)
link->flags |= ATA_LFLAG_DISABLED;
}
+ } else if (vendor == 0x1b4b && devid == 0x9705) {
+ /*
+ * Marvell 88SM9705 quirks
+ *
+ * 1-to-5 PMP with an extra SEMB port. It reports 6 ports and
+ * port 5 is the SEMB, which never answers SRST: EH spends about
+ * a minute per round retrying it, and disk discovery waits for
+ * that. Seen behind a Marvell 88SE9235 in the Synology DX1215
+ * and DX1222.
+ */
+ ata_for_each_link(link, ap, EDGE) {
+ if (link->pmp == 5)
+ link->flags |= ATA_LFLAG_DISABLED;
+ }
}
}
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index b86ac50f16b9..023de0e633b7 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1984,6 +1984,18 @@ static int ata_scsi_translate(struct ata_device *dev, struct scsi_cmnd *cmd,
if (xlat_func(qc))
goto done;
+ /*
+ * A data-out request may contain pages pinned without write access.
+ * Reject a translated data-in command before issuing it, since its
+ * completion handler may write to the buffer even if the device fails.
+ */
+ if (unlikely(cmd->sc_data_direction == DMA_TO_DEVICE &&
+ ata_is_data(qc->tf.protocol) &&
+ !(qc->tf.flags & ATA_TFLAG_WRITE))) {
+ ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0x4b, 0x0a);
+ goto done;
+ }
+
return ata_scsi_qc_issue(ap, qc);
done:
@@ -2015,6 +2027,12 @@ static void ata_scsi_rbuf_fill(struct ata_device *dev, struct scsi_cmnd *cmd,
unsigned long flags;
unsigned int len;
+ /* A data-out request may contain pages pinned without write access. */
+ if (cmd->sc_data_direction == DMA_TO_DEVICE) {
+ ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0x4b, 0x0a);
+ return;
+ }
+
spin_lock_irqsave(&ata_scsi_rbuf_lock, flags);
memset(ata_scsi_rbuf, 0, ATA_SCSI_RBUF_SIZE);
@@ -3108,6 +3126,13 @@ static unsigned int atapi_xlat(struct ata_queued_cmd *qc)
qc->complete_fn = atapi_qc_complete;
+ /* atapi_fixup_inquiry() writes to the buffer on successful completion. */
+ if (scmd->sc_data_direction == DMA_TO_DEVICE &&
+ scmd->cmnd[0] == INQUIRY) {
+ ata_scsi_set_sense(dev, scmd, ABORTED_COMMAND, 0x4b, 0x0a);
+ return 1;
+ }
+
qc->tf.flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_DEVICE;
if (scmd->sc_data_direction == DMA_TO_DEVICE) {
qc->tf.flags |= ATA_TFLAG_WRITE;
diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
index 1ef01ce18f4f..959e6594013a 100644
--- a/drivers/block/ublk_drv.c
+++ b/drivers/block/ublk_drv.c
@@ -5833,6 +5833,7 @@ static int ublk_ctrl_uring_cmd_permission(struct ublk_device *ub,
case UBLK_CMD_STOP_DEV:
case UBLK_CMD_ADD_DEV:
case UBLK_CMD_DEL_DEV:
+ case UBLK_CMD_DEL_DEV_ASYNC:
case UBLK_CMD_SET_PARAMS:
case UBLK_CMD_START_USER_RECOVERY:
case UBLK_CMD_END_USER_RECOVERY:
diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
index c8bfbf8435ac..10ba8b0b863b 100644
--- a/drivers/block/virtio_blk.c
+++ b/drivers/block/virtio_blk.c
@@ -1596,7 +1596,7 @@ static int virtblk_freeze_priv(struct virtio_device *vdev)
/* Ensure no requests in virtqueues before deleting vqs. */
memflags = blk_mq_freeze_queue(q);
- blk_mq_quiesce_queue_nowait(q);
+ blk_mq_quiesce_queue(q);
blk_mq_unfreeze_queue(q, memflags);
/* Ensure we don't receive any more interrupts */
diff --git a/drivers/bluetooth/btqca.c b/drivers/bluetooth/btqca.c
index 22b08ab05b82..002307000d3c 100644
--- a/drivers/bluetooth/btqca.c
+++ b/drivers/bluetooth/btqca.c
@@ -426,7 +426,8 @@ static int qca_tlv_check_data(struct hci_dev *hdev,
tlv_nvm->data[0] |= 0x80;
/* UART Baud Rate */
- if (soc_type >= QCA_WCN3991)
+ if (soc_type == QCA_WCN3950 ||
+ soc_type >= QCA_WCN3991)
tlv_nvm->data[1] = nvm_baud_rate;
else
tlv_nvm->data[2] = nvm_baud_rate;
diff --git a/drivers/char/hw_random/virtio-rng.c b/drivers/char/hw_random/virtio-rng.c
index 5e83ffa105e4..a9ff520975a7 100644
--- a/drivers/char/hw_random/virtio-rng.c
+++ b/drivers/char/hw_random/virtio-rng.c
@@ -190,6 +190,7 @@ static int probe_common(struct virtio_device *vdev)
err_find:
ida_free(&rng_index_ida, index);
err_ida:
+ vdev->priv = NULL;
kfree(vi);
return err;
}
@@ -198,6 +199,11 @@ static void remove_common(struct virtio_device *vdev)
{
struct virtrng_info *vi = vdev->priv;
+ if (!vi) {
+ virtio_reset_device(vdev);
+ return;
+ }
+
vi->hwrng_removed = true;
vi->data_avail = 0;
vi->data_idx = 0;
@@ -207,6 +213,7 @@ static void remove_common(struct virtio_device *vdev)
virtio_reset_device(vdev);
vdev->config->del_vqs(vdev);
ida_free(&rng_index_ida, vi->index);
+ vdev->priv = NULL;
kfree(vi);
}
diff --git a/drivers/clk/nuvoton/clk-ma35d1.c b/drivers/clk/nuvoton/clk-ma35d1.c
index 9f65d0623517..c914079cee2d 100644
--- a/drivers/clk/nuvoton/clk-ma35d1.c
+++ b/drivers/clk/nuvoton/clk-ma35d1.c
@@ -17,6 +17,7 @@
static DEFINE_SPINLOCK(ma35d1_lock);
#define PLL_MAX_NUM 5
+#define MA35D1_CLK_COUNT 238
/* Clock Control Registers Offset */
#define REG_CLK_PWRCTL 0x00
@@ -467,12 +468,12 @@ static int ma35d1_clocks_probe(struct platform_device *pdev)
int ret;
ma35d1_hw_data = devm_kzalloc(dev,
- struct_size(ma35d1_hw_data, hws, CLK_MAX_IDX),
+ struct_size(ma35d1_hw_data, hws, MA35D1_CLK_COUNT),
GFP_KERNEL);
if (!ma35d1_hw_data)
return -ENOMEM;
- ma35d1_hw_data->num = CLK_MAX_IDX;
+ ma35d1_hw_data->num = MA35D1_CLK_COUNT;
hws = ma35d1_hw_data->hws;
clk_base = devm_platform_ioremap_resource(pdev, 0);
@@ -543,6 +544,8 @@ static int ma35d1_clocks_probe(struct platform_device *pdev)
hws[HCLK3] = ma35d1_clk_fixed_factor(dev, "hclk3", "sysclk1_mux", 1, 2);
hws[PCLK3] = ma35d1_clk_fixed_factor(dev, "pclk3", "sysclk1_mux", 1, 2);
hws[PCLK4] = ma35d1_clk_fixed_factor(dev, "pclk4", "sysclk1_mux", 1, 2);
+ hws[PCLK3_DIV4096] = ma35d1_clk_fixed_factor(dev, "pclk3_div4096", "pclk3", 1, 4096);
+ hws[PCLK4_DIV4096] = ma35d1_clk_fixed_factor(dev, "pclk4_div4096", "pclk4", 1, 4096);
hws[USBPHY0] = ma35d1_clk_fixed("usbphy0", 480000000);
hws[USBPHY1] = ma35d1_clk_fixed("usbphy1", 480000000);
diff --git a/drivers/cpufreq/amd-pstate.c b/drivers/cpufreq/amd-pstate.c
index 6e00d200920a..28b9d4d062b9 100644
--- a/drivers/cpufreq/amd-pstate.c
+++ b/drivers/cpufreq/amd-pstate.c
@@ -1796,7 +1796,7 @@ EXPORT_SYMBOL_FOR_PSTATE_UT(amd_pstate_get_status);
int amd_pstate_update_status(const char *buf, size_t size)
{
- int mode_idx;
+ int cpu, mode_idx, ret = 0;
if (size > strlen("passive") || size < strlen("active"))
return -EINVAL;
@@ -1805,12 +1805,27 @@ int amd_pstate_update_status(const char *buf, size_t size)
if (mode_idx < 0)
return mode_idx;
- if (mode_state_machine[cppc_state][mode_idx]) {
- guard(mutex)(&amd_pstate_driver_lock);
- return mode_state_machine[cppc_state][mode_idx](mode_idx);
+ guard(mutex)(&amd_pstate_driver_lock);
+
+ if (!mode_state_machine[cppc_state][mode_idx])
+ return 0;
+
+ if (mode_idx == AMD_PSTATE_PASSIVE &&
+ !cpu_feature_enabled(X86_FEATURE_CPPC)) {
+ cpus_read_lock();
+ /* Check offline CPUs too, before changing or removing the driver. */
+ for_each_present_cpu(cpu) {
+ if (cppc_auto_sel_is_immutable(cpu)) {
+ ret = -EOPNOTSUPP;
+ break;
+ }
+ }
+ cpus_read_unlock();
+ if (ret)
+ return ret;
}
- return 0;
+ return mode_state_machine[cppc_state][mode_idx](mode_idx);
}
EXPORT_SYMBOL_FOR_PSTATE_UT(amd_pstate_update_status);
diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c
index 5f20c4a7ba2f..782b97acf5dc 100644
--- a/drivers/cpufreq/cppc_cpufreq.c
+++ b/drivers/cpufreq/cppc_cpufreq.c
@@ -42,6 +42,7 @@ MODULE_PARM_DESC(fie_disabled, "Disable Frequency Invariance Engine (FIE)");
/* Frequency invariance support */
struct cppc_freq_invariance {
int cpu;
+ bool pcc_work_initialized;
struct irq_work irq_work;
struct kthread_work work;
struct cppc_perf_fb_ctrs prev_perf_fb_ctrs;
@@ -152,7 +153,7 @@ static struct scale_freq_data cppc_sftd_pcc = {
static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy)
{
- struct scale_freq_data *sftd = &cppc_sftd;
+ struct scale_freq_data *sftd;
struct cppc_freq_invariance *cppc_fi;
int cpu, ret;
@@ -169,7 +170,7 @@ static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy)
kthread_init_work(&cppc_fi->work, cppc_scale_freq_workfn);
init_irq_work(&cppc_fi->irq_work, cppc_irq_work);
- sftd = &cppc_sftd_pcc;
+ cppc_fi->pcc_work_initialized = true;
}
ret = cppc_get_perf_ctrs(cpu, &cppc_fi->prev_perf_fb_ctrs);
@@ -185,8 +186,15 @@ static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy)
}
}
- /* Register for freq-invariance */
- topology_set_scale_freq_source(sftd, policy->cpus);
+ /* A shared policy may contain both PCC and non-PCC counters. */
+ for_each_cpu(cpu, policy->cpus) {
+ cppc_fi = &per_cpu(cppc_freq_inv, cpu);
+ if (cppc_fi->pcc_work_initialized)
+ sftd = &cppc_sftd_pcc;
+ else
+ sftd = &cppc_sftd;
+ topology_set_scale_freq_source(sftd, cpumask_of(cpu));
+ }
}
/*
@@ -209,11 +217,12 @@ static void cppc_cpufreq_cpu_fie_exit(struct cpufreq_policy *policy)
topology_clear_scale_freq_source(SCALE_FREQ_SOURCE_CPPC, policy->related_cpus);
for_each_cpu(cpu, policy->related_cpus) {
- if (!cppc_perf_ctrs_in_pcc_cpu(cpu))
- continue;
cppc_fi = &per_cpu(cppc_freq_inv, cpu);
+ if (!cppc_fi->pcc_work_initialized)
+ continue;
irq_work_sync(&cppc_fi->irq_work);
kthread_cancel_work_sync(&cppc_fi->work);
+ cppc_fi->pcc_work_initialized = false;
}
}
@@ -890,6 +899,7 @@ static ssize_t store_auto_select(struct cpufreq_policy *policy,
const char *buf, size_t count)
{
struct cppc_cpudata *cpu_data = policy->driver_data;
+ bool old_auto_sel = cpu_data->perf_ctrls.auto_sel;
bool val;
int ret;
@@ -918,8 +928,8 @@ static ssize_t store_auto_select(struct cpufreq_policy *policy,
if (ret) {
cpu_data->perf_ctrls.min_perf = old_min_perf;
cpu_data->perf_ctrls.max_perf = old_max_perf;
- cppc_set_auto_sel(policy->cpu, false);
- cpu_data->perf_ctrls.auto_sel = false;
+ cppc_set_auto_sel(policy->cpu, old_auto_sel);
+ cpu_data->perf_ctrls.auto_sel = old_auto_sel;
return ret;
}
}
diff --git a/drivers/cxl/core/features.c b/drivers/cxl/core/features.c
index ba6d2a5acb74..754fe5eb4b76 100644
--- a/drivers/cxl/core/features.c
+++ b/drivers/cxl/core/features.c
@@ -97,6 +97,7 @@ get_supported_features(struct cxl_features_state *cxlfs)
kvmalloc_flex(*entries, ent, count);
if (!entries)
return NULL;
+ entries->num_features = count;
struct cxl_mbox_get_sup_feats_out *mbox_out __free(kvfree) =
kvmalloc(cxl_mbox->payload_size, GFP_KERNEL);
@@ -174,7 +175,6 @@ get_supported_features(struct cxl_features_state *cxlfs)
start += num_entries;
} while (remain_feats);
- entries->num_features = count;
entries->num_user_features = user_feats;
return no_free_ptr(entries);
diff --git a/drivers/dma-buf/dma-buf-mapping.c b/drivers/dma-buf/dma-buf-mapping.c
index 833be519e1e6..ef0f22854d01 100644
--- a/drivers/dma-buf/dma-buf-mapping.c
+++ b/drivers/dma-buf/dma-buf-mapping.c
@@ -156,7 +156,7 @@ struct sg_table *dma_buf_phys_vec_to_sgt(struct dma_buf_attachment *attach,
phys_vec[i].paddr);
} else if (dma_use_iova(dma->state)) {
ret = dma_iova_link(attach->dev, dma->state,
- phys_vec[i].paddr, 0,
+ phys_vec[i].paddr, mapped_len,
phys_vec[i].len, dir,
DMA_ATTR_MMIO);
if (ret)
diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c
index 05a3b1f9e0c2..f67a93a37d48 100644
--- a/drivers/dma/qcom/bam_dma.c
+++ b/drivers/dma/qcom/bam_dma.c
@@ -1332,8 +1332,8 @@ static int bam_dma_probe(struct platform_device *pdev)
for (i = 0; i < bdev->num_channels; i++)
bam_channel_init(bdev, &bdev->channels[i], i);
- ret = devm_request_irq(bdev->dev, bdev->irq, bam_dma_irq,
- IRQF_TRIGGER_HIGH, "bam_dma", bdev);
+ ret = request_irq(bdev->irq, bam_dma_irq, IRQF_TRIGGER_NONE,
+ "bam_dma", bdev);
if (ret)
goto err_bam_channel_exit;
@@ -1366,7 +1366,7 @@ static int bam_dma_probe(struct platform_device *pdev)
ret = dma_async_device_register(&bdev->common);
if (ret) {
dev_err(bdev->dev, "failed to register dma async device\n");
- goto err_bam_channel_exit;
+ goto err_free_irq;
}
ret = of_dma_controller_register(pdev->dev.of_node, bam_dma_xlate,
@@ -1385,6 +1385,8 @@ static int bam_dma_probe(struct platform_device *pdev)
err_unregister_dma:
dma_async_device_unregister(&bdev->common);
+err_free_irq:
+ free_irq(bdev->irq, bdev);
err_bam_channel_exit:
for (i = 0; i < bdev->num_channels; i++)
tasklet_kill(&bdev->channels[i].vc.task);
@@ -1410,7 +1412,7 @@ static void bam_dma_remove(struct platform_device *pdev)
/* mask all interrupts for this execution environment */
writel_relaxed(0, bam_addr(bdev, 0, BAM_IRQ_SRCS_MSK_EE));
- devm_free_irq(bdev->dev, bdev->irq, bdev);
+ free_irq(bdev->irq, bdev);
for (i = 0; i < bdev->num_channels; i++) {
bam_dma_terminate_all(&bdev->channels[i].vc.chan);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
index 8b8acf98fdfe..53475f8a3ff1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
@@ -405,7 +405,11 @@ int amdgpu_atif_query_backlight_caps(struct amdgpu_atif *atif)
characteristics.max_input_signal;
atif->backlight_caps.ac_level = characteristics.ac_level;
atif->backlight_caps.dc_level = characteristics.dc_level;
- atif->backlight_caps.data_points = characteristics.number_of_points;
+ if (characteristics.number_of_points > ATIF_QBTC_MAX_DATA_POINTS)
+ pr_warn(FW_BUG "ATIF: backlight curve has %u points, using the first %u\n",
+ characteristics.number_of_points, ATIF_QBTC_MAX_DATA_POINTS);
+ atif->backlight_caps.data_points = min_t(u8, characteristics.number_of_points,
+ ATIF_QBTC_MAX_DATA_POINTS);
memcpy(atif->backlight_caps.luminance_data,
characteristics.data_points,
sizeof(atif->backlight_caps.luminance_data));
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index f462b40220eb..8e0b3db5c15e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -345,7 +345,8 @@ int amdgpu_amdkfd_gpuvm_map_bo_to_kernel(struct kgd_mem *mem, void **kptr,
u64 *size, u32 domain);
void amdgpu_amdkfd_gpuvm_unmap_bo_from_kernel(struct kgd_mem *mem);
-int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart);
+int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart,
+ struct amdkfd_process_info *process_info);
int amdgpu_amdkfd_gpuvm_restore_process_bos(void *process_info,
struct dma_fence __rcu **ef);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index c5c2a73ddd94..6c657d6060d6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -975,11 +975,11 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem,
ret = amdgpu_vm_lock_pd(vm, &exec, 0);
drm_exec_retry_on_contention(&exec);
if (unlikely(ret))
- goto unwind;
+ goto unlock;
ret = drm_exec_lock_obj(&exec, &bo[i]->tbo.base);
drm_exec_retry_on_contention(&exec);
if (unlikely(ret))
- goto unwind;
+ goto unlock;
}
bo_va = amdgpu_vm_bo_find(vm, bo[i]);
@@ -1005,6 +1005,8 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem,
return 0;
+unlock:
+ drm_exec_fini(&exec);
unwind:
for (; i >= 0; i--) {
if (!attachment[i])
@@ -2261,11 +2263,13 @@ int amdgpu_amdkfd_gpuvm_sync_memory(
* amdgpu_amdkfd_map_gtt_bo_to_gart - Map BO to GART and increment reference count
* @bo: Buffer object to be mapped
* @bo_gart: Return bo reference
+ * @process_info: KFD process info of the queue the BO is for
*
* Before return, bo reference count is incremented. To release the reference and unpin/
* unmap the BO, call amdgpu_amdkfd_free_kernel_mem.
*/
-int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart)
+int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart,
+ struct amdkfd_process_info *process_info)
{
int ret;
@@ -2287,8 +2291,8 @@ int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo
goto err_map_bo_gart_failed;
}
- amdgpu_amdkfd_remove_eviction_fence(
- bo, bo->vm_bo->vm->process_info->eviction_fence);
+ /* bo->vm_bo can be the VM of any file which opened the BO */
+ amdgpu_amdkfd_remove_eviction_fence(bo, process_info->eviction_fence);
amdgpu_bo_unreserve(bo);
@@ -2473,6 +2477,9 @@ static int import_obj_create(struct amdgpu_device *adev,
err_remove_mem:
remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info);
+ amdgpu_sync_free(&(*mem)->sync);
+ mutex_destroy(&(*mem)->lock);
+ dma_buf_put(dma_buf);
drm_vma_node_revoke(&obj->vma_node, drm_priv);
err_free_mem:
kfree(*mem);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 86a3dc3655b3..a03a15cfca96 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -149,7 +149,12 @@ static int amdgpu_cs_p1_user_fence(struct amdgpu_cs_parser *p,
if (size != PAGE_SIZE || data->offset > (size - 8))
return -EINVAL;
- if (amdgpu_ttm_tt_get_usermm(p->uf_bo->tbo.ttm))
+ if (amdgpu_ttm_tt_is_userptr(p->uf_bo->tbo.ttm))
+ return -EINVAL;
+
+ /* The GPU writes the user fence through the GART */
+ if (p->uf_bo->preferred_domains != AMDGPU_GEM_DOMAIN_GTT &&
+ p->uf_bo->allowed_domains != AMDGPU_GEM_DOMAIN_GTT)
return -EINVAL;
*offset = data->offset;
@@ -188,18 +193,22 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p,
int ret;
int i;
+ if (cs->in.flags)
+ return -EINVAL;
+
chunk_array = memdup_array_user(u64_to_user_ptr(cs->in.chunks),
cs->in.num_chunks,
sizeof(uint64_t));
if (IS_ERR(chunk_array))
return PTR_ERR(chunk_array);
- p->nchunks = cs->in.num_chunks;
- p->chunks = kvmalloc_objs(struct amdgpu_cs_chunk, p->nchunks);
+ /* parser_fini() walks nchunks chunks, set it once they exist */
+ p->chunks = kvmalloc_objs(struct amdgpu_cs_chunk, cs->in.num_chunks);
if (!p->chunks) {
ret = -ENOMEM;
goto free_chunk;
}
+ p->nchunks = cs->in.num_chunks;
for (i = 0; i < p->nchunks; i++) {
struct drm_amdgpu_cs_chunk __user *chunk_ptr = NULL;
@@ -1071,7 +1080,7 @@ static int amdgpu_cs_patch_ibs(struct amdgpu_cs_parser *p,
if (r)
return r;
- kptr += va_start - (m->start * AMDGPU_GPU_PAGE_SIZE);
+ kptr += va_start - (m->start * AMDGPU_GPU_PAGE_SIZE) + m->offset;
if (ring->funcs->parse_cs) {
memcpy(ib->ptr, kptr, ib->length_dw * 4);
@@ -1309,10 +1318,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
fence = &p->jobs[i]->base.s_fence->scheduled;
dma_fence_get(fence);
r = drm_sched_job_add_dependency(&leader->base, fence);
- if (r) {
- dma_fence_put(fence);
+ if (r)
return r;
- }
}
if (p->gang_size > 1) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 5b72e4afb650..948886fffb4a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -5515,7 +5515,21 @@ static void amdgpu_device_halt_activities(struct amdgpu_device *adev,
struct amdgpu_device *tmp_adev = NULL;
int i;
- /* block all schedulers and reset given job's ring */
+ /* Stop job submission first, the preparation below can take seconds */
+ list_for_each_entry(tmp_adev, device_list, reset_list) {
+ for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
+ struct amdgpu_ring *ring = tmp_adev->rings[i];
+
+ if (!amdgpu_ring_sched_ready(ring))
+ continue;
+
+ drm_sched_wqueue_stop(&ring->sched);
+
+ if (need_emergency_restart)
+ amdgpu_job_stop_all_jobs_on_sched(&ring->sched);
+ }
+ }
+
list_for_each_entry(tmp_adev, device_list, reset_list) {
amdgpu_device_set_mp1_state(tmp_adev);
@@ -5551,18 +5565,6 @@ static void amdgpu_device_halt_activities(struct amdgpu_device *adev,
amdgpu_ras_suspend(tmp_adev);
amdgpu_userq_pre_reset(tmp_adev);
-
- for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
- struct amdgpu_ring *ring = tmp_adev->rings[i];
-
- if (!amdgpu_ring_sched_ready(ring))
- continue;
-
- drm_sched_wqueue_stop(&ring->sched);
-
- if (need_emergency_restart)
- amdgpu_job_stop_all_jobs_on_sched(&ring->sched);
- }
atomic_inc(&tmp_adev->gpu_reset_counter);
}
}
@@ -5797,8 +5799,10 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
amdgpu_device_halt_activities(adev, job, reset_context, &device_list,
hive, need_emergency_restart);
- if (need_emergency_restart)
- goto skip_sched_resume;
+ if (need_emergency_restart) {
+ amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart);
+ goto reset_unlock;
+ }
/*
* Must check guilty signal here since after this point all old
* HW fences are force signaled.
@@ -5808,18 +5812,17 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
if (job && (dma_fence_get_status(&job->hw_fence->base) > 0)) {
job_signaled = true;
dev_info(adev->dev, "Guilty job already signaled, skipping HW reset");
- goto skip_hw_reset;
+ goto post_resume;
}
r = amdgpu_device_asic_reset(adev, &device_list, reset_context);
if (r)
goto reset_unlock;
-skip_hw_reset:
+post_resume:
+ amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart);
r = amdgpu_device_sched_resume(&device_list, reset_context, job_signaled);
if (r)
goto reset_unlock;
-skip_sched_resume:
- amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart);
reset_unlock:
amdgpu_device_recovery_put_reset_lock(adev, &device_list);
amdgpu_ras_post_reset(adev, &device_list);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
index 7ab2ad2c5ba9..6d9a3e3b3546 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
@@ -1402,6 +1402,40 @@ static int amdgpu_discovery_sysfs_recurse(struct amdgpu_device *adev,
return 0;
}
+#define list_to_kobj(el) container_of(el, struct kobject, entry)
+
+static void amdgpu_discovery_sysfs_update_harvest(struct amdgpu_device *adev)
+{
+ struct ip_discovery_top *ip_top = adev->discovery.ip_top;
+ struct ip_die_entry *ip_die_entry;
+ struct list_head *el_die, *el_hw_id, *el_hw_inst;
+ struct ip_hw_id *hw_id;
+ struct kset *die_kset;
+ struct ip_hw_instance *ip_inst;
+
+ if (!ip_top)
+ return;
+
+ die_kset = &ip_top->die_kset;
+
+ guard(spinlock)(&die_kset->list_lock);
+ list_for_each(el_die, &die_kset->list) {
+ ip_die_entry = to_ip_die_entry(list_to_kobj(el_die));
+
+ list_for_each(el_hw_id, &ip_die_entry->ip_kset.list) {
+ hw_id = to_ip_hw_id(list_to_kobj(el_hw_id));
+
+ list_for_each(el_hw_inst, &hw_id->hw_id_kset.list) {
+ ip_inst = to_ip_hw_instance(list_to_kobj(el_hw_inst));
+ ip_inst->harvest =
+ amdgpu_discovery_get_harvest_info(
+ adev, ip_inst->hw_id,
+ ip_inst->num_instance);
+ }
+ }
+ }
+}
+
static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev)
{
uint8_t *discovery_bin = adev->discovery.bin;
@@ -1412,9 +1446,13 @@ static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev)
if (!discovery_bin)
return -EINVAL;
- /* If early init already created sysfs in standalone mode, skip normal init */
- if (adev->discovery.ip_top && adev->discovery.ip_top->standalone_mode)
+ /* If early init already created sysfs in standalone mode, update harvest info
+ * now that adev is fully initialized, then skip the rest of normal init.
+ */
+ if (adev->discovery.ip_top && adev->discovery.ip_top->standalone_mode) {
+ amdgpu_discovery_sysfs_update_harvest(adev);
return 0;
+ }
ip_top = kzalloc_obj(*ip_top);
if (!ip_top)
@@ -1469,8 +1507,6 @@ static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev)
/* -------------------------------------------------- */
-#define list_to_kobj(el) container_of(el, struct kobject, entry)
-
static void amdgpu_discovery_sysfs_ip_hw_free(struct ip_hw_id *ip_hw_id)
{
struct list_head *el, *tmp;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
index 9adf3eed8822..ccadbcf38d3f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
@@ -547,6 +547,11 @@ amdgpu_dma_buf_move_notify(struct dma_buf_attachment *attach)
continue;
}
+ if (!amdgpu_vm_ready(vm)) {
+ dma_resv_unlock(resv);
+ continue;
+ }
+
/* Reserve fences for two SDMA page table updates */
r = dma_resv_reserve_fences(resv, 2);
if (!r)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index afd8502a2bc4..cd05921aede5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -1033,6 +1033,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data,
struct drm_file *filp)
{
+ struct amdgpu_device *adev = drm_to_adev(dev);
struct drm_amdgpu_gem_op *args = data;
struct drm_gem_object *gobj;
struct amdgpu_vm_bo_base *base;
@@ -1139,7 +1140,8 @@ int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data,
amdgpu_vm_bo_va_for_each_valid_mapping(bo_va, mapping) {
if (num_mappings < args->num_entries) {
- vm_entries[num_mappings].addr = mapping->start * AMDGPU_GPU_PAGE_SIZE;
+ vm_entries[num_mappings].addr =
+ amdgpu_gmc_sign_extend(mapping->start * AMDGPU_GPU_PAGE_SIZE);
vm_entries[num_mappings].size = (mapping->last - mapping->start + 1) * AMDGPU_GPU_PAGE_SIZE;
vm_entries[num_mappings].offset = mapping->offset;
vm_entries[num_mappings].flags = mapping->flags;
@@ -1149,7 +1151,8 @@ int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data,
amdgpu_vm_bo_va_for_each_invalid_mapping(bo_va, mapping) {
if (num_mappings < args->num_entries) {
- vm_entries[num_mappings].addr = mapping->start * AMDGPU_GPU_PAGE_SIZE;
+ vm_entries[num_mappings].addr =
+ amdgpu_gmc_sign_extend(mapping->start * AMDGPU_GPU_PAGE_SIZE);
vm_entries[num_mappings].size = (mapping->last - mapping->start + 1) * AMDGPU_GPU_PAGE_SIZE;
vm_entries[num_mappings].offset = mapping->offset;
vm_entries[num_mappings].flags = mapping->flags;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
index 35eb87591740..37cf1b808950 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
@@ -1485,7 +1485,8 @@ int amdgpu_ras_reset_error_count(struct amdgpu_device *adev,
return -EOPNOTSUPP;
}
- if (!amdgpu_ras_is_supported(adev, block))
+ if (!amdgpu_ras_is_supported(adev, block) ||
+ !amdgpu_ras_mgr_get_debug_mode(adev))
return -EOPNOTSUPP;
if (amdgpu_sriov_vf(adev))
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
index d6ae9974c952..4f31021b2152 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
@@ -447,10 +447,8 @@ int amdgpu_sync_push_to_job(struct amdgpu_sync *sync, struct amdgpu_job *job)
dma_fence_get(f);
r = drm_sched_job_add_dependency(&job->base, f);
- if (r) {
- dma_fence_put(f);
+ if (r)
return r;
- }
}
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
index b3dc5dd49a64..053ead0d690f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
@@ -365,6 +365,8 @@ static int amdgpu_userq_unmap_helper(struct amdgpu_usermode_queue *queue)
adev->userq_funcs[queue->queue_type];
int r;
+ /* The reset changes the queue state and the MES is unusable meanwhile */
+ guard(rwsem_read)(&adev->reset_domain->sem);
if ((queue->state == AMDGPU_USERQ_STATE_MAPPED) ||
(queue->state == AMDGPU_USERQ_STATE_PREEMPTED)) {
trace_amdgpu_userq_state_start(queue);
@@ -391,6 +393,8 @@ static int amdgpu_userq_map_helper(struct amdgpu_usermode_queue *queue)
adev->userq_funcs[queue->queue_type];
int r;
+ /* The reset changes the queue state and the MES is unusable meanwhile */
+ guard(rwsem_read)(&adev->reset_domain->sem);
if (queue->state == AMDGPU_USERQ_STATE_UNMAPPED) {
trace_amdgpu_userq_state_start(queue);
@@ -751,12 +755,7 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args)
if (!adev->userq_halt_for_enforce_isolation ||
((queue->queue_type != AMDGPU_HW_IP_GFX) &&
(queue->queue_type != AMDGPU_HW_IP_COMPUTE))) {
- /* Serialize the map against an in-progress GPU reset (MES is
- * unresponsive during recovery), matching amdgpu_userq_detach_doorbell().
- */
- down_read(&adev->reset_domain->sem);
r = amdgpu_userq_map_helper(queue);
- up_read(&adev->reset_domain->sem);
if (r) {
drm_file_err(uq_mgr->file, "Failed to map Queue\n");
trace_amdgpu_userq_create_end(queue, r);
@@ -973,7 +972,11 @@ static int amdgpu_userq_validate_vm(void *param, struct amdgpu_bo *bo)
return ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
}
-/* Handle all BOs on the invalidated list, validate them and update the PTs */
+/*
+ * Handle all individual BOs which were moved, lock and validate them and
+ * update the PTs. An evicted individual BO lands on the needs_update list,
+ * the evicted one only holds imported DMA-bufs of compute VMs.
+ */
static int
amdgpu_userq_bo_validate(struct amdgpu_device *adev, struct drm_exec *exec,
struct amdgpu_vm *vm)
@@ -984,8 +987,8 @@ amdgpu_userq_bo_validate(struct amdgpu_device *adev, struct drm_exec *exec,
int ret;
spin_lock(&vm->individual_lock);
- while (!list_empty(&vm->individual.evicted)) {
- bo_va = list_first_entry(&vm->individual.evicted,
+ while (!list_empty(&vm->individual.needs_update)) {
+ bo_va = list_first_entry(&vm->individual.needs_update,
struct amdgpu_bo_va,
base.vm_status);
bo = bo_va->base.bo;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index fc3a8d38d514..c07f1115f654 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -269,19 +269,27 @@ static void amdgpu_vm_bo_reset_state_machine(struct amdgpu_vm *vm)
/*
* Don't use list splice here, we need the special handling for the root
- * PD and set the moved flag appropriately.
+ * PD. All mappings must be written again, also of the vm_bos which are
+ * already waiting for an update of some of them, e.g. after a map with
+ * a delayed update, so set the moved flag for everything that needs an
+ * update.
*/
amdgpu_vm_assert_locked(vm);
list_for_each_entry_safe(vm_bo, tmp, &vm->kernel.idle, vm_status)
amdgpu_vm_bo_needs_update(vm_bo);
+ list_for_each_entry(vm_bo, &vm->kernel.needs_update, vm_status)
+ vm_bo->moved = true;
+
list_for_each_entry_safe(vm_bo, tmp, &vm->always_valid.idle, vm_status)
amdgpu_vm_bo_needs_update(vm_bo);
+ list_for_each_entry(vm_bo, &vm->always_valid.needs_update, vm_status)
+ vm_bo->moved = true;
spin_lock(&vm->individual_lock);
- list_for_each_entry_safe(vm_bo, tmp, &vm->individual.idle, vm_status) {
- vm_bo->moved = true;
+ list_for_each_entry_safe(vm_bo, tmp, &vm->individual.idle, vm_status)
list_move(&vm_bo->vm_status, &vm->individual.needs_update);
- }
+ list_for_each_entry(vm_bo, &vm->individual.needs_update, vm_status)
+ vm_bo->moved = true;
spin_unlock(&vm->individual_lock);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
index eb8cf0d9d9c5..652266e93766 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
@@ -234,7 +234,6 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p,
dma_fence_get(fence);
r = drm_sched_job_add_dependency(&p->job->base, fence);
if (r) {
- dma_fence_put(fence);
dma_resv_iter_end(&cursor);
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
index f2977fe6d824..04b18056a906 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
@@ -1705,8 +1705,9 @@ static void dce_v10_0_afmt_setmode(struct drm_encoder *encoder,
WREG32(mmAFMT_RAMP_CONTROL2 + dig->afmt->offset, 0x00000001);
WREG32(mmAFMT_RAMP_CONTROL3 + dig->afmt->offset, 0x00000001);
- /* enable audio after to setting up hw */
- dce_v10_0_audio_enable(adev, dig->afmt->pin, true);
+ /* Only enable audio if the display supports it */
+ if (connector->display_info.has_audio)
+ dce_v10_0_audio_enable(adev, dig->afmt->pin, true);
}
static void dce_v10_0_afmt_enable(struct drm_encoder *encoder, bool enable)
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
index c68de0fe1d7d..585b0a50279f 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
@@ -1748,8 +1748,9 @@ static void dce_v6_0_afmt_setmode(struct drm_encoder *encoder,
dce_v6_0_audio_dp_enable(encoder, 1);
}
- /* enable audio after setting up hw */
- dce_v6_0_audio_enable(adev, dig->afmt->pin, true);
+ /* Only enable audio if the display supports it */
+ if (connector->display_info.has_audio)
+ dce_v6_0_audio_enable(adev, dig->afmt->pin, true);
}
static void dce_v6_0_afmt_enable(struct drm_encoder *encoder, bool enable)
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
index c3906270f25e..1390efa4ea51 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
@@ -1653,8 +1653,9 @@ static void dce_v8_0_afmt_setmode(struct drm_encoder *encoder,
WREG32(mmAFMT_RAMP_CONTROL2 + offset, 0x00000001);
WREG32(mmAFMT_RAMP_CONTROL3 + offset, 0x00000001);
- /* enable audio after setting up hw */
- dce_v8_0_audio_enable(adev, dig->afmt->pin, true);
+ /* Only enable audio if the display supports it */
+ if (connector->display_info.has_audio)
+ dce_v8_0_audio_enable(adev, dig->afmt->pin, true);
}
static void dce_v8_0_afmt_enable(struct drm_encoder *encoder, bool enable)
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
index d69c43f1e194..60debaa23762 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
@@ -4080,6 +4080,7 @@ static void gfx_v12_0_update_spm_vmid(struct amdgpu_device *adev,
{
u32 reg, data;
+ amdgpu_gfx_off_ctrl(adev, false);
reg = SOC15_REG_OFFSET(GC, 0, regRLC_SPM_MC_CNTL);
if (amdgpu_sriov_is_pp_one_vf(adev))
data = RREG32_NO_KIQ(reg);
@@ -4093,6 +4094,7 @@ static void gfx_v12_0_update_spm_vmid(struct amdgpu_device *adev,
WREG32_SOC15_NO_KIQ(GC, 0, regRLC_SPM_MC_CNTL, data);
else
WREG32_SOC15(GC, 0, regRLC_SPM_MC_CNTL, data);
+ amdgpu_gfx_off_ctrl(adev, true);
if (ring
&& amdgpu_sriov_is_pp_one_vf(adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
index 333e9c30c091..535c84e55558 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
@@ -168,8 +168,19 @@ static int ih_v6_0_toggle_ring_interrupts(struct amdgpu_device *adev,
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0));
if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
- if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ /*
+ * PSP programming of IH_RB_CNTL is slow enough for MSI to
+ * arrive before this call returns. Publish ih->enabled first
+ * so amdgpu_ih_process() consumes that IV and rearms.
+ */
+ if (enable) {
+ ih->enabled = true;
+ smp_wmb();
+ }
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) {
+ ih->enabled = false;
return -ETIMEDOUT;
+ }
} else {
WREG32(ih_regs->ih_rb_cntl, tmp);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
index c57bcb9a98b1..10bed6f38f30 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
@@ -218,8 +218,10 @@ static int mes_v11_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes,
r = amdgpu_fence_wait_polling(ring,
seq - ring->fence_drv.num_fences_mask,
timeout);
- if (r < 1)
+ if (r < 1) {
+ r = -ETIMEDOUT;
goto error_undo;
+ }
api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off);
api_status->api_completion_fence_addr = status_gpu_addr;
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
index ee72afdb0a1d..bb9892d1bd05 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
@@ -195,8 +195,10 @@ static int mes_v12_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes,
r = amdgpu_fence_wait_polling(ring,
seq - ring->fence_drv.num_fences_mask,
timeout);
- if (r < 1)
+ if (r < 1) {
+ r = -ETIMEDOUT;
goto error_undo;
+ }
api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off);
api_status->api_completion_fence_addr = status_gpu_addr;
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
index 4cd325149b63..bed0eb4b6ec3 100644
--- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
@@ -161,13 +161,52 @@ static int navi10_ih_toggle_ring_interrupts(struct amdgpu_device *adev,
tmp = RREG32(ih_regs->ih_rb_cntl);
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_ENABLE, (enable ? 1 : 0));
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_GPU_TS_ENABLE, 1);
+
+ if (enable) {
+ /* Unset the CLEAR_OVERFLOW bit to make sure the next step
+ * is switching the bit from 0 to 1
+ */
+ tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 0);
+ if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ return -ETIMEDOUT;
+ } else {
+ WREG32_NO_KIQ(ih_regs->ih_rb_cntl, tmp);
+ }
+
+ /* Clear RB_OVERFLOW bit */
+ tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 1);
+ if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ return -ETIMEDOUT;
+ } else {
+ WREG32_NO_KIQ(ih_regs->ih_rb_cntl, tmp);
+ }
+
+ /* Unset the CLEAR_OVERFLOW bit immediately so new overflows
+ * can be detected.
+ */
+ tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 0);
+ }
+
/* enable_intr field is only valid in ring0 */
if (ih == &adev->irq.ih)
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0));
if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
- if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ /*
+ * PSP programming of IH_RB_CNTL is slow enough for MSI to
+ * arrive before this call returns. Publish ih->enabled first
+ * so amdgpu_ih_process() consumes that IV and rearms.
+ */
+ if (enable) {
+ ih->enabled = true;
+ smp_wmb();
+ }
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) {
+ ih->enabled = false;
return -ETIMEDOUT;
+ }
} else {
WREG32(ih_regs->ih_rb_cntl, tmp);
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
index 87fb42d3c713..64673007ccde 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
@@ -303,7 +303,8 @@ static int init_user_queue(struct process_queue_manager *pqm,
}
retval = amdgpu_amdkfd_map_gtt_bo_to_gart(q_properties->wptr_bo,
- &(*q)->wptr_bo_gart);
+ &(*q)->wptr_bo_gart,
+ pqm->process->kgd_process_info);
if (retval) {
pr_err("Failed to map wptr bo to GART\n");
goto free_gang_ctx_bo;
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_freesync.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_freesync.c
index 7e484adb4120..a51cf46d0e44 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_freesync.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_freesync.c
@@ -128,6 +128,14 @@ void amdgpu_dm_get_freesync_config_for_crtc(
vrefresh >= aconnector->min_vfreq &&
vrefresh <= aconnector->max_vfreq;
+ /*
+ * DC reads allow_freesync while validating the new state, for example
+ * to keep SubVP off a single FreeSync display, but it is otherwise set
+ * only on the first flip after the commit. Set it here, so that a
+ * stream created by this commit is not validated without it.
+ */
+ new_crtc_state->stream->allow_freesync = new_crtc_state->vrr_supported;
+
if (new_crtc_state->vrr_supported) {
new_crtc_state->stream->ignore_msa_timing_param = true;
fs_vid_mode = new_crtc_state->freesync_config.state == VRR_STATE_ACTIVE_FIXED;
@@ -154,6 +162,18 @@ void amdgpu_dm_get_freesync_config_for_crtc(
}
EXPORT_IF_KUNIT(amdgpu_dm_get_freesync_config_for_crtc);
+#define DP_HDR_SDP_EOTF_IDX 2
+
+static bool dm_dp_in_hdr_mode(const struct dc_stream_state *stream)
+{
+ const struct dc_info_packet *hdr = &stream->hdr_static_metadata;
+
+ if (!dc_is_dp_signal(stream->signal) || !hdr->valid)
+ return false;
+
+ return hdr->sb[DP_HDR_SDP_EOTF_IDX] == HDMI_EOTF_SMPTE_ST2084;
+}
+
void amdgpu_dm_update_freesync_state_on_stream(
struct amdgpu_display_manager *dm,
struct dm_crtc_state *new_crtc_state,
@@ -169,6 +189,7 @@ void amdgpu_dm_update_freesync_state_on_stream(
bool pack_sdp_v1_3 = false;
struct amdgpu_dm_connector *aconn;
enum vrr_packet_type packet_type = PACKET_TYPE_VRR;
+ enum color_transfer_func app_tf;
if (!new_stream)
return;
@@ -220,12 +241,16 @@ void amdgpu_dm_update_freesync_state_on_stream(
&new_stream->adaptive_sync_infopacket);
}
+ app_tf = dm_dp_in_hdr_mode(new_stream) ?
+ TRANSFER_FUNC_PQ2084 :
+ TRANSFER_FUNC_UNKNOWN;
+
mod_freesync_build_vrr_infopacket(
dm->freesync_module,
new_stream,
&vrr_params,
packet_type,
- TRANSFER_FUNC_UNKNOWN,
+ app_tf,
&vrr_infopacket,
pack_sdp_v1_3);
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c
index 6ee3da89d058..135eeef68ac4 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c
@@ -538,6 +538,9 @@ static void dcn401_update_clocks_update_dtb_dto(struct clk_mgr_internal *clk_mgr
otg_master = resource_get_otg_master_for_stream(
&context->res_ctx, context->streams[i]);
ASSERT(otg_master);
+ if (!otg_master)
+ continue;
+
ASSERT(otg_master->clock_source);
ASSERT(otg_master->clock_source->funcs->program_pix_clk);
ASSERT(otg_master->stream_res.pix_clk_params.controller_id >= CONTROLLER_ID_D0);
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc.c b/drivers/gpu/drm/amd/display/dc/core/dc.c
index 519ac878ada1..60b903edff2f 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc.c
@@ -2452,6 +2452,8 @@ static enum dc_status dc_commit_state_no_check(struct dc *dc, struct dc_state *c
* update pending complete before optimizing bandwidth.
*/
hwss_wait_for_odm_update_pending_complete(dc, context);
+ /* No DET may stay allocated to a HUBP without a plane (e.g. left over from VBIOS) */
+ hwss_reset_unused_det(dc, context);
/* pplib is notified if disp_num changed */
dc->hwss.optimize_bandwidth(dc, context);
/* Need to do otg sync again as otg could be out of sync due to otg
@@ -4885,8 +4887,10 @@ static void commit_planes_for_stream(struct dc *dc,
dc_exit_ips_for_hw_access(dc);
dc_z10_restore(dc);
- if (update_type == UPDATE_TYPE_FULL && dc->optimized_required)
+ if (update_type == UPDATE_TYPE_FULL && dc->optimized_required) {
+ hwss_reset_unused_det(dc, dc->current_state);
hwss_process_outstanding_hw_updates(dc, dc->current_state);
+ }
if (update_type != UPDATE_TYPE_FAST && dc->res_pool->funcs->prepare_mcache_programming)
dc->res_pool->funcs->prepare_mcache_programming(dc, context);
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c
index ef4987d449cb..d4dbdf7d0506 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c
@@ -3047,6 +3047,37 @@ void hwss_wait_for_odm_update_pending_complete(struct dc *dc, struct dc_state *c
hwss_wait_for_all_blank_complete(dc, context);
}
+void hwss_reset_unused_det(struct dc *dc, struct dc_state *context)
+{
+ struct hubbub *hubbub = dc->res_pool->hubbub;
+ unsigned int i, j;
+
+ if (!hubbub || !hubbub->funcs->program_det_segments)
+ return;
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct hubp *hubp = dc->res_pool->hubps[i];
+ bool hubp_in_use = false;
+
+ if (!hubp)
+ continue;
+
+ for (j = 0; j < dc->res_pool->pipe_count; j++) {
+ const struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[j];
+
+ /* A plane-less pipe renders from the DPG, its HUBP (assigned with the stream) needs no DET */
+ if (pipe->plane_state && pipe->plane_res.hubp &&
+ pipe->plane_res.hubp->inst == hubp->inst) {
+ hubp_in_use = true;
+ break;
+ }
+ }
+
+ if (!hubp_in_use)
+ hubbub->funcs->program_det_segments(hubbub, hubp->inst, 0);
+ }
+}
+
void hwss_wait_for_no_pipes_pending(struct dc *dc, struct dc_state *context)
{
int i;
diff --git a/drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h b/drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h
index cdf2562a9a40..0d64a0c1913d 100644
--- a/drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h
+++ b/drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h
@@ -1807,6 +1807,9 @@ void hwss_wait_for_odm_update_pending_complete(struct dc *dc,
void hwss_wait_for_no_pipes_pending(struct dc *dc,
struct dc_state *context);
+void hwss_reset_unused_det(struct dc *dc,
+ struct dc_state *context);
+
void hwss_wait_for_outstanding_hw_updates(struct dc *dc,
struct dc_state *dc_context);
diff --git a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c
index 24e3bbec15ff..8ff364d465de 100644
--- a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c
+++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c
@@ -2601,12 +2601,8 @@ bool dp_is_sink_present(struct dc_link *link)
* which indicates we need additional delay
*/
- if (dal_ddc_open(ddc, GPIO_MODE_INPUT,
- GPIO_DDC_CONFIG_TYPE_MODE_I2C) != GPIO_RESULT_OK) {
- dal_ddc_close(ddc);
-
+ if (dal_ddc_open(ddc, GPIO_MODE_INPUT, GPIO_DDC_CONFIG_TYPE_MODE_I2C) != GPIO_RESULT_OK)
return present;
- }
/*
* Read GPIO: DP sink is present if both clock and data pins are zero
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c b/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c
index 73cc8a713556..2cdfe3d7a0ea 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c
@@ -449,6 +449,13 @@ void optc2_lock_doublebuffer_disable(struct timing_generator *optc)
void optc2_setup_manual_trigger(struct timing_generator *optc)
{
struct optc *optc1 = DCN10TG_FROM_TG(optc);
+ uint32_t v_total, v_total_max;
+
+ REG_GET(OTG_V_TOTAL_MAX, OTG_V_TOTAL_MAX, &v_total_max);
+ if (!v_total_max) {
+ REG_GET(OTG_V_TOTAL, OTG_V_TOTAL, &v_total);
+ optc->funcs->set_vtotal_min_max(optc, v_total, v_total);
+ }
/* Set the min/max selectors unconditionally so that
* DMCUB fw may change OTG timings when necessary
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn30/dcn30_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn30/dcn30_optc.h
index 16c5610b49ac..b801989f10cf 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn30/dcn30_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn30/dcn30_optc.h
@@ -91,6 +91,12 @@
SRI(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst),\
SRI(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst),\
SRI(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_DATA_RG, OTG, inst),\
+ SRI(OTG_CRC1_DATA_B, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst),\
SR(GSL_SOURCE_SELECT),\
SRI(OTG_TRIGA_MANUAL_TRIG, OTG, inst),\
SRI(OTG_DRR_CONTROL, OTG, inst)
@@ -229,6 +235,7 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL2, OTG_CRC_DSC_MODE, mask_sh),\
SF(OTG0_OTG_CRC_CNTL2, OTG_CRC_DATA_STREAM_COMBINE_MODE, mask_sh),\
@@ -245,6 +252,17 @@
SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\
SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h
index 119a954fe3e5..85223414c909 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h
@@ -83,6 +83,12 @@
SRI(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst),\
SRI(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst),\
SRI(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_DATA_RG, OTG, inst),\
+ SRI(OTG_CRC1_DATA_B, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst),\
SR(GSL_SOURCE_SELECT),\
SRI(OTG_TRIGA_MANUAL_TRIG, OTG, inst),\
SRI(OTG_GLOBAL_CONTROL1, OTG, inst),\
@@ -205,7 +211,9 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_RG, CRC0_R_CR, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_RG, CRC0_G_Y, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_B, CRC0_B_CB, mask_sh),\
@@ -217,6 +225,17 @@
SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\
SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\
@@ -264,8 +283,7 @@
SF(OTG0_INTERRUPT_DEST, OTG0_IHC_OTG_VERTICAL_INTERRUPT2_DEST, mask_sh)
#define OPTC_COMMON_MASK_SH_LIST_DCN31X(mask_sh)\
- OPTC_COMMON_MASK_SH_LIST_DCN3_1(mask_sh),\
- SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh)
+ OPTC_COMMON_MASK_SH_LIST_DCN3_1(mask_sh)
void dcn31_timing_generator_init(struct optc *optc1);
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.h
index 7b43c3ecd46b..956eecf0a43a 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.h
@@ -84,6 +84,12 @@
SRI(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst),\
SRI(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst),\
SRI(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_DATA_RG, OTG, inst),\
+ SRI(OTG_CRC1_DATA_B, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst),\
SR(GSL_SOURCE_SELECT),\
SRI(OTG_TRIGA_MANUAL_TRIG, OTG, inst),\
SRI(OTG_GLOBAL_CONTROL1, OTG, inst),\
@@ -204,6 +210,7 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_RG, CRC0_R_CR, mask_sh),\
@@ -217,6 +224,17 @@
SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\
SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.h
index 60c01ec28b65..492760779226 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.h
@@ -146,6 +146,17 @@
SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\
SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.h
index 82e0818c8f9f..196596cc9fb6 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.h
@@ -32,23 +32,12 @@
#define OPTC_COMMON_MASK_SH_LIST_DCN3_5(mask_sh)\
OPTC_COMMON_MASK_SH_LIST_DCN3_2(mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_WINDOW_DB_EN, mask_sh),\
- SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\
- SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\
- SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\
SF(OTG0_OTG_CRC2_DATA_RG, CRC2_R_CR, mask_sh),\
SF(OTG0_OTG_CRC2_DATA_RG, CRC2_G_Y, mask_sh),\
SF(OTG0_OTG_CRC2_DATA_B, CRC2_B_CB, mask_sh),\
SF(OTG0_OTG_CRC3_DATA_RG, CRC3_R_CR, mask_sh),\
SF(OTG0_OTG_CRC3_DATA_RG, CRC3_G_Y, mask_sh),\
SF(OTG0_OTG_CRC3_DATA_B, CRC3_B_CB, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWA_X_CONTROL_READBACK, OTG_CRC0_WINDOWA_X_START_READBACK, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWA_X_CONTROL_READBACK, OTG_CRC0_WINDOWA_X_END_READBACK, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWA_Y_CONTROL_READBACK, OTG_CRC0_WINDOWA_Y_START_READBACK, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.h
index 3add972ca2ff..9e4e9d75400a 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.h
@@ -110,6 +110,7 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_RG, CRC0_R_CR, mask_sh),\
@@ -123,6 +124,17 @@
SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\
SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn42/dcn42_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn42/dcn42_optc.h
index bc641f6ecd7e..bdda94489a23 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn42/dcn42_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn42/dcn42_optc.h
@@ -137,6 +137,7 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_R, CRC0_R_CR, mask_sh),\
@@ -316,6 +317,7 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_R, CRC0_R_CR, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn60/dcn60_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn60/dcn60_optc.h
index ba207c0e498e..ec40c9180030 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn60/dcn60_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn60/dcn60_optc.h
@@ -109,7 +109,9 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_FORCE_DELAY, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_R, CRC0_R_CR, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_G, CRC0_G_Y, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_B, CRC0_B_CB, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
index a2e66761b8ec..81227df68680 100644
--- a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
@@ -1941,8 +1941,21 @@ static bool dml1_validate(struct dc *dc, struct dc_state *context, enum dc_valid
out = dcn32_internal_validate_bw(dc, context, pipes, &pipe_cnt, &vlevel, validate_mode);
DC_FP_END();
- if (pipe_cnt == 0)
+ if (pipe_cnt == 0) {
+ /* The context is a copy of the previous state, so without this the
+ * clocks the last display needed stay applied with all displays off.
+ */
+ context->bw_ctx.bw.dcn.clk.socclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dppclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dcfclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dcfclk_deep_sleep_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dramclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.fclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.p_state_change_support = true;
+ context->bw_ctx.bw.dcn.clk.fclk_p_state_change_support = true;
+ context->bw_ctx.bw.dcn.clk.num_ways = 0;
goto validate_out;
+ }
if (!out)
goto validate_fail;
diff --git a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.h b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.h
index 68e7140f1505..fa8cc89cb3f7 100644
--- a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.h
+++ b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.h
@@ -1054,6 +1054,12 @@ unsigned int dcn32_get_max_hw_cursor_size(const struct dc *dc,
SRI_ARR(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst), \
SRI_ARR(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst), \
SRI_ARR(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_DATA_RG, OTG, inst), \
+ SRI_ARR(OTG_CRC1_DATA_B, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst), \
SR_ARR(GSL_SOURCE_SELECT, inst), \
SRI_ARR(OTG_TRIGA_MANUAL_TRIG, OTG, inst), \
SRI_ARR(OTG_GLOBAL_CONTROL1, OTG, inst), \
diff --git a/drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h b/drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h
index 8e3473f86490..3cfc1a39a103 100644
--- a/drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h
+++ b/drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h
@@ -535,6 +535,12 @@ int dcn401_get_power_profile(const struct dc_state *context);
SRI_ARR(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst), \
SRI_ARR(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst), \
SRI_ARR(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_DATA_RG, OTG, inst), \
+ SRI_ARR(OTG_CRC1_DATA_B, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst), \
SR_ARR(GSL_SOURCE_SELECT, inst), \
SRI_ARR(OTG_TRIGA_MANUAL_TRIG, OTG, inst), \
SRI_ARR(OTG_GLOBAL_CONTROL1, OTG, inst), \
diff --git a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
index 4bd060e30799..f154a619b533 100644
--- a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
+++ b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
@@ -1281,10 +1281,8 @@ void mod_freesync_handle_v_update(struct mod_freesync *mod_freesync,
* If VRR is inactive, set vtotal min and max to nominal vtotal
*/
if (in_out_vrr->state == VRR_STATE_INACTIVE) {
- in_out_vrr->adjust.v_total_min =
- mod_freesync_calc_v_total_from_refresh(stream,
- in_out_vrr->max_refresh_in_uhz);
- in_out_vrr->adjust.v_total_max = in_out_vrr->adjust.v_total_min;
+ in_out_vrr->adjust.v_total_min = stream->timing.v_total;
+ in_out_vrr->adjust.v_total_max = stream->timing.v_total;
return;
}
}
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
index 8285d4a91589..13e7e33fa0eb 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
@@ -2912,6 +2912,25 @@ static int smu_v13_0_7_mode1_reset(struct smu_context *smu)
return ret;
}
+static void smu_v13_0_7_get_unique_id(struct smu_context *smu)
+{
+ SmuMetricsExternal_t metrics_external;
+ SmuMetrics_t *metrics = &metrics_external.SmuMetrics;
+ struct amdgpu_device *adev = smu->adev;
+ u32 upper32 = 0, lower32 = 0;
+ int ret;
+
+ ret = smu_cmn_get_metrics_table(smu, &metrics_external, false);
+ if (ret)
+ goto out;
+
+ upper32 = metrics->PublicSerialNumberUpper;
+ lower32 = metrics->PublicSerialNumberLower;
+
+out:
+ adev->unique_id = ((u64)upper32 << 32) | lower32;
+}
+
static void smu_v13_0_7_init_msg_ctl(struct smu_context *smu)
{
struct amdgpu_device *adev = smu->adev;
@@ -2995,6 +3014,7 @@ static const struct pptable_funcs smu_v13_0_7_ppt_funcs = {
.enable_uclk_shadow = smu_v13_0_enable_uclk_shadow,
.set_wbrf_exclusion_ranges = smu_v13_0_set_wbrf_exclusion_ranges,
.interrupt_work = smu_v13_0_interrupt_work,
+ .get_unique_id = smu_v13_0_7_get_unique_id,
};
void smu_v13_0_7_set_ppt_funcs(struct smu_context *smu)
diff --git a/drivers/gpu/drm/bridge/nwl-dsi.c b/drivers/gpu/drm/bridge/nwl-dsi.c
index 09992529f3d8..1aab46e1a168 100644
--- a/drivers/gpu/drm/bridge/nwl-dsi.c
+++ b/drivers/gpu/drm/bridge/nwl-dsi.c
@@ -110,6 +110,8 @@ struct nwl_dsi {
int error;
struct nwl_dsi_transfer *xfer;
+
+ unsigned int endpoint;
};
static const struct regmap_config nwl_dsi_regmap_config = {
@@ -821,10 +823,13 @@ static int nwl_dsi_bridge_atomic_check(struct drm_bridge *bridge,
struct drm_connector_state *conn_state)
{
struct drm_display_mode *adjusted_mode = &crtc_state->adjusted_mode;
+ struct nwl_dsi *dsi = bridge_to_dsi(bridge);
/* At least LCDIF + NWL needs active high sync */
- adjusted_mode->flags |= (DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC);
- adjusted_mode->flags &= ~(DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC);
+ if (dsi->endpoint == NWL_DSI_ENDPOINT_LCDIF) {
+ adjusted_mode->flags |= (DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC);
+ adjusted_mode->flags &= ~(DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC);
+ }
/*
* Do a full modeset if crtc_state->active is changed to be true.
@@ -1088,13 +1093,12 @@ static int nwl_dsi_parse_dt(struct nwl_dsi *dsi)
static int nwl_dsi_select_input(struct nwl_dsi *dsi)
{
struct device_node *remote;
- u32 use_dcss = 1;
int ret;
remote = of_graph_get_remote_node(dsi->dev->of_node, 0,
NWL_DSI_ENDPOINT_LCDIF);
if (remote) {
- use_dcss = 0;
+ dsi->endpoint = NWL_DSI_ENDPOINT_LCDIF;
} else {
remote = of_graph_get_remote_node(dsi->dev->of_node, 0,
NWL_DSI_ENDPOINT_DCSS);
@@ -1103,11 +1107,12 @@ static int nwl_dsi_select_input(struct nwl_dsi *dsi)
"No valid input endpoint found\n");
return -EINVAL;
}
+ dsi->endpoint = NWL_DSI_ENDPOINT_DCSS;
}
DRM_DEV_INFO(dsi->dev, "Using %s as input source\n",
- (use_dcss) ? "DCSS" : "LCDIF");
- ret = mux_control_try_select(dsi->mux, use_dcss);
+ (dsi->endpoint == NWL_DSI_ENDPOINT_DCSS) ? "DCSS" : "LCDIF");
+ ret = mux_control_try_select(dsi->mux, dsi->endpoint);
if (ret < 0)
DRM_DEV_ERROR(dsi->dev, "Failed to select input: %d\n", ret);
diff --git a/drivers/gpu/drm/display/drm_dp_helper.c b/drivers/gpu/drm/display/drm_dp_helper.c
index 488fd123a01a..7676285c19f8 100644
--- a/drivers/gpu/drm/display/drm_dp_helper.c
+++ b/drivers/gpu/drm/display/drm_dp_helper.c
@@ -2192,7 +2192,8 @@ static int drm_dp_i2c_xfer(struct i2c_adapter *adapter, struct i2c_msg *msgs,
msg.buffer = msgs[i].buf + j;
msg.size = min(transfer_size, msgs[i].len - j);
- if (j + msg.size == msgs[i].len && aux->no_zero_sized)
+ if (aux->no_zero_sized && i == num - 1 &&
+ j + msg.size == msgs[i].len)
msg.request &= ~DP_AUX_I2C_MOT;
err = drm_dp_i2c_drain_msg(aux, &msg);
diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c
index 63ead8ba6756..5295ec122e0d 100644
--- a/drivers/gpu/drm/drm_crtc.c
+++ b/drivers/gpu/drm/drm_crtc.c
@@ -154,11 +154,8 @@ static void drm_crtc_crc_fini(struct drm_crtc *crtc)
#endif
}
-static const struct dma_fence_ops drm_crtc_fence_ops;
-
static struct drm_crtc *fence_to_crtc(struct dma_fence *fence)
{
- BUG_ON(rcu_access_pointer(fence->ops) != &drm_crtc_fence_ops);
return container_of(fence->extern_lock, struct drm_crtc, fence_lock);
}
@@ -493,14 +490,23 @@ EXPORT_SYMBOL(__drmm_crtc_alloc_with_planes);
* drm_crtc_cleanup - Clean up the core crtc usage
* @crtc: CRTC to cleanup
*
- * This function cleans up @crtc and removes it from the DRM mode setting
- * core. Note that the function does *not* free the crtc structure itself,
- * this is the responsibility of the caller.
+ * This function cleans up @crtc and removes it from the DRM mode setting core,
+ * after first waiting an RCU grace period to ensure @crtc->dev can safely be
+ * dereferenced by our dma_fence_ops.
+ *
+ * Note that the function does *not* free the crtc structure itself, this is the
+ * responsibility of the caller.
*/
void drm_crtc_cleanup(struct drm_crtc *crtc)
{
struct drm_device *dev = crtc->dev;
+ /* Ensure our dma_fence_ops remain valid for an RCU grace period after
+ * the fence is signaled. This is necessary because our dma_fence_ops
+ * dereference crtc->dev.
+ */
+ synchronize_rcu();
+
/* Note that the crtc_list is considered to be static; should we
* remove the drm_crtc at runtime we would have to decrement all
* the indices on the drm_crtc after us in the crtc_list.
diff --git a/drivers/gpu/drm/drm_panic.c b/drivers/gpu/drm/drm_panic.c
index d28ce8878e05..fef47625c526 100644
--- a/drivers/gpu/drm/drm_panic.c
+++ b/drivers/gpu/drm/drm_panic.c
@@ -558,7 +558,8 @@ static void draw_panic_screen_kmsg(struct drm_scanout_buffer *sb)
const struct font_desc *font = get_default_font(sb->width, sb->height, NULL, NULL);
struct drm_rect r_screen = DRM_RECT_INIT(0, 0, sb->width, sb->height);
struct kmsg_dump_iter iter;
- char kmsg_buf[512];
+ /* the size printk formats one record into, too big for the stack */
+ static char kmsg_buf[2048];
size_t kmsg_len;
struct drm_panic_line line;
int yoffset;
@@ -576,6 +577,10 @@ static void draw_panic_screen_kmsg(struct drm_scanout_buffer *sb)
char *start;
char *end;
+ /* a record that does not fit comes back empty, every time */
+ if (!kmsg_len)
+ break;
+
/* ignore terminating NUL and newline */
start = kmsg_buf + kmsg_len - 2;
end = kmsg_buf + kmsg_len - 1;
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
index 03b2576af67f..99c91a9eecf1 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
@@ -168,6 +168,17 @@ u16 etnaviv_buffer_config_pta(struct etnaviv_gpu *gpu, unsigned short id)
return buffer->user_size / 8;
}
+static void etnaviv_cmd_flush_ts(struct etnaviv_cmdbuf *buffer, bool has_blt)
+{
+ CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE, VIVS_TS_FLUSH_CACHE_FLUSH);
+
+ if (has_blt) {
+ CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1);
+ CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1);
+ CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
+ }
+}
+
void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
{
struct etnaviv_cmdbuf *buffer = &gpu->buffer;
@@ -191,7 +202,7 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
unsigned int dwords = 7;
if (has_blt)
- dwords += 10;
+ dwords += 11;
link_target = etnaviv_buffer_reserve(gpu, buffer, dwords);
@@ -204,16 +215,8 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
}
CMD_LOAD_STATE(buffer, VIVS_GL_FLUSH_CACHE, flush);
- if (gpu->exec_state == ETNA_PIPE_3D) {
- if (has_blt) {
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
- } else {
- CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE,
- VIVS_TS_FLUSH_CACHE_FLUSH);
- }
- }
+ if (gpu->exec_state == ETNA_PIPE_3D)
+ etnaviv_cmd_flush_ts(buffer, has_blt);
CMD_SEM(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
CMD_STALL(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
if (has_blt) {
@@ -400,12 +403,12 @@ void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, u32 exec_state,
return_dwords = 7;
/*
- * When the BLT engine is present we need 6 more dwords in the return
+ * When the BLT engine is present we need 7 more dwords in the return
* target: 3 enable/flush/disable + 4 enable/semaphore stall/disable,
- * but we don't need the normal TS flush state.
+ * on top of the normal TS flush state.
*/
if (has_blt)
- return_dwords += 6;
+ return_dwords += 7;
return_target = etnaviv_buffer_reserve(gpu, buffer, return_dwords);
CMD_LINK(cmdbuf, return_dwords, return_target);
@@ -422,14 +425,7 @@ void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, u32 exec_state,
VIVS_GL_FLUSH_CACHE_DEPTH |
VIVS_GL_FLUSH_CACHE_COLOR |
VIVS_GL_FLUSH_CACHE_SHADER_L1);
- if (has_blt) {
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
- } else {
- CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE,
- VIVS_TS_FLUSH_CACHE_FLUSH);
- }
+ etnaviv_cmd_flush_ts(buffer, has_blt);
}
CMD_SEM(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
CMD_STALL(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.c b/drivers/gpu/drm/etnaviv/etnaviv_drv.c
index 08aca9035fc1..ec1352c1b09f 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_drv.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.c
@@ -601,8 +601,10 @@ static void etnaviv_unbind(struct device *dev)
component_unbind_all(dev, drm);
- etnaviv_cmdbuf_free(priv->flop_reset_data_ppu);
- kfree(priv->flop_reset_data_ppu);
+ if (priv->flop_reset_data_ppu) {
+ etnaviv_cmdbuf_free(priv->flop_reset_data_ppu);
+ kfree(priv->flop_reset_data_ppu);
+ }
etnaviv_cmdbuf_suballoc_destroy(priv->cmdbuf_suballoc);
diff --git a/drivers/gpu/drm/i915/display/intel_ddi.c b/drivers/gpu/drm/i915/display/intel_ddi.c
index 02a53c9848e1..8de6e13cf2dd 100644
--- a/drivers/gpu/drm/i915/display/intel_ddi.c
+++ b/drivers/gpu/drm/i915/display/intel_ddi.c
@@ -4594,7 +4594,7 @@ intel_ddi_port_sync_transcoders(const struct intel_crtc_state *ref_crtc_state,
* From, xe3lpd onwards we have defeatured this with reference to
* Wa_16024710867
*/
- if (!IS_DISPLAY_VER(display, 9, 20))
+ if (!IS_DISPLAY_VER(display, 9, 30))
return 0;
if (!intel_crtc_has_type(ref_crtc_state, INTEL_OUTPUT_DP))
diff --git a/drivers/gpu/drm/i915/display/intel_display_device.c b/drivers/gpu/drm/i915/display/intel_display_device.c
index f17fc2c68472..c74256359c25 100644
--- a/drivers/gpu/drm/i915/display/intel_display_device.c
+++ b/drivers/gpu/drm/i915/display/intel_display_device.c
@@ -1813,7 +1813,10 @@ static void __intel_display_device_info_runtime_init(struct intel_display *displ
if (intel_display_wa(display, INTEL_DISPLAY_WA_14011765242))
for_each_pipe(display, pipe)
display_runtime->num_scalers[pipe] = 0;
- else if (DISPLAY_VER(display) >= 11) {
+ else if (DISPLAY_VERx100(display) == 3002) {
+ for_each_pipe(display, pipe)
+ display_runtime->num_scalers[pipe] = 1;
+ } else if (DISPLAY_VER(display) >= 11) {
for_each_pipe(display, pipe)
display_runtime->num_scalers[pipe] = 2;
} else if (DISPLAY_VER(display) >= 9) {
diff --git a/drivers/gpu/drm/i915/display/intel_dp.c b/drivers/gpu/drm/i915/display/intel_dp.c
index 3152122e6aef..d79b6367cd57 100644
--- a/drivers/gpu/drm/i915/display/intel_dp.c
+++ b/drivers/gpu/drm/i915/display/intel_dp.c
@@ -475,9 +475,9 @@ bool intel_dp_has_joiner(struct intel_dp *intel_dp)
!connector->panel.vbt.edp.pipe_joiner_enable)
return false;
- return DISPLAY_VER(display) >= 12 ||
+ return ((DISPLAY_VER(display) >= 12 && DISPLAY_VERx100(display) != 3002) ||
(DISPLAY_VER(display) == 11 &&
- encoder->port != PORT_A);
+ encoder->port != PORT_A));
}
static int dg2_max_source_rate(struct intel_dp *intel_dp)
diff --git a/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c b/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c
index 266e042e0023..d84def79c4bf 100644
--- a/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c
+++ b/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c
@@ -41,6 +41,7 @@
#include "intel_display_types.h"
#include "intel_dp.h"
#include "intel_dp_aux_backlight.h"
+#include "intel_quirks.h"
/*
* DP AUX registers for Intel's proprietary HDR backlight interface. We define
@@ -687,11 +688,30 @@ int intel_dp_aux_init_backlight_funcs(struct intel_connector *connector)
struct drm_device *dev = connector->base.dev;
struct intel_panel *panel = &connector->panel;
bool try_intel_interface = false, try_vesa_interface = false;
+ int enable_dpcd_backlight;
+
+ /*
+ * Some devices have a broken VBT in some Panels, but not others,
+ * advertising INTEL_BACKLIGHT_DISPLAY_DDI, but actually failing to use
+ * the Intel interface. Adding a fallback to the Vesa interface gives
+ * these devices a working backlight for both the broken and non broken
+ * panel VBTs.
+ *
+ * This fallback can not be made the general default as this has broken
+ * other devices in the past whose VBT correctly reports
+ * INTEL_BACKLIGHT_DISPLAY_DDI and whose PWM path is the actual
+ * backlight control, but whose DPCD optimistically advertises
+ * DP_EDP_BACKLIGHT_AUX_ENABLE_CAP / _BRIGHTNESS_AUX_SET_CAP.
+ */
+ enable_dpcd_backlight = display->params.enable_dpcd_backlight;
+ if (enable_dpcd_backlight == INTEL_DP_AUX_BACKLIGHT_AUTO &&
+ intel_has_dpcd_quirk(intel_dp, QUIRK_ENABLE_DPCD_BACKLIGHT))
+ enable_dpcd_backlight = INTEL_DP_AUX_BACKLIGHT_ON;
/* Check the VBT and user's module parameters to figure out which
* interfaces to probe
*/
- switch (display->params.enable_dpcd_backlight) {
+ switch (enable_dpcd_backlight) {
case INTEL_DP_AUX_BACKLIGHT_OFF:
return -ENODEV;
case INTEL_DP_AUX_BACKLIGHT_AUTO:
diff --git a/drivers/gpu/drm/i915/display/intel_plane.c b/drivers/gpu/drm/i915/display/intel_plane.c
index d0f99a87c42e..c1c5a6128614 100644
--- a/drivers/gpu/drm/i915/display/intel_plane.c
+++ b/drivers/gpu/drm/i915/display/intel_plane.c
@@ -140,6 +140,7 @@ intel_plane_duplicate_state(struct drm_plane *plane)
intel_state->dpt_vma = NULL;
intel_state->fence_id = -1;
intel_state->damage = DRM_RECT_INIT(0, 0, 0, 0);
+ intel_state->psr2_sel_fetch_area = DRM_RECT_INIT(0, 0, 0, 0);
/* add reference to fb */
if (intel_state->hw.fb)
@@ -1930,6 +1931,16 @@ static int intel_add_affected_planes(struct intel_atomic_state *state)
ret = intel_joiner_add_affected_planes(state, intel_crtc_joined_pipe_mask(crtc_state));
if (ret)
return ret;
+
+ /*
+ * Add affected plane here to ensure UV and Y plane linking is
+ * in place if UV plane is added as a part of selective update.
+ */
+ if (crtc_state->enable_psr2_sel_fetch) {
+ ret = intel_plane_add_affected(state, crtc);
+ if (ret)
+ return ret;
+ }
}
return 0;
diff --git a/drivers/gpu/drm/i915/display/intel_psr.c b/drivers/gpu/drm/i915/display/intel_psr.c
index c28bcb1a0184..7aaf5b4d3503 100644
--- a/drivers/gpu/drm/i915/display/intel_psr.c
+++ b/drivers/gpu/drm/i915/display/intel_psr.c
@@ -2702,23 +2702,25 @@ static u32 psr2_pipe_srcsz_early_tpt_calc(struct intel_crtc_state *crtc_state,
}
static void clip_area_update(struct drm_rect *overlap_damage_area,
- struct drm_rect *damage_area,
- struct drm_rect *display_area)
+ const struct drm_rect *damage_area,
+ const struct drm_rect *display_area)
{
- if (!drm_rect_intersect(damage_area, display_area))
+ struct drm_rect inter = *damage_area;
+
+ if (!drm_rect_intersect(&inter, display_area))
return;
if (overlap_damage_area->y1 == -1) {
- overlap_damage_area->y1 = damage_area->y1;
- overlap_damage_area->y2 = damage_area->y2;
+ overlap_damage_area->y1 = inter.y1;
+ overlap_damage_area->y2 = inter.y2;
return;
}
- if (damage_area->y1 < overlap_damage_area->y1)
- overlap_damage_area->y1 = damage_area->y1;
+ if (inter.y1 < overlap_damage_area->y1)
+ overlap_damage_area->y1 = inter.y1;
- if (damage_area->y2 > overlap_damage_area->y2)
- overlap_damage_area->y2 = damage_area->y2;
+ if (inter.y2 > overlap_damage_area->y2)
+ overlap_damage_area->y2 = inter.y2;
}
static bool intel_psr2_sel_fetch_pipe_alignment(struct intel_crtc_state *crtc_state)
@@ -2895,7 +2897,7 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
.y2 = crtc_state->hw.adjusted_mode.crtc_vdisplay,
};
bool full_update = false, su_area_changed;
- int i, ret;
+ int i;
/*
* Selective fetch is not always usable, for instance it is dropped
@@ -2913,16 +2915,16 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
if (!crtc_state->enable_psr2_sel_fetch)
return 0;
- if (!psr2_sel_fetch_pipe_state_supported(crtc_state)) {
- full_update = true;
- goto skip_sel_fetch_set_loop;
- }
-
crtc_state->psr2_su_area.x1 = 0;
crtc_state->psr2_su_area.y1 = -1;
crtc_state->psr2_su_area.x2 = drm_rect_width(&display_area);
crtc_state->psr2_su_area.y2 = -1;
+ if (!psr2_sel_fetch_pipe_state_supported(crtc_state)) {
+ full_update = true;
+ goto skip_su_area_calc;
+ }
+
/*
* Calculate minimal selective fetch area of each plane and calculate
* the pipe damaged area.
@@ -3006,23 +3008,19 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
}
if (full_update)
- goto skip_sel_fetch_set_loop;
+ goto skip_su_area_calc;
intel_psr_apply_su_area_workarounds(crtc_state);
- ret = drm_atomic_add_affected_planes(&state->base, &crtc->base);
- if (ret)
- return ret;
-
do {
bool cursor_in_su_area = false;
/*
* Adjust su area to cover cursor fully as necessary
- * (early transport). This needs to be done after
- * drm_atomic_add_affected_planes to ensure visible
- * cursor is added into affected planes even when
- * cursor is not updated by itself.
+ * (early transport). A visible cursor is in the state
+ * even when it is not updated by itself as all planes
+ * of the pipe are added into the state before the plane
+ * checks when selective fetch is enabled.
*/
intel_psr2_sel_fetch_et_alignment(state, crtc, &display_area,
&cursor_in_su_area);
@@ -3042,6 +3040,11 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
break;
} while (su_area_changed);
+skip_su_area_calc:
+ if (full_update)
+ clip_area_update(&crtc_state->psr2_su_area, &display_area,
+ &display_area);
+
/*
* Now that we have the pipe damaged area check if it intersect with
* every plane, if it does set the plane selective fetch area.
@@ -3057,28 +3060,14 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
inter = crtc_state->psr2_su_area;
sel_fetch_area = &new_plane_state->psr2_sel_fetch_area;
- if (!drm_rect_intersect(&inter, &new_plane_state->uapi.dst)) {
- sel_fetch_area->y1 = -1;
- sel_fetch_area->y2 = -1;
- /*
- * if plane sel fetch was previously enabled ->
- * disable it
- */
- if (drm_rect_height(&old_plane_state->psr2_sel_fetch_area) > 0)
- crtc_state->update_planes |= BIT(plane->id);
-
- continue;
- }
-
- if (!psr2_sel_fetch_plane_state_supported(new_plane_state)) {
- full_update = true;
- break;
+ if (drm_rect_intersect(&inter, &new_plane_state->uapi.dst)) {
+ sel_fetch_area->y1 = inter.y1 - new_plane_state->uapi.dst.y1;
+ sel_fetch_area->y2 = inter.y2 - new_plane_state->uapi.dst.y1;
}
- sel_fetch_area = &new_plane_state->psr2_sel_fetch_area;
- sel_fetch_area->y1 = inter.y1 - new_plane_state->uapi.dst.y1;
- sel_fetch_area->y2 = inter.y2 - new_plane_state->uapi.dst.y1;
- crtc_state->update_planes |= BIT(plane->id);
+ if (!drm_rect_equals(&old_plane_state->psr2_sel_fetch_area,
+ sel_fetch_area))
+ crtc_state->update_planes |= BIT(plane->id);
/*
* Sel_fetch_area is calculated for UV plane. Use
@@ -3086,24 +3075,25 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
*/
if (linked) {
struct intel_plane_state *linked_new_plane_state;
+ struct intel_plane_state *linked_old_plane_state;
struct drm_rect *linked_sel_fetch_area;
- linked_new_plane_state = intel_atomic_get_plane_state(state, linked);
- if (IS_ERR(linked_new_plane_state))
- return PTR_ERR(linked_new_plane_state);
+ linked_new_plane_state = intel_atomic_get_new_plane_state(state, linked);
+ if (!linked_new_plane_state)
+ return -EINVAL;
+
+ linked_old_plane_state = intel_atomic_get_old_plane_state(state, linked);
linked_sel_fetch_area = &linked_new_plane_state->psr2_sel_fetch_area;
linked_sel_fetch_area->y1 = sel_fetch_area->y1;
linked_sel_fetch_area->y2 = sel_fetch_area->y2;
- crtc_state->update_planes |= BIT(linked->id);
+
+ if (!drm_rect_equals(&linked_old_plane_state->psr2_sel_fetch_area,
+ linked_sel_fetch_area))
+ crtc_state->update_planes |= BIT(linked->id);
}
}
-skip_sel_fetch_set_loop:
- if (full_update)
- clip_area_update(&crtc_state->psr2_su_area, &display_area,
- &display_area);
-
psr2_man_trk_ctl_calc(crtc_state, full_update);
crtc_state->pipe_srcsz_early_tpt =
psr2_pipe_srcsz_early_tpt_calc(crtc_state, full_update);
diff --git a/drivers/gpu/drm/i915/display/intel_psr_regs.h b/drivers/gpu/drm/i915/display/intel_psr_regs.h
index 16a9e3af198d..428f3da04500 100644
--- a/drivers/gpu/drm/i915/display/intel_psr_regs.h
+++ b/drivers/gpu/drm/i915/display/intel_psr_regs.h
@@ -254,12 +254,12 @@
#define _LNL_SFF_CTL_A 0x60918
#define _LNL_SFF_CTL_B 0x61918
#define LNL_SFF_CTL(tran) _MMIO_TRANS(tran, _LNL_SFF_CTL_A, _LNL_SFF_CTL_B)
-#define LNL_SFF_CTL_SF_SINGLE_FULL_FRAME REG_BIT(1)
+#define LNL_SFF_CTL_SF_SINGLE_FULL_FRAME REG_BIT(0)
#define _LNL_CFF_CTL_A 0x6091c
#define _LNL_CFF_CTL_B 0x6191c
#define LNL_CFF_CTL(tran) _MMIO_TRANS(tran, _LNL_CFF_CTL_A, _LNL_CFF_CTL_B)
-#define LNL_CFF_CTL_SF_CONTINUOUS_FULL_FRAME REG_BIT(1)
+#define LNL_CFF_CTL_SF_CONTINUOUS_FULL_FRAME REG_BIT(0)
/* PSR2 Early transport */
#define _PIPE_SRCSZ_ERLY_TPT_A 0x70074
diff --git a/drivers/gpu/drm/i915/display/intel_quirks.c b/drivers/gpu/drm/i915/display/intel_quirks.c
index 7d7db774d8c7..96eb9a40df9b 100644
--- a/drivers/gpu/drm/i915/display/intel_quirks.c
+++ b/drivers/gpu/drm/i915/display/intel_quirks.c
@@ -100,6 +100,14 @@ static void quirk_disable_psr2(struct intel_display *display)
drm_info(display->drm, "PSR2 support not currently available for this setup, applying disable PSR2 quirk\n");
}
+static void quirk_enable_dpcd_backlight(struct intel_dp *intel_dp)
+{
+ struct intel_display *display = to_intel_display(intel_dp);
+
+ intel_set_dpcd_quirk(intel_dp, QUIRK_ENABLE_DPCD_BACKLIGHT);
+ drm_info(display->drm, "Applying Enable DPCD Backlight quirk\n");
+}
+
struct intel_quirk {
int device;
int subsystem_vendor;
@@ -289,6 +297,22 @@ static const struct intel_dpcd_quirk intel_dpcd_quirks[] = {
.sink_oui = SINK_OUI(0x00, 0x22, 0xb9),
.hook = quirk_disable_edp_panel_replay,
},
+ /* TUXEDO InsanityBook 15 v1 */
+ {
+ .device = 0x591b,
+ .subsystem_vendor = 0x1558,
+ .subsystem_device = 0x9501,
+ .sink_oui = SINK_OUI(0x38, 0xec, 0x11),
+ .hook = quirk_enable_dpcd_backlight,
+ },
+ /* TUXEDO DX1708 */
+ {
+ .device = 0x3e9b,
+ .subsystem_vendor = 0x1558,
+ .subsystem_device = 0x8500,
+ .sink_oui = SINK_OUI(0x38, 0xec, 0x11),
+ .hook = quirk_enable_dpcd_backlight,
+ },
};
void intel_init_quirks(struct intel_display *display)
diff --git a/drivers/gpu/drm/i915/display/intel_quirks.h b/drivers/gpu/drm/i915/display/intel_quirks.h
index 970a4fe52faf..4996419ae76b 100644
--- a/drivers/gpu/drm/i915/display/intel_quirks.h
+++ b/drivers/gpu/drm/i915/display/intel_quirks.h
@@ -23,6 +23,7 @@ enum intel_quirk_id {
QUIRK_EDP_LIMIT_RATE_HBR2,
QUIRK_DISABLE_EDP_PANEL_REPLAY,
QUIRK_DISABLE_PSR2,
+ QUIRK_ENABLE_DPCD_BACKLIGHT,
};
void intel_init_quirks(struct intel_display *display);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index d5aadced7da0..7fdd41a190b4 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -2790,7 +2790,6 @@ add_timeline_fence_array(struct i915_execbuffer *eb,
u64 __user *user_values;
struct eb_fence *f;
u64 nfences;
- int err = 0;
nfences = timeline_fences->fence_count;
if (!nfences)
@@ -2828,6 +2827,7 @@ add_timeline_fence_array(struct i915_execbuffer *eb,
struct drm_syncobj *syncobj;
struct dma_fence *fence = NULL;
u64 point;
+ int err = 0;
if (__copy_from_user(&user_fence,
user_fences++,
diff --git a/drivers/gpu/drm/loongson/lsdc_gem.c b/drivers/gpu/drm/loongson/lsdc_gem.c
index 2fb03487c983..9eba4e1a5ba6 100644
--- a/drivers/gpu/drm/loongson/lsdc_gem.c
+++ b/drivers/gpu/drm/loongson/lsdc_gem.c
@@ -157,14 +157,18 @@ struct drm_gem_object *lsdc_gem_object_create(struct drm_device *ddev,
return ERR_PTR(ret);
}
+ gobj = &lbo->tbo.base;
+ gobj->funcs = &lsdc_gem_object_funcs;
+
if (!sg) {
/* VRAM is filled with random data */
- lsdc_bo_clear(lbo);
+ ret = lsdc_bo_clear(lbo);
+ if (ret) {
+ drm_gem_object_put(gobj);
+ return ERR_PTR(ret);
+ }
}
- gobj = &lbo->tbo.base;
- gobj->funcs = &lsdc_gem_object_funcs;
-
/* tracking the BOs we created */
mutex_lock(&ldev->gem.mutex);
list_add_tail(&lbo->list, &ldev->gem.objects);
diff --git a/drivers/gpu/drm/loongson/lsdc_plane.c b/drivers/gpu/drm/loongson/lsdc_plane.c
index bcc0ffa17bdf..7b9c65c30a9d 100644
--- a/drivers/gpu/drm/loongson/lsdc_plane.c
+++ b/drivers/gpu/drm/loongson/lsdc_plane.c
@@ -230,7 +230,7 @@ static void lsdc_cursor_plane_atomic_async_update(struct drm_plane *plane,
new_state = drm_atomic_get_new_plane_state(state, plane);
- new_fb = plane->state->fb;
+ new_fb = new_state->fb;
plane->state->crtc_x = new_state->crtc_x;
plane->state->crtc_y = new_state->crtc_y;
diff --git a/drivers/gpu/drm/loongson/lsdc_ttm.c b/drivers/gpu/drm/loongson/lsdc_ttm.c
index d7441d96a0dc..7b30dea039ee 100644
--- a/drivers/gpu/drm/loongson/lsdc_ttm.c
+++ b/drivers/gpu/drm/loongson/lsdc_ttm.c
@@ -388,9 +388,13 @@ void lsdc_bo_kunmap(struct lsdc_bo *lbo)
ttm_bo_kunmap(&lbo->kmap);
}
-void lsdc_bo_clear(struct lsdc_bo *lbo)
+int lsdc_bo_clear(struct lsdc_bo *lbo)
{
- lsdc_bo_kmap(lbo);
+ int ret;
+
+ ret = lsdc_bo_kmap(lbo);
+ if (ret)
+ return ret;
if (lbo->is_iomem)
memset_io((void __iomem *)lbo->kptr, 0, lbo->size);
@@ -398,6 +402,8 @@ void lsdc_bo_clear(struct lsdc_bo *lbo)
memset(lbo->kptr, 0, lbo->size);
lsdc_bo_kunmap(lbo);
+
+ return 0;
}
int lsdc_bo_evict_vram(struct drm_device *ddev)
@@ -475,10 +481,8 @@ struct lsdc_bo *lsdc_bo_create(struct drm_device *ddev,
ret = ttm_bo_init_validate(bdev, tbo, bo_type, &lbo->placement, 0,
false, sg, resv, lsdc_bo_destroy);
- if (ret) {
- kfree(lbo);
+ if (ret)
return ERR_PTR(ret);
- }
return lbo;
}
diff --git a/drivers/gpu/drm/loongson/lsdc_ttm.h b/drivers/gpu/drm/loongson/lsdc_ttm.h
index 843e1475064e..df47d9bb0d03 100644
--- a/drivers/gpu/drm/loongson/lsdc_ttm.h
+++ b/drivers/gpu/drm/loongson/lsdc_ttm.h
@@ -89,7 +89,7 @@ size_t lsdc_bo_size(struct lsdc_bo *lbo);
int lsdc_bo_kmap(struct lsdc_bo *lbo);
void lsdc_bo_kunmap(struct lsdc_bo *lbo);
-void lsdc_bo_clear(struct lsdc_bo *lbo);
+int lsdc_bo_clear(struct lsdc_bo *lbo);
int lsdc_bo_evict_vram(struct drm_device *ddev);
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
index aab5a73f467a..ff6858f5e5d3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
@@ -157,6 +157,7 @@ nvkm_subdev_init_(struct nvkm_subdev *subdev)
ret = subdev->func->init(subdev);
if (ret) {
nvkm_error(subdev, "init failed, %d\n", ret);
+ subdev->use.enabled = false;
return ret;
}
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
index 5368e705e7fd..186e09ea0cb1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
@@ -27,6 +27,7 @@
#include <subdev/bios/init.h>
#include <subdev/bios/pll.h>
#include <subdev/clk/pll.h>
+#include <subdev/vga.h>
int
gf100_devinit_pll_set(struct nvkm_devinit *init, u32 type, u32 freq)
@@ -99,6 +100,25 @@ gf100_devinit_preinit(struct nvkm_devinit *base)
* can use it as a reliable way to know whether we should run devinit.
*/
base->post = ((nvkm_rd32(device, 0x2240c) & BIT(1)) == 0);
+
+ /*
+ * However some VBIOS init sequences miss this bit. So fall back to
+ * the nv50 method of checking if the bit is not set. Running the
+ * VBIOS multiple times may have detrimental effects.
+ */
+ if (base->post) {
+ u64 disable = nvkm_devinit_disable(base);
+ /* magic to detect whether or not x86 vbios code has
+ * executed the devinit scripts to initialise the
+ * board. only works if there's a display engine.
+ */
+ if (!(disable & (1ULL << NVKM_ENGINE_DISP))) {
+ if (nvkm_rdvgac(device, 0, 0x00) ||
+ nvkm_rdvgac(device, 0, 0x1a)) {
+ base->post = false;
+ }
+ }
+ }
}
static const struct nvkm_devinit_func
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index 673ca9cbf362..0206bbad2ce0 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -130,10 +130,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,
INIT_LIST_HEAD(&entity->list);
entity->guilty = guilty;
- entity->priority = priority;
entity->last_user = current->group_leader;
- entity->rq_priority = drm_sched_policy == DRM_SCHED_POLICY_FAIR ?
- DRM_SCHED_PRIORITY_KERNEL : priority;
entity->num_sched_list = num_sched_list;
entity->sched_list = num_sched_list > 1 ? sched_list : NULL;
RCU_INIT_POINTER(entity->last_scheduled, NULL);
@@ -145,26 +142,26 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,
* not to do this and to fix their DRM calling order.
*/
pr_warn("%s: called with uninitialized scheduler\n", __func__);
- } else {
- enum drm_sched_priority p = entity->priority;
-
+ } else if (priority >= sched_list[0]->num_user_rqs) {
/*
* The "priority" of an entity cannot exceed the number of
* run-queues of a scheduler. Protect against num_rqs being 0,
* by converting to signed. Choose the lowest priority
* available.
*/
- if (p >= sched_list[0]->num_user_rqs) {
- dev_err(sched_list[0]->dev, "entity with out-of-bounds priority:%u num_user_rqs:%u\n",
- p, sched_list[0]->num_user_rqs);
- p = max_t(s32,
- (s32)sched_list[0]->num_user_rqs - 1,
- (s32)DRM_SCHED_PRIORITY_KERNEL);
- entity->priority = p;
- }
- entity->rq = sched_list[0]->sched_rq[entity->rq_priority];
+ dev_err(sched_list[0]->dev, "entity with out-of-bounds priority:%u num_user_rqs:%u\n",
+ priority, sched_list[0]->num_user_rqs);
+ priority = max_t(s32,
+ (s32)sched_list[0]->num_user_rqs - 1,
+ (s32)DRM_SCHED_PRIORITY_KERNEL);
}
+ entity->priority = priority;
+ entity->rq_priority = drm_sched_policy == DRM_SCHED_POLICY_FAIR ?
+ DRM_SCHED_PRIORITY_KERNEL : priority;
+ if (sched_list[0]->sched_rq)
+ entity->rq = sched_list[0]->sched_rq[entity->rq_priority];
+
init_completion(&entity->entity_idle);
/* We start in an idle state. */
@@ -457,6 +454,8 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
{
spin_lock(&entity->lock);
entity->priority = priority;
+ if (drm_sched_policy != DRM_SCHED_POLICY_FAIR)
+ entity->rq_priority = priority;
spin_unlock(&entity->lock);
}
EXPORT_SYMBOL(drm_sched_entity_set_priority);
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index bc88ebdcdbea..cf3df526e773 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -217,6 +217,8 @@ v3d_clean_caches(struct v3d_dev *v3d)
struct drm_device *dev = &v3d->drm;
int core = 0;
+ guard(mutex)(&v3d->cache_clean_lock);
+
trace_v3d_cache_clean_begin(dev);
/* GFXH-1897: Ensure pending flushes complete before writing L2TCACTL */
@@ -233,7 +235,6 @@ v3d_clean_caches(struct v3d_dev *v3d)
drm_err(dev, "Timeout waiting for TMU write combiner flush\n");
}
- mutex_lock(&v3d->cache_clean_lock);
V3D_CORE_WRITE(core, V3D_CTL_L2TCACTL,
V3D_L2TCACTL_L2TFLS |
V3D_SET_FIELD(V3D_L2TCACTL_FLM_CLEAN, V3D_L2TCACTL_FLM));
@@ -243,8 +244,6 @@ v3d_clean_caches(struct v3d_dev *v3d)
drm_err(dev, "Timeout waiting for L2T clean\n");
}
- mutex_unlock(&v3d->cache_clean_lock);
-
trace_v3d_cache_clean_end(dev);
}
diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c
index e6283a75bcdf..002116efc874 100644
--- a/drivers/gpu/drm/v3d/v3d_sched.c
+++ b/drivers/gpu/drm/v3d/v3d_sched.c
@@ -360,6 +360,8 @@ v3d_csd_job_run(struct drm_sched_job *sched_job)
dma_fence_put(job->base.irq_fence);
job->base.irq_fence = dma_fence_get(fence);
+ job->timedout_batches = job->args.cfg[4];
+
trace_v3d_submit_csd(dev, to_v3d_fence(fence)->seqno);
v3d_job_start_stats(&job->base);
diff --git a/drivers/gpu/drm/vc4/vc4_gem.c b/drivers/gpu/drm/vc4/vc4_gem.c
index 6ee4bd77d433..0fef9b48d3c6 100644
--- a/drivers/gpu/drm/vc4/vc4_gem.c
+++ b/drivers/gpu/drm/vc4/vc4_gem.c
@@ -639,13 +639,14 @@ vc4_queue_submit(struct drm_device *dev, struct vc4_exec_info *exec,
fence->seqno = exec->seqno;
exec->fence = &fence->base;
+ /* The job can complete and drop exec->fence as soon as job_lock is
+ * released, so hold our own reference.
+ */
if (out_sync)
- drm_syncobj_replace_fence(out_sync, exec->fence);
+ dma_fence_get(&fence->base);
vc4_attach_fences(exec);
- drm_exec_fini(exec_ctx);
-
list_add_tail(&exec->head, &vc4->bin_job_list);
/* If no bin job was executing and if the render job (if any) has the
@@ -662,6 +663,13 @@ vc4_queue_submit(struct drm_device *dev, struct vc4_exec_info *exec,
spin_unlock_irqrestore(&vc4->job_lock, irqflags);
+ drm_exec_fini(exec_ctx);
+
+ if (out_sync) {
+ drm_syncobj_replace_fence(out_sync, &fence->base);
+ dma_fence_put(&fence->base);
+ }
+
return 0;
}
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi.c b/drivers/gpu/drm/vc4/vc4_hdmi.c
index d4672aa81d85..f2043f907dc5 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi.c
+++ b/drivers/gpu/drm/vc4/vc4_hdmi.c
@@ -842,8 +842,7 @@ static void vc4_hdmi_disable_scrambling(struct drm_encoder *encoder)
vc4_hdmi->scdc_enabled = false;
- if (delayed_work_pending(&vc4_hdmi->scrambling_work))
- cancel_delayed_work_sync(&vc4_hdmi->scrambling_work);
+ cancel_delayed_work_sync(&vc4_hdmi->scrambling_work);
if (!drm_dev_enter(drm, &idx))
return;
@@ -918,8 +917,6 @@ static void vc4_hdmi_encoder_post_crtc_disable(struct drm_encoder *encoder,
spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
- vc4_hdmi_disable_scrambling(encoder);
-
drm_dev_exit(idx);
out:
@@ -947,6 +944,8 @@ static void vc4_hdmi_encoder_post_crtc_powerdown(struct drm_encoder *encoder,
if (vc4_hdmi->variant->phy_disable)
vc4_hdmi->variant->phy_disable(vc4_hdmi);
+ vc4_hdmi_disable_scrambling(encoder);
+
clk_disable_unprepare(vc4_hdmi->pixel_bvb_clock);
clk_disable_unprepare(vc4_hdmi->pixel_clock);
@@ -1554,6 +1553,8 @@ static void vc4_hdmi_encoder_pre_crtc_configure(struct drm_encoder *encoder,
goto err_disable_pixel_clock;
}
+ vc4_hdmi_enable_scrambling(encoder);
+
if (vc4_hdmi->variant->phy_init)
vc4_hdmi->variant->phy_init(vc4_hdmi, conn_state);
@@ -1688,7 +1689,6 @@ static void vc4_hdmi_encoder_post_crtc_enable(struct drm_encoder *encoder,
}
vc4_hdmi_recenter_fifo(vc4_hdmi);
- vc4_hdmi_enable_scrambling(encoder);
drm_dev_exit(idx);
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi_phy.c b/drivers/gpu/drm/vc4/vc4_hdmi_phy.c
index 56e6a35da357..5040f014bf7e 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi_phy.c
+++ b/drivers/gpu/drm/vc4/vc4_hdmi_phy.c
@@ -1197,4 +1197,9 @@ void vc6_hdmi_phy_init(struct vc4_hdmi *vc4_hdmi,
void vc6_hdmi_phy_disable(struct vc4_hdmi *vc4_hdmi)
{
+ unsigned long flags;
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+ vc6_hdmi_reset_phy(vc4_hdmi);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
diff --git a/drivers/hwmon/aquacomputer_d5next.c b/drivers/hwmon/aquacomputer_d5next.c
index 1ca70e726298..3d3786c82931 100644
--- a/drivers/hwmon/aquacomputer_d5next.c
+++ b/drivers/hwmon/aquacomputer_d5next.c
@@ -19,6 +19,7 @@
#include <linux/hwmon.h>
#include <linux/jiffies.h>
#include <linux/ktime.h>
+#include <linux/limits.h>
#include <linux/module.h>
#include <linux/seq_file.h>
#include <linux/unaligned.h>
@@ -132,6 +133,7 @@ static u8 aquaero_secondary_ctrl_report[] = {
#define AQUAERO_FAN_CURRENT_OFFSET 0x06
#define AQUAERO_FAN_POWER_OFFSET 0x08
#define AQUAERO_FAN_SPEED_OFFSET 0x00
+#define AQUAERO_MIN_STATUS_REPORT_SIZE (0x18B + AQUAERO_FAN_POWER_OFFSET + sizeof(u16))
static u16 aquaero_sensor_fan_offsets[] = { 0x167, 0x173, 0x17f, 0x18B };
/* Control report offsets for the Aquaero fan controllers */
@@ -155,6 +157,8 @@ static u16 aquaero_ctrl_fan_offsets[] = { 0x20c, 0x220, 0x234, 0x248 };
#define D5NEXT_5V_VOLTAGE 0x39
#define D5NEXT_12V_VOLTAGE 0x37
#define D5NEXT_VIRTUAL_SENSORS_START 0x3f
+#define D5NEXT_MIN_STATUS_REPORT_SIZE (D5NEXT_PUMP_OFFSET + AQC_FAN_SPEED_OFFSET + \
+ sizeof(u16))
static u16 d5next_sensor_fan_offsets[] = { D5NEXT_PUMP_OFFSET, D5NEXT_FAN_OFFSET };
/* Control report offsets for the D5 Next pump */
@@ -179,11 +183,14 @@ static u16 d5next_ctrl_fan_offsets[] = { 0x97, 0x42 }; /* Pump and fan speed (fr
#define AQUASTREAMULT_FAN_CURRENT_OFFSET 0x00
#define AQUASTREAMULT_FAN_POWER_OFFSET 0x04
#define AQUASTREAMULT_FAN_SPEED_OFFSET 0x06
+#define AQUASTREAMULT_MIN_STATUS_REPORT_SIZE (AQUASTREAMULT_PRESSURE_OFFSET + sizeof(u16))
static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET };
/* Spec and sensor report offset for the Farbwerk RGB controller */
#define FARBWERK_NUM_SENSORS 4
#define FARBWERK_SENSOR_START 0x2f
+#define FARBWERK_MIN_STATUS_REPORT_SIZE (FARBWERK_SENSOR_START + \
+ FARBWERK_NUM_SENSORS * AQC_SENSOR_SIZE)
/* Specs of the Farbwerk 360 RGB controller */
#define FARBWERK360_NUM_SENSORS 4
@@ -193,6 +200,8 @@ static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET };
/* Sensor report offsets for the Farbwerk 360 */
#define FARBWERK360_SENSOR_START 0x32
#define FARBWERK360_VIRTUAL_SENSORS_START 0x3a
+#define FARBWERK360_MIN_STATUS_REPORT_SIZE (FARBWERK360_VIRTUAL_SENSORS_START + \
+ FARBWERK360_NUM_VIRTUAL_SENSORS * AQC_SENSOR_SIZE)
/* Control report offsets for the Farbwerk 360 */
#define FARBWERK360_TEMP_CTRL_OFFSET 0x8
@@ -209,6 +218,7 @@ static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET };
#define OCTO_SENSOR_START 0x3D
#define OCTO_VIRTUAL_SENSORS_START 0x45
#define OCTO_FLOW_SENSOR_OFFSET 0x7B
+#define OCTO_MIN_STATUS_REPORT_SIZE (0xD8 + AQC_FAN_SPEED_OFFSET + sizeof(u16))
static u16 octo_sensor_fan_offsets[] = { 0x7D, 0x8A, 0x97, 0xA4, 0xB1, 0xBE, 0xCB, 0xD8 };
/* Control report offsets for the Octo */
@@ -229,6 +239,7 @@ static u16 octo_ctrl_fan_offsets[] = { 0x5B, 0xB0, 0x105, 0x15A, 0x1AF, 0x204, 0
#define QUADRO_SENSOR_START 0x34
#define QUADRO_VIRTUAL_SENSORS_START 0x3c
#define QUADRO_FLOW_SENSOR_OFFSET 0x6e
+#define QUADRO_MIN_STATUS_REPORT_SIZE (0x97 + AQC_FAN_SPEED_OFFSET + sizeof(u16))
static u16 quadro_sensor_fan_offsets[] = { 0x70, 0x7D, 0x8A, 0x97 };
/* Control report offsets for the Quadro */
@@ -248,6 +259,7 @@ static u16 quadro_ctrl_fan_offsets[] = { 0x37, 0x8c, 0xe1, 0x136 }; /* Fan speed
#define HIGHFLOWNEXT_CONDUCTIVITY 95
#define HIGHFLOWNEXT_5V_VOLTAGE 97
#define HIGHFLOWNEXT_5V_VOLTAGE_USB 99
+#define HIGHFLOWNEXT_MIN_STATUS_REPORT_SIZE (HIGHFLOWNEXT_5V_VOLTAGE_USB + sizeof(u16))
/* Specs of the Leakshield */
#define LEAKSHIELD_NUM_SENSORS 2
@@ -263,6 +275,7 @@ static u16 quadro_ctrl_fan_offsets[] = { 0x37, 0x8c, 0xe1, 0x136 }; /* Fan speed
#define LEAKSHIELD_FLOW_IN 111
#define LEAKSHIELD_RESERVOIR_VOLUME 313
#define LEAKSHIELD_RESERVOIR_FILLED 311
+#define LEAKSHIELD_MIN_STATUS_REPORT_SIZE (LEAKSHIELD_RESERVOIR_VOLUME + sizeof(u16))
/* Specs of the Aquastream XT pump */
#define AQUASTREAMXT_SERIAL_START 0x3a
@@ -554,6 +567,7 @@ struct aqc_data {
const char *name;
int status_report_id; /* Used for legacy devices, report is stored in buffer */
+ u16 min_report_size;
int ctrl_report_id;
int secondary_ctrl_report_id;
int secondary_ctrl_report_size;
@@ -599,9 +613,9 @@ struct aqc_data {
u32 speed_input_min[1];
u32 speed_input_target[1];
u32 speed_input_max[1];
- u32 power_input[8];
+ long power_input[8];
u16 voltage_input[8];
- u16 current_input[8];
+ u32 current_input[8];
/* Label values */
const char *const *temp_label;
@@ -976,7 +990,7 @@ static int aqc_legacy_read(struct aqc_data *priv)
/* Calculation derived from linear regression */
sensor_value = get_unaligned_le16(priv->buffer + AQUASTREAMXT_PUMP_CURR_OFFSET);
- priv->current_input[0] = DIV_ROUND_CLOSEST(sensor_value * 176, 100) - 52;
+ priv->current_input[0] = max(DIV_ROUND_CLOSEST(sensor_value * 176, 100) - 52, 0);
sensor_value = get_unaligned_le16(priv->buffer + AQUASTREAMXT_PUMP_VOLTAGE_OFFSET);
priv->voltage_input[0] = DIV_ROUND_CLOSEST(sensor_value * 1000, 61);
@@ -1006,6 +1020,7 @@ static int aqc_legacy_read(struct aqc_data *priv)
static int aqc_read(struct device *dev, enum hwmon_sensor_types type, u32 attr,
int channel, long *val)
{
+ long power;
int ret;
struct aqc_data *priv = dev_get_drvdata(dev);
@@ -1070,7 +1085,10 @@ static int aqc_read(struct device *dev, enum hwmon_sensor_types type, u32 attr,
}
break;
case hwmon_power:
- *val = priv->power_input[channel];
+ power = READ_ONCE(priv->power_input[channel]);
+ if (power == -ENODATA)
+ return -ENODATA;
+ *val = power;
break;
case hwmon_pwm:
switch (priv->kind) {
@@ -1333,6 +1351,8 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
return 0;
priv = hid_get_drvdata(hdev);
+ if (!priv->min_report_size || size < priv->min_report_size)
+ return 0;
/* Info provided with every report */
priv->serial_number[0] = get_unaligned_be16(data + priv->serial_number_start_offset);
@@ -1368,9 +1388,9 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
priv->speed_input[i] =
get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
priv->fan_structure->speed);
- priv->power_input[i] =
- get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
- priv->fan_structure->power) * 10000;
+ WRITE_ONCE(priv->power_input[i],
+ get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
+ priv->fan_structure->power) * 10000);
priv->voltage_input[i] =
get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
priv->fan_structure->voltage) * 10;
@@ -1410,7 +1430,8 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
priv->speed_input[2] = get_unaligned_be16(data + AQUASTREAMULT_PRESSURE_OFFSET);
priv->speed_input[3] = get_unaligned_be16(data + AQUASTREAMULT_FLOW_SENSOR_OFFSET);
- priv->power_input[1] = get_unaligned_be16(data + AQUASTREAMULT_PUMP_POWER) * 10000;
+ WRITE_ONCE(priv->power_input[1],
+ get_unaligned_be16(data + AQUASTREAMULT_PUMP_POWER) * 10000);
priv->voltage_input[1] = get_unaligned_be16(data + AQUASTREAMULT_PUMP_VOLTAGE) * 10;
@@ -1423,10 +1444,11 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
case highflownext:
/* If external temp sensor is not connected, its power reading is also N/A */
if (priv->temp_input[1] == -ENODATA)
- priv->power_input[0] = -ENODATA;
+ WRITE_ONCE(priv->power_input[0], -ENODATA);
else
- priv->power_input[0] =
- get_unaligned_be16(data + HIGHFLOWNEXT_POWER) * 1000000;
+ WRITE_ONCE(priv->power_input[0],
+ min_t(u64, get_unaligned_be16(data + HIGHFLOWNEXT_POWER) *
+ 1000000ULL, LONG_MAX));
priv->voltage_input[0] = get_unaligned_be16(data + HIGHFLOWNEXT_5V_VOLTAGE) * 10;
priv->voltage_input[1] =
@@ -1518,6 +1540,7 @@ static void aqc_debugfs_init(struct aqc_data *priv)
static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
{
struct aqc_data *priv;
+ u16 min_report_size = 0;
int ret;
priv = devm_kzalloc(&hdev->dev, sizeof(*priv), GFP_KERNEL);
@@ -1554,7 +1577,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
* they present. The two other devices have the type of the second element in
* their respective collections set to 1, while the real device has it set to 0.
*/
- if (hdev->collection[1].type != 0) {
+ if (hdev->maxcollection < 2 || hdev->collection[1].type != 0) {
ret = -ENODEV;
goto fail_and_close;
}
@@ -1585,6 +1608,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_fan_power;
priv->voltage_label = label_fan_voltage;
priv->current_label = label_fan_current;
+ min_report_size = AQUAERO_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_D5NEXT:
priv->kind = d5next;
@@ -1610,6 +1634,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_d5next_power;
priv->voltage_label = label_d5next_voltages;
priv->current_label = label_d5next_current;
+ min_report_size = D5NEXT_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_FARBWERK:
priv->kind = farbwerk;
@@ -1620,6 +1645,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->temp_sensor_start_offset = FARBWERK_SENSOR_START;
priv->temp_label = label_temp_sensors;
+ min_report_size = FARBWERK_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_FARBWERK360:
priv->kind = farbwerk360;
@@ -1636,6 +1662,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->temp_label = label_temp_sensors;
priv->virtual_temp_label = label_virtual_temp_sensors;
+ min_report_size = FARBWERK360_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_OCTO:
priv->kind = octo;
@@ -1665,6 +1692,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_fan_power;
priv->voltage_label = label_fan_voltage;
priv->current_label = label_fan_current;
+ min_report_size = OCTO_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_QUADRO:
priv->kind = quadro;
@@ -1694,6 +1722,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_fan_power;
priv->voltage_label = label_fan_voltage;
priv->current_label = label_fan_current;
+ min_report_size = QUADRO_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_HIGHFLOWNEXT:
priv->kind = highflownext;
@@ -1711,6 +1740,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->speed_label = label_highflownext_fan_speed;
priv->power_label = label_highflownext_power;
priv->voltage_label = label_highflownext_voltage;
+ min_report_size = HIGHFLOWNEXT_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_LEAKSHIELD:
/*
@@ -1730,6 +1760,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->temp_label = label_leakshield_temp_sensors;
priv->speed_label = label_leakshield_fan_speed;
+ min_report_size = LEAKSHIELD_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_AQUASTREAMXT:
priv->kind = aquastreamxt;
@@ -1760,6 +1791,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_aquastreamult_power;
priv->voltage_label = label_aquastreamult_voltages;
priv->current_label = label_aquastreamult_current;
+ min_report_size = AQUASTREAMULT_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_POWERADJUST3:
priv->kind = poweradjust3;
@@ -1833,6 +1865,8 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
break;
}
+ priv->min_report_size = min_report_size;
+
if (priv->buffer_size != 0) {
priv->checksum_start = 0x01;
priv->checksum_length = priv->buffer_size - 3;
diff --git a/drivers/hwmon/asus_rog_ryujin.c b/drivers/hwmon/asus_rog_ryujin.c
index e297557ca346..a400e523d2e9 100644
--- a/drivers/hwmon/asus_rog_ryujin.c
+++ b/drivers/hwmon/asus_rog_ryujin.c
@@ -49,6 +49,7 @@ static const struct rog_ryujin_device_info rog_ryujin_iii_info = {
#define MAX_REPORT_LENGTH 65
/* Cooler duty report offsets */
+#define RYUJIN_PUMP_SOURCE 3
#define RYUJIN_PUMP_DUTY 4
#define RYUJIN_INTERNAL_FAN_DUTY 5
@@ -69,10 +70,15 @@ static const u8 get_cooler_duty_cmd[] = { RYUJIN_CMD_PREFIX, 0x9A };
static const u8 get_controller_speed_cmd[] = { RYUJIN_CMD_PREFIX, 0xA0 };
static const u8 get_controller_duty_cmd[] = { RYUJIN_CMD_PREFIX, 0xA1 };
+#define RYUJIN_SET_COOLER_CHANNEL_OFFSET 2
#define RYUJIN_SET_COOLER_PUMP_DUTY_OFFSET 3
#define RYUJIN_SET_COOLER_FAN_DUTY_OFFSET 4
static const u8 set_cooler_duty_cmd[] = { RYUJIN_CMD_PREFIX, 0x1A, 0x00, 0x00, 0x00 };
+/* Pump control sources in the set cooler duty command and the cooler duty report */
+#define RYUJIN_SOURCE_HEADER 0
+#define RYUJIN_SOURCE_USB 1
+
#define RYUJIN_SET_CONTROLLER_FAN_DUTY_OFFSET 4
static const u8 set_controller_duty_cmd[] = { RYUJIN_CMD_PREFIX, 0x21, 0x00, 0x00, 0x00 };
@@ -116,6 +122,7 @@ struct rog_ryujin_data {
s32 temp_input[1];
u16 speed_input[6]; /* Pump, internal fan and four controller fan speeds in RPM */
u8 duty_input[3]; /* Pump, internal fan and controller fan duty in PWM */
+ u8 pump_source; /* Pump control source from the cooler duty report */
u8 *buffer;
unsigned long updated; /* jiffies */
@@ -305,6 +312,44 @@ static int rog_ryujin_read_string(struct device *dev, enum hwmon_sensor_types ty
return 0;
}
+/*
+ * On devices with a non-zero duty_channel (the RYUJIN III), the channel byte of
+ * the set cooler duty command selects the pump control source.
+ * RYUJIN_SOURCE_HEADER hands the pump to its internal controller, which follows
+ * the PWM of the header its 4-pin cable is plugged into. RYUJIN_SOURCE_USB
+ * makes the pump follow the pump duty in the command. The cooler duty report
+ * returns the current source.
+ *
+ * A fan duty write sends the reported source back, so that it leaves the pump
+ * control source as it is. A pump duty write selects USB. The cooler does not
+ * pass on a pump duty equal to the one it reports, and after power-on it
+ * reports 40% while a pump handed over at that duty runs at full speed. So
+ * when a pump duty write asks for the reported duty, write a duty 1% away
+ * first.
+ */
+static int rog_ryujin_select_pump_source(struct rog_ryujin_data *priv, u8 *set_cmd,
+ int channel)
+{
+ u8 duty = set_cmd[RYUJIN_SET_COOLER_PUMP_DUTY_OFFSET];
+ int ret;
+
+ if (channel == 1) {
+ set_cmd[RYUJIN_SET_COOLER_CHANNEL_OFFSET] = priv->pump_source;
+ return 0;
+ }
+
+ set_cmd[RYUJIN_SET_COOLER_CHANNEL_OFFSET] = RYUJIN_SOURCE_USB;
+
+ if (duty != rog_ryujin_pwm_to_percent(priv->duty_input[0]))
+ return 0;
+
+ set_cmd[RYUJIN_SET_COOLER_PUMP_DUTY_OFFSET] = duty < 100 ? duty + 1 : duty - 1;
+ ret = rog_ryujin_execute_cmd(priv, set_cmd, SET_CMD_LENGTH, &priv->cooler_duty_set);
+ set_cmd[RYUJIN_SET_COOLER_PUMP_DUTY_OFFSET] = duty;
+
+ return ret;
+}
+
static int rog_ryujin_write_fixed_duty(struct rog_ryujin_data *priv, int channel, int val)
{
u8 set_cmd[SET_CMD_LENGTH];
@@ -322,7 +367,7 @@ static int rog_ryujin_write_fixed_duty(struct rog_ryujin_data *priv, int channel
return ret;
memcpy(set_cmd, set_cooler_duty_cmd, SET_CMD_LENGTH);
- set_cmd[2] = priv->info->duty_channel;
+ set_cmd[RYUJIN_SET_COOLER_CHANNEL_OFFSET] = priv->info->duty_channel;
/* Cooler duties are set as 0-100% */
val = rog_ryujin_pwm_to_percent(val);
@@ -339,6 +384,12 @@ static int rog_ryujin_write_fixed_duty(struct rog_ryujin_data *priv, int channel
set_cmd[RYUJIN_SET_COOLER_FAN_DUTY_OFFSET] = val;
}
+ if (priv->info->duty_channel) {
+ ret = rog_ryujin_select_pump_source(priv, set_cmd, channel);
+ if (ret < 0)
+ return ret;
+ }
+
return rog_ryujin_execute_cmd(priv, set_cmd, SET_CMD_LENGTH, &priv->cooler_duty_set);
} else {
/*
@@ -481,6 +532,7 @@ static int rog_ryujin_raw_event(struct hid_device *hdev, struct hid_report *repo
goto unlock;
}
read_cooler_duty:
+ priv->pump_source = data[RYUJIN_PUMP_SOURCE];
priv->duty_input[0] = rog_ryujin_percent_to_pwm(data[RYUJIN_PUMP_DUTY]);
priv->duty_input[1] = rog_ryujin_percent_to_pwm(data[RYUJIN_INTERNAL_FAN_DUTY]);
diff --git a/drivers/hwmon/asus_wmi_sensors.c b/drivers/hwmon/asus_wmi_sensors.c
index c2dd7ff882f2..502996e97cf1 100644
--- a/drivers/hwmon/asus_wmi_sensors.c
+++ b/drivers/hwmon/asus_wmi_sensors.c
@@ -164,6 +164,7 @@ struct asus_wmi_wmi_info {
struct asus_wmi_sensors {
struct asus_wmi_wmi_info wmi;
+ struct hwmon_chip_info chip_info;
/* lock access to internal cache */
struct mutex lock;
};
@@ -496,11 +497,6 @@ static const struct hwmon_ops asus_wmi_hwmon_ops = {
.read_string = asus_wmi_hwmon_read_string,
};
-static struct hwmon_chip_info asus_wmi_chip_info = {
- .ops = &asus_wmi_hwmon_ops,
- .info = NULL,
-};
-
static int asus_wmi_configure_sensor_setup(struct device *dev,
struct asus_wmi_sensors *sensor_data)
{
@@ -508,7 +504,7 @@ static int asus_wmi_configure_sensor_setup(struct device *dev,
struct hwmon_channel_info *asus_wmi_hwmon_chan;
int nr_count[hwmon_max] = {}, nr_types = 0;
struct asus_wmi_sensor_info *temp_sensor;
- const struct hwmon_chip_info *chip_info;
+ struct hwmon_chip_info *chip_info;
enum hwmon_sensor_types type;
struct device *hwdev;
int i, idx;
@@ -549,8 +545,9 @@ static int asus_wmi_configure_sensor_setup(struct device *dev,
if (!ptr_asus_wmi_ci)
return -ENOMEM;
- asus_wmi_chip_info.info = ptr_asus_wmi_ci;
- chip_info = &asus_wmi_chip_info;
+ chip_info = &sensor_data->chip_info;
+ chip_info->ops = &asus_wmi_hwmon_ops;
+ chip_info->info = ptr_asus_wmi_ci;
sensor_data->wmi.info_by_id = devm_kcalloc(dev, sensor_data->wmi.sensor_count,
sizeof(*sensor_data->wmi.info_by_id),
diff --git a/drivers/hwmon/chipcap2.c b/drivers/hwmon/chipcap2.c
index 9bef767b589e..00f62a33d463 100644
--- a/drivers/hwmon/chipcap2.c
+++ b/drivers/hwmon/chipcap2.c
@@ -20,7 +20,9 @@
#include <linux/i2c.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
+#include <linux/mutex.h>
#include <linux/module.h>
+#include <linux/unaligned.h>
#include <linux/regulator/consumer.h>
#include <linux/mod_devicetable.h>
#include <linux/property.h>
@@ -90,6 +92,9 @@ struct cc2_data {
int irq_low;
int irq_high;
bool process_irqs;
+ /* protects process_irqs and the alarm indicators */
+ struct mutex irq_lock;
+ bool regulator_enabled;
};
/* %RH as a per cent mille from a register value */
@@ -118,20 +123,33 @@ static int cc2_enable(struct cc2_data *data)
{
int ret;
+ if (data->irq_ready > 0) {
+ scoped_guard(mutex, &data->irq_lock)
+ data->process_irqs = false;
+
+ synchronize_irq(data->irq_ready);
+
+ scoped_guard(mutex, &data->irq_lock)
+ reinit_completion(&data->complete);
+ }
+
/* exclusive regulator, check in case a disable failed */
- if (regulator_is_enabled(data->regulator))
+ if (data->regulator_enabled) {
+ scoped_guard(mutex, &data->irq_lock)
+ data->process_irqs = true;
return 0;
-
- /* clear any pending completion */
- try_wait_for_completion(&data->complete);
+ }
ret = regulator_enable(data->regulator);
if (ret < 0)
return ret;
+ data->regulator_enabled = true;
+
usleep_range(CC2_STARTUP_TIME_US, CC2_STARTUP_TIME_US + 125);
- data->process_irqs = true;
+ scoped_guard(mutex, &data->irq_lock)
+ data->process_irqs = true;
return 0;
}
@@ -140,15 +158,36 @@ static void cc2_disable(struct cc2_data *data)
{
int err;
+ /* let the alarms already raised by the last measurement be processed */
+ if (data->irq_low > 0)
+ synchronize_irq(data->irq_low);
+
+ if (data->irq_high > 0)
+ synchronize_irq(data->irq_high);
+
/* ignore alarms triggered by voltage toggling when powering up */
- data->process_irqs = false;
+ scoped_guard(mutex, &data->irq_lock)
+ data->process_irqs = false;
+
+ if (data->irq_ready > 0)
+ synchronize_irq(data->irq_ready);
/* exclusive regulator, check in case an enable failed */
- if (regulator_is_enabled(data->regulator)) {
- err = regulator_disable(data->regulator);
- if (err)
- dev_dbg(&data->client->dev, "Failed to disable device");
+ if (!data->regulator_enabled)
+ return;
+
+ err = regulator_disable(data->regulator);
+ if (err) {
+ dev_dbg(&data->client->dev, "Failed to disable device");
+ return;
}
+
+ data->regulator_enabled = false;
+}
+
+static void cc2_disable_action(void *data)
+{
+ cc2_disable(data);
}
static int cc2_cmd_response_diagnostic(struct device *dev, u8 status)
@@ -167,13 +206,13 @@ static int cc2_cmd_response_diagnostic(struct device *dev, u8 status)
case CC2_RESPONSE_BUSY:
return -EBUSY;
case CC2_RESPONSE_NACK:
- if (resp & CC2_ERR_CORR_EEPROM)
+ if (status & CC2_ERR_CORR_EEPROM)
dev_dbg(dev, "Command failed: corrected EEPROM\n");
- if (resp & CC2_ERR_UNCORR_EEPROM)
+ if (status & CC2_ERR_UNCORR_EEPROM)
dev_dbg(dev, "Command failed: uncorrected EEPROM\n");
- if (resp & CC2_ERR_RAM_PARITY)
+ if (status & CC2_ERR_RAM_PARITY)
dev_dbg(dev, "Command failed: RAM parity\n");
- if (resp & CC2_ERR_RAM_PARITY)
+ if (status & CC2_ERR_CONFIG_LOAD)
dev_dbg(dev, "Command failed: configuration error\n");
return -ENODATA;
default:
@@ -226,7 +265,7 @@ static int cc2_command_mode_start(struct cc2_data *data)
2 * CC2_RESP_START_CM_US);
}
ret = cc2_read_command_status(data->client);
- if (ret != -ETIMEDOUT || i == CC2_CM_RETRIES)
+ if (ret != -ETIMEDOUT || i == CC2_CM_RETRIES - 1)
break;
/* command window missed, prepare for a retry */
@@ -260,8 +299,7 @@ static int cc2_write_reg(struct cc2_data *data, u8 reg, u16 val)
if (ret < 0)
goto disable;
- cpu_to_be16s(&val);
- ret = i2c_smbus_write_word_data(data->client, reg, val);
+ ret = i2c_smbus_write_word_swapped(data->client, reg, val);
if (ret < 0)
goto disable;
@@ -311,7 +349,7 @@ static int cc2_read_reg(struct cc2_data *data, u8 reg, u16 *val)
if (ret != CC2_EEPROM_DATA_LEN)
return ret < 0 ? ret : -EIO;
- *val = be16_to_cpup((__be16 *)&buf[1]);
+ *val = get_unaligned_be16(&buf[1]);
return cc2_read_command_status(data->client);
}
@@ -351,10 +389,10 @@ static int cc2_data_fetch(struct i2c_client *client,
switch (type) {
case hwmon_humidity:
- *val = cc2_rh_convert(be16_to_cpup((__be16 *)&data[0]));
+ *val = cc2_rh_convert(get_unaligned_be16(&data[0]));
break;
case hwmon_temp:
- *val = cc2_temp_convert(be16_to_cpup((__be16 *)&data[2]));
+ *val = cc2_temp_convert(get_unaligned_be16(&data[2]));
break;
default:
return -EINVAL;
@@ -482,8 +520,10 @@ static irqreturn_t cc2_ready_interrupt(int irq, void *data)
{
struct cc2_data *cc2 = data;
- if (cc2->process_irqs)
- complete(&cc2->complete);
+ scoped_guard(mutex, &cc2->irq_lock) {
+ if (cc2->process_irqs)
+ complete(&cc2->complete);
+ }
return IRQ_HANDLED;
}
@@ -492,10 +532,12 @@ static irqreturn_t cc2_low_interrupt(int irq, void *data)
{
struct cc2_data *cc2 = data;
- if (cc2->process_irqs) {
- hwmon_notify_event(cc2->hwmon, hwmon_humidity,
- hwmon_humidity_min_alarm, 0);
- cc2->rh_alarm.low_alarm = true;
+ scoped_guard(mutex, &cc2->irq_lock) {
+ if (cc2->process_irqs) {
+ hwmon_notify_event(cc2->hwmon, hwmon_humidity,
+ hwmon_humidity_min_alarm, 0);
+ cc2->rh_alarm.low_alarm = true;
+ }
}
return IRQ_HANDLED;
@@ -505,10 +547,12 @@ static irqreturn_t cc2_high_interrupt(int irq, void *data)
{
struct cc2_data *cc2 = data;
- if (cc2->process_irqs) {
- hwmon_notify_event(cc2->hwmon, hwmon_humidity,
- hwmon_humidity_max_alarm, 0);
- cc2->rh_alarm.high_alarm = true;
+ scoped_guard(mutex, &cc2->irq_lock) {
+ if (cc2->process_irqs) {
+ hwmon_notify_event(cc2->hwmon, hwmon_humidity,
+ hwmon_humidity_max_alarm, 0);
+ cc2->rh_alarm.high_alarm = true;
+ }
}
return IRQ_HANDLED;
@@ -524,11 +568,13 @@ static int cc2_humidity_min_alarm_status(struct cc2_data *data, long *val)
if (ret < 0)
return ret;
- if (data->rh_alarm.low_alarm) {
- *val = (measurement < min_hyst) ? 1 : 0;
- data->rh_alarm.low_alarm = *val;
- } else {
- *val = 0;
+ scoped_guard(mutex, &data->irq_lock) {
+ if (data->rh_alarm.low_alarm) {
+ *val = (measurement < min_hyst) ? 1 : 0;
+ data->rh_alarm.low_alarm = *val;
+ } else {
+ *val = 0;
+ }
}
return 0;
@@ -544,11 +590,13 @@ static int cc2_humidity_max_alarm_status(struct cc2_data *data, long *val)
if (ret < 0)
return ret;
- if (data->rh_alarm.high_alarm) {
- *val = (measurement > max_hyst) ? 1 : 0;
- data->rh_alarm.high_alarm = *val;
- } else {
- *val = 0;
+ scoped_guard(mutex, &data->irq_lock) {
+ if (data->rh_alarm.high_alarm) {
+ *val = (measurement > max_hyst) ? 1 : 0;
+ data->rh_alarm.high_alarm = *val;
+ } else {
+ *val = 0;
+ }
}
return 0;
@@ -631,28 +679,47 @@ static int cc2_write(struct device *dev, enum hwmon_sensor_types type, u32 attr,
}
}
-static int cc2_request_ready_irq(struct cc2_data *data, struct device *dev)
+static int cc2_get_irqs(struct cc2_data *data, struct device *dev)
{
- int ret = 0;
-
data->irq_ready = fwnode_irq_get_byname(dev_fwnode(dev), "ready");
- if (data->irq_ready > 0) {
+ if (data->irq_ready == -EPROBE_DEFER)
+ return data->irq_ready;
+ if (data->irq_ready > 0)
init_completion(&data->complete);
+
+ data->irq_low = fwnode_irq_get_byname(dev_fwnode(dev), "low");
+ if (data->irq_low == -EPROBE_DEFER)
+ return data->irq_low;
+ if (data->irq_low > 0)
+ data->rh_alarm.low_alarm_visible = true;
+
+ data->irq_high = fwnode_irq_get_byname(dev_fwnode(dev), "high");
+ if (data->irq_high == -EPROBE_DEFER)
+ return data->irq_high;
+ if (data->irq_high > 0)
+ data->rh_alarm.high_alarm_visible = true;
+
+ return 0;
+}
+
+static int cc2_request_data_ready(struct cc2_data *data, struct device *dev)
+{
+ int ret = 0;
+
+ if (data->irq_ready > 0)
ret = devm_request_threaded_irq(dev, data->irq_ready, NULL,
cc2_ready_interrupt,
IRQF_ONESHOT |
IRQF_TRIGGER_RISING,
dev_name(dev), data);
- }
return ret;
}
-static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev)
+static int cc2_request_alarms(struct cc2_data *data, struct device *dev)
{
int ret = 0;
- data->irq_low = fwnode_irq_get_byname(dev_fwnode(dev), "low");
if (data->irq_low > 0) {
ret = devm_request_threaded_irq(dev, data->irq_low, NULL,
cc2_low_interrupt,
@@ -661,11 +728,8 @@ static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev)
dev_name(dev), data);
if (ret)
return ret;
-
- data->rh_alarm.low_alarm_visible = true;
}
- data->irq_high = fwnode_irq_get_byname(dev_fwnode(dev), "high");
if (data->irq_high > 0) {
ret = devm_request_threaded_irq(dev, data->irq_high, NULL,
cc2_high_interrupt,
@@ -674,8 +738,6 @@ static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev)
dev_name(dev), data);
if (ret)
return ret;
-
- data->rh_alarm.high_alarm_visible = true;
}
return ret;
@@ -717,6 +779,8 @@ static int cc2_probe(struct i2c_client *client)
i2c_set_clientdata(client, data);
+ mutex_init(&data->irq_lock);
+
data->client = client;
data->regulator = devm_regulator_get_exclusive(dev, "vdd");
@@ -726,13 +790,17 @@ static int cc2_probe(struct i2c_client *client)
device_property_read_string(dev, "label", &data->label);
- ret = cc2_request_ready_irq(data, dev);
+ ret = cc2_get_irqs(data, dev);
if (ret)
- return dev_err_probe(dev, ret, "Failed to request ready irq\n");
+ return dev_err_probe(dev, ret, "Failed to get IRQs\n");
- ret = cc2_request_alarm_irqs(data, dev);
+ ret = devm_add_action_or_reset(dev, cc2_disable_action, data);
if (ret)
- return dev_err_probe(dev, ret, "Failed to request alarm irqs\n");
+ return ret;
+
+ ret = cc2_request_data_ready(data, dev);
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to request ready irq\n");
data->hwmon = devm_hwmon_device_register_with_info(dev, client->name,
data, &cc2_chip_info,
@@ -741,14 +809,11 @@ static int cc2_probe(struct i2c_client *client)
return dev_err_probe(dev, PTR_ERR(data->hwmon),
"Failed to register hwmon device\n");
- return 0;
-}
-
-static void cc2_remove(struct i2c_client *client)
-{
- struct cc2_data *data = i2c_get_clientdata(client);
+ ret = cc2_request_alarms(data, dev);
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to request alarm irqs\n");
- cc2_disable(data);
+ return 0;
}
static const struct i2c_device_id cc2_id[] = {
@@ -783,7 +848,6 @@ static struct i2c_driver cc2_driver = {
.of_match_table = cc2_of_match,
},
.probe = cc2_probe,
- .remove = cc2_remove,
.id_table = cc2_id,
};
module_i2c_driver(cc2_driver);
diff --git a/drivers/hwmon/coretemp.c b/drivers/hwmon/coretemp.c
index 9ed21c8c093a..2d7487b2f80a 100644
--- a/drivers/hwmon/coretemp.c
+++ b/drivers/hwmon/coretemp.c
@@ -491,11 +491,12 @@ init_temp_data(struct platform_data *pdata, unsigned int cpu, int pkg_flag)
struct temp_data *tdata;
if (!pdata->core_data) {
- pdata->nr_cores = topology_num_cores_per_package();
- pdata->core_data = kzalloc_objs(struct temp_data *,
- pdata->nr_cores);
+ int nr_cores = topology_num_cores_per_package();
+
+ pdata->core_data = kzalloc_objs(struct temp_data *, nr_cores);
if (!pdata->core_data)
return NULL;
+ pdata->nr_cores = nr_cores;
}
tdata = kzalloc_obj(struct temp_data);
@@ -695,6 +696,9 @@ static int coretemp_cpu_online(unsigned int cpu)
if (!housekeeping_cpu(cpu, HK_TYPE_MISC))
return 0;
+ if (!pdev)
+ return 0;
+
pdata = platform_get_drvdata(pdev);
if (!pdata->hwmon_dev) {
struct device *hwmon;
@@ -746,6 +750,9 @@ static int coretemp_cpu_offline(unsigned int cpu)
return 0;
/* If the physical CPU device does not exist, just return */
+ if (!pdev)
+ return 0;
+
pd = platform_get_drvdata(pdev);
if (!pd->hwmon_dev)
return 0;
diff --git a/drivers/hwmon/corsair-cpro.c b/drivers/hwmon/corsair-cpro.c
index e6d31d367698..6ce5da37de03 100644
--- a/drivers/hwmon/corsair-cpro.c
+++ b/drivers/hwmon/corsair-cpro.c
@@ -215,10 +215,11 @@ static int set_target(struct ccp_device *ccp, int channel, long val)
int ret;
val = clamp_val(val, 0, 0xFFFF);
- ccp->target[channel] = val;
mutex_lock(&ccp->mutex);
ret = send_usb_cmd(ccp, CTL_SET_FAN_TARGET, channel, val >> 8, val);
+ if (!ret)
+ ccp->target[channel] = val;
mutex_unlock(&ccp->mutex);
return ret;
diff --git a/drivers/hwmon/gpd-fan.c b/drivers/hwmon/gpd-fan.c
index ed212615b2fe..b3b75f99f588 100644
--- a/drivers/hwmon/gpd-fan.c
+++ b/drivers/hwmon/gpd-fan.c
@@ -444,6 +444,7 @@ static void gpd_duo_set_pwm_enable(struct gpd_fan_data *data, enum FAN_PWM_ENABL
break;
case AUTOMATIC:
gpd_ecram_write(data->drvdata, data->drvdata->pwm_write, 0);
+ gpd_ecram_write(data->drvdata, data->drvdata->pwm_write + 1, 0);
break;
}
}
diff --git a/drivers/hwmon/gpio-fan.c b/drivers/hwmon/gpio-fan.c
index 3f78375eeb44..08b58b49be5b 100644
--- a/drivers/hwmon/gpio-fan.c
+++ b/drivers/hwmon/gpio-fan.c
@@ -150,7 +150,10 @@ static int set_fan_speed(struct gpio_fan_data *fan_data, int speed_index)
__set_fan_ctrl(fan_data, fan_data->speed[speed_index].ctrl_val);
- if (fan_data->speed_index > 0 && speed_index == 0) {
+ /* The GPIOs have changed, and a PM put drops the reference on error. */
+ fan_data->speed_index = speed_index;
+
+ if (speed_index == 0) {
int ret;
ret = pm_runtime_put_sync(fan_data->dev);
@@ -158,8 +161,6 @@ static int set_fan_speed(struct gpio_fan_data *fan_data, int speed_index)
return ret;
}
- fan_data->speed_index = speed_index;
-
return 0;
}
diff --git a/drivers/hwmon/hp-wmi-sensors.c b/drivers/hwmon/hp-wmi-sensors.c
index cfa0d6ad0fb3..12950666c981 100644
--- a/drivers/hwmon/hp-wmi-sensors.c
+++ b/drivers/hwmon/hp-wmi-sensors.c
@@ -390,6 +390,7 @@ struct hp_wmi_info {
*/
struct hp_wmi_sensors {
struct wmi_device *wdev;
+ struct hwmon_chip_info chip_info;
struct hp_wmi_info **info_map[hwmon_max];
u8 channel_count[hwmon_max];
bool has_intrusion;
@@ -738,15 +739,21 @@ static long scale_numeric_sensor(const struct hp_wmi_numeric_sensor *nsensor)
val = current_reading;
- for (; unit_modifier < target_modifier; unit_modifier++)
+ for (; val && unit_modifier < target_modifier; unit_modifier++)
val = DIV_ROUND_CLOSEST(val, 10);
- for (; unit_modifier > target_modifier; unit_modifier--) {
- if (val > LONG_MAX / 10) {
- val = LONG_MAX;
- break;
+ if (val) {
+ for (; unit_modifier > target_modifier; unit_modifier--) {
+ if (val > LONG_MAX / 10) {
+ val = LONG_MAX;
+ break;
+ }
+ if (val < LONG_MIN / 10) {
+ val = LONG_MIN;
+ break;
+ }
+ val *= 10;
}
- val *= 10;
}
if (sensor_type == HP_WMI_TYPE_TEMPERATURE) {
@@ -1531,11 +1538,6 @@ static const struct hwmon_ops hp_wmi_hwmon_ops = {
.write = hp_wmi_hwmon_write,
};
-static struct hwmon_chip_info hp_wmi_chip_info = {
- .ops = &hp_wmi_hwmon_ops,
- .info = NULL,
-};
-
static struct hp_wmi_info *match_fan_event(struct hp_wmi_sensors *state,
const char *event_description)
{
@@ -1962,7 +1964,8 @@ static int make_chip_info(struct hp_wmi_sensors *state, bool has_events)
if (!ptr_channel_info)
return -ENOMEM;
- hp_wmi_chip_info.info = ptr_channel_info;
+ state->chip_info.ops = &hp_wmi_hwmon_ops;
+ state->chip_info.info = ptr_channel_info;
for (type = hwmon_chip; type < hwmon_max; type++) {
count = channel_count[type];
@@ -2053,7 +2056,7 @@ static int hp_wmi_sensors_init(struct hp_wmi_sensors *state)
return err;
hwdev = devm_hwmon_device_register_with_info(dev, "hp_wmi_sensors",
- state, &hp_wmi_chip_info,
+ state, &state->chip_info,
NULL);
return PTR_ERR_OR_ZERO(hwdev);
}
diff --git a/drivers/hwmon/lm75.c b/drivers/hwmon/lm75.c
index 2d2d752aeac9..917ac250a648 100644
--- a/drivers/hwmon/lm75.c
+++ b/drivers/hwmon/lm75.c
@@ -123,8 +123,7 @@ struct lm75_data {
static const u8 lm75_sample_set_masks[] = { 0 << 5, 1 << 5, 2 << 5, 3 << 5 };
-#define LM75_ALERT_POLARITY_HIGH_8_BIT (BIT(2))
-#define LM75_ALERT_POLARITY_HIGH_16_BIT (BIT(2) << 8)
+#define LM75_ALERT_POLARITY_HIGH_BIT BIT(2)
#define LM75_SAMPLE_CLEAR_MASK (3 << 5)
/* The structure below stores the configuration values of the supported devices.
@@ -770,11 +769,7 @@ static int lm75_generic_probe(struct device *dev, const char *name,
data->orig_conf = status;
/* Enforce polarity active-low (default) or active-high (devicetree) */
- if (!data->params->config_reg_16bits)
- pol_mask = LM75_ALERT_POLARITY_HIGH_8_BIT;
- else
- pol_mask = LM75_ALERT_POLARITY_HIGH_16_BIT;
-
+ pol_mask = LM75_ALERT_POLARITY_HIGH_BIT;
clr_mask = data->params->clr_mask | pol_mask;
set_mask = data->params->set_mask & ~pol_mask;
if (device_property_read_bool(dev, "ti,alert-polarity-active-high"))
diff --git a/drivers/hwmon/lm87.c b/drivers/hwmon/lm87.c
index c09074f44708..bb4e15b66355 100644
--- a/drivers/hwmon/lm87.c
+++ b/drivers/hwmon/lm87.c
@@ -606,13 +606,16 @@ static DEVICE_ATTR_RO(cpu0_vid);
static ssize_t vrm_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- struct lm87_data *data = dev_get_drvdata(dev);
+ struct i2c_client *client = dev_get_drvdata(dev);
+ struct lm87_data *data = i2c_get_clientdata(client);
+
return sprintf(buf, "%d\n", data->vrm);
}
static ssize_t vrm_store(struct device *dev, struct device_attribute *attr,
const char *buf, size_t count)
{
- struct lm87_data *data = dev_get_drvdata(dev);
+ struct i2c_client *client = dev_get_drvdata(dev);
+ struct lm87_data *data = i2c_get_clientdata(client);
unsigned long val;
int err;
diff --git a/drivers/hwmon/nct6683.c b/drivers/hwmon/nct6683.c
index e1c36c95affb..f34cb62305be 100644
--- a/drivers/hwmon/nct6683.c
+++ b/drivers/hwmon/nct6683.c
@@ -920,7 +920,7 @@ static ssize_t
show_pwm(struct device *dev, struct device_attribute *attr, char *buf)
{
struct nct6683_data *data = nct6683_update_device(dev);
- struct sensor_device_attribute_2 *sattr = to_sensor_dev_attr_2(attr);
+ struct sensor_device_attribute *sattr = to_sensor_dev_attr(attr);
int index = sattr->index;
return sprintf(buf, "%d\n", data->pwm[index]);
@@ -930,7 +930,7 @@ static ssize_t
store_pwm(struct device *dev, struct device_attribute *attr, const char *buf,
size_t count)
{
- struct sensor_device_attribute_2 *sattr = to_sensor_dev_attr_2(attr);
+ struct sensor_device_attribute *sattr = to_sensor_dev_attr(attr);
struct nct6683_data *data = dev_get_drvdata(dev);
int index = sattr->index;
unsigned long val;
diff --git a/drivers/hwmon/peci/dimmtemp.c b/drivers/hwmon/peci/dimmtemp.c
index bd3e8715dfec..617a4cc0f65c 100644
--- a/drivers/hwmon/peci/dimmtemp.c
+++ b/drivers/hwmon/peci/dimmtemp.c
@@ -4,7 +4,6 @@
#include <linux/auxiliary_bus.h>
#include <linux/bitfield.h>
#include <linux/bitops.h>
-#include <linux/devm-helpers.h>
#include <linux/hwmon.h>
#include <linux/jiffies.h>
#include <linux/module.h>
@@ -326,11 +325,13 @@ static int create_dimm_temp_info(struct peci_dimmtemp *priv)
if (ret == -ENODEV) {
dev_dbg(priv->dev, "No DIMMs found\n");
return 0;
- } else if (ret) {
+ }
+ if (ret == -EAGAIN) {
schedule_delayed_work(&priv->detect_work, DIMM_MASK_CHECK_DELAY_JIFFIES);
dev_dbg(priv->dev, "Deferred populating DIMM temp info\n");
- return ret;
}
+ if (ret)
+ return ret;
channels = priv->gen_info->chan_rank_max * priv->gen_info->dimm_idx_max;
@@ -398,10 +399,8 @@ static int peci_dimmtemp_probe(struct auxiliary_device *adev, const struct auxil
"Unexpected PECI revision %#x, some features may be unavailable\n",
peci_dev->info.peci_revision);
- ret = devm_delayed_work_autocancel(priv->dev, &priv->detect_work,
- create_dimm_temp_info_delayed);
- if (ret)
- return ret;
+ auxiliary_set_drvdata(adev, priv);
+ INIT_DELAYED_WORK(&priv->detect_work, create_dimm_temp_info_delayed);
ret = create_dimm_temp_info(priv);
if (ret && ret != -EAGAIN) {
@@ -412,6 +411,13 @@ static int peci_dimmtemp_probe(struct auxiliary_device *adev, const struct auxil
return 0;
}
+static void peci_dimmtemp_remove(struct auxiliary_device *adev)
+{
+ struct peci_dimmtemp *priv = auxiliary_get_drvdata(adev);
+
+ cancel_delayed_work_sync(&priv->detect_work);
+}
+
static int
read_thresholds_hsx(struct peci_dimmtemp *priv, int dimm_order, int chan_rank, u32 *data)
{
@@ -665,6 +671,7 @@ MODULE_DEVICE_TABLE(auxiliary, peci_dimmtemp_ids);
static struct auxiliary_driver peci_dimmtemp_driver = {
.probe = peci_dimmtemp_probe,
+ .remove = peci_dimmtemp_remove,
.id_table = peci_dimmtemp_ids,
};
diff --git a/drivers/hwmon/pmbus/pmbus_core.c b/drivers/hwmon/pmbus/pmbus_core.c
index 5f69c1420b4e..b033bd3b1527 100644
--- a/drivers/hwmon/pmbus/pmbus_core.c
+++ b/drivers/hwmon/pmbus/pmbus_core.c
@@ -1507,11 +1507,14 @@ static int pmbus_thermal_add_sensor(struct pmbus_data *pmbus_data,
tzd = devm_thermal_of_zone_register(dev, index, tdata,
&pmbus_thermal_ops);
/*
- * If CONFIG_THERMAL_OF is disabled, this returns -ENODEV,
- * so ignore that error but forward any other error.
+ * If CONFIG_THERMAL_OF is disabled, devm_thermal_of_zone_register()
+ * returns -ENOTSUPP. If no thermal zone is configured for the sensor,
+ * it returns -ENODEV. Ignore those errors but forward any other error.
*/
- if (IS_ERR(tzd) && (PTR_ERR(tzd) != -ENODEV))
+ if (IS_ERR(tzd) && PTR_ERR(tzd) != -ENODEV && PTR_ERR(tzd) != -ENOTSUPP) {
+ devm_kfree(dev, tdata);
return PTR_ERR(tzd);
+ }
return 0;
}
@@ -1526,10 +1529,11 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data,
{
struct sensor_device_attribute *a;
struct pmbus_sensor *sensor;
+ int err;
sensor = devm_kzalloc(data->dev, sizeof(*sensor), GFP_KERNEL);
if (!sensor)
- return NULL;
+ return ERR_PTR(-ENOMEM);
a = &sensor->attribute;
if (type)
@@ -1554,14 +1558,17 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data,
pmbus_set_sensor, -1);
if (pmbus_add_attribute(data, &a->dev_attr.attr))
- return NULL;
+ return ERR_PTR(-ENOMEM);
sensor->next = data->sensors;
data->sensors = sensor;
/* temperature sensors with _input values are registered with thermal */
- if (class == PSC_TEMPERATURE && strcmp(type, "input") == 0)
- pmbus_thermal_add_sensor(data, sensor, seq);
+ if (class == PSC_TEMPERATURE && strcmp(type, "input") == 0) {
+ err = pmbus_thermal_add_sensor(data, sensor, seq);
+ if (err)
+ return ERR_PTR(err);
+ }
return sensor;
}
@@ -1662,8 +1669,8 @@ static int pmbus_add_limit_attrs(struct i2c_client *client,
page, 0xff, l->reg, attr->class,
attr->update || l->update,
l->readonly, l->writeonly, true);
- if (!curr)
- return -ENOMEM;
+ if (IS_ERR(curr))
+ return PTR_ERR(curr);
if (l->sbit && (info->func[page] & attr->sfunc)) {
ret = pmbus_add_boolean(data, name,
l->alarm, index,
@@ -1702,8 +1709,8 @@ static int pmbus_add_sensor_attrs_one(struct i2c_client *client,
}
base = pmbus_add_sensor(data, name, "input", index, page, phase,
attr->reg, attr->class, true, true, false, true);
- if (!base)
- return -ENOMEM;
+ if (IS_ERR(base))
+ return PTR_ERR(base);
/* No limit and alarm attributes for phase specific sensors */
if (attr->sfunc && phase == 0xff) {
ret = pmbus_add_limit_attrs(client, data, info, name,
@@ -2376,8 +2383,8 @@ static int pmbus_add_fan_ctrl(struct i2c_client *client,
0xff, PMBUS_VIRT_FAN_TARGET_1 + id, PSC_FAN,
false, false, false, true);
- if (!sensor)
- return -ENOMEM;
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
if (!((data->info->func[page] & PMBUS_HAVE_PWM12) ||
(data->info->func[page] & PMBUS_HAVE_PWM34)))
@@ -2387,15 +2394,15 @@ static int pmbus_add_fan_ctrl(struct i2c_client *client,
0xff, PMBUS_VIRT_PWM_1 + id, PSC_PWM,
false, false, false, true);
- if (!sensor)
- return -ENOMEM;
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
sensor = pmbus_add_sensor(data, "pwm", "enable", index, page,
0xff, PMBUS_VIRT_PWM_ENABLE_1 + id, PSC_PWM,
true, false, false, false);
- if (!sensor)
- return -ENOMEM;
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
return 0;
}
@@ -2412,6 +2419,7 @@ static int pmbus_add_fan_attributes(struct i2c_client *client,
int f;
for (f = 0; f < ARRAY_SIZE(pmbus_fan_registers); f++) {
+ struct pmbus_sensor *sensor;
int regval;
if (!(info->func[page] & pmbus_fan_flags[f]))
@@ -2432,10 +2440,11 @@ static int pmbus_add_fan_attributes(struct i2c_client *client,
(!(regval & (PB_FAN_1_INSTALLED >> ((f & 1) * 4)))))
continue;
- if (pmbus_add_sensor(data, "fan", "input", index,
- page, 0xff, pmbus_fan_registers[f],
- PSC_FAN, true, true, false, true) == NULL)
- return -ENOMEM;
+ sensor = pmbus_add_sensor(data, "fan", "input", index,
+ page, 0xff, pmbus_fan_registers[f],
+ PSC_FAN, true, true, false, true);
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
/* Fan control */
if (pmbus_check_word_register(client, page,
diff --git a/drivers/hwmon/pmbus/q54sj108a2.c b/drivers/hwmon/pmbus/q54sj108a2.c
index ecac405d7536..b018a611d9b6 100644
--- a/drivers/hwmon/pmbus/q54sj108a2.c
+++ b/drivers/hwmon/pmbus/q54sj108a2.c
@@ -351,6 +351,8 @@ static int q54sj108a2_probe(struct i2c_client *client)
psu->client = client;
debugfs = pmbus_get_debugfs_dir(client);
+ if (!debugfs)
+ return 0;
q54sj108a2_dir = debugfs_create_dir(client->name, debugfs);
diff --git a/drivers/hwmon/sht15.c b/drivers/hwmon/sht15.c
index 7bb2105d1ae2..48d93c61345c 100644
--- a/drivers/hwmon/sht15.c
+++ b/drivers/hwmon/sht15.c
@@ -31,6 +31,7 @@
#include <linux/bitrev.h>
#include <linux/gpio/consumer.h>
#include <linux/of.h>
+#include <linux/workqueue.h>
/* Commands */
#define SHT15_MEASURE_TEMP 0x03
@@ -976,25 +977,22 @@ static int sht15_probe(struct platform_device *pdev)
goto err_release_reg;
}
- ret = devm_request_irq(&pdev->dev, gpiod_to_irq(data->data),
- sht15_interrupt_fired,
- IRQF_TRIGGER_FALLING,
- "sht15 data",
- data);
+ ret = request_irq(gpiod_to_irq(data->data), sht15_interrupt_fired,
+ IRQF_TRIGGER_FALLING, "sht15 data", data);
if (ret)
goto err_release_reg;
disable_irq_nosync(gpiod_to_irq(data->data));
ret = sht15_connection_reset(data);
if (ret)
- goto err_release_reg;
+ goto err_free_irq;
ret = sht15_soft_reset(data);
if (ret)
- goto err_release_reg;
+ goto err_free_irq;
ret = sysfs_create_group(&pdev->dev.kobj, &sht15_attr_group);
if (ret) {
dev_err(&pdev->dev, "sysfs create failed\n");
- goto err_release_reg;
+ goto err_free_irq;
}
data->hwmon_dev = hwmon_device_register(data->dev);
@@ -1007,9 +1005,13 @@ static int sht15_probe(struct platform_device *pdev)
err_release_sysfs_group:
sysfs_remove_group(&pdev->dev.kobj, &sht15_attr_group);
+err_free_irq:
+ disable_work_sync(&data->read_work);
+ free_irq(gpiod_to_irq(data->data), data);
err_release_reg:
if (!IS_ERR(data->reg)) {
regulator_unregister_notifier(data->reg, &data->nb);
+ cancel_work_sync(&data->update_supply_work);
regulator_disable(data->reg);
}
return ret;
@@ -1023,12 +1025,16 @@ static void sht15_remove(struct platform_device *pdev)
hwmon_device_unregister(data->hwmon_dev);
sysfs_remove_group(&pdev->dev.kobj, &sht15_attr_group);
+ disable_work_sync(&data->read_work);
+ free_irq(gpiod_to_irq(data->data), data);
+
ret = sht15_soft_reset(data);
if (ret)
dev_err(&pdev->dev, "Failed to reset device (%pe)\n", ERR_PTR(ret));
if (!IS_ERR(data->reg)) {
regulator_unregister_notifier(data->reg, &data->nb);
+ cancel_work_sync(&data->update_supply_work);
regulator_disable(data->reg);
}
}
diff --git a/drivers/hwmon/sht4x.c b/drivers/hwmon/sht4x.c
index 0bb9227674a8..f38c106b7090 100644
--- a/drivers/hwmon/sht4x.c
+++ b/drivers/hwmon/sht4x.c
@@ -99,7 +99,9 @@ static int sht4x_read_values(struct sht4x_data *data)
msleep(jiffies_to_msecs(data->heating_complete - curr_jiffies));
if (data->data_pending &&
- time_before64(get_jiffies_64(), data->heating_complete + data->update_interval)) {
+ time_before64(get_jiffies_64(),
+ data->heating_complete +
+ msecs_to_jiffies(data->update_interval))) {
data->data_pending = false;
} else {
next_update = data->last_updated +
diff --git a/drivers/hwmon/tmp108.c b/drivers/hwmon/tmp108.c
index 91d20a1ce112..862b944d3e60 100644
--- a/drivers/hwmon/tmp108.c
+++ b/drivers/hwmon/tmp108.c
@@ -13,7 +13,7 @@
#include <linux/i2c.h>
#include <linux/i3c/device.h>
#include <linux/init.h>
-#include <linux/jiffies.h>
+#include <linux/ktime.h>
#include <linux/regmap.h>
#include <linux/regulator/consumer.h>
#include <linux/slab.h>
@@ -78,7 +78,7 @@
struct tmp108 {
struct regmap *regmap;
u16 orig_config;
- u64 ready_time;
+ ktime_t ready_time;
const struct tmp108_params *params;
};
@@ -121,6 +121,7 @@ static int tmp108_read(struct device *dev, enum hwmon_sensor_types type,
struct tmp108 *tmp108 = dev_get_drvdata(dev);
unsigned int regval;
int err, hyst;
+ s64 delay;
if (type == hwmon_chip) {
if (attr == hwmon_chip_update_interval) {
@@ -137,12 +138,10 @@ static int tmp108_read(struct device *dev, enum hwmon_sensor_types type,
switch (attr) {
case hwmon_temp_input:
- /* Is it too early to return a conversion ? */
- if (time_before64(get_jiffies_64(), tmp108->ready_time)) {
- dev_dbg(dev, "%s: Conversion not ready yet..\n",
- __func__);
- return -EAGAIN;
- }
+ /* Wait if it is too early to return a conversion */
+ delay = ktime_us_delta(tmp108->ready_time, ktime_get());
+ if (delay > 0)
+ usleep_range(delay, delay + 1000);
err = regmap_read(tmp108->regmap, TMP108_REG_TEMP, ®val);
if (err < 0)
return err;
@@ -477,11 +476,8 @@ static int tmp108_common_probe(struct device *dev, struct regmap *regmap, char *
return err;
}
- tmp108->ready_time = get_jiffies_64();
- if ((tmp108->orig_config & TMP108_CONF_MODE_MASK) ==
- TMP108_MODE_SHUTDOWN)
- tmp108->ready_time +=
- msecs_to_jiffies(TMP108_CONVERSION_TIME_MS);
+ tmp108->ready_time = ktime_add_ms(ktime_get(),
+ TMP108_CONVERSION_TIME_MS);
err = devm_add_action_or_reset(dev, tmp108_restore_config, tmp108);
if (err) {
@@ -528,8 +524,8 @@ static int tmp108_resume(struct device *dev)
err = regmap_update_bits(tmp108->regmap, TMP108_REG_CONF,
TMP108_CONF_MODE_MASK, TMP108_MODE_CONTINUOUS);
- tmp108->ready_time = get_jiffies_64() +
- msecs_to_jiffies(TMP108_CONVERSION_TIME_MS);
+ tmp108->ready_time = ktime_add_ms(ktime_get(),
+ TMP108_CONVERSION_TIME_MS);
return err;
}
diff --git a/drivers/idle/intel_idle.c b/drivers/idle/intel_idle.c
index 651408df9c24..ec951c148d4a 100644
--- a/drivers/idle/intel_idle.c
+++ b/drivers/idle/intel_idle.c
@@ -1905,8 +1905,13 @@ static void __init intel_idle_init_cstates_acpi_lpi(struct cpuidle_driver *drv)
state = &drv->states[drv->state_count++];
- scnprintf(state->name, CPUIDLE_NAME_LEN, "C%d_LPI", index + 1);
- strscpy(state->desc, lpi_state->desc, CPUIDLE_DESC_LEN);
+ if (strlen(lpi_state->desc) > 0)
+ strscpy(state->name, lpi_state->desc);
+ else
+ scnprintf(state->name, CPUIDLE_NAME_LEN, "C%d_LPI", index + 1);
+
+ scnprintf(state->desc, CPUIDLE_DESC_LEN, "ACPI LPI MWAIT 0x%02llx",
+ lpi_state->address);
state->exit_latency = lpi_state->wake_latency;
state->target_residency = lpi_state->min_residency;
state->flags = MWAIT2flg(lpi_state->address);
@@ -1956,7 +1961,7 @@ static void __init intel_idle_init_cstates_acpi_cst(struct cpuidle_driver *drv)
state = &drv->states[drv->state_count++];
snprintf(state->name, CPUIDLE_NAME_LEN, "C%d_ACPI", cstate);
- strscpy(state->desc, cx->desc, CPUIDLE_DESC_LEN);
+ strscpy(state->desc, cx->desc);
state->exit_latency = cx->latency;
/*
* For C1-type C-states use the same number for both the exit
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index 0506617b27c2..cfc061a4247a 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -1720,6 +1720,21 @@ static int __init init_iommu_from_acpi(struct amd_iommu *iommu,
return 0;
}
+/*
+ * Apply workaround for Zen1 to Zen5(BRH) generation of processors.
+ * References: CVE-2024-21963
+ */
+static bool __init amd_iommu_force_max_dev_table(void)
+{
+ if (boot_cpu_data.x86 >= 0x17 && boot_cpu_data.x86 <= 0x1a) {
+ if (cpu_feature_enabled(X86_FEATURE_ZEN6))
+ return false;
+ else
+ return true;
+ }
+ return false;
+}
+
/* Allocate PCI segment data structure */
static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id,
struct acpi_table_header *ivrs_base)
@@ -1742,9 +1757,15 @@ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id,
pci_seg->last_bdf = last_bdf;
DUMP_printk("PCI segment : 0x%0x, last bdf : 0x%04x\n", id, last_bdf);
- pci_seg->dev_table_size =
- max(roundup_pow_of_two((last_bdf + 1) * DEV_TABLE_ENTRY_SIZE),
- SZ_4K);
+ if (amd_iommu_force_max_dev_table()) {
+ pr_info("Force device table to maximum size\n");
+ pci_seg->dev_table_size = SZ_2M;
+ } else {
+ pci_seg->dev_table_size =
+ max(roundup_pow_of_two((last_bdf + 1) *
+ DEV_TABLE_ENTRY_SIZE),
+ SZ_4K);
+ }
pci_seg->id = id;
init_llist_head(&pci_seg->dev_data_list);
@@ -2708,6 +2729,15 @@ static int __init init_unity_map_range(struct ivmd_header *m,
return 0;
}
+static bool ivrs_block_type_ivmd(u8 type)
+{
+ if (type == ACPI_IVMD_TYPE ||
+ type == ACPI_IVMD_TYPE_ALL ||
+ type == ACPI_IVMD_TYPE_RANGE)
+ return true;
+ return false;
+}
+
/* iterates over all memory definitions we find in the ACPI table */
static int __init init_memory_definitions(struct acpi_table_header *table)
{
@@ -2719,10 +2749,11 @@ static int __init init_memory_definitions(struct acpi_table_header *table)
while (p < end) {
m = (struct ivmd_header *)p;
+ p += m->length;
+ if (!ivrs_block_type_ivmd(m->type))
+ continue;
if (m->flags & (IVMD_FLAG_UNITY_MAP | IVMD_FLAG_EXCL_RANGE))
init_unity_map_range(m, table);
-
- p += m->length;
}
return 0;
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 893cd2be11a2..b19496f8c1b0 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -489,6 +489,10 @@ static void setup_aliases(struct amd_iommu *iommu, struct device *dev)
clone_aliases(iommu, dev);
}
+/*
+ * We keep dev_data around for unplugged devices and reuse it when the
+ * device is re-plugged - not doing so would introduce a ton of races.
+ */
static struct iommu_dev_data *find_dev_data(struct amd_iommu *iommu, u16 devid)
{
struct iommu_dev_data *dev_data;
@@ -2548,18 +2552,6 @@ static struct iommu_device *amd_iommu_probe_device(struct device *dev)
return iommu_dev;
}
-static void amd_iommu_release_device(struct device *dev)
-{
- struct iommu_dev_data *dev_data = dev_iommu_priv_get(dev);
-
- WARN_ON(dev_data->domain);
-
- /*
- * We keep dev_data around for unplugged devices and reuse it when the
- * device is re-plugged - not doing so would introduce a ton of races.
- */
-}
-
static struct iommu_group *amd_iommu_device_group(struct device *dev)
{
if (dev_is_pci(dev))
@@ -3205,7 +3197,6 @@ const struct iommu_ops amd_iommu_ops = {
.domain_alloc_paging_flags = amd_iommu_domain_alloc_paging_flags,
.domain_alloc_sva = amd_iommu_domain_alloc_sva,
.probe_device = amd_iommu_probe_device,
- .release_device = amd_iommu_release_device,
.device_group = amd_iommu_device_group,
.get_resv_regions = amd_iommu_get_resv_regions,
.is_attach_deferred = amd_iommu_is_attach_deferred,
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index 0a429c64fbf3..3f50298a1c14 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -215,6 +215,13 @@ bool arm_smmu_sva_supported(struct arm_smmu_device *smmu)
if (system_supports_haft())
feat_mask |= ARM_SMMU_FEAT_HAFT;
+ /*
+ * The workaround for ARM_SMMU_OPT_FULL_CONT_RANGE_INV requires range
+ * invalidation support.
+ */
+ if (smmu->options & ARM_SMMU_OPT_FULL_CONT_RANGE_INV)
+ feat_mask |= ARM_SMMU_FEAT_RANGE_INV;
+
if ((smmu->features & feat_mask) != feat_mask)
return false;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 81baafaffc41..b510dd47f231 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2538,6 +2538,37 @@ static void arm_smmu_cmdq_batch_add_range(struct arm_smmu_device *smmu,
}
}
+/*
+ * Generate a range invalidation for ARM_SMMU_OPT_FULL_CONT_RANGE_INV by
+ * ensuring the entire SVA requested range is covered with a single range
+ * invalidation command. The scale is adjusted so that the range invalidation
+ * may extend past the end of the requested range. This ensures that any CONT
+ * the MM is invalidating is covered by a single range invalidation. TTL and
+ * LEAF are always 0 because this is only used by SVA.
+ */
+static bool arm_smmu_cmdq_batch_add_range_inv(struct arm_smmu_device *smmu,
+ struct arm_smmu_cmdq_batch *cmds,
+ struct arm_smmu_cmd *cmd,
+ unsigned long iova, size_t size,
+ u8 tgsz_lg2)
+{
+ u64 cur_tg = iova >> tgsz_lg2;
+ u64 num_tg = ((iova + size - 1) >> tgsz_lg2) - cur_tg + 1;
+ unsigned int scale = fls64((num_tg - 1) / 32);
+
+ if (scale > 31)
+ return false;
+
+ cmd->data[0] |=
+ FIELD_PREP(CMDQ_TLBI_0_NUM,
+ DIV_ROUND_UP_ULL(num_tg, 1ULL << scale) - 1) |
+ FIELD_PREP(CMDQ_TLBI_0_SCALE, scale);
+ cmd->data[1] = FIELD_PREP(CMDQ_TLBI_1_TG, (tgsz_lg2 - 10) / 2) |
+ (cur_tg << tgsz_lg2);
+ arm_smmu_cmdq_batch_add_cmd_p(smmu, cmds, cmd);
+ return true;
+}
+
static bool arm_smmu_inv_size_too_big(struct arm_smmu_device *smmu, size_t size,
size_t granule)
{
@@ -2565,21 +2596,30 @@ static bool arm_smmu_inv_size_too_big(struct arm_smmu_device *smmu, size_t size,
static void arm_smmu_inv_to_cmdq_batch(struct arm_smmu_inv *inv,
struct arm_smmu_cmdq_batch *cmds,
struct arm_smmu_cmd *cmd,
- bool leaf,
+ bool single_range_inv, bool leaf,
unsigned long iova, size_t size,
unsigned int granule)
{
- if (arm_smmu_inv_size_too_big(inv->smmu, size, granule)) {
- struct arm_smmu_cmd nsize_cmd = *cmd;
+ struct arm_smmu_cmd nsize_cmd;
+
+ if (arm_smmu_inv_size_too_big(inv->smmu, size, granule))
+ goto full_inv;
- u64p_replace_bits(&nsize_cmd.data[0], inv->nsize_opcode,
- CMDQ_0_OP);
- arm_smmu_cmdq_batch_add_cmd_p(inv->smmu, cmds, &nsize_cmd);
+ if (single_range_inv && size > granule) {
+ if (!arm_smmu_cmdq_batch_add_range_inv(inv->smmu, cmds, cmd,
+ iova, size, inv->pgsize))
+ goto full_inv;
return;
}
- arm_smmu_cmdq_batch_add_range(inv->smmu, cmds, cmd, leaf,
- iova, size, granule, inv->pgsize);
+ arm_smmu_cmdq_batch_add_range(inv->smmu, cmds, cmd, leaf, iova, size,
+ granule, inv->pgsize);
+ return;
+
+full_inv:
+ nsize_cmd = *cmd;
+ u64p_replace_bits(&nsize_cmd.data[0], inv->nsize_opcode, CMDQ_0_OP);
+ arm_smmu_cmdq_batch_add_cmd_p(inv->smmu, cmds, &nsize_cmd);
}
static inline bool arm_smmu_invs_end_batch(struct arm_smmu_inv *cur,
@@ -2600,7 +2640,8 @@ static inline bool arm_smmu_invs_end_batch(struct arm_smmu_inv *cur,
static void __arm_smmu_domain_inv_range(struct arm_smmu_invs *invs,
unsigned long iova, size_t size,
- unsigned int granule, bool leaf)
+ unsigned int granule,
+ bool single_range_inv, bool leaf)
{
struct arm_smmu_cmdq_batch cmds = {};
struct arm_smmu_inv *cur;
@@ -2630,14 +2671,16 @@ static void __arm_smmu_domain_inv_range(struct arm_smmu_invs *invs,
case INV_TYPE_S1_ASID:
cmd = arm_smmu_make_cmd_tlbi(cur->size_opcode,
cur->id, 0);
- arm_smmu_inv_to_cmdq_batch(cur, &cmds, &cmd, leaf,
- iova, size, granule);
+ arm_smmu_inv_to_cmdq_batch(cur, &cmds, &cmd,
+ single_range_inv, leaf, iova,
+ size, granule);
break;
case INV_TYPE_S2_VMID:
cmd = arm_smmu_make_cmd_tlbi(cur->size_opcode,
0, cur->id);
- arm_smmu_inv_to_cmdq_batch(cur, &cmds, &cmd, leaf,
- iova, size, granule);
+ arm_smmu_inv_to_cmdq_batch(cur, &cmds, &cmd,
+ single_range_inv, leaf, iova,
+ size, granule);
break;
case INV_TYPE_S2_VMID_S1_CLEAR:
/* CMDQ_OP_TLBI_S12_VMALL already flushed S1 entries */
@@ -2684,6 +2727,9 @@ void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain,
unsigned int granule, bool leaf)
{
struct arm_smmu_invs *invs;
+ bool single_range_inv =
+ smmu_domain->stage == ARM_SMMU_DOMAIN_SVA &&
+ (smmu_domain->smmu->options & ARM_SMMU_OPT_FULL_CONT_RANGE_INV);
/*
* An invalidation request must follow some IOPTE change and then load
@@ -2723,10 +2769,12 @@ void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain,
unsigned long flags;
read_lock_irqsave(&invs->rwlock, flags);
- __arm_smmu_domain_inv_range(invs, iova, size, granule, leaf);
+ __arm_smmu_domain_inv_range(invs, iova, size, granule,
+ single_range_inv, leaf);
read_unlock_irqrestore(&invs->rwlock, flags);
} else {
- __arm_smmu_domain_inv_range(invs, iova, size, granule, leaf);
+ __arm_smmu_domain_inv_range(invs, iova, size, granule,
+ single_range_inv, leaf);
}
rcu_read_unlock();
@@ -5009,11 +5057,20 @@ static void arm_smmu_device_iidr_probe(struct arm_smmu_device *smmu)
/* Arm errata 2268618, 2812531 */
smmu->features &= ~ARM_SMMU_FEAT_NESTING;
}
+ /* Arm errata 3777127 */
+ smmu->options |= ARM_SMMU_OPT_FULL_CONT_RANGE_INV;
break;
case IIDR_PRODUCTID_ARM_MMU_L1:
+ /* Arm errata 3878312 */
+ smmu->features &= ~ARM_SMMU_FEAT_BTM;
+ break;
case IIDR_PRODUCTID_ARM_MMU_S3:
- /* Arm errata 3878312/3995052 */
+ /* Arm errata 3995052 */
smmu->features &= ~ARM_SMMU_FEAT_BTM;
+ /* Arm errata 3673557 */
+ if (variant < 1 || (variant == 1 && revision < 1))
+ smmu->options |=
+ ARM_SMMU_OPT_FULL_CONT_RANGE_INV;
break;
}
break;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index dd2fee2f560e..094ce28c862f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -934,6 +934,11 @@ struct arm_smmu_device {
#define ARM_SMMU_OPT_MSIPOLL (1 << 2)
#define ARM_SMMU_OPT_CMDQ_FORCE_SYNC (1 << 3)
#define ARM_SMMU_OPT_TEGRA241_CMDQV (1 << 4)
+/*
+ * Range invalidation is mandatory and one range invalidation must fully span an
+ * invalidated CONT
+ */
+#define ARM_SMMU_OPT_FULL_CONT_RANGE_INV (1 << 6)
u32 options;
struct arm_smmu_cmdq cmdq;
diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c
index 283ab372e8da..5ac736cf5bb5 100644
--- a/drivers/iommu/iommufd/io_pagetable.c
+++ b/drivers/iommu/iommufd/io_pagetable.c
@@ -589,7 +589,7 @@ iommu_read_and_clear_dirty(struct iommu_domain *domain,
arg.iopt = iopt;
arg.domain = domain;
arg.dirty = &dirty;
- iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty);
+ ret = iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty);
if (!(flags & IOMMU_DIRTY_NO_CLEAR))
iommu_iotlb_sync(domain, &gather);
diff --git a/drivers/iommu/iommufd/iova_bitmap.c b/drivers/iommu/iommufd/iova_bitmap.c
index dac3e657d498..908c70f5362e 100644
--- a/drivers/iommu/iommufd/iova_bitmap.c
+++ b/drivers/iommu/iommufd/iova_bitmap.c
@@ -116,6 +116,9 @@ struct iova_bitmap {
/* length of the IOVA range for the whole bitmap */
size_t length;
+
+ /* first error from recording an IOVA range */
+ int err;
};
/*
@@ -384,7 +387,9 @@ static int iova_bitmap_advance_to(struct iova_bitmap *bitmap,
int iova_bitmap_for_each(struct iova_bitmap *bitmap, void *opaque,
iova_bitmap_fn_t fn)
{
- return fn(bitmap, bitmap->iova, bitmap->length, opaque);
+ int ret = fn(bitmap, bitmap->iova, bitmap->length, opaque);
+
+ return ret ? ret : bitmap->err;
}
EXPORT_SYMBOL_NS_GPL(iova_bitmap_for_each, "IOMMUFD");
@@ -404,6 +409,18 @@ void iova_bitmap_set(struct iova_bitmap *bitmap,
struct iova_bitmap_map *mapped = &bitmap->mapped;
unsigned long cur_bit, last_bit, last_page_idx;
+ /*
+ * If the range cannot be recorded then the system call returns failure.
+ * This is to prevent loosing dirties if userspace is expecting to step
+ * the bitmap through the IOVA space.
+ */
+ if (!length || length > bitmap->length || iova < bitmap->iova ||
+ iova - bitmap->iova > bitmap->length - length) {
+ if (!bitmap->err)
+ bitmap->err = -EPROTO;
+ return;
+ }
+
update_indexes:
if (unlikely(!iova_bitmap_mapped_range(mapped, iova, length))) {
/*
diff --git a/drivers/iommu/virtio-iommu.c b/drivers/iommu/virtio-iommu.c
index 587fc13197f1..fa72ae23b8af 100644
--- a/drivers/iommu/virtio-iommu.c
+++ b/drivers/iommu/virtio-iommu.c
@@ -1227,12 +1227,12 @@ static int viommu_probe(struct virtio_device *vdev)
/* Populate the event queue with buffers */
ret = viommu_fill_evtq(viommu);
if (ret)
- goto err_free_vqs;
+ goto err_reset_vdev;
ret = iommu_device_sysfs_add(&viommu->iommu, dev, NULL, "%s",
virtio_bus_name(vdev));
if (ret)
- goto err_free_vqs;
+ goto err_reset_vdev;
vdev->priv = viommu;
@@ -1244,6 +1244,8 @@ static int viommu_probe(struct virtio_device *vdev)
return 0;
+err_reset_vdev:
+ virtio_reset_device(vdev);
err_free_vqs:
vdev->config->del_vqs(vdev);
diff --git a/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c b/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c
index 62ed92ff1d30..9f5bd9c4ddf9 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c
@@ -32,7 +32,7 @@ static int handle_proxy_response(struct ipu6_isys *isys, unsigned int req_id)
struct ipu6_fw_isys_proxy_resp_info_abi *resp;
int ret;
- resp = ipu6_recv_get_token(isys->fwcom, IPU6_BASE_PROXY_RECV_QUEUES);
+ resp = ipu6_recv_get_token(isys->fwctx, IPU6_BASE_PROXY_RECV_QUEUES);
if (!resp)
return 1;
@@ -42,7 +42,7 @@ static int handle_proxy_response(struct ipu6_isys *isys, unsigned int req_id)
ret = req_id == resp->request_id ? 0 : -EIO;
- ipu6_recv_put_token(isys->fwcom, IPU6_BASE_PROXY_RECV_QUEUES);
+ ipu6_recv_put_token(isys->fwctx, IPU6_BASE_PROXY_RECV_QUEUES);
return ret;
}
@@ -52,7 +52,7 @@ int ipu6_fw_isys_send_proxy_token(struct ipu6_isys *isys,
unsigned int index,
unsigned int offset, u32 value)
{
- struct ipu6_fw_com_context *ctx = isys->fwcom;
+ struct ipu6_fw_com_context *ctx = isys->fwctx;
struct device *dev = &isys->adev->auxdev.dev;
struct ipu6_fw_proxy_send_queue_token *token;
unsigned int timeout = 1000;
@@ -96,7 +96,7 @@ int ipu6_fw_isys_complex_cmd(struct ipu6_isys *isys,
dma_addr_t dma_mapped_buf,
size_t size, u16 send_type)
{
- struct ipu6_fw_com_context *ctx = isys->fwcom;
+ struct ipu6_fw_com_context *ctx = isys->fwctx;
struct device *dev = &isys->adev->auxdev.dev;
struct ipu6_fw_send_queue_token *token;
@@ -138,7 +138,7 @@ int ipu6_fw_isys_close(struct ipu6_isys *isys)
struct device *dev = &isys->adev->auxdev.dev;
int retry = IPU6_ISYS_CLOSE_RETRY;
unsigned long flags;
- void *fwcom;
+ void *fwctx;
int ret;
/*
@@ -148,9 +148,9 @@ int ipu6_fw_isys_close(struct ipu6_isys *isys)
* spinlock to wait the interrupt handler to be finished
*/
spin_lock_irqsave(&isys->power_lock, flags);
- ret = ipu6_fw_com_close(isys->fwcom);
- fwcom = isys->fwcom;
- isys->fwcom = NULL;
+ ret = ipu6_fw_com_close(isys->fwctx);
+ fwctx = isys->fwctx;
+ isys->fwctx = NULL;
spin_unlock_irqrestore(&isys->power_lock, flags);
if (ret)
dev_err(dev, "Device close failure: %d\n", ret);
@@ -158,14 +158,14 @@ int ipu6_fw_isys_close(struct ipu6_isys *isys)
/* release probably fails if the close failed. Let's try still */
do {
usleep_range(400, 500);
- ret = ipu6_fw_com_release(fwcom, 0);
+ ret = ipu6_fw_com_release(fwctx, 0);
retry--;
} while (ret && retry);
if (ret) {
dev_err(dev, "Device release time out %d\n", ret);
spin_lock_irqsave(&isys->power_lock, flags);
- isys->fwcom = fwcom;
+ isys->fwctx = fwctx;
spin_unlock_irqrestore(&isys->power_lock, flags);
}
@@ -176,11 +176,11 @@ void ipu6_fw_isys_cleanup(struct ipu6_isys *isys)
{
int ret;
- ret = ipu6_fw_com_release(isys->fwcom, 1);
+ ret = ipu6_fw_com_release(isys->fwctx, 1);
if (ret < 0)
dev_warn(&isys->adev->auxdev.dev,
"Device busy, fw_com release failed.");
- isys->fwcom = NULL;
+ isys->fwctx = NULL;
}
static void start_sp(struct ipu6_bus_device *adev)
@@ -212,13 +212,12 @@ static int query_sp(struct ipu6_bus_device *adev)
}
static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
- struct ipu6_fw_com_cfg *fwcom,
+ struct ipu6_fw_com_cfg *fwcom_cfg,
unsigned int num_streams)
{
unsigned int max_send_queues, max_sram_blocks, max_devq_size;
struct ipu6_fw_syscom_queue_config *input_queue_cfg;
struct ipu6_fw_syscom_queue_config *output_queue_cfg;
- struct device *dev = &isys->adev->auxdev.dev;
int type_proxy = IPU6_FW_ISYS_QUEUE_TYPE_PROXY;
int type_dev = IPU6_FW_ISYS_QUEUE_TYPE_DEV;
int type_msg = IPU6_FW_ISYS_QUEUE_TYPE_MSG;
@@ -228,7 +227,6 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
struct ipu6_fw_isys_fw_config *isys_fw_cfg;
u32 num_in_message_queues;
unsigned int max_streams;
- unsigned int size;
unsigned int i;
max_streams = isys->pdata->ipdata->max_streams;
@@ -236,7 +234,7 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
max_sram_blocks = isys->pdata->ipdata->max_sram_blocks;
max_devq_size = isys->pdata->ipdata->max_devq_size;
num_in_message_queues = clamp(num_streams, 1U, max_streams);
- isys_fw_cfg = devm_kzalloc(dev, sizeof(*isys_fw_cfg), GFP_KERNEL);
+ isys_fw_cfg = kzalloc_obj(*isys_fw_cfg);
if (!isys_fw_cfg)
return -ENOMEM;
@@ -248,24 +246,25 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
isys_fw_cfg->num_recv_queues[type_dev] = 0;
isys_fw_cfg->num_recv_queues[type_msg] = 1;
- size = sizeof(*input_queue_cfg) * max_send_queues;
- input_queue_cfg = devm_kzalloc(dev, size, GFP_KERNEL);
+ input_queue_cfg = kzalloc_objs(*input_queue_cfg, max_send_queues);
if (!input_queue_cfg)
- return -ENOMEM;
+ goto err_free_fw_cfg;
- size = sizeof(*output_queue_cfg) * IPU6_N_MAX_RECV_QUEUES;
- output_queue_cfg = devm_kzalloc(dev, size, GFP_KERNEL);
+ output_queue_cfg = kzalloc_objs(*output_queue_cfg,
+ IPU6_N_MAX_RECV_QUEUES);
if (!output_queue_cfg)
- return -ENOMEM;
+ goto err_free_input_queue_cfg;
- fwcom->input = input_queue_cfg;
- fwcom->output = output_queue_cfg;
+ fwcom_cfg->input = input_queue_cfg;
+ fwcom_cfg->output = output_queue_cfg;
- fwcom->num_input_queues = isys_fw_cfg->num_send_queues[type_proxy] +
+ fwcom_cfg->num_input_queues =
+ isys_fw_cfg->num_send_queues[type_proxy] +
isys_fw_cfg->num_send_queues[type_dev] +
isys_fw_cfg->num_send_queues[type_msg];
- fwcom->num_output_queues = isys_fw_cfg->num_recv_queues[type_proxy] +
+ fwcom_cfg->num_output_queues =
+ isys_fw_cfg->num_recv_queues[type_proxy] +
isys_fw_cfg->num_recv_queues[type_dev] +
isys_fw_cfg->num_recv_queues[type_msg];
@@ -280,7 +279,7 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
isys_fw_cfg->buffer_partition.num_gda_pages[i] = 0;
}
- /* FW assumes proxy interface at fwcom queue 0 */
+ /* FW assumes proxy interface at fwcom_cfg queue 0 */
for (i = 0; i < isys_fw_cfg->num_send_queues[type_proxy]; i++) {
input_queue_cfg[i].token_size =
sizeof(struct ipu6_fw_proxy_send_queue_token);
@@ -314,34 +313,51 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
IPU6_ISYS_SIZE_RECV_QUEUE;
}
- fwcom->dmem_addr = isys->pdata->ipdata->hw_variant.dmem_offset;
- fwcom->specific_addr = isys_fw_cfg;
- fwcom->specific_size = sizeof(*isys_fw_cfg);
+ fwcom_cfg->dmem_addr = isys->pdata->ipdata->hw_variant.dmem_offset;
+ fwcom_cfg->specific_addr = isys_fw_cfg;
+ fwcom_cfg->specific_size = sizeof(*isys_fw_cfg);
return 0;
+
+err_free_input_queue_cfg:
+ kfree(input_queue_cfg);
+err_free_fw_cfg:
+ kfree(isys_fw_cfg);
+
+ return -ENOMEM;
+}
+
+static void ipu6_isys_fwcom_cfg_free(struct ipu6_fw_com_cfg *fwcom_cfg)
+{
+ kfree(fwcom_cfg->specific_addr);
+ kfree(fwcom_cfg->output);
+ kfree(fwcom_cfg->input);
}
int ipu6_fw_isys_init(struct ipu6_isys *isys, unsigned int num_streams)
{
struct device *dev = &isys->adev->auxdev.dev;
int retry = IPU6_ISYS_OPEN_RETRY;
- struct ipu6_fw_com_cfg fwcom = {
+ struct ipu6_fw_com_cfg fwcom_cfg = {
.cell_start = start_sp,
.cell_ready = query_sp,
.buttress_boot_offset = SYSCOM_BUTTRESS_FW_PARAMS_ISYS_OFFSET,
};
int ret;
- ipu6_isys_fwcom_cfg_init(isys, &fwcom, num_streams);
+ ret = ipu6_isys_fwcom_cfg_init(isys, &fwcom_cfg, num_streams);
+ if (ret)
+ return ret;
- isys->fwcom = ipu6_fw_com_prepare(&fwcom, isys->adev,
+ isys->fwctx = ipu6_fw_com_prepare(&fwcom_cfg, isys->adev,
isys->pdata->base);
- if (!isys->fwcom) {
+ ipu6_isys_fwcom_cfg_free(&fwcom_cfg);
+ if (!isys->fwctx) {
dev_err(dev, "isys fw com prepare failed\n");
return -EIO;
}
- ret = ipu6_fw_com_open(isys->fwcom);
+ ret = ipu6_fw_com_open(isys->fwctx);
if (ret) {
dev_err(dev, "isys fw com open failed %d\n", ret);
return ret;
@@ -349,7 +365,7 @@ int ipu6_fw_isys_init(struct ipu6_isys *isys, unsigned int num_streams)
do {
usleep_range(400, 500);
- if (ipu6_fw_com_ready(isys->fwcom))
+ if (ipu6_fw_com_ready(isys->fwctx))
break;
retry--;
} while (retry > 0);
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
index 3ac48d2076da..89eb265737d7 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
@@ -1114,7 +1114,7 @@ int ipu6_isys_fw_open(struct ipu6_isys *isys)
*/
ipu6_cleanup_fw_msg_bufs(isys);
- if (isys->fwcom) {
+ if (isys->fwctx) {
/*
* Something went wrong in previous shutdown. As we are now
* restarting isys we can safely delete old context.
@@ -1147,7 +1147,7 @@ void ipu6_isys_fw_close(struct ipu6_isys *isys)
isys->ref_count--;
if (!isys->ref_count) {
ipu6_fw_isys_close(isys);
- if (isys->fwcom) {
+ if (isys->fwctx) {
isys->need_reset = true;
dev_warn(&isys->adev->auxdev.dev,
"failed to close fw isys\n");
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys.c b/drivers/media/pci/intel/ipu6/ipu6-isys.c
index 24db2763de54..e13fbbae8c98 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys.c
@@ -1212,10 +1212,10 @@ static int isys_isr_one(struct ipu6_bus_device *adev)
u32 index;
u64 ts;
- if (!isys->fwcom)
+ if (!isys->fwctx)
return 1;
- resp = ipu6_fw_isys_get_resp(isys->fwcom, IPU6_BASE_MSG_RECV_QUEUES);
+ resp = ipu6_fw_isys_get_resp(isys->fwctx, IPU6_BASE_MSG_RECV_QUEUES);
if (!resp)
return 1;
@@ -1324,7 +1324,7 @@ static int isys_isr_one(struct ipu6_bus_device *adev)
ipu6_isys_put_stream(stream);
leave:
- ipu6_fw_isys_put_resp(isys->fwcom, IPU6_BASE_MSG_RECV_QUEUES);
+ ipu6_fw_isys_put_resp(isys->fwctx, IPU6_BASE_MSG_RECV_QUEUES);
return 0;
}
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys.h b/drivers/media/pci/intel/ipu6/ipu6-isys.h
index 7fb8cb820912..0b139ab24431 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys.h
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys.h
@@ -114,8 +114,7 @@ struct sensor_async_sd {
* @csi2_rx_ctrl_cached: cached shared value between all CSI2 receivers
* @streams_lock: serialise access to streams
* @streams: streams per firmware stream ID
- * @fwcom: fw communication layer private pointer
- * or optional external library private pointer
+ * @fwctx: fw communication layer context pointer
* @phy_termcal_val: the termination calibration value, only used for DWC PHY
* @need_reset: Isys requires d0i0->i3 transition
* @ref_count: total number of callers fw open
@@ -136,7 +135,7 @@ struct ipu6_isys {
spinlock_t streams_lock;
struct ipu6_isys_stream streams[IPU6_ISYS_MAX_STREAMS];
int streams_ref_count[IPU6_ISYS_MAX_STREAMS];
- void *fwcom;
+ void *fwctx;
u32 phy_termcal_val;
bool need_reset;
bool icache_prefetch;
diff --git a/drivers/media/platform/microchip/microchip-isc-base.c b/drivers/media/platform/microchip/microchip-isc-base.c
index 21d108eb93c7..963d5fd53b58 100644
--- a/drivers/media/platform/microchip/microchip-isc-base.c
+++ b/drivers/media/platform/microchip/microchip-isc-base.c
@@ -8,6 +8,7 @@
* Author: Eugen Hristev <eugen.hristev@microchip.com>
*
*/
+#include <linux/bitfield.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/math64.h>
@@ -62,17 +63,17 @@ static inline void isc_update_awb_ctrls(struct isc_device *isc)
/* In here we set our actual hw pipeline config */
regmap_write(isc->regmap, ISC_WB_O_RGR,
- ((ctrls->offset[ISC_HIS_CFG_MODE_R])) |
- ((ctrls->offset[ISC_HIS_CFG_MODE_GR]) << 16));
+ FIELD_PREP(ISC_WB_O_LO, ctrls->offset[ISC_HIS_CFG_MODE_R]) |
+ FIELD_PREP(ISC_WB_O_HI, ctrls->offset[ISC_HIS_CFG_MODE_GR]));
regmap_write(isc->regmap, ISC_WB_O_BGB,
- ((ctrls->offset[ISC_HIS_CFG_MODE_B])) |
- ((ctrls->offset[ISC_HIS_CFG_MODE_GB]) << 16));
+ FIELD_PREP(ISC_WB_O_LO, ctrls->offset[ISC_HIS_CFG_MODE_B]) |
+ FIELD_PREP(ISC_WB_O_HI, ctrls->offset[ISC_HIS_CFG_MODE_GB]));
regmap_write(isc->regmap, ISC_WB_G_RGR,
- ctrls->gain[ISC_HIS_CFG_MODE_R] |
- (ctrls->gain[ISC_HIS_CFG_MODE_GR] << 16));
+ FIELD_PREP(ISC_WB_G_LO, ctrls->gain[ISC_HIS_CFG_MODE_R]) |
+ FIELD_PREP(ISC_WB_G_HI, ctrls->gain[ISC_HIS_CFG_MODE_GR]));
regmap_write(isc->regmap, ISC_WB_G_BGB,
- ctrls->gain[ISC_HIS_CFG_MODE_B] |
- (ctrls->gain[ISC_HIS_CFG_MODE_GB] << 16));
+ FIELD_PREP(ISC_WB_G_LO, ctrls->gain[ISC_HIS_CFG_MODE_B]) |
+ FIELD_PREP(ISC_WB_G_HI, ctrls->gain[ISC_HIS_CFG_MODE_GB]));
}
static inline void isc_reset_awb_ctrls(struct isc_device *isc)
@@ -291,7 +292,8 @@ static int isc_configure(struct isc_device *isc)
struct isc_subdev_entity *subdev = isc->current_subdev;
int ret;
- pfe_cfg0 = isc->config.sd_format->pfe_cfg0_bps;
+ pfe_cfg0 = FIELD_PREP(ISC_PFE_CFG0_BPS_MASK,
+ isc->config.sd_format->pfe_cfg0_bps);
pipeline = isc->config.bits_pipeline;
dcfg = isc->config.dcfg_imode | isc->dcfg;
diff --git a/drivers/media/platform/microchip/microchip-isc-regs.h b/drivers/media/platform/microchip/microchip-isc-regs.h
index e77e1d9a1db8..fe145b142b82 100644
--- a/drivers/media/platform/microchip/microchip-isc-regs.h
+++ b/drivers/media/platform/microchip/microchip-isc-regs.h
@@ -31,11 +31,11 @@
#define ISC_PFE_CFG0_MODE_PROGRESSIVE (0x0 << 4)
#define ISC_PFE_CFG0_MODE_MASK GENMASK(6, 4)
-#define ISC_PFE_CFG0_BPS_EIGHT (0x4 << 28)
-#define ISC_PFG_CFG0_BPS_NINE (0x3 << 28)
-#define ISC_PFG_CFG0_BPS_TEN (0x2 << 28)
-#define ISC_PFG_CFG0_BPS_ELEVEN (0x1 << 28)
-#define ISC_PFG_CFG0_BPS_TWELVE (0x0 << 28)
+#define ISC_PFE_CFG0_BPS_EIGHT 0x4
+#define ISC_PFE_CFG0_BPS_NINE 0x3
+#define ISC_PFE_CFG0_BPS_TEN 0x2
+#define ISC_PFE_CFG0_BPS_ELEVEN 0x1
+#define ISC_PFE_CFG0_BPS_TWELVE 0x0
#define ISC_PFE_CFG0_BPS_MASK GENMASK(30, 28)
#define ISC_PFE_CFG0_COLEN BIT(12)
@@ -149,6 +149,12 @@
/* ISC White Balance Gain for B, GB Register */
#define ISC_WB_G_BGB 0x0000006c
+/* Each WB offset/gain register packs two 13-bit fields, low and high */
+#define ISC_WB_O_LO GENMASK(12, 0) /* R or B offset [12:0] */
+#define ISC_WB_O_HI GENMASK(28, 16) /* GR or GB offset [28:16] */
+#define ISC_WB_G_LO GENMASK(12, 0) /* R or B gain [12:0] */
+#define ISC_WB_G_HI GENMASK(28, 16) /* GR or GB gain [28:16] */
+
/* ISC Color Filter Array Control Register */
#define ISC_CFA_CTRL 0x00000070
diff --git a/drivers/media/platform/microchip/microchip-isc.h b/drivers/media/platform/microchip/microchip-isc.h
index f5e322c2e36b..d7bcd74efff9 100644
--- a/drivers/media/platform/microchip/microchip-isc.h
+++ b/drivers/media/platform/microchip/microchip-isc.h
@@ -62,7 +62,9 @@ struct isc_subdev_entity {
* @mbus_code: V4L2 media bus format code.
* @cfa_baycfg: If this format is RAW BAYER, indicate the type of bayer.
this is either BGBG, RGRG, etc.
- * @pfe_cfg0_bps: Number of hardware data lines connected to the ISC
+ * @pfe_cfg0_bps: ISC_PFE_CFG0 BPS field value (e.g. ISC_PFE_CFG0_BPS_EIGHT),
+ written into PFE_CFG0 with FIELD_PREP(ISC_PFE_CFG0_BPS_MASK)
+ at configure time.
* @raw: If the format is raw bayer.
*/
diff --git a/drivers/media/platform/microchip/microchip-sama5d2-isc.c b/drivers/media/platform/microchip/microchip-sama5d2-isc.c
index 5bd109e93a8f..5e41eee45dbd 100644
--- a/drivers/media/platform/microchip/microchip-sama5d2-isc.c
+++ b/drivers/media/platform/microchip/microchip-sama5d2-isc.c
@@ -147,49 +147,49 @@ static struct isc_format sama5d2_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_SBGGR10,
.mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
- .cfa_baycfg = ISC_BAY_CFG_RGRG,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
+ .cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG10,
.mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG10,
.mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB10,
.mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
.fourcc = V4L2_PIX_FMT_SBGGR12,
.mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG12,
.mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG12,
.mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB12,
.mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
@@ -210,7 +210,7 @@ static struct isc_format sama5d2_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_Y10,
.mbus_code = MEDIA_BUS_FMT_Y10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
},
};
diff --git a/drivers/media/platform/microchip/microchip-sama7g5-isc.c b/drivers/media/platform/microchip/microchip-sama7g5-isc.c
index d978ce448a4d..9e78ea07b013 100644
--- a/drivers/media/platform/microchip/microchip-sama7g5-isc.c
+++ b/drivers/media/platform/microchip/microchip-sama7g5-isc.c
@@ -156,49 +156,49 @@ static struct isc_format sama7g5_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_SBGGR10,
.mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
- .cfa_baycfg = ISC_BAY_CFG_RGRG,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
+ .cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG10,
.mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG10,
.mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB10,
.mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
.fourcc = V4L2_PIX_FMT_SBGGR12,
.mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG12,
.mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG12,
.mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB12,
.mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
@@ -224,7 +224,7 @@ static struct isc_format sama7g5_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_Y10,
.mbus_code = MEDIA_BUS_FMT_Y10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
},
};
diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index e289cc71ba98..73b97ec578ec 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -2851,6 +2851,26 @@ static const struct usb_device_id uvc_ids[] = {
.bInterfaceSubClass = 1,
.bInterfaceProtocol = 0,
.driver_info = (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
+ /* Avermedia GC515 */
+ { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
+ | USB_DEVICE_ID_MATCH_INT_INFO,
+ .idVendor = 0x07ca,
+ .idProduct = 0x1515,
+ .bInterfaceClass = USB_CLASS_VIDEO,
+ .bInterfaceSubClass = 1,
+ .bInterfaceProtocol = UVC_PC_PROTOCOL_15,
+ .driver_info = (kernel_ulong_t)&(const struct uvc_device_info){
+ .uvc_version = 0x0110,
+ } },
+ /* AVerMedia Live Gamer HD 2 capture card */
+ { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
+ | USB_DEVICE_ID_MATCH_INT_INFO,
+ .idVendor = 0x07ca,
+ .idProduct = 0x1570,
+ .bInterfaceClass = USB_CLASS_VIDEO,
+ .bInterfaceSubClass = 1,
+ .bInterfaceProtocol = 0,
+ .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_DISABLE_AUTOSUSPEND) },
/* ViMicro Vega */
{ .match_flags = USB_DEVICE_ID_MATCH_DEVICE
| USB_DEVICE_ID_MATCH_INT_INFO,
diff --git a/drivers/mtd/spi-nor/macronix.c b/drivers/mtd/spi-nor/macronix.c
index e97f5cbd9aad..c854a1163a47 100644
--- a/drivers/mtd/spi-nor/macronix.c
+++ b/drivers/mtd/spi-nor/macronix.c
@@ -132,7 +132,9 @@ static const struct flash_info macronix_nor_parts[] = {
}, {
/* MX25L12805D */
.id = SNOR_ID(0xc2, 0x20, 0x18),
+ .size = SZ_16M,
.flags = SPI_NOR_HAS_LOCK | SPI_NOR_4BIT_BP,
+ .no_sfdp_flags = SECT_4K,
}, {
/* MX25L25635E, MX25L25645G */
.id = SNOR_ID(0xc2, 0x20, 0x19),
diff --git a/drivers/net/can/dev/netlink.c b/drivers/net/can/dev/netlink.c
index 766d455950f5..b958c04d53db 100644
--- a/drivers/net/can/dev/netlink.c
+++ b/drivers/net/can/dev/netlink.c
@@ -812,7 +812,7 @@ static size_t can_get_size(const struct net_device *dev)
size += nla_total_size(sizeof(u32)); /* IFLA_CAN_STATE */
size += nla_total_size(sizeof(struct can_ctrlmode)); /* IFLA_CAN_CTRLMODE */
size += nla_total_size(sizeof(u32)); /* IFLA_CAN_RESTART_MS */
- if (priv->do_get_berr_counter) /* IFLA_CAN_BERR_COUNTER */
+ if (priv->do_get_berr_counter && netif_running(dev)) /* IFLA_CAN_BERR_COUNTER */
size += nla_total_size(sizeof(struct can_berr_counter));
if (priv->termination_const) {
size += nla_total_size(sizeof(priv->termination)); /* IFLA_CAN_TERMINATION */
@@ -1006,7 +1006,7 @@ static int can_fill_info(struct sk_buff *skb, const struct net_device *dev)
nla_put(skb, IFLA_CAN_CTRLMODE, sizeof(cm), &cm) ||
nla_put_u32(skb, IFLA_CAN_RESTART_MS, priv->restart_ms) ||
- (priv->do_get_berr_counter &&
+ (priv->do_get_berr_counter && netif_running(dev) &&
!priv->do_get_berr_counter(dev, &bec) &&
nla_put(skb, IFLA_CAN_BERR_COUNTER, sizeof(bec), &bec)) ||
diff --git a/drivers/net/can/dev/skb.c b/drivers/net/can/dev/skb.c
index 14edec5afb57..9e274147ffe5 100644
--- a/drivers/net/can/dev/skb.c
+++ b/drivers/net/can/dev/skb.c
@@ -4,6 +4,7 @@
* Copyright (C) 2008-2009 Wolfgang Grandegger <wg@grandegger.com>
*/
+#include <linux/can/can-ml.h>
#include <linux/can/dev.h>
#include <linux/module.h>
#include <net/can.h>
@@ -54,6 +55,7 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev,
if (idx >= priv->echo_skb_max) {
netdev_err(dev, "%s: BUG! Trying to access can_priv::echo_skb out of bounds (%u/max %u)\n",
__func__, idx, priv->echo_skb_max);
+ dev_kfree_skb_any(skb);
return -EINVAL;
}
@@ -62,7 +64,7 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev,
(skb->protocol != htons(ETH_P_CAN) &&
skb->protocol != htons(ETH_P_CANFD) &&
skb->protocol != htons(ETH_P_CANXL))) {
- kfree_skb(skb);
+ dev_kfree_skb_any(skb);
return 0;
}
@@ -90,7 +92,7 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev,
} else {
/* locking problem with netif_stop_queue() ?? */
netdev_err(dev, "%s: BUG! echo_skb %d is occupied!\n", __func__, idx);
- kfree_skb(skb);
+ dev_kfree_skb_any(skb);
return -EBUSY;
}
@@ -388,7 +390,7 @@ bool can_dropped_invalid_skb(struct net_device *dev, struct sk_buff *skb)
break;
case ETH_P_CANXL:
- if (!can_is_canxl_skb(skb))
+ if (!can_cap_enabled(dev, CAN_CAP_XL) || !can_is_canxl_skb(skb))
goto inval_skb;
break;
@@ -402,7 +404,7 @@ bool can_dropped_invalid_skb(struct net_device *dev, struct sk_buff *skb)
return false;
inval_skb:
- kfree_skb(skb);
+ dev_kfree_skb_any(skb);
dev->stats.tx_dropped++;
return true;
}
diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
index 0d46f4ce5dca..39f0740421c3 100644
--- a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
+++ b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
@@ -755,7 +755,7 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb)
u16 rx_msg_type, rx_msg_size;
size_t rx_msg_min_size;
- if (msg_end - msg_ptr < sizeof(*rx_msg)) {
+ if (msg_end - msg_ptr < sizeof(rx_msg->size)) {
err = -EBADMSG;
break;
}
@@ -766,6 +766,11 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb)
break;
}
+ if (msg_end - msg_ptr < sizeof(*rx_msg)) {
+ err = -EBADMSG;
+ break;
+ }
+
rx_msg_size = le16_to_cpu(rx_msg->size);
rx_msg_type = le16_to_cpu(rx_msg->type);
diff --git a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
index e58f1d2c26bd..eddd8406867f 100644
--- a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
+++ b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
@@ -1934,6 +1934,39 @@ static int atl1c_clean_rx(struct napi_struct *napi, int budget)
break;
}
atl1c_clean_rfd(rfd_ring, rrs, rfd_num);
+
+ /* A link partner whose l2mtu is larger than ours can hand us a
+ * frame that does not fit the buffer the hardware was given:
+ * the descriptor reports the untruncated length while at most
+ * buffer_info->length bytes were written. skb_put() of such a
+ * length would pull in whatever the tail of the buffer holds
+ * and, past the end of the skb, trips skb_over_panic and takes
+ * the host down. Bound it by what the device actually wrote
+ * rather than by skb_tailroom(): the skb carries 64 bytes of
+ * slack for the DMA erratum, so the two are not the same.
+ *
+ * Frames shorter than an Ethernet header plus its FCS are
+ * rejected by the same test; before this, a length below
+ * ETH_FCS_LEN underflowed in skb_put().
+ */
+ if (unlikely(length < ETH_HLEN + ETH_FCS_LEN ||
+ length > buffer_info->length)) {
+ dev_core_stats_rx_dropped_inc(netdev);
+ if (netif_msg_rx_err(adapter))
+ dev_warn_ratelimited(&pdev->dev,
+ "dropped rx frame: %u bytes, buffer %u\n",
+ length, buffer_info->length);
+ kfree_skb_reason(skb, SKB_DROP_REASON_PKT_TOO_BIG);
+ /* atl1c_clean_rfd() has already recycled the RFD, so
+ * count the drop as work: the refill at the end of the
+ * poll only runs when work_done is non-zero, and
+ * sustained oversized traffic would otherwise drain the
+ * ring and stop RX for good.
+ */
+ work_done++;
+ continue;
+ }
+
skb_put(skb, length - ETH_FCS_LEN);
skb->protocol = eth_type_trans(skb, netdev);
atl1c_rx_checksum(adapter, skb, rrs);
diff --git a/drivers/net/ethernet/jme.c b/drivers/net/ethernet/jme.c
index 25ec5f757c68..acb4d41f56e1 100644
--- a/drivers/net/ethernet/jme.c
+++ b/drivers/net/ethernet/jme.c
@@ -1502,7 +1502,8 @@ jme_intr_msi(struct jme_adapter *jme, u32 intrstat)
* all other events are ignored
*/
jwrite32(jme, JME_IEVE, intrstat);
- schedule_work(&jme->linkch_task);
+ if (!test_bit(JME_FLAG_WOL_QUIESCE, &jme->flags))
+ schedule_work(&jme->linkch_task);
goto out_reenable;
}
@@ -1546,7 +1547,8 @@ jme_intr_msi(struct jme_adapter *jme, u32 intrstat)
/*
* Re-enable interrupt
*/
- jwrite32f(jme, JME_IENS, INTR_ENABLE);
+ if (!test_bit(JME_FLAG_WOL_QUIESCE, &jme->flags))
+ jwrite32f(jme, JME_IENS, INTR_ENABLE);
}
static irqreturn_t
@@ -1892,6 +1894,7 @@ jme_powersave_phy(struct jme_adapter *jme)
jme_set_100m_half(jme);
if (jme->reg_pmcs & (PMCS_LFEN | PMCS_LREN))
jme_wait_link(jme);
+ jme_mac_rxclk_on(jme);
jme_clear_pm_enable_wol(jme);
} else {
jme_phy_off(jme);
@@ -3158,6 +3161,21 @@ jme_shutdown(struct pci_dev *pdev)
struct net_device *netdev = pci_get_drvdata(pdev);
struct jme_adapter *jme = netdev_priv(netdev);
+ if (netif_running(netdev)) {
+ set_bit(JME_FLAG_WOL_QUIESCE, &jme->flags);
+ netif_device_detach(netdev);
+ netif_stop_queue(netdev);
+ jme_stop_irq(jme);
+ synchronize_irq(pdev->irq);
+ jme_stop_irq(jme);
+ cancel_work_sync(&jme->linkch_task);
+ tasklet_disable(&jme->pcc_task);
+ napi_disable(&jme->napi);
+ tasklet_disable(&jme->txclean_task);
+ tasklet_disable(&jme->rxclean_task);
+ tasklet_disable(&jme->rxempty_task);
+ }
+
jme_powersave_phy(jme);
pci_pme_active(pdev, true);
}
@@ -3172,11 +3190,16 @@ jme_suspend(struct device *dev)
if (!netif_running(netdev))
return 0;
- atomic_dec(&jme->link_changing);
-
+ set_bit(JME_FLAG_WOL_QUIESCE, &jme->flags);
netif_device_detach(netdev);
netif_stop_queue(netdev);
jme_stop_irq(jme);
+ synchronize_irq(jme->pdev->irq);
+ jme_stop_irq(jme);
+ cancel_work_sync(&jme->linkch_task);
+ tasklet_disable(&jme->pcc_task);
+ napi_disable(&jme->napi);
+ atomic_dec(&jme->link_changing);
tasklet_disable(&jme->txclean_task);
tasklet_disable(&jme->rxclean_task);
@@ -3215,6 +3238,7 @@ jme_resume(struct device *dev)
return 0;
jme_clear_pm_disable_wol(jme);
+ jme_mac_rxclk_off(jme);
jme_phy_on(jme);
if (test_bit(JME_FLAG_SSET, &jme->flags))
jme_set_link_ksettings(netdev, &jme->old_cmd);
@@ -3226,8 +3250,11 @@ jme_resume(struct device *dev)
atomic_inc(&jme->link_changing);
- jme_reset_link(jme);
+ napi_enable(&jme->napi);
+ tasklet_enable(&jme->pcc_task);
+ clear_bit(JME_FLAG_WOL_QUIESCE, &jme->flags);
+ jme_reset_link(jme);
jme_start_irq(jme);
return 0;
diff --git a/drivers/net/ethernet/jme.h b/drivers/net/ethernet/jme.h
index 860494ff3714..ec5dfddf7322 100644
--- a/drivers/net/ethernet/jme.h
+++ b/drivers/net/ethernet/jme.h
@@ -450,6 +450,7 @@ enum jme_flags_bits {
JME_FLAG_SSET = 2,
JME_FLAG_POLL = 5,
JME_FLAG_SHUTDOWN = 6,
+ JME_FLAG_WOL_QUIESCE = 7,
};
#define TX_TIMEOUT (5 * HZ)
diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.c b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
index 6330540979d7..f54cfcb7d153 100644
--- a/drivers/net/ethernet/ti/am65-cpsw-nuss.c
+++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
@@ -128,6 +128,7 @@
#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_319 BIT(21)
#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_320 BIT(22)
#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO BIT(23)
+#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_UNI_EN BIT(24)
/* The PTP event messages - Sync, Delay_Req, Pdelay_Req, and Pdelay_Resp. */
#define AM65_CPSW_TS_EVENT_MSG_TYPE_BITS (BIT(0) | BIT(1) | BIT(2) | BIT(3))
@@ -1857,7 +1858,8 @@ static int am65_cpsw_nuss_hwtstamp_set(struct net_device *ndev,
AM65_CPSW_PN_TS_CTL_LTYPE2_TS_132 |
AM65_CPSW_PN_TS_CTL_LTYPE2_TS_319 |
AM65_CPSW_PN_TS_CTL_LTYPE2_TS_320 |
- AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO;
+ AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO |
+ AM65_CPSW_PN_TS_CTL_LTYPE2_TS_UNI_EN;
ts_ctrl = AM65_CPSW_TS_EVENT_MSG_TYPE_BITS <<
AM65_CPSW_PN_TS_CTL_MSG_TYPE_EN_SHIFT;
diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 7db389ae866d..c1859ffcdaf1 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -1372,9 +1372,10 @@ static int veth_set_channels(struct net_device *dev,
* to identify the range we have to disable
*/
veth_disable_range_safe(dev, new_rx_count, old_rx_count);
- netif_carrier_on(dev);
- if (peer)
+ if (peer && (peer->flags & IFF_UP)) {
+ netif_carrier_on(dev);
netif_carrier_on(peer);
+ }
}
/* update XDP supported features */
diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c
index 3390e341d1e5..7ecd73a5a374 100644
--- a/drivers/net/vxlan/vxlan_core.c
+++ b/drivers/net/vxlan/vxlan_core.c
@@ -996,12 +996,6 @@ static int vxlan_fdb_update_existing(struct vxlan_dev *vxlan,
return -EOPNOTSUPP;
}
- if (rcu_access_pointer(f->nh) &&
- !(state & (NUD_PERMANENT | NUD_NOARP))) {
- NL_SET_ERR_MSG(extack, "Cannot make a nexthop fdb dynamic");
- return -EOPNOTSUPP;
- }
-
/* Do not allow an externally learned entry to take over an entry added
* by the user.
*/
@@ -1263,11 +1257,6 @@ static int vxlan_fdb_add(struct ndmsg *ndm, struct nlattr *tb[],
if (err)
return err;
- if (nhid && !(ndm->ndm_state & (NUD_PERMANENT | NUD_NOARP))) {
- NL_SET_ERR_MSG(extack, "A nexthop fdb cannot be dynamic");
- return -EINVAL;
- }
-
if (vxlan->default_dst.remote_ip.sa.sa_family != ip.sa.sa_family)
return -EAFNOSUPPORT;
@@ -2887,6 +2876,9 @@ static void vxlan_cleanup(struct timer_list *t)
if (f->flags & NTF_EXT_LEARNED)
continue;
+ if (rcu_access_pointer(f->nh))
+ continue;
+
timeout = READ_ONCE(f->updated) + vxlan->cfg.age_interval * HZ;
if (time_before_eq(timeout, jiffies)) {
spin_lock(&vxlan->hash_lock);
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
index 872c48806d09..62d072b6369e 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
@@ -6504,6 +6504,7 @@ brcmf_bss_connect_done(struct brcmf_cfg80211_info *cfg,
struct brcmf_cfg80211_profile *profile = &ifp->vif->profile;
struct brcmf_cfg80211_connect_info *conn_info = cfg_to_conn(cfg);
struct cfg80211_connect_resp_params conn_params;
+ bool authorized;
brcmf_dbg(TRACE, "Enter\n");
@@ -6528,7 +6529,13 @@ brcmf_bss_connect_done(struct brcmf_cfg80211_info *cfg,
conn_params.req_ie_len = conn_info->req_ie_len;
conn_params.resp_ie = conn_info->resp_ie;
conn_params.resp_ie_len = conn_info->resp_ie_len;
+ authorized = completed &&
+ ((profile->use_fwsup == BRCMF_PROFILE_FWSUP_SAE) ||
+ (profile->use_fwsup == BRCMF_PROFILE_FWSUP_PSK));
cfg80211_connect_done(ndev, &conn_params, GFP_KERNEL);
+ if (authorized)
+ cfg80211_port_authorized(ndev, profile->bssid, NULL, 0,
+ GFP_KERNEL);
brcmf_dbg(CONN, "Report connect result - connection %s\n",
completed ? "succeeded" : "failed");
}
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c
index 069ba7016654..ff47d4827722 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c
@@ -1639,7 +1639,7 @@ int brcmf_proto_msgbuf_attach(struct brcmf_pub *drvr)
if (!msgbuf->tx_pktids)
goto fail;
msgbuf->rx_pktids = brcmf_msgbuf_init_pktids(NR_RX_PKTIDS,
- DMA_FROM_DEVICE);
+ DMA_BIDIRECTIONAL);
if (!msgbuf->rx_pktids)
goto fail;
diff --git a/drivers/net/wireless/realtek/rtw88/mac80211.c b/drivers/net/wireless/realtek/rtw88/mac80211.c
index 0b622d7663b6..2a9b09fa76e7 100644
--- a/drivers/net/wireless/realtek/rtw88/mac80211.c
+++ b/drivers/net/wireless/realtek/rtw88/mac80211.c
@@ -474,6 +474,7 @@ static int rtw_ops_start_ap(struct ieee80211_hw *hw,
mutex_lock(&rtwdev->mutex);
rtw_write32_set(rtwdev, REG_TCR, BIT_TCR_UPDATE_HGQMD);
+ rtw_write16(rtwdev, REG_ATIMWND, ATIMWND_AP);
rtwdev->ap_active = true;
rtw_store_op_chan(rtwdev, true);
chip->ops->phy_calibration(rtwdev);
@@ -490,6 +491,7 @@ static void rtw_ops_stop_ap(struct ieee80211_hw *hw,
mutex_lock(&rtwdev->mutex);
rtw_write32_clr(rtwdev, REG_TCR, BIT_TCR_UPDATE_HGQMD);
+ rtw_write16(rtwdev, REG_ATIMWND, ATIMWND_DEFAULT);
rtwdev->ap_active = false;
if (!rtw_core_check_sta_active(rtwdev))
rtw_clear_op_chan(rtwdev);
diff --git a/drivers/net/wireless/realtek/rtw88/reg.h b/drivers/net/wireless/realtek/rtw88/reg.h
index 08e9494977e0..0b9ba6f37680 100644
--- a/drivers/net/wireless/realtek/rtw88/reg.h
+++ b/drivers/net/wireless/realtek/rtw88/reg.h
@@ -483,6 +483,8 @@
#define REG_DRVERLYINT 0x0558
#define REG_BCNDMATIM 0x0559
#define REG_ATIMWND 0x055A
+#define ATIMWND_DEFAULT 0x2
+#define ATIMWND_AP 0xa
#define REG_USTIME_TSF 0x055C
#define REG_BCN_MAX_ERR 0x055D
#define REG_RXTSF_OFFSET_CCK 0x055E
diff --git a/drivers/net/wireless/realtek/rtw88/tx.c b/drivers/net/wireless/realtek/rtw88/tx.c
index 8786bbb421c2..701ed7c4f3f7 100644
--- a/drivers/net/wireless/realtek/rtw88/tx.c
+++ b/drivers/net/wireless/realtek/rtw88/tx.c
@@ -196,7 +196,7 @@ void rtw_tx_report_purge_timer(struct timer_list *t)
void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn)
{
struct rtw_tx_report *tx_report = &rtwdev->tx_report;
- unsigned long timeout = RTW_TX_PROBE_TIMEOUT;
+ unsigned long timeout;
unsigned long flags;
u8 *drv_data;
@@ -208,9 +208,10 @@ void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn)
__skb_queue_tail(&tx_report->queue, skb);
spin_unlock_irqrestore(&tx_report->q_lock, flags);
- if (rtwdev->chip->id == RTW_CHIP_TYPE_8723D &&
- rtwdev->hci.type == RTW_HCI_TYPE_USB)
- timeout = msecs_to_jiffies(2500);
+ if (rtwdev->hci.type == RTW_HCI_TYPE_PCIE)
+ timeout = RTW_TX_PROBE_TIMEOUT_SHORT;
+ else
+ timeout = RTW_TX_PROBE_TIMEOUT_LONG;
mod_timer(&tx_report->purge_timer, jiffies + timeout);
}
diff --git a/drivers/net/wireless/realtek/rtw88/tx.h b/drivers/net/wireless/realtek/rtw88/tx.h
index d34cdeca16f1..168f48e87853 100644
--- a/drivers/net/wireless/realtek/rtw88/tx.h
+++ b/drivers/net/wireless/realtek/rtw88/tx.h
@@ -7,7 +7,8 @@
#define RTK_TX_MAX_AGG_NUM_MASK 0x1f
-#define RTW_TX_PROBE_TIMEOUT msecs_to_jiffies(500)
+#define RTW_TX_PROBE_TIMEOUT_SHORT msecs_to_jiffies(500)
+#define RTW_TX_PROBE_TIMEOUT_LONG msecs_to_jiffies(2500)
struct rtw_tx_desc {
__le32 w0;
diff --git a/drivers/net/wireless/realtek/rtw88/usb.c b/drivers/net/wireless/realtek/rtw88/usb.c
index 64e1c3420e0a..5c766701e632 100644
--- a/drivers/net/wireless/realtek/rtw88/usb.c
+++ b/drivers/net/wireless/realtek/rtw88/usb.c
@@ -2,9 +2,11 @@
/* Copyright(c) 2018-2019 Realtek Corporation
*/
+#include <linux/ip.h>
#include <linux/module.h>
-#include <linux/usb.h>
#include <linux/mutex.h>
+#include <linux/udp.h>
+#include <linux/usb.h>
#include "main.h"
#include "debug.h"
#include "mac.h"
@@ -557,9 +559,61 @@ static int rtw_usb_write_data_h2c(struct rtw_dev *rtwdev, u8 *buf, u32 size)
return rtw_usb_write_data(rtwdev, &pkt_info, buf);
}
-static u8 rtw_usb_tx_queue_mapping_to_qsel(struct sk_buff *skb)
+static bool rtw_usb_bmc_needs_dtim(struct sk_buff *skb)
+{
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
+ struct udphdr *udphdr;
+
+ if (info->control.flags & IEEE80211_TX_CTRL_PORT_CTRL_PROTO)
+ return true;
+
+ if (skb->protocol == htons(ETH_P_ARP))
+ return true;
+
+ if (skb->protocol == htons(ETH_P_IP) &&
+ ip_hdr(skb)->protocol == IPPROTO_UDP) {
+ udphdr = udp_hdr(skb);
+
+ return udphdr->dest == htons(67) || udphdr->dest == htons(68);
+ }
+
+ return false;
+}
+
+#define RTW_USB_HIQ_REFILL_INTERVAL (HZ / 10) /* one unit of budget per 100 ms */
+#define RTW_USB_HIQ_BUDGET_MAX 16
+
+static bool rtw_usb_hiq_take_budget(struct rtw_usb *rtwusb)
+{
+ unsigned long flags, elapsed, add;
+ bool ok;
+
+ spin_lock_irqsave(&rtwusb->hiq_lock, flags);
+
+ elapsed = jiffies - rtwusb->hiq_refill;
+ add = elapsed / RTW_USB_HIQ_REFILL_INTERVAL;
+ if (add) {
+ rtwusb->hiq_budget = min_t(unsigned long, rtwusb->hiq_budget + add,
+ RTW_USB_HIQ_BUDGET_MAX);
+
+ /* keep the unfinished part of the interval for the next unit */
+ rtwusb->hiq_refill = jiffies - elapsed % RTW_USB_HIQ_REFILL_INTERVAL;
+ }
+
+ ok = rtwusb->hiq_budget > 0;
+ if (ok)
+ rtwusb->hiq_budget--;
+
+ spin_unlock_irqrestore(&rtwusb->hiq_lock, flags);
+
+ return ok;
+}
+
+static u8 rtw_usb_tx_queue_mapping_to_qsel(struct rtw_usb *rtwusb,
+ struct sk_buff *skb)
{
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
__le16 fc = hdr->frame_control;
u8 qsel;
@@ -567,7 +621,9 @@ static u8 rtw_usb_tx_queue_mapping_to_qsel(struct sk_buff *skb)
qsel = TX_DESC_QSEL_MGMT;
else if (is_broadcast_ether_addr(hdr->addr1) ||
is_multicast_ether_addr(hdr->addr1))
- qsel = TX_DESC_QSEL_HIGH;
+ qsel = (info->flags & IEEE80211_TX_CTL_SEND_AFTER_DTIM) &&
+ rtw_usb_bmc_needs_dtim(skb) && rtw_usb_hiq_take_budget(rtwusb) ?
+ TX_DESC_QSEL_HIGH : skb->priority;
else if (skb_get_queue_mapping(skb) <= IEEE80211_AC_BK)
qsel = skb->priority;
else
@@ -586,7 +642,7 @@ static int rtw_usb_tx_write(struct rtw_dev *rtwdev,
u8 *pkt_desc;
int ep;
- pkt_info->qsel = rtw_usb_tx_queue_mapping_to_qsel(skb);
+ pkt_info->qsel = rtw_usb_tx_queue_mapping_to_qsel(rtwusb, skb);
pkt_desc = skb_push(skb, chip->tx_pkt_desc_sz);
memset(pkt_desc, 0, chip->tx_pkt_desc_sz);
ep = qsel_to_ep(rtwusb, pkt_info->qsel);
@@ -1027,6 +1083,10 @@ static int rtw_usb_init_tx(struct rtw_dev *rtwdev)
struct rtw_usb *rtwusb = rtw_get_usb_priv(rtwdev);
int i;
+ spin_lock_init(&rtwusb->hiq_lock);
+ rtwusb->hiq_budget = RTW_USB_HIQ_BUDGET_MAX;
+ rtwusb->hiq_refill = jiffies;
+
rtwusb->txwq = create_singlethread_workqueue("rtw88_usb: tx wq");
if (!rtwusb->txwq) {
rtw_err(rtwdev, "failed to create TX work queue\n");
diff --git a/drivers/net/wireless/realtek/rtw88/usb.h b/drivers/net/wireless/realtek/rtw88/usb.h
index 9b695b688b24..94ff21dd0846 100644
--- a/drivers/net/wireless/realtek/rtw88/usb.h
+++ b/drivers/net/wireless/realtek/rtw88/usb.h
@@ -75,6 +75,11 @@ struct rtw_usb {
u8 out_ep[RTW_USB_EP_MAX];
int qsel_to_ep[TX_DESC_QSEL_MAX];
+ /* protects hiq_budget and hiq_refill */
+ spinlock_t hiq_lock;
+ u32 hiq_budget;
+ unsigned long hiq_refill;
+
struct workqueue_struct *txwq, *rxwq;
struct sk_buff_head tx_queue[RTW_USB_EP_MAX];
diff --git a/drivers/net/wireless/rsi/rsi_91x_core.c b/drivers/net/wireless/rsi/rsi_91x_core.c
index ead4d4e04328..3281fc94e107 100644
--- a/drivers/net/wireless/rsi/rsi_91x_core.c
+++ b/drivers/net/wireless/rsi/rsi_91x_core.c
@@ -458,7 +458,7 @@ void rsi_core_xmit(struct rsi_common *common, struct sk_buff *skb)
tx_params->sta_id = 0;
}
- if (rsta) {
+ if (rsta && tid < IEEE80211_NUM_TIDS) {
/* Start aggregation if not done for this tid */
if (!rsta->start_tx_aggr[tid]) {
rsta->start_tx_aggr[tid] = true;
diff --git a/drivers/nvme/common/keyring.c b/drivers/nvme/common/keyring.c
index 32d16c53133b..87cb51e1f5a7 100644
--- a/drivers/nvme/common/keyring.c
+++ b/drivers/nvme/common/keyring.c
@@ -35,6 +35,7 @@ struct key *nvme_tls_key_lookup(key_serial_t key_id)
}
if (nvme_tls_psk_revoked(key)) {
pr_err("key id %08x revoked\n", key_id);
+ key_put(key);
return ERR_PTR(-EKEYREVOKED);
}
return key;
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index af71d02b8476..a0811b49710c 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3941,10 +3941,16 @@ static int nvme_subsys_check_duplicate_ids(struct nvme_subsystem *subsys,
return 0;
}
+/* The char device of a namespace rather than of a namespace head */
+static bool nvme_is_ns_cdev(struct device *dev)
+{
+ return dev->parent->class == &nvme_class;
+}
+
static void nvme_cdev_rel(struct device *dev)
{
ida_free(&nvme_ns_chr_minor_ida, MINOR(dev->devt));
- if (dev->parent->class == &nvme_class)
+ if (nvme_is_ns_cdev(dev))
nvme_put_ns(container_of(dev, struct nvme_ns, cdev_device));
else
nvme_put_ns_head(container_of(dev, struct nvme_ns_head,
@@ -3978,6 +3984,14 @@ int nvme_cdev_add(struct cdev *cdev, struct device *cdev_device,
device_initialize(cdev_device);
cdev_init(cdev, fops);
cdev->owner = owner;
+ /* Undone in nvme_cdev_rel() */
+ if (nvme_is_ns_cdev(cdev_device))
+ nvme_get_ns(container_of(cdev_device, struct nvme_ns,
+ cdev_device));
+ else
+ nvme_get_ns_head(container_of(cdev_device,
+ struct nvme_ns_head,
+ cdev_device));
ret = cdev_device_add(cdev, cdev_device);
if (ret)
put_device(cdev_device);
@@ -4010,13 +4024,11 @@ static void nvme_add_ns_cdev(struct nvme_ns *ns)
{
ns->cdev_device.parent = ns->ctrl->device;
- nvme_get_ns(ns); /* Undone in nvme_cdev_rel() */
if (nvme_cdev_add(&ns->cdev, &ns->cdev_device,
&nvme_ns_chr_fops, ns->ctrl->ops->module,
ns->ctrl->instance, ns->head->instance)) {
dev_err(ns->ctrl->device, "Unable to create the ng%dn%d device\n",
ns->ctrl->instance, ns->head->instance);
- nvme_put_ns(ns);
return;
}
set_bit(NVME_NS_CDEV_LIVE, &ns->flags);
@@ -4743,9 +4755,8 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING_NOIO);
mutex_lock(&ctrl->namespaces_lock);
- list_splice_init_rcu(&ctrl->namespaces, &ns_list, synchronize_rcu);
+ list_splice_init_srcu(&ctrl->namespaces, &ns_list, &ctrl->srcu);
mutex_unlock(&ctrl->namespaces_lock);
- synchronize_srcu(&ctrl->srcu);
list_for_each_entry_safe(ns, next, &ns_list, list)
nvme_ns_remove(ns);
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 99ecc73393e5..0920fa9616de 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -648,14 +648,12 @@ static void nvme_add_ns_head_cdev(struct nvme_ns_head *head)
{
head->cdev_device.parent = &head->subsys->dev;
- nvme_get_ns_head(head); /* Undone in nvme_cdev_rel() */
if (nvme_cdev_add(&head->cdev, &head->cdev_device,
&nvme_ns_head_chr_fops, THIS_MODULE,
head->subsys->instance, head->instance)) {
dev_err(disk_to_dev(head->disk),
"Unable to create the ng%dn%d device\n",
head->subsys->instance, head->instance);
- nvme_put_ns_head(head);
return;
}
set_bit(NVME_NSHEAD_CDEV_LIVE, &head->flags);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 77317e5d00f9..b7c5a18b899b 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -4198,6 +4198,8 @@ static const struct pci_device_id nvme_id_table[] = {
.driver_data = NVME_QUIRK_BOGUS_NID, },
{ PCI_DEVICE(0x1344, 0x6004), /* Micron 4100AT */
.driver_data = NVME_QUIRK_DMAPOOL_ALIGN_512, },
+ { PCI_DEVICE(0x1e0f, 0x0032), /* KIOXIA XG10d */
+ .driver_data = NVME_QUIRK_DMAPOOL_ALIGN_512, },
{ PCI_DEVICE(0x1c5c, 0x1504), /* SK Hynix PC400 */
.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
{ PCI_DEVICE(0x1c5c, 0x174a), /* SK Hynix P31 SSD */
diff --git a/drivers/nvme/host/pr.c b/drivers/nvme/host/pr.c
index fe7dbe264815..d8b6ac4d83bb 100644
--- a/drivers/nvme/host/pr.c
+++ b/drivers/nvme/host/pr.c
@@ -128,8 +128,12 @@ static int nvme_pr_register(struct block_device *bdev, u64 old_key, u64 new_key,
data.crkey = cpu_to_le64(old_key);
data.nrkey = cpu_to_le64(new_key);
- cdw10 = old_key ? NVME_PR_REGISTER_ACT_REPLACE :
- NVME_PR_REGISTER_ACT_REG;
+ if (!new_key)
+ cdw10 = NVME_PR_REGISTER_ACT_UNREG;
+ else if (old_key)
+ cdw10 = NVME_PR_REGISTER_ACT_REPLACE;
+ else
+ cdw10 = NVME_PR_REGISTER_ACT_REG;
cdw10 |= (flags & PR_FL_IGNORE_KEY) ? NVME_PR_IGNORE_KEY : 0;
cdw10 |= NVME_PR_CPTPL_PERSIST;
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 5b9b540693b5..ff7b9e379614 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -735,7 +735,6 @@ static int nvme_rdma_setup_io_queues(struct nvme_rdma_ctrl *ctrl,
for (i = 0; i < nr_queues; i++) {
struct nvme_rdma_queue *queue = &ctrl->queues[first + i];
- queue->ctrl = ctrl;
queue->queue_size = queue_size;
ctxs[i].queue = queue;
@@ -798,7 +797,6 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
bool pi_capable = false;
int error;
- ctrl->queues[0].ctrl = ctrl;
ctrl->queues[0].queue_size = NVME_AQ_DEPTH;
error = nvme_rdma_alloc_queue(&ctrl->queues[0]);
if (error)
@@ -2284,7 +2282,7 @@ static struct nvme_rdma_ctrl *nvme_rdma_alloc_ctrl(struct device *dev,
struct nvmf_ctrl_options *opts)
{
struct nvme_rdma_ctrl *ctrl;
- int ret;
+ int ret, i;
ctrl = kzalloc_obj(*ctrl);
if (!ctrl)
@@ -2343,6 +2341,9 @@ static struct nvme_rdma_ctrl *nvme_rdma_alloc_ctrl(struct device *dev,
if (!ctrl->queues)
goto out_free_ctrl;
+ for (i = 0; i < ctrl->ctrl.queue_count; i++)
+ ctrl->queues[i].ctrl = ctrl;
+
ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops,
0 /* no quirks, we're perfect! */);
if (ret)
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index ce28c9b03817..50297ce8bdde 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -70,6 +70,12 @@ MODULE_PARM_DESC(tls_handshake_timeout,
"nvme TLS handshake timeout in seconds (default 10)");
#endif
+/* read a TLS control record in chunks of this size */
+#define NVME_TCP_TLS_REC_CHUNK 256
+
+/* TLS records read while waiting for the ICResp, the ICResp included */
+#define NVME_TCP_ICRESP_MAX_RECORDS 8
+
static atomic_t nvme_tcp_cpu_queues[NR_CPUS];
enum nvme_tcp_send_state {
@@ -111,6 +117,9 @@ enum nvme_tcp_queue_flags {
NVME_TCP_Q_IO_CPU_SET = 3,
};
+#define NVME_TCP_OPT_NR_QUEUES(opts) ((opts)->nr_io_queues + \
+ (opts)->nr_write_queues + (opts)->nr_poll_queues + 1)
+
enum nvme_tcp_recv_state {
NVME_TCP_RECV_PDU = 0,
NVME_TCP_RECV_DATA,
@@ -1550,23 +1559,161 @@ static void nvme_tcp_free_queue(struct nvme_ctrl *nctrl, int qid)
* nvme_reset_ctrl_work(), queue teardown is immediately followed by
* re-allocation. fput() defers socket cleanup to delayed_fput_work
* in workqueue context, which can race with new queue setup.
+ *
+ * Clear queue->sock under the queue lock so that a sysfs reader in
+ * nvme_tcp_get_address() either observes the NVME_TCP_Q_LIVE bit
+ * cleared or a still valid socket. The queue lock itself is owned
+ * by the controller and only destroyed when the controller is
+ * freed, so readers blocked on it cannot race its teardown.
*/
+ mutex_lock(&queue->queue_lock);
__fput_sync(queue->sock->file);
queue->sock = NULL;
+ mutex_unlock(&queue->queue_lock);
memalloc_noio_restore(noio_flag);
kfree(queue->pdu);
mutex_destroy(&queue->send_mutex);
- mutex_destroy(&queue->queue_lock);
mutex_destroy(&queue->pf_cache_lock);
}
+static int nvme_tcp_recv_tls_alert(struct nvme_tcp_queue *queue,
+ struct msghdr *msg, int len)
+{
+ struct sock *sk = queue->sock->sk;
+ u8 level, description;
+
+ if (len < 2) {
+ dev_err(queue->ctrl->ctrl.device,
+ "queue %d: truncated TLS alert\n",
+ nvme_tcp_queue_id(queue));
+ return -ENOTCONN;
+ }
+
+ /* tls_alert_recv() reads through the iterator */
+ iov_iter_revert(&msg->msg_iter, len);
+ tls_alert_recv(sk, msg, &level, &description);
+
+ if (level == TLS_ALERT_LEVEL_FATAL) {
+ dev_err(queue->ctrl->ctrl.device,
+ "queue %d: fatal TLS alert %u\n",
+ nvme_tcp_queue_id(queue), description);
+ return -ENOTCONN;
+ }
+
+ /* a peer shutting down normally is not worth a warning */
+ if (description == TLS_ALERT_DESC_CLOSE_NOTIFY)
+ dev_dbg(queue->ctrl->ctrl.device,
+ "queue %d: TLS close_notify\n",
+ nvme_tcp_queue_id(queue));
+ else
+ dev_warn(queue->ctrl->ctrl.device,
+ "queue %d: TLS alert %u\n",
+ nvme_tcp_queue_id(queue), description);
+
+ return 0;
+}
+
+/*
+ * Discard a TLS record that arrived in place of the ICResp. A peer may send
+ * a NewSessionTicket as soon as the handshake completes.
+ *
+ * Returns 0 when the record was discarded and the ICResp can be read, or a
+ * negative errno if the connection cannot continue.
+ */
+static int nvme_tcp_skip_tls_record(struct nvme_tcp_queue *queue,
+ struct msghdr *msg, u8 ctype, int len)
+{
+ u8 buf[NVME_TCP_TLS_REC_CHUNK];
+ int ret;
+
+ switch (ctype) {
+ case TLS_RECORD_TYPE_HANDSHAKE:
+ break;
+ case TLS_RECORD_TYPE_ALERT:
+ ret = nvme_tcp_recv_tls_alert(queue, msg, len);
+ return ret ? : -ENOTCONN;
+ default:
+ pr_err("queue %d: unhandled TLS record %u\n",
+ nvme_tcp_queue_id(queue), ctype);
+ return -ENOTCONN;
+ }
+
+ /* the ICResp is in a later record, drain this one first */
+ while (!(msg->msg_flags & MSG_EOR)) {
+ struct kvec iov = {
+ .iov_base = buf,
+ .iov_len = sizeof(buf),
+ };
+ char cbuf[CMSG_LEN(sizeof(char))] = {};
+ /* kTLS fails a control record read without a cmsg buffer */
+ struct msghdr drain = {
+ .msg_control = cbuf,
+ .msg_controllen = sizeof(cbuf),
+ };
+
+ ret = kernel_recvmsg(queue->sock, &drain, &iov, 1, sizeof(buf),
+ 0);
+ if (ret <= 0) {
+ pr_warn("queue %d: failed to drain TLS record: %d\n",
+ nvme_tcp_queue_id(queue), ret);
+ return ret ? : -ECONNRESET;
+ }
+ msg->msg_flags = drain.msg_flags;
+ }
+
+ return 0;
+}
+
+/*
+ * Receive the ICResp. Over TLS, skip the records that a peer may send
+ * ahead of it.
+ *
+ * Returns the length received, or a negative errno.
+ */
+static int nvme_tcp_recv_icresp(struct nvme_tcp_queue *queue,
+ struct nvme_tcp_icresp_pdu *icresp)
+{
+ char cbuf[CMSG_LEN(sizeof(char))];
+ struct msghdr msg;
+ struct kvec iov;
+ u8 ctype;
+ int i, ret;
+
+ for (i = 0; i < NVME_TCP_ICRESP_MAX_RECORDS; i++) {
+ memset(&msg, 0, sizeof(msg));
+ memset(cbuf, 0, sizeof(cbuf));
+ iov.iov_base = icresp;
+ iov.iov_len = sizeof(*icresp);
+ if (nvme_tcp_queue_tls(queue)) {
+ msg.msg_control = cbuf;
+ msg.msg_controllen = sizeof(cbuf);
+ }
+ msg.msg_flags = MSG_WAITALL;
+ ret = kernel_recvmsg(queue->sock, &msg, &iov, 1,
+ iov.iov_len, msg.msg_flags);
+ if (ret <= 0 || !nvme_tcp_queue_tls(queue))
+ return ret;
+
+ ctype = tls_get_record_type(queue->sock->sk,
+ (struct cmsghdr *)cbuf);
+ if (ctype == TLS_RECORD_TYPE_DATA)
+ return ret;
+
+ ret = nvme_tcp_skip_tls_record(queue, &msg, ctype, ret);
+ if (ret < 0)
+ return ret;
+ }
+
+ pr_err("queue %d: too many TLS records before the ICResp\n",
+ nvme_tcp_queue_id(queue));
+ return -EPROTO;
+}
+
static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue)
{
struct nvme_tcp_icreq_pdu *icreq;
struct nvme_tcp_icresp_pdu *icresp;
- char cbuf[CMSG_LEN(sizeof(char))] = {};
- u8 ctype;
struct msghdr msg = {};
struct kvec iov;
bool ctrl_hdgst, ctrl_ddgst;
@@ -1604,16 +1751,7 @@ static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue)
goto free_icresp;
}
- memset(&msg, 0, sizeof(msg));
- iov.iov_base = icresp;
- iov.iov_len = sizeof(*icresp);
- if (nvme_tcp_queue_tls(queue)) {
- msg.msg_control = cbuf;
- msg.msg_controllen = sizeof(cbuf);
- }
- msg.msg_flags = MSG_WAITALL;
- ret = kernel_recvmsg(queue->sock, &msg, &iov, 1,
- iov.iov_len, msg.msg_flags);
+ ret = nvme_tcp_recv_icresp(queue, icresp);
if (ret >= 0 && ret < sizeof(*icresp))
ret = -ECONNRESET;
if (ret < 0) {
@@ -1621,16 +1759,6 @@ static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue)
nvme_tcp_queue_id(queue), ret);
goto free_icresp;
}
- ret = -ENOTCONN;
- if (nvme_tcp_queue_tls(queue)) {
- ctype = tls_get_record_type(queue->sock->sk,
- (struct cmsghdr *)cbuf);
- if (ctype != TLS_RECORD_TYPE_DATA) {
- pr_err("queue %d: unhandled TLS record %d\n",
- nvme_tcp_queue_id(queue), ctype);
- goto free_icresp;
- }
- }
ret = -EINVAL;
if (icresp->hdr.type != nvme_tcp_icresp) {
pr_err("queue %d: bad type returned %d\n",
@@ -1877,7 +2005,6 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
int ret, rcv_pdu_size;
struct file *sock_file;
- mutex_init(&queue->queue_lock);
queue->ctrl = ctrl;
init_llist_head(&queue->req_list);
INIT_LIST_HEAD(&queue->send_list);
@@ -2015,7 +2142,6 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
queue->sock = NULL;
err_destroy_mutex:
mutex_destroy(&queue->send_mutex);
- mutex_destroy(&queue->queue_lock);
mutex_destroy(&queue->pf_cache_lock);
return ret;
}
@@ -2665,6 +2791,9 @@ static void nvme_tcp_stop_ctrl(struct nvme_ctrl *ctrl)
static void nvme_tcp_free_ctrl(struct nvme_ctrl *nctrl)
{
struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
+ /* the same count used to size and initialize the queue array */
+ unsigned int nr_queues = NVME_TCP_OPT_NR_QUEUES(nctrl->opts);
+ unsigned int i;
if (list_empty_careful(&ctrl->list))
goto free_ctrl;
@@ -2675,6 +2804,8 @@ static void nvme_tcp_free_ctrl(struct nvme_ctrl *nctrl)
nvmf_free_options(nctrl->opts);
free_ctrl:
+ for (i = 0; i < nr_queues; i++)
+ mutex_destroy(&ctrl->queues[i].queue_lock);
kfree(ctrl->queues);
kfree(ctrl);
}
@@ -2927,17 +3058,18 @@ static int nvme_tcp_get_address(struct nvme_ctrl *ctrl, char *buf, int size)
len = nvmf_get_address(ctrl, buf, size);
- if (!test_bit(NVME_TCP_Q_LIVE, &queue->flags))
- return len;
-
mutex_lock(&queue->queue_lock);
- ret = kernel_getsockname(queue->sock, (struct sockaddr *)&src_addr);
- if (ret > 0) {
- if (len > 0)
- len--; /* strip trailing newline */
- len += scnprintf(buf + len, size - len, "%ssrc_addr=%pISc\n",
- (len) ? "," : "", &src_addr);
+ if (test_bit(NVME_TCP_Q_LIVE, &queue->flags)) {
+ ret = kernel_getsockname(queue->sock,
+ (struct sockaddr *)&src_addr);
+ if (ret > 0) {
+ if (len > 0)
+ len--; /* strip trailing newline */
+ len += scnprintf(buf + len, size - len,
+ "%ssrc_addr=%pISc\n",
+ (len) ? "," : "", &src_addr);
+ }
}
mutex_unlock(&queue->queue_lock);
@@ -3003,7 +3135,7 @@ static struct nvme_tcp_ctrl *nvme_tcp_alloc_ctrl(struct device *dev,
struct nvmf_ctrl_options *opts)
{
struct nvme_tcp_ctrl *ctrl;
- int ret;
+ int ret, i;
ctrl = kzalloc_obj(*ctrl);
if (!ctrl)
@@ -3014,8 +3146,7 @@ static struct nvme_tcp_ctrl *nvme_tcp_alloc_ctrl(struct device *dev,
*/
context_unsafe(INIT_LIST_HEAD(&ctrl->list));
ctrl->ctrl.opts = opts;
- ctrl->ctrl.queue_count = opts->nr_io_queues + opts->nr_write_queues +
- opts->nr_poll_queues + 1;
+ ctrl->ctrl.queue_count = NVME_TCP_OPT_NR_QUEUES(opts);
ctrl->ctrl.sqsize = opts->queue_size - 1;
ctrl->ctrl.kato = opts->kato;
@@ -3073,6 +3204,14 @@ static struct nvme_tcp_ctrl *nvme_tcp_alloc_ctrl(struct device *dev,
goto out_free_ctrl;
}
+ /*
+ * The queue locks are owned by the controller, not the queues:
+ * they outlive the per-queue connect/disconnect cycles so that
+ * sysfs readers can never race their destruction.
+ */
+ for (i = 0; i < ctrl->ctrl.queue_count; i++)
+ mutex_init(&ctrl->queues[i].queue_lock);
+
ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_tcp_ctrl_ops, 0);
if (ret)
goto out_kfree_queues;
diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 7764a3c0195c..3704493dbec9 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -543,6 +543,15 @@ static void nvmet_execute_get_log_page_ana(struct nvmet_req *req)
u16 ngrps = 0;
u16 status;
+ /*
+ * The header and one descriptor per enabled group are the only bytes
+ * written below; zero the whole transfer first so the unwritten tail
+ * does not leak stale kernel memory to the host.
+ */
+ status = nvmet_zero_sgl(req, 0, req->transfer_len);
+ if (status)
+ goto out;
+
status = NVME_SC_INTERNAL;
desc = kmalloc_flex(*desc, nsids, NVMET_MAX_NAMESPACES);
if (!desc)
@@ -1225,8 +1234,13 @@ u16 nvmet_set_feat_kato(struct nvmet_req *req)
{
u32 val32 = le32_to_cpu(req->cmd->common.cdw11);
+ if (val32 > NVMET_MAX_KATO * MSEC_PER_SEC) {
+ req->error_loc = offsetof(struct nvme_common_command, cdw11);
+ return NVME_SC_KA_TIMEOUT_INVALID | NVME_STATUS_DNR;
+ }
+
nvmet_stop_keep_alive_timer(req->sq->ctrl);
- req->sq->ctrl->kato = DIV_ROUND_UP(val32, 1000);
+ req->sq->ctrl->kato = DIV_ROUND_UP_ULL(val32, MSEC_PER_SEC);
nvmet_start_keep_alive_timer(req->sq->ctrl);
nvmet_set_result(req, req->sq->ctrl->kato);
diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 83b86180c50f..61107bcfc895 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -150,7 +150,7 @@ static ssize_t nvmet_addr_traddr_store(struct config_item *item,
if (nvmet_is_port_enabled(port, __func__))
return -EACCES;
- if (sscanf(page, "%s\n", port->disc_addr.traddr) != 1)
+ if (sscanf(page, "%255s", port->disc_addr.traddr) != 1)
return -EINVAL;
return count;
}
@@ -242,7 +242,7 @@ static ssize_t nvmet_addr_trsvcid_store(struct config_item *item,
if (nvmet_is_port_enabled(port, __func__))
return -EACCES;
- if (sscanf(page, "%s\n", port->disc_addr.trsvcid) != 1)
+ if (sscanf(page, "%31s", port->disc_addr.trsvcid) != 1)
return -EINVAL;
return count;
}
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 07a74c2e1f58..0f5e94bf12bd 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1612,6 +1612,12 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
u8 dhchap_status;
int ret;
+ if (kato > NVMET_MAX_KATO * MSEC_PER_SEC) {
+ args->status = NVME_SC_KA_TIMEOUT_INVALID | NVME_STATUS_DNR;
+ args->error_loc = offsetof(struct nvmf_connect_command, kato);
+ return NULL;
+ }
+
args->status = NVME_SC_CONNECT_INVALID_PARAM | NVME_STATUS_DNR;
subsys = nvmet_find_get_subsys(args->port, args->subsysnqn);
if (!subsys) {
@@ -1678,7 +1684,7 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
kato = NVMET_DISC_KATO_MS;
/* keep-alive timeout in seconds */
- ctrl->kato = DIV_ROUND_UP(kato, 1000);
+ ctrl->kato = DIV_ROUND_UP_ULL(kato, MSEC_PER_SEC);
ctrl->err_counter = 0;
spin_lock_init(&ctrl->error_lock);
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 2032db380f99..215315ebe42d 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -7,6 +7,7 @@
#define _NVMET_H
#include <linux/dma-mapping.h>
+#include <linux/jiffies.h>
#include <linux/types.h>
#include <linux/device.h>
#include <linux/kref.h>
@@ -702,6 +703,13 @@ void nvmet_add_async_event(struct nvmet_ctrl *ctrl, u8 event_type,
#define NVMET_KAS 10
#define NVMET_DISC_KATO_MS 120000
+/*
+ * Largest keep-alive timeout in seconds that can be reported back in
+ * milliseconds and converted to jiffies without overflowing.
+ */
+#define NVMET_MAX_KATO \
+ min3(U32_MAX / MSEC_PER_SEC, U32_MAX / HZ, MAX_JIFFY_OFFSET / HZ)
+
int __init nvmet_init_configfs(void);
void __exit nvmet_exit_configfs(void);
diff --git a/drivers/pci/controller/cadence/pcie-cadence-plat.c b/drivers/pci/controller/cadence/pcie-cadence-plat.c
index a1ea24fc3b63..4a24302cc5b2 100644
--- a/drivers/pci/controller/cadence/pcie-cadence-plat.c
+++ b/drivers/pci/controller/cadence/pcie-cadence-plat.c
@@ -14,14 +14,6 @@
#define CDNS_PLAT_CPU_TO_BUS_ADDR 0x0FFFFFFF
-/**
- * struct cdns_plat_pcie - private data for this PCIe platform driver
- * @pcie: Cadence PCIe controller
- */
-struct cdns_plat_pcie {
- struct cdns_pcie *pcie;
-};
-
static const struct of_device_id cdns_plat_pcie_of_match[];
static u64 cdns_plat_cpu_addr_fixup(struct cdns_pcie *pcie, u64 cpu_addr)
@@ -36,12 +28,11 @@ static const struct cdns_pcie_ops cdns_plat_ops = {
static int cdns_plat_pcie_probe(struct platform_device *pdev)
{
const struct cdns_plat_pcie_of_data *data;
- struct cdns_plat_pcie *cdns_plat_pcie;
struct device *dev = &pdev->dev;
struct pci_host_bridge *bridge;
+ struct cdns_pcie *pcie;
struct cdns_pcie_ep *ep;
struct cdns_pcie_rc *rc;
- int phy_count;
bool is_rc;
int ret;
@@ -52,11 +43,6 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
is_rc = data->is_rc;
pr_debug(" Started %s with is_rc: %d\n", __func__, is_rc);
- cdns_plat_pcie = devm_kzalloc(dev, sizeof(*cdns_plat_pcie), GFP_KERNEL);
- if (!cdns_plat_pcie)
- return -ENOMEM;
-
- platform_set_drvdata(pdev, cdns_plat_pcie);
if (is_rc) {
if (!IS_ENABLED(CONFIG_PCIE_CADENCE_PLAT_HOST))
return -ENODEV;
@@ -68,9 +54,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
rc = pci_host_bridge_priv(bridge);
rc->pcie.dev = dev;
rc->pcie.ops = &cdns_plat_ops;
- cdns_plat_pcie->pcie = &rc->pcie;
+ pcie = &rc->pcie;
- ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie);
+ ret = cdns_pcie_init_phy(dev, pcie);
if (ret) {
dev_err(dev, "failed to init phy\n");
return ret;
@@ -95,9 +81,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
ep->pcie.dev = dev;
ep->pcie.ops = &cdns_plat_ops;
- cdns_plat_pcie->pcie = &ep->pcie;
+ pcie = &ep->pcie;
- ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie);
+ ret = cdns_pcie_init_phy(dev, pcie);
if (ret) {
dev_err(dev, "failed to init phy\n");
return ret;
@@ -115,18 +101,17 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
goto err_init;
}
+ platform_set_drvdata(pdev, pcie);
+
return 0;
err_init:
err_get_sync:
pm_runtime_put_sync(dev);
pm_runtime_disable(dev);
- cdns_pcie_disable_phy(cdns_plat_pcie->pcie);
- phy_count = cdns_plat_pcie->pcie->phy_count;
- while (phy_count--)
- device_link_del(cdns_plat_pcie->pcie->link[phy_count]);
+ cdns_pcie_disable_phy(pcie);
- return 0;
+ return ret;
}
static void cdns_plat_pcie_shutdown(struct platform_device *pdev)
diff --git a/drivers/pci/controller/cadence/pcie-cadence.c b/drivers/pci/controller/cadence/pcie-cadence.c
index a1eada56edba..0ac980249941 100644
--- a/drivers/pci/controller/cadence/pcie-cadence.c
+++ b/drivers/pci/controller/cadence/pcie-cadence.c
@@ -222,7 +222,6 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie)
struct device_node *np = dev->of_node;
int phy_count;
struct phy **phy;
- struct device_link **link;
int i;
int ret;
const char *name;
@@ -238,10 +237,6 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie)
if (!phy)
return -ENOMEM;
- link = devm_kcalloc(dev, phy_count, sizeof(*link), GFP_KERNEL);
- if (!link)
- return -ENOMEM;
-
for (i = 0; i < phy_count; i++) {
of_property_read_string_index(np, "phy-names", i, &name);
phy[i] = devm_phy_get(dev, name);
@@ -249,17 +244,10 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie)
ret = PTR_ERR(phy[i]);
goto err_phy;
}
- link[i] = device_link_add(dev, &phy[i]->dev, DL_FLAG_STATELESS);
- if (!link[i]) {
- devm_phy_put(dev, phy[i]);
- ret = -EINVAL;
- goto err_phy;
- }
}
pcie->phy_count = phy_count;
pcie->phy = phy;
- pcie->link = link;
ret = cdns_pcie_enable_phy(pcie);
if (ret)
@@ -268,10 +256,8 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie)
return 0;
err_phy:
- while (--i >= 0) {
- device_link_del(link[i]);
+ while (--i >= 0)
devm_phy_put(dev, phy[i]);
- }
return ret;
}
diff --git a/drivers/pci/controller/cadence/pcie-cadence.h b/drivers/pci/controller/cadence/pcie-cadence.h
index b4d51adefea5..4a96e1869e53 100644
--- a/drivers/pci/controller/cadence/pcie-cadence.h
+++ b/drivers/pci/controller/cadence/pcie-cadence.h
@@ -260,7 +260,6 @@ struct cdns_plat_pcie_of_data {
* @is_hpa: indicates if the architecture is HPA
* @phy_count: number of supported PHY devices
* @phy: list of pointers to specific PHY control blocks
- * @link: list of pointers to corresponding device link representations
* @ops: Platform-specific ops to control various inputs from Cadence PCIe
* wrapper
* @cdns_pcie_reg_offsets: Register bank offsets for different SoC
@@ -276,7 +275,6 @@ struct cdns_pcie {
bool is_hpa;
int phy_count;
struct phy **phy;
- struct device_link **link;
const struct cdns_pcie_ops *ops;
const struct cdns_plat_pcie_of_data *cdns_pcie_reg_offsets;
int max_link_speed;
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index 593388f29bdd..9297d0e607e3 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -577,6 +577,8 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci,
/* The data-less messages only for now */
val |= PCIE_ATU_INHIBIT_PAYLOAD | atu->code;
}
+ if (dw_pcie_ver_is_ge(pci, 460A))
+ val |= PCIE_ATU_DMA_BYPASS;
dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL2, val);
/*
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 0735ae940924..668a5ba1c6f5 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -201,6 +201,7 @@
#define PCIE_ATU_ENABLE BIT(31)
#define PCIE_ATU_BAR_MODE_ENABLE BIT(30)
#define PCIE_ATU_CFG_SHIFT_MODE_ENABLE BIT(28)
+#define PCIE_ATU_DMA_BYPASS BIT(27)
#define PCIE_ATU_INHIBIT_PAYLOAD BIT(22)
#define PCIE_ATU_FUNC_NUM_MATCH_EN BIT(19)
#define PCIE_ATU_LOWER_BASE 0x008
diff --git a/drivers/platform/chrome/cros_ec_ishtp.c b/drivers/platform/chrome/cros_ec_ishtp.c
index 3766cef81fe8..b21a27ad3762 100644
--- a/drivers/platform/chrome/cros_ec_ishtp.c
+++ b/drivers/platform/chrome/cros_ec_ishtp.c
@@ -369,7 +369,7 @@ static void ish_event_cb(struct ishtp_cl_device *cl_device)
* @cros_ish_cl: ISHTP client instance
* @reset: true if called from reset handler
*
- * This function complete the initializtion of the client.
+ * This function complete the initialization of the client.
*
* Return: 0 for success, negative error code for failure.
*/
diff --git a/drivers/platform/chrome/cros_usbpd_notify.c b/drivers/platform/chrome/cros_usbpd_notify.c
index d842c05f4db0..25da31ec1996 100644
--- a/drivers/platform/chrome/cros_usbpd_notify.c
+++ b/drivers/platform/chrome/cros_usbpd_notify.c
@@ -64,7 +64,7 @@ static void cros_usbpd_get_event_and_notify(struct device *dev,
/*
* We still send a 0 event out to older devices which don't
- * have the updated device heirarchy.
+ * have the updated device hierarchy.
*/
if (!ec_dev) {
dev_dbg(dev,
@@ -110,26 +110,29 @@ static int cros_usbpd_notify_probe_acpi(struct platform_device *pdev)
if (!pdnotify)
return -ENOMEM;
- /* Get the EC device pointer needed to talk to the EC. */
- ec_dev = dev_get_drvdata(dev->parent);
- if (!ec_dev) {
- /*
- * We continue even for older devices which don't have the
- * correct device heirarchy, namely, GOOG0003 is a child
- * of GOOG0004. If GOOG0003 is a child of GOOG0004 and we
- * can't get a pointer to the Chrome EC device, defer the
- * probe function.
- */
- parent_fwnode = fwnode_get_parent(dev->fwnode);
- if (parent_fwnode) {
- parent_adev = to_acpi_device_node(parent_fwnode);
- if (parent_adev &&
- acpi_dev_hid_match(parent_adev, CREC_DRV_NAME)) {
- return -EPROBE_DEFER;
- }
+ /*
+ * Get the EC device pointer needed to talk to the EC. The parent's
+ * driver data is only a struct cros_ec_device when GOOG0003 is a
+ * child of GOOG0004. On older devices without that hierarchy the
+ * parent may be bound to an unrelated driver (e.g. the ACPI EC
+ * driver for PNP0C09), so don't touch its driver data and continue
+ * without an EC pointer. If GOOG0003 is a child of GOOG0004 and we
+ * can't get a pointer to the Chrome EC device yet, defer the probe.
+ */
+ ec_dev = NULL;
+ parent_fwnode = fwnode_get_parent(dev->fwnode);
+ parent_adev = to_acpi_device_node(parent_fwnode);
+ if (parent_adev && acpi_dev_hid_match(parent_adev, CREC_DRV_NAME)) {
+ ec_dev = dev_get_drvdata(dev->parent);
+ if (!ec_dev) {
+ fwnode_handle_put(parent_fwnode);
+ return -EPROBE_DEFER;
}
- dev_warn(dev, "Couldn't get Chrome EC device pointer.\n");
}
+ fwnode_handle_put(parent_fwnode);
+
+ if (!ec_dev)
+ dev_warn(dev, "Couldn't get Chrome EC device pointer.\n");
pdnotify->dev = dev;
pdnotify->ec = ec_dev;
diff --git a/drivers/ptp/ptp_sysfs.c b/drivers/ptp/ptp_sysfs.c
index dc398c6b7528..7d09256aca72 100644
--- a/drivers/ptp/ptp_sysfs.c
+++ b/drivers/ptp/ptp_sysfs.c
@@ -80,13 +80,17 @@ static ssize_t extts_fifo_show(struct device *dev,
size_t qcnt;
int cnt = 0;
+ spin_lock_irqsave(&ptp->tsevqs_lock, flags);
cnt = list_count_nodes(&ptp->tsevqs);
- if (cnt <= 0)
+ if (cnt <= 0) {
+ spin_unlock_irqrestore(&ptp->tsevqs_lock, flags);
goto out;
+ }
/* The sysfs fifo will always draw from the fist queue */
queue = list_first_entry(&ptp->tsevqs, struct timestamp_event_queue,
qlist);
+ spin_unlock_irqrestore(&ptp->tsevqs_lock, flags);
memset(&event, 0, sizeof(event));
spin_lock_irqsave(&queue->lock, flags);
diff --git a/drivers/scsi/aic7xxx/aic79xx_core.c b/drivers/scsi/aic7xxx/aic79xx_core.c
index 3602a1fc40e4..4abc7948187f 100644
--- a/drivers/scsi/aic7xxx/aic79xx_core.c
+++ b/drivers/scsi/aic7xxx/aic79xx_core.c
@@ -8942,8 +8942,10 @@ ahd_handle_scsi_status(struct ahd_softc *ahd, struct scb *scb)
}
#endif
- if (ahd_perform_autosense(scb) == 0)
+ if (ahd_perform_autosense(scb) == 0) {
+ ahd_done(ahd, scb);
break;
+ }
ahd_compile_devinfo(&devinfo, SCB_GET_OUR_ID(scb),
SCB_GET_TARGET(ahd, scb),
diff --git a/drivers/scsi/aic7xxx/aic79xx_osm.h b/drivers/scsi/aic7xxx/aic79xx_osm.h
index 793fe19993a9..3bcfcfe18f08 100644
--- a/drivers/scsi/aic7xxx/aic79xx_osm.h
+++ b/drivers/scsi/aic7xxx/aic79xx_osm.h
@@ -499,6 +499,8 @@ int ahd_linux_show_info(struct seq_file *,struct Scsi_Host *);
static inline
void ahd_cmd_set_transaction_status(struct scsi_cmnd *cmd, uint32_t status)
{
+ if (!cmd)
+ return;
cmd->result &= ~(CAM_STATUS_MASK << 16);
cmd->result |= status << 16;
}
@@ -512,6 +514,8 @@ void ahd_set_transaction_status(struct scb *scb, uint32_t status)
static inline
void ahd_cmd_set_scsi_status(struct scsi_cmnd *cmd, uint32_t status)
{
+ if (!cmd)
+ return;
cmd->result &= ~0xFFFF;
cmd->result |= status;
}
@@ -525,6 +529,8 @@ void ahd_set_scsi_status(struct scb *scb, uint32_t status)
static inline
uint32_t ahd_cmd_get_transaction_status(struct scsi_cmnd *cmd)
{
+ if (!cmd)
+ return CAM_REQ_INPROG;
return ((cmd->result >> 16) & CAM_STATUS_MASK);
}
@@ -537,6 +543,8 @@ uint32_t ahd_get_transaction_status(struct scb *scb)
static inline
uint32_t ahd_cmd_get_scsi_status(struct scsi_cmnd *cmd)
{
+ if (!cmd)
+ return SAM_STAT_GOOD;
return (cmd->result & 0xFFFF);
}
@@ -564,13 +572,16 @@ u_long ahd_get_transfer_length(struct scb *scb)
static inline
int ahd_get_transfer_dir(struct scb *scb)
{
+ if (!scb->io_ctx)
+ return DMA_NONE;
return (scb->io_ctx->sc_data_direction);
}
static inline
void ahd_set_residual(struct scb *scb, u_long resid)
{
- scsi_set_resid(scb->io_ctx, resid);
+ if (scb->io_ctx)
+ scsi_set_resid(scb->io_ctx, resid);
}
static inline
@@ -582,6 +593,8 @@ void ahd_set_sense_residual(struct scb *scb, u_long resid)
static inline
u_long ahd_get_residual(struct scb *scb)
{
+ if (!scb->io_ctx)
+ return 0;
return scsi_get_resid(scb->io_ctx);
}
@@ -595,11 +608,12 @@ static inline
int ahd_perform_autosense(struct scb *scb)
{
/*
- * We always perform autosense in Linux.
- * On other platforms this is set on a
- * per-transaction basis.
+ * We always perform autosense in Linux, except for the device
+ * reset SCB, which has no command to return sense data to.
*/
- return (1);
+ if (!scb->io_ctx)
+ return 0;
+ return 1;
}
static inline uint32_t
@@ -629,6 +643,8 @@ void ahd_platform_freeze_devq(struct ahd_softc *ahd, struct scb *scb);
static inline void
ahd_freeze_scb(struct scb *scb)
{
+ if (!scb->io_ctx)
+ return;
if ((scb->io_ctx->result & (CAM_DEV_QFRZN << 16)) == 0) {
scb->io_ctx->result |= CAM_DEV_QFRZN << 16;
scb->platform_data->dev->qfrozen++;
diff --git a/drivers/scsi/aic7xxx/aic7xxx_core.c b/drivers/scsi/aic7xxx/aic7xxx_core.c
index b9761f9f0bbc..8ad82171e925 100644
--- a/drivers/scsi/aic7xxx/aic7xxx_core.c
+++ b/drivers/scsi/aic7xxx/aic7xxx_core.c
@@ -5266,9 +5266,13 @@ ahc_init(struct ahc_softc *ahc)
/*
* Assume we have a board at this stage and it has been reset.
+ * PCI controllers keep the host ID ahc_pci_config() chose.
*/
- if ((ahc->flags & AHC_USEDEFAULTS) != 0)
- ahc->our_id = ahc->our_id_b = 7;
+ if ((ahc->flags & AHC_USEDEFAULTS) != 0) {
+ if ((ahc->chip & AHC_PCI) == 0)
+ ahc->our_id = 7;
+ ahc->our_id_b = 7;
+ }
/*
* Default to allowing initiator operations.
diff --git a/drivers/scsi/aic7xxx/aic7xxx_pci.c b/drivers/scsi/aic7xxx/aic7xxx_pci.c
index 2d4c85426dc3..5a04c7d55e4b 100644
--- a/drivers/scsi/aic7xxx/aic7xxx_pci.c
+++ b/drivers/scsi/aic7xxx/aic7xxx_pci.c
@@ -701,6 +701,21 @@ ahc_find_pci_device(ahc_dev_softc_t pci)
return (NULL);
}
+/*
+ * The SGI O2 firmware runs its onboard controllers, in slots 1 and 2 of the
+ * root bus, without the Adaptec BIOS and at host ID 0. It leaves no settings
+ * in scratch RAM, which may still hold power-on contents with bad parity.
+ */
+static bool
+ahc_pci_sgi_o2_onboard(struct ahc_softc *ahc)
+{
+ int slot = ahc_get_pci_slot(ahc->dev_softc);
+
+ return IS_ENABLED(CONFIG_SGI_IP32) &&
+ ahc_get_pci_bus(ahc->dev_softc) == 0 &&
+ (slot == 1 || slot == 2);
+}
+
int
ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry)
{
@@ -782,6 +797,9 @@ ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry)
ahc->bus_intr = ahc_pci_intr;
ahc->bus_chip_init = ahc_pci_chip_init;
+ if (ahc_pci_sgi_o2_onboard(ahc))
+ ahc->flags |= AHC_NO_BIOS_INIT;
+
/* Remember how the card was setup in case there is no SEEPROM */
if ((ahc_inb(ahc, HCNTRL) & POWRDN) == 0) {
ahc_pause(ahc);
@@ -898,7 +916,7 @@ ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry)
* Assume only one connector and always turn
* on termination.
*/
- our_id = 0x07;
+ our_id = ahc_pci_sgi_o2_onboard(ahc) ? 0 : 0x07;
sxfrctl1 = STPWEN;
}
ahc_outb(ahc, SCSICONF, our_id|ENSPCHK|RESET_SCSI);
diff --git a/drivers/scsi/scsi_bsg.c b/drivers/scsi/scsi_bsg.c
index 5eec248a77a6..e4a65d7a2374 100644
--- a/drivers/scsi/scsi_bsg.c
+++ b/drivers/scsi/scsi_bsg.c
@@ -41,7 +41,7 @@ static void scsi_bsg_uring_task_cb(struct io_tw_req tw_req, io_tw_token_t tw)
u8 sense_len_wr = 0;
if (pdu->bio)
- blk_rq_unmap_user(pdu->bio);
+ ret = blk_rq_unmap_user(pdu->bio);
if (scsi_status_is_check_condition(scmd->result)) {
driver_status = DRIVER_SENSE;
@@ -199,7 +199,7 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr,
struct scsi_cmnd *scmd;
struct request *rq;
struct bio *bio;
- int ret;
+ int ret, unmap_ret;
if (hdr->protocol != BSG_PROTOCOL_SCSI ||
hdr->subprotocol != BSG_SUB_PROTOCOL_SCSI_CMD)
@@ -273,7 +273,9 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr,
else
hdr->dout_resid = scmd->resid_len;
- blk_rq_unmap_user(bio);
+ unmap_ret = blk_rq_unmap_user(bio);
+ if (!ret)
+ ret = unmap_ret;
out_put_request:
blk_mq_free_request(rq);
diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 9db2201aa553..c5d5d6a91fee 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -349,6 +349,19 @@ int core_enable_device_list_for_node(
new->attach_count++;
mutex_lock(&nacl->lun_entry_mutex);
+ /*
+ * Do not attach a new dev entry to an se_lun that is being torn down.
+ * lun_shutdown is set by core_tpg_remove_lun() before its
+ * core_clear_lun_from_tpg() sweep; checking it here under the same
+ * nacl->lun_entry_mutex the sweep takes guarantees that once the sweep
+ * has passed this ACL no dev entry can be left dangling on the
+ * se_lun->lun_deve_list that is about to be freed.
+ */
+ if (lun->lun_shutdown) {
+ mutex_unlock(&nacl->lun_entry_mutex);
+ ret = -ENODEV;
+ goto free_stats;
+ }
orig = target_nacl_find_deve(nacl, mapped_lun);
if (orig && orig->se_lun) {
struct se_lun *orig_lun = orig->se_lun;
@@ -375,12 +388,12 @@ int core_enable_device_list_for_node(
new->se_lun_acl = lun_acl;
hlist_del_rcu(&orig->link);
hlist_add_head_rcu(&new->link, &nacl->lun_entry_hlist);
- mutex_unlock(&nacl->lun_entry_mutex);
spin_lock(&lun->lun_deve_lock);
list_del(&orig->lun_link);
list_add_tail(&new->lun_link, &lun->lun_deve_list);
spin_unlock(&lun->lun_deve_lock);
+ mutex_unlock(&nacl->lun_entry_mutex);
kref_put(&orig->pr_kref, target_pr_kref_release);
wait_for_completion(&orig->pr_comp);
@@ -393,11 +406,10 @@ int core_enable_device_list_for_node(
new->se_lun = lun;
new->se_lun_acl = lun_acl;
hlist_add_head_rcu(&new->link, &nacl->lun_entry_hlist);
- mutex_unlock(&nacl->lun_entry_mutex);
-
spin_lock(&lun->lun_deve_lock);
list_add_tail(&new->lun_link, &lun->lun_deve_list);
spin_unlock(&lun->lun_deve_lock);
+ mutex_unlock(&nacl->lun_entry_mutex);
target_luns_data_has_changed(nacl, new, true);
return 0;
diff --git a/drivers/target/target_core_fabric_lib.c b/drivers/target/target_core_fabric_lib.c
index 2853b95b2c59..ad024009df56 100644
--- a/drivers/target/target_core_fabric_lib.c
+++ b/drivers/target/target_core_fabric_lib.c
@@ -371,6 +371,16 @@ static bool iscsi_parse_pr_out_transport_id(
pr_err("Missing ISID in iSCSI Initiator port TransportID\n");
return false;
}
+ /*
+ * The registration code reads the ISID as a C string,
+ * and its first 8 bytes with get_unaligned_be64(), so
+ * require 8 bytes after the separator and a NUL inside
+ * the rest of the descriptor.
+ */
+ if (buf + tid_len - p < 8 || !memchr(p, '\0', buf + tid_len - p)) {
+ pr_err("Incomplete or unterminated ISID in iSCSI Initiator port TransportID\n");
+ return false;
+ }
*port_nexus_ptr = p;
/*
* Go ahead and do the lower case conversion of the received
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index ccff9f6e4826..e615e5e30427 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -266,6 +266,15 @@ struct se_node_acl *core_tpg_check_initiator_node_acl(
kref_get(&acl->acl_kref);
acl->dynamic_node_acl = 1;
+ /*
+ * Publish the dynamic NodeACL on tpg->acl_node_list before creating
+ * its demo-mode MappedLUN dev entries below, so that a concurrent LUN
+ * teardown sweep (core_clear_lun_from_tpg(), which walks acl_node_list)
+ * can observe and disable those dev entries. Otherwise a dev entry can
+ * be left on an se_lun->lun_deve_list that is about to be freed.
+ */
+ target_add_node_acl(acl);
+
/*
* Here we only create demo-mode MappedLUNs from the active
* TPG LUNs if the fabric is not explicitly asking for
@@ -275,7 +284,6 @@ struct se_node_acl *core_tpg_check_initiator_node_acl(
(tpg->se_tpg_tfo->tpg_check_demo_mode_login_only(tpg) != 1))
core_tpg_add_node_to_devs(acl, tpg, NULL);
- target_add_node_acl(acl);
return acl;
}
EXPORT_SYMBOL(core_tpg_check_initiator_node_acl);
diff --git a/drivers/target/target_core_user.c b/drivers/target/target_core_user.c
index edc2afd5f4ee..839d5b2e7250 100644
--- a/drivers/target/target_core_user.c
+++ b/drivers/target/target_core_user.c
@@ -2524,6 +2524,11 @@ static int tcmu_set_data_pages_per_blk(struct tcmu_dev *udev, substring_t *arg)
return ret;
}
+ if (val <= 0) {
+ pr_err("Invalid data_pages_per_blk %d.\n", val);
+ return -EINVAL;
+ }
+
if (val > TCMU_MBS_TO_PAGES(udev->data_area_mb)) {
pr_err("Invalid data_pages_per_blk %d: greater than max_data_area_mb %d -> %zd pages).\n",
val, udev->data_area_mb,
diff --git a/drivers/thermal/intel/int340x_thermal/int3403_thermal.c b/drivers/thermal/intel/int340x_thermal/int3403_thermal.c
index 264c9bc8e645..2262de1e14d9 100644
--- a/drivers/thermal/intel/int340x_thermal/int3403_thermal.c
+++ b/drivers/thermal/intel/int340x_thermal/int3403_thermal.c
@@ -217,8 +217,7 @@ static int int3403_add(struct platform_device *pdev)
}
- status = acpi_evaluate_integer(priv->adev->handle, "_TMP",
- NULL, &tmp);
+ status = acpi_evaluate_integer_exclusive(priv->adev->handle, "_TMP", NULL, &tmp);
if (ACPI_FAILURE(status)) {
status = acpi_evaluate_integer(priv->adev->handle, "PTYP",
NULL, &priv->type);
diff --git a/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.c b/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.c
index 965700ee6962..9451fea1f65c 100644
--- a/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.c
+++ b/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.c
@@ -18,7 +18,11 @@ static int int340x_thermal_get_zone_temp(struct thermal_zone_device *zone,
unsigned long long tmp;
acpi_status status;
- status = acpi_evaluate_integer(d->adev->handle, "_TMP", NULL, &tmp);
+ /*
+ * On some ThinkPads _TMP switches the EC page selector without
+ * taking the mutex the battery methods use.
+ */
+ status = acpi_evaluate_integer_exclusive(d->adev->handle, "_TMP", NULL, &tmp);
if (ACPI_FAILURE(status))
return -EIO;
diff --git a/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c b/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c
index b0284c2e2e74..9bdb65f54b6a 100644
--- a/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c
+++ b/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c
@@ -287,7 +287,7 @@ int proc_thermal_add(struct device *dev, struct proc_thermal_device *proc_priv)
if (ret)
return ret;
- status = acpi_evaluate_integer(adev->handle, "_TMP", NULL, &tmp);
+ status = acpi_evaluate_integer_exclusive(adev->handle, "_TMP", NULL, &tmp);
if (ACPI_FAILURE(status)) {
/* there is no _TMP method, add local method */
if (intel_tcc_get_tjmax(-1) > 0)
diff --git a/drivers/vdpa/solidrun/snet_ctrl.c b/drivers/vdpa/solidrun/snet_ctrl.c
index e284c3a06717..09afa0487fd6 100644
--- a/drivers/vdpa/solidrun/snet_ctrl.c
+++ b/drivers/vdpa/solidrun/snet_ctrl.c
@@ -291,7 +291,6 @@ static int snet_send_ctrl_msg(struct snet *snet, u16 opcode, u16 vq_idx)
/* Check for errors */
val = snet_read_ctrl(regs);
- ret = SNET_VAL_TO_ERR(val);
/* Clear the chunk ready bit */
val &= ~SNET_CTRL_CHUNK_RDY_MASK;
@@ -301,6 +300,8 @@ static int snet_send_ctrl_msg(struct snet *snet, u16 opcode, u16 vq_idx)
if (ret)
SNET_WARN(pdev, "Timeout waiting for DPU to complete a control command, err %d\n",
ret);
+ else
+ ret = SNET_VAL_TO_ERR(val);
exit:
mutex_unlock(&snet->ctrl_lock);
diff --git a/drivers/vdpa/vdpa_user/iova_domain.c b/drivers/vdpa/vdpa_user/iova_domain.c
index b6c958224b7c..28a977248a0d 100644
--- a/drivers/vdpa/vdpa_user/iova_domain.c
+++ b/drivers/vdpa/vdpa_user/iova_domain.c
@@ -76,6 +76,10 @@ int vduse_domain_set_map(struct vduse_iova_domain *domain,
for (map = vhost_iotlb_itree_first(iotlb, start, last); map;
map = vhost_iotlb_itree_next(map, start, last)) {
map_file = (struct vdpa_map_file *)map->opaque;
+ if (!map_file || map_file->file == domain->file) {
+ ret = -EINVAL;
+ goto err;
+ }
ret = vduse_iotlb_add_range(domain, map->start, map->last,
map->addr, map->perm,
map_file->file,
@@ -226,6 +230,7 @@ vduse_domain_get_coherent_page(struct vduse_iova_domain *domain, u64 iova)
{
u64 start = iova & PAGE_MASK;
u64 last = start + PAGE_SIZE - 1;
+ struct vdpa_map_file *map_file;
struct vhost_iotlb_map *map;
struct page *page = NULL;
@@ -234,6 +239,10 @@ vduse_domain_get_coherent_page(struct vduse_iova_domain *domain, u64 iova)
if (!map)
goto out;
+ map_file = (struct vdpa_map_file *)map->opaque;
+ if (!map_file || map_file->file != domain->file)
+ goto out;
+
page = pfn_to_page((map->addr + iova - map->start) >> PAGE_SHIFT);
get_page(page);
out:
@@ -579,7 +588,7 @@ static int vduse_domain_mmap(struct file *file, struct vm_area_struct *vma)
{
struct vduse_iova_domain *domain = file->private_data;
- vm_flags_set(vma, VM_DONTDUMP | VM_DONTEXPAND);
+ vm_flags_set(vma, VM_IO | VM_DONTDUMP | VM_DONTEXPAND);
vma->vm_private_data = domain;
vma->vm_ops = &vduse_domain_mmap_ops;
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 4f8c0260bc9e..ddc9bea190de 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -937,7 +937,12 @@ vhost_scsi_copy_iov_to_sgl(struct vhost_scsi_cmd *cmd, struct iov_iter *iter,
}
for_each_sgtable_sg(sg_table, sg, i) {
- page = alloc_page(GFP_KERNEL);
+ /*
+ * Completion copies the full bounce buffer, including bytes the
+ * target did not write. This is already the slow fallback for
+ * misaligned I/O, so zeroing is acceptable here.
+ */
+ page = alloc_page(GFP_KERNEL | __GFP_ZERO);
if (!page) {
ret = -ENOMEM;
goto err;
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index a31786796d4c..c7a9702c2447 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -1151,7 +1151,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
unsigned long list_size = PAGE_SIZE / sizeof(struct page *);
unsigned int gup_flags = FOLL_LONGTERM;
unsigned long npages, cur_base, map_pfn, last_pfn = 0;
- unsigned long lock_limit, sz2pin, nchunks, i;
+ unsigned long lock_limit, sz2pin, i;
unsigned long page_offset;
u64 start = iova;
long pinned;
@@ -1187,7 +1187,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
cur_base = uaddr & PAGE_MASK;
iova &= PAGE_MASK;
- nchunks = 0;
while (npages) {
sz2pin = min_t(unsigned long, npages, list_size);
@@ -1202,7 +1201,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
}
goto out;
}
- nchunks++;
if (!last_pfn)
map_pfn = page_to_pfn(page_list[0]);
@@ -1233,7 +1231,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
map_pfn = this_pfn;
iova += csize;
- nchunks = 0;
}
last_pfn = this_pfn;
@@ -1248,7 +1245,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
PFN_PHYS(map_pfn), perm, NULL);
out:
if (ret) {
- if (nchunks) {
+ if (last_pfn) {
unsigned long pfn;
/*
@@ -1260,7 +1257,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
* the corresponding unpinning will be handled by
* vdpa_unmap().
*/
- WARN_ON(!last_pfn);
for (pfn = map_pfn; pfn <= last_pfn; pfn++)
unpin_user_page(pfn_to_page(pfn));
}
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index 44cac11b68d2..0226c4c59709 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -3172,8 +3172,7 @@ static int vhost_add_used_n_in_order(struct vhost_virtqueue *vq,
}
start += nheads[i];
new += nheads[i];
- if (start >= vq->num)
- start -= vq->num;
+ start &= vq->num - 1;
}
if (unlikely(vq->log_used)) {
diff --git a/drivers/vhost/vringh.c b/drivers/vhost/vringh.c
index 9066f9f12ba1..f027f669460f 100644
--- a/drivers/vhost/vringh.c
+++ b/drivers/vhost/vringh.c
@@ -197,8 +197,9 @@ static int move_to_indirect(const struct vringh *vrh,
}
len = vringh32_to_cpu(vrh, desc->len);
- if (unlikely(len % sizeof(struct vring_desc))) {
- vringh_bad("Strange indirect len %u", desc->len);
+ if (unlikely(len < sizeof(struct vring_desc) ||
+ len % sizeof(struct vring_desc))) {
+ vringh_bad("Invalid indirect len %u", desc->len);
return -EINVAL;
}
@@ -333,6 +334,17 @@ __vringh_iov(struct vringh *vrh, u16 i,
if (unlikely(err))
goto fail;
+ if (up_next == -1)
+ count++;
+ else
+ indirect_count++;
+
+ if (count > vrh->vring.num || indirect_count > desc_max) {
+ vringh_bad("Descriptor loop in %p", descs);
+ err = -ELOOP;
+ goto fail;
+ }
+
if (unlikely(desc.flags &
cpu_to_vringh16(vrh, VRING_DESC_F_INDIRECT))) {
u64 a = vringh64_to_cpu(vrh, desc.addr);
@@ -358,17 +370,6 @@ __vringh_iov(struct vringh *vrh, u16 i,
continue;
}
- if (up_next == -1)
- count++;
- else
- indirect_count++;
-
- if (count > vrh->vring.num || indirect_count > desc_max) {
- vringh_bad("Descriptor loop in %p", descs);
- err = -ELOOP;
- goto fail;
- }
-
if (desc.flags & cpu_to_vringh16(vrh, VRING_DESC_F_WRITE))
iov = wiov;
else {
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index abed1fbcf66c..35f75e23c749 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -364,7 +364,7 @@ static struct sk_buff *
vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,
unsigned int out, unsigned int in)
{
- struct virtio_vsock_hdr *hdr;
+ struct virtio_vsock_hdr hdr;
struct iov_iter iov_iter;
struct sk_buff *skb;
size_t payload_len;
@@ -382,34 +382,32 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,
len > VIRTIO_VSOCK_MAX_PKT_BUF_SIZE + VIRTIO_VSOCK_SKB_HEADROOM)
return NULL;
- /* len contains both payload and hdr */
- skb = virtio_vsock_alloc_skb(len, GFP_KERNEL);
- if (!skb)
- return NULL;
-
iov_iter_init(&iov_iter, ITER_SOURCE, vq->iov, out, len);
- hdr = virtio_vsock_hdr(skb);
- nbytes = copy_from_iter(hdr, sizeof(*hdr), &iov_iter);
- if (nbytes != sizeof(*hdr)) {
+ nbytes = copy_from_iter(&hdr, sizeof(hdr), &iov_iter);
+ if (nbytes != sizeof(hdr)) {
vq_err(vq, "Expected %zu bytes for pkt->hdr, got %zu bytes\n",
- sizeof(*hdr), nbytes);
- kfree_skb(skb);
+ sizeof(hdr), nbytes);
return NULL;
}
- payload_len = le32_to_cpu(hdr->len);
+ payload_len = le32_to_cpu(hdr.len);
+
+ /* The pkt is too big or the length in the header is invalid */
+ if (payload_len > len - sizeof(hdr))
+ return NULL;
+
+ /* Allocate only for the payload declared in the header. */
+ skb = virtio_vsock_alloc_skb(payload_len + sizeof(hdr), GFP_KERNEL);
+ if (!skb)
+ return NULL;
+
+ memcpy(virtio_vsock_hdr(skb), &hdr, sizeof(hdr));
/* No payload */
if (!payload_len)
return skb;
- /* The pkt is too big or the length in the header is invalid */
- if (payload_len + sizeof(*hdr) > len) {
- kfree_skb(skb);
- return NULL;
- }
-
virtio_vsock_skb_put(skb, payload_len);
if (skb_copy_datagram_from_iter(skb, 0, &iov_iter, payload_len)) {
diff --git a/fs/9p/v9fs.c b/fs/9p/v9fs.c
index 274c5157135d..b30ca5145fae 100644
--- a/fs/9p/v9fs.c
+++ b/fs/9p/v9fs.c
@@ -17,6 +17,7 @@
#include <linux/fs_context.h>
#include <linux/slab.h>
#include <linux/seq_file.h>
+#include <linux/sysfs.h>
#include <net/9p/9p.h>
#include <net/9p/client.h>
#include <net/9p/transport.h>
@@ -406,7 +407,7 @@ int v9fs_parse_param(struct fs_context *fc, struct fs_parameter *param)
break;
case Opt_privport:
fd_opts->privport = true;
- rdma_opts->port = true;
+ rdma_opts->privport = true;
break;
}
@@ -617,21 +618,14 @@ static ssize_t caches_show(struct kobject *kobj,
struct kobj_attribute *attr,
char *buf)
{
- ssize_t n = 0, count = 0, limit = PAGE_SIZE;
+ ssize_t count = 0;
struct v9fs_session_info *v9ses;
spin_lock(&v9fs_sessionlist_lock);
list_for_each_entry(v9ses, &v9fs_sessionlist, slist) {
- if (v9ses->cachetag) {
- n = snprintf(buf + count, limit, "%s\n", v9ses->cachetag);
- if (n < 0) {
- count = n;
- break;
- }
-
- count += n;
- limit -= n;
- }
+ if (v9ses->cachetag)
+ count += sysfs_emit_at(buf, count, "%s\n",
+ v9ses->cachetag);
}
spin_unlock(&v9fs_sessionlist_lock);
diff --git a/fs/9p/vfs_inode.c b/fs/9p/vfs_inode.c
index 3829554ca369..2d7e0c1a5353 100644
--- a/fs/9p/vfs_inode.c
+++ b/fs/9p/vfs_inode.c
@@ -495,6 +495,12 @@ static int v9fs_at_to_dotl_flags(int flags)
* nothing and races with concurrent metadata fetches that may already
* have observed the post-unlink value (nlink == 0).
*
+ * The same applies to a regular file whose cached nlink has already been
+ * driven to zero by the server: in the cached modes the attributes come
+ * from GETATTR and are applied verbatim, so decrementing a link count the
+ * server does not maintain only trips the drop_nlink() WARN_ON and
+ * underflows i_nlink.
+ *
* @inode: inode whose nlink is being dropped
*/
static void v9fs_dec_count(struct inode *inode)
@@ -503,6 +509,8 @@ static void v9fs_dec_count(struct inode *inode)
if (!(v9ses->cache & (CACHE_META | CACHE_LOOSE)))
return;
+ if (inode->i_nlink == 0)
+ return;
if (!S_ISDIR(inode->i_mode) || inode->i_nlink > 2)
drop_nlink(inode);
}
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 576cd76ca0bf..beae9dcb5a3e 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -375,6 +375,19 @@ int btrfs_fileattr_set(struct mnt_idmap *idmap,
}
}
+ /*
+ * Callers write back the whole flag set, so only touch the compression
+ * flags and the compression property if FS_COMPR_FL or FS_NOCOMP_FL
+ * changes.
+ */
+ if (!((fsflags ^ old_fsflags) & (FS_COMPR_FL | FS_NOCOMP_FL))) {
+ /* 1 item for the inode */
+ trans = btrfs_start_transaction(root, 1);
+ if (IS_ERR(trans))
+ return PTR_ERR(trans);
+ goto update_flags;
+ }
+
/*
* The COMPRESS flag can only be changed by users, while the NOCOMPRESS
* flag may be changed automatically if compression code won't make
@@ -2380,6 +2393,17 @@ static noinline int btrfs_ioctl_snap_destroy(struct file *file,
inode = d_inode(dentry);
dest = BTRFS_I(inode)->root;
+
+ /*
+ * A concurrent rename may have changed which subvolume the name
+ * refers to. Verify that the resolved ID still matches the requested ID.
+ */
+ if (destroy_v2 && (vol_args2->flags & BTRFS_SUBVOL_SPEC_BY_ID) &&
+ btrfs_root_id(dest) != vol_args2->subvolid) {
+ ret = -EAGAIN;
+ goto out_end_removing;
+ }
+
if (!capable(CAP_SYS_ADMIN)) {
/*
* Regular user. Only allow this with a special mount
diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 39cb01ee441a..71b1ec6c0643 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -1857,6 +1857,7 @@ void btrfs_sysfs_add_block_group_type(struct btrfs_block_group *cache)
if (space_info->block_group_kobjs[index]) {
spin_unlock(&space_info->lock);
kobject_put(&rkobj->kobj);
+ memalloc_nofs_restore(nofs_flag);
return;
} else {
space_info->block_group_kobjs[index] = &rkobj->kobj;
diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
index e07a7b1d3663..f32a533f4100 100644
--- a/fs/ceph/addr.c
+++ b/fs/ceph/addr.c
@@ -2220,6 +2220,12 @@ void ceph_fill_inline_data(struct inode *inode, struct page *locked_page,
doutc(cl, "%p %llx.%llx len %zu locked_page %p\n", inode,
ceph_vinop(inode), len, locked_page);
+ if (len > PAGE_SIZE) {
+ pr_warn_ratelimited_client(cl, "oversized inline data %zu\n",
+ len);
+ len = PAGE_SIZE;
+ }
+
if (len > 0) {
void *kaddr = kmap_atomic(page);
memcpy(kaddr, data, len);
diff --git a/fs/erofs/fileio.c b/fs/erofs/fileio.c
index ebb81a7ffd4c..ac846cfbc563 100644
--- a/fs/erofs/fileio.c
+++ b/fs/erofs/fileio.c
@@ -201,17 +201,29 @@ int erofs_read_meta_folio(struct file *file, struct folio *folio)
.dev = { .m_pa = folio_pos(folio), },
};
struct inode *inode = folio_inode(folio);
+ size_t len = folio_size(folio);
+ loff_t pos, isize;
int err;
err = erofs_map_dev(inode->i_sb, &io.dev);
if (err)
return err;
+ pos = io.dev.m_dif->fsoff + io.dev.m_pa;
+ isize = i_size_read(file_inode(io.dev.m_dif->file));
+ if (pos + folio_size(folio) > isize) {
+ len = isize > pos ? isize - pos : 0;
+ folio_zero_segment(folio, len, folio_size(folio));
+ if (!len) {
+ folio_end_read(folio, true);
+ return 0;
+ }
+ }
+
io.rq = erofs_fileio_rq_alloc(&io.dev);
- io.rq->bio.bi_iter.bi_sector =
- (io.dev.m_dif->fsoff + io.dev.m_pa) >> 9;
+ io.rq->bio.bi_iter.bi_sector = pos >> 9;
erofs_onlinefolio_init(folio);
- bio_add_folio_nofail(&io.rq->bio, folio, folio_size(folio), 0);
+ bio_add_folio_nofail(&io.rq->bio, folio, len, 0);
erofs_fileio_rq_submit(io.rq);
return 0;
}
diff --git a/fs/ext2/file.c b/fs/ext2/file.c
index b9020df7d89e..d4a0a528c342 100644
--- a/fs/ext2/file.c
+++ b/fs/ext2/file.c
@@ -138,7 +138,8 @@ static ssize_t ext2_dio_write_iter(struct kiocb *iocb, struct iov_iter *from)
pos = iocb->ki_pos;
status = generic_perform_write(iocb, from);
if (unlikely(status < 0)) {
- ret = status;
+ if (!ret)
+ ret = status;
goto out_unlock;
}
diff --git a/fs/ext2/inode.c b/fs/ext2/inode.c
index a9245f0cda4d..53edda7217f3 100644
--- a/fs/ext2/inode.c
+++ b/fs/ext2/inode.c
@@ -801,6 +801,12 @@ static int ext2_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
(first_block << blkbits) < i_size_read(inode))
create = 0;
+ ret = ext2_get_blocks(inode, first_block, max_blocks,
+ &bno, &new, &boundary, create);
+ if (ret < 0)
+ return ret;
+
+ iomap->flags = 0;
/*
* Writes that span EOF might trigger an IO size update on completion,
* so consider them to be dirty for the purposes of O_DSYNC even if
@@ -808,13 +814,6 @@ static int ext2_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
*/
if ((flags & IOMAP_WRITE) && offset + length > i_size_read(inode))
iomap->flags |= IOMAP_F_DIRTY;
-
- ret = ext2_get_blocks(inode, first_block, max_blocks,
- &bno, &new, &boundary, create);
- if (ret < 0)
- return ret;
-
- iomap->flags = 0;
iomap->offset = (u64)first_block << blkbits;
iomap->bdev = inode->i_sb->s_bdev;
diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c
index 4b59f30ef45d..a84517023e54 100644
--- a/fs/f2fs/checkpoint.c
+++ b/fs/f2fs/checkpoint.c
@@ -291,7 +291,7 @@ static struct folio *__get_meta_folio(struct f2fs_sb_info *sbi, pgoff_t index,
f2fs_update_iostat(sbi, NULL, FS_META_READ_IO, F2FS_BLKSIZE);
folio_lock(folio);
- if (unlikely(!is_meta_folio(folio))) {
+ if (unlikely(!is_meta_folio(sbi, folio))) {
f2fs_folio_put(folio, true);
goto repeat;
}
@@ -610,7 +610,7 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, long nr_to_write,
enum iostat_type io_type)
{
struct address_space *mapping = META_MAPPING(sbi);
- pgoff_t index = 0, prev = ULONG_MAX;
+ pgoff_t index = 0, next = ULONG_MAX;
struct folio_batch fbatch;
long nwritten = 0;
int nr_folios;
@@ -629,16 +629,15 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, long nr_to_write,
for (i = 0; i < nr_folios; i++) {
struct folio *folio = fbatch.folios[i];
- if (nr_to_write != LONG_MAX && i != 0 &&
- folio->index != prev +
- folio_nr_pages(fbatch.folios[i-1])) {
+ if (nr_to_write != LONG_MAX && next != ULONG_MAX &&
+ folio->index != next) {
folio_batch_release(&fbatch);
goto stop;
}
folio_lock(folio);
- if (unlikely(!is_meta_folio(folio))) {
+ if (unlikely(!is_meta_folio(sbi, folio))) {
continue_unlock:
folio_unlock(folio);
continue;
@@ -659,7 +658,7 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, long nr_to_write,
break;
}
nwritten += folio_nr_pages(folio);
- prev = folio->index;
+ next = folio->index + folio_nr_pages(folio);
if (unlikely(nwritten >= nr_to_write))
break;
}
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index cfd81bfc28a2..e0abc159b796 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -172,7 +172,8 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task)
dec_page_count(F2FS_F_SB(folio), __read_io_type(folio));
if (bio->bi_status == BLK_STS_OK &&
- F2FS_F_SB(folio)->node_inode && is_node_folio(folio) &&
+ F2FS_F_SB(folio)->node_inode &&
+ is_node_folio(F2FS_F_SB(folio), folio) &&
f2fs_sanity_check_node_footer(F2FS_F_SB(folio),
folio, folio->index, NODE_TYPE_REGULAR, true))
bio->bi_status = BLK_STS_IOERR;
@@ -359,12 +360,12 @@ static void f2fs_write_end_bio(struct bio *bio)
}
}
- if (is_node_folio(folio)) {
+ if (is_node_folio(sbi, folio)) {
f2fs_sanity_check_node_footer(sbi, folio,
folio->index, NODE_TYPE_REGULAR, true);
f2fs_bug_on(sbi, folio->index != nid_of_node(folio));
}
- if (f2fs_in_warm_node_list(folio))
+ if (f2fs_in_warm_node_list(sbi, folio))
f2fs_del_fsync_node_entry(sbi, folio);
dec_page_count(sbi, type);
@@ -399,7 +400,8 @@ static void f2fs_write_end_io(struct bio *bio)
sbi = bio->bi_private;
- if (in_atomic() && bio->bi_iter.bi_size > sbi->max_atc_write_bio_size) {
+ if (bio_in_atomic() &&
+ bio->bi_iter.bi_size > sbi->max_atc_write_bio_size) {
struct work_struct *w;
w = &container_of(bio, struct f2fs_bio, bio)->work;
@@ -500,6 +502,10 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages)
struct bio *bio;
bdev = f2fs_target_device(sbi, fio->new_blkaddr, §or);
+ /* avoid reversed IO if bio is split due to exceeding max_segments */
+ if (npages > 1 && !is_read_io(fio->op) &&
+ f2fs_is_sequential_zone_area(sbi, fio->new_blkaddr))
+ npages = min_t(int, npages, bdev_max_segments(bdev));
bio = bio_alloc_bioset(bdev, npages,
fio->op | fio->op_flags | f2fs_io_flags(fio),
GFP_NOIO, &f2fs_bioset);
@@ -790,7 +796,19 @@ static bool page_is_mergeable(struct f2fs_sb_info *sbi, struct bio *bio,
return false;
if (last_blkaddr + 1 != cur_blkaddr)
return false;
- return bio->bi_bdev == f2fs_target_device(sbi, cur_blkaddr, NULL);
+ if (bio->bi_bdev != f2fs_target_device(sbi, cur_blkaddr, NULL))
+ return false;
+ /* avoid reversed IO if bio is split due to exceeding max_segments/sectors */
+ if (f2fs_is_sequential_zone_area(sbi, cur_blkaddr)) {
+ struct request_queue *q = bdev_get_queue(bio->bi_bdev);
+
+ if (bio->bi_vcnt >= queue_max_segments(q))
+ return false;
+ if (bio_sectors(bio) + (F2FS_BLKSIZE >> SECTOR_SHIFT) >
+ queue_max_sectors(q))
+ return false;
+ }
+ return true;
}
static bool io_type_is_mergeable(struct f2fs_bio_info *io,
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 0c85d90307f7..f0024647153d 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -2316,14 +2316,14 @@ static inline struct address_space *NODE_MAPPING(struct f2fs_sb_info *sbi)
return sbi->node_inode->i_mapping;
}
-static inline bool is_meta_folio(struct folio *folio)
+static inline bool is_meta_folio(struct f2fs_sb_info *sbi, struct folio *folio)
{
- return folio->mapping == META_MAPPING(F2FS_F_SB(folio));
+ return folio->mapping == META_MAPPING(sbi);
}
-static inline bool is_node_folio(struct folio *folio)
+static inline bool is_node_folio(struct f2fs_sb_info *sbi, struct folio *folio)
{
- return folio->mapping == NODE_MAPPING(F2FS_F_SB(folio));
+ return folio->mapping == NODE_MAPPING(sbi);
}
static inline bool is_sbi_flag_set(struct f2fs_sb_info *sbi, unsigned int type)
@@ -3994,7 +3994,7 @@ enum node_type;
int f2fs_check_nid_range(struct f2fs_sb_info *sbi, nid_t nid);
bool f2fs_available_free_memory(struct f2fs_sb_info *sbi, int type);
-bool f2fs_in_warm_node_list(struct folio *folio);
+bool f2fs_in_warm_node_list(struct f2fs_sb_info *sbi, struct folio *folio);
void f2fs_init_fsync_node_info(struct f2fs_sb_info *sbi);
void f2fs_del_fsync_node_entry(struct f2fs_sb_info *sbi, struct folio *folio);
void f2fs_reset_fsync_node_info(struct f2fs_sb_info *sbi);
diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
index 0a00180c21dc..6220d55709e9 100644
--- a/fs/f2fs/gc.c
+++ b/fs/f2fs/gc.c
@@ -1429,7 +1429,7 @@ static int move_data_block(struct inode *inode, block_t bidx,
F2FS_BLKSIZE);
folio_lock(mfolio);
- if (unlikely(!is_meta_folio(mfolio) ||
+ if (unlikely(!is_meta_folio(fio.sbi, mfolio) ||
!folio_test_uptodate(mfolio))) {
err = -EIO;
f2fs_folio_put(mfolio, true);
@@ -1857,6 +1857,9 @@ static int do_garbage_collect(struct f2fs_sb_info *sbi,
if (block_end_segno > end_segno)
block_end_segno = end_segno;
+ if (unlikely(freezing(current)))
+ goto next_block;
+
if (is_cursec(sbi, GET_SEC_FROM_SEG(sbi, segno))) {
f2fs_err(sbi, "%s: segment %u is used by log",
__func__, segno);
@@ -1924,18 +1927,12 @@ static int do_garbage_collect(struct f2fs_sb_info *sbi,
sbi->next_victim_seg[gc_type] =
(cur_segno + 1 < sec_end_segno) ?
cur_segno + 1 : NULL_SEGNO;
-
- if (unlikely(freezing(current))) {
- folio_put_refs(sum_folio, 2);
- goto stop;
- }
}
next_block:
folio_put_refs(sum_folio, 2);
segno = block_end_segno;
}
-stop:
if (submitted)
f2fs_submit_merged_write(sbi, data_type);
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index 86c2e67e43b6..e166c5a5d170 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -330,9 +330,9 @@ static unsigned int __gang_lookup_nat_set(struct f2fs_nm_info *nm_i,
start, nr);
}
-bool f2fs_in_warm_node_list(struct folio *folio)
+bool f2fs_in_warm_node_list(struct f2fs_sb_info *sbi, struct folio *folio)
{
- return is_node_folio(folio) && IS_DNODE(folio) && is_cold_node(folio);
+ return is_node_folio(sbi, folio) && IS_DNODE(folio) && is_cold_node(folio);
}
void f2fs_init_fsync_node_info(struct f2fs_sb_info *sbi)
@@ -835,8 +835,20 @@ int f2fs_get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode)
if (!dn->inode_folio) {
nfolio[0] = f2fs_get_inode_folio(sbi, nids[0]);
- if (IS_ERR(nfolio[0]))
- return PTR_ERR(nfolio[0]);
+ if (IS_ERR(nfolio[0])) {
+ err = PTR_ERR(nfolio[0]);
+ /*
+ * cp_error drops dirty node caches, so the inode block
+ * of an inode that was never written reads as NEW_ADDR
+ * and returns -ENOENT. It is not a hole; the inode is
+ * gone. Callers treat -ENOENT as a hole and keep
+ * looking block by block (e.g. fiemap up to maxbytes).
+ */
+ if (err == -ENOENT && (f2fs_cp_error(sbi) ||
+ is_sbi_flag_set(sbi, SBI_IS_SHUTDOWN)))
+ err = -EIO;
+ return err;
+ }
} else {
nfolio[0] = dn->inode_folio;
}
@@ -1290,7 +1302,7 @@ int f2fs_truncate_inode_blocks(struct inode *inode, pgoff_t from)
goto fail;
if (offset[1] == 0 && get_nid(folio, offset[0], true)) {
folio_lock(folio);
- BUG_ON(!is_node_folio(folio));
+ BUG_ON(!is_node_folio(sbi, folio));
set_nid(folio, offset[0], 0, true);
folio_unlock(folio);
}
@@ -1605,7 +1617,7 @@ static struct folio *__get_node_folio(struct f2fs_sb_info *sbi, pgoff_t nid,
folio_lock(folio);
- if (unlikely(!is_node_folio(folio))) {
+ if (unlikely(!is_node_folio(sbi, folio))) {
f2fs_folio_put(folio, true);
goto repeat;
}
@@ -1724,7 +1736,7 @@ static struct folio *last_fsync_dnode(struct f2fs_sb_info *sbi, nid_t ino)
folio_lock(folio);
- if (unlikely(!is_node_folio(folio))) {
+ if (unlikely(!is_node_folio(sbi, folio))) {
continue_unlock:
folio_unlock(folio);
continue;
@@ -1834,7 +1846,7 @@ static bool __write_node_folio(struct folio *folio, bool atomic, bool do_fsync,
f2fs_need_dentry_mark(sbi, ino_of_node(folio)));
/* should add to global list before clearing PAGECACHE status */
- if (f2fs_in_warm_node_list(folio)) {
+ if (f2fs_in_warm_node_list(sbi, folio)) {
seq = f2fs_add_fsync_node_entry(sbi, folio);
if (seq_id)
*seq_id = seq;
@@ -1958,7 +1970,7 @@ int f2fs_fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode,
folio_lock(folio);
- if (unlikely(!is_node_folio(folio))) {
+ if (unlikely(!is_node_folio(sbi, folio))) {
continue_unlock:
folio_unlock(folio);
continue;
@@ -2016,7 +2028,7 @@ int f2fs_fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode,
f2fs_debug(sbi, "Retry to write fsync mark: ino=%u, idx=%lx",
ino, last_folio->index);
folio_lock(last_folio);
- if (unlikely(!is_node_folio(last_folio))) {
+ if (unlikely(!is_node_folio(sbi, last_folio))) {
f2fs_folio_put(last_folio, true);
ret = -EAGAIN;
goto out;
@@ -2094,7 +2106,7 @@ void f2fs_flush_inline_data(struct f2fs_sb_info *sbi)
folio_lock(folio);
- if (unlikely(!is_node_folio(folio)))
+ if (unlikely(!is_node_folio(sbi, folio)))
goto unlock;
if (!folio_test_dirty(folio))
goto unlock;
@@ -2166,7 +2178,7 @@ int f2fs_sync_node_pages(struct f2fs_sb_info *sbi,
else if (!folio_trylock(folio))
continue;
- if (unlikely(!is_node_folio(folio))) {
+ if (unlikely(!is_node_folio(sbi, folio))) {
continue_unlock:
folio_unlock(folio);
continue;
@@ -2871,6 +2883,45 @@ int f2fs_recover_inline_xattr(struct inode *inode, struct folio *folio)
f2fs_folio_wait_writeback(ifolio, NODE, true, true);
memcpy(dst_addr, src_addr, inline_size);
update_inode:
+ /*
+ * The xattr node of the checkpoint was removed before fsync, e.g. the
+ * xattrs became small enough to fit inline. Release it as well, or
+ * the inode keeps pointing to the stale xattrs.
+ */
+ if (!ri->i_xattr_nid && F2FS_I(inode)->i_xattr_nid) {
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ struct node_info ni;
+ int err;
+
+ err = f2fs_get_node_info(sbi, F2FS_I(inode)->i_xattr_nid,
+ &ni, false);
+ if (err) {
+ f2fs_folio_put(ifolio, true);
+ return err;
+ }
+
+ if (ni.blk_addr != NEW_ADDR &&
+ !f2fs_is_valid_blkaddr(sbi, ni.blk_addr,
+ DATA_GENERIC_ENHANCE)) {
+ f2fs_err_ratelimited(sbi,
+ "nat entry is corrupted, run fsck to fix it, ino:%u, nid:%u, blkaddr:%u",
+ ni.ino, ni.nid, ni.blk_addr);
+ set_sbi_flag(sbi, SBI_NEED_FSCK);
+ f2fs_handle_error(sbi, ERROR_INCONSISTENT_NAT);
+ f2fs_folio_put(ifolio, true);
+ return -EFSCORRUPTED;
+ }
+
+ f2fs_invalidate_blocks(sbi, ni.blk_addr, 1);
+ dec_valid_node_count(sbi, inode, false);
+ set_node_addr(sbi, &ni, NULL_ADDR, false);
+ /* drop the cached xattr node, which may be dirty */
+ truncate_inode_pages_range(NODE_MAPPING(sbi),
+ F2FS_BLK_TO_BYTES((loff_t)ni.nid),
+ F2FS_BLK_END_BYTES((loff_t)ni.nid));
+ f2fs_i_xnid_write(inode, 0);
+ }
+
f2fs_update_inode(inode, ifolio);
f2fs_folio_put(ifolio, true);
return 0;
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index f3a73b6c24c3..75f78ed7eb90 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -3234,6 +3234,7 @@ static int change_curseg(struct f2fs_sb_info *sbi, int type)
if (IS_ERR(sum_folio)) {
/* GC won't be able to use stale summary pages by cp_error */
memset(curseg->sum_blk, 0, sbi->sum_entry_size);
+ reset_curseg_fields(curseg);
return PTR_ERR(sum_folio);
}
sum_node = SUM_BLK_PAGE_ADDR(sbi, sum_folio, new_segno);
@@ -4158,7 +4159,7 @@ static void do_write_page(struct f2fs_summary *sum, struct f2fs_io_info *fio)
__func__, fio->ino, folio->index, type,
fio->old_blkaddr, fio->new_blkaddr, err);
folio_end_writeback(folio);
- if (f2fs_in_warm_node_list(folio))
+ if (f2fs_in_warm_node_list(fio->sbi, folio))
f2fs_del_fsync_node_entry(fio->sbi, folio);
f2fs_bug_on(fio->sbi, !is_set_ckpt_flags(fio->sbi,
CP_ERROR_FLAG));
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index b9adad85e18c..b18ea09bcbe2 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -4255,6 +4255,7 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi)
block_t avail_node_count, valid_node_count;
unsigned int nat_blocks, nat_bits_bytes, nat_bits_blocks;
unsigned int sit_blk_cnt;
+ unsigned long long ckpt_size, sit_end, nat_end;
int i, j;
total = le32_to_cpu(raw_super->segment_count);
@@ -4386,6 +4387,29 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi)
return 1;
}
+ /*
+ * Nothing above ties the bitmap lengths to sbi->ckpt, which is
+ * (1 + cp_payload) blocks. Bound both ends the way __bitmap_ptr()
+ * lays them out.
+ */
+ ckpt_size = (unsigned long long)(cp_payload + 1) * F2FS_BLKSIZE;
+ if (__is_set_ckpt_flags(ckpt, CP_LARGE_NAT_BITMAP_FLAG)) {
+ nat_end = CP_MIN_CHKSUM_OFFSET + sizeof(__le32) + nat_bitmap_size;
+ sit_end = nat_end + sit_bitmap_size;
+ } else if (cp_payload > 0) {
+ nat_end = CP_MIN_CHKSUM_OFFSET + nat_bitmap_size;
+ sit_end = F2FS_BLKSIZE + sit_bitmap_size;
+ } else {
+ sit_end = CP_MIN_CHKSUM_OFFSET + sit_bitmap_size;
+ nat_end = sit_end + nat_bitmap_size;
+ }
+
+ if (sit_end > ckpt_size || nat_end > ckpt_size) {
+ f2fs_err(sbi, "Bitmaps do not fit the checkpoint pack: sit ends at %llu, nat at %llu, pack is %llu bytes",
+ sit_end, nat_end, ckpt_size);
+ return 1;
+ }
+
if (__is_set_ckpt_flags(ckpt, CP_LARGE_NAT_BITMAP_FLAG) &&
le32_to_cpu(ckpt->checksum_offset) != CP_MIN_CHKSUM_OFFSET) {
f2fs_warn(sbi, "using deprecated layout of large_nat_bitmap, "
diff --git a/fs/f2fs/sysfs.c b/fs/f2fs/sysfs.c
index 811e350a1430..5ee12547171c 100644
--- a/fs/f2fs/sysfs.c
+++ b/fs/f2fs/sysfs.c
@@ -1404,6 +1404,7 @@ F2FS_FEATURE_RO_ATTR(linear_lookup);
#endif
F2FS_FEATURE_RO_ATTR(packed_ssa);
F2FS_FEATURE_RO_ATTR(fserror);
+F2FS_FEATURE_RO_ATTR(resizable_tail);
#define ATTR_LIST(name) (&f2fs_attr_##name.attr)
static struct attribute *f2fs_attrs[] = {
@@ -1574,6 +1575,7 @@ static struct attribute *f2fs_feat_attrs[] = {
#endif
BASE_ATTR_LIST(packed_ssa),
BASE_ATTR_LIST(fserror),
+ BASE_ATTR_LIST(resizable_tail),
NULL,
};
ATTRIBUTE_GROUPS(f2fs_feat);
diff --git a/fs/fat/inode.c b/fs/fat/inode.c
index f775a004cae1..065658642e7f 100644
--- a/fs/fat/inode.c
+++ b/fs/fat/inode.c
@@ -216,7 +216,9 @@ static void fat_write_failed(struct address_space *mapping, loff_t to)
if (to > inode->i_size) {
truncate_pagecache(inode, inode->i_size);
+ down_write(&MSDOS_I(inode)->truncate_lock);
fat_truncate_blocks(inode, inode->i_size);
+ up_write(&MSDOS_I(inode)->truncate_lock);
}
}
diff --git a/fs/nfsd/nfs2acl.c b/fs/nfsd/nfs2acl.c
index 190f5a001900..113ec827ae06 100644
--- a/fs/nfsd/nfs2acl.c
+++ b/fs/nfsd/nfs2acl.c
@@ -15,6 +15,62 @@
#define NFSDDBG_FACILITY NFSDDBG_PROC
+/*
+ * These maps are identical to the NFSv3 maps (nfs3proc.c). This enables
+ * the behavior of the two versions to diverge if needed.
+ */
+static const struct nfsd_access_map nfsd2_regaccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_map nfsd2_diraccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_LOOKUP, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE },
+ { NFS3_ACCESS_DELETE, NFSD_MAY_REMOVE },
+ { 0, 0 }
+};
+
+/*
+ * Some clients - Solaris 2.6 at least, make an access call to the NFS
+ * server to check for access for things like /dev/null (which really,
+ * NFSD doesn't care about). So NFSD provides simple access checking
+ * for those objects, looking mainly at mode bits, ignoring read-only
+ * filesystem checks.
+ */
+static const struct nfsd_access_map nfsd2_otheraccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_maps nfsd2_access_maps = {
+ .regular = nfsd2_regaccess,
+ .directory = nfsd2_diraccess,
+ .other = nfsd2_otheraccess,
+};
+
+static __be32 nfsacld_map_status(__be32 status)
+{
+ switch (status) {
+ case nfserr_nofilehandle:
+ case nfserr_badhandle:
+ status = nfserr_stale;
+ break;
+ case nfserr_wrongsec:
+ status = nfserr_acces;
+ break;
+ }
+ return status;
+}
+
/*
* NULL call.
*/
@@ -79,6 +135,7 @@ static __be32 nfsacld_proc_getacl(struct svc_rqst *rqstp)
/* resp->acl_{access,default} are released in nfssvc_release_getacl. */
out:
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
fail:
@@ -137,6 +194,7 @@ static __be32 nfsacld_proc_setacl(struct svc_rqst *rqstp)
out:
/* argp->acl_{access,default} are released in nfsaclsvc_release_setacl. */
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
out_drop_lock:
@@ -163,6 +221,7 @@ static __be32 nfsacld_proc_getattr(struct svc_rqst *rqstp)
goto out;
resp->status = fh_getattr(&resp->fh, &resp->stat);
out:
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
}
@@ -180,11 +239,14 @@ static __be32 nfsacld_proc_access(struct svc_rqst *rqstp)
fh_copy(&resp->fh, &argp->fh);
resp->access = argp->access;
- resp->status = nfsd_access(rqstp, &resp->fh, &resp->access, NULL);
+
+ resp->status = nfsd_access(rqstp, &resp->fh, &nfsd2_access_maps,
+ &resp->access, NULL);
if (resp->status != nfs_ok)
goto out;
resp->status = fh_getattr(&resp->fh, &resp->stat);
out:
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
}
diff --git a/fs/nfsd/nfs3proc.c b/fs/nfsd/nfs3proc.c
index 0904d953d10e..b3c634190701 100644
--- a/fs/nfsd/nfs3proc.c
+++ b/fs/nfsd/nfs3proc.c
@@ -48,6 +48,58 @@ static bool nfsd3_time_in_range(const struct iattr *iap)
return true;
}
+static const struct nfsd_access_map nfsd3_regaccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_map nfsd3_diraccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_LOOKUP, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE },
+ { NFS3_ACCESS_DELETE, NFSD_MAY_REMOVE },
+ { 0, 0 }
+};
+
+/*
+ * Some clients - Solaris 2.6 at least, make an access call to the NFS
+ * server to check for access for things like /dev/null (which really,
+ * NFSD doesn't care about). So NFSD provides simple access checking
+ * for those objects, looking mainly at mode bits, ignoring read-only
+ * filesystem checks.
+ */
+static const struct nfsd_access_map nfsd3_otheraccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_maps nfsd3_access_maps = {
+ .regular = nfsd3_regaccess,
+ .directory = nfsd3_diraccess,
+ .other = nfsd3_otheraccess,
+};
+
+static int nfsd3_iocb_flags(enum nfs3_stable_how how)
+{
+ switch (how) {
+ case NFS_FILE_SYNC:
+ /* persist data and timestamps */
+ return IOCB_DSYNC | IOCB_SYNC;
+ case NFS_DATA_SYNC:
+ /* persist data only */
+ return IOCB_DSYNC;
+ default:
+ return 0;
+ }
+}
+
static __be32 nfsd3_map_status(__be32 status)
{
switch (status) {
@@ -171,7 +223,8 @@ nfsd3_proc_access(struct svc_rqst *rqstp)
fh_copy(&resp->fh, &argp->fh);
resp->access = argp->access;
- resp->status = nfsd_access(rqstp, &resp->fh, &resp->access, NULL);
+ resp->status = nfsd_access(rqstp, &resp->fh, &nfsd3_access_maps,
+ &resp->access, NULL);
resp->status = nfsd3_map_status(resp->status);
return rpc_success;
}
@@ -260,7 +313,8 @@ nfsd3_proc_write(struct svc_rqst *rqstp)
resp->committed = argp->stable;
resp->status = nfsd_write(rqstp, &resp->fh, argp->offset,
&argp->payload, &cnt,
- resp->committed, resp->verf);
+ nfsd3_iocb_flags(resp->committed),
+ resp->verf);
resp->count = cnt;
resp->status = nfsd3_map_status(resp->status);
return rpc_success;
diff --git a/fs/nfsd/nfs3xdr.c b/fs/nfsd/nfs3xdr.c
index e481804bb120..f418cab0bd61 100644
--- a/fs/nfsd/nfs3xdr.c
+++ b/fs/nfsd/nfs3xdr.c
@@ -556,6 +556,8 @@ nfs3svc_decode_writeargs(struct svc_rqst *rqstp, struct xdr_stream *xdr)
return false;
if (xdr_stream_decode_u32(xdr, &args->stable) < 0)
return false;
+ if (args->stable > NFS_FILE_SYNC)
+ return false;
/* opaque data */
if (xdr_stream_decode_u32(xdr, &args->len) < 0)
diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index f3f7f14a9368..f19d8ed80a9d 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -69,6 +69,57 @@ MODULE_PARM_DESC(nfsd4_ssc_umount_timeout,
#define NFSDDBG_FACILITY NFSDDBG_PROC
+static const struct nfsd_access_map nfsd4_regaccess[] = {
+ { NFS4_ACCESS_READ, NFSD_MAY_READ },
+ { NFS4_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS4_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS4_ACCESS_EXTEND, NFSD_MAY_WRITE },
+ { NFS4_ACCESS_XAREAD, NFSD_MAY_READ },
+ { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE },
+ { NFS4_ACCESS_XALIST, NFSD_MAY_READ },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_map nfsd4_diraccess[] = {
+ { NFS4_ACCESS_READ, NFSD_MAY_READ },
+ { NFS4_ACCESS_LOOKUP, NFSD_MAY_EXEC },
+ { NFS4_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS4_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE },
+ { NFS4_ACCESS_DELETE, NFSD_MAY_REMOVE },
+ { NFS4_ACCESS_XAREAD, NFSD_MAY_READ },
+ { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE },
+ { NFS4_ACCESS_XALIST, NFSD_MAY_READ },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_map nfsd4_otheraccess[] = {
+ { NFS4_ACCESS_READ, NFSD_MAY_READ },
+ { NFS4_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS4_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { NFS4_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_maps nfsd4_access_maps = {
+ .regular = nfsd4_regaccess,
+ .directory = nfsd4_diraccess,
+ .other = nfsd4_otheraccess,
+};
+
+static int nfsd4_iocb_flags(enum stable_how4 how)
+{
+ switch (how) {
+ case FILE_SYNC4:
+ /* persist data and timestamps */
+ return IOCB_DSYNC | IOCB_SYNC;
+ case DATA_SYNC4:
+ /* persist data only */
+ return IOCB_DSYNC;
+ default:
+ return 0;
+ }
+}
+
static u32 nfsd_attrmask[] = {
NFSD_WRITEABLE_ATTRS_WORD0,
NFSD_WRITEABLE_ATTRS_WORD1,
@@ -795,10 +846,9 @@ nfsd4_access(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
if (access->ac_req_access & ~access_full)
return nfserr_inval;
-
access->ac_resp_access = access->ac_req_access;
- return nfsd_access(rqstp, &cstate->current_fh, &access->ac_resp_access,
- &access->ac_supported);
+ return nfsd_access(rqstp, &cstate->current_fh, &nfsd4_access_maps,
+ &access->ac_resp_access, &access->ac_supported);
}
static __be32
@@ -1377,7 +1427,7 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
write->wr_how_written = write->wr_stable_how;
status = nfsd_vfs_write(rqstp, &cstate->current_fh, nf,
write->wr_offset, &write->wr_payload,
- &cnt, write->wr_how_written,
+ &cnt, nfsd4_iocb_flags(write->wr_how_written),
(__be32 *)write->wr_verifier.data);
nfsd_file_put(nf);
@@ -1974,7 +2024,7 @@ static void nfsd4_init_copy_res(struct nfsd4_copy *copy, bool sync)
{
copy->cp_res.wr_stable_how =
test_bit(NFSD4_COPY_F_COMMITTED, ©->cp_flags) ?
- NFS_FILE_SYNC : NFS_UNSTABLE;
+ FILE_SYNC4 : UNSTABLE4;
nfsd4_copy_set_sync(copy, sync);
}
diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
index 6441e54283a3..907a914b04cd 100644
--- a/fs/nfsd/nfs4recover.c
+++ b/fs/nfsd/nfs4recover.c
@@ -649,6 +649,7 @@ struct cld_upcall {
struct completion cu_done;
/* daemon has read the whole upcall; protected by cn_lock */
bool cu_inflight;
+ struct rpc_pipe_msg cu_pipe_msg;
union {
struct cld_msg_hdr cu_hdr;
struct cld_msg cu_msg;
@@ -660,22 +661,22 @@ static int
__cld_pipe_upcall(struct rpc_pipe *pipe, void *cmsg, struct nfsd_net *nn)
{
int ret;
- struct rpc_pipe_msg msg;
struct cld_upcall *cup = container_of(cmsg, struct cld_upcall, cu_u);
+ struct rpc_pipe_msg *msg = &cup->cu_pipe_msg;
- memset(&msg, 0, sizeof(msg));
- msg.data = cmsg;
- msg.len = nn->client_tracking_ops->msglen;
+ memset(msg, 0, sizeof(*msg));
+ msg->data = cmsg;
+ msg->len = nn->client_tracking_ops->msglen;
- ret = rpc_queue_upcall(pipe, &msg);
+ ret = rpc_queue_upcall(pipe, msg);
if (ret < 0) {
goto out;
}
wait_for_completion(&cup->cu_done);
- if (msg.errno < 0)
- ret = msg.errno;
+ if (msg->errno < 0)
+ ret = msg->errno;
out:
return ret;
}
@@ -838,8 +839,11 @@ cld_pipe_downcall(struct file *filp, const char __user *src, size_t mlen)
if (status == -EINPROGRESS)
return __cld_pipe_inprogress_downcall(cmsg, nn);
- if (copy_from_user(&cup->cu_u.cu_msg_v2, src, mlen) != 0)
+ if (copy_from_user(&cup->cu_u.cu_msg_v2, src, mlen) != 0) {
+ cup->cu_pipe_msg.errno = -EFAULT;
+ complete(&cup->cu_done);
return -EFAULT;
+ }
complete(&cup->cu_done);
return mlen;
@@ -848,9 +852,8 @@ cld_pipe_downcall(struct file *filp, const char __user *src, size_t mlen)
static void
cld_pipe_destroy_msg(struct rpc_pipe_msg *msg)
{
- struct cld_msg *cmsg = msg->data;
- struct cld_upcall *cup = container_of(cmsg, struct cld_upcall,
- cu_u.cu_msg);
+ struct cld_upcall *cup = container_of(msg, struct cld_upcall,
+ cu_pipe_msg);
struct cld_net *cn = cup->cu_net;
/*
@@ -867,10 +870,34 @@ cld_pipe_destroy_msg(struct rpc_pipe_msg *msg)
complete(&cup->cu_done);
}
+/*
+ * An upcall the daemon has consumed is off every pipe list, so the
+ * purge in rpc_pipe_release() and rpc_close_pipes() cannot reach it.
+ * Release its waiter here instead; no downcall can arrive now.
+ */
+static void
+cld_release_pipe(struct inode *inode)
+{
+ struct nfsd_net *nn = net_generic(inode->i_sb->s_fs_info, nfsd_net_id);
+ struct cld_net *cn = nn->cld_net;
+ struct cld_upcall *cup;
+
+ spin_lock(&cn->cn_lock);
+ list_for_each_entry(cup, &cn->cn_list, cu_list) {
+ if (!cup->cu_inflight)
+ continue;
+ cup->cu_inflight = false;
+ cup->cu_pipe_msg.errno = -EPIPE;
+ complete(&cup->cu_done);
+ }
+ spin_unlock(&cn->cn_lock);
+}
+
static const struct rpc_pipe_ops cld_upcall_ops = {
.upcall = rpc_pipe_generic_upcall,
.downcall = cld_pipe_downcall,
.destroy_msg = cld_pipe_destroy_msg,
+ .release_pipe = cld_release_pipe,
};
static int
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 3bdde4c1bb8e..f0a795940137 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1517,6 +1517,7 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp)
dp->dl_stid.sc_type = SC_TYPE_DELEG;
list_add(&dp->dl_perfile, &fp->fi_delegations);
list_add(&dp->dl_perclnt, &clp->cl_delegations);
+ clp->cl_deleg_count++;
return 0;
}
@@ -1548,6 +1549,7 @@ unhash_delegation_locked(struct nfs4_delegation *dp, unsigned short statusmask)
++dp->dl_time;
spin_lock(&fp->fi_lock);
list_del_init(&dp->dl_perclnt);
+ dp->dl_stid.sc_client->cl_deleg_count--;
list_del_init(&dp->dl_recall_lru);
list_del_init(&dp->dl_perfile);
spin_unlock(&fp->fi_lock);
@@ -7938,6 +7940,7 @@ deleg_reaper(struct nfsd_net *nn)
{
struct list_head *pos, *next;
struct nfs4_client *clp;
+ unsigned int count;
spin_lock(&nn->client_lock);
list_for_each_safe(pos, next, &nn->client_lru) {
@@ -7947,23 +7950,36 @@ deleg_reaper(struct nfsd_net *nn)
continue;
if (clp->cl_state != NFSD4_ACTIVE)
continue;
- if (list_empty(&clp->cl_delegations))
- continue;
if (atomic_read(&clp->cl_delegs_in_recall))
continue;
if (ktime_get_boottime_seconds() - clp->cl_ra_time < 5)
continue;
if (clp->cl_cb_state != NFSD4_CB_UP)
continue;
+ /*
+ * This read races with hash_delegation_locked() and
+ * unhash_delegation_locked() on other CPUs. A stale
+ * count only skews the keep value; the next
+ * laundromat pass sees a more current one.
+ */
+ count = data_race(READ_ONCE(clp->cl_deleg_count));
+ if (!count)
+ continue;
if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &clp->cl_ra->ra_cb.cb_flags))
continue;
/* release in nfsd4_cb_recall_any_release */
kref_get(&clp->cl_nfsdfs.cl_ref);
clp->cl_ra_time = ktime_get_boottime_seconds();
- clp->cl_ra->ra_keep = 0;
+ /*
+ * Ask for a single delegation. Recalling one before it
+ * is needed costs the client an OPEN when it next
+ * touches the file.
+ */
+ clp->cl_ra->ra_keep = count - 1;
clp->cl_ra->ra_bmval[0] = BIT(RCA4_TYPE_MASK_RDATA_DLG) |
- BIT(RCA4_TYPE_MASK_WDATA_DLG);
+ BIT(RCA4_TYPE_MASK_WDATA_DLG) |
+ BIT(RCA4_TYPE_MASK_DIR_DLG);
trace_nfsd_cb_recall_any(clp->cl_ra);
nfsd4_run_cb(&clp->cl_ra->ra_cb);
}
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index ebd0dc8f00fc..f7d2884aba44 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -1606,7 +1606,7 @@ nfsd4_decode_write(struct nfsd4_compoundargs *argp, union nfsd4_op_u *u)
return nfserr_bad_xdr;
if (xdr_stream_decode_u32(argp->xdr, &write->wr_stable_how) < 0)
return nfserr_bad_xdr;
- if (write->wr_stable_how > NFS_FILE_SYNC)
+ if (write->wr_stable_how > FILE_SYNC4)
return nfserr_bad_xdr;
if (xdr_stream_decode_u32(argp->xdr, &write->wr_buflen) < 0)
return nfserr_bad_xdr;
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 86aa8d202b4b..3a5d0f5cc0fc 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -2086,8 +2086,13 @@ static int nfsd_nl_validate_listeners(struct genl_info *info)
int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
{
struct net *net = genl_info_net(info);
+ const struct nlattr *bad_attr = NULL;
struct svc_xprt *xprt, *tmp;
+ const char *bad_xprt = NULL;
+ unsigned int rpcb_failures;
const struct nlattr *attr;
+ bool skipped_rpcb = false;
+ bool bad_rpcb = false;
struct svc_serv *serv;
LIST_HEAD(permsocks);
struct nfsd_net *nn;
@@ -2177,13 +2182,16 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
if (delete)
svc_xprt_destroy_all(serv, net, false);
+ rpcb_failures = svc_rpcb_failure_count(serv);
+
/* walk list of addrs again, open any that still don't exist */
nlmsg_for_each_attr_type(attr, NFSD_A_SERVER_SOCK_ADDR, info->nlhdr,
GENL_HDRLEN, rem) {
struct nlattr *tb[NFSD_A_SOCK_MAX + 1];
const char *xcl_name;
struct sockaddr *sa;
- int ret;
+ bool hit_rpcb;
+ int flags, ret;
/* validated up front in nfsd_nl_validate_listeners() */
if (nla_parse_nested(tb, NFSD_A_SOCK_MAX, attr,
@@ -2202,11 +2210,53 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
continue;
}
- ret = svc_xprt_create_from_sa(serv, xcl_name, net, sa, 0,
+ flags = skipped_rpcb ? SVC_SOCK_ANONYMOUS : 0;
+ ret = svc_xprt_create_from_sa(serv, xcl_name, net, sa, flags,
current_cred());
+
+ hit_rpcb = false;
+ if (!skipped_rpcb &&
+ svc_rpcb_failure_count(serv) != rpcb_failures) {
+ skipped_rpcb = true;
+ hit_rpcb = true;
+ if (ret < 0)
+ ret = svc_xprt_create_from_sa(serv, xcl_name,
+ net, sa,
+ SVC_SOCK_ANONYMOUS,
+ current_cred());
+ }
+
/* always save the latest error */
- if (ret < 0)
+ if (ret < 0) {
+ bad_attr = attr;
+ bad_xprt = xcl_name;
+ bad_rpcb = hit_rpcb;
err = ret;
+ }
+ }
+
+ /*
+ * The ack carries the errno of the last entry that failed. Point at
+ * that entry as well, since several entries can share a transport
+ * name and the errno alone cannot tell them apart.
+ */
+ if (err) {
+ NL_SET_BAD_ATTR(info->extack, bad_attr);
+ if (bad_rpcb)
+ NL_SET_ERR_MSG_FMT(info->extack,
+ "cannot create %s listener; rpcbind did not answer",
+ bad_xprt);
+ else if (skipped_rpcb)
+ NL_SET_ERR_MSG_FMT(info->extack,
+ "cannot create %s listener; rpcbind did not answer earlier",
+ bad_xprt);
+ else
+ NL_SET_ERR_MSG_FMT(info->extack,
+ "cannot create %s listener",
+ bad_xprt);
+ } else if (skipped_rpcb) {
+ NL_SET_ERR_MSG(info->extack,
+ "rpcbind did not answer, some listeners are not registered");
}
if (!serv->sv_nrthreads && list_empty(&nn->nfsd_serv->sv_permsocks))
diff --git a/fs/nfsd/nfsproc.c b/fs/nfsd/nfsproc.c
index aaeb222b62ba..48182f78d6be 100644
--- a/fs/nfsd/nfsproc.c
+++ b/fs/nfsd/nfsproc.c
@@ -280,7 +280,7 @@ nfsd_proc_write(struct svc_rqst *rqstp)
fh_copy(&resp->fh, &argp->fh);
resp->status = nfsd_write(rqstp, &resp->fh, argp->offset,
- &argp->payload, &cnt, NFS_DATA_SYNC, NULL);
+ &argp->payload, &cnt, IOCB_DSYNC, NULL);
if (resp->status == nfs_ok)
resp->status = fh_getattr(&resp->fh, &resp->stat);
else if (resp->status == nfserr_jukebox)
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index 9661c08b879e..df35c35890d4 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -610,6 +610,8 @@ struct nfs4_client {
unsigned int cl_state;
atomic_t cl_delegs_in_recall;
+ /* Length of cl_delegations, updated under nn->deleg_lock */
+ unsigned int cl_deleg_count;
struct nfsd4_cb_recall_any *cl_ra;
time64_t cl_ra_time;
diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c
index 8923a9910a08..792c0b327755 100644
--- a/fs/nfsd/vfs.c
+++ b/fs/nfsd/vfs.c
@@ -773,64 +773,21 @@ __be32 nfsd4_vfs_fallocate(struct svc_rqst *rqstp, struct svc_fh *fhp,
}
#endif /* defined(CONFIG_NFSD_V4) */
-/*
- * Check server access rights to a file system object
+/**
+ * nfsd_access - Check caller's access rights to a file system object
+ * @rqstp: RPC transaction context
+ * @fhp: target NFS filehandle
+ * @maps: tables mapping on-the-wire access bits to NFSD_MAY flags
+ * @access: requested access bits on entry, permitted bits on return
+ * @supported: optional output of the access bits the server supports
+ *
+ * Return: nfs_ok on success, otherwise an nfserr status code
*/
-struct accessmap {
- u32 access;
- int how;
-};
-static struct accessmap nfs3_regaccess[] = {
- { NFS3_ACCESS_READ, NFSD_MAY_READ },
- { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
- { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
- { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE },
-
-#ifdef CONFIG_NFSD_V4
- { NFS4_ACCESS_XAREAD, NFSD_MAY_READ },
- { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE },
- { NFS4_ACCESS_XALIST, NFSD_MAY_READ },
-#endif
-
- { 0, 0 }
-};
-
-static struct accessmap nfs3_diraccess[] = {
- { NFS3_ACCESS_READ, NFSD_MAY_READ },
- { NFS3_ACCESS_LOOKUP, NFSD_MAY_EXEC },
- { NFS3_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC},
- { NFS3_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE },
- { NFS3_ACCESS_DELETE, NFSD_MAY_REMOVE },
-
-#ifdef CONFIG_NFSD_V4
- { NFS4_ACCESS_XAREAD, NFSD_MAY_READ },
- { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE },
- { NFS4_ACCESS_XALIST, NFSD_MAY_READ },
-#endif
-
- { 0, 0 }
-};
-
-static struct accessmap nfs3_anyaccess[] = {
- /* Some clients - Solaris 2.6 at least, make an access call
- * to the server to check for access for things like /dev/null
- * (which really, the server doesn't care about). So
- * We provide simple access checking for them, looking
- * mainly at mode bits, and we make sure to ignore read-only
- * filesystem checks
- */
- { NFS3_ACCESS_READ, NFSD_MAY_READ },
- { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
- { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
- { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
-
- { 0, 0 }
-};
-
-__be32
-nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, u32 *access, u32 *supported)
+__be32 nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp,
+ const struct nfsd_access_maps *maps,
+ u32 *access, u32 *supported)
{
- struct accessmap *map;
+ const struct nfsd_access_map *map;
struct svc_export *export;
struct dentry *dentry;
u32 query, result = 0, sresult = 0;
@@ -844,12 +801,11 @@ nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, u32 *access, u32 *suppor
dentry = fhp->fh_dentry;
if (d_is_reg(dentry))
- map = nfs3_regaccess;
+ map = maps->regular;
else if (d_is_dir(dentry))
- map = nfs3_diraccess;
+ map = maps->directory;
else
- map = nfs3_anyaccess;
-
+ map = maps->other;
query = *access;
for (; map->access; map++) {
@@ -859,7 +815,7 @@ nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, u32 *access, u32 *suppor
sresult |= map->access;
err2 = nfsd_permission(&rqstp->rq_cred, export,
- dentry, map->how);
+ dentry, map->may);
switch (err2) {
case nfs_ok:
result |= map->access;
@@ -1424,7 +1380,7 @@ nfsd_direct_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
* @offset: Byte offset of start
* @payload: xdr_buf containing the write payload
* @cnt: IN: number of bytes to write, OUT: number of bytes actually written
- * @stable: An NFS stable_how value
+ * @iocb_flags: VFS IOCB_* flags expressing the requested write stability
* @verf: NFS WRITE verifier
*
* Upon return, caller must invoke fh_put on @fhp.
@@ -1436,7 +1392,7 @@ __be32
nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
struct nfsd_file *nf, loff_t offset,
const struct xdr_buf *payload, unsigned long *cnt,
- int stable, __be32 *verf)
+ int iocb_flags, __be32 *verf)
{
struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
struct file *file = nf->nf_file;
@@ -1473,21 +1429,11 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
exp = fhp->fh_export;
if (!EX_ISSYNC(exp))
- stable = NFS_UNSTABLE;
+ iocb_flags = 0;
init_sync_kiocb(&kiocb, file);
kiocb.ki_pos = offset;
- if (likely(!fhp->fh_use_wgather)) {
- switch (stable) {
- case NFS_FILE_SYNC:
- /* persist data and timestamps */
- kiocb.ki_flags |= IOCB_DSYNC | IOCB_SYNC;
- break;
- case NFS_DATA_SYNC:
- /* persist data only */
- kiocb.ki_flags |= IOCB_DSYNC;
- break;
- }
- }
+ if (likely(!fhp->fh_use_wgather))
+ kiocb.ki_flags |= iocb_flags;
nvecs = xdr_buf_to_bvec(rqstp->rq_bvec, rqstp->rq_maxpages, payload);
if (nvecs < 0) {
@@ -1528,7 +1474,7 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
goto out_nfserr;
}
- if (stable && fhp->fh_use_wgather) {
+ if (iocb_flags && fhp->fh_use_wgather) {
host_err = wait_for_concurrent_writes(file);
if (host_err < 0)
commit_reset_write_verifier(nn, rqstp, host_err);
@@ -1619,7 +1565,7 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
* @offset: Byte offset of start
* @payload: xdr_buf containing the write payload
* @cnt: IN: number of bytes to write, OUT: number of bytes actually written
- * @stable: An NFS stable_how value
+ * @iocb_flags: VFS IOCB_* flags expressing the requested write stability
* @verf: NFS WRITE verifier
*
* Upon return, caller must invoke fh_put on @fhp.
@@ -1629,8 +1575,8 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
*/
__be32
nfsd_write(struct svc_rqst *rqstp, struct svc_fh *fhp, loff_t offset,
- const struct xdr_buf *payload, unsigned long *cnt, int stable,
- __be32 *verf)
+ const struct xdr_buf *payload, unsigned long *cnt,
+ int iocb_flags, __be32 *verf)
{
struct nfsd_file *nf;
__be32 err;
@@ -1642,7 +1588,7 @@ nfsd_write(struct svc_rqst *rqstp, struct svc_fh *fhp, loff_t offset,
goto out;
err = nfsd_vfs_write(rqstp, fhp, nf, offset, payload, cnt,
- stable, verf);
+ iocb_flags, verf);
nfsd_file_put(nf);
out:
trace_nfsd_write_done(rqstp, fhp, offset, *cnt);
diff --git a/fs/nfsd/vfs.h b/fs/nfsd/vfs.h
index 4af2ff9e9dfe..bdb58422a3fb 100644
--- a/fs/nfsd/vfs.h
+++ b/fs/nfsd/vfs.h
@@ -37,6 +37,17 @@
#define NFSD_MAY_CREATE (NFSD_MAY_EXEC|NFSD_MAY_WRITE)
#define NFSD_MAY_REMOVE (NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC)
+struct nfsd_access_map {
+ u32 access;
+ int may;
+};
+
+struct nfsd_access_maps {
+ const struct nfsd_access_map *regular;
+ const struct nfsd_access_map *directory;
+ const struct nfsd_access_map *other;
+};
+
struct nfsd_file;
/*
@@ -100,7 +111,9 @@ __be32 nfsd_create_locked(struct svc_rqst *, struct svc_fh *,
__be32 nfsd_create(struct svc_rqst *, struct svc_fh *,
char *name, int len, struct nfsd_attrs *attrs,
int type, dev_t rdev, struct svc_fh *res);
-__be32 nfsd_access(struct svc_rqst *, struct svc_fh *, u32 *, u32 *);
+__be32 nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp,
+ const struct nfsd_access_maps *maps,
+ u32 *access, u32 *supported);
__be32 nfsd_create_setattr(struct svc_rqst *rqstp, struct svc_fh *fhp,
struct svc_fh *resfhp, struct nfsd_attrs *iap);
__be32 nfsd_commit(struct svc_rqst *rqst, struct svc_fh *fhp,
@@ -135,11 +148,13 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
u32 *eof);
__be32 nfsd_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
loff_t offset, const struct xdr_buf *payload,
- unsigned long *cnt, int stable, __be32 *verf);
+ unsigned long *cnt, int iocb_flags,
+ __be32 *verf);
__be32 nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
struct nfsd_file *nf, loff_t offset,
const struct xdr_buf *payload,
- unsigned long *cnt, int stable, __be32 *verf);
+ unsigned long *cnt, int iocb_flags,
+ __be32 *verf);
__be32 nfsd_readlink(struct svc_rqst *, struct svc_fh *,
char *, int *);
__be32 nfsd_symlink(struct svc_rqst *, struct svc_fh *,
diff --git a/fs/nfsd/xdr3.h b/fs/nfsd/xdr3.h
index 344203874b4c..cad875d14231 100644
--- a/fs/nfsd/xdr3.h
+++ b/fs/nfsd/xdr3.h
@@ -39,7 +39,7 @@ struct nfsd3_writeargs {
svc_fh fh;
__u64 offset;
__u32 count;
- int stable;
+ __u32 stable;
__u32 len;
struct xdr_buf payload;
};
diff --git a/fs/notify/inotify/inotify_fsnotify.c b/fs/notify/inotify/inotify_fsnotify.c
index 7c326ec2e8a8..6e3af6452dcc 100644
--- a/fs/notify/inotify/inotify_fsnotify.c
+++ b/fs/notify/inotify/inotify_fsnotify.c
@@ -36,7 +36,7 @@ static bool event_compare(struct fsnotify_event *old_fsn,
old = INOTIFY_E(old_fsn);
new = INOTIFY_E(new_fsn);
- if (old->mask & FS_IN_IGNORED)
+ if (old->mask & (FS_IN_IGNORED | FS_MOVE))
return false;
if ((old->mask == new->mask) &&
(old->wd == new->wd) &&
diff --git a/fs/ntfs/ea.c b/fs/ntfs/ea.c
index 6ac551fe8ea6..9e00588ff9b5 100644
--- a/fs/ntfs/ea.c
+++ b/fs/ntfs/ea.c
@@ -699,6 +699,11 @@ static int ntfs_new_attr_flags(struct ntfs_inode *ni, __le32 fattr)
goto err_out;
}
+ if ((new_aflags & ATTR_IS_COMPRESSED) && ni->vol->cluster_size > 4096) {
+ err = -EOPNOTSUPP;
+ goto err_out;
+ }
+
if (!a->non_resident) {
if (!(new_aflags & (ATTR_IS_SPARSE | ATTR_IS_COMPRESSED))) {
err = 0;
diff --git a/fs/ntfs3/xattr.c b/fs/ntfs3/xattr.c
index 594ef6860b93..fb7276d4a428 100644
--- a/fs/ntfs3/xattr.c
+++ b/fs/ntfs3/xattr.c
@@ -146,29 +146,32 @@ static int ntfs_read_ea(struct ntfs_inode *ni, struct EA_FULL **ea,
for (off = 0; off < size; off += ea_size) {
const struct EA_FULL *ef = Add2Ptr(ea_p, off);
u32 bytes = size - off;
- size_t need;
-
- /* Check if we can use field ea->size. */
- if (bytes < sizeof(ef->size))
- goto out1;
+ u32 packed_size;
/* Check if we can use fields ef->name_len and ef->elength. */
if (bytes < offsetof(struct EA_FULL, name))
goto out1;
- /* Size needed to hold this record's name and value. */
- need = struct_size(ef, name,
- 1 + ef->name_len + le16_to_cpu(ef->elength));
+ /*
+ * The name and the value have to fit in the entry whether or
+ * not ef->size is set, because ntfs_get_ea() copies elength
+ * bytes out of this buffer and checks them only against the
+ * caller supplied buffer size.
+ */
+ packed_size =
+ ALIGN(struct_size(ef, name,
+ 1 + ef->name_len +
+ le16_to_cpu(ef->elength)),
+ 4);
if (ef->size) {
ea_size = le32_to_cpu(ef->size);
- /* ef->size must fit the list and cover the record. */
- if (ea_size > bytes || ea_size < need)
+ if (ea_size > bytes || packed_size > ea_size)
goto out1;
continue;
}
- ea_size = ALIGN(need, 4);
+ ea_size = packed_size;
if (ea_size > bytes)
goto out1;
}
diff --git a/fs/ocfs2/cluster/tcp.c b/fs/ocfs2/cluster/tcp.c
index 474fe1414cee..0ba26bd2eba1 100644
--- a/fs/ocfs2/cluster/tcp.c
+++ b/fs/ocfs2/cluster/tcp.c
@@ -391,6 +391,9 @@ static void sc_kref_release(struct kref *kref)
sclog(sc, "releasing\n");
+ /* Unlink before freeing anything the debugfs readers dereference. */
+ o2net_debug_del_sc(sc);
+
if (sc->sc_sock) {
sock_release(sc->sc_sock);
sc->sc_sock = NULL;
@@ -400,8 +403,6 @@ static void sc_kref_release(struct kref *kref)
o2nm_node_put(sc->sc_node);
sc->sc_node = NULL;
- o2net_debug_del_sc(sc);
-
if (sc->sc_page)
__free_page(sc->sc_page);
kfree(sc);
diff --git a/fs/ocfs2/dcache.h b/fs/ocfs2/dcache.h
index 7f246c5692d8..046ab9a80769 100644
--- a/fs/ocfs2/dcache.h
+++ b/fs/ocfs2/dcache.h
@@ -17,9 +17,8 @@ struct ocfs2_dentry_lock {
u64 dl_parent_blkno;
/*
- * The ocfs2_dentry_lock keeps an inode reference until
- * dl_lockres has been destroyed. This is usually done in
- * ->d_iput() anyway, so there should be minimal impact.
+ * The inode reference is dropped before dl_lockres is unlocked.
+ * Callbacks must not use it to find the mount context during teardown.
*/
struct inode *dl_inode;
struct ocfs2_lock_res dl_lockres;
diff --git a/fs/ocfs2/dlm/dlmmaster.c b/fs/ocfs2/dlm/dlmmaster.c
index 612969867ff9..ea48c013f7fb 100644
--- a/fs/ocfs2/dlm/dlmmaster.c
+++ b/fs/ocfs2/dlm/dlmmaster.c
@@ -2525,6 +2525,52 @@ static int dlm_is_lockres_migratable(struct dlm_ctxt *dlm,
*/
+/*
+ * Drop the sender's references on a migration MLE after the attempt has
+ * failed. The finishing assert from the target may already have unlinked
+ * the MLE and consumed its base reference; decide under the locks the
+ * assert handler holds, and only drop the base reference while the MLE is
+ * still hashed. Returns true when the assert got there first: the target
+ * has become the master, so the caller must complete the migration.
+ */
+static bool dlm_put_migration_mle(struct dlm_ctxt *dlm,
+ struct dlm_master_list_entry *mle)
+{
+ bool asserted;
+
+ spin_lock(&dlm->spinlock);
+ spin_lock(&dlm->master_lock);
+ __dlm_mle_detach_hb_events(dlm, mle);
+ asserted = hlist_unhashed(&mle->master_hash_node);
+ if (!asserted) {
+ __dlm_unlink_mle(dlm, mle);
+ __dlm_put_mle(mle);
+ }
+ spin_unlock(&dlm->master_lock);
+ spin_unlock(&dlm->spinlock);
+
+ dlm_put_mle_inuse(mle);
+ return asserted;
+}
+
+/*
+ * The finishing assert got there first: it switched the owner to the
+ * target and cleared DLM_LOCK_RES_MIGRATING. Drop the other nodes' locks
+ * as a completed migration does, unless recovery has taken the lockres
+ * over since, and let the usage be recalculated.
+ */
+static void dlm_migrate_lockres_asserted(struct dlm_ctxt *dlm,
+ struct dlm_lock_resource *res,
+ u8 target)
+{
+ spin_lock(&res->spinlock);
+ if (res->owner == target)
+ dlm_remove_nonlocal_locks(dlm, res);
+ spin_unlock(&res->spinlock);
+
+ dlm_lockres_calc_usage(dlm, res);
+}
+
static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
struct dlm_lock_resource *res, u8 target)
{
@@ -2609,9 +2655,10 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
if (ret < 0) {
if (mle_added) {
- dlm_mle_detach_hb_events(dlm, mle);
- dlm_put_mle(mle);
- dlm_put_mle_inuse(mle);
+ if (dlm_put_migration_mle(dlm, mle)) {
+ dlm_migrate_lockres_asserted(dlm, res, target);
+ ret = 0;
+ }
} else if (mle) {
kmem_cache_free(dlm_mle_cache, mle);
mle = NULL;
@@ -2640,9 +2687,11 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
mlog(0, "migration to node %u failed with %d\n",
target, ret);
/* migration failed, detach and clean up mle */
- dlm_mle_detach_hb_events(dlm, mle);
- dlm_put_mle(mle);
- dlm_put_mle_inuse(mle);
+ if (dlm_put_migration_mle(dlm, mle)) {
+ dlm_migrate_lockres_asserted(dlm, res, target);
+ ret = 0;
+ goto leave;
+ }
spin_lock(&res->spinlock);
res->state &= ~DLM_LOCK_RES_MIGRATING;
wake = 1;
@@ -2686,9 +2735,12 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
res->lockname.name, target);
ret = -EINVAL;
/* migration failed, detach and clean up mle */
- dlm_mle_detach_hb_events(dlm, mle);
- dlm_put_mle(mle);
- dlm_put_mle_inuse(mle);
+ if (dlm_put_migration_mle(dlm, mle)) {
+ dlm_migrate_lockres_asserted(dlm, res,
+ target);
+ ret = 0;
+ goto leave;
+ }
spin_lock(&res->spinlock);
res->state &= ~DLM_LOCK_RES_MIGRATING;
wake = 1;
diff --git a/fs/ocfs2/dlmglue.c b/fs/ocfs2/dlmglue.c
index 1a83241b6d41..a2a33c34fdf9 100644
--- a/fs/ocfs2/dlmglue.c
+++ b/fs/ocfs2/dlmglue.c
@@ -57,7 +57,6 @@ struct ocfs2_mask_waiter {
#endif
};
-static struct ocfs2_super *ocfs2_get_dentry_osb(struct ocfs2_lock_res *lockres);
static struct ocfs2_super *ocfs2_get_inode_osb(struct ocfs2_lock_res *lockres);
static struct ocfs2_super *ocfs2_get_file_osb(struct ocfs2_lock_res *lockres);
static struct ocfs2_super *ocfs2_get_qinfo_osb(struct ocfs2_lock_res *lockres);
@@ -257,7 +256,6 @@ static const struct ocfs2_lock_res_ops ocfs2_orphan_scan_lops = {
};
static const struct ocfs2_lock_res_ops ocfs2_dentry_lops = {
- .get_osb = ocfs2_get_dentry_osb,
.post_unlock = ocfs2_dentry_post_unlock,
.downconvert_worker = ocfs2_dentry_convert_worker,
.flags = 0,
@@ -308,7 +306,7 @@ static inline struct ocfs2_dentry_lock *ocfs2_lock_res_dl(struct ocfs2_lock_res
{
BUG_ON(lockres->l_type != OCFS2_LOCK_TYPE_DENTRY);
- return (struct ocfs2_dentry_lock *)lockres->l_priv;
+ return container_of(lockres, struct ocfs2_dentry_lock, dl_lockres);
}
static inline struct ocfs2_mem_dqinfo *ocfs2_lock_res_qinfo(struct ocfs2_lock_res *lockres)
@@ -609,13 +607,6 @@ static __u64 ocfs2_get_dentry_lock_ino(struct ocfs2_lock_res *lockres)
return be64_to_cpu(inode_blkno_be);
}
-static struct ocfs2_super *ocfs2_get_dentry_osb(struct ocfs2_lock_res *lockres)
-{
- struct ocfs2_dentry_lock *dl = lockres->l_priv;
-
- return OCFS2_SB(dl->dl_inode->i_sb);
-}
-
void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl,
u64 parent, struct inode *inode)
{
@@ -623,6 +614,7 @@ void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl,
u64 inode_blkno = OCFS2_I(inode)->ip_blkno;
__be64 inode_blkno_be = cpu_to_be64(inode_blkno);
struct ocfs2_lock_res *lockres = &dl->dl_lockres;
+ struct ocfs2_super *osb = OCFS2_SB(inode->i_sb);
ocfs2_lock_res_init_once(lockres);
@@ -651,9 +643,9 @@ void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl,
memcpy(&lockres->l_name[OCFS2_DENTRY_LOCK_INO_START], &inode_blkno_be,
sizeof(__be64));
- ocfs2_lock_res_init_common(OCFS2_SB(inode->i_sb), lockres,
+ ocfs2_lock_res_init_common(osb, lockres,
OCFS2_LOCK_TYPE_DENTRY, &ocfs2_dentry_lops,
- dl);
+ osb);
}
static void ocfs2_super_lock_res_init(struct ocfs2_lock_res *res,
@@ -3335,6 +3327,14 @@ static void ocfs2_dlm_shutdown_debug(struct ocfs2_super *osb)
{
struct ocfs2_dlm_debug *dlm_debug = osb->osb_dlm_debug;
+ /*
+ * Drain the readers before the cluster stack goes away: the
+ * locking_state reader resolves ocfs2_dlm_lvb() through the
+ * stack. Every caller removes the parent directory later.
+ */
+ debugfs_lookup_and_remove("locking_state", osb->osb_debug_root);
+ debugfs_lookup_and_remove("locking_filter", osb->osb_debug_root);
+
if (dlm_debug)
ocfs2_put_dlm_debug(dlm_debug);
}
@@ -3349,8 +3349,6 @@ int ocfs2_dlm_init(struct ocfs2_super *osb)
goto local;
}
- ocfs2_dlm_init_debug(osb);
-
/* launch downconvert thread */
osb->dc_task = kthread_run(ocfs2_downconvert_thread, osb, "ocfs2dc-%s",
osb->uuid_str);
@@ -3390,9 +3388,10 @@ int ocfs2_dlm_init(struct ocfs2_super *osb)
ocfs2_orphan_scan_lock_res_init(&osb->osb_orphan_scan.os_lockres, osb);
osb->cconn = conn;
+ if (conn)
+ ocfs2_dlm_init_debug(osb);
bail:
if (status < 0) {
- ocfs2_dlm_shutdown_debug(osb);
if (osb->dc_task)
kthread_stop(osb->dc_task);
}
@@ -3422,10 +3421,11 @@ void ocfs2_dlm_shutdown(struct ocfs2_super *osb,
ocfs2_lock_res_free(&osb->osb_orphan_scan.os_lockres);
if (osb->cconn) {
+ /* The disconnect below may release the selected stack. */
+ ocfs2_dlm_shutdown_debug(osb);
+
ocfs2_cluster_disconnect(osb->cconn, hangup_pending);
osb->cconn = NULL;
-
- ocfs2_dlm_shutdown_debug(osb);
}
}
diff --git a/fs/ocfs2/filecheck.c b/fs/ocfs2/filecheck.c
index 75334e4bd082..e3b79bc749f5 100644
--- a/fs/ocfs2/filecheck.c
+++ b/fs/ocfs2/filecheck.c
@@ -189,6 +189,8 @@ int ocfs2_filecheck_create_sysfs(struct ocfs2_super *osb)
/* Mount failure still calls ocfs2_filecheck_remove_sysfs(). */
entry->fs_fcheck = NULL;
kobject_put(&entry->fs_kobj);
+ /* Keep ocfs2_super alive until the kobject is released. */
+ wait_for_completion(&entry->fs_kobj_unregister);
kfree(fcheck);
return ret;
}
diff --git a/fs/ocfs2/journal.c b/fs/ocfs2/journal.c
index a3938a03e93b..674e70dfbb5c 100644
--- a/fs/ocfs2/journal.c
+++ b/fs/ocfs2/journal.c
@@ -227,13 +227,22 @@ void ocfs2_recovery_disable_quota(struct ocfs2_super *osb)
ocfs2_recovery_disable(osb, OCFS2_REC_QUOTA_WANT_DISABLE);
}
+/* Stop producers and drain completion work without freeing recovery state. */
+void ocfs2_recovery_stop(struct ocfs2_super *osb)
+{
+ ocfs2_recovery_disable(osb, OCFS2_REC_WANT_DISABLE);
+}
+
void ocfs2_recovery_exit(struct ocfs2_super *osb)
{
struct ocfs2_recovery_map *rm;
- /* disable any new recovery threads and wait for any currently
- * running ones to exit. Do this before setting the vol_state. */
- ocfs2_recovery_disable(osb, OCFS2_REC_WANT_DISABLE);
+ /*
+ * Disable producers and drain work before freeing recovery state.
+ * Do this before setting vol_state to VOLUME_DISMOUNTED: completion
+ * work may still be waiting in __ocfs2_wait_on_mount().
+ */
+ ocfs2_recovery_stop(osb);
/*
* Now that recovery is shut down, and the osb is about to be
diff --git a/fs/ocfs2/journal.h b/fs/ocfs2/journal.h
index 19fc920d26b1..4b52bf7b0938 100644
--- a/fs/ocfs2/journal.h
+++ b/fs/ocfs2/journal.h
@@ -147,6 +147,7 @@ void ocfs2_complete_recovery(struct work_struct *work);
void ocfs2_wait_for_recovery(struct ocfs2_super *osb);
int ocfs2_recovery_init(struct ocfs2_super *osb);
+void ocfs2_recovery_stop(struct ocfs2_super *osb);
void ocfs2_recovery_exit(struct ocfs2_super *osb);
void ocfs2_recovery_disable_quota(struct ocfs2_super *osb);
diff --git a/fs/ocfs2/move_extents.c b/fs/ocfs2/move_extents.c
index ad1678ee7cc4..93b45531bf7d 100644
--- a/fs/ocfs2/move_extents.c
+++ b/fs/ocfs2/move_extents.c
@@ -907,6 +907,30 @@ static int __ocfs2_move_extents_range(struct buffer_head *di_bh,
return ret;
}
+static int ocfs2_validate_and_adjust_move_range(struct inode *inode,
+ struct ocfs2_move_extents *range)
+{
+ u64 i_size = i_size_read(inode);
+
+ if (range->me_start > i_size)
+ return -EINVAL;
+
+ if (range->me_len > i_size - range->me_start)
+ range->me_len = i_size - range->me_start;
+
+ /*
+ * The default threshold for defragmentation is 1 MiB, which
+ * matches the maximum OCFS2 cluster size.
+ */
+ if (!range->me_threshold)
+ range->me_threshold = 1024 * 1024;
+
+ if (range->me_threshold > i_size)
+ range->me_threshold = i_size;
+
+ return 0;
+}
+
static int ocfs2_move_extents(struct ocfs2_move_extents_context *context)
{
int status;
@@ -936,6 +960,21 @@ static int ocfs2_move_extents(struct ocfs2_move_extents_context *context)
goto out_rw_unlock;
}
+ status = ocfs2_validate_and_adjust_move_range(inode, context->range);
+ if (status) {
+ mlog_errno(status);
+ goto out_inode_unlock;
+ }
+
+ /* The goal check depends on the adjusted range length. */
+ if (!context->auto_defrag) {
+ status = ocfs2_validate_and_adjust_move_goal(inode, context->range);
+ if (status) {
+ mlog_errno(status);
+ goto out_inode_unlock;
+ }
+ }
+
/*
* remember ip_xattr_sem also needs to be held if necessary
*/
@@ -1028,27 +1067,8 @@ int ocfs2_ioctl_move_extents(struct file *filp, void __user *argp)
goto out_free;
}
- if (range.me_start > i_size_read(inode)) {
- status = -EINVAL;
- goto out_free;
- }
-
- if (range.me_start + range.me_len > i_size_read(inode))
- range.me_len = i_size_read(inode) - range.me_start;
-
context->range = ⦥
- /*
- * ok, the default threshold for the defragmentation
- * is 1M, since our maximum clustersize was 1M also.
- * any thought?
- */
- if (!range.me_threshold)
- range.me_threshold = 1024 * 1024;
-
- if (range.me_threshold > i_size_read(inode))
- range.me_threshold = i_size_read(inode);
-
if (range.me_flags & ~(OCFS2_MOVE_EXT_FL_AUTO_DEFRAG |
OCFS2_MOVE_EXT_FL_PART_DEFRAG)) {
status = -EINVAL;
@@ -1060,23 +1080,11 @@ int ocfs2_ioctl_move_extents(struct file *filp, void __user *argp)
if (range.me_flags & OCFS2_MOVE_EXT_FL_PART_DEFRAG)
context->partial = 1;
- } else {
- /*
- * first best-effort attempt to validate and adjust the goal
- * (physical address in block), while it can't guarantee later
- * operation can succeed all the time since global_bitmap may
- * change a bit over time.
- */
-
- status = ocfs2_validate_and_adjust_move_goal(inode, &range);
- if (status)
- goto out_copy;
}
status = ocfs2_move_extents(context);
if (status)
mlog_errno(status);
-out_copy:
/*
* movement/defragmentation may end up being partially completed,
* that's the reason why we need to return userspace the finished
diff --git a/fs/ocfs2/super.c b/fs/ocfs2/super.c
index 169c576f06bf..e5482c556552 100644
--- a/fs/ocfs2/super.c
+++ b/fs/ocfs2/super.c
@@ -1828,6 +1828,9 @@ static void ocfs2_dismount_volume(struct super_block *sb, int mnt_err)
/* Wait for worker to be done with the work structure in osb */
cancel_work_sync(&osb->dquot_drop_work);
+ /* Stop recovery producers before releasing their local allocators. */
+ ocfs2_recovery_stop(osb);
+
ocfs2_shutdown_local_alloc(osb);
ocfs2_truncate_log_shutdown(osb);
diff --git a/fs/ocfs2/xattr.c b/fs/ocfs2/xattr.c
index 18ebe3cc64ec..4d00a26ece3c 100644
--- a/fs/ocfs2/xattr.c
+++ b/fs/ocfs2/xattr.c
@@ -260,6 +260,7 @@ static int ocfs2_xattr_tree_list_index_block(struct inode *inode,
size_t buffer_size);
static int ocfs2_xattr_create_index_block(struct inode *inode,
+ struct ocfs2_xattr_info *xi,
struct ocfs2_xattr_search *xs,
struct ocfs2_xattr_set_ctxt *ctxt);
@@ -3251,7 +3252,7 @@ static int ocfs2_xattr_block_set(struct inode *inode,
else if ((ret != -ENOSPC) || ctxt->set_abort)
goto end;
else {
- ret = ocfs2_xattr_create_index_block(inode, xs, ctxt);
+ ret = ocfs2_xattr_create_index_block(inode, xi, xs, ctxt);
if (ret)
goto end;
}
@@ -4533,6 +4534,21 @@ static void ocfs2_cp_xattr_block_to_bucket(struct inode *inode,
for (i = 0; i < blks; i++)
memset(bucket_block(bucket, i), 0, blocksize);
+ /*
+ * An empty block has no names or values to carry over, and the
+ * xh_entries[count - 1] below would be xh_entries[-1]. Leave the
+ * bucket empty, but carry over the hash ocfs2_xa_remove_entry() left
+ * in the block's xh_entries[0] so that the bucket still holds the key
+ * it is looked up by.
+ */
+ if (!count) {
+ xh->xh_num_buckets = cpu_to_le16(1);
+ xh->xh_free_start = cpu_to_le16(OCFS2_XATTR_BUCKET_SIZE);
+ xh->xh_entries[0].xe_name_hash =
+ xb_xh->xh_entries[0].xe_name_hash;
+ return;
+ }
+
/*
* Since the xe_name_offset is based on ocfs2_xattr_header,
* there is a offset change corresponding to the change of
@@ -4579,13 +4595,10 @@ static void ocfs2_cp_xattr_block_to_bucket(struct inode *inode,
* real place of the xattr.
*/
static void ocfs2_xattr_update_xattr_search(struct inode *inode,
- struct ocfs2_xattr_search *xs,
- struct buffer_head *old_bh)
+ struct ocfs2_xattr_info *xi,
+ struct ocfs2_xattr_search *xs)
{
- char *buf = old_bh->b_data;
- struct ocfs2_xattr_block *old_xb = (struct ocfs2_xattr_block *)buf;
- struct ocfs2_xattr_header *old_xh = &old_xb->xb_attrs.xb_header;
- int i;
+ int i, count;
xs->header = bucket_xh(xs->bucket);
xs->base = bucket_block(xs->bucket, 0);
@@ -4594,11 +4607,47 @@ static void ocfs2_xattr_update_xattr_search(struct inode *inode,
if (xs->not_found)
return;
- i = xs->here - old_xh->xh_entries;
- xs->here = &xs->header->xh_entries[i];
+ /*
+ * A block holds its entries in insertion order while a bucket holds
+ * them by name hash, so ocfs2_cp_xattr_block_to_bucket() sorted the
+ * copies and the index this entry had in the block no longer says
+ * where it is. It copied every entry and every name unchanged, and
+ * (type, name) pairs are unique within a block, so find it again by
+ * name.
+ *
+ * Search for the name being set, not for the name xs->here carries.
+ * A failed set may have removed that entry, and ocfs2_xa_remove_entry()
+ * then memmoves its successor into the slot. Comparing every entry
+ * rather than narrowing on the hash also finds an entry stored under
+ * the legacy signed hash, which ocfs2_find_xe_in_bucket() would skip.
+ */
+ count = le16_to_cpu(xs->header->xh_count);
+ for (i = 0; i < count; i++) {
+ struct ocfs2_xattr_entry *xe = &xs->header->xh_entries[i];
+
+ if (ocfs2_xattr_get_type(xe) != xi->xi_name_index)
+ continue;
+ if (xe->xe_name_len != xi->xi_name_len)
+ continue;
+ if (!memcmp(xs->base + le16_to_cpu(xe->xe_name_offset),
+ xi->xi_name, xi->xi_name_len)) {
+ xs->here = xe;
+ return;
+ }
+ }
+
+ /*
+ * The entry did not survive the failed set, so this has become an
+ * insert. Point xs->here back inside the bucket: it still refers to
+ * the block being converted, which is about to be rewritten as a tree
+ * root.
+ */
+ xs->not_found = -ENODATA;
+ xs->here = xs->header->xh_entries;
}
static int ocfs2_xattr_create_index_block(struct inode *inode,
+ struct ocfs2_xattr_info *xi,
struct ocfs2_xattr_search *xs,
struct ocfs2_xattr_set_ctxt *ctxt)
{
@@ -4665,7 +4714,7 @@ static int ocfs2_xattr_create_index_block(struct inode *inode,
ocfs2_cp_xattr_block_to_bucket(inode, xb_bh, xs->bucket);
ocfs2_xattr_bucket_journal_dirty(handle, xs->bucket);
- ocfs2_xattr_update_xattr_search(inode, xs, xb_bh);
+ ocfs2_xattr_update_xattr_search(inode, xi, xs);
/* Change from ocfs2_xattr_header to ocfs2_xattr_tree_root */
memset(&xb->xb_attrs, 0, inode->i_sb->s_blocksize -
@@ -5018,7 +5067,15 @@ static int ocfs2_divide_xattr_bucket(struct inode *inode,
start = ocfs2_xattr_find_divide_pos(xh);
if (start == count) {
- xe = &xh->xh_entries[start-1];
+ /*
+ * start == count means every entry shares one hash, so the
+ * first of them carries it too. An empty bucket has no
+ * entries[start - 1], but its entries[0] still carries the
+ * bucket key: either the hash ocfs2_xa_remove_entry()
+ * deliberately leaves behind, or the sentinel a previous
+ * divide wrote.
+ */
+ xe = &xh->xh_entries[0];
/*
* initialized a new empty bucket here.
@@ -5848,11 +5905,20 @@ static int ocfs2_check_xattr_bucket_collision(struct ocfs2_xattr_bucket *bucket,
u32 name_hash)
{
struct ocfs2_xattr_header *xh = bucket_xh(bucket);
+ u16 count = le16_to_cpu(xh->xh_count);
+
+ /*
+ * An empty bucket keeps the hash of the entry that was removed in
+ * xh_entries[0] so that it can still be found, but it holds no
+ * entries to collide with.
+ */
+ if (!count)
+ return 0;
if (name_hash != le32_to_cpu(xh->xh_entries[0].xe_name_hash))
return 0;
- if (xh->xh_entries[le16_to_cpu(xh->xh_count) - 1].xe_name_hash ==
+ if (xh->xh_entries[count - 1].xe_name_hash ==
xh->xh_entries[0].xe_name_hash) {
mlog(ML_ERROR, "Too much hash collision in xattr bucket %llu, "
"hash = %u\n",
@@ -5873,13 +5939,18 @@ static int ocfs2_xattr_set_entry_bucket(struct inode *inode,
struct ocfs2_xattr_search *xs,
struct ocfs2_xattr_set_ctxt *ctxt)
{
- int ret;
+ int ret, found = 0;
+ u16 xe_index;
+ u32 name_hash;
struct ocfs2_xa_loc loc;
trace_ocfs2_xattr_set_entry_bucket(xi->xi_name);
ocfs2_init_xattr_bucket_xa_loc(&loc, xs->bucket,
xs->not_found ? NULL : xs->here);
+ /* Defragmenting below may move this entry, so remember its hash. */
+ name_hash = loc.xl_entry ? le32_to_cpu(loc.xl_entry->xe_name_hash) : 0;
+
ret = ocfs2_xa_set(&loc, xi, ctxt);
if (!ret) {
xs->here = loc.xl_entry;
@@ -5898,6 +5969,29 @@ static int ocfs2_xattr_set_entry_bucket(struct inode *inode,
goto out;
}
+ /*
+ * ocfs2_defrag_xattr_bucket() sorts the entries by name hash and
+ * lib/sort.c is not stable, so entries sharing a hash may have traded
+ * slots. Look the entry up again before writing through it.
+ */
+ if (loc.xl_entry) {
+ ret = ocfs2_find_xe_in_bucket(inode, xs->bucket,
+ xi->xi_name_index, xi->xi_name,
+ name_hash, &xe_index, &found);
+ if (ret) {
+ mlog_errno(ret);
+ goto out;
+ }
+ if (!found) {
+ mlog(ML_ERROR, "xattr entry not found in bucket %llu after defrag\n",
+ (unsigned long long)bucket_blkno(xs->bucket));
+ ret = -EFSCORRUPTED;
+ goto out;
+ }
+ loc.xl_entry = &bucket_xh(xs->bucket)->xh_entries[xe_index];
+ xs->here = loc.xl_entry;
+ }
+
ret = ocfs2_xa_set(&loc, xi, ctxt);
if (!ret) {
xs->here = loc.xl_entry;
diff --git a/fs/open.c b/fs/open.c
index 6b1c14e684a9..d7d9fcc666ab 100644
--- a/fs/open.c
+++ b/fs/open.c
@@ -1147,6 +1147,7 @@ struct file *dentry_open_nonotify(const struct path *path, int flags,
}
return f;
}
+EXPORT_SYMBOL_GPL(dentry_open_nonotify);
/**
* kernel_file_open - open a file for kernel internal use
diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c
index f716ac13badf..c77af9a79d88 100644
--- a/fs/overlayfs/dir.c
+++ b/fs/overlayfs/dir.c
@@ -74,39 +74,34 @@ static struct dentry *ovl_start_creating_temp(struct ovl_fs *ofs,
&QSTR(name));
}
+/*
+ * The whiteout is the source of a rename, so it has to be hashed.
+ * ovl_create_temp() takes care of that for a casefolded workdir.
+ */
static struct dentry *ovl_whiteout(struct ovl_fs *ofs)
{
int err;
struct dentry *whiteout, *link;
struct dentry *workdir = ofs->workdir;
- struct inode *wdir = workdir->d_inode;
- char name[OVL_TEMPNAME_SIZE];
guard(mutex)(&ofs->whiteout_lock);
if (!ofs->whiteout) {
- whiteout = ovl_start_creating_temp(ofs, workdir, name);
+ whiteout = ovl_create_temp(ofs, workdir, &(struct ovl_cattr) {
+ .mode = S_IFCHR | WHITEOUT_MODE,
+ .rdev = WHITEOUT_DEV });
if (IS_ERR(whiteout))
return whiteout;
- err = ovl_do_whiteout(ofs, wdir, whiteout);
- if (!err)
- ofs->whiteout = dget(whiteout);
- end_creating(whiteout);
- if (err)
- return ERR_PTR(err);
+ ofs->whiteout = whiteout;
}
if (!ofs->no_shared_whiteout) {
- link = ovl_start_creating_temp(ofs, workdir, name);
- if (IS_ERR(link))
+ link = ovl_create_temp(ofs, workdir, &(struct ovl_cattr) {
+ .hardlink = ofs->whiteout });
+ if (!IS_ERR(link))
return link;
- err = ovl_do_link(ofs, ofs->whiteout, wdir, link);
- if (!err)
- whiteout = dget(link);
- end_creating(link);
- if (!err)
- return whiteout;
+ err = PTR_ERR(link);
if (err != -EMLINK) {
pr_warn("Failed to link whiteout - disabling whiteout inode sharing(nlink=%u, err=%u)\n",
ofs->whiteout->d_inode->i_nlink,
@@ -120,9 +115,10 @@ static struct dentry *ovl_whiteout(struct ovl_fs *ofs)
}
int ovl_cleanup_and_whiteout(struct ovl_fs *ofs, struct dentry *dir,
- struct dentry *dentry)
+ struct dentry *dentry, const struct qstr *name)
{
struct dentry *whiteout;
+ struct qstr last = QSTR_LEN(name->name, name->len);
struct renamedata rd = {};
int err;
int flags = 0;
@@ -139,9 +135,17 @@ int ovl_cleanup_and_whiteout(struct ovl_fs *ofs, struct dentry *dir,
rd.old_parent = ofs->workdir;
rd.new_parent = dir;
rd.flags = flags;
- err = start_renaming_two_dentries(&rd, whiteout, dentry);
+ /*
+ * The caller looked up @dentry by @name without a lock on @dir, and a
+ * casefolded directory leaves a negative dentry unhashed. Nothing tells
+ * if such a dentry is still valid, so look up @name again with @dir locked.
+ */
+ err = start_renaming_dentry(&rd, 0, whiteout, &last);
if (!err) {
- err = ovl_do_rename_rd(&rd);
+ if (d_inode(rd.new_dentry) != d_inode(dentry))
+ err = -ESTALE;
+ else
+ err = ovl_do_rename_rd(&rd);
end_renaming(&rd);
}
if (err)
@@ -847,7 +851,7 @@ static int ovl_remove_and_whiteout(struct dentry *dentry,
goto out_dput_upper;
}
- err = ovl_cleanup_and_whiteout(ofs, upperdir, upper);
+ err = ovl_cleanup_and_whiteout(ofs, upperdir, upper, &dentry->d_name);
if (!err)
ovl_dir_modified(dentry->d_parent, true);
diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
index 9aae20c59608..bd998a3d8b95 100644
--- a/fs/overlayfs/export.c
+++ b/fs/overlayfs/export.c
@@ -713,9 +713,9 @@ static struct dentry *ovl_lower_fh_to_d(struct super_block *sb,
}
}
- /* Then lookup indexed upper/whiteout by origin fh */
+ /* Then lookup indexed upper/whiteout by origin */
if (ovl_indexdir(sb)) {
- index = ovl_get_index_fh(ofs, fh);
+ index = ovl_get_index_origin(ofs, origin.dentry);
err = PTR_ERR(index);
if (IS_ERR(index)) {
index = NULL;
diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 7e1ac70c5d97..e3aa0931bea9 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -658,6 +658,14 @@ int ovl_fileattr_set(struct mnt_idmap *idmap,
unsigned int flags;
int err;
+ /*
+ * Lookup refuses a directory that does not agree with the casefold
+ * state of the mount. Only FS_IOC_SETFLAGS can ask for a change.
+ */
+ if (S_ISDIR(inode->i_mode) && fa->flags_valid &&
+ !!(fa->flags & FS_CASEFOLD_FL) != OVL_FS(inode->i_sb)->casefold)
+ return -EOPNOTSUPP;
+
err = ovl_copy_up(dentry);
if (!err) {
ovl_path_real(dentry, &upperpath);
diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index c6127edb9be9..64d63b260709 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -781,14 +781,21 @@ int ovl_get_index_name(struct ovl_fs *ofs, struct dentry *origin,
return err;
}
-/* Lookup index by file handle for NFS export */
-struct dentry *ovl_get_index_fh(struct ovl_fs *ofs, struct ovl_fh *fh)
+/*
+ * Lookup index by decoded origin for NFS export.
+ *
+ * The name of an index is the hex of the file handle that copy up encoded.
+ * The bytes of the handle to decode cannot be used as the name, because a
+ * lower fs may decode many handles to the same inode. For example, it may
+ * ignore the fid type, extra fid bytes or a zero generation.
+ */
+struct dentry *ovl_get_index_origin(struct ovl_fs *ofs, struct dentry *origin)
{
struct dentry *index;
struct qstr name;
int err;
- err = ovl_get_index_name_fh(fh, &name);
+ err = ovl_get_index_name(ofs, origin, &name);
if (err)
return ERR_PTR(err);
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 91cdd7e15696..a5aa22cc79a8 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -394,14 +394,6 @@ static inline int ovl_do_rename(struct ovl_fs *ofs, struct dentry *olddir,
return ovl_do_rename_rd(&rd);
}
-static inline int ovl_do_whiteout(struct ovl_fs *ofs,
- struct inode *dir, struct dentry *dentry)
-{
- int err = vfs_whiteout(ovl_upper_mnt_idmap(ofs), dir, dentry);
- pr_debug("whiteout(%pd2) = %i\n", dentry, err);
- return err;
-}
-
static inline struct file *ovl_do_tmpfile(struct ovl_fs *ofs,
struct dentry *dentry, umode_t mode)
{
@@ -537,7 +529,14 @@ void ovl_dir_modified(struct dentry *dentry, bool impurity);
u64 ovl_inode_version_get(struct inode *inode);
bool ovl_is_whiteout(struct dentry *dentry);
bool ovl_path_is_whiteout(struct ovl_fs *ofs, const struct path *path);
-struct file *ovl_path_open(const struct path *path, int flags);
+struct file *ovl_path_open_realfile(const struct file *user_file,
+ const struct path *path, int flags);
+
+static inline struct file *ovl_path_open(const struct path *path, int flags)
+{
+ return ovl_path_open_realfile(NULL, path, flags);
+}
+
int ovl_copy_up_start(struct dentry *dentry, int flags);
void ovl_copy_up_end(struct dentry *dentry);
bool ovl_already_copied_up(struct dentry *dentry, int flags);
@@ -749,7 +748,7 @@ int ovl_verify_index(struct ovl_fs *ofs, struct dentry *index);
int ovl_get_index_name_fh(const struct ovl_fh *fh, struct qstr *name);
int ovl_get_index_name(struct ovl_fs *ofs, struct dentry *origin,
struct qstr *name);
-struct dentry *ovl_get_index_fh(struct ovl_fs *ofs, struct ovl_fh *fh);
+struct dentry *ovl_get_index_origin(struct ovl_fs *ofs, struct dentry *origin);
struct dentry *ovl_lookup_index(struct ovl_fs *ofs, struct dentry *upper,
struct dentry *origin, bool verify);
int ovl_path_next(int idx, struct dentry *dentry, struct path *path,
@@ -897,7 +896,7 @@ static inline void ovl_copyflags(struct inode *from, struct inode *to)
/* dir.c */
extern const struct inode_operations ovl_dir_inode_operations;
int ovl_cleanup_and_whiteout(struct ovl_fs *ofs, struct dentry *dir,
- struct dentry *dentry);
+ struct dentry *dentry, const struct qstr *name);
struct ovl_cattr {
dev_t rdev;
umode_t mode;
diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c
index 80366424b3dc..e23331212a29 100644
--- a/fs/overlayfs/readdir.c
+++ b/fs/overlayfs/readdir.c
@@ -973,7 +973,8 @@ static struct file *ovl_dir_open_realfile(const struct file *file,
const struct path *realpath)
{
with_ovl_creds(file_inode(file)->i_sb)
- return ovl_path_open(realpath, O_RDONLY | (file->f_flags & O_LARGEFILE));
+ return ovl_path_open_realfile(file, realpath,
+ O_RDONLY | (file->f_flags & O_LARGEFILE));
}
/*
@@ -1322,7 +1323,8 @@ int ovl_indexdir_cleanup(struct ovl_fs *ofs)
* Whiteout orphan index to block future open by
* handle after overlay nlink dropped to zero.
*/
- err = ovl_cleanup_and_whiteout(ofs, indexdir, index);
+ err = ovl_cleanup_and_whiteout(ofs, indexdir, index,
+ &QSTR_LEN(p->name, p->len));
} else {
/* Cleanup orphan index entries */
err = ovl_cleanup(ofs, indexdir, index);
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 861b0775f9dd..c06bfe9765e2 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -1366,7 +1366,6 @@ static struct dentry *ovl_get_root(struct super_block *sb,
ovl_dentry_set_flag(OVL_E_CONNECTED, root);
ovl_set_upperdata(d_inode(root));
ovl_inode_init(d_inode(root), &oip, ino, fsid);
- WARN_ON(!!IS_CASEFOLDED(d_inode(root)) != ofs->casefold);
ovl_dentry_init_flags(root, upperdentry, oe, DCACHE_OP_WEAK_REVALIDATE);
/* root keeps a reference of upperdentry */
dget(upperdentry);
@@ -1550,6 +1549,17 @@ static int ovl_fill_super_creds(struct fs_context *fc, struct super_block *sb)
if (!sb->s_root)
goto out_free_oe;
+ /*
+ * The casefold flag of an empty layer root may have changed since the
+ * mount options were parsed. The root inode owns oe by now.
+ */
+ if (!!IS_CASEFOLDED(d_inode(sb->s_root)) != ofs->casefold) {
+ pr_err("casefold flag of the root directory has changed\n");
+ dput(sb->s_root);
+ sb->s_root = NULL;
+ return -EINVAL;
+ }
+
return 0;
out_free_oe:
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index b41f4788e4f0..04103a6741ba 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -654,7 +654,8 @@ bool ovl_path_is_whiteout(struct ovl_fs *ofs, const struct path *path)
ovl_path_check_xwhiteout_xattr(ofs, path);
}
-struct file *ovl_path_open(const struct path *path, int flags)
+struct file *ovl_path_open_realfile(const struct file *user_file,
+ const struct path *path, int flags)
{
struct inode *inode = d_inode(path->dentry);
struct mnt_idmap *real_idmap = mnt_idmap(path->mnt);
@@ -682,6 +683,14 @@ struct file *ovl_path_open(const struct path *path, int flags)
if (inode_owner_or_capable(real_idmap, inode))
flags |= O_NOATIME;
+ /*
+ * Directory opens use dentry_open() here, not backing_file_open().
+ * Inherit FMODE_NONOTIFY from a fanotify permission fd so a listener
+ * that also watches the real filesystem does not deadlock.
+ */
+ if (user_file && FMODE_FSNOTIFY_NONE(user_file->f_mode))
+ return dentry_open_nonotify(path, flags, current_cred());
+
return dentry_open(path, flags, current_cred());
}
@@ -1118,7 +1127,7 @@ static void ovl_cleanup_index(struct dentry *dentry)
} else if (ovl_index_all(dentry->d_sb)) {
/* Whiteout orphan index to block future open by handle */
err = ovl_cleanup_and_whiteout(OVL_FS(dentry->d_sb),
- indexdir, index);
+ indexdir, index, &name);
} else {
/* Cleanup orphan index entries */
err = ovl_cleanup(ofs, indexdir, index);
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index a2db957aa799..2eaa79ab1138 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -3314,7 +3314,7 @@ static long do_pagemap_cmd(struct file *file, unsigned int cmd,
return do_pagemap_scan(mm, arg);
default:
- return -EINVAL;
+ return -ENOIOCTLCMD;
}
}
diff --git a/fs/smb/server/mgmt/user_session.c b/fs/smb/server/mgmt/user_session.c
index 1b30e432ebfd..d131809dd08a 100644
--- a/fs/smb/server/mgmt/user_session.c
+++ b/fs/smb/server/mgmt/user_session.c
@@ -89,6 +89,8 @@ static const char *session_state_string(struct ksmbd_session *session)
static const char *session_user_name(struct ksmbd_session *session)
{
+ if (!session->user)
+ return "(Unauthenticated)";
if (user_guest(session->user))
return "(Guest)";
else if (ksmbd_anonymous_user(session->user))
@@ -98,6 +100,8 @@ static const char *session_user_name(struct ksmbd_session *session)
static const char *session_account_type(struct ksmbd_session *session)
{
+ if (!session->user)
+ return "unauthenticated";
if (user_guest(session->user))
return "guest";
if (ksmbd_anonymous_user(session->user))
diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c
index 3f1ab9a20307..1c8037c85de9 100644
--- a/fs/smb/server/smb2pdu.c
+++ b/fs/smb/server/smb2pdu.c
@@ -3097,9 +3097,10 @@ int smb2_tree_disconnect(struct ksmbd_work *work)
goto err_out;
}
+ err = ksmbd_tree_conn_disconnect(sess, tcon);
+
ksmbd_close_tree_conn_fds(work);
- err = ksmbd_tree_conn_disconnect(sess, tcon);
if (err) {
rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED;
goto err_out;
@@ -3452,7 +3453,8 @@ static noinline int smb2_set_stream_name_xattr(const struct path *path,
if (fp->cdoption == FILE_OPEN_LE) {
if (!strcmp(stream_name, "AFP_AfpInfo") &&
test_share_config_flag(fp->tcon->share_conf,
- KSMBD_SHARE_FLAG_TIME_MACHINE)) {
+ KSMBD_SHARE_FLAG_TIME_MACHINE) &&
+ test_tree_conn_flag(fp->tcon, KSMBD_TREE_CONN_FLAG_WRITABLE)) {
/*
* Synthesize an empty AFP_AfpInfo xattr on first access.
* type=0/creator=0 tells macOS to use the file extension
@@ -3478,6 +3480,9 @@ static noinline int smb2_set_stream_name_xattr(const struct path *path,
return -EBADF;
}
+ if (!test_tree_conn_flag(fp->tcon, KSMBD_TREE_CONN_FLAG_WRITABLE))
+ return -EACCES;
+
rc = ksmbd_vfs_setxattr(idmap, path, xattr_stream_name, NULL, 0, 0, false);
if (rc < 0)
pr_err("Failed to store XATTR stream name :%d\n", rc);
@@ -4427,7 +4432,7 @@ int smb2_open(struct ksmbd_work *work)
file_info = FILE_OPENED;
rc = ksmbd_vfs_getattr(&fp->filp->f_path, &stat);
if (rc)
- goto err_out2;
+ goto err_out1;
goto reconnected_fp;
}
@@ -4439,8 +4444,12 @@ int smb2_open(struct ksmbd_work *work)
goto err_out2;
rc = ksmbd_reopen_durable_fd(work, dh_info.fp);
- if (rc)
+ if (rc) {
+ if (rc == -ENOTCONN)
+ rsp->hdr.Status =
+ STATUS_NETWORK_NAME_DELETED;
goto err_out2;
+ }
fp = dh_info.fp;
@@ -4453,7 +4462,7 @@ int smb2_open(struct ksmbd_work *work)
rc = ksmbd_vfs_getattr(&fp->filp->f_path, &stat);
if (rc)
- goto err_out2;
+ goto err_out1;
goto reconnected_fp;
}
@@ -4503,6 +4512,14 @@ int smb2_open(struct ksmbd_work *work)
goto err_out2;
}
}
+
+ /* MS-FSA 2.1.5.1: delete-on-close requires DELETE access */
+ if (req->CreateOptions & FILE_DELETE_ON_CLOSE_LE &&
+ !(smb_map_generic_desired_access(req->DesiredAccess) &
+ FILE_DELETE_LE)) {
+ rc = -EINVAL;
+ goto err_out2;
+ }
}
if (le32_to_cpu(req->CreateDisposition) >
@@ -4772,6 +4789,13 @@ int smb2_open(struct ksmbd_work *work)
maximal_access = daccess;
}
+ /* MAXIMUM_ALLOWED may not have granted the requested DELETE */
+ if (req->CreateOptions & FILE_DELETE_ON_CLOSE_LE &&
+ !(daccess & FILE_DELETE_LE)) {
+ rc = -EACCES;
+ goto err_out;
+ }
+
open_flags = smb2_create_open_flags(file_present, daccess,
req->CreateDisposition,
&may_flags,
@@ -4874,6 +4898,8 @@ int smb2_open(struct ksmbd_work *work)
if (IS_ERR(fp)) {
fput(filp);
rc = PTR_ERR(fp);
+ if (rc == -ENOTCONN)
+ rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED;
fp = NULL;
goto err_out;
}
@@ -5477,11 +5503,20 @@ int smb2_open(struct ksmbd_work *work)
err_out2:
if (!rc) {
- if (!dh_info.replay)
+ if (!dh_info.replay) {
+ smb2_complete_request_open(work);
rc = ksmbd_update_fstate(&work->sess->file_table, fp,
FP_INITED);
- if (!rc)
+ if (rc == -ENOENT)
+ rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED;
+ if (!rc) {
+ work->request_open = fp;
+ work->request_open_chseq =
+ smb3_hdr_channel_sequence(&req->hdr);
+ }
+ } else {
rc = smb2_set_request_open(work, fp, &req->hdr, false, false);
+ }
if (!rc)
rc = ksmbd_iov_pin_rsp(work, (void *)rsp, iov_len);
}
@@ -5496,8 +5531,7 @@ int smb2_open(struct ksmbd_work *work)
rsp->hdr.Status = STATUS_PRIVILEGE_NOT_HELD;
else
rsp->hdr.Status = STATUS_ACCESS_DENIED;
- }
- else if (rc == -ENOENT)
+ } else if (rc == -ENOENT && !rsp->hdr.Status)
rsp->hdr.Status = STATUS_OBJECT_NAME_INVALID;
else if (rc == -EPERM)
rsp->hdr.Status = STATUS_SHARING_VIOLATION;
@@ -7082,17 +7116,8 @@ static int get_file_stream_info(struct ksmbd_work *work,
streamlen *= 2;
kfree(stream_buf);
file_info->StreamNameLength = cpu_to_le32(streamlen);
- /*
- * stream_name_len is the byte length of the xattr's *name*,
- * not its value -- same class of bug ksmbd_stream_eof()
- * (smb2pdu.c) already fixes for EndOfFile/AllocationSize on
- * a stream handle; this enumeration path needs the same
- * real xattr value length, not the name length reused as a
- * size.
- */
- slen = ksmbd_vfs_casexattr_len(file_mnt_idmap(fp->filp),
- path->dentry, stream_name,
- strlen(stream_name) + 1, NULL);
+ slen = ksmbd_vfs_xattr_len(file_mnt_idmap(fp->filp),
+ path->dentry, stream_name);
ssize = slen < 0 ? 0 : (loff_t)slen;
file_info->StreamSize = cpu_to_le64(ssize);
file_info->StreamAllocationSize = cpu_to_le64(ssize);
diff --git a/fs/smb/server/smbacl.c b/fs/smb/server/smbacl.c
index f8bd8e2ed33e..cc3a450e4cb2 100644
--- a/fs/smb/server/smbacl.c
+++ b/fs/smb/server/smbacl.c
@@ -1470,6 +1470,7 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
int i, found = 0;
unsigned int access_bits = 0, denied = 0;
struct smb_ace *others_ace = NULL;
+ bool or_ace = false;
struct posix_acl_entry *pa_entry;
unsigned int sid_type = SIDOWNER;
unsigned short ace_size;
@@ -1502,6 +1503,8 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
is_owner = uid == from_kuid(&init_user_ns, vfsuid_into_kuid(vfsuid));
if (!pdacl->num_aces) {
+ unsigned int allowed = 0;
+
/*
* An empty (present, zero-ACE) DACL grants no access to
* anyone except the object owner's implicit READ_CONTROL
@@ -1510,9 +1513,14 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
* not let trailing bytes after a zero-ACE DACL become an
* implicit grant.
*/
- if (!is_owner ||
- (*pdaccess & ~(FILE_READ_CONTROL_LE | FILE_WRITE_DAC_LE)))
+ if (is_owner)
+ allowed = READ_CONTROL | WRITE_DAC;
+ if (*pdaccess & FILE_MAXIMAL_ACCESS_LE)
+ requested |= allowed;
+ if (!allowed || requested & ~allowed)
rc = -EACCES;
+ else
+ *pdaccess = cpu_to_le32(requested);
goto err_out;
}
@@ -1597,6 +1605,9 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
found = 1;
break;
}
+ if (!compare_sids(&sid_owner_rights, &ace->sid) &&
+ !(ace->flags & INHERIT_ONLY_ACE))
+ or_ace = true;
if (!compare_sids(&sid_everyone, &ace->sid) ||
!compare_sids(&sid_authusers, &ace->sid))
others_ace = ace;
@@ -1618,7 +1629,10 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
else
continue;
- if (id == uid) {
+ /* A group entry grants only to the group's members */
+ if (id == uid &&
+ (pa_entry->e_tag == ACL_USER ||
+ in_group_p(make_kgid(&init_user_ns, id)))) {
mode_to_access_flags(pa_entry->e_perm,
0777,
&access_bits);
@@ -1638,8 +1652,23 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
if (others_ace) {
ace = others_ace;
} else {
- ksmbd_debug(SMB, "Can't find corresponding sid\n");
- rc = -EACCES;
+ unsigned int allowed = 0;
+
+ /*
+ * No ACE names the owner, who may still read and
+ * change the DACL unless OWNER RIGHTS applies
+ * (MS-DTYP 2.5.3.2).
+ */
+ if (is_owner && !or_ace)
+ allowed = READ_CONTROL | WRITE_DAC;
+ if (*pdaccess & FILE_MAXIMAL_ACCESS_LE)
+ requested |= allowed;
+ if (!allowed || requested & ~allowed) {
+ ksmbd_debug(SMB, "Can't find corresponding sid\n");
+ rc = -EACCES;
+ goto err_out;
+ }
+ *pdaccess = cpu_to_le32(requested);
goto err_out;
}
}
@@ -1665,8 +1694,8 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
}
if (granted & ~access_bits) {
- ksmbd_debug(SMB, "Access denied with winACL, granted : %x, access_req : %x\n",
- granted, le32_to_cpu(ace->access_req));
+ ksmbd_debug(SMB, "Access denied with winACL, requested : %x, allowed : %x\n",
+ granted, access_bits);
rc = -EACCES;
goto err_out;
}
diff --git a/fs/smb/server/vfs.c b/fs/smb/server/vfs.c
index 7453b6e72bd4..674e4fcd80cb 100644
--- a/fs/smb/server/vfs.c
+++ b/fs/smb/server/vfs.c
@@ -902,8 +902,8 @@ ssize_t ksmbd_vfs_listxattr(struct dentry *dentry, char **list)
return size;
}
-static ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap,
- struct dentry *dentry, char *xattr_name)
+ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap,
+ struct dentry *dentry, char *xattr_name)
{
return vfs_getxattr(idmap, dentry, xattr_name, NULL, 0);
}
diff --git a/fs/smb/server/vfs.h b/fs/smb/server/vfs.h
index 53044549571a..7cdd49678667 100644
--- a/fs/smb/server/vfs.h
+++ b/fs/smb/server/vfs.h
@@ -109,6 +109,8 @@ ssize_t ksmbd_vfs_getxattr(struct mnt_idmap *idmap,
struct dentry *dentry,
char *xattr_name,
char **xattr_buf);
+ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap,
+ struct dentry *dentry, char *xattr_name);
ssize_t ksmbd_vfs_casexattr_len(struct mnt_idmap *idmap,
struct dentry *dentry, char *attr_name,
int attr_name_len, char *actual_name);
diff --git a/fs/smb/server/vfs_cache.c b/fs/smb/server/vfs_cache.c
index 1efa9f814560..38196befa03a 100644
--- a/fs/smb/server/vfs_cache.c
+++ b/fs/smb/server/vfs_cache.c
@@ -698,16 +698,18 @@ static void __ksmbd_close_fd(struct ksmbd_file_table *ft, struct ksmbd_file *fp)
}
/*
- * Drop fp's strong reference on conn (taken in ksmbd_open_fd() /
- * ksmbd_reopen_durable_fd()). Durable fps that reached the
- * scavenger have already had fp->conn cleared by session_fd_check(),
- * in which case there is nothing to drop here.
+ * Drop fp's strong connection references. Durable fps that reached
+ * the scavenger have already had these cleared by session_fd_check().
*/
if (fp->conn) {
atomic_dec(&fp->conn->stats.open_files_count);
ksmbd_conn_put(fp->conn);
fp->conn = NULL;
}
+ if (fp->tcon) {
+ ksmbd_tree_connect_put(fp->tcon);
+ fp->tcon = NULL;
+ }
if (ksmbd_stream_fd(fp))
kfree(fp->stream.name);
@@ -1331,14 +1333,14 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp)
fp->filp = filp;
/*
- * fp owns a strong reference on fp->conn for as long as fp->conn is
- * non-NULL, so session_fd_check() and __ksmbd_close_fd() never
- * dereference a dangling pointer. Paired with ksmbd_conn_put() in
- * session_fd_check() (durable preserve), in __ksmbd_close_fd()
- * (final close), and on the error paths below.
+ * fp owns strong references on fp->conn and fp->tcon for as long as
+ * they are non-NULL. Paired with the puts in session_fd_check()
+ * (durable preserve), in __ksmbd_close_fd() (final close), and on the
+ * error paths below.
*/
fp->conn = ksmbd_conn_get(work->conn);
fp->tcon = work->tcon;
+ atomic_inc(&fp->tcon->refcount);
fp->share_conf = work->tcon->share_conf;
atomic_inc(&fp->share_conf->refcount);
fp->volatile_id = KSMBD_NO_FID;
@@ -1351,7 +1353,13 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp)
goto err_out;
}
- ret = __open_id(&work->sess->file_table, fp, OPEN_ID_TYPE_VOLATILE_ID);
+ down_read(&work->sess->tree_conns_lock);
+ if (work->tcon->t_state == TREE_CONNECTED)
+ ret = __open_id(&work->sess->file_table, fp,
+ OPEN_ID_TYPE_VOLATILE_ID);
+ else
+ ret = -ENOTCONN;
+ up_read(&work->sess->tree_conns_lock);
if (ret) {
ksmbd_inode_put(fp->f_ci);
goto err_out;
@@ -1361,7 +1369,8 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp)
return fp;
err_out:
- /* fp->conn was set and refcounted before every branch here. */
+ /* fp->conn and fp->tcon were refcounted before every branch here. */
+ ksmbd_tree_connect_put(fp->tcon);
ksmbd_conn_put(fp->conn);
ksmbd_share_config_put(fp->share_conf);
kmem_cache_free(filp_cache, fp);
@@ -1376,8 +1385,9 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp)
*
* Return: 0 on success. The FP_NEW -> FP_INITED transition is special:
* -ENOENT if teardown already unpublished @fp by advancing the state or
- * clearing the volatile id. Other state updates preserve the historical
- * fire-and-forget behavior.
+ * clearing the volatile id. On success it also takes a reference for the
+ * opening request, which must be released with ksmbd_fd_put(). Other state
+ * updates preserve the historical fire-and-forget behavior.
*/
int ksmbd_update_fstate(struct ksmbd_file_table *ft, struct ksmbd_file *fp,
unsigned int state)
@@ -1393,6 +1403,8 @@ int ksmbd_update_fstate(struct ksmbd_file_table *ft, struct ksmbd_file *fp,
ret = -ENOENT;
} else {
fp->f_state = state;
+ if (state == FP_INITED)
+ atomic_inc(&fp->refcount);
ret = 0;
}
write_unlock(&ft->lock);
@@ -1813,6 +1825,7 @@ static bool session_fd_check(struct ksmbd_tree_connect *tcon,
struct oplock_info *op;
struct ksmbd_conn *conn;
struct ksmbd_lock *smb_lock, *tmp_lock;
+ struct ksmbd_tree_connect *tree_conn;
if (!is_reconnectable(fp))
return false;
@@ -1834,6 +1847,7 @@ static bool session_fd_check(struct ksmbd_tree_connect *tcon,
ci = fp->f_ci;
down_write(&ci->m_lock);
conn = fp->conn;
+ tree_conn = fp->tcon;
list_for_each_entry_rcu(op, &ci->m_op_list, op_entry,
lockdep_is_held(&ci->m_lock)) {
if (op->o_fp != fp)
@@ -1869,6 +1883,7 @@ static bool session_fd_check(struct ksmbd_tree_connect *tcon,
/* Drop fp's own reference on conn. */
ksmbd_conn_put(conn);
+ ksmbd_tree_connect_put(tree_conn);
return true;
}
@@ -1972,7 +1987,8 @@ int ksmbd_reopen_durable_fd(struct ksmbd_work *work, struct ksmbd_file *fp)
* concurrent teardown that iterates the table can observe a valid
* volatile_id with fp->conn == NULL and preserve a
* partially-initialized fp. fp owns a strong reference on the new
- * conn (see ksmbd_open_fd()); undo it on __open_id() failure.
+ * conn (see ksmbd_open_fd()); undo it on __open_id() failure. Serialize
+ * publication with tree disconnect just as ksmbd_open_fd() does.
*/
spin_lock(&fp->f_lock);
old_f_state = fp->f_state;
@@ -1982,12 +1998,19 @@ int ksmbd_reopen_durable_fd(struct ksmbd_work *work, struct ksmbd_file *fp)
atomic_inc(&fp->share_conf->refcount);
fp->conn = ksmbd_conn_get(conn);
fp->tcon = work->tcon;
+ atomic_inc(&fp->tcon->refcount);
spin_unlock(&fp->f_lock);
atomic_inc(&conn->stats.open_files_count);
write_unlock(&global_ft.lock);
- __open_id(&work->sess->file_table, fp, OPEN_ID_TYPE_VOLATILE_ID);
- if (!has_file_id(fp->volatile_id)) {
+ down_read(&work->sess->tree_conns_lock);
+ if (work->tcon->t_state == TREE_CONNECTED)
+ ret = __open_id(&work->sess->file_table, fp,
+ OPEN_ID_TYPE_VOLATILE_ID);
+ else
+ ret = -ENOTCONN;
+ up_read(&work->sess->tree_conns_lock);
+ if (ret) {
write_lock(&global_ft.lock);
spin_lock(&fp->f_lock);
fp->conn = NULL;
@@ -1998,7 +2021,10 @@ int ksmbd_reopen_durable_fd(struct ksmbd_work *work, struct ksmbd_file *fp)
atomic_dec(&conn->stats.open_files_count);
write_unlock(&global_ft.lock);
ksmbd_conn_put(conn);
+ ksmbd_tree_connect_put(work->tcon);
ksmbd_share_config_put(work->tcon->share_conf);
+ if (ret != -ENOTCONN)
+ ret = -EBADF;
goto out;
}
diff --git a/fs/udf/balloc.c b/fs/udf/balloc.c
index 30cec5600149..75124188e745 100644
--- a/fs/udf/balloc.c
+++ b/fs/udf/balloc.c
@@ -457,6 +457,7 @@ static void udf_table_free_blocks(struct super_block *sb,
int adsize;
+ eloc.partitionReferenceNum = bloc->partitionReferenceNum;
eloc.logicalBlockNum = start;
elen = EXT_RECORDED_ALLOCATED |
(count << sb->s_blocksize_bits);
@@ -532,7 +533,7 @@ static int udf_table_prealloc_blocks(struct super_block *sb,
eloc.logicalBlockNum, elen, first_block);
}
- if (first_block == eloc.logicalBlockNum) {
+ if (ret > 0 && first_block == eloc.logicalBlockNum) {
epos.offset -= adsize;
alloc_count = (elen >> sb->s_blocksize_bits);
@@ -572,7 +573,7 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb,
uint32_t elen, goal_elen = 0;
struct kernel_lb_addr eloc, goal_eloc;
struct extent_position epos, goal_epos;
- int8_t etype;
+ int8_t etype, goal_etype = 0;
struct udf_inode_info *iinfo = UDF_I(table);
int ret = 0;
@@ -623,7 +624,8 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb,
goal_epos.block = epos.block;
goal_epos.offset = epos.offset - adsize;
goal_eloc = eloc;
- goal_elen = (etype << 30) | elen;
+ goal_elen = elen;
+ goal_etype = etype;
}
}
@@ -647,7 +649,8 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb,
goal_elen -= sb->s_blocksize;
if (goal_elen)
- udf_write_aext(table, &goal_epos, &goal_eloc, goal_elen, 1);
+ udf_write_aext(table, &goal_epos, &goal_eloc,
+ (goal_etype << 30) | goal_elen, 1);
else
udf_delete_aext(table, goal_epos, &freed);
brelse(goal_epos.bh);
diff --git a/fs/udf/super.c b/fs/udf/super.c
index 5351755aca3e..7570506ff2dd 100644
--- a/fs/udf/super.c
+++ b/fs/udf/super.c
@@ -1106,6 +1106,73 @@ static int check_partition_desc(struct super_block *sb,
return 0;
}
+/*
+ * Check the Unallocated Space Table once when it is loaded: the allocator
+ * hands out blocks from the start of each extent and trusts that every
+ * extent covers at least one whole block inside the partition.
+ */
+static int udf_check_unalloc_table(struct super_block *sb,
+ struct inode *table, u32 partition_len)
+{
+ struct extent_position epos = {
+ .block = UDF_I(table)->i_location,
+ .offset = sizeof(struct unallocSpaceEntry),
+ };
+ struct kernel_lb_addr eloc;
+ uint32_t elen, blocks;
+ struct kernel_lb_addr seen_block = {};
+ uint32_t seen_offset = 0;
+ u64 steps = 0, period = 1;
+ int8_t etype;
+ int ret;
+
+ /* The walk below assumes the layout of an Unallocated Space Entry */
+ if (!UDF_I(table)->i_use) {
+ udf_err(sb, "unallocated space table is not an unallocated space entry\n");
+ return -EFSCORRUPTED;
+ }
+
+ while ((ret = udf_next_aext(table, &epos, &eloc, &elen, &etype, 1)) > 0) {
+ blocks = elen >> sb->s_blocksize_bits;
+ if (!blocks || (elen & (sb->s_blocksize - 1)) ||
+ eloc.logicalBlockNum >= partition_len ||
+ blocks > partition_len - eloc.logicalBlockNum) {
+ udf_err(sb, "invalid unallocated space table extent (block %u, length %u)\n",
+ eloc.logicalBlockNum, elen);
+ ret = -EFSCORRUPTED;
+ break;
+ }
+ /*
+ * A chain of allocation extents can lead back to itself, and
+ * then the walk returns the same extents forever. Remember a
+ * position at doubling intervals; a walk that comes back to it
+ * is in a loop.
+ */
+ if (epos.block.logicalBlockNum == seen_block.logicalBlockNum &&
+ epos.block.partitionReferenceNum ==
+ seen_block.partitionReferenceNum &&
+ epos.offset == seen_offset) {
+ udf_err(sb, "unallocated space table loops back on itself\n");
+ ret = -EFSCORRUPTED;
+ break;
+ }
+ if (++steps == period) {
+ seen_block = epos.block;
+ seen_offset = epos.offset;
+ steps = 0;
+ period <<= 1;
+ }
+ if (fatal_signal_pending(current)) {
+ udf_err(sb, "interrupted while checking the unallocated space table\n");
+ ret = -EINTR;
+ break;
+ }
+ cond_resched();
+ }
+ brelse(epos.bh);
+ return ret;
+}
+
static int udf_fill_partdesc_info(struct super_block *sb,
struct partitionDesc *p, int p_index)
{
@@ -1166,6 +1233,16 @@ static int udf_fill_partdesc_info(struct super_block *sb,
p_index);
return PTR_ERR(inode);
}
+ err = udf_check_unalloc_table(sb, inode, map->s_partition_len);
+ if (err) {
+ iput(inode);
+ if (err == -EINTR)
+ return err;
+ if (!sb_rdonly(sb))
+ return -EACCES;
+ UDF_SET_FLAG(sb, UDF_FLAG_RW_INCOMPAT);
+ return 0;
+ }
map->s_uspace.s_table = inode;
map->s_partition_flags |= UDF_PART_FLAG_UNALLOC_TABLE;
udf_debug("unallocSpaceTable (part %d) @ %llu\n",
@@ -2233,8 +2310,9 @@ static int udf_fill_super(struct super_block *sb, struct fs_context *fc)
/*
* EACCES is special - we want to propagate to
* upper layers that we cannot handle RW mount.
+ * EINTR means that a fatal signal is pending.
*/
- if (ret == -EACCES)
+ if (ret == -EACCES || ret == -EINTR)
break;
} else
break;
diff --git a/fs/udf/truncate.c b/fs/udf/truncate.c
index 0990f94b8551..75151a59f0da 100644
--- a/fs/udf/truncate.c
+++ b/fs/udf/truncate.c
@@ -124,8 +124,8 @@ void udf_discard_prealloc(struct inode *inode)
{
struct extent_position epos = {};
struct extent_position prev_epos = {};
- struct kernel_lb_addr eloc;
- uint32_t elen;
+ struct kernel_lb_addr eloc, tmpeloc;
+ uint32_t elen, tmpelen;
uint64_t lbcount = 0;
int8_t etype = -1;
struct udf_inode_info *iinfo = UDF_I(inode);
@@ -141,7 +141,8 @@ void udf_discard_prealloc(struct inode *inode)
/* Find the last extent in the file */
while (1) {
- ret = udf_next_aext(inode, &epos, &eloc, &elen, &tmpetype, 0);
+ ret = udf_next_aext(inode, &epos, &tmpeloc, &tmpelen,
+ &tmpetype, 0);
if (ret < 0)
goto out;
if (ret == 0)
diff --git a/fs/xfs/libxfs/xfs_attr_leaf.c b/fs/xfs/libxfs/xfs_attr_leaf.c
index 02ca50442b46..5b1fd6271420 100644
--- a/fs/xfs/libxfs/xfs_attr_leaf.c
+++ b/fs/xfs/libxfs/xfs_attr_leaf.c
@@ -3042,7 +3042,14 @@ xfs_attr3_leaf_setflag(
#endif
entry = &xfs_attr3_leaf_entryp(leaf)[args->index];
- ASSERT((entry->flags & XFS_ATTR_INCOMPLETE) == 0);
+ if (entry->flags & XFS_ATTR_INCOMPLETE) {
+ if (!(args->op_flags & XFS_DA_OP_RECOVERY)) {
+ ASSERT(args->op_flags & XFS_DA_OP_RECOVERY);
+ return -EFSCORRUPTED;
+ }
+ return 0;
+ }
+
entry->flags |= XFS_ATTR_INCOMPLETE;
xfs_trans_log_buf(args->trans, bp,
XFS_DA_LOGRANGE(leaf, entry, sizeof(*entry)));
diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index d64defeda645..fce2705c67d2 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -4128,6 +4128,8 @@ xfs_bmapi_finish(
* Returns a negative error code on failure, including -ENOSPC when it could not
* allocate any blocks and -ENOSR when it did allocate blocks to convert a
* delalloc range, but those blocks were before the passed in range.
+ * With XFS_BMAPI_CONVERT_ONLY, it stops at a hole or a delalloc extent instead
+ * of allocating blocks, and returns -ENOSR if it stops before mapping anything.
*/
int
xfs_bmapi_write(
@@ -4187,6 +4189,8 @@ xfs_bmapi_write(
*/
ASSERT((flags & (XFS_BMAPI_PREALLOC | XFS_BMAPI_ZERO)) !=
(XFS_BMAPI_PREALLOC | XFS_BMAPI_ZERO));
+ ASSERT(!(flags & XFS_BMAPI_CONVERT_ONLY) ||
+ (flags & XFS_BMAPI_CONVERT));
if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) ||
XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) {
@@ -4235,6 +4239,9 @@ xfs_bmapi_write(
* that we found, if any.
*/
if (need_alloc || wasdelay) {
+ if (flags & XFS_BMAPI_CONVERT_ONLY)
+ break;
+
bma.eof = eof;
bma.conv = !!(flags & XFS_BMAPI_CONVERT);
bma.wasdel = wasdelay;
@@ -4330,10 +4337,13 @@ xfs_bmapi_write(
*
* To avoid a successful return with *nmap set to 0, return the magic
* -ENOSR error code for this particular case so that the caller can
- * handle it.
+ * handle it. The same goes for XFS_BMAPI_CONVERT_ONLY stopping at a
+ * hole or a delalloc extent at the start of the range, in which case
+ * nothing has been changed.
*/
if (!n) {
- ASSERT(bma.nallocs >= *nmap);
+ ASSERT((flags & XFS_BMAPI_CONVERT_ONLY) ? bma.nallocs == 0 :
+ bma.nallocs >= *nmap);
return -ENOSR;
}
*nmap = n;
diff --git a/fs/xfs/libxfs/xfs_bmap.h b/fs/xfs/libxfs/xfs_bmap.h
index d5f2729305fa..76d25e5d5d9c 100644
--- a/fs/xfs/libxfs/xfs_bmap.h
+++ b/fs/xfs/libxfs/xfs_bmap.h
@@ -90,6 +90,13 @@ struct xfs_bmalloca {
/* Try to align allocations to the extent size hint */
#define XFS_BMAPI_EXTSZALIGN (1u << 11)
+/*
+ * Use with XFS_BMAPI_CONVERT to convert unwritten extents without allocating
+ * blocks. Stop at a hole or a delalloc extent: return the mappings found so
+ * far, or -ENOSR if there are none.
+ */
+#define XFS_BMAPI_CONVERT_ONLY (1u << 12)
+
#define XFS_BMAPI_FLAGS \
{ XFS_BMAPI_ENTIRE, "ENTIRE" }, \
{ XFS_BMAPI_METADATA, "METADATA" }, \
@@ -102,7 +109,8 @@ struct xfs_bmalloca {
{ XFS_BMAPI_COWFORK, "COWFORK" }, \
{ XFS_BMAPI_NODISCARD, "NODISCARD" }, \
{ XFS_BMAPI_NORMAP, "NORMAP" },\
- { XFS_BMAPI_EXTSZALIGN, "EXTSZALIGN" }
+ { XFS_BMAPI_EXTSZALIGN, "EXTSZALIGN" }, \
+ { XFS_BMAPI_CONVERT_ONLY, "CONVERT_ONLY" }
static inline int xfs_bmapi_aflag(int w)
diff --git a/fs/xfs/libxfs/xfs_btree.c b/fs/xfs/libxfs/xfs_btree.c
index 60ef7f08b1d3..b978ddc13c15 100644
--- a/fs/xfs/libxfs/xfs_btree.c
+++ b/fs/xfs/libxfs/xfs_btree.c
@@ -2567,7 +2567,7 @@ xfs_btree_lshift(
if (XFS_IS_CORRUPT(tcur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
- goto error0;
+ goto error1;
}
error = xfs_btree_decrement(tcur, level, &i);
@@ -2737,7 +2737,7 @@ xfs_btree_rshift(
if (XFS_IS_CORRUPT(tcur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
- goto error0;
+ goto error1;
}
error = xfs_btree_increment(tcur, level, &i);
diff --git a/fs/xfs/scrub/attr.c b/fs/xfs/scrub/attr.c
index 842d65834afb..f909e2b25158 100644
--- a/fs/xfs/scrub/attr.c
+++ b/fs/xfs/scrub/attr.c
@@ -293,6 +293,78 @@ xchk_xattr_set_map(
return ret;
}
+static unsigned int
+xchk_xattr_local_entry(
+ struct xchk_da_btree *ds,
+ int level,
+ char *buf_end,
+ struct xfs_attr_leafblock *leaf,
+ int idx)
+{
+ struct xfs_attr_leaf_name_local *lentry =
+ xfs_attr3_leaf_name_local(leaf, idx);
+ char *name_end;
+ unsigned int namesize;
+
+ name_end = (char *)lentry + offsetof(typeof(*lentry), nameval[0]);
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ namesize = xfs_attr_leaf_entsize_local(lentry->namelen,
+ be16_to_cpu(lentry->valuelen));
+ name_end = (char *)lentry + namesize;
+ if (lentry->namelen == 0) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ return namesize;
+}
+
+static unsigned int
+xchk_xattr_remote_entry(
+ struct xchk_da_btree *ds,
+ int level,
+ char *buf_end,
+ struct xfs_attr_leafblock *leaf,
+ int idx,
+ struct xfs_attr_leaf_entry *ent)
+{
+ struct xfs_attr_leaf_name_remote *rentry =
+ xfs_attr3_leaf_name_remote(leaf, idx);
+ char *name_end;
+ unsigned int namesize;
+
+ name_end = (char *)rentry + offsetof(typeof(*rentry), name[0]);
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ namesize = xfs_attr_leaf_entsize_remote(rentry->namelen);
+ name_end = (char *)rentry + namesize;
+ if (rentry->namelen == 0) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+ if (rentry->valueblk == 0 && !(ent->flags & XFS_ATTR_INCOMPLETE)) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ return namesize;
+}
+
/*
* Check this leaf entry's relations to everything else.
* Returns the number of bytes used for the name/value data.
@@ -311,9 +383,6 @@ xchk_xattr_entry(
{
struct xfs_mount *mp = ds->state->mp;
struct xchk_xattr_buf *ab = ds->sc->buf;
- char *name_end;
- struct xfs_attr_leaf_name_local *lentry;
- struct xfs_attr_leaf_name_remote *rentry;
unsigned int nameidx;
unsigned int namesize;
@@ -333,25 +402,14 @@ xchk_xattr_entry(
}
/* Check the name information. */
- if (ent->flags & XFS_ATTR_LOCAL) {
- lentry = xfs_attr3_leaf_name_local(leaf, idx);
- namesize = xfs_attr_leaf_entsize_local(lentry->namelen,
- be16_to_cpu(lentry->valuelen));
- name_end = (char *)lentry + namesize;
- if (lentry->namelen == 0)
- xchk_da_set_corrupt(ds, level);
- } else {
- rentry = xfs_attr3_leaf_name_remote(leaf, idx);
- namesize = xfs_attr_leaf_entsize_remote(rentry->namelen);
- name_end = (char *)rentry + namesize;
- if (rentry->namelen == 0)
- xchk_da_set_corrupt(ds, level);
- if (rentry->valueblk == 0 &&
- !(ent->flags & XFS_ATTR_INCOMPLETE))
- xchk_da_set_corrupt(ds, level);
- }
- if (name_end > buf_end)
- xchk_da_set_corrupt(ds, level);
+ if (ent->flags & XFS_ATTR_LOCAL)
+ namesize = xchk_xattr_local_entry(ds, level, buf_end, leaf,
+ idx);
+ else
+ namesize = xchk_xattr_remote_entry(ds, level, buf_end, leaf,
+ idx, ent);
+ if (!namesize)
+ return;
if (!xchk_xattr_set_map(ds->sc, ab->usedmap, nameidx, namesize))
xchk_da_set_corrupt(ds, level);
diff --git a/fs/xfs/scrub/attr_repair.c b/fs/xfs/scrub/attr_repair.c
index c7695d4cc8e7..ad688f9f7ce9 100644
--- a/fs/xfs/scrub/attr_repair.c
+++ b/fs/xfs/scrub/attr_repair.c
@@ -285,6 +285,7 @@ xrep_xattr_salvage_local_attr(
struct xfs_attr_leaf_name_local *lentry)
{
struct xchk_xattr_buf *ab = rx->sc->buf;
+ char *name_end;
unsigned char *value;
unsigned int valuelen;
unsigned int namesize;
@@ -293,10 +294,14 @@ xrep_xattr_salvage_local_attr(
* Decode the leaf local entry format. If something seems wrong, we
* junk the attribute.
*/
+ name_end = (char *)lentry + offsetof(typeof(*lentry), nameval[0]);
+ if (name_end > buf_end)
+ return 0;
value = &lentry->nameval[lentry->namelen];
valuelen = be16_to_cpu(lentry->valuelen);
namesize = xfs_attr_leaf_entsize_local(lentry->namelen, valuelen);
- if ((char *)lentry + namesize > buf_end)
+ name_end = (char *)lentry + namesize;
+ if (name_end > buf_end)
return 0;
if (!xrep_xattr_want_salvage(rx, ent->flags, lentry->nameval,
lentry->namelen, value, valuelen))
@@ -333,8 +338,8 @@ xrep_xattr_salvage_remote_attr(
.attr_filter = ent->flags & XFS_ATTR_NSP_ONDISK_MASK,
.namelen = rentry->namelen,
.name = rentry->name,
- .valuelen = be32_to_cpu(rentry->valuelen),
};
+ char *name_end;
unsigned int namesize;
int error;
@@ -343,9 +348,14 @@ xrep_xattr_salvage_remote_attr(
* junk the attribute. Note that we should never find a zero-length
* remote attribute value.
*/
+ name_end = (char *)rentry + offsetof(typeof(*rentry), name[0]);
+ if (name_end > buf_end)
+ return 0;
namesize = xfs_attr_leaf_entsize_remote(rentry->namelen);
- if ((char *)rentry + namesize > buf_end)
+ name_end = (char *)rentry + namesize;
+ if (name_end > buf_end)
return 0;
+ args.valuelen = be32_to_cpu(rentry->valuelen);
if (args.valuelen == 0 ||
!xrep_xattr_want_salvage(rx, ent->flags, rentry->name,
rentry->namelen, NULL, args.valuelen))
diff --git a/fs/xfs/scrub/dir.c b/fs/xfs/scrub/dir.c
index 4ab7e6469248..05fe8bc71427 100644
--- a/fs/xfs/scrub/dir.c
+++ b/fs/xfs/scrub/dir.c
@@ -313,6 +313,74 @@ xchk_dir_actor(
return error;
}
+/*
+ * Deal with an unused directory entry. Returns a positive number of bytes to
+ * advance in the directory block, zero to stop processing, or -1 if corruption
+ * was found.
+ */
+static int
+xchk_dir_check_unused(
+ struct xchk_da_btree *ds,
+ struct xfs_buf *bp,
+ unsigned int iter_off,
+ unsigned int end,
+ xfs_dablk_t rec_bno)
+{
+ struct xfs_dir2_data_unused *dup = bp->b_addr + iter_off;
+ unsigned int advance;
+
+ if (iter_off + sizeof(*dup) > end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ advance = xfs_dir2_data_unusedsize(be16_to_cpu(dup->length));
+ if (!advance || iter_off + advance > end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ return advance;
+}
+
+/*
+ * Check a directory entry. Returns a positive number of bytes to advance
+ * in the directory block, zero to stop processing, or -1 if corruption was
+ * found.
+ */
+static int
+xchk_dir_check_entry(
+ struct xchk_da_btree *ds,
+ struct xfs_buf *bp,
+ unsigned int iter_off,
+ unsigned int end,
+ struct xfs_dir2_data_entry *dent,
+ xfs_dablk_t rec_bno)
+{
+ struct xfs_dir2_data_entry *dep = bp->b_addr + iter_off;
+ struct xfs_mount *mp = ds->state->mp;
+ unsigned int advance;
+
+ /* must have namelen */
+ advance = offsetof(typeof(*dep), name);
+ if (iter_off + advance >= end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ /* Don't walk off the end of the block. */
+ advance = xfs_dir2_data_entsize(mp, dep->namelen);
+ if (!advance || iter_off + advance > end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ if (dep == dent)
+ return 0;
+
+ return advance;
+}
+
/* Scrub a directory btree record. */
STATIC int
xchk_dir_rec(
@@ -338,6 +406,7 @@ xchk_dir_rec(
xfs_dahash_t hash;
struct xfs_dir3_icleaf_hdr hdr;
unsigned int tag;
+ bool foundit = false;
int error;
ASSERT(blk->magic == XFS_DIR2_LEAF1_MAGIC ||
@@ -382,28 +451,42 @@ xchk_dir_rec(
dent = bp->b_addr + off;
/* Make sure we got a real directory entry. */
- iter_off = geo->data_entry_offset;
end = xfs_dir3_data_end_offset(geo, bp->b_addr);
if (!end) {
xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
goto out_relse;
}
- for (;;) {
- struct xfs_dir2_data_entry *dep = bp->b_addr + iter_off;
+ iter_off = geo->data_entry_offset;
+ while (iter_off < end) {
struct xfs_dir2_data_unused *dup = bp->b_addr + iter_off;
+ int advance;
- if (iter_off >= end) {
+ /* must have freetag */
+ advance = offsetof(typeof(*dup), length);
+ if (iter_off + advance >= end) {
xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
goto out_relse;
}
- if (be16_to_cpu(dup->freetag) == XFS_DIR2_DATA_FREE_TAG) {
- iter_off += be16_to_cpu(dup->length);
- continue;
- }
- if (dep == dent)
+ if (dup->freetag == cpu_to_be16(XFS_DIR2_DATA_FREE_TAG))
+ advance = xchk_dir_check_unused(ds, bp, iter_off, end,
+ rec_bno);
+ else
+ advance = xchk_dir_check_entry(ds, bp, iter_off, end,
+ dent, rec_bno);
+ if (advance < 0)
+ goto out_relse;
+ if (!advance) {
+ foundit = true;
break;
- iter_off += xfs_dir2_data_entsize(mp, dep->namelen);
+ }
+ iter_off += advance;
+ }
+
+ /* Hash tree must point to the exact dirent */
+ if (!foundit) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ goto out_relse;
}
/* Retrieve the entry, sanity check it, and compare hashes. */
@@ -417,6 +500,13 @@ xchk_dir_rec(
goto out_relse;
}
+ /* Name must not overflow end of block */
+ if ((char *)dent->name + dent->namelen >
+ (char *)bp->b_addr + BBTOB(bp->b_length)) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ goto out_relse;
+ }
+
/* Does the directory hash match? */
dname.name = dent->name;
dname.len = dent->namelen;
diff --git a/fs/xfs/scrub/dir_repair.c b/fs/xfs/scrub/dir_repair.c
index bb3f2f462bde..84fe3dda0d42 100644
--- a/fs/xfs/scrub/dir_repair.c
+++ b/fs/xfs/scrub/dir_repair.c
@@ -462,6 +462,63 @@ xrep_dir_salvage_data_entry(
return xrep_dir_salvage_entry(rd, dep->name, dep->namelen, ino);
}
+/*
+ * Deal with an unused directory entry. Returns a positive number of bytes to
+ * advance in the directory block, or zero to stop processing.
+ */
+static int
+xrep_dir_recover_unused(
+ struct xfs_buf *bp,
+ unsigned int offset,
+ unsigned int end)
+{
+ struct xfs_dir2_data_unused *dup = bp->b_addr + offset;
+ unsigned int advance;
+
+ if (offset + sizeof(*dup) > end)
+ return 0;
+
+ advance = xfs_dir2_data_unusedsize(be16_to_cpu(dup->length));
+ if (!advance || offset + advance > end)
+ return 0;
+
+ return advance;
+}
+
+/*
+ * Salvage a directory entry. Returns a positive number of bytes to advance
+ * in the directory block, zero to stop processing, or a negative errno.
+ */
+static int
+xrep_dir_recover_entry(
+ struct xrep_dir *rd,
+ struct xfs_buf *bp,
+ unsigned int offset,
+ unsigned int end)
+{
+ struct xfs_dir2_data_entry *dep = bp->b_addr + offset;
+ unsigned int advance;
+ int error;
+
+ /* must have namelen */
+ advance = offsetof(typeof(*dep), name);
+ if (offset + advance >= end)
+ return 0;
+
+ /* Don't walk off the end of the block. */
+ advance = xfs_dir2_data_entsize(rd->sc->mp, dep->namelen);
+ if (!advance || offset + advance > end)
+ return 0;
+
+ /* Ok, let's save this entry. */
+ error = xrep_dir_salvage_data_entry(rd, dep);
+ if (error < 0)
+ return error;
+
+ return advance;
+}
+
+
/* Try to recover block/data format directory entries. */
STATIC int
xrep_dir_recover_data(
@@ -483,33 +540,25 @@ xrep_dir_recover_data(
while (offset < end) {
struct xfs_dir2_data_unused *dup = bp->b_addr + offset;
- struct xfs_dir2_data_entry *dep = bp->b_addr + offset;
- unsigned int advance;
+ int advance;
if (xchk_should_terminate(rd->sc, &error))
return error;
- /* Skip unused entries. */
- if (be16_to_cpu(dup->freetag) == XFS_DIR2_DATA_FREE_TAG) {
- if (!dup->length)
- break;
- offset += be16_to_cpu(dup->length);
- continue;
- }
+ /* must have freetag */
+ advance = offsetof(typeof(*dup), length);
+ if (offset + advance >= end)
+ break;
- /* Don't walk off the end of the block. */
- advance = xfs_dir2_data_entsize(rd->sc->mp, dep->namelen);
+ if (dup->freetag == cpu_to_be16(XFS_DIR2_DATA_FREE_TAG))
+ advance = xrep_dir_recover_unused(bp, offset, end);
+ else
+ advance = xrep_dir_recover_entry(rd, bp, offset, end);
+ if (advance < 0)
+ return advance;
if (!advance)
break;
offset += advance;
- if (offset > end)
- break;
-
- /* Ok, let's save this entry. */
- error = xrep_dir_salvage_data_entry(rd, dep);
- if (error)
- return error;
-
}
return 0;
@@ -532,6 +581,19 @@ xrep_dir_recover_sf(
hdr = ifp->if_data;
end = (unsigned char *)ifp->if_data + ifp->if_bytes;
+ /* sf header must be big enough for count/i8count */
+ if (!hdr || ifp->if_bytes < 2)
+ return 0;
+
+ /* and large enough for the parent inumber */
+ if (hdr->i8count) {
+ if (ifp->if_bytes < offsetof(struct xfs_dir2_sf_hdr, parent[8]))
+ return 0;
+ } else {
+ if (ifp->if_bytes < offsetof(struct xfs_dir2_sf_hdr, parent[4]))
+ return 0;
+ }
+
ino = xfs_dir2_sf_get_parent_ino(hdr);
trace_xrep_dir_salvaged_parent(rd->sc->ip, ino);
diff --git a/fs/xfs/scrub/quota_repair.c b/fs/xfs/scrub/quota_repair.c
index b4f33f432b12..3be56d73dd66 100644
--- a/fs/xfs/scrub/quota_repair.c
+++ b/fs/xfs/scrub/quota_repair.c
@@ -297,7 +297,6 @@ xrep_quota_block(
xfs_dqid_t id)
{
struct xfs_dqblk *dqblk;
- struct xfs_disk_dquot *ddq;
struct xfs_quotainfo *qi = sc->mp->m_quotainfo;
struct xfs_def_quota *defq = xfs_get_defquota(qi, dqtype);
struct xfs_buf *bp = NULL;
@@ -319,14 +318,21 @@ xrep_quota_block(
break;
case 0:
dqblk = bp->b_addr;
- ddq = &dqblk[0].dd_diskdq;
+ error = 0;
/*
* If there's nothing that would impede a dqiterate, we're
* done.
*/
- if ((ddq->d_type & XFS_DQTYPE_REC_MASK) == dqtype &&
- id == be32_to_cpu(ddq->d_id)) {
+ for (i = 0; i < qi->qi_dqperchunk; i++, dqblk++) {
+ struct xfs_disk_dquot *ddq = &dqblk->dd_diskdq;
+
+ if ((ddq->d_type & XFS_DQTYPE_REC_MASK) != dqtype ||
+ id + i != be32_to_cpu(ddq->d_id))
+ error++;
+ }
+
+ if (!error) {
xfs_trans_brelse(sc->tp, bp);
return 0;
}
@@ -339,7 +345,9 @@ xrep_quota_block(
dqblk = bp->b_addr;
bp->b_ops = &xfs_dquot_buf_ops;
for (i = 0; i < qi->qi_dqperchunk; i++, dqblk++) {
- ddq = &dqblk->dd_diskdq;
+ struct xfs_disk_dquot *ddq = &dqblk->dd_diskdq;
+ bool was_bigtime =
+ !!(ddq->d_type & XFS_DQTYPE_BIGTIME);
trace_xrep_disk_dquot(sc->mp, dqtype, id + i);
@@ -348,8 +356,21 @@ xrep_quota_block(
ddq->d_type = dqtype;
ddq->d_id = cpu_to_be32(id + i);
- if (xfs_has_bigtime(sc->mp) && ddq->d_id)
+ if (xfs_has_bigtime(sc->mp) && ddq->d_id) {
+ /*
+ * If something was broken with this dquot and it was a
+ * non-bigtime dquot, we'll grant everyone a fresh
+ * grace period by zeroing the timer field rather than
+ * try to interpret what might be garbage.
+ */
+ if (!was_bigtime) {
+ ddq->d_btimer = 0;
+ ddq->d_itimer = 0;
+ ddq->d_rtbtimer = 0;
+ }
+
ddq->d_type |= XFS_DQTYPE_BIGTIME;
+ }
xrep_quota_fix_timer(sc->mp, ddq, ddq->d_blk_softlimit,
ddq->d_bcount, &ddq->d_btimer,
diff --git a/fs/xfs/xfs_discard.c b/fs/xfs/xfs_discard.c
index 906221ffe9ba..cdf660344c28 100644
--- a/fs/xfs/xfs_discard.c
+++ b/fs/xfs/xfs_discard.c
@@ -155,6 +155,8 @@ struct xfs_trim_cur {
xfs_agblock_t end;
xfs_extlen_t minlen;
bool by_bno;
+ /* restart point (with count) for the by-length walk */
+ xfs_agblock_t restart_bno;
};
static int
@@ -198,14 +200,15 @@ xfs_trim_gather_extents(
error = xfs_alloc_lookup_le(cur, tcur->start, 0, &i);
if (!error && !i)
error = xfs_alloc_lookup_ge(cur, tcur->start, 0, &i);
- } else if (tcur->start == 0) {
- /* first time through a by-len starts with max length */
- cur = xfs_cntbt_init_cursor(mp, tp, agbp, pag);
- error = xfs_alloc_lookup_ge(cur, 0, tcur->count, &i);
} else {
- /* nth time through a by-len starts where we left off */
+ /*
+ * A by-len walk starts at the last record of the longest
+ * length (restart_bno is NULLAGBLOCK until the first batch
+ * ends) and then where the previous batch stopped.
+ */
cur = xfs_cntbt_init_cursor(mp, tp, agbp, pag);
- error = xfs_alloc_lookup_le(cur, tcur->start, tcur->count, &i);
+ error = xfs_alloc_lookup_le(cur, tcur->restart_bno,
+ tcur->count, &i);
}
if (error)
goto out_del_cursor;
@@ -235,9 +238,15 @@ xfs_trim_gather_extents(
if (--batch <= 0) {
/*
* Update the cursor to point at this extent so we
- * restart the next batch from this extent.
+ * restart the next batch from this extent. The
+ * by-length walk does not visit extents in block
+ * order, so it must keep the start of the range and
+ * restart from restart_bno instead.
*/
- tcur->start = fbno;
+ if (tcur->by_bno)
+ tcur->start = fbno;
+ else
+ tcur->restart_bno = fbno;
tcur->count = flen;
break;
}
@@ -340,6 +349,7 @@ xfs_trim_perag_extents(
.start = start,
.end = end,
.minlen = minlen,
+ .restart_bno = NULLAGBLOCK,
};
int error = 0;
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index d8202da15aca..59bd9a24b3ef 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -668,7 +668,7 @@ xfs_dio_write_end_io(
* they are converted.
*/
if (flags & IOMAP_DIO_UNWRITTEN) {
- error = xfs_iomap_write_unwritten(ip, offset, size, true);
+ error = xfs_iomap_write_unwritten(ip, offset, size, true, 0);
goto out;
}
diff --git a/fs/xfs/xfs_healthmon.c b/fs/xfs/xfs_healthmon.c
index 2bdd747f1ead..05d6f0494139 100644
--- a/fs/xfs/xfs_healthmon.c
+++ b/fs/xfs/xfs_healthmon.c
@@ -114,8 +114,8 @@ xfs_healthmon_attach(
}
refcount_inc(&hm->ref);
- rcu_assign_pointer(mp->m_healthmon, hm);
hm->mount_cookie = (uintptr_t)mp->m_super;
+ rcu_assign_pointer(mp->m_healthmon, hm);
spin_unlock(&xfs_healthmon_lock);
return 0;
diff --git a/fs/xfs/xfs_ioctl32.c b/fs/xfs/xfs_ioctl32.c
index c66e192448a8..e411f9453bfb 100644
--- a/fs/xfs/xfs_ioctl32.c
+++ b/fs/xfs/xfs_ioctl32.c
@@ -123,8 +123,8 @@ xfs_ioctl32_bstat_copyin(
xfs_ioctl32_bstime_copyin(&bstat->bs_atime, &bstat32->bs_atime) ||
xfs_ioctl32_bstime_copyin(&bstat->bs_mtime, &bstat32->bs_mtime) ||
xfs_ioctl32_bstime_copyin(&bstat->bs_ctime, &bstat32->bs_ctime) ||
- get_user(bstat->bs_blocks, &bstat32->bs_size) ||
- get_user(bstat->bs_xflags, &bstat32->bs_size) ||
+ get_user(bstat->bs_blocks, &bstat32->bs_blocks) ||
+ get_user(bstat->bs_xflags, &bstat32->bs_xflags) ||
get_user(bstat->bs_extsize, &bstat32->bs_extsize) ||
get_user(bstat->bs_extents, &bstat32->bs_extents) ||
get_user(bstat->bs_gen, &bstat32->bs_gen) ||
diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c
index 40695d18dac0..aced9a023db1 100644
--- a/fs/xfs/xfs_ioend.c
+++ b/fs/xfs/xfs_ioend.c
@@ -102,7 +102,7 @@ xfs_end_ioend_write(
else if (ioend->io_flags & IOMAP_IOEND_SHARED)
error = xfs_reflink_end_cow(ip, offset, size);
else if (ioend->io_flags & IOMAP_IOEND_UNWRITTEN)
- error = xfs_iomap_write_unwritten(ip, offset, size, false);
+ error = xfs_iomap_write_unwritten(ip, offset, size, false, 0);
if (!error &&
!(ioend->io_flags & IOMAP_IOEND_DIRECT) &&
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index 7c6238fed61e..373ff0e64f14 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -285,6 +285,14 @@ xfs_iomap_write_direct(
ASSERT(count_fsb > 0);
+ /*
+ * xfs_bmapi_write() maps at most XFS_MAX_BMBT_EXTLEN blocks in one
+ * mapping, so don't reserve blocks for more. A pNFS layout range
+ * comes from a client, and its block count may not even fit the
+ * 32-bit reservation. The caller maps the rest with a later call.
+ */
+ count_fsb = min_t(xfs_filblks_t, count_fsb, XFS_MAX_BMBT_EXTLEN);
+
resaligned = xfs_aligned_fsb_count(offset_fsb, count_fsb,
xfs_get_extsz_hint(ip));
if (unlikely(XFS_IS_REALTIME_INODE(ip))) {
@@ -614,7 +622,8 @@ xfs_iomap_write_unwritten(
xfs_inode_t *ip,
xfs_off_t offset,
xfs_off_t count,
- bool update_isize)
+ bool update_isize,
+ uint32_t bmapi_flags)
{
xfs_mount_t *mp = ip->i_mount;
xfs_fileoff_t offset_fsb;
@@ -676,8 +685,21 @@ xfs_iomap_write_unwritten(
*/
nimaps = 1;
error = xfs_bmapi_write(tp, ip, offset_fsb, count_fsb,
- XFS_BMAPI_CONVERT, resblks, &imap,
- &nimaps);
+ XFS_BMAPI_CONVERT | bmapi_flags,
+ resblks, &imap, &nimaps);
+ if (error == -ENOSR && (bmapi_flags & XFS_BMAPI_CONVERT_ONLY)) {
+ /*
+ * Nothing was mapped: the rest of the range starts
+ * with a hole or a delalloc extent. Nothing was
+ * changed either, but xfs_iext_count_extend() may
+ * have logged the inode, and cancelling a dirty
+ * transaction shuts the filesystem down. Commit it,
+ * as xfs_alloc_file_space() does on -ENOSR.
+ */
+ error = xfs_trans_commit(tp);
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+ return error ? error : -EIO;
+ }
if (error)
goto error_on_bmapi_transaction;
diff --git a/fs/xfs/xfs_iomap.h b/fs/xfs/xfs_iomap.h
index f2520a9b3a13..8fc0c6fc8bea 100644
--- a/fs/xfs/xfs_iomap.h
+++ b/fs/xfs/xfs_iomap.h
@@ -15,7 +15,8 @@ struct xfs_zone_alloc_ctx;
int xfs_iomap_write_direct(struct xfs_inode *ip, xfs_fileoff_t offset_fsb,
xfs_fileoff_t count_fsb, unsigned int flags,
struct xfs_bmbt_irec *imap, u64 *sequence);
-int xfs_iomap_write_unwritten(struct xfs_inode *, xfs_off_t, xfs_off_t, bool);
+int xfs_iomap_write_unwritten(struct xfs_inode *ip, xfs_off_t offset,
+ xfs_off_t count, bool update_isize, uint32_t bmapi_flags);
xfs_fileoff_t xfs_iomap_eof_align_last_fsb(struct xfs_inode *ip,
xfs_fileoff_t end_fsb);
diff --git a/fs/xfs/xfs_pnfs.c b/fs/xfs/xfs_pnfs.c
index ab385617003a..7153e7a1e1de 100644
--- a/fs/xfs/xfs_pnfs.c
+++ b/fs/xfs/xfs_pnfs.c
@@ -110,7 +110,15 @@ xfs_fs_map_update_inode(
ip->i_diflags |= XFS_DIFLAG_PREALLOC;
xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
- return xfs_trans_commit(tp);
+ error = xfs_trans_commit(tp);
+ if (error)
+ return error;
+
+ /*
+ * Force the log so that the blocks handed out to the client are
+ * guaranteed to be present even after a server crash.
+ */
+ return xfs_log_force_inode(ip);
}
/*
@@ -134,6 +142,7 @@ xfs_fs_map_blocks(
uint lock_flags;
int error = 0;
u64 seq;
+ bool update_inode = false;
if (xfs_is_shutdown(mp))
return -EIO;
@@ -167,10 +176,9 @@ xfs_fs_map_blocks(
if (!write)
limit = max(limit, round_up(i_size_read(inode),
inode->i_sb->s_blocksize));
- if (offset > limit)
+ if (offset < 0 || offset >= limit)
goto out_unlock;
- if (offset > limit - length)
- length = limit - offset;
+ length = min_t(u64, length, limit - offset);
error = filemap_write_and_wait(inode->i_mapping);
if (error)
@@ -221,20 +229,27 @@ xfs_fs_map_blocks(
end_fsb - offset_fsb, 0, &imap, &seq);
if (error)
goto out_unlock;
+ update_inode = true;
+ } else {
+ xfs_iunlock(ip, lock_flags);
/*
- * Ensure the next transaction is committed synchronously so
- * that the blocks allocated and handed out to the client are
- * guaranteed to be present even after a server crash.
+ * Blocks past EOF that were allocated for buffered writes, such
+ * as speculative preallocation, can be freed by
+ * xfs_free_eofblocks() without breaking the layout unless the
+ * inode is marked as preallocated. Mark it before handing them
+ * out for writing, as for blocks allocated above.
*/
+ if (write && !(ip->i_diflags & XFS_DIFLAG_PREALLOC) &&
+ imap.br_startoff + imap.br_blockcount >
+ XFS_B_TO_FSB(mp, XFS_ISIZE(ip)))
+ update_inode = true;
+ }
+
+ if (update_inode) {
error = xfs_fs_map_update_inode(ip);
- if (!error)
- error = xfs_log_force_inode(ip);
if (error)
goto out_unlock;
-
- } else {
- xfs_iunlock(ip, lock_flags);
}
xfs_iunlock(ip, XFS_IOLOCK_EXCL);
@@ -312,9 +327,7 @@ xfs_fs_commit_blocks(
if (start > size)
continue;
- end = start + maps[i].length;
- if (end > size)
- end = size;
+ end = start + min_t(u64, maps[i].length, size - start);
length = end - start;
if (!length)
@@ -328,7 +341,14 @@ xfs_fs_commit_blocks(
(end - 1) >> PAGE_SHIFT);
WARN_ON_ONCE(error);
- error = xfs_iomap_write_unwritten(ip, start, length, false);
+ /*
+ * A client may only write to blocks that a layout handed out,
+ * and xfs_fs_map_blocks() allocates blocks for any hole in a
+ * write layout, so refuse to allocate blocks for a hole or a
+ * delalloc extent in the committed range.
+ */
+ error = xfs_iomap_write_unwritten(ip, start, length, false,
+ XFS_BMAPI_CONVERT_ONLY);
if (error)
goto out_drop_iolock;
}
diff --git a/include/acpi/acpi_bus.h b/include/acpi/acpi_bus.h
index a10a591c18b2..8b1d329455f7 100644
--- a/include/acpi/acpi_bus.h
+++ b/include/acpi/acpi_bus.h
@@ -32,6 +32,12 @@ acpi_status
acpi_evaluate_integer(acpi_handle handle,
acpi_string pathname,
struct acpi_object_list *arguments, unsigned long long *data);
+acpi_status acpi_evaluate_object_exclusive(acpi_handle handle, acpi_string pathname,
+ struct acpi_object_list *arguments,
+ struct acpi_buffer *buffer);
+acpi_status acpi_evaluate_integer_exclusive(acpi_handle handle, acpi_string pathname,
+ struct acpi_object_list *arguments,
+ unsigned long long *data);
bool acpi_evaluate_reference(acpi_handle handle, acpi_string pathname,
struct acpi_object_list *arguments,
struct acpi_handle_list *list);
@@ -48,6 +54,7 @@ acpi_evaluate_ost(acpi_handle handle, u32 source_event, u32 status_code,
bool acpi_has_method(acpi_handle handle, char *name);
acpi_status acpi_execute_simple_method(acpi_handle handle, char *method,
u64 arg);
+acpi_status acpi_execute_simple_method_exclusive(acpi_handle handle, char *method, u64 arg);
acpi_status acpi_evaluate_ej0(acpi_handle handle);
acpi_status acpi_evaluate_lck(acpi_handle handle, int lock);
acpi_status acpi_evaluate_reg(acpi_handle handle, u8 space_id, u32 function);
diff --git a/include/acpi/cppc_acpi.h b/include/acpi/cppc_acpi.h
index 94a6277edab2..a32b09ec3394 100644
--- a/include/acpi/cppc_acpi.h
+++ b/include/acpi/cppc_acpi.h
@@ -188,6 +188,7 @@ extern int cppc_set_epp(int cpu, u64 epp_val);
extern int cppc_get_auto_act_window(int cpu, u64 *auto_act_window);
extern int cppc_set_auto_act_window(int cpu, u64 auto_act_window);
extern int cppc_get_auto_sel(int cpu, bool *enable);
+bool cppc_auto_sel_is_immutable(int cpu);
extern int cppc_set_auto_sel(int cpu, bool enable);
extern int cppc_get_perf_limited(int cpu, u64 *perf_limited);
extern int cppc_set_perf_limited(int cpu, u64 bits_to_clear);
@@ -289,6 +290,11 @@ static inline int cppc_get_auto_sel(int cpu, bool *enable)
{
return -EOPNOTSUPP;
}
+static inline bool cppc_auto_sel_is_immutable(int cpu)
+{
+ return false;
+}
+
static inline int cppc_set_auto_sel(int cpu, bool enable)
{
return -EOPNOTSUPP;
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index bdcc2778ac64..32ffffd9d7b9 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -398,8 +398,7 @@ struct mmu_gather {
void tlb_flush_mmu(struct mmu_gather *tlb);
static inline void __tlb_adjust_range(struct mmu_gather *tlb,
- unsigned long address,
- unsigned int range_size)
+ unsigned long address, unsigned long range_size)
{
tlb->start = min(tlb->start, address);
tlb->end = max(tlb->end, address + range_size);
diff --git a/include/crypto/mldsa.h b/include/crypto/mldsa.h
index 3ef2676787c9..b156f166a141 100644
--- a/include/crypto/mldsa.h
+++ b/include/crypto/mldsa.h
@@ -48,11 +48,13 @@ enum mldsa_alg {
* Return:
* * 0 if the signature is valid
* * -EBADMSG if the signature and/or public key is malformed
+ * * -EINVAL if @alg is not a valid value
* * -EKEYREJECTED if the signature is invalid but otherwise well-formed
* * -ENOMEM if out of memory so the validity of the signature is unknown
*/
-int mldsa_verify(enum mldsa_alg alg, const u8 *sig, size_t sig_len,
- const u8 *msg, size_t msg_len, const u8 *pk, size_t pk_len);
+int __must_check mldsa_verify(enum mldsa_alg alg, const u8 *sig, size_t sig_len,
+ const u8 *msg, size_t msg_len, const u8 *pk,
+ size_t pk_len);
#if IS_ENABLED(CONFIG_CRYPTO_LIB_MLDSA_KUNIT_TEST)
/* Internal function, exposed only for unit testing */
diff --git a/include/dt-bindings/clock/nuvoton,ma35d1-clk.h b/include/dt-bindings/clock/nuvoton,ma35d1-clk.h
index ba2d70f776a6..764d837d6162 100644
--- a/include/dt-bindings/clock/nuvoton,ma35d1-clk.h
+++ b/include/dt-bindings/clock/nuvoton,ma35d1-clk.h
@@ -248,6 +248,7 @@
#define ADC_GATE 233
#define EADC_DIV 234
#define EADC_GATE 235
-#define CLK_MAX_IDX 236
+#define PCLK3_DIV4096 236
+#define PCLK4_DIV4096 237
#endif /* __DT_BINDINGS_CLOCK_NUVOTON_MA35D1_CLK_H */
diff --git a/include/linux/assoc_array_priv.h b/include/linux/assoc_array_priv.h
index dca733ef6750..37a9f76e1505 100644
--- a/include/linux/assoc_array_priv.h
+++ b/include/linux/assoc_array_priv.h
@@ -80,7 +80,7 @@ struct assoc_array_edit {
struct assoc_array_ptr **leaf_p;
struct assoc_array_ptr *dead_leaf;
struct assoc_array_ptr *new_meta[3];
- struct assoc_array_ptr *excised_meta[1];
+ struct assoc_array_ptr *excised_meta[2];
struct assoc_array_ptr *excised_subtree;
struct assoc_array_ptr **set_backpointers[ASSOC_ARRAY_FAN_OUT];
struct assoc_array_ptr *set_backpointers_to;
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 4f7905c3412b..3989bb0dd43b 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -675,7 +675,7 @@ enum {
QUEUE_FLAG_HCTX_ACTIVE, /* at least one blk-mq hctx is active */
QUEUE_FLAG_SQ_SCHED, /* single queue style io dispatch */
QUEUE_FLAG_DISABLE_WBT_DEF, /* for sched to disable/enable wbt */
- QUEUE_FLAG_NO_ELV_SWITCH, /* can't switch elevator any more */
+ QUEUE_FLAG_NO_ELV_SWITCH, /* a switch is running or disk is going away */
QUEUE_FLAG_QOS_ENABLED, /* qos is enabled */
QUEUE_FLAG_BIO_ISSUE_TIME, /* record bio->issue_time_ns */
QUEUE_FLAG_ZONED_QD1_WRITES, /* Limit zoned devices writes to QD=1 */
@@ -716,8 +716,6 @@ void blk_queue_flag_clear(unsigned int flag, struct request_queue *q);
((q)->limits.features & BLK_FEAT_SKIP_TAGSET_QUIESCE)
#define blk_queue_disable_wbt(q) \
test_bit(QUEUE_FLAG_DISABLE_WBT_DEF, &(q)->queue_flags)
-#define blk_queue_no_elv_switch(q) \
- test_bit(QUEUE_FLAG_NO_ELV_SWITCH, &(q)->queue_flags)
#define blk_queue_zoned_qd1_writes(q) \
test_bit(QUEUE_FLAG_ZONED_QD1_WRITES, &(q)->queue_flags)
diff --git a/include/linux/can/can-ml.h b/include/linux/can/can-ml.h
index 1e99fda2b380..cfefa4a385be 100644
--- a/include/linux/can/can-ml.h
+++ b/include/linux/can/can-ml.h
@@ -45,6 +45,7 @@
#include <linux/can.h>
#include <linux/list.h>
#include <linux/netdevice.h>
+#include <linux/spinlock.h>
/* exposed CAN device capabilities for network layer */
#define CAN_CAP_CC BIT(0) /* CAN CC aka Classical CAN */
@@ -59,6 +60,7 @@
enum { RX_ERR, RX_ALL, RX_FIL, RX_INV, RX_MAX };
struct can_dev_rcv_lists {
+ spinlock_t lock; /* protects the receive lists and entries */
struct hlist_head rx[RX_MAX];
struct hlist_head rx_sff[CAN_SFF_RCV_ARRAY_SZ];
struct hlist_head rx_eff[CAN_EFF_RCV_ARRAY_SZ];
@@ -81,6 +83,7 @@ static inline struct can_ml_priv *can_get_ml_priv(struct net_device *dev)
static inline void can_set_ml_priv(struct net_device *dev,
struct can_ml_priv *ml_priv)
{
+ spin_lock_init(&ml_priv->dev_rcv_lists.lock);
netdev_set_ml_priv(dev, ml_priv, ML_PRIV_CAN);
}
diff --git a/include/linux/can/dev.h b/include/linux/can/dev.h
index 6d0710d6f571..346faddee02e 100644
--- a/include/linux/can/dev.h
+++ b/include/linux/can/dev.h
@@ -176,7 +176,7 @@ static inline bool can_dev_dropped_skb(struct net_device *dev, struct sk_buff *s
return can_dropped_invalid_skb(dev, skb);
invalid_skb:
- kfree_skb(skb);
+ dev_kfree_skb_any(skb);
dev->stats.tx_dropped++;
return true;
}
diff --git a/include/linux/can/skb.h b/include/linux/can/skb.h
index 5d27843862fc..87ace09c0d8b 100644
--- a/include/linux/can/skb.h
+++ b/include/linux/can/skb.h
@@ -78,12 +78,12 @@ static inline struct sk_buff *can_create_echo_skb(struct sk_buff *skb)
nskb = skb_clone(skb, GFP_ATOMIC);
if (unlikely(!nskb)) {
- kfree_skb(skb);
+ dev_kfree_skb_any(skb);
return NULL;
}
can_skb_set_owner(nskb, skb->sk);
- consume_skb(skb);
+ dev_consume_skb_any(skb);
return nskb;
}
diff --git a/include/linux/damon.h b/include/linux/damon.h
index 0c8b7ddef9ab..9772309b2b58 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -914,6 +914,16 @@ struct damon_ctx {
/* @rnd_state: Per-ctx PRNG state for damon_rand(). */
struct rnd_state rnd_state;
+ /*
+ * number of regions after last merge. Read kdamond_split_regions()
+ * for more details.
+ */
+ unsigned long last_nr_regions;
+ /*
+ * number of damon_split_some_regions() invocations for this context.
+ * Read damon_split_some_regions() for more details.
+ */
+ unsigned long region_split_rotation;
};
/* Get a random number in [@l, @r) using @ctx's lockless PRNG. */
diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index c745f7ad2298..fac05120ab77 100644
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -12,9 +12,8 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr,
struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma);
bool huge_pmd_set_accessed(struct vm_fault *vmf);
-int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm,
- pud_t *dst_pud, pud_t *src_pud, unsigned long addr,
- struct vm_area_struct *vma);
+int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
+ pud_t *dst_pud, pud_t *src_pud, unsigned long addr);
#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud);
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 45ada75dc04e..77d62c930dc3 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -121,10 +121,9 @@ void hugepage_put_subpool(struct hugepage_subpool *spool);
void hugetlb_dup_vma_private(struct vm_area_struct *vma);
void clear_vma_resv_huge_pages(struct vm_area_struct *vma);
-int move_hugetlb_page_tables(struct vm_area_struct *vma,
- struct vm_area_struct *new_vma,
- unsigned long old_addr, unsigned long new_addr,
- unsigned long len);
+unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma,
+ struct vm_area_struct *new_vma, unsigned long old_addr,
+ unsigned long new_addr, unsigned long len);
int copy_hugetlb_page_range(struct mm_struct *, struct mm_struct *,
struct vm_area_struct *, struct vm_area_struct *);
void unmap_hugepage_range(struct vm_area_struct *,
@@ -339,11 +338,9 @@ static inline int copy_hugetlb_page_range(struct mm_struct *dst,
return 0;
}
-static inline int move_hugetlb_page_tables(struct vm_area_struct *vma,
- struct vm_area_struct *new_vma,
- unsigned long old_addr,
- unsigned long new_addr,
- unsigned long len)
+static inline unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma,
+ struct vm_area_struct *new_vma, unsigned long old_addr,
+ unsigned long new_addr, unsigned long len)
{
BUG();
return 0;
diff --git a/include/linux/ksm.h b/include/linux/ksm.h
index d39d0d5483a2..a0d358a1feb6 100644
--- a/include/linux/ksm.h
+++ b/include/linux/ksm.h
@@ -91,8 +91,8 @@ static inline void ksm_exit(struct mm_struct *mm)
* We'd like to make this conditional on vma->vm_flags & VM_MERGEABLE,
* but what if the vma was unmerged while the page was swapped out?
*/
-struct folio *ksm_might_need_to_copy(struct folio *folio,
- struct vm_area_struct *vma, unsigned long addr);
+struct folio *ksm_might_need_to_copy(struct folio *folio, struct page *page,
+ struct vm_area_struct *vma, unsigned long addr);
void rmap_walk_ksm(struct folio *folio, struct rmap_walk_control *rwc);
void folio_migrate_ksm(struct folio *newfolio, struct folio *folio);
@@ -145,7 +145,7 @@ static inline int ksm_madvise(struct vm_area_struct *vma, unsigned long start,
}
static inline struct folio *ksm_might_need_to_copy(struct folio *folio,
- struct vm_area_struct *vma, unsigned long addr)
+ struct page *page, struct vm_area_struct *vma, unsigned long addr)
{
return folio;
}
diff --git a/include/linux/memfd.h b/include/linux/memfd.h
index c159e40e3f34..272479317cf1 100644
--- a/include/linux/memfd.h
+++ b/include/linux/memfd.h
@@ -2,10 +2,19 @@
#ifndef __LINUX_MEMFD_H
#define __LINUX_MEMFD_H
+#include <linux/fcntl.h>
#include <linux/file.h>
#define MEMFD_ANON_NAME "[memfd]"
+static inline int memfd_check_seals_mode(unsigned int seals, umode_t old_mode,
+ umode_t new_mode)
+{
+ if ((seals & F_SEAL_EXEC) && ((old_mode ^ new_mode) & 0111))
+ return -EPERM;
+ return 0;
+}
+
#ifdef CONFIG_MEMFD_CREATE
extern long memfd_fcntl(struct file *file, unsigned int cmd, unsigned int arg);
struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx);
diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h
index a11a44eef521..5db78e5761c5 100644
--- a/include/linux/mmu_notifier.h
+++ b/include/linux/mmu_notifier.h
@@ -483,7 +483,8 @@ mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range)
/*
* This version of mmu_notifier_invalidate_range_start() avoids blocking, but it
* can return an error if a notifier can't proceed without blocking, in which
- * case you're not allowed to modify PTEs in the specified range.
+ * case you're not allowed to modify PTEs in the specified range or call
+ * mmu_notifier_invalidate_range_end().
*
* This is mainly intended for OOM handling.
*/
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 94f9c3ff5416..da8dd04fb462 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -492,6 +492,8 @@ enum lruvec_flags {
* folio->flags, masked by LRU_REFS_MASK.
*/
#define MAX_NR_TIERS 4U
+#define LRU_TIER_MIN 0U
+#define LRU_TIER_MAX (MAX_NR_TIERS - 1)
#ifndef __GENERATING_BOUNDS_H
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index 1a3981c26b23..41b7cdcc674f 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -263,6 +263,12 @@ enum why_no_delegation4 { /* new to v4.1 */
WND4_IS_DIR = 8,
};
+enum stable_how4 {
+ UNSTABLE4 = 0,
+ DATA_SYNC4 = 1,
+ FILE_SYNC4 = 2,
+};
+
enum lock_type4 {
NFS4_UNLOCK_LT = 0,
NFS4_READ_LT = 1,
diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h
index 5663dff53186..842cc0f96147 100644
--- a/include/linux/shmem_fs.h
+++ b/include/linux/shmem_fs.h
@@ -111,8 +111,14 @@ extern unsigned long shmem_get_unmapped_area(struct file *, unsigned long addr,
unsigned long len, unsigned long pgoff, unsigned long flags);
extern int shmem_lock(struct file *file, int lock, struct ucounts *ucounts);
#ifdef CONFIG_SHMEM
+bool shmem_file(struct file *file);
bool shmem_mapping(const struct address_space *mapping);
#else
+static inline bool shmem_file(struct file *file)
+{
+ return false;
+}
+
static inline bool shmem_mapping(const struct address_space *mapping)
{
return false;
@@ -187,15 +193,6 @@ static inline struct page *shmem_read_mapping_page(
mapping_gfp_mask(mapping));
}
-static inline bool shmem_file(struct file *file)
-{
- if (!IS_ENABLED(CONFIG_SHMEM))
- return false;
- if (!file || !file->f_mapping)
- return false;
- return shmem_mapping(file->f_mapping);
-}
-
/* Must be called with inode lock taken exclusive. */
static inline void shmem_freeze(struct inode *inode, bool freeze)
{
diff --git a/include/linux/srcu.h b/include/linux/srcu.h
index 7d9bc06df98d..1c221861ba43 100644
--- a/include/linux/srcu.h
+++ b/include/linux/srcu.h
@@ -18,6 +18,7 @@
#include <linux/mutex.h>
#include <linux/rcupdate.h>
+#include <linux/rculist.h>
#include <linux/workqueue.h>
#include <linux/rcu_segcblist.h>
@@ -103,6 +104,41 @@ void call_srcu(struct srcu_struct *ssp, struct rcu_head *head,
void cleanup_srcu_struct(struct srcu_struct *ssp);
void synchronize_srcu(struct srcu_struct *ssp);
+/**
+ * list_splice_init_srcu - splice an SRCU-protected list into an existing list
+ * @list: the SRCU-protected list to splice
+ * @head: the place in the existing list to splice the first list into
+ * @ssp: the srcu_struct that readers of @list use
+ *
+ * Like list_splice_init_rcu(), but wait for the SRCU readers of @list
+ * instead of RCU ones, so that none of them can follow a next pointer
+ * onto @head.
+ *
+ * Note that this function blocks, and that the caller must prevent any
+ * other updates to @list.
+ */
+static inline void list_splice_init_srcu(struct list_head *list,
+ struct list_head *head,
+ struct srcu_struct *ssp)
+{
+ struct list_head *first = list->next;
+ struct list_head *last = list->prev;
+ struct list_head *next = head->next;
+
+ if (list_empty(list))
+ return;
+
+ INIT_LIST_HEAD_RCU(list);
+ synchronize_srcu(ssp);
+ ASSERT_EXCLUSIVE_ACCESS(*first);
+ ASSERT_EXCLUSIVE_ACCESS(*last);
+
+ last->next = next;
+ rcu_assign_pointer(list_next_rcu(head), first);
+ first->prev = head;
+ next->prev = last;
+}
+
#define SRCU_GET_STATE_COMPLETED 0x1
/**
diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index 3c2b8c355ab3..30344c0d6a9d 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -199,7 +199,8 @@ struct rpc_xprt *rpc_task_get_xprt(struct rpc_clnt *clnt,
int rpcb_create_local(struct net *);
void rpcb_put_local(struct net *);
-int rpcb_register(struct net *, u32, u32, int, unsigned short);
+int rpcb_register(struct net *net, u32 prog, u32 vers, int prot,
+ unsigned short port);
int rpcb_v4_register(struct net *net, const u32 program,
const u32 version,
const struct sockaddr *address,
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index 2db1b9ec5658..5fa9417e034d 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -78,6 +78,7 @@ struct svc_serv {
unsigned int sv_max_payload; /* datagram payload size */
unsigned int sv_max_mesg; /* max_payload + 1 page for overheads */
unsigned int sv_xdrsize; /* XDR buffer size */
+ atomic_t sv_rpcb_failures; /* unanswered rpcbind calls */
struct list_head sv_permsocks; /* all permanent sockets */
struct list_head sv_tempsocks; /* all temporary sockets */
int sv_tmpcnt; /* count of temporary "valid" sockets */
@@ -451,6 +452,7 @@ int sunrpc_set_pool_mode(const char *val);
int sunrpc_get_pool_mode(char *val, size_t size);
void svc_rpcb_cleanup(struct svc_serv *serv, struct net *net);
int svc_bind(struct svc_serv *serv, struct net *net);
+unsigned int svc_rpcb_failure_count(struct svc_serv *serv);
struct svc_serv *svc_create(struct svc_program *, unsigned int,
int (*threadfn)(void *data));
bool svc_rqst_replace_page(struct svc_rqst *rqstp,
@@ -471,8 +473,9 @@ unsigned int svc_serv_maxthreads(const struct svc_serv *serv);
int svc_pool_stats_open(struct svc_info *si, struct file *file);
void svc_process(struct svc_rqst *rqstp);
void svc_process_bc(struct rpc_rqst *req, struct svc_rqst *rqstp);
-int svc_register(const struct svc_serv *, struct net *, const int,
- const unsigned short, const unsigned short);
+int svc_register(struct svc_serv *serv, struct net *net,
+ const int family, const unsigned short proto,
+ const unsigned short port);
void svc_wake_up(struct svc_serv *);
void svc_reserve(struct svc_rqst *rqstp, int space);
diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h
index a4351cffc60c..4c0924c73166 100644
--- a/include/linux/userfaultfd_k.h
+++ b/include/linux/userfaultfd_k.h
@@ -154,8 +154,6 @@ extern int mrwprotect_range(struct userfaultfd_ctx *ctx, unsigned long start,
unsigned long len, bool enable_rwp);
/* move_pages */
-void double_pt_lock(spinlock_t *ptl1, spinlock_t *ptl2);
-void double_pt_unlock(spinlock_t *ptl1, spinlock_t *ptl2);
int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pmd_t dst_pmdval,
struct vm_area_struct *dst_vma,
struct vm_area_struct *src_vma,
diff --git a/include/net/espintcp.h b/include/net/espintcp.h
index c70efd704b6d..7cad3c6822df 100644
--- a/include/net/espintcp.h
+++ b/include/net/espintcp.h
@@ -34,7 +34,8 @@ static inline struct espintcp_ctx *espintcp_getctx(const struct sock *sk)
{
const struct inet_connection_sock *icsk = inet_csk(sk);
- /* RCU is only needed for diag */
- return (__force void *)icsk->icsk_ulp_data;
+ /* pairs with smp_wmb() in espintcp_init_sk() */
+ smp_rmb();
+ return (__force void *)READ_ONCE(icsk->icsk_ulp_data);
}
#endif
diff --git a/include/net/inet_frag.h b/include/net/inet_frag.h
index b02d2472b003..c1fb419a9cf0 100644
--- a/include/net/inet_frag.h
+++ b/include/net/inet_frag.h
@@ -79,6 +79,7 @@ struct frag_v6_compare_key {
* @tstamp_type: stamp has a mono delivery time (EDT)
* @flags: fragment queue flags
* @max_size: maximum received fragment size
+ * @truesize: sum of the truesize of queued fragments
* @fqdir: pointer to struct fqdir
* @rcu: rcu head for freeing deferall
*/
@@ -100,6 +101,7 @@ struct inet_frag_queue {
u8 tstamp_type;
__u8 flags;
u16 max_size;
+ unsigned int truesize;
struct fqdir *fqdir;
struct rcu_head rcu;
};
@@ -172,6 +174,13 @@ extern const u8 ip_frag_ecn_table[16];
#define IPFRAG_DUP 1
#define IPFRAG_OVERLAP 2
#define IPFRAG_MIXED_SECPATH 3
+#define IPFRAG_TOOBIG 4
+
+/* Max sum of the truesize of fragments in a queue. This keeps the sums
+ * done in inet_frag_reasm_prepare() and inet_frag_reasm_finish() well
+ * below INT_MAX. Legitimate traffic stays far below this limit.
+ */
+#define INET_FRAG_MAX_TRUESIZE (INT_MAX / 2)
int inet_frag_queue_insert(struct inet_frag_queue *q, struct sk_buff *skb,
int offset, int end);
void *inet_frag_reasm_prepare(struct inet_frag_queue *q, struct sk_buff *skb,
diff --git a/include/net/netns/can.h b/include/net/netns/can.h
index 48b79f7e6236..54f23681e0d4 100644
--- a/include/net/netns/can.h
+++ b/include/net/netns/can.h
@@ -29,7 +29,7 @@ struct netns_can {
/* receive filters subscribed for 'all' CAN devices */
struct can_dev_rcv_lists *rx_alldev_list;
- spinlock_t rcvlists_lock;
+ spinlock_t rcvlists_lock; /* protects the rcv_entries(_max) stats */
struct timer_list stattimer; /* timer for statistics update */
struct can_pkg_stats *pkg_stats;
struct can_rcv_lists_stats *rcv_lists_stats;
diff --git a/include/net/sctp/sm.h b/include/net/sctp/sm.h
index 3bfd261a53cc..76605d1ee839 100644
--- a/include/net/sctp/sm.h
+++ b/include/net/sctp/sm.h
@@ -252,6 +252,7 @@ struct sctp_chunk *sctp_make_fwdtsn(const struct sctp_association *asoc,
struct sctp_fwdtsn_skip *skiplist);
struct sctp_chunk *sctp_make_auth(const struct sctp_association *asoc,
__u16 key_id);
+void sctp_control_set_owner_w(struct sctp_chunk *chunk);
struct sctp_chunk *sctp_make_strreset_req(const struct sctp_association *asoc,
__u16 stream_num, __be16 *stream_list,
bool out, bool in);
diff --git a/include/sound/cs35l56.h b/include/sound/cs35l56.h
index 747e10ec1a27..ce6383c2c60c 100644
--- a/include/sound/cs35l56.h
+++ b/include/sound/cs35l56.h
@@ -209,6 +209,9 @@ struct snd_ctl_elem_value;
#define CS35L56_GPIO_FN_GPIO 0x00000001
+/* DSP1_FW_VER */
+#define CS35L56_FW_MAIN_VERSION_MASK GENMASK(19, 0)
+
/* Mixer input sources */
#define CS35L56_INPUT_SRC_NONE 0x00
#define CS35L56_INPUT_SRC_ASP1RX1 0x08
@@ -446,6 +449,7 @@ int cs35l56_cal_set_status_get(struct cs35l56_base *cs35l56_base,
int cs35l56_read_prot_status(struct cs35l56_base *cs35l56_base,
bool *fw_missing, unsigned int *fw_version);
void cs35l56_warn_if_firmware_missing(struct cs35l56_base *cs35l56_base);
+bool cs35l56_needs_wait_for_firmware_timer_expiry(struct cs35l56_base *cs35l56_base);
void cs35l56_log_tuning(struct cs35l56_base *cs35l56_base, struct cs_dsp *cs_dsp);
int cs35l56_hw_init(struct cs35l56_base *cs35l56_base);
int cs35l56_get_speaker_id(struct cs35l56_base *cs35l56_base);
diff --git a/include/sound/hwdep.h b/include/sound/hwdep.h
index b0da633184cd..7357727f0660 100644
--- a/include/sound/hwdep.h
+++ b/include/sound/hwdep.h
@@ -57,6 +57,7 @@ struct snd_hwdep {
struct mutex open_mutex;
int used; /* reference counter */
+ struct mutex dsp_mutex; /* serializes DSP load requests */
unsigned int dsp_loaded; /* bit fields of loaded dsp indices */
unsigned int exclusive:1; /* exclusive access mode */
};
diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h
index 5d47d1421c1f..63a86354e822 100644
--- a/include/uapi/linux/iommufd.h
+++ b/include/uapi/linux/iommufd.h
@@ -871,6 +871,12 @@ enum iommufd_hwpt_get_dirty_bitmap_flags {
* Walk the IOMMU pagetables for a given IOVA range to return a bitmap
* with the dirty IOVAs. In doing so it will also by default clear any
* dirty bit metadata set in the IOPTE.
+ *
+ * The IOVA range given must fully enclose any IOPTE. If the range starts or
+ * ends inside a dirty IOPTE then the ioctl will fail. This avoids partially
+ * reporting a dirty range.
+ *
+ * On failure the dirty state becomes unknown and the userspace must restart.
*/
struct iommu_hwpt_get_dirty_bitmap {
__u32 size;
diff --git a/io_uring/cancel.c b/io_uring/cancel.c
index 75a21ecfe69c..6be224ab53dc 100644
--- a/io_uring/cancel.c
+++ b/io_uring/cancel.c
@@ -198,7 +198,8 @@ static int __io_async_cancel(struct io_cancel_data *cd,
break;
if (!all)
return ret;
- nr++;
+ /* poll, futex and waitid return how many they cancelled */
+ nr += max(ret, 1);
} while (1);
/* slow path, try all io-wq's */
@@ -650,7 +651,16 @@ __cold void io_uring_cancel_generic(bool cancel_all, struct io_sq_data *sqd)
}
if (loop) {
+ /*
+ * A request that runs in io-wq and doesn't end makes
+ * each round come here. Don't hold sqd->lock for that
+ * long either.
+ */
+ if (sqd)
+ mutex_unlock(&sqd->lock);
cond_resched();
+ if (sqd)
+ mutex_lock(&sqd->lock);
continue;
}
@@ -678,8 +688,20 @@ __cold void io_uring_cancel_generic(bool cancel_all, struct io_sq_data *sqd)
* avoids a race where a completion comes in before we did
* prepare_to_wait().
*/
- if (inflight == tctx_inflight(tctx, !cancel_all))
+ if (inflight == tctx_inflight(tctx, !cancel_all)) {
+ /*
+ * A zero-copy notification can't be canceled, and a
+ * peer that doesn't read keeps it pending. Don't hold
+ * sqd->lock while waiting for it, or whoever wants the
+ * lock sleeps as long. No ring is used across this,
+ * the next round walks ->ctx_list anew.
+ */
+ if (sqd)
+ mutex_unlock(&sqd->lock);
schedule();
+ if (sqd)
+ mutex_lock(&sqd->lock);
+ }
end_wait:
finish_wait(&tctx->wait, &wait);
} while (1);
diff --git a/io_uring/poll.c b/io_uring/poll.c
index 4f04bda42f7f..fc1800c7097b 100644
--- a/io_uring/poll.c
+++ b/io_uring/poll.c
@@ -128,12 +128,24 @@ static void io_poll_req_insert(struct io_kiocb *req)
hlist_add_head(&req->hash_node, &table->hbs[index].list);
}
+#define IO_POLL_UNMASK (EPOLLERR|EPOLLHUP|EPOLLNVAL)
+
+static __poll_t io_poll_unmask(__poll_t events)
+{
+ /*
+ * A peer that stopped sending is an event for a reader, but a write
+ * only poll must stay armed until the file is writable again.
+ */
+ if (events & (EPOLLIN | EPOLLRDNORM | EPOLLRDBAND))
+ return IO_POLL_UNMASK | EPOLLRDHUP;
+ return IO_POLL_UNMASK;
+}
+
static void io_init_poll_iocb(struct io_poll *poll, __poll_t events)
{
poll->head = NULL;
-#define IO_POLL_UNMASK (EPOLLERR|EPOLLHUP|EPOLLNVAL|EPOLLRDHUP)
/* mask in events that we always want/need */
- poll->events = events | IO_POLL_UNMASK;
+ poll->events = events | io_poll_unmask(events);
INIT_LIST_HEAD(&poll->wait.entry);
init_waitqueue_func_entry(&poll->wait, io_poll_wake);
}
@@ -727,7 +739,7 @@ int io_arm_apoll(struct io_kiocb *req, unsigned issue_flags, __poll_t mask)
int io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags)
{
const struct io_issue_def *def = &io_issue_defs[req->opcode];
- __poll_t mask = POLLPRI | POLLERR;
+ __poll_t mask = EPOLLERR;
if (!def->pollin && !def->pollout)
return IO_APOLL_ABORTED;
@@ -735,7 +747,8 @@ int io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags)
return IO_APOLL_ABORTED;
if (def->pollin) {
- mask |= EPOLLIN | EPOLLRDNORM;
+ /* urgent data is an event for a reader only */
+ mask |= EPOLLIN | EPOLLRDNORM | EPOLLPRI;
/* If reading from MSG_ERRQUEUE using recvmsg, ignore POLLIN */
if (req->flags & REQ_F_CLEAR_POLLIN)
@@ -827,12 +840,47 @@ static int io_poll_disarm(struct io_kiocb *req)
return 0;
}
+/*
+ * Cancel every request matching @cd in one pass. Cancelled requests stay
+ * hashed until their task work runs, so finding them one at a time would
+ * walk past all the earlier ones on each lookup.
+ */
+static int io_poll_cancel_all(struct io_ring_ctx *ctx, struct io_cancel_data *cd)
+{
+ unsigned int nr_buckets = 1U << ctx->cancel_table.hash_bits;
+ unsigned int i = 0;
+ struct hlist_node *tmp;
+ struct io_kiocb *req;
+ int nr = 0;
+
+ if (!(cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP |
+ IORING_ASYNC_CANCEL_ANY))) {
+ /* keyed by user_data, so only one bucket can match */
+ i = hash_long(cd->data, ctx->cancel_table.hash_bits);
+ nr_buckets = i + 1;
+ }
+
+ for (; i < nr_buckets; i++) {
+ struct io_hash_bucket *hb = &ctx->cancel_table.hbs[i];
+
+ hlist_for_each_entry_safe(req, tmp, &hb->list, hash_node) {
+ if (!io_cancel_req_match(req, cd))
+ continue;
+ io_poll_cancel_req(req);
+ nr++;
+ }
+ }
+ return nr ?: -ENOENT;
+}
+
static int __io_poll_cancel(struct io_ring_ctx *ctx, struct io_cancel_data *cd)
{
struct io_kiocb *req;
- if (cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP |
- IORING_ASYNC_CANCEL_ANY))
+ if (cd->flags & (IORING_ASYNC_CANCEL_ALL | IORING_ASYNC_CANCEL_ANY))
+ return io_poll_cancel_all(ctx, cd);
+
+ if (cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP))
req = io_poll_file_find(ctx, cd);
else
req = io_poll_find(ctx, false, cd);
@@ -971,7 +1019,7 @@ int io_poll_remove(struct io_kiocb *req, unsigned int issue_flags)
poll->events &= ~0xffff;
poll->events |= poll_update->events & 0xffff;
- poll->events |= IO_POLL_UNMASK;
+ poll->events |= io_poll_unmask(poll->events);
}
if (poll_update->update_user_data)
preq->cqe.user_data = poll_update->new_user_data;
diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
index d69baa47c7e7..c1ab760c4591 100644
--- a/io_uring/rsrc.c
+++ b/io_uring/rsrc.c
@@ -521,11 +521,12 @@ static int io_files_update_with_index_alloc(struct io_kiocb *req,
{
struct io_rsrc_update *up = io_kiocb_to_cmd(req, struct io_rsrc_update);
__s32 __user *fds = u64_to_user_ptr(up->arg);
+ struct io_ring_ctx *ctx = req->ctx;
unsigned int done;
struct file *file;
int ret, fd;
- if (!req->ctx->file_table.data.nr)
+ if (!ctx->file_table.data.nr)
return -ENXIO;
for (done = 0; done < up->nr_args; done++) {
@@ -539,15 +540,25 @@ static int io_files_update_with_index_alloc(struct io_kiocb *req,
ret = -EBADF;
break;
}
- ret = io_fixed_fd_install(req, issue_flags, file,
- IORING_FILE_INDEX_ALLOC);
- if (ret < 0)
+ /*
+ * Hold the lock across the put_user(), if it's dropped the
+ * slot can be closed and reused before the error path runs.
+ */
+ io_ring_submit_lock(ctx, issue_flags);
+ ret = __io_fixed_fd_install(ctx, file,
+ IORING_FILE_INDEX_ALLOC);
+ if (ret < 0) {
+ io_ring_submit_unlock(ctx, issue_flags);
+ fput(file);
break;
+ }
if (put_user(ret, &fds[done])) {
- __io_close_fixed(req->ctx, issue_flags, ret);
+ io_fixed_fd_remove(ctx, ret);
+ io_ring_submit_unlock(ctx, issue_flags);
ret = -EFAULT;
break;
}
+ io_ring_submit_unlock(ctx, issue_flags);
}
if (done)
diff --git a/io_uring/sqpoll.c b/io_uring/sqpoll.c
index 49c52284ecdd..d387488db2de 100644
--- a/io_uring/sqpoll.c
+++ b/io_uring/sqpoll.c
@@ -29,6 +29,7 @@
enum {
IO_SQ_THREAD_SHOULD_STOP = 0,
IO_SQ_THREAD_SHOULD_PARK,
+ IO_SQ_THREAD_EXITING,
};
void io_sq_thread_unpark(struct io_sq_data *sqd)
@@ -348,6 +349,18 @@ static int io_sq_thread(void *data)
timeout = jiffies + sqd->sq_thread_idle;
}
+ /*
+ * The SQPOLL thread never returns to userspace, so do the
+ * return-to-userspace work, like memory.high reclaim, here.
+ * It can sleep for a long time, so drop sqd->lock.
+ */
+ if (test_thread_flag(TIF_NOTIFY_RESUME)) {
+ mutex_unlock(&sqd->lock);
+ resume_user_mode_work(NULL);
+ mutex_lock(&sqd->lock);
+ sqd->sq_cpu = raw_smp_processor_id();
+ }
+
cap_entries = !list_is_singular(&sqd->ctx_list);
list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) {
int ret;
@@ -433,6 +446,13 @@ static int io_sq_thread(void *data)
if (io_sq_tw_pending())
io_sq_tw(UINT_MAX);
+ /*
+ * Cancelation drops sqd->lock while it waits, and a ring could attach
+ * to a thread that is on its way out. A fatal signal doesn't set the
+ * stop bit, so mark the thread as exiting here for
+ * io_sq_offload_create() to see.
+ */
+ set_bit(IO_SQ_THREAD_EXITING, &sqd->state);
io_uring_cancel_generic(true, sqd);
rcu_assign_pointer(sqd->thread, NULL);
put_task_struct(current);
@@ -502,7 +522,11 @@ __cold int io_sq_offload_create(struct io_ring_ctx *ctx,
list_add(&ctx->sqd_list, &sqd->ctx_list);
io_sqd_update_thread_idle(sqd);
/* don't attach to a dying SQPOLL thread, would be racy */
- ret = (attached && !sqd->thread) ? -ENXIO : 0;
+ ret = 0;
+ if (attached && (!sqd->thread ||
+ test_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state) ||
+ test_bit(IO_SQ_THREAD_EXITING, &sqd->state)))
+ ret = -ENXIO;
io_sq_thread_unpark(sqd);
if (ret < 0)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index c4217c996f5a..9b90b6831de7 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -7634,6 +7634,15 @@ static void perf_sigtrap(struct perf_event *event)
if (event->ctx->task == TASK_TOMBSTONE)
return;
+ /*
+ * The event was removed after this signal was queued, e.g. by exec()
+ * (remove_on_exec), which may also have closed its fd (close-on-exec).
+ * The new program has default signal handlers, so a SIGTRAP would
+ * kill it.
+ */
+ if (READ_ONCE(event->state) <= PERF_EVENT_STATE_EXIT)
+ return;
+
/*
* We'd expect this to only occur if the irq_work is delayed and either
* ctx->task or current has changed in the meantime. This can be the
diff --git a/kernel/fork.c b/kernel/fork.c
index 8ad34646962d..48d7ca064404 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1535,12 +1535,19 @@ static struct mm_struct *dup_mm(struct task_struct *tsk,
if (!mm)
goto fail_nomem;
+ uprobe_start_dup_mmap();
+ if (mmap_write_lock_killable(oldmm)) {
+ free_mm(mm);
+ goto fail_uprobe;
+ }
+
memcpy(mm, oldmm, sizeof(*mm));
- if (!mm_init(mm, tsk))
- goto fail_nomem;
+ if (!mm_init(mm, tsk)) {
+ mmap_write_unlock(oldmm);
+ goto fail_uprobe;
+ }
- uprobe_start_dup_mmap();
err = dup_mmap(mm, oldmm);
if (err)
goto free_pt;
@@ -1559,9 +1566,10 @@ static struct mm_struct *dup_mm(struct task_struct *tsk,
mm->binfmt = NULL;
mm_init_owner(mm, NULL);
mmput(mm);
- if (err)
- uprobe_end_dup_mmap();
-
+ if (!err)
+ return NULL;
+fail_uprobe:
+ uprobe_end_dup_mmap();
fail_nomem:
return NULL;
}
diff --git a/kernel/panic.c b/kernel/panic.c
index cec271601935..f08453f0d8b4 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -893,7 +893,7 @@ static __init int alloc_taint_buf(void)
size += 1; /* For NULL terminator */
- buf = kmalloc(size, GFP_KERNEL);
+ buf = kzalloc(size, GFP_KERNEL);
if (!buf) {
panic("Failed to allocate taint string buffer");
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index b35e23f93bcd..913630c0b059 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7021,8 +7021,8 @@ static void dequeue_throttled_task(struct task_struct *p, int flags)
WARN_ON_ONCE(p->se.on_rq);
list_del_init(&p->throttle_node);
- /* task blocked after throttled */
- if (flags & DEQUEUE_SLEEP) {
+ /* Task blocked after throttling or is leaving the fair class: */
+ if (flags & (DEQUEUE_SLEEP | DEQUEUE_CLASS)) {
p->throttled = false;
return;
}
diff --git a/lib/assoc_array.c b/lib/assoc_array.c
index b6c9723e12ce..3a7cddb1290b 100644
--- a/lib/assoc_array.c
+++ b/lib/assoc_array.c
@@ -470,33 +470,32 @@ static bool assoc_array_insert_in_empty_tree(struct assoc_array_edit *edit)
}
/*
- * Handle insertion into a terminal node.
+ * The pointer that leads to a node or shortcut with the given back pointer
+ * and parent slot: the root, a slot in a node, or a shortcut's next node.
*/
-static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
- const struct assoc_array_ops *ops,
- const void *index_key,
- struct assoc_array_walk_result *result)
+static struct assoc_array_ptr **assoc_array_parent_link(struct assoc_array *array,
+ struct assoc_array_ptr *back_pointer,
+ int parent_slot)
{
- struct assoc_array_shortcut *shortcut, *new_s0;
- struct assoc_array_node *node, *new_n0, *new_n1, *side;
- struct assoc_array_ptr *ptr;
- unsigned long dissimilarity, base_seg, blank;
- size_t keylen;
- bool have_meta;
- int level, diff;
- int slot, next_slot, free_slot, i, j;
-
- node = result->terminal_node.node;
- level = result->terminal_node.level;
- edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = result->terminal_node.slot;
+ if (!back_pointer)
+ return &array->root;
+ if (assoc_array_ptr_is_node(back_pointer))
+ return &assoc_array_ptr_to_node(back_pointer)->slots[parent_slot];
+ return &assoc_array_ptr_to_shortcut(back_pointer)->next_node;
+}
- pr_devel("-->%s()\n", __func__);
+/*
+ * Insert into a terminal node without changing its shape, if that can be
+ * done: in place of a leaf with the same index key, or in a free slot.
+ */
+static bool assoc_array_insert_in_node(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const void *index_key,
+ struct assoc_array_node *node)
+{
+ struct assoc_array_ptr *ptr;
+ int free_slot, i;
- /* We arrived at a node which doesn't have an onward node or shortcut
- * pointer that we have to follow. This means that (a) the leaf we
- * want must go here (either by insertion or replacement) or (b) we
- * need to split this node and insert in one of the fragments.
- */
free_slot = -1;
/* Firstly, we have to check the leaves in this node to see if there's
@@ -514,7 +513,6 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
pr_devel("replace in slot %d\n", i);
edit->leaf_p = &node->slots[i];
edit->dead_leaf = node->slots[i];
- pr_devel("<--%s() = ok [replace]\n", __func__);
return true;
}
}
@@ -526,28 +524,28 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
pr_devel("insert in free slot %d\n", free_slot);
edit->leaf_p = &node->slots[free_slot];
edit->adjust_count_on = node;
- pr_devel("<--%s() = ok [insert]\n", __func__);
return true;
}
- /* The node has no spare slots - so we're either going to have to split
- * it or insert another node before it.
- *
- * Whatever, we're going to need at least two new nodes - so allocate
- * those now. We may also need a new shortcut, but we deal with that
- * when we need it.
- */
- new_n0 = kzalloc_obj(struct assoc_array_node);
- if (!new_n0)
- return false;
- edit->new_meta[0] = assoc_array_node_to_ptr(new_n0);
- new_n1 = kzalloc_obj(struct assoc_array_node);
- if (!new_n1)
- return false;
- edit->new_meta[1] = assoc_array_node_to_ptr(new_n1);
+ return false;
+}
+
+/*
+ * Fill in the edit script's segment cache for a full node: the segment at
+ * this level of each leaf's index key, or 0xff for a metadata pointer.
+ * The new key's segment, in the last entry, is left to the caller. Returns
+ * true if there are any metadata pointers.
+ */
+static bool assoc_array_fill_segment_cache(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const struct assoc_array_node *node,
+ int level)
+{
+ struct assoc_array_ptr *ptr;
+ unsigned long base_seg;
+ bool have_meta;
+ int i;
- /* We need to find out how similar the leaves are. */
- pr_devel("no spare slots\n");
have_meta = false;
for (i = 0; i < ASSOC_ARRAY_FAN_OUT; i++) {
ptr = node->slots[i];
@@ -561,65 +559,42 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK;
edit->segment_cache[i] = base_seg & ASSOC_ARRAY_FAN_MASK;
}
+ return have_meta;
+}
- if (have_meta) {
- pr_devel("have meta\n");
- goto split_node;
- }
-
- /* The node contains only leaves */
- dissimilarity = 0;
- base_seg = edit->segment_cache[0];
- for (i = 1; i < ASSOC_ARRAY_FAN_OUT; i++)
- dissimilarity |= edit->segment_cache[i] ^ base_seg;
-
- pr_devel("only leaves; dissimilarity=%lx\n", dissimilarity);
+/*
+ * Whether the leaves of a full node and the new key all share one segment,
+ * so that splitting the node cannot make room.
+ */
+static bool assoc_array_all_one_segment(const struct assoc_array_edit *edit)
+{
+ int i;
- if ((dissimilarity & ASSOC_ARRAY_FAN_MASK) == 0) {
- /* The old leaves all cluster in the same slot. We will need
- * to insert a shortcut if the new node wants to cluster with them.
- */
- if ((edit->segment_cache[ASSOC_ARRAY_FAN_OUT] ^ base_seg) == 0)
- goto all_leaves_cluster_together;
+ for (i = 1; i < ASSOC_ARRAY_FAN_OUT + 1; i++)
+ if (edit->segment_cache[i] != edit->segment_cache[0])
+ return false;
+ return true;
+}
- /* Otherwise all the old leaves cluster in the same slot, but
- * the new leaf wants to go into a different slot - so we
- * create a new node (n0) to hold the new leaf and a pointer to
- * a new node (n1) holding all the old leaves.
- *
- * This can be done by falling through to the node splitting
- * path.
- */
- pr_devel("present leaves cluster but not new leaf\n");
- }
+/*
+ * Split a full node into new_n0, which takes its place, and new_n1, below
+ * new_n0 in the slot of a segment that at least two of the leaves share,
+ * the new one included. The caller has filled in the segment cache and
+ * set new_n0's back pointer and parent slot.
+ */
+static void assoc_array_split_node(struct assoc_array_edit *edit,
+ struct assoc_array_node *node,
+ struct assoc_array_node *new_n0,
+ struct assoc_array_node *new_n1)
+{
+ struct assoc_array_shortcut *shortcut;
+ struct assoc_array_node *side;
+ struct assoc_array_ptr *ptr;
+ int slot, next_slot, free_slot, i, j;
-split_node:
pr_devel("split node\n");
- /* We need to split the current node. The node must contain anything
- * from a single leaf (in the one leaf case, this leaf will cluster
- * with the new leaf) and the rest meta-pointers, to all leaves, some
- * of which may cluster.
- *
- * It won't contain the case in which all the current leaves plus the
- * new leaves want to cluster in the same slot.
- *
- * We need to expel at least two leaves out of a set consisting of the
- * leaves in the node and the new leaf. The current meta pointers can
- * just be copied as they shouldn't cluster with any of the leaves.
- *
- * We need a new node (n0) to replace the current one and a new node to
- * take the expelled nodes (n1).
- */
- edit->set[0].to = assoc_array_node_to_ptr(new_n0);
- new_n0->back_pointer = node->back_pointer;
- new_n0->parent_slot = node->parent_slot;
new_n1->back_pointer = assoc_array_node_to_ptr(new_n0);
- new_n1->parent_slot = -1; /* Need to calculate this */
-
-do_split_node:
- pr_devel("do_split_node\n");
-
new_n0->nr_leaves_on_branch = node->nr_leaves_on_branch;
new_n1->nr_leaves_on_branch = 0;
@@ -699,26 +674,29 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
}
}
}
+}
- ptr = node->back_pointer;
- if (!ptr)
- edit->set[0].ptr = &edit->array->root;
- else if (assoc_array_ptr_is_node(ptr))
- edit->set[0].ptr = &assoc_array_ptr_to_node(ptr)->slots[node->parent_slot];
- else
- edit->set[0].ptr = &assoc_array_ptr_to_shortcut(ptr)->next_node;
- edit->excised_meta[0] = assoc_array_node_to_ptr(node);
- pr_devel("<--%s() = ok [split node]\n", __func__);
- return true;
+/*
+ * All the leaves, new and old, want to cluster together in this node in the
+ * same slot, so we have to replace this node with a shortcut to skip over
+ * the identical parts of the key, leading to new_n0. If the node is the
+ * target of a shortcut, the new one replaces that shortcut as well. The
+ * caller then splits the leaves between new_n0 and new_n1 at the shortcut's
+ * far end, for which this fills in the segment cache.
+ */
+static struct assoc_array_shortcut *
+assoc_array_insert_shortcut(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const void *index_key,
+ struct assoc_array_node *node, int level,
+ struct assoc_array_node *new_n0)
+{
+ struct assoc_array_shortcut *new_s0;
+ unsigned long base_seg, blank;
+ size_t keylen;
+ int diff, i;
-all_leaves_cluster_together:
- /* All the leaves, new and old, want to cluster together in this node
- * in the same slot, so we have to replace this node with a shortcut to
- * skip over the identical parts of the key and then place a pair of
- * nodes, one inside the other, at the end of the shortcut and
- * distribute the keys between them.
- *
- * Firstly we need to work out where the leaves start diverging as a
+ /* Firstly we need to work out where the leaves start diverging as a
* bit position into their keys so that we know how big the shortcut
* needs to be.
*
@@ -744,17 +722,28 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
new_s0 = kzalloc_flex(*new_s0, index_key, keylen);
if (!new_s0)
- return false;
+ return NULL;
edit->new_meta[2] = assoc_array_shortcut_to_ptr(new_s0);
- edit->set[0].to = assoc_array_shortcut_to_ptr(new_s0);
new_s0->back_pointer = node->back_pointer;
new_s0->parent_slot = node->parent_slot;
+
+ /* A shortcut has to lead to a node, so a node at the end of a shortcut
+ * is replaced together with that shortcut. The new one holds the index
+ * key from the root, so it covers what the old one did.
+ */
+ if (node->back_pointer &&
+ assoc_array_ptr_is_shortcut(node->back_pointer)) {
+ struct assoc_array_shortcut *above =
+ assoc_array_ptr_to_shortcut(node->back_pointer);
+
+ new_s0->back_pointer = above->back_pointer;
+ new_s0->parent_slot = above->parent_slot;
+ edit->excised_meta[1] = node->back_pointer;
+ }
new_s0->next_node = assoc_array_node_to_ptr(new_n0);
new_n0->back_pointer = assoc_array_shortcut_to_ptr(new_s0);
new_n0->parent_slot = 0;
- new_n1->back_pointer = assoc_array_node_to_ptr(new_n0);
- new_n1->parent_slot = -1; /* Need to calculate this */
new_s0->skip_to_level = level = diff & ~ASSOC_ARRAY_LEVEL_STEP_MASK;
pr_devel("skip_to_level = %d [diff %d]\n", level, diff);
@@ -773,18 +762,104 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
/* This now reduces to a node splitting exercise for which we'll need
* to regenerate the disparity table.
*/
- for (i = 0; i < ASSOC_ARRAY_FAN_OUT; i++) {
- ptr = node->slots[i];
- base_seg = ops->get_object_key_chunk(assoc_array_ptr_to_leaf(ptr),
- level);
- base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK;
- edit->segment_cache[i] = base_seg & ASSOC_ARRAY_FAN_MASK;
- }
-
+ assoc_array_fill_segment_cache(edit, ops, node, level);
base_seg = ops->get_key_chunk(index_key, level);
base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK;
edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = base_seg & ASSOC_ARRAY_FAN_MASK;
- goto do_split_node;
+ return new_s0;
+}
+
+/*
+ * Handle insertion into a terminal node.
+ */
+static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const void *index_key,
+ struct assoc_array_walk_result *result)
+{
+ struct assoc_array_shortcut *new_s0;
+ struct assoc_array_node *node, *new_n0, *new_n1;
+ bool have_meta;
+ int level;
+
+ node = result->terminal_node.node;
+ level = result->terminal_node.level;
+
+ pr_devel("-->%s()\n", __func__);
+
+ /* We arrived at a node which doesn't have an onward node or shortcut
+ * pointer that we have to follow. This means that (a) the leaf we
+ * want must go here (either by insertion or replacement) or (b) we
+ * need to split this node and insert in one of the fragments.
+ */
+ if (assoc_array_insert_in_node(edit, ops, index_key, node)) {
+ pr_devel("<--%s() = ok [in node]\n", __func__);
+ return true;
+ }
+
+ /* The node has no spare slots - so we're either going to have to split
+ * it or insert another node before it.
+ *
+ * Whatever, we're going to need at least two new nodes - so allocate
+ * those now. We may also need a new shortcut, but we deal with that
+ * when we need it.
+ */
+ new_n0 = kzalloc_obj(struct assoc_array_node);
+ if (!new_n0)
+ return false;
+ edit->new_meta[0] = assoc_array_node_to_ptr(new_n0);
+ new_n1 = kzalloc_obj(struct assoc_array_node);
+ if (!new_n1)
+ return false;
+ edit->new_meta[1] = assoc_array_node_to_ptr(new_n1);
+
+ /* We need to find out how similar the leaves are. */
+ pr_devel("no spare slots\n");
+ have_meta = assoc_array_fill_segment_cache(edit, ops, node, level);
+ edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = result->terminal_node.slot;
+
+ if (!have_meta && assoc_array_all_one_segment(edit)) {
+ /* The node only contains leaves and all the leaves cluster in
+ * the same slot, so we need to insert a shortcut to skip the
+ * common portion of the key and split the node at the
+ * shortcut's far end. The shortcut goes wherever its back
+ * pointer says, which may be further up than the node.
+ */
+ new_s0 = assoc_array_insert_shortcut(edit, ops, index_key,
+ node, level, new_n0);
+ if (!new_s0)
+ return false;
+ edit->set[0].ptr = assoc_array_parent_link(edit->array,
+ new_s0->back_pointer,
+ new_s0->parent_slot);
+ edit->set[0].to = assoc_array_shortcut_to_ptr(new_s0);
+ } else {
+ /* We need to split the current node. The node must contain
+ * anything from a single leaf (in the one leaf case, this leaf
+ * will cluster with the new leaf) and the rest meta-pointers,
+ * to all leaves, some of which may cluster.
+ *
+ * We need to expel at least two leaves out of a set consisting
+ * of the leaves in the node and the new leaf. The current meta
+ * pointers can just be copied as they shouldn't cluster with
+ * any of the leaves. If all the old leaves cluster in one slot
+ * and the new leaf does not, they all go and the new leaf stays.
+ *
+ * We need a new node (n0) to replace the current one and a new
+ * node to take the expelled nodes (n1).
+ */
+ new_n0->back_pointer = node->back_pointer;
+ new_n0->parent_slot = node->parent_slot;
+ edit->set[0].ptr = assoc_array_parent_link(edit->array,
+ new_n0->back_pointer,
+ new_n0->parent_slot);
+ edit->set[0].to = assoc_array_node_to_ptr(new_n0);
+ }
+
+ assoc_array_split_node(edit, node, new_n0, new_n1);
+ edit->excised_meta[0] = assoc_array_node_to_ptr(node);
+ pr_devel("<--%s() = ok [split node]\n", __func__);
+ return true;
}
/*
@@ -843,7 +918,8 @@ static bool assoc_array_insert_mid_shortcut(struct assoc_array_edit *edit,
* zero length - otherwise we just connect the new node directly to the
* parent.
*/
- level += ASSOC_ARRAY_LEVEL_STEP;
+ if (shortcut->back_pointer)
+ level += ASSOC_ARRAY_LEVEL_STEP;
if (diff > level) {
pr_devel("pre-shortcut %d...%d\n", level, diff);
keylen = round_up(diff, ASSOC_ARRAY_KEY_CHUNK_SIZE);
@@ -865,9 +941,11 @@ static bool assoc_array_insert_mid_shortcut(struct assoc_array_edit *edit,
memcpy(new_s0->index_key, shortcut->index_key,
flex_array_size(new_s0, index_key, keylen));
- blank = ULONG_MAX << (diff & ASSOC_ARRAY_KEY_CHUNK_MASK);
- pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, diff, blank);
- new_s0->index_key[keylen - 1] &= ~blank;
+ if (diff & ASSOC_ARRAY_KEY_CHUNK_MASK) {
+ blank = ULONG_MAX << (diff & ASSOC_ARRAY_KEY_CHUNK_MASK);
+ pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, diff, blank);
+ new_s0->index_key[keylen - 1] &= ~blank;
+ }
} else {
pr_devel("no pre-shortcut\n");
edit->set[0].to = assoc_array_node_to_ptr(new_n0);
@@ -1482,16 +1560,25 @@ int assoc_array_gc(struct assoc_array *array,
cursor = array->root;
descend:
- /* If this point is a shortcut, then we need to duplicate it and
- * advance the target cursor.
+ /* The node at this position is duplicated, and so is the shortcut in
+ * front of it if there is one. The node is allocated first: the copy
+ * of the shortcut starts out pointing at the old node, and must not be
+ * in the new tree if the node cannot be allocated, or the cleanup
+ * would follow it into the old tree.
*/
+ new_n = kzalloc_obj(struct assoc_array_node);
+ if (!new_n)
+ goto enomem;
+
if (assoc_array_ptr_is_shortcut(cursor)) {
shortcut = assoc_array_ptr_to_shortcut(cursor);
keylen = round_up(shortcut->skip_to_level, ASSOC_ARRAY_KEY_CHUNK_SIZE);
keylen >>= ASSOC_ARRAY_KEY_CHUNK_SHIFT;
new_s = kmalloc_flex(*new_s, index_key, keylen);
- if (!new_s)
+ if (!new_s) {
+ kfree(new_n);
goto enomem;
+ }
pr_devel("dup shortcut %p -> %p\n", shortcut, new_s);
memcpy(new_s, shortcut, struct_size(new_s, index_key, keylen));
new_s->back_pointer = new_parent;
@@ -1501,11 +1588,7 @@ int assoc_array_gc(struct assoc_array *array,
cursor = shortcut->next_node;
}
- /* Duplicate the node at this position */
node = assoc_array_ptr_to_node(cursor);
- new_n = kzalloc_obj(struct assoc_array_node);
- if (!new_n)
- goto enomem;
pr_devel("dup node %p -> %p\n", node, new_n);
new_n->back_pointer = new_parent;
new_n->parent_slot = node->parent_slot;
diff --git a/lib/base64.c b/lib/base64.c
index 325c7332b049..e46be3a55585 100644
--- a/lib/base64.c
+++ b/lib/base64.c
@@ -52,11 +52,14 @@ static const char base64_tables[][65] = {
INIT_32(0x60, ch_62, ch_63), \
[0x80 ... 0xff] = -1 }
+__diag_push();
+__diag_ignore(clang, all, "-Wconstant-conversion", "https://github.com/llvm/llvm-project/issues/223923");
static const s8 base64_rev_maps[][256] = {
[BASE64_STD] = BASE64_REV_INIT('+', '/'),
[BASE64_URLSAFE] = BASE64_REV_INIT('-', '_'),
[BASE64_IMAP] = BASE64_REV_INIT('+', ',')
};
+__diag_pop();
#undef BASE64_REV_INIT
#undef INIT_32
diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig
index 56f308dd0273..83b4a211c15d 100644
--- a/lib/crypto/Kconfig
+++ b/lib/crypto/Kconfig
@@ -237,7 +237,7 @@ config CRYPTO_LIB_SHA1_ARCH
depends on CRYPTO_LIB_SHA1 && !UML
default y if ARM
default y if ARM64
- default y if MIPS && CPU_CAVIUM_OCTEON
+ default y if MIPS && CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN
default y if PPC
default y if S390
default y if SPARC64
@@ -255,7 +255,7 @@ config CRYPTO_LIB_SHA256_ARCH
depends on CRYPTO_LIB_SHA256 && !UML
default y if ARM && !CPU_V7M
default y if ARM64
- default y if MIPS && CPU_CAVIUM_OCTEON
+ default y if MIPS && CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN
default y if PPC && SPE
default y if RISCV && 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \
RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS
@@ -275,7 +275,7 @@ config CRYPTO_LIB_SHA512_ARCH
depends on CRYPTO_LIB_SHA512 && !UML
default y if ARM && !CPU_V7M
default y if ARM64
- default y if MIPS && CPU_CAVIUM_OCTEON
+ default y if MIPS && CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN
default y if RISCV && 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \
RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS
default y if S390
diff --git a/lib/crypto/mldsa.c b/lib/crypto/mldsa.c
index c96fddc4e7dc..80e0e63b418a 100644
--- a/lib/crypto/mldsa.c
+++ b/lib/crypto/mldsa.c
@@ -306,7 +306,7 @@ static const u8 *decode_t1_elem(struct mldsa_ring_elem *out,
* Reference: FIPS 204 Algorithm 27, sigDecode.
*
* This also validates that the coefficients of z are in range, corresponding
- * the infinity norm check at the end of Algorithm 8, ML-DSA.Verify_internal.
+ * to the infinity norm check at the end of Algorithm 8, ML-DSA.Verify_internal.
*
* Finally, this also converts z to NTT form.
*/
@@ -528,15 +528,21 @@ static size_t encode_w1(u8 out[MAX_W1_ENCODED_LEN],
int mldsa_verify(enum mldsa_alg alg, const u8 *sig, size_t sig_len,
const u8 *msg, size_t msg_len, const u8 *pk, size_t pk_len)
{
- const struct mldsa_parameter_set *params = &mldsa_parameter_sets[alg];
- const int k = params->k, l = params->l;
+ const struct mldsa_parameter_set *params;
+ int k, l;
/* For now this just does pure ML-DSA with an empty context string. */
static const u8 msg_prefix[2] = { /* dom_sep= */ 0, /* ctx_len= */ 0 };
const u8 *ctilde; /* The signer's commitment hash */
- const u8 *t1_encoded = &pk[RHO_LEN]; /* Next encoded element of t_1 */
+ const u8 *t1_encoded; /* Next encoded element of t_1 */
u8 *h; /* The signer's hint vector, length k * N */
size_t w1_enc_len;
+ if (alg != MLDSA44 && alg != MLDSA65 && alg != MLDSA87)
+ return -EINVAL;
+ params = &mldsa_parameter_sets[alg];
+ k = params->k;
+ l = params->l;
+
/* Validate the public key and signature lengths. */
if (pk_len != params->pk_len || sig_len != params->sig_len)
return -EBADMSG;
@@ -587,6 +593,7 @@ int mldsa_verify(enum mldsa_alg alg, const u8 *sig, size_t sig_len,
* The computation is the same for each of the k rows. Just do each row
* before moving on to the next, resulting in only one loop over k.
*/
+ t1_encoded = &pk[RHO_LEN];
for (int i = 0; i < k; i++) {
/*
* tmp = NTT(A) * NTT(z) * 2^-32
diff --git a/lib/crypto/tests/mldsa_kunit.c b/lib/crypto/tests/mldsa_kunit.c
index ec64e10c97f2..d0ee704a3694 100644
--- a/lib/crypto/tests/mldsa_kunit.c
+++ b/lib/crypto/tests/mldsa_kunit.c
@@ -372,6 +372,14 @@ static void test_mldsa_use_hint(struct kunit *test)
}
}
+static void test_mldsa_invalid_alg(struct kunit *test)
+{
+ KUNIT_ASSERT_EQ(test, -EINVAL,
+ mldsa_verify(MLDSA44 - 1, NULL, 0, NULL, 0, NULL, 0));
+ KUNIT_ASSERT_EQ(test, -EINVAL,
+ mldsa_verify(MLDSA87 + 1, NULL, 0, NULL, 0, NULL, 0));
+}
+
static void benchmark_mldsa(struct kunit *test,
const struct mldsa_testvector *tv)
{
@@ -414,6 +422,7 @@ static struct kunit_case mldsa_kunit_cases[] = {
KUNIT_CASE(test_mldsa65),
KUNIT_CASE(test_mldsa87),
KUNIT_CASE(test_mldsa_use_hint),
+ KUNIT_CASE(test_mldsa_invalid_alg),
KUNIT_CASE(benchmark_mldsa44),
KUNIT_CASE(benchmark_mldsa65),
KUNIT_CASE(benchmark_mldsa87),
diff --git a/mm/cma.c b/mm/cma.c
index a10ea37a261d..9ec1d1fd5dff 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -381,6 +381,8 @@ static int __init cma_fixed_reserve(phys_addr_t base, phys_addr_t size)
return -EBUSY;
}
+ accept_memory(base, size);
+
return 0;
}
@@ -678,6 +680,8 @@ int __init cma_declare_contiguous_multi(phys_addr_t total_size,
break;
}
+ accept_memory(mlp->base, size);
+
pr_debug("created region %d: %016llx - %016llx\n",
nr, (u64)mlp->base, (u64)mlp->base + size);
cmrp = &cma->ranges[nr++];
@@ -1020,7 +1024,7 @@ bool cma_release(struct cma *cma, const struct page *pages,
unsigned long count)
{
struct cma_memrange *cmr;
- unsigned long ret = 0;
+ unsigned long leaked = 0;
unsigned long i, pfn;
cmr = find_cma_memrange(cma, pages, count);
@@ -1029,9 +1033,13 @@ bool cma_release(struct cma *cma, const struct page *pages,
pfn = page_to_pfn(pages);
for (i = 0; i < count; i++, pfn++)
- ret += !put_page_testzero(pfn_to_page(pfn));
+ leaked += !page_ref_freeze(pfn_to_page(pfn), 1);
- WARN(ret, "%lu pages are still in use!\n", ret);
+ if (leaked) {
+ pr_warn_ratelimited("%lu pages are still in use, not freeing CMA region!\n",
+ leaked);
+ return true;
+ }
__cma_release_frozen(cma, cmr, pages, count);
diff --git a/mm/damon/core.c b/mm/damon/core.c
index e018635774be..85bcb6da9ad0 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -3200,6 +3200,26 @@ static void damos_trace_esz(struct damon_ctx *c, struct damos *s,
trace_damos_esz(cidx, sidx, quota->esz);
}
+static void damos_fix_stale_charge_pointers(struct damon_ctx *c,
+ struct damos *s)
+{
+ struct damos_quota *quota;
+ struct damon_target *t;
+
+ quota = &s->quota;
+ if (!quota->charge_target_from)
+ return;
+ damon_for_each_target(t, c) {
+ if (t == quota->charge_target_from) {
+ if (damon_nr_regions(t) > 0)
+ return;
+ break;
+ }
+ }
+ quota->charge_target_from = NULL;
+ quota->charge_addr_from = 0;
+}
+
static void damos_adjust_quota(struct damon_ctx *c, struct damos *s)
{
struct damos_quota *quota = &s->quota;
@@ -3235,6 +3255,8 @@ static void damos_adjust_quota(struct damon_ctx *c, struct damos *s)
damos_trace_esz(c, s, quota);
}
+ damos_fix_stale_charge_pointers(c, s);
+
if (!c->ops.get_scheme_score)
return;
@@ -3546,12 +3568,13 @@ static void damon_split_regions_of(struct damon_ctx *ctx,
static void damon_split_some_regions(struct damon_ctx *ctx,
unsigned long split_step)
{
- static unsigned long rotation;
struct damon_target *t;
struct damon_region *r, *next;
- unsigned long offset = rotation++ % split_step;
+ unsigned long offset;
unsigned long idx = 0;
+ offset = ctx->region_split_rotation++ % split_step;
+
damon_for_each_target(t, ctx) {
damon_for_each_region_safe(r, next, t) {
unsigned long sz_region, sz_sub;
@@ -3588,7 +3611,6 @@ static void kdamond_split_regions(struct damon_ctx *ctx)
struct damon_target *t;
unsigned long nr_regions = 0;
unsigned long max_nr_regions = ctx->attrs.max_nr_regions;
- static unsigned long last_nr_regions;
int nr_subregions = 2;
damon_for_each_target(t, ctx)
@@ -3604,7 +3626,7 @@ static void kdamond_split_regions(struct damon_ctx *ctx)
}
/* Maybe the middle of the region has different access frequency */
- if (last_nr_regions == nr_regions &&
+ if (ctx->last_nr_regions == nr_regions &&
nr_regions < max_nr_regions / 3)
nr_subregions = 3;
@@ -3613,7 +3635,7 @@ static void kdamond_split_regions(struct damon_ctx *ctx)
ctx->min_region_sz);
done:
- last_nr_regions = nr_regions;
+ ctx->last_nr_regions = nr_regions;
}
/*
diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c
index 974c456a9e09..20eb8c242f2f 100644
--- a/mm/damon/sysfs-schemes.c
+++ b/mm/damon/sysfs-schemes.c
@@ -1300,7 +1300,10 @@ static ssize_t target_metric_store(struct kobject *kobj,
metric_name = &damos_sysfs_qgoal_metric_names[i];
if (sysfs_streq(buf, metric_name->name)) {
+ if (!mutex_trylock(&damon_sysfs_lock))
+ return -EBUSY;
goal->metric = metric_name->metric;
+ mutex_unlock(&damon_sysfs_lock);
return count;
}
}
@@ -2831,12 +2834,11 @@ static int damon_sysfs_add_scheme_filters(struct damos *scheme,
} else if (filter->type == DAMOS_FILTER_TYPE_TARGET) {
filter->target_idx = sysfs_filter->target_idx;
} else if (filter->type == DAMOS_FILTER_TYPE_HUGEPAGE_SIZE) {
- if (sysfs_filter->sz_range.min >
- sysfs_filter->sz_range.max) {
+ filter->sz_range = sysfs_filter->sz_range;
+ if (filter->sz_range.min > filter->sz_range.max) {
damos_destroy_filter(filter);
return -EINVAL;
}
- filter->sz_range = sysfs_filter->sz_range;
}
damos_add_filter(scheme, filter);
diff --git a/mm/filemap.c b/mm/filemap.c
index 00fd89cf6f55..73a26cd5b6d5 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -227,18 +227,6 @@ void __filemap_remove_folio(struct folio *folio, void *shadow)
page_cache_delete(mapping, folio, shadow);
}
-static void filemap_free_folio(const struct address_space *mapping,
- struct folio *folio)
-{
- void (*free_folio)(struct folio *);
-
- free_folio = mapping->a_ops->free_folio;
- if (free_folio)
- free_folio(folio);
-
- folio_put_refs(folio, folio_nr_pages(folio));
-}
-
/**
* filemap_remove_folio - Remove folio from page cache.
* @folio: The folio.
@@ -250,6 +238,7 @@ static void filemap_free_folio(const struct address_space *mapping,
void filemap_remove_folio(struct folio *folio)
{
struct address_space *mapping = folio->mapping;
+ void (*free_folio)(struct folio *folio);
BUG_ON(!folio_test_locked(folio));
spin_lock(&mapping->host->i_lock);
@@ -258,9 +247,12 @@ void filemap_remove_folio(struct folio *folio)
xa_unlock_irq(&mapping->i_pages);
if (mapping_shrinkable(mapping))
inode_lru_list_add(mapping->host);
+ free_folio = mapping->a_ops->free_folio;
spin_unlock(&mapping->host->i_lock);
- filemap_free_folio(mapping, folio);
+ if (free_folio)
+ free_folio(folio);
+ folio_put_refs(folio, folio_nr_pages(folio));
}
/*
@@ -320,6 +312,7 @@ static void page_cache_delete_batch(struct address_space *mapping,
void delete_from_page_cache_batch(struct address_space *mapping,
struct folio_batch *fbatch)
{
+ void (*free_folio)(struct folio *folio);
int i;
if (!folio_batch_count(fbatch))
@@ -337,10 +330,16 @@ void delete_from_page_cache_batch(struct address_space *mapping,
xa_unlock_irq(&mapping->i_pages);
if (mapping_shrinkable(mapping))
inode_lru_list_add(mapping->host);
+ free_folio = mapping->a_ops->free_folio;
spin_unlock(&mapping->host->i_lock);
- for (i = 0; i < folio_batch_count(fbatch); i++)
- filemap_free_folio(mapping, fbatch->folios[i]);
+ for (i = 0; i < folio_batch_count(fbatch); i++) {
+ struct folio *folio = fbatch->folios[i];
+
+ if (free_folio)
+ free_folio(folio);
+ folio_put_refs(folio, folio_nr_pages(folio));
+ }
}
int filemap_check_errors(struct address_space *mapping)
@@ -1375,10 +1374,22 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr,
* return value based on that state without races.
*
* Also note that WQ_FLAG_WOKEN is sufficient for a non-exclusive
- * waiter, but an exclusive one requires WQ_FLAG_DONE.
+ * waiter, but an exclusive one must also acquire the bit.
*/
- if (behavior == EXCLUSIVE)
- return wait->flags & WQ_FLAG_DONE ? 0 : -EINTR;
+ if (behavior == EXCLUSIVE) {
+ if (wait->flags & WQ_FLAG_DONE)
+ return 0;
+ /*
+ * We might have just been woken up and the waker will not
+ * wake up anybody else. If we ignore that now, the remaining
+ * waiters may not get woken up: take the bit now if it is
+ * still free.
+ */
+ if ((wait->flags & WQ_FLAG_WOKEN) &&
+ !test_and_set_bit(bit_nr, folio_flags(folio, 0)))
+ return 0;
+ return -EINTR;
+ }
return wait->flags & WQ_FLAG_WOKEN ? 0 : -EINTR;
}
@@ -3279,11 +3290,11 @@ loff_t mapping_seek_hole_data(struct address_space *mapping, loff_t start,
* @folio - the folio to lock.
* @fpin - the pointer to the file we may pin (or is already pinned).
*
- * This works similar to lock_folio_or_retry in that it can drop the
- * mmap_lock. It differs in that it actually returns the folio locked
- * if it returns 1 and 0 if it couldn't lock the folio. If we did have
- * to drop the mmap_lock then fpin will point to the pinned file and
- * needs to be fput()'ed at a later point.
+ * Returns 1 with the folio locked, or 0 if locking failed.
+ *
+ * On both entry and return, a non-NULL *fpin means the fault lock has
+ * already been dropped and the caller must eventually fput() the file.
+ * Otherwise the fault lock is still held, including on failure.
*/
static int lock_folio_maybe_drop_mmap(struct vm_fault *vmf, struct folio *folio,
struct file **fpin)
@@ -3300,21 +3311,9 @@ static int lock_folio_maybe_drop_mmap(struct vm_fault *vmf, struct folio *folio,
return 0;
*fpin = maybe_unlock_mmap_for_io(vmf, *fpin);
- if (vmf->flags & FAULT_FLAG_KILLABLE) {
- if (__folio_lock_killable(folio)) {
- /*
- * We didn't have the right flags to drop the
- * fault lock, but all fault_handlers only check
- * for fatal signals if we return VM_FAULT_RETRY,
- * so we need to drop the fault lock here and
- * return 0 if we don't have a fpin.
- */
- if (*fpin == NULL)
- release_fault_lock(vmf);
- return 0;
- }
- } else
- __folio_lock(folio);
+ if (vmf->flags & FAULT_FLAG_KILLABLE)
+ return !__folio_lock_killable(folio);
+ __folio_lock(folio);
return 1;
}
@@ -3543,7 +3542,7 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf)
* vma->vm_mm->mmap_lock must be held on entry.
*
* If our return value has VM_FAULT_RETRY set, it's because the mmap_lock
- * may be dropped before doing I/O or by lock_folio_maybe_drop_mmap().
+ * may be dropped before doing I/O or when aborting an interrupted wait.
*
* If our return value does not have VM_FAULT_RETRY set, the mmap_lock
* has not been released.
@@ -3699,9 +3698,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf)
out_retry:
/*
- * We dropped the mmap_lock, we need to return to the fault handler to
- * re-find the vma and come back and find our hopefully still populated
- * page.
+ * With fpin set, the fault lock was already dropped. Otherwise it is
+ * still held. Release invalidate_lock before dropping either protection.
*/
if (!IS_ERR(folio))
folio_put(folio);
@@ -3709,6 +3707,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf)
filemap_invalidate_unlock_shared(mapping);
if (fpin)
fput(fpin);
+ else if (!(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
+ release_fault_lock(vmf);
return ret | VM_FAULT_RETRY;
}
EXPORT_SYMBOL(filemap_fault);
diff --git a/mm/gup.c b/mm/gup.c
index eb898ea1ee22..4f58ad22a756 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -658,9 +658,6 @@ static struct page *follow_huge_pud(struct vm_area_struct *vma,
assert_spin_locked(pud_lockptr(mm, pudp));
- if (!pud_present(pud))
- return NULL;
-
if ((flags & FOLL_WRITE) &&
!can_follow_write_pud(pud, pfn_to_page(pfn), vma, flags))
return NULL;
@@ -896,15 +893,17 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
}
static struct page *follow_pmd_mask(struct vm_area_struct *vma,
- unsigned long address, pud_t *pudp,
- unsigned int flags,
- unsigned long *page_mask)
+ unsigned long address, pud_t *pudp, pud_t pud,
+ unsigned int flags, unsigned long *page_mask)
{
pmd_t *pmd, pmdval;
spinlock_t *ptl;
struct page *page;
struct mm_struct *mm = vma->vm_mm;
+ if (unlikely(pud_bad(pud)))
+ return no_page_table(vma, flags, address);
+
pmd = pmd_offset(pudp, address);
pmdval = pmdp_get_lockless(pmd);
if (pmd_none(pmdval))
@@ -953,18 +952,23 @@ static struct page *follow_pud_mask(struct vm_area_struct *vma,
pud = pudp_get(pudp);
if (!pud_present(pud))
return no_page_table(vma, flags, address);
- if (pud_leaf(pud)) {
- ptl = pud_lock(mm, pudp);
- page = follow_huge_pud(vma, address, pudp, flags, page_mask);
+ if (!pud_leaf(pud))
+ return follow_pmd_mask(vma, address, pudp, pud, flags, page_mask);
+
+ ptl = pud_lock(mm, pudp);
+ pud = *pudp;
+ if (unlikely(!pud_present(pud))) {
spin_unlock(ptl);
- if (page)
- return page;
return no_page_table(vma, flags, address);
}
- if (unlikely(pud_bad(pud)))
- return no_page_table(vma, flags, address);
+ if (unlikely(!pud_leaf(pud))) {
+ spin_unlock(ptl);
+ return follow_pmd_mask(vma, address, pudp, pud, flags, page_mask);
+ }
- return follow_pmd_mask(vma, address, pudp, flags, page_mask);
+ page = follow_huge_pud(vma, address, pudp, flags, page_mask);
+ spin_unlock(ptl);
+ return page;
}
static struct page *follow_p4d_mask(struct vm_area_struct *vma,
@@ -3186,6 +3190,9 @@ static int gup_fast_fallback(unsigned long start, unsigned long nr_pages,
FOLL_PCI_P2PDMA | FOLL_HONOR_NUMA_FAULT)))
return -EINVAL;
+ if (unlikely(!nr_pages))
+ return 0;
+
if (gup_flags & FOLL_PIN)
mm_set_has_pinned_flag(current->mm);
@@ -3193,8 +3200,8 @@ static int gup_fast_fallback(unsigned long start, unsigned long nr_pages,
might_lock_read(¤t->mm->mmap_lock);
start = untagged_addr(start) & PAGE_MASK;
- len = nr_pages << PAGE_SHIFT;
- if (check_add_overflow(start, len, &end))
+ if (check_shl_overflow(nr_pages, PAGE_SHIFT, &len) ||
+ check_add_overflow(start, len, &end))
return -EOVERFLOW;
if (end > TASK_SIZE_MAX)
return -EFAULT;
@@ -3439,7 +3446,7 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
{
unsigned int flags, nr_folios, nr_found;
unsigned int i, pgshift = PAGE_SHIFT;
- pgoff_t start_idx, end_idx;
+ pgoff_t start_idx, end_idx, retry_idx;
struct folio *folio = NULL;
struct folio_batch fbatch;
struct hstate *h;
@@ -3480,6 +3487,7 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
* reason, we try to allocate them and add them to the
* page cache.
*/
+ retry_idx = start_idx;
nr_found = filemap_get_folios_contig(memfd->f_mapping,
&start_idx,
end_idx,
@@ -3492,12 +3500,38 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
for (i = 0; i < nr_found; i++) {
folio = fbatch.folios[i];
+ /*
+ * The folio might have been removed from the
+ * page cache (e.g., truncated or collapsed)
+ * after the lookup. Recheck under the folio
+ * lock.
+ */
+ folio_lock(folio);
+ if (unlikely(folio->mapping != memfd->f_mapping)) {
+ folio_unlock(folio);
+ start_idx = retry_idx;
+ break;
+ }
+
if (try_grab_folio(folio, 1, FOLL_PIN)) {
+ folio_unlock(folio);
folio_batch_release(&fbatch);
ret = -EINVAL;
goto err;
}
+ /*
+ * fallocate() can leave folios in the
+ * page cache that were never cleared.
+ */
+ if (!folio_test_uptodate(folio)) {
+ folio_zero_range(folio, 0, folio_size(folio));
+ flush_dcache_folio(folio);
+ folio_mark_uptodate(folio);
+ }
+ retry_idx = folio_next_index(folio);
+ folio_unlock(folio);
+
if (nr_folios == 0)
*offset = offset_in_folio(folio, start);
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 779c02e0bf6c..5f7e3320d4d0 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2073,11 +2073,13 @@ void touch_pud(struct vm_area_struct *vma, unsigned long addr,
update_mmu_cache_pud(vma, addr, pud);
}
-int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm,
- pud_t *dst_pud, pud_t *src_pud, unsigned long addr,
- struct vm_area_struct *vma)
+int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
+ pud_t *dst_pud, pud_t *src_pud, unsigned long addr)
{
+ struct mm_struct *dst_mm = dst_vma->vm_mm;
+ struct mm_struct *src_mm = src_vma->vm_mm;
spinlock_t *dst_ptl, *src_ptl;
+ struct page *page;
pud_t pud;
int ret;
@@ -2090,11 +2092,20 @@ int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm,
if (unlikely(!pud_trans_huge(pud)))
goto out_unlock;
- /*
- * TODO: once we support anonymous pages, use
- * folio_try_dup_anon_rmap_*() and split if duplicating fails.
- */
- if (vma_is_cow_mapping(vma) && pud_write(pud)) {
+ page = vm_normal_page_pud(src_vma, addr, pud);
+ if (page) {
+ struct folio *folio = page_folio(page);
+
+ folio_get(folio);
+ /*
+ * TODO: once we support anonymous pages, use
+ * folio_try_dup_anon_rmap_*() and split if duplicating fails.
+ */
+ folio_add_file_rmap_pud(folio, page, dst_vma);
+ add_mm_counter(dst_mm, mm_counter_file(folio), HPAGE_PUD_NR);
+ }
+
+ if (vma_is_cow_mapping(src_vma) && pud_write(pud)) {
pudp_set_wrprotect(src_mm, addr, src_pud);
pud = pud_wrprotect(pud);
}
@@ -2483,11 +2494,7 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)
static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
pmd_t pmdval, struct folio *folio, bool is_present)
{
- const bool is_device_private = folio_is_device_private(folio);
-
- /* Present and device private folios are rmappable. */
- if (is_present || is_device_private)
- folio_remove_rmap_pmd(folio, &folio->page, vma);
+ const bool is_device_private = pmd_is_device_private_entry(pmdval);
if (folio_test_anon(folio)) {
add_mm_counter(mm, MM_ANONPAGES, -HPAGE_PMD_NR);
@@ -2502,6 +2509,13 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
folio_mark_accessed(folio);
}
+ /*
+ * Present and device private folios are rmappable. Remove the
+ * rmap after transferring young+dirty to the folio.
+ */
+ if (is_present || is_device_private)
+ folio_remove_rmap_pmd(folio, &folio->page, vma);
+
/* Device private folios are pinned. */
if (is_device_private)
folio_put(folio);
@@ -2645,15 +2659,11 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,
return false;
}
- /*
- * We don't have to worry about the ordering of src and dst
- * ptlocks because exclusive mmap_lock prevents deadlock.
- */
- old_ptl = __pmd_trans_huge_lock(old_pmd, vma);
- if (old_ptl) {
- new_ptl = pmd_lockptr(mm, new_pmd);
- if (new_ptl != old_ptl)
- spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
+ old_ptl = pmd_lockptr(mm, old_pmd);
+ new_ptl = pmd_lockptr(mm, new_pmd);
+
+ double_pt_lock(old_ptl, new_ptl);
+ if (likely(pmd_is_huge(*old_pmd))) {
pmd = pmdp_huge_get_and_clear(mm, old_addr, old_pmd);
if (pmd_present(pmd))
force_flush = true;
@@ -2681,11 +2691,10 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,
set_pmd_at(mm, new_addr, new_pmd, pmd);
if (force_flush)
flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
- if (new_ptl != old_ptl)
- spin_unlock(new_ptl);
- spin_unlock(old_ptl);
+ double_pt_unlock(old_ptl, new_ptl);
return true;
}
+ double_pt_unlock(old_ptl, new_ptl);
return false;
}
@@ -3052,6 +3061,8 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma,
spinlock_t *ptl;
pud_t orig_pud;
+ tlb_change_page_size(tlb, HPAGE_PUD_SIZE);
+
ptl = __pud_trans_huge_lock(pud, vma);
if (!ptl)
return 0;
@@ -3507,16 +3518,66 @@ void split_huge_pmd_address(struct vm_area_struct *vma, unsigned long address,
__split_huge_pmd(vma, pmd, address, freeze);
}
-static inline void split_huge_pmd_if_needed(struct vm_area_struct *vma, unsigned long address)
+static bool pud_split_required(const struct vm_area_struct *vma,
+ unsigned long address)
+{
+ if (!IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD))
+ return false;
+ if (IS_ALIGNED(address, HPAGE_PUD_SIZE))
+ return false;
+ /*
+ * Only selected VMAs support mapping PUDs for now. Lift the
+ * restriction once this changes.
+ */
+ if (!vma_is_dax(vma) && !vma_test_any(vma, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT))
+ return false;
+ return range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PUD_SIZE),
+ ALIGN(address, HPAGE_PUD_SIZE));
+}
+
+static bool pmd_split_required(const struct vm_area_struct *vma,
+ unsigned long address)
{
+ if (IS_ALIGNED(address, HPAGE_PMD_SIZE))
+ return false;
+
+ return range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PMD_SIZE),
+ ALIGN(address, HPAGE_PMD_SIZE));
+}
+
+static void split_huge_leaf(struct vm_area_struct *vma,
+ unsigned long address)
+{
+ const bool split_pud = pud_split_required(vma, address);
+ const bool split_pmd = pmd_split_required(vma, address);
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+
/*
- * If the new address isn't hpage aligned and it could previously
- * contain an hugepage: check if we need to split an huge pmd.
+ * mmap, VMA and rmap write locks protect the walk; the split helpers
+ * take the page table locks.
*/
- if (!IS_ALIGNED(address, HPAGE_PMD_SIZE) &&
- range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PMD_SIZE),
- ALIGN(address, HPAGE_PMD_SIZE)))
- split_huge_pmd_address(vma, address, false);
+ if (!split_pud && !split_pmd)
+ return;
+
+ pgd = pgd_offset(vma->vm_mm, address);
+ if (!pgd_present(*pgd))
+ return;
+
+ p4d = p4d_offset(pgd, address);
+ if (!p4d_present(*p4d))
+ return;
+
+ pud = pud_offset(p4d, address);
+ if (pud_leaf(*pud))
+ __split_huge_pud(vma, pud, address);
+ if (!split_pmd || !pud_present(*pud))
+ return;
+
+ pmd = pmd_offset(pud, address);
+ __split_huge_pmd(vma, pmd, address, false);
}
void vma_adjust_trans_huge(struct vm_area_struct *vma,
@@ -3525,14 +3586,14 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,
struct vm_area_struct *next)
{
/* Check if we need to split start first. */
- split_huge_pmd_if_needed(vma, start);
+ split_huge_leaf(vma, start);
/* Check if we need to split end next. */
- split_huge_pmd_if_needed(vma, end);
+ split_huge_leaf(vma, end);
/* If we're incrementing next->vm_start, we might need to split it. */
if (next)
- split_huge_pmd_if_needed(next, end);
+ split_huge_leaf(next, end);
}
static void unmap_folio(struct folio *folio)
@@ -4511,8 +4572,7 @@ int split_folio_to_list(struct folio *folio, struct list_head *list)
*
* It is unsafe to call folio_unqueue_deferred_split() until folio refcount is
* zero: because even when the list_lru lock is held, a non-empty
- * _deferred_list might be in use on deferred_split_scan()'s unlocked
- * on-stack list.
+ * _deferred_list might be in use on deferred_split_scan()'s on-stack list.
*
* The list_lru sublist is determined by folio's memcg: it is therefore
* important to unqueue deferred split before changing folio memcg.
@@ -4658,10 +4718,22 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
deferred_split_isolate, &dispose);
list_for_each_entry_safe(folio, next, &dispose, _deferred_list) {
+ struct mem_cgroup *memcg;
+ struct list_lru_one *lru;
bool did_split = false;
bool underused = false;
+ /*
+ * Sync with deferred_split_folio(), which can requeue the
+ * folio once list_empty() is true, and sets PG_partially_mapped.
+ */
+ rcu_read_lock();
+ memcg = folio_memcg(folio);
+ lru = list_lru_lock_irq(&deferred_split_lru, folio_nid(folio),
+ &memcg);
list_del_init(&folio->_deferred_list);
+ list_lru_unlock_irq(lru);
+ rcu_read_unlock();
if (!folio_test_partially_mapped(folio)) {
/*
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index e71e0ab116bf..860c3c6b7673 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -125,6 +125,10 @@ static int __huge_pmd_unshare(struct mmu_gather *tlb,
bool check_locks);
static void hugetlb_unshare_pmds(struct vm_area_struct *vma,
unsigned long start, unsigned long end, bool take_locks);
+static void hugetlb_unmap_range(struct mmu_gather *tlb,
+ struct vm_area_struct *vma, unsigned long start,
+ unsigned long end, struct folio *folio,
+ zap_flags_t zap_flags);
static struct resv_map *vma_resv_map(struct vm_area_struct *vma);
static inline bool subpool_is_free(struct hugepage_subpool *spool)
@@ -4935,10 +4939,11 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
raw_write_seqcount_begin(&src->write_protect_seq);
} else {
/*
- * For shared mappings the vma lock must be held before
- * calling hugetlb_walk() in the src vma. Otherwise, the
- * returned ptep could go away if part of a shared pmd and
- * another thread calls huge_pmd_unshare.
+ * Shared mappings only need copying when we could have
+ * information stored in page tables (e.g., userfaultfd PTE
+ * bits or markers), which is incompatible with PMD sharing.
+ * Still take the hugetlb VMA lock for now to satisfy
+ * hugetlb_walk()'s locking requirements.
*/
hugetlb_vma_lock_read(src_vma);
}
@@ -4946,6 +4951,13 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
last_addr_mask = hugetlb_mask_last_page(h);
for (addr = src_vma->vm_start; addr < src_vma->vm_end; addr += sz) {
spinlock_t *src_ptl, *dst_ptl;
+
+ if (WARN_ON_ONCE(want_pmd_share(src_vma, addr) ||
+ want_pmd_share(dst_vma, addr))) {
+ ret = -EINVAL;
+ break;
+ }
+
src_pte = hugetlb_walk(src_vma, addr, sz);
if (!src_pte) {
addr |= last_addr_mask;
@@ -4957,14 +4969,6 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
break;
}
-#ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING
- /* If the pagetables are shared, there is nothing to do */
- if (ptdesc_pmd_is_shared(virt_to_ptdesc(dst_pte))) {
- addr |= last_addr_mask;
- continue;
- }
-#endif
-
dst_ptl = huge_pte_lock(h, dst, dst_pte);
src_ptl = huge_pte_lockptr(h, src, src_pte);
spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING);
@@ -5115,15 +5119,9 @@ static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr,
spinlock_t *src_ptl, *dst_ptl;
pte_t pte;
- dst_ptl = huge_pte_lock(h, mm, dst_pte);
+ dst_ptl = huge_pte_lockptr(h, mm, dst_pte);
src_ptl = huge_pte_lockptr(h, mm, src_pte);
-
- /*
- * We don't have to worry about the ordering of src and dst ptlocks
- * because exclusive mmap_lock (or the i_mmap_lock) prevents deadlock.
- */
- if (src_ptl != dst_ptl)
- spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING);
+ double_pt_lock(dst_ptl, src_ptl);
pte = huge_ptep_get_and_clear(mm, old_addr, src_pte, sz);
@@ -5152,15 +5150,12 @@ static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr,
set_huge_pte_at(mm, new_addr, dst_pte, pte, sz);
}
- if (src_ptl != dst_ptl)
- spin_unlock(src_ptl);
- spin_unlock(dst_ptl);
+ double_pt_unlock(dst_ptl, src_ptl);
}
-int move_hugetlb_page_tables(struct vm_area_struct *vma,
- struct vm_area_struct *new_vma,
- unsigned long old_addr, unsigned long new_addr,
- unsigned long len)
+unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma,
+ struct vm_area_struct *new_vma, unsigned long old_addr,
+ unsigned long new_addr, unsigned long len)
{
struct hstate *h = hstate_vma(vma);
struct address_space *mapping = vma->vm_file->f_mapping;
@@ -5190,6 +5185,8 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma,
for (; old_addr < old_end; old_addr += sz, new_addr += sz) {
const unsigned long offset_to_last_entry =
(old_addr | last_addr_mask) - old_addr;
+ const unsigned long offset_to_last_dst_entry =
+ (new_addr | last_addr_mask) - new_addr;
src_pte = hugetlb_walk(vma, old_addr, sz);
if (!src_pte) {
@@ -5206,6 +5203,24 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma,
continue;
}
+ /*
+ * Avoid entering the destination PMD sharing path while holding
+ * i_mmap_rwsem for write. Leave destination page table
+ * population to the fault path, which can share the PMD lazily.
+ */
+ if (sz == PMD_SIZE && want_pmd_share(new_vma, new_addr)) {
+ const unsigned long unmap_size =
+ min(old_end - old_addr,
+ offset_to_last_dst_entry + sz);
+
+ hugetlb_unmap_range(&tlb, vma, old_addr,
+ old_addr + unmap_size, NULL,
+ ZAP_FLAG_DROP_MARKER);
+ old_addr += unmap_size - sz;
+ new_addr += unmap_size - sz;
+ continue;
+ }
+
dst_pte = huge_pte_alloc(mm, new_vma, new_addr, sz);
if (!dst_pte)
break;
@@ -5225,9 +5240,10 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma,
return len + old_addr - old_end;
}
-void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
- unsigned long start, unsigned long end,
- struct folio *folio, zap_flags_t zap_flags)
+static void hugetlb_unmap_range(struct mmu_gather *tlb,
+ struct vm_area_struct *vma, unsigned long start,
+ unsigned long end, struct folio *folio,
+ zap_flags_t zap_flags)
{
struct mm_struct *mm = vma->vm_mm;
const bool folio_provided = !!folio;
@@ -5245,13 +5261,6 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
BUG_ON(start & ~huge_page_mask(h));
BUG_ON(end & ~huge_page_mask(h));
- /*
- * This is a hugetlb vma, all the pte entries should point
- * to huge page.
- */
- tlb_change_page_size(tlb, sz);
- tlb_start_vma(tlb, vma);
-
last_addr_mask = hugetlb_mask_last_page(h);
address = start;
for (; address < end; address += sz) {
@@ -5380,6 +5389,15 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
if (folio_provided)
break;
}
+}
+
+void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
+ unsigned long start, unsigned long end,
+ struct folio *folio, zap_flags_t zap_flags)
+{
+ tlb_change_page_size(tlb, huge_page_size(hstate_vma(vma)));
+ tlb_start_vma(tlb, vma);
+ hugetlb_unmap_range(tlb, vma, start, end, folio, zap_flags);
tlb_end_vma(tlb, vma);
huge_pmd_unshare_flush(tlb, vma);
diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c
index 917db0984143..8e5981888937 100644
--- a/mm/hugetlb_vmemmap.c
+++ b/mm/hugetlb_vmemmap.c
@@ -222,10 +222,12 @@ static void vmemmap_remap_pte(pte_t *pte, unsigned long addr,
VM_WARN_ON_ONCE(!PageTail((const struct page *)addr));
/*
- * Remap the tail pages as read-only to catch illegal write
- * operation to the tail pages.
+ * Map the shared tail page writable. A lockless get that read
+ * a non-zero refcount before the remap can still do its
+ * cmpxchg after it. The refcount is zero, so the cmpxchg
+ * fails and leaves the page unchanged.
*/
- entry = mk_pte(walk->vmemmap_tail, PAGE_KERNEL_RO);
+ entry = mk_pte(walk->vmemmap_tail, PAGE_KERNEL);
}
list_add(&page->lru, walk->vmemmap_pages);
diff --git a/mm/internal.h b/mm/internal.h
index 38b1165212c9..526ad50e3c1d 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -584,14 +584,16 @@ void free_pgtables(struct mmu_gather *tlb, struct unmap_desc *desc);
void pmd_install(struct mm_struct *mm, pmd_t *pmd, pgtable_t *pte);
/**
- * sync_with_folio_pmd_zap - sync with concurrent zapping of a folio PMD
+ * sync_with_folio_pmd_zap - sync with concurrent zapping of folios
* @mm: The mm_struct.
* @pmdp: Pointer to the pmd that was found to be pmd_none().
*
* When we find a pmd_none() while unmapping a folio without holding the PTL,
- * zap_huge_pmd() may have cleared the PMD but not yet modified the folio to
- * indicate that it's unmapped. Skipping the PMD without synchronization could
- * make folio unmapping code assume that unmapping failed.
+ * zap_huge_pmd() may have cleared the PMD but not yet updated the folio
+ * mapcount, and zap_empty_pte_table() clears the PMD only after the rmaps
+ * for the table have been removed, but nothing orders the pmd_none() read
+ * against later mapcount reads. Skipping the PMD without synchronization
+ * could make folio unmapping code assume that unmapping failed.
*
* Wait for concurrent zapping to complete by grabbing the PTL.
*/
@@ -619,6 +621,34 @@ static inline void force_page_cache_readahead(struct address_space *mapping,
force_page_cache_ra(&ractl, nr_to_read);
}
+/*
+ * Take two page table locks at the same level in address order such that
+ * we cannot deadlock.
+ */
+static inline void double_pt_lock(spinlock_t *ptl1, spinlock_t *ptl2)
+ __acquires(ptl1)
+ __acquires(ptl2)
+{
+ if (ptl1 > ptl2)
+ swap(ptl1, ptl2);
+ spin_lock(ptl1);
+ if (ptl1 != ptl2)
+ spin_lock_nested(ptl2, SINGLE_DEPTH_NESTING);
+ else
+ __acquire(ptl2);
+}
+
+static inline void double_pt_unlock(spinlock_t *ptl1, spinlock_t *ptl2)
+ __releases(ptl1)
+ __releases(ptl2)
+{
+ spin_unlock(ptl1);
+ if (ptl1 != ptl2)
+ spin_unlock(ptl2);
+ else
+ __release(ptl2);
+}
+
unsigned find_lock_entries(struct address_space *mapping, pgoff_t *start,
pgoff_t end, struct folio_batch *fbatch, pgoff_t *indices);
unsigned find_get_entries(struct address_space *mapping, pgoff_t *start,
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 792166950bba..47ed64f51748 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1928,7 +1928,7 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
result = SCAN_FAIL;
start_pte = pte_offset_map_lock(mm, pmd, haddr, &ptl);
- if (!start_pte) /* mmap_lock + page lock should prevent this */
+ if (!start_pte)
goto drop_folio;
/* step 1: check all mapped PTEs are to the right huge page */
@@ -1959,6 +1959,7 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
}
pte_unmap_unlock(start_pte, ptl);
+ ptl = NULL;
mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, mm,
haddr, haddr + HPAGE_PMD_SIZE);
mmu_notifier_invalidate_range_start(&range);
@@ -1976,7 +1977,7 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
pml = pmd_lock(mm, pmd);
start_pte = pte_offset_map_rw_nolock(mm, pmd, haddr, &pgt_pmd, &ptl);
- if (!start_pte) /* mmap_lock + page lock should prevent this */
+ if (!start_pte)
goto abort;
if (!pml)
spin_lock(ptl);
@@ -2018,9 +2019,10 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
/*
* Must clear entry, or a racing truncate may re-remove it.
* TLB flush can be left until pmdp_collapse_flush() does it.
- * PTE dirty? Shmem page is already dirty; file is read-only.
*/
- clear_ptes(mm, addr, pte, nr_batch_ptes);
+ ptent = get_and_clear_ptes(mm, addr, pte, nr_batch_ptes);
+ if (pte_dirty(ptent))
+ folio_mark_dirty(folio);
folio_remove_rmap_ptes(folio, page, nr_batch_ptes, vma);
nr_mapped_ptes += nr_batch_ptes;
}
@@ -2037,12 +2039,11 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
/* step 4: remove empty page table */
if (!pml) {
pml = pmd_lock(mm, pmd);
- if (ptl != pml) {
+ if (ptl != pml)
spin_lock_nested(ptl, SINGLE_DEPTH_NESTING);
- if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) {
- flush_tlb_mm(mm);
- goto unlock;
- }
+ if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) {
+ flush_tlb_mm(mm);
+ goto unlock;
}
}
pgt_pmd = pmdp_collapse_flush(vma, haddr, pmd);
@@ -2251,6 +2252,7 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
struct file *file, pgoff_t start, struct collapse_control *cc)
{
struct address_space *mapping = file->f_mapping;
+ const bool is_shmem = shmem_mapping(mapping);
struct page *dst;
struct folio *folio, *tmp, *new_folio;
unsigned long new_pfn = -1;
@@ -2259,7 +2261,8 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
XA_STATE_ORDER(xas, &mapping->i_pages, start, HPAGE_PMD_ORDER);
enum scan_result result = SCAN_SUCCEED;
int nr_none = 0;
- bool is_shmem = shmem_file(file);
+ bool do_readahead = false;
+ bool readahead_done = false;
/*
* MADV_COLLAPSE ignores shmem huge config, so do not check shmem
@@ -2277,6 +2280,10 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
mapping_set_update(&xas, mapping);
+retry:
+ result = SCAN_SUCCEED;
+ do_readahead = false;
+ xas_set_order(&xas, start, HPAGE_PMD_ORDER);
__folio_set_locked(new_folio);
if (is_shmem)
__folio_set_swapbacked(new_folio);
@@ -2341,17 +2348,9 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
}
} else { /* !is_shmem */
if (!folio || xa_is_value(folio)) {
- xas_unlock_irq(&xas);
- page_cache_sync_readahead(mapping, &file->f_ra,
- file, index,
- end - index);
- /* drain lru cache to help folio_isolate_lru() */
- lru_add_drain();
- folio = filemap_lock_folio(mapping, index);
- if (IS_ERR(folio)) {
- result = SCAN_FAIL;
- goto xa_unlocked;
- }
+ do_readahead = !readahead_done;
+ result = SCAN_FAIL;
+ goto xa_locked;
} else if (folio_test_dirty(folio)) {
/*
* This page is dirty because it hasn't
@@ -2488,6 +2487,11 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
index += folio_nr_pages(folio);
continue;
out_unlock:
+ /*
+ * The folio may have been unmapped with TTU_BATCH_FLUSH.
+ * Flush before releasing the lock and our last reference.
+ */
+ try_to_unmap_flush();
folio_unlock(folio);
folio_put(folio);
goto xa_unlocked;
@@ -2498,9 +2502,8 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
xa_unlocked:
/*
- * If collapse is successful, flush must be done now before copying.
- * If collapse is unsuccessful, does flush actually need to be done?
- * Do it anyway, to clear the state.
+ * Flush before copying the folios, or releasing them in rollback.
+ * This is a no-op if out_unlock already flushed the batch.
*/
try_to_unmap_flush();
@@ -2674,6 +2677,30 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
new_folio->mapping = NULL;
folio_unlock(new_folio);
+
+ /* Readahead may take invalidate_lock; source folios are unlocked. */
+ if (do_readahead) {
+ for (index = start; index < end;) {
+ folio = filemap_lock_folio(mapping, index);
+ if (IS_ERR(folio)) {
+ page_cache_sync_readahead(mapping, &file->f_ra,
+ file, index, end - index);
+ folio = filemap_lock_folio(mapping, index);
+ if (IS_ERR(folio)) {
+ result = SCAN_FAIL;
+ goto out_put;
+ }
+ }
+ index = folio_next_index(folio);
+ folio_unlock(folio);
+ folio_put(folio);
+ }
+ /* Drain the local LRU batch before retrying isolation. */
+ lru_add_drain();
+ readahead_done = true;
+ goto retry;
+ }
+out_put:
folio_put(new_folio);
out:
VM_BUG_ON(!list_empty(&pagelist));
diff --git a/mm/ksm.c b/mm/ksm.c
index fe42c17490b8..a554e5c0ed07 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -34,6 +34,7 @@
#include <linux/memory.h>
#include <linux/mmu_notifier.h>
#include <linux/swap.h>
+#include <linux/userfaultfd_k.h>
#include <linux/ksm.h>
#include <linux/hashtable.h>
#include <linux/freezer.h>
@@ -475,6 +476,7 @@ static void advisor_stop_scan(void)
#ifdef CONFIG_NUMA
/* Zeroed when merging across nodes is not allowed */
static unsigned int ksm_merge_across_nodes = 1;
+/* Number of allocated stable and unstable tree roots. */
static int ksm_nr_node_ids = 1;
#else
#define ksm_merge_across_nodes 1U
@@ -1462,6 +1464,15 @@ static int replace_page(struct vm_area_struct *vma, struct page *page,
dec_mm_counter(mm, MM_ANONPAGES);
}
+ /* Preserve soft-dirty and uffd protection when replacing the page. */
+ if (pte_soft_dirty(orig_pte))
+ newpte = pte_mksoft_dirty(newpte);
+ if (pte_uffd(orig_pte)) {
+ newpte = pte_mkuffd(newpte);
+ if (userfaultfd_rwp(vma))
+ newpte = pte_modify(newpte, PAGE_NONE);
+ }
+
flush_cache_page(vma, addr, pte_pfn(ptep_get(ptep)));
/*
* No need to notify as we are replacing a read only page with another
@@ -3135,24 +3146,52 @@ void __ksm_exit(struct mm_struct *mm)
trace_ksm_exit(mm);
}
-struct folio *ksm_might_need_to_copy(struct folio *folio,
- struct vm_area_struct *vma, unsigned long addr)
+static bool folio_anon_rmap_compatible(const struct folio *folio,
+ const struct page *page, struct vm_area_struct *vma,
+ unsigned long addr)
{
- struct page *page = folio_page(folio, 0);
- struct anon_vma *anon_vma = folio_anon_vma(folio);
- struct folio *new_folio;
+ const struct anon_vma *anon_vma = folio_anon_vma(folio);
+ struct anon_vma_chain *avc;
- if (folio_test_large(folio))
- return folio;
+ VM_WARN_ON_ONCE_FOLIO(!folio_test_locked(folio), folio);
+ VM_WARN_ON_ONCE_FOLIO(page_folio(page) != folio, folio);
+ VM_WARN_ON_ONCE_FOLIO(folio_test_ksm(folio), folio);
+ vma_assert_stabilised(vma);
+
+ if (!anon_vma)
+ return true;
+ if (page_pgoff(folio, page) != linear_anon_page_index(vma, addr))
+ return false;
+
+ /*
+ * Careful: anon_vma might have been freed, so we cannot safely
+ * dereference it. Add a fast path before we walk the anon_vma chain.
+ */
+ if (anon_vma == vma->anon_vma || anon_vma == vma->anon_vma->root)
+ return true;
+ list_for_each_entry(avc, &vma->anon_vma_chain, same_vma) {
+ if (avc->anon_vma == anon_vma)
+ return true;
+ }
+ return false;
+}
+
+struct folio *ksm_might_need_to_copy(struct folio *folio, struct page *page,
+ struct vm_area_struct *vma, unsigned long addr)
+{
+ struct folio *new_folio;
+
+ /*
+ * A KSM folio is small, but the slot that it left behind can be swapped
+ * in as a part of a large folio by one of the processes that shared it.
+ * So test, and copy, the page of the slot.
+ */
if (folio_test_ksm(folio)) {
if (folio_stable_node(folio) &&
!(ksm_run & KSM_RUN_UNMERGE))
return folio; /* no need to copy it */
- } else if (!anon_vma) {
- return folio; /* no need to copy it */
- } else if (folio->index == linear_anon_page_index(vma, addr) &&
- anon_vma->root == vma->anon_vma->root) {
+ } else if (folio_anon_rmap_compatible(folio, page, vma, addr)) {
return folio; /* still no need to copy it */
}
if (PageHWPoison(page))
@@ -3635,14 +3674,13 @@ static ssize_t merge_across_nodes_store(struct kobject *kobj,
else {
root_stable_tree = buf;
root_unstable_tree = buf + nr_node_ids;
+ ksm_nr_node_ids = nr_node_ids;
/* Stable tree is empty but not the unstable */
root_unstable_tree[0] = one_unstable_tree[0];
}
}
- if (!err) {
+ if (!err)
ksm_merge_across_nodes = knob;
- ksm_nr_node_ids = knob ? 1 : nr_node_ids;
- }
}
mutex_unlock(&ksm_thread_mutex);
diff --git a/mm/madvise.c b/mm/madvise.c
index 54d8397059ba..222c10eb61eb 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -755,6 +755,15 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
if (folio_test_swapcache(folio) || folio_test_dirty(folio)) {
if (!folio_trylock(folio))
continue;
+ /*
+ * KSM folios can gain new mappings at any time, so we must
+ * not clear their dirty flag.
+ */
+ if (folio_test_ksm(folio)) {
+ folio_unlock(folio);
+ continue;
+ }
+
/*
* If we have a large folio at this point, we know it is
* fully mapped so if its mapcount is the same as its
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index b5137e86bed7..b303bfc6b46e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -2419,7 +2419,10 @@ static void high_work_func(struct work_struct *work)
struct mem_cgroup *memcg;
memcg = container_of(work, struct mem_cgroup, high_work);
+ if (!css_tryget_online(&memcg->css))
+ return;
reclaim_high(memcg, MEMCG_CHARGE_BATCH, GFP_KERNEL);
+ css_put(&memcg->css);
}
/*
@@ -4369,6 +4372,8 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css)
page_counter_set_min(&memcg->memory, 0);
page_counter_set_low(&memcg->memory, 0);
+ page_counter_set_high(&memcg->memory, PAGE_COUNTER_MAX);
+ page_counter_set_high(&memcg->swap, PAGE_COUNTER_MAX);
zswap_memcg_offline_cleanup(memcg);
diff --git a/mm/memfd.c b/mm/memfd.c
index 0f6fff004f5e..0e3ff3eebfa8 100644
--- a/mm/memfd.c
+++ b/mm/memfd.c
@@ -482,6 +482,8 @@ struct file *memfd_alloc_file(const char *name, unsigned int flags)
}
file->f_mode |= FMODE_LSEEK | FMODE_PREAD | FMODE_PWRITE;
+ /* a regular file like any other: POSIX.1-2008/SUSv4 Section XSI 2.9.7 */
+ file->f_mode |= FMODE_ATOMIC_POS;
file->f_flags |= O_LARGEFILE;
if (flags & MFD_NOEXEC_SEAL) {
diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c
index 54851d8a195b..ddc135f55eb8 100644
--- a/mm/memory-tiers.c
+++ b/mm/memory-tiers.c
@@ -571,21 +571,14 @@ static struct memory_tier *set_node_memory_tier(int node)
return memtier;
}
-static void destroy_memory_tier(struct memory_tier *memtier)
+static struct memory_tier *clear_node_memory_tier(int node)
{
- list_del(&memtier->list);
- device_unregister(&memtier->dev);
-}
-
-static bool clear_node_memory_tier(int node)
-{
- bool cleared = false;
pg_data_t *pgdat;
struct memory_tier *memtier;
pgdat = NODE_DATA(node);
if (!pgdat)
- return false;
+ return NULL;
/*
* Make sure that anybody looking at NODE_DATA who finds
@@ -603,14 +596,10 @@ static bool clear_node_memory_tier(int node)
synchronize_rcu();
memtype = node_memory_types[node].memtype;
node_clear(node, memtype->nodes);
- if (nodes_empty(memtype->nodes)) {
+ if (nodes_empty(memtype->nodes))
list_del_init(&memtype->tier_sibling);
- if (list_empty(&memtier->memory_types))
- destroy_memory_tier(memtier);
- }
- cleared = true;
}
- return cleared;
+ return memtier;
}
static void release_memtype(struct kref *kref)
@@ -883,13 +872,21 @@ static int __meminit memtier_hotplug_callback(struct notifier_block *self,
{
struct memory_tier *memtier;
struct node_notify *nn = _arg;
+ bool empty = false;
switch (action) {
case NODE_REMOVED_LAST_MEMORY:
mutex_lock(&memory_tier_lock);
- if (clear_node_memory_tier(nn->nid))
+ memtier = clear_node_memory_tier(nn->nid);
+ if (memtier) {
+ empty = list_empty(&memtier->memory_types);
+ if (empty)
+ list_del(&memtier->list);
establish_demotion_targets();
+ }
mutex_unlock(&memory_tier_lock);
+ if (empty)
+ device_unregister(&memtier->dev);
break;
case NODE_ADDED_FIRST_MEMORY:
mutex_lock(&memory_tier_lock);
diff --git a/mm/memory.c b/mm/memory.c
index 9e395ad33ffd..77e0a9bf6321 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -1500,7 +1500,6 @@ copy_pud_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
unsigned long end)
{
struct mm_struct *dst_mm = dst_vma->vm_mm;
- struct mm_struct *src_mm = src_vma->vm_mm;
pud_t *src_pud, *dst_pud;
unsigned long next;
@@ -1514,8 +1513,7 @@ copy_pud_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
int err;
VM_BUG_ON_VMA(next-addr != HPAGE_PUD_SIZE, src_vma);
- err = copy_huge_pud(dst_mm, src_mm,
- dst_pud, src_pud, addr, src_vma);
+ err = copy_huge_pud(dst_vma, src_vma, dst_pud, src_pud, addr);
if (err == -ENOMEM)
return -ENOMEM;
if (!err)
@@ -2073,6 +2071,15 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,
}
} while (pte += nr, addr += PAGE_SIZE * nr, addr != end);
+ add_mm_rss_vec(mm, rss);
+ lazy_mmu_mode_disable();
+
+ /* Do the actual TLB flush before dropping ptl */
+ if (force_flush) {
+ tlb_flush_mmu_tlbonly(tlb);
+ tlb_flush_rmaps(tlb, vma);
+ }
+
/*
* Fast path: try to hold the pmd lock and unmap the PTE page.
*
@@ -2080,18 +2087,12 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,
* to hold the pmd lock failed, then we need to recheck all pte entries
* to ensure they are still none, thereby preventing the pte entries
* from being repopulated by another thread.
+ *
+ * Clear the PMD after the rmaps for the table have been removed, in case
+ * an incorrect mapcount is observed after unmapping the folio.
*/
if (can_reclaim_pt && direct_reclaim && addr == end)
direct_reclaim = zap_empty_pte_table(mm, pmd, ptl, &pmdval);
-
- add_mm_rss_vec(mm, rss);
- lazy_mmu_mode_disable();
-
- /* Do the actual TLB flush before dropping ptl */
- if (force_flush) {
- tlb_flush_mmu_tlbonly(tlb);
- tlb_flush_rmaps(tlb, vma);
- }
pte_unmap_unlock(start_pte, ptl);
/*
@@ -2138,13 +2139,15 @@ static inline unsigned long zap_pmd_range(struct mmu_gather *tlb,
addr = next;
continue;
}
+ /* We already synchronized with concurrent zapping. */
+ if (pmd_none(*pmd)) {
+ addr = next;
+ continue;
+ }
/* fall through */
- } else if (details && details->single_folio &&
- folio_test_pmd_mappable(details->single_folio) &&
- next - addr == HPAGE_PMD_SIZE && pmd_none(*pmd)) {
- sync_with_folio_pmd_zap(tlb->mm, pmd);
- }
- if (pmd_none(*pmd)) {
+ } else if (pmd_none(*pmd)) {
+ if (details && details->single_folio)
+ sync_with_folio_pmd_zap(tlb->mm, pmd);
addr = next;
continue;
}
@@ -3700,6 +3703,13 @@ static inline int __wp_page_copy_user(struct page *dst, struct page *src,
return 0;
}
+ /*
+ * __copy_from_user_inatomic() only works as expected for local faults.
+ * Reject remote faults.
+ */
+ if (unlikely(mm != current->mm))
+ return -EFAULT;
+
/*
* If the source page was a PFN mapping, we don't have
* a "struct page" for it. We do a best-effort copy by
@@ -4025,6 +4035,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
folio_put(old_folio);
delayacct_wpcopy_end();
+ if (err == -EFAULT)
+ return VM_FAULT_SIGBUS;
return err == -EHWPOISON ? VM_FAULT_HWPOISON : 0;
}
kmsan_copy_page_meta(&new_folio->page, vmf->page);
@@ -4057,8 +4069,11 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
if (unlikely(unshare)) {
if (pte_soft_dirty(vmf->orig_pte))
entry = pte_mksoft_dirty(entry);
- if (pte_uffd(vmf->orig_pte))
+ if (pte_uffd(vmf->orig_pte)) {
entry = pte_mkuffd(entry);
+ if (userfaultfd_rwp(vma))
+ entry = pte_modify(entry, PAGE_NONE);
+ }
} else {
entry = maybe_mkwrite(pte_mkdirty(entry), vma);
}
@@ -4579,8 +4594,8 @@ EXPORT_SYMBOL_GPL(unmap_mapping_pages);
void unmap_mapping_range(struct address_space *mapping,
loff_t const holebegin, loff_t const holelen, int even_cows)
{
- pgoff_t hba = (pgoff_t)(holebegin) >> PAGE_SHIFT;
- pgoff_t hlen = ((pgoff_t)(holelen) + PAGE_SIZE - 1) >> PAGE_SHIFT;
+ pgoff_t hba = (u64)holebegin >> PAGE_SHIFT;
+ pgoff_t hlen = ((u64)holelen + PAGE_SIZE - 1) >> PAGE_SHIFT;
/* Check for overflow. */
if (sizeof(holelen) > sizeof(hlen)) {
@@ -4870,6 +4885,34 @@ static void check_swap_exclusive(struct folio *folio, swp_entry_t entry,
} while (--nr_pages);
}
+/*
+ * Get the address range in which a folio might be mapped, assuming its pages
+ * are mapped consecutively and the folio page at the given idx is mapped at
+ * the given address.
+ */
+static void folio_pte_range(const struct folio *folio,
+ const struct vm_area_struct *vma, unsigned long addr,
+ unsigned long idx, unsigned long *start, unsigned long *end)
+{
+ /* Stay within the page table and VMA boundaries. */
+ const unsigned long pmd_start = addr & PMD_MASK;
+ const unsigned long pmd_end = pmd_start + PMD_SIZE;
+ const unsigned long lower = max(pmd_start, vma->vm_start);
+ /*
+ * We cannot use pmd_addr_end(), as it does not clamp reliably. To
+ * avoid wraparounds, compute using the inclusive end.
+ */
+ const unsigned long upper = min(pmd_end - 1, vma->vm_end - 1) + 1;
+ const unsigned long offset = idx << PAGE_SHIFT;
+
+ /*
+ * Avoid start/end addresses wrapping by not adding/subtracting more
+ * than the lower/upper limits would allow.
+ */
+ *start = addr - min(addr - lower, offset);
+ *end = addr + min(upper - addr, folio_size(folio) - offset);
+}
+
/*
* We enter with either the VMA lock or the mmap_lock held (see
* FAULT_FLAG_VMA_LOCK), and pte mapped but not yet locked.
@@ -5020,7 +5063,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
* folio->index of non-ksm folios would be nonlinear inside the
* anon VMA -- the ksm flag is lost on actual swapout.
*/
- folio = ksm_might_need_to_copy(folio, vma, vmf->address);
+ folio = ksm_might_need_to_copy(folio, page, vma, vmf->address);
if (unlikely(!folio)) {
ret = VM_FAULT_OOM;
folio = swapcache;
@@ -5052,16 +5095,15 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
address = vmf->address;
ptep = vmf->pte;
if (folio_test_large(folio) && folio_test_swapcache(folio)) {
- int nr = folio_nr_pages(folio);
- unsigned long idx = folio_page_idx(folio, page);
- unsigned long folio_start = address - idx * PAGE_SIZE;
- unsigned long folio_end = folio_start + nr * PAGE_SIZE;
+ const int nr = folio_nr_pages(folio);
+ const unsigned long idx = folio_page_idx(folio, page);
+ unsigned long folio_start, folio_end;
pte_t *folio_ptep;
pte_t folio_pte;
- if (unlikely(folio_start < max(address & PMD_MASK, vma->vm_start)))
- goto check_folio;
- if (unlikely(folio_end > pmd_addr_end(address, vma->vm_end)))
+ folio_pte_range(folio, vma, address, idx, &folio_start,
+ &folio_end);
+ if (unlikely((folio_end - folio_start) >> PAGE_SHIFT != nr))
goto check_folio;
folio_ptep = vmf->pte - idx;
@@ -5202,7 +5244,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
if (unlikely(folio != swapcache)) {
folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE);
folio_add_lru_vma(folio, vma);
- folio_put_swap(swapcache, NULL);
+ folio_put_swap(swapcache,
+ folio_file_page(swapcache, swp_offset(entry)));
} else if (!folio_test_anon(folio)) {
/*
* We currently only expect !anon folios that are fully
@@ -6211,16 +6254,11 @@ static void numa_rebuild_large_mapping(struct vm_fault *vmf, struct vm_area_stru
struct folio *folio, pte_t fault_pte,
bool ignore_writable, bool pte_write_upgrade)
{
- int nr = pte_pfn(fault_pte) - folio_pfn(folio);
+ const unsigned long idx = pte_pfn(fault_pte) - folio_pfn(folio);
unsigned long start, end, addr = vmf->address;
- unsigned long addr_start = addr - (nr << PAGE_SHIFT);
- unsigned long pt_start = ALIGN_DOWN(addr, PMD_SIZE);
pte_t *start_ptep;
- /* Stay within the VMA and within the page table. */
- start = max3(addr_start, pt_start, vma->vm_start);
- end = min3(addr_start + folio_size(folio), pt_start + PMD_SIZE,
- vma->vm_end);
+ folio_pte_range(folio, vma, addr, idx, &start, &end);
start_ptep = vmf->pte - ((addr - start) >> PAGE_SHIFT);
/* Restore all PTEs' mapping of the large folio */
@@ -6566,9 +6604,12 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)
* VM_UFFD_RWP VMA, a protnone PTE without the uffd bit is
* NUMA hinting and must still fall through to do_numa_page().
*/
- if (userfaultfd_pte_rwp(vmf->vma, vmf->orig_pte))
- return do_uffd_rwp(vmf);
- return do_numa_page(vmf);
+ if (userfaultfd_pte_rwp(vmf->vma, vmf->orig_pte)) {
+ if (!(vmf->flags & FAULT_FLAG_UNSHARE))
+ return do_uffd_rwp(vmf);
+ } else {
+ return do_numa_page(vmf);
+ }
}
spin_lock(vmf->ptl);
@@ -6883,8 +6924,12 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
lru_gen_exit_fault();
- /* If the mapping is droppable, then errors due to OOM aren't fatal. */
- if (is_droppable)
+ /*
+ * If the mapping is droppable, then errors due to OOM aren't fatal: the
+ * task returns to user mode, handles its signals and tries again. A
+ * fault from kernel mode has no such return, so it must see the error.
+ */
+ if (is_droppable && (flags & FAULT_FLAG_USER))
ret &= ~VM_FAULT_OOM;
if (flags & FAULT_FLAG_USER) {
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 839c3fe985a0..d9de3b5d5ebe 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -3346,11 +3346,11 @@ void __init numa_policy_init(void)
policy_cache = kmem_cache_create("numa_policy",
sizeof(struct mempolicy),
- 0, SLAB_PANIC, NULL);
+ 0, SLAB_PANIC | SLAB_ACCOUNT, NULL);
sn_cache = kmem_cache_create("shared_policy_node",
sizeof(struct sp_node),
- 0, SLAB_PANIC, NULL);
+ 0, SLAB_PANIC | SLAB_ACCOUNT, NULL);
for_each_node(nid) {
preferred_node_policy[nid] = (struct mempolicy) {
diff --git a/mm/migrate.c b/mm/migrate.c
index 15b45832bcfa..ee78577177d6 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2462,7 +2462,7 @@ static void do_pages_stat_array(struct mm_struct *mm, unsigned long nr_pages,
mmap_read_lock(mm);
for (i = 0; i < nr_pages; i++) {
- unsigned long addr = (unsigned long)(*pages);
+ unsigned long addr = (unsigned long)untagged_addr_remote(mm, *pages);
struct vm_area_struct *vma;
struct folio_walk fw;
struct folio *folio;
@@ -2757,6 +2757,7 @@ int migrate_misplaced_folio(struct folio *folio, int node)
pg_data_t *pgdat = NODE_DATA(node);
int nr_remaining;
unsigned int nr_succeeded;
+ int src_node = folio_nid(folio);
LIST_HEAD(migratepages);
struct mem_cgroup *memcg = get_mem_cgroup_from_folio(folio);
struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat);
@@ -2771,7 +2772,7 @@ int migrate_misplaced_folio(struct folio *folio, int node)
count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded);
count_memcg_events(memcg, NUMA_PAGE_MIGRATE, nr_succeeded);
if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING)
- && !node_is_toptier(folio_nid(folio))
+ && !node_is_toptier(src_node)
&& node_is_toptier(node))
mod_lruvec_state(lruvec, PGPROMOTE_SUCCESS, nr_succeeded);
}
diff --git a/mm/mm_init.c b/mm/mm_init.c
index 1533aebafb68..36ec1ef0712c 100644
--- a/mm/mm_init.c
+++ b/mm/mm_init.c
@@ -2422,8 +2422,6 @@ void __init memblock_free_pages(unsigned long pfn, unsigned int order)
return;
}
- /* pages were reserved and not allocated */
- clear_page_tag_ref(page);
__free_pages_core(page, order, MEMINIT_EARLY);
}
diff --git a/mm/mmap.c b/mm/mmap.c
index 4bf26b0f1e6e..6f22cd76be4f 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1713,8 +1713,7 @@ __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
LIST_HEAD(uf);
VMA_ITERATOR(vmi, mm, 0);
- if (mmap_write_lock_killable(oldmm))
- return -EINTR;
+ mmap_assert_write_locked(oldmm);
flush_cache_dup_mm(oldmm);
uprobe_dup_mmap(oldmm, mm);
/*
@@ -1725,11 +1724,6 @@ __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
/* No ordering required: file already has been exposed. */
dup_mm_exe_file(mm, oldmm);
- mm->total_vm = oldmm->total_vm;
- mm->data_vm = oldmm->data_vm;
- mm->exec_vm = oldmm->exec_vm;
- mm->stack_vm = oldmm->stack_vm;
-
/* Use __mt_dup() to efficiently build an identical maple tree. */
retval = __mt_dup(&oldmm->mm_mt, &mm->mm_mt, GFP_KERNEL);
if (unlikely(retval))
diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c
index df69ba6e797f..a7bae5de5fed 100644
--- a/mm/mmu_notifier.c
+++ b/mm/mmu_notifier.c
@@ -569,16 +569,25 @@ int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range)
{
struct mmu_notifier_subscriptions *subscriptions =
range->mm->notifier_subscriptions;
- int ret;
+ const bool has_itree = subscriptions->has_itree;
+ int ret = 0;
- if (subscriptions->has_itree) {
+ if (has_itree) {
ret = mn_itree_invalidate(subscriptions, range);
if (ret)
return ret;
}
if (!hlist_empty(&subscriptions->list))
- return mn_hlist_invalidate_range_start(subscriptions, range);
- return 0;
+ ret = mn_hlist_invalidate_range_start(subscriptions, range);
+
+ /*
+ * On error, the non-blocking caller is not allowed to call
+ * invalidate_range_end(), as in mn_itree_invalidate().
+ */
+ if (ret && has_itree && !mmu_notifier_range_blockable(range))
+ mn_itree_inv_end(subscriptions);
+
+ return ret;
}
static void
@@ -938,7 +947,6 @@ static int __mmu_interval_notifier_insert(
struct mmu_notifier_subscriptions *subscriptions, unsigned long start,
unsigned long length, const struct mmu_interval_notifier_ops *ops)
{
- interval_sub->mm = mm;
interval_sub->ops = ops;
RB_CLEAR_NODE(&interval_sub->interval_tree.rb);
interval_sub->interval_tree.start = start;
@@ -955,8 +963,12 @@ static int __mmu_interval_notifier_insert(
if (WARN_ON(atomic_read(&mm->mm_users) <= 0))
return -EINVAL;
- /* pairs with mmdrop in mmu_interval_notifier_remove() */
+ /*
+ * Pairs with mmdrop in mmu_interval_notifier_remove(). Drivers take a
+ * non-NULL ->mm for "registered", so set it only with the reference.
+ */
mmgrab(mm);
+ interval_sub->mm = mm;
/*
* If some invalidate_range_start/end region is going on in parallel
diff --git a/mm/mremap.c b/mm/mremap.c
index d404e4b1b6cb..23498d0783c4 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -87,7 +87,7 @@ static pud_t *get_old_pud(struct mm_struct *mm, unsigned long addr)
return NULL;
pud = pud_offset(p4d, addr);
- if (pud_none_or_clear_bad(pud))
+ if (!pud_trans_huge(*pud) && pud_none_or_clear_bad(pud))
return NULL;
return pud;
@@ -202,7 +202,7 @@ static int move_ptes(struct pagetable_move_control *pmc,
struct mm_struct *mm = vma->vm_mm;
pte_t *old_ptep, *new_ptep;
pte_t old_pte, pte;
- pmd_t dummy_pmdval;
+ pmd_t old_pmdval, new_pmdval;
spinlock_t *old_ptl, *new_ptl;
bool force_flush = false;
unsigned long old_addr = pmc->old_addr;
@@ -234,31 +234,30 @@ static int move_ptes(struct pagetable_move_control *pmc,
if (pmc->need_rmap_locks)
take_rmap_locks(vma);
- /*
- * We don't have to worry about the ordering of src and dst
- * pte locks because exclusive mmap_lock prevents deadlock.
- */
- old_ptep = pte_offset_map_lock(mm, old_pmd, old_addr, &old_ptl);
+ old_ptep = pte_offset_map_rw_nolock(mm, old_pmd, old_addr, &old_pmdval,
+ &old_ptl);
if (!old_ptep) {
err = -EAGAIN;
goto out;
}
- /*
- * Now new_pte is none, so collapse_scan_file() path can not find
- * this by traversing file->f_mapping, so there is no concurrency with
- * retract_page_tables(). In addition, we already hold the exclusive
- * mmap_lock, so this new_pte page is stable, so there is no need to get
- * pmdval and do pmd_same() check.
- */
- new_ptep = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &dummy_pmdval,
- &new_ptl);
+ new_ptep = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &new_pmdval,
+ &new_ptl);
if (!new_ptep) {
- pte_unmap_unlock(old_ptep, old_ptl);
+ pte_unmap(old_ptep);
+ err = -EAGAIN;
+ goto out;
+ }
+
+ double_pt_lock(old_ptl, new_ptl);
+ if (unlikely(!pmd_same(old_pmdval, pmdp_get_lockless(old_pmd)) ||
+ !pmd_same(new_pmdval, pmdp_get_lockless(new_pmd)))) {
+ double_pt_unlock(old_ptl, new_ptl);
+ pte_unmap(new_ptep);
+ pte_unmap(old_ptep);
err = -EAGAIN;
goto out;
}
- if (new_ptl != old_ptl)
- spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
+
flush_tlb_batched_pending(vma->vm_mm);
lazy_mmu_mode_enable();
@@ -317,10 +316,9 @@ static int move_ptes(struct pagetable_move_control *pmc,
lazy_mmu_mode_disable();
if (force_flush)
flush_tlb_range(vma, old_end - len, old_end);
- if (new_ptl != old_ptl)
- spin_unlock(new_ptl);
+ double_pt_unlock(old_ptl, new_ptl);
pte_unmap(new_ptep - 1);
- pte_unmap_unlock(old_ptep - 1, old_ptl);
+ pte_unmap(old_ptep - 1);
out:
if (pmc->need_rmap_locks)
drop_rmap_locks(vma);
@@ -395,14 +393,9 @@ static bool move_normal_pmd(struct pagetable_move_control *pmc,
if (WARN_ON_ONCE(!pmd_none(*new_pmd)))
return false;
- /*
- * We don't have to worry about the ordering of src and dst
- * ptlocks because exclusive mmap_lock prevents deadlock.
- */
- old_ptl = pmd_lock(mm, old_pmd);
+ old_ptl = pmd_lockptr(mm, old_pmd);
new_ptl = pmd_lockptr(mm, new_pmd);
- if (new_ptl != old_ptl)
- spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
+ double_pt_lock(old_ptl, new_ptl);
pmd = *old_pmd;
@@ -418,9 +411,7 @@ static bool move_normal_pmd(struct pagetable_move_control *pmc,
pmd_populate(mm, new_pmd, pmd_pgtable(pmd));
flush_tlb_range(vma, pmc->old_addr, pmc->old_addr + PMD_SIZE);
out_unlock:
- if (new_ptl != old_ptl)
- spin_unlock(new_ptl);
- spin_unlock(old_ptl);
+ double_pt_unlock(old_ptl, new_ptl);
return res;
}
@@ -452,26 +443,24 @@ static bool move_normal_pud(struct pagetable_move_control *pmc,
if (WARN_ON_ONCE(!pud_none(*new_pud)))
return false;
- /*
- * We don't have to worry about the ordering of src and dst
- * ptlocks because exclusive mmap_lock prevents deadlock.
- */
- old_ptl = pud_lock(mm, old_pud);
+ old_ptl = pud_lockptr(mm, old_pud);
new_ptl = pud_lockptr(mm, new_pud);
- if (new_ptl != old_ptl)
- spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
- /* Clear the pud */
+ double_pt_lock(old_ptl, new_ptl);
+
pud = *old_pud;
+ if (unlikely(pud_none(pud)))
+ goto out_unlock;
+
+ /* Clear the pud */
pud_clear(old_pud);
VM_BUG_ON(!pud_none(*new_pud));
pud_populate(mm, new_pud, pud_pgtable(pud));
flush_tlb_range(vma, pmc->old_addr, pmc->old_addr + PUD_SIZE);
- if (new_ptl != old_ptl)
- spin_unlock(new_ptl);
- spin_unlock(old_ptl);
+out_unlock:
+ double_pt_unlock(old_ptl, new_ptl);
return true;
}
@@ -499,18 +488,17 @@ static bool move_huge_pud(struct pagetable_move_control *pmc,
if (WARN_ON_ONCE(!pud_none(*new_pud)))
return false;
- /*
- * We don't have to worry about the ordering of src and dst
- * ptlocks because exclusive mmap_lock prevents deadlock.
- */
- old_ptl = pud_lock(mm, old_pud);
+ old_ptl = pud_lockptr(mm, old_pud);
new_ptl = pud_lockptr(mm, new_pud);
- if (new_ptl != old_ptl)
- spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
+
+ double_pt_lock(old_ptl, new_ptl);
+ if (unlikely(!pud_trans_huge(*old_pud))) {
+ double_pt_unlock(old_ptl, new_ptl);
+ return false;
+ }
/* Clear the pud */
- pud = *old_pud;
- pud_clear(old_pud);
+ pud = pudp_huge_get_and_clear(mm, pmc->old_addr, old_pud);
VM_BUG_ON(!pud_none(*new_pud));
@@ -518,9 +506,7 @@ static bool move_huge_pud(struct pagetable_move_control *pmc,
/* mark soft_ditry when we add pud level soft dirty support */
set_pud_at(mm, pmc->new_addr, new_pud, pud);
flush_pud_tlb_range(vma, pmc->old_addr, pmc->old_addr + HPAGE_PUD_SIZE);
- if (new_ptl != old_ptl)
- spin_unlock(new_ptl);
- spin_unlock(old_ptl);
+ double_pt_unlock(old_ptl, new_ptl);
return true;
}
@@ -842,11 +828,10 @@ unsigned long move_page_tables(struct pagetable_move_control *pmc)
if (!new_pud)
break;
if (pud_trans_huge(*old_pud)) {
- if (extent == HPAGE_PUD_SIZE) {
- move_pgt_entry(pmc, HPAGE_PUD, old_pud, new_pud);
- /* We ignore and continue on error? */
+ if (extent == HPAGE_PUD_SIZE &&
+ move_pgt_entry(pmc, HPAGE_PUD, old_pud, new_pud))
continue;
- }
+ split_huge_pud(pmc->old, old_pud, pmc->old_addr);
} else if (IS_ENABLED(CONFIG_HAVE_MOVE_PUD) && extent == PUD_SIZE) {
if (move_pgt_entry(pmc, NORMAL_PUD, old_pud, new_pud))
continue;
diff --git a/mm/nommu.c b/mm/nommu.c
index ed44510e3770..6e0f9c7da236 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1926,7 +1926,7 @@ subsys_initcall(init_admin_reserve);
int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
{
- mmap_write_lock(oldmm);
+ mmap_assert_write_locked(oldmm);
dup_mm_exe_file(mm, oldmm);
mmap_write_unlock(oldmm);
return 0;
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index eeab25d6ce36..1d6aab8781a5 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -883,6 +883,7 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc,
{
struct wb_domain *dom = dtc_dom(dtc);
struct bdi_writeback *wb = dtc->wb;
+ u64 threshold = thresh;
u64 wb_thresh;
u64 wb_max_thresh;
unsigned long numerator, denominator;
@@ -894,13 +895,14 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc,
fprop_fraction_percpu(&dom->completions, dtc->wb_completions,
&numerator, &denominator);
- wb_thresh = (thresh * (100 * BDI_RATIO_SCALE - bdi_min_ratio)) / (100 * BDI_RATIO_SCALE);
+ wb_thresh = div_u64(threshold * (100 * BDI_RATIO_SCALE - bdi_min_ratio),
+ 100 * BDI_RATIO_SCALE);
wb_thresh *= numerator;
wb_thresh = div64_ul(wb_thresh, denominator);
wb_min_max_ratio(wb, &wb_min_ratio, &wb_max_ratio);
- wb_thresh += (thresh * wb_min_ratio) / (100 * BDI_RATIO_SCALE);
+ wb_thresh += div_u64(threshold * wb_min_ratio, 100 * BDI_RATIO_SCALE);
/*
* It's very possible that wb_thresh is close to 0 not because the
@@ -916,7 +918,7 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc,
wb_thresh = max(wb_thresh, (thresh - dtc->dirty) / 8);
}
- wb_max_thresh = thresh * wb_max_ratio / (100 * BDI_RATIO_SCALE);
+ wb_max_thresh = div_u64(threshold * wb_max_ratio, 100 * BDI_RATIO_SCALE);
if (wb_thresh > wb_max_thresh)
wb_thresh = wb_max_thresh;
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 8ca2289d0491..8714425ffe84 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1634,6 +1634,9 @@ void __meminit __free_pages_core(struct page *page, unsigned int order,
atomic_long_add(nr_pages, &page_zone(page)->managed_pages);
}
+ /* pages were never allocated */
+ clear_page_tag_ref(page);
+
if (page_contains_unaccepted(page, order)) {
if (order == MAX_PAGE_ORDER && __free_unaccepted(page))
return;
@@ -2947,7 +2950,7 @@ static bool free_frozen_page_commit(struct zone *zone,
* kswapd work again by resetting kswapd_failures.
*/
if (kswapd_test_hopeless(pgdat) &&
- next_memory_node(pgdat->node_id) < MAX_NUMNODES)
+ num_node_state(N_MEMORY) > 1)
kswapd_clear_hopeless(pgdat, KSWAPD_CLEAR_HOPELESS_PCP);
}
return ret;
@@ -3256,7 +3259,9 @@ struct page *rmqueue_buddy(struct zone *preferred_zone, struct zone *zone,
* reserves as failing now is worse than failing a
* high-order atomic allocation in the future.
*/
- if (!page && (alloc_flags & (ALLOC_OOM|ALLOC_NON_BLOCK)))
+ if (!page &&
+ ((alloc_flags & ALLOC_OOM) ||
+ (alloc_flags & ALLOC_MASK_ATOMIC) == ALLOC_MASK_ATOMIC))
page = __rmqueue_smallest(zone, order, MIGRATE_HIGHATOMIC);
if (!page) {
@@ -4786,10 +4791,10 @@ static inline struct page *
__alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
struct alloc_context *ac)
{
- bool can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM;
- bool can_compact = can_direct_reclaim && gfp_compaction_allowed(gfp_mask);
- bool nofail = gfp_mask & __GFP_NOFAIL;
const bool costly_order = order > PAGE_ALLOC_COSTLY_ORDER;
+ bool can_direct_reclaim;
+ bool can_compact;
+ bool nofail;
struct page *page = NULL;
unsigned int alloc_flags;
unsigned long did_some_progress;
@@ -4804,6 +4809,18 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
bool can_retry_reserves = true;
unsigned long alloc_start_time = jiffies;
+ /*
+ * Costly __GFP_NORETRY callers have a cheap fallback, so don't stall
+ * them in reclaim or compaction. __GFP_THISNODE callers are exempt.
+ */
+ if (costly_order && (gfp_mask & __GFP_NORETRY) &&
+ !(gfp_mask & __GFP_THISNODE))
+ gfp_mask &= ~__GFP_DIRECT_RECLAIM;
+
+ can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM;
+ can_compact = can_direct_reclaim && gfp_compaction_allowed(gfp_mask);
+ nofail = gfp_mask & __GFP_NOFAIL;
+
if (unlikely(nofail)) {
/*
* Also we don't support __GFP_NOFAIL without __GFP_DIRECT_RECLAIM,
@@ -4920,8 +4937,11 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
* Reclaim/compaction cannot run, so defrag_mode's strategy
* of enforcing ALLOC_NOFRAGMENT cannot be fulfilled. Allow
* fallbacks rather than failing the allocation outright.
+ * Not for __GFP_NORETRY: those have a cheap lower order
+ * fallback, so failing beats fragmenting.
*/
if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT) &&
+ !(gfp_mask & __GFP_NORETRY) &&
(gfp_mask & __GFP_KSWAPD_RECLAIM)) {
alloc_flags &= ~ALLOC_NOFRAGMENT;
goto retry;
diff --git a/mm/page_alloc.h b/mm/page_alloc.h
index 2b1017a91928..a9b72cb9520e 100644
--- a/mm/page_alloc.h
+++ b/mm/page_alloc.h
@@ -32,12 +32,11 @@
#define ALLOC_OOM ALLOC_NO_WATERMARKS
#endif
-#define ALLOC_NON_BLOCK 0x10 /* Caller cannot block. Allow access
- * to 25% of the min watermark or
- * 62.5% if __GFP_HIGH is set.
- */
+#define ALLOC_NON_BLOCK 0x10 /* Caller cannot block. */
#define ALLOC_MIN_RESERVE 0x20 /* __GFP_HIGH set. Allow access to 50%
- * of the min watermark.
+ * of the min watermark, or 62.5% if
+ * the caller cannot block either
+ * (ALLOC_NON_BLOCK).
*/
#define ALLOC_CPUSET 0x40 /* check for correct cpuset */
#define ALLOC_CMA 0x80 /* allow allocations from CMA areas */
@@ -58,7 +57,10 @@
#define ALLOC_NO_CODETAG 0x1000
/* Flags that allow allocations below the min watermark. */
-#define ALLOC_RESERVES (ALLOC_NON_BLOCK|ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM)
+#define ALLOC_RESERVES (ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM)
+
+/* Flags that mean GFP_ATOMIC */
+#define ALLOC_MASK_ATOMIC (ALLOC_NON_BLOCK|ALLOC_MIN_RESERVE)
/*
* Structure for holding the mostly immutable allocation parameters passed
diff --git a/mm/rmap.c b/mm/rmap.c
index 6b3480330b92..9ffd3e833e08 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -926,6 +926,7 @@ static bool folio_referenced_one(struct folio *folio,
DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, address, 0);
int ptes = 0, referenced = 0;
unsigned int nr;
+ pte_t pteval;
while (page_vma_mapped_walk(&pvmw)) {
address = pvmw.address;
@@ -970,10 +971,12 @@ static bool folio_referenced_one(struct folio *folio,
return false;
}
- if (pvmw.pte && folio_test_large(folio)) {
+ if (pvmw.pte)
+ pteval = ptep_get(pvmw.pte);
+
+ if (pvmw.pte && pte_present(pteval) && folio_test_large(folio)) {
const unsigned long end_addr = pmd_addr_end(address, vma->vm_end);
const unsigned int max_nr = (end_addr - address) >> PAGE_SHIFT;
- pte_t pteval = ptep_get(pvmw.pte);
nr = folio_pte_batch(folio, pvmw.pte, pteval, max_nr);
}
@@ -983,11 +986,23 @@ static bool folio_referenced_one(struct folio *folio,
* are. —they could be on active/inactive lists or on MGLRU. So the
* simplest approach is to disable this look-around optimization.
*/
- if (lru_gen_enabled() && !lru_gen_switching() && pvmw.pte) {
+ if (lru_gen_enabled() && !lru_gen_switching() && pvmw.pte &&
+ pte_present(pteval)) {
if (lru_gen_look_around(&pvmw, nr))
referenced++;
} else if (pvmw.pte) {
- if (clear_flush_young_ptes_notify(vma, address, pvmw.pte, nr))
+ bool young = false;
+
+ /*
+ * PFN swap PTEs, such as device-exclusive ones, that
+ * actually map pages are "old" from a CPU perspective.
+ * The MMU notifier takes care of any device aspects.
+ */
+ if (pte_present(pteval))
+ young = clear_flush_young_ptes(vma, address, pvmw.pte, nr);
+ young |= mmu_notifier_clear_flush_young(vma->vm_mm, address,
+ address + nr * PAGE_SIZE);
+ if (young)
referenced++;
} else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) {
if (pmdp_clear_flush_young_notify(vma, address,
diff --git a/mm/shmem.c b/mm/shmem.c
index de2362209cc3..597bbc2e7321 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -77,6 +77,7 @@ static struct vfsmount *shm_mnt __ro_after_init;
#include <linux/magic.h>
#include <linux/syscalls.h>
#include <linux/fcntl.h>
+#include <linux/memfd.h>
#include <uapi/linux/memfd.h>
#include <linux/rmap.h>
#include <linux/uuid.h>
@@ -275,6 +276,11 @@ static const struct vm_operations_struct shmem_vm_ops;
static const struct vm_operations_struct shmem_anon_vm_ops;
static struct file_system_type shmem_fs_type;
+bool shmem_file(struct file *file)
+{
+ return file && file_inode(file)->i_fop == &shmem_file_operations;
+}
+
bool shmem_mapping(const struct address_space *mapping)
{
return mapping->a_ops == &shmem_aops;
@@ -968,12 +974,11 @@ static void shmem_delete_from_page_cache(struct folio *folio, void *radswap)
}
/*
- * Remove swap entry from page cache, free the swap and its page cache. Returns
- * the number of pages being freed. 0 means entry not found in XArray (0 pages
- * being freed).
+ * Remove a swap entry or swapin-error marker wholly within [index, end].
+ * Return whether it was removed, counting freed swap pages in nr_swaps_freed.
*/
-static long shmem_free_swap(struct address_space *mapping,
- pgoff_t index, pgoff_t end, void *radswap)
+static bool shmem_free_swap(struct address_space *mapping,
+ pgoff_t index, pgoff_t end, void *radswap, long *nr_swaps_freed)
{
XA_STATE(xas, &mapping->i_pages, index);
const softleaf_t swp = radix_to_swp_entry(radswap);
@@ -994,8 +999,10 @@ static long shmem_free_swap(struct address_space *mapping,
xas_unlock_irq(&xas);
/* A swapin-error marker holds no swap slot, so just drop it. */
- if (nr_pages && softleaf_is_swap(swp))
+ if (nr_pages && softleaf_is_swap(swp)) {
swap_put_entries_direct(swp, nr_pages);
+ *nr_swaps_freed += nr_pages;
+ }
return nr_pages;
}
@@ -1089,12 +1096,10 @@ void shmem_unlock_mapping(struct address_space *mapping)
static struct folio *shmem_get_partial_folio(struct inode *inode, pgoff_t index)
{
+ const gfp_t gfp = mapping_gfp_mask(inode->i_mapping);
struct folio *folio;
- /*
- * At first avoid shmem_get_folio(,,,SGP_READ): that fails
- * beyond i_size, and reports fallocated folios as holes.
- */
+retry:
folio = filemap_get_entry(inode->i_mapping, index);
if (!folio)
return folio;
@@ -1105,13 +1110,14 @@ static struct folio *shmem_get_partial_folio(struct inode *inode, pgoff_t index)
/* The folio has been swapped out */
folio_unlock(folio);
folio_put(folio);
+ goto retry;
}
/*
- * But read a folio back from swap if any of it is within i_size
- * (although in some cases this is just a waste of time).
+ * Truncation can look up a large swap entry at or beyond i_size.
+ * Swap it in directly to bypass the size check in shmem_get_folio().
*/
- folio = NULL;
- shmem_get_folio(inode, index, 0, &folio, SGP_READ);
+ if (shmem_swapin_folio(inode, index, &folio, SGP_READ, gfp, NULL, NULL) == -EEXIST)
+ goto retry;
return folio;
}
@@ -1150,8 +1156,8 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,
if (xa_is_value(folio)) {
if (unfalloc)
continue;
- nr_swaps_freed += shmem_free_swap(mapping, indices[i],
- end - 1, folio);
+ shmem_free_swap(mapping, indices[i], end - 1,
+ folio, &nr_swaps_freed);
continue;
}
@@ -1218,32 +1224,31 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,
if (xa_is_value(folio)) {
int order;
- long swaps_freed;
if (unfalloc)
continue;
- swaps_freed = shmem_free_swap(mapping, indices[i],
- end - 1, folio);
- if (!swaps_freed) {
+ if (!shmem_free_swap(mapping, indices[i], end - 1,
+ folio, &nr_swaps_freed)) {
pgoff_t base = indices[i];
order = shmem_confirm_swap(mapping, indices[i],
radix_to_swp_entry(folio));
/*
- * If found a large swap entry cross the end or start
- * border, skip it as the truncate_inode_partial_folio
- * above should have at least zerod its content once.
+ * Skip boundary entries when restarting
+ * the scan to avoid endless loops after
+ * swapin failure.
*/
if (order > 0) {
base = round_down(base, 1 << order);
- if (base < start || base + (1 << order) > end)
+ if (base < start || base + (1 << order) > end) {
+ start = max(start, base + (1 << order));
continue;
+ }
}
/* Swap was replaced by page or extended, retry */
index = base;
break;
}
- nr_swaps_freed += swaps_freed;
continue;
}
@@ -1259,22 +1264,34 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,
VM_BUG_ON_FOLIO(folio_test_writeback(folio),
folio);
- if (!folio_test_large(folio)) {
+ /*
+ * A shared mapping can have dirty PTEs while
+ * the folio is still clean. Mark the folio
+ * dirty so a failed split cannot discard data
+ * outside the range.
+ */
+ if (!unfalloc && folio_test_large(folio))
+ folio_mark_dirty(folio);
+
+ if (unfalloc || !folio_test_large(folio)) {
truncate_inode_folio(mapping, folio);
} else if (truncate_inode_partial_folio(folio, lstart, lend)) {
/*
- * If we split a page, reset the loop so
- * that we pick up the new sub pages.
- * Otherwise the THP was entirely
- * dropped or the target range was
- * zeroed, so just continue the loop as
- * is.
+ * The folio was split. Retry the scan
+ * to find the split folios.
*/
- if (!folio_test_large(folio)) {
- folio_unlock(folio);
- index = start;
- break;
- }
+ folio_unlock(folio);
+ index = start;
+ break;
+ } else if (folio->mapping) {
+ /*
+ * Splitting failed, but the part in the
+ * range is already zeroed. The folio
+ * was not dropped, so skip it when
+ * restarting the scan to avoid retrying
+ * it forever.
+ */
+ start = max(start, folio_next_index(folio));
}
}
folio_unlock(folio);
@@ -1341,10 +1358,11 @@ static int shmem_setattr(struct mnt_idmap *idmap,
if (error)
return error;
- if ((info->seals & F_SEAL_EXEC) && (attr->ia_valid & ATTR_MODE)) {
- if ((inode->i_mode ^ attr->ia_mode) & 0111) {
- return -EPERM;
- }
+ if (attr->ia_valid & ATTR_MODE) {
+ error = memfd_check_seals_mode(info->seals, inode->i_mode,
+ attr->ia_mode);
+ if (error)
+ return error;
}
if (S_ISREG(inode->i_mode) && (attr->ia_valid & ATTR_SIZE)) {
@@ -1373,7 +1391,8 @@ static int shmem_setattr(struct mnt_idmap *idmap,
if (oldsize > holebegin)
unmap_mapping_range(inode->i_mapping,
holebegin, 0, 1);
- if (info->alloced)
+ /* Swapin-error markers are not counted in alloced. */
+ if (info->alloced || !mapping_empty(inode->i_mapping))
shmem_truncate_range(inode,
newsize, (loff_t)-1);
/* unmap again to remove racily COWed private pages */
@@ -1544,6 +1563,10 @@ static int shmem_unuse_inode(struct inode *inode, unsigned int type)
if (ret < 0)
break;
+ if (signal_pending(current)) {
+ ret = -ERESTARTSYS;
+ break;
+ }
start = indices[folio_batch_count(&fbatch) - 1];
} while (true);
@@ -2171,11 +2194,10 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index,
folio_put_swap(folio, NULL);
swap_cache_del_folio(folio);
/*
- * Don't treat swapin error folio as alloced. Otherwise inode->i_blocks
- * won't be 0 when inode is released and thus trigger WARN_ON(i_blocks)
- * in shmem_evict_inode().
+ * Let shmem_recalc_inode() reduce alloced and release the block charge
+ * for the freed swap pages.
*/
- shmem_recalc_inode(inode, -nr_pages, -nr_pages);
+ shmem_recalc_inode(inode, 0, -nr_pages);
}
static int shmem_split_large_entry(struct inode *inode, pgoff_t index,
@@ -2316,6 +2338,8 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index,
else
error = -ENOMEM;
folio = NULL;
+ /* Make sure to validate the original value. */
+ swap = index_entry;
goto failed;
}
if (fault_type) {
@@ -5198,12 +5222,35 @@ static const struct file_operations shmem_file_operations = {
#endif
};
+#ifdef CONFIG_TMPFS_POSIX_ACL
+static int shmem_set_acl(struct mnt_idmap *idmap, struct dentry *dentry,
+ struct posix_acl *acl, int type)
+{
+ struct inode *inode = d_inode(dentry);
+ umode_t mode = inode->i_mode;
+ int error;
+
+ if (type == ACL_TYPE_ACCESS) {
+ error = posix_acl_equiv_mode(acl, &mode);
+ if (error < 0)
+ return error;
+ error = memfd_check_seals_mode(SHMEM_I(inode)->seals,
+ inode->i_mode, mode);
+ if (error)
+ return error;
+ }
+ return simple_set_acl(idmap, dentry, acl, type);
+}
+#else
+#define shmem_set_acl simple_set_acl
+#endif
+
static const struct inode_operations shmem_inode_operations = {
.getattr = shmem_getattr,
.setattr = shmem_setattr,
#ifdef CONFIG_TMPFS_XATTR
.listxattr = shmem_listxattr,
- .set_acl = simple_set_acl,
+ .set_acl = shmem_set_acl,
.fileattr_get = shmem_fileattr_get,
.fileattr_set = shmem_fileattr_set,
#endif
diff --git a/mm/shmem_quota.c b/mm/shmem_quota.c
index d0b92d6da50f..eeca9efd448a 100644
--- a/mm/shmem_quota.c
+++ b/mm/shmem_quota.c
@@ -234,7 +234,7 @@ static int shmem_acquire_dquot(struct dquot *dquot)
return ret;
}
-static bool shmem_is_empty_dquot(struct dquot *dquot)
+static bool shmem_is_default_dquot(struct dquot *dquot)
{
struct shmem_sb_info *sbinfo = dquot->dq_sb->s_fs_info;
qsize_t bhardlimit;
@@ -248,19 +248,21 @@ static bool shmem_is_empty_dquot(struct dquot *dquot)
ihardlimit = sbinfo->qlimits.grpquota_ihardlimit;
}
- if (test_bit(DQ_FAKE_B, &dquot->dq_flags) ||
- (dquot->dq_dqb.dqb_curspace == 0 &&
- dquot->dq_dqb.dqb_curinodes == 0 &&
- dquot->dq_dqb.dqb_bhardlimit == bhardlimit &&
- dquot->dq_dqb.dqb_ihardlimit == ihardlimit))
- return true;
-
- return false;
+ /*
+ * Defaults are zero usage and soft limits, and the mount's hard limits.
+ * DQ_FAKE_B only means that *all* limits are zero, which need not match
+ * the mount defaults.
+ */
+ return dquot->dq_dqb.dqb_curspace == 0 &&
+ dquot->dq_dqb.dqb_curinodes == 0 &&
+ dquot->dq_dqb.dqb_bsoftlimit == 0 &&
+ dquot->dq_dqb.dqb_isoftlimit == 0 &&
+ dquot->dq_dqb.dqb_bhardlimit == bhardlimit &&
+ dquot->dq_dqb.dqb_ihardlimit == ihardlimit;
}
/*
- * Store limits from dquot in the tree unless it's fake. If it is fake
- * remove the id from the tree since there is no useful information in
- * there.
+ * Remove the ID from the tree if it has no usage and its limits match
+ * the mount defaults. Otherwise, save its limits for the next acquire.
*/
static int shmem_release_dquot(struct dquot *dquot)
{
@@ -295,7 +297,7 @@ static int shmem_release_dquot(struct dquot *dquot)
return -ENOENT;
found:
- if (shmem_is_empty_dquot(dquot)) {
+ if (shmem_is_default_dquot(dquot)) {
/* Remove entry from the tree */
rb_erase(&entry->node, info->dqi_priv);
kfree(entry);
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 601979b97f95..278060e24113 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -2434,7 +2434,8 @@ static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd,
return 0;
swapcache = folio;
- folio = ksm_might_need_to_copy(folio, vma, addr);
+ folio = ksm_might_need_to_copy(folio,
+ folio_file_page(folio, swp_offset(entry)), vma, addr);
if (unlikely(!folio))
return -ENOMEM;
else if (unlikely(folio == ERR_PTR(-EHWPOISON))) {
@@ -2609,6 +2610,8 @@ static inline int unuse_pmd_range(struct vm_area_struct *vma, pud_t *pud,
pmd = pmd_offset(pud, addr);
do {
cond_resched();
+ if (signal_pending(current))
+ return -ERESTARTSYS;
next = pmd_addr_end(addr, end);
ret = unuse_pte_range(vma, pmd, addr, next, type);
if (ret)
@@ -2823,7 +2826,7 @@ static int try_to_unuse(unsigned int type)
if (swap_usage_in_pages(si)) {
if (!signal_pending(current))
goto retry;
- return -EINTR;
+ return -ERESTARTSYS;
}
success:
diff --git a/mm/truncate.c b/mm/truncate.c
index 29782aced8ba..b7c5d3384748 100644
--- a/mm/truncate.c
+++ b/mm/truncate.c
@@ -436,8 +436,11 @@ void truncate_inode_pages_range(struct address_space *mapping,
}
}
+ if (start >= end)
+ return;
+
index = start;
- while (index < end) {
+ while (true) {
cond_resched();
if (!find_get_entries(mapping, &index, end - 1, &fbatch,
indices)) {
@@ -460,6 +463,7 @@ void truncate_inode_pages_range(struct address_space *mapping,
folio_lock(folio);
VM_BUG_ON_FOLIO(!folio_contains(folio, indices[i]), folio);
folio_wait_writeback(folio);
+ /* Do not handle errors, we'll do another lookup */
truncate_inode_folio(mapping, folio);
folio_unlock(folio);
}
@@ -721,7 +725,12 @@ int invalidate_inode_pages2_range(struct address_space *mapping,
folio_lock(folio);
if (unlikely(folio->mapping != mapping)) {
folio_unlock(folio);
- continue;
+ /*
+ * Folio might have been replaced: look up
+ * again.
+ */
+ index = indices[i];
+ break;
}
VM_BUG_ON_FOLIO(!folio_contains(folio, indices[i]), folio);
folio_wait_writeback(folio);
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 500962be1162..f8e2b7f2f61c 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -115,8 +115,7 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
vma = vma_lookup(mm, addr);
if (!vma)
vma = ERR_PTR(-ENOENT);
- else if (!(vma->vm_flags & VM_SHARED) &&
- unlikely(anon_vma_prepare(vma)))
+ else if (vma_is_cow_mapping(vma) && unlikely(anon_vma_prepare(vma)))
vma = ERR_PTR(-ENOMEM);
return vma;
@@ -144,7 +143,7 @@ static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,
* We know we're going to need to use anon_vma, so check
* that early.
*/
- if (!(vma->vm_flags & VM_SHARED) && unlikely(!vma->anon_vma))
+ if (vma_is_cow_mapping(vma) && unlikely(!vma->anon_vma))
vma_end_read(vma);
else
return vma;
@@ -1234,33 +1233,6 @@ int mrwprotect_range(struct userfaultfd_ctx *ctx, unsigned long start,
return 0;
}
-void double_pt_lock(spinlock_t *ptl1,
- spinlock_t *ptl2)
- __acquires(ptl1)
- __acquires(ptl2)
-{
- if (ptl1 > ptl2)
- swap(ptl1, ptl2);
- /* lock in virtual address order to avoid lock inversion */
- spin_lock(ptl1);
- if (ptl1 != ptl2)
- spin_lock_nested(ptl2, SINGLE_DEPTH_NESTING);
- else
- __acquire(ptl2);
-}
-
-void double_pt_unlock(spinlock_t *ptl1,
- spinlock_t *ptl2)
- __releases(ptl1)
- __releases(ptl2)
-{
- spin_unlock(ptl1);
- if (ptl1 != ptl2)
- spin_unlock(ptl2);
- else
- __release(ptl2);
-}
-
static inline bool is_pte_pages_stable(pte_t *dst_pte, pte_t *src_pte,
pte_t orig_dst_pte, pte_t orig_src_pte,
pmd_t *dst_pmd, pmd_t dst_pmdval)
@@ -4021,6 +3993,8 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx,
ret = 0;
for_each_vma_range(vmi, vma, end) {
+ struct userfaultfd_wake_range range;
+
cond_resched();
/* VMA not registered with userfaultfd. */
@@ -4035,18 +4009,17 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx,
start = vma->vm_start;
vma_end = min(end, vma->vm_end);
- if (userfaultfd_missing(vma)) {
- /*
- * Wake any concurrent pending userfault while
- * we unregister, so they will not hang
- * permanently and it avoids userland to call
- * UFFDIO_WAKE explicitly.
- */
- struct userfaultfd_wake_range range;
- range.start = start;
- range.len = vma_end - start;
- wake_userfault(vma->vm_userfaultfd_ctx.ctx, &range);
- }
+ /* Prevent new userfaults from being queued after the wakeup. */
+ vma_start_write(vma);
+
+ /*
+ * Wake any concurrent pending userfault while we unregister,
+ * so they will not hang permanently and it avoids userland to
+ * call UFFDIO_WAKE explicitly.
+ */
+ range.start = start;
+ range.len = vma_end - start;
+ wake_userfault(ctx, &range);
vma = userfaultfd_clear_vma(&vmi, prev, vma,
start, vma_end);
diff --git a/mm/util.c b/mm/util.c
index bf0513d1d3d0..e31eb565445c 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -697,6 +697,19 @@ void *vcalloc_noprof(size_t n, size_t size)
}
EXPORT_SYMBOL(vcalloc_noprof);
+/**
+ * folio_anon_vma - Get a folio's raw anon_vma pointer.
+ * @folio: The folio.
+ *
+ * The returned pointer may be stale for an unmapped folio. The caller
+ * must ensure the anon_vma stays alive before dereferencing it. A folio
+ * reference or lock alone is insufficient; use folio_get_anon_vma() when
+ * the anon_vma's lifetime is not otherwise protected. Note that
+ * folio_get_anon_vma() is only useful for folios that are still mapped.
+ *
+ * Return: The raw anon_vma pointer for an anonymous, non-KSM folio, or
+ * %NULL otherwise.
+ */
struct anon_vma *folio_anon_vma(const struct folio *folio)
{
unsigned long mapping = (unsigned long)folio->mapping;
diff --git a/mm/vma.c b/mm/vma.c
index 3d63f73e723b..e382fd32f0ff 100644
--- a/mm/vma.c
+++ b/mm/vma.c
@@ -535,6 +535,15 @@ void unmap_region(struct unmap_desc *unmap)
tlb_finish_mmu(&tlb);
}
+static int check_split(struct vm_area_struct *vma, unsigned long addr)
+{
+ if (!vma->vm_ops)
+ return 0;
+ if (!vma->vm_ops->may_split)
+ return 0;
+ return vma->vm_ops->may_split(vma, addr);
+}
+
/*
* __split_vma() bypasses sysctl_max_map_count checking. We use this where it
* has already been checked or doesn't make sense to fail.
@@ -551,12 +560,11 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
WARN_ON(vma->vm_start >= addr);
WARN_ON(vma->vm_end <= addr);
- if (vma->vm_ops && vma->vm_ops->may_split) {
- err = vma->vm_ops->may_split(vma, addr);
- if (err)
- return err;
- }
+ err = check_split(vma, addr);
+ if (err)
+ return err;
+ vma_start_write(vma);
new = vm_area_dup(vma);
if (!new)
return -ENOMEM;
@@ -587,7 +595,6 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
if (new->vm_ops && new->vm_ops->open)
new->vm_ops->open(new);
- vma_start_write(vma);
vma_start_write(new);
init_vma_prep(&vp, vma);
@@ -964,6 +971,10 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(
if (vmg->__remove_middle && !can_merge_remove_vma(middle))
return NULL;
+ /* If middle isn't removed then it's resized, i.e. splits. */
+ if (!vmg->__remove_middle && check_split(middle, left_side ? end : start))
+ return NULL;
+
/*
* If we merge both VMAs, then next is also deleted. This implies
* merge_will_delete_vma also.
@@ -1143,6 +1154,7 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
struct vm_area_struct *next = vmg->next;
unsigned long end = vmg->end;
bool can_merge_left, can_merge_right;
+ bool move_iter_back = false;
mmap_assert_write_locked(vmg->mm);
VM_WARN_ON_VMG(vmg->middle, vmg);
@@ -1182,23 +1194,30 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
vmg->end = end;
/* In expand-only case we are already positioned at prev. */
- if (!vmg->just_expand) {
- /* Equivalent to going to the previous range. */
- vma_prev(vmg->vmi);
- }
+ if (!vmg->just_expand)
+ move_iter_back = true;
}
+ if (!can_merge_left && !can_merge_right)
+ return NULL;
+
+ if (move_iter_back)
+ vma_prev(vmg->vmi);
+
/*
* Now try to expand adjacent VMA(s). This takes care of removing the
* following VMA if we have VMAs on both sides.
*/
- if (vmg->target && !vma_expand(vmg)) {
- khugepaged_enter_vma(vmg->target, vmg->vm_flags);
- vmg->state = VMA_MERGE_SUCCESS;
- return vmg->target;
+ if (vma_expand(vmg)) {
+ /* Error occurred, restore iterator. */
+ if (move_iter_back)
+ vma_iter_next_range(vmg->vmi);
+ return NULL;
}
- return NULL;
+ khugepaged_enter_vma(vmg->target, vmg->vm_flags);
+ vmg->state = VMA_MERGE_SUCCESS;
+ return vmg->target;
}
/*
@@ -1293,7 +1312,7 @@ int vma_expand(struct vma_merge_struct *vmg)
if (!ret && vmg->copied_from)
ret = dup_anon_vma(target, vmg->copied_from, &anon_dup);
if (ret)
- return ret;
+ goto nomem;
if (remove_next) {
vma_flags_t next_sticky;
@@ -3241,7 +3260,8 @@ static int acct_stack_growth(struct vm_area_struct *vma,
* Overcommit.. This must be the final test, as it will
* update security statistics.
*/
- if (security_vm_enough_memory_mm(mm, grow))
+ if (vma_test(vma, VMA_ACCOUNT_BIT) &&
+ security_vm_enough_memory_mm(mm, grow))
return -ENOMEM;
return 0;
diff --git a/mm/vma_init.c b/mm/vma_init.c
index baa7e82f47e3..d661f229a734 100644
--- a/mm/vma_init.c
+++ b/mm/vma_init.c
@@ -127,6 +127,9 @@ struct vm_area_struct *vm_area_dup(struct vm_area_struct *orig)
{
struct vm_area_struct *new = kmem_cache_alloc(vm_area_cachep, GFP_KERNEL);
+ /* Must hold a write lock to stabilise atomic VMA flags. */
+ vma_assert_write_locked(orig);
+
if (!new)
return NULL;
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 6a53762d5544..88fd31d13585 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3194,8 +3194,11 @@ struct ctrl_pos {
int gain;
};
-static void read_ctrl_pos(struct lruvec *lruvec, int type, int tier, int gain,
- struct ctrl_pos *pos)
+/*
+ * __noipa works around gcc-16 warning for uninitialized use of pos->refaulted
+ */
+static void __noipa read_ctrl_pos(struct lruvec *lruvec, int type, int tier_min,
+ int tier_max, int gain, struct ctrl_pos *pos)
{
int i;
struct lru_gen_folio *lrugen = &lruvec->lrugen;
@@ -3204,7 +3207,7 @@ static void read_ctrl_pos(struct lruvec *lruvec, int type, int tier, int gain,
pos->gain = gain;
pos->refaulted = pos->total = 0;
- for (i = tier % MAX_NR_TIERS; i <= min(tier, MAX_NR_TIERS - 1); i++) {
+ for (i = tier_min; i <= tier_max; i++) {
pos->refaulted += lrugen->avg_refaulted[type][i] +
atomic_long_read(&lrugen->refaulted[hist][type][i]);
pos->total += lrugen->avg_total[type][i] +
@@ -3771,7 +3774,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end,
walk->mm_stats[MM_NONLEAF_FOUND]++;
- if (!walk_pte_range(&val, addr, next, args))
+ if (!walk_pte_range(pmd + i, addr, next, args))
continue;
walk->mm_stats[MM_NONLEAF_ADDED]++;
@@ -3807,7 +3810,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long start, unsigned long end,
if (!pud_present(val) || WARN_ON_ONCE(pud_leaf(val)))
continue;
- walk_pmd_range(&val, addr, next, args);
+ walk_pmd_range(pud + i, addr, next, args);
if (need_resched() || walk->batched >= MAX_LRU_BATCH) {
end = (addr | ~PUD_MASK) + 1;
@@ -4801,16 +4804,16 @@ static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
static int get_tier_idx(struct lruvec *lruvec, int type)
{
int tier;
- struct ctrl_pos sp, pv = {};
+ struct ctrl_pos sp, pv;
/*
* To leave a margin for fluctuations, use a larger gain factor (2:3).
* This value is chosen because any other tier would have at least twice
* as many refaults as the first tier.
*/
- read_ctrl_pos(lruvec, type, 0, 2, &sp);
- for (tier = 1; tier < MAX_NR_TIERS; tier++) {
- read_ctrl_pos(lruvec, type, tier, 3, &pv);
+ read_ctrl_pos(lruvec, type, LRU_TIER_MIN, LRU_TIER_MIN, 2, &sp);
+ for (tier = LRU_TIER_MIN + 1; tier <= LRU_TIER_MAX; tier++) {
+ read_ctrl_pos(lruvec, type, tier, tier, 3, &pv);
if (!positive_ctrl_err(&sp, &pv))
break;
}
@@ -4820,7 +4823,7 @@ static int get_tier_idx(struct lruvec *lruvec, int type)
static int get_type_to_scan(struct lruvec *lruvec, int swappiness)
{
- struct ctrl_pos sp, pv = {};
+ struct ctrl_pos sp, pv;
if (swappiness <= MIN_SWAPPINESS + 1)
return LRU_GEN_FILE;
@@ -4831,8 +4834,10 @@ static int get_type_to_scan(struct lruvec *lruvec, int swappiness)
* Compare the sum of all tiers of anon with that of file to determine
* which type to scan.
*/
- read_ctrl_pos(lruvec, LRU_GEN_ANON, MAX_NR_TIERS, swappiness, &sp);
- read_ctrl_pos(lruvec, LRU_GEN_FILE, MAX_NR_TIERS, MAX_SWAPPINESS - swappiness, &pv);
+ read_ctrl_pos(lruvec, LRU_GEN_ANON, LRU_TIER_MIN, LRU_TIER_MAX,
+ swappiness, &sp);
+ read_ctrl_pos(lruvec, LRU_GEN_FILE, LRU_TIER_MIN, LRU_TIER_MAX,
+ MAX_SWAPPINESS - swappiness, &pv);
return positive_ctrl_err(&sp, &pv);
}
diff --git a/net/9p/trans_usbg.c b/net/9p/trans_usbg.c
index 419cda13a7b5..7d600e1288a3 100644
--- a/net/9p/trans_usbg.c
+++ b/net/9p/trans_usbg.c
@@ -149,7 +149,8 @@ static void usb9pfs_tx_complete(struct usb_ep *ep, struct usb_request *req)
{
struct f_usb9pfs *usb9pfs = ep->driver_data;
struct usb_composite_dev *cdev = usb9pfs->function.config->cdev;
- struct p9_req_t *p9_tx_req = req->context;
+ struct p9_client *client;
+ struct p9_req_t *p9_tx_req;
unsigned long flags;
/* reset zero packages */
@@ -165,18 +166,25 @@ static void usb9pfs_tx_complete(struct usb_ep *ep, struct usb_request *req)
ep->name, req->status, req->actual, req->length);
spin_lock_irqsave(&usb9pfs->lock, flags);
- WRITE_ONCE(p9_tx_req->status, REQ_STATUS_SENT);
+ client = usb9pfs->client;
+ p9_tx_req = req->context;
+ req->context = NULL;
- p9_req_put(usb9pfs->client, p9_tx_req);
+ if (!client || !p9_tx_req)
+ goto unlock_complete;
- req->context = NULL;
+ WRITE_ONCE(p9_tx_req->status, REQ_STATUS_SENT);
+
+ p9_req_put(client, p9_tx_req);
+unlock_complete:
spin_unlock_irqrestore(&usb9pfs->lock, flags);
complete(&usb9pfs->send);
}
-static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
+static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs,
+ struct p9_client *client, void *buf)
{
struct p9_req_t *p9_rx_req;
struct p9_fcall rc;
@@ -202,7 +210,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
"mux %p pkt: size: %d bytes tag: %d\n",
usb9pfs, rc.size, rc.tag);
- p9_rx_req = p9_tag_lookup(usb9pfs->client, rc.tag);
+ p9_rx_req = p9_tag_lookup(client, rc.tag);
if (!p9_rx_req || p9_rx_req->status != REQ_STATUS_SENT) {
p9_debug(P9_DEBUG_ERROR, "Unexpected packet tag %d\n", rc.tag);
return NULL;
@@ -212,7 +220,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
p9_debug(P9_DEBUG_ERROR,
"requested packet size too big: %d for tag %d with capacity %zd\n",
rc.size, rc.tag, p9_rx_req->rc.capacity);
- p9_req_put(usb9pfs->client, p9_rx_req);
+ p9_req_put(client, p9_rx_req);
return NULL;
}
@@ -220,7 +228,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
p9_debug(P9_DEBUG_ERROR,
"No recv fcall for tag %d (req %p), disconnecting!\n",
rc.tag, p9_rx_req);
- p9_req_put(usb9pfs->client, p9_rx_req);
+ p9_req_put(client, p9_rx_req);
return NULL;
}
@@ -231,8 +239,10 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req)
{
struct f_usb9pfs *usb9pfs = ep->driver_data;
struct usb_composite_dev *cdev = usb9pfs->function.config->cdev;
+ struct p9_client *client;
struct p9_req_t *p9_rx_req;
unsigned int req_size = req->actual;
+ unsigned long flags;
int status = REQ_STATUS_RCVD;
if (req->status) {
@@ -241,9 +251,16 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req)
return;
}
- p9_rx_req = usb9pfs_rx_header(usb9pfs, req->buf);
- if (!p9_rx_req)
+ spin_lock_irqsave(&usb9pfs->lock, flags);
+ client = usb9pfs->client;
+ if (!client) {
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
return;
+ }
+
+ p9_rx_req = usb9pfs_rx_header(usb9pfs, client, req->buf);
+ if (!p9_rx_req)
+ goto out_unlock;
if (req_size > p9_rx_req->rc.capacity) {
dev_err(&cdev->gadget->dev,
@@ -257,8 +274,11 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req)
p9_rx_req->rc.size = req_size;
- p9_client_cb(usb9pfs->client, p9_rx_req, status);
- p9_req_put(usb9pfs->client, p9_rx_req);
+ p9_client_cb(client, p9_rx_req, status);
+ p9_req_put(client, p9_rx_req);
+
+out_unlock:
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
complete(&usb9pfs->received);
}
@@ -278,18 +298,18 @@ static void disable_usb9pfs(struct f_usb9pfs *usb9pfs)
struct usb_composite_dev *cdev =
usb9pfs->function.config->cdev;
+ disable_ep(cdev, usb9pfs->in_ep);
+ disable_ep(cdev, usb9pfs->out_ep);
+
if (usb9pfs->in_req) {
usb_ep_free_request(usb9pfs->in_ep, usb9pfs->in_req);
usb9pfs->in_req = NULL;
}
if (usb9pfs->out_req) {
- usb_ep_free_request(usb9pfs->out_ep, usb9pfs->out_req);
+ free_ep_req(usb9pfs->out_ep, usb9pfs->out_req);
usb9pfs->out_req = NULL;
}
-
- disable_ep(cdev, usb9pfs->in_ep);
- disable_ep(cdev, usb9pfs->out_ep);
dev_dbg(&cdev->gadget->dev, "%s disabled\n",
usb9pfs->function.name);
}
@@ -314,10 +334,6 @@ static int alloc_requests(struct usb_composite_dev *cdev,
usb9pfs->in_req->complete = usb9pfs_tx_complete;
usb9pfs->out_req->complete = usb9pfs_rx_complete;
- /* length will be set in complete routine */
- usb9pfs->in_req->context = usb9pfs;
- usb9pfs->out_req->context = usb9pfs;
-
return 0;
fail_in:
@@ -391,7 +407,7 @@ static int p9_usbg_create(struct p9_client *client, struct fs_context *fc)
guard(mutex)(&usb9pfs_lock);
list_for_each_entry(dev, &usbg_instance_list, usb9pfs_instance) {
- if (!strncmp(devname, dev->tag, strlen(devname))) {
+ if (!strcmp(devname, dev->tag)) {
if (!dev->inuse) {
dev->inuse = true;
found = 1;
@@ -416,7 +432,9 @@ static int p9_usbg_create(struct p9_client *client, struct fs_context *fc)
client->status = Disconnected;
else
client->status = Connected;
+ spin_lock_irq(&usb9pfs->lock);
usb9pfs->client = client;
+ spin_unlock_irq(&usb9pfs->lock);
client->trans_mod->maxsize = usb9pfs->buflen;
@@ -427,18 +445,26 @@ static int p9_usbg_create(struct p9_client *client, struct fs_context *fc)
static void usb9pfs_clear_tx(struct f_usb9pfs *usb9pfs)
{
+ struct p9_client *client;
struct p9_req_t *req;
+ unsigned long flags;
- guard(spinlock_irqsave)(&usb9pfs->lock);
+ spin_lock_irqsave(&usb9pfs->lock, flags);
+ client = usb9pfs->client;
+ usb9pfs->client = NULL;
+ req = usb9pfs->in_req ? usb9pfs->in_req->context : NULL;
+ if (usb9pfs->in_req)
+ usb9pfs->in_req->context = NULL;
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
- req = usb9pfs->in_req->context;
- if (!req)
+ if (!req || !client)
return;
if (!req->t_err)
req->t_err = -ECONNRESET;
- p9_client_cb(usb9pfs->client, req, REQ_STATUS_ERROR);
+ p9_client_cb(client, req, REQ_STATUS_ERROR);
+ usb9pfs->in_req->context = NULL;
}
static void p9_usbg_close(struct p9_client *client)
@@ -466,6 +492,9 @@ static void p9_usbg_close(struct p9_client *client)
mutex_lock(&usb9pfs_lock);
dev->inuse = false;
mutex_unlock(&usb9pfs_lock);
+
+ disable_usb9pfs(usb9pfs);
+ reinit_completion(&usb9pfs->send);
}
static int p9_usbg_request(struct p9_client *client, struct p9_req_t *p9_req)
@@ -748,8 +777,18 @@ static int usb9pfs_set_alt(struct usb_function *f,
static void usb9pfs_disable(struct usb_function *f)
{
struct f_usb9pfs *usb9pfs = func_to_usb9pfs(f);
+ unsigned long flags;
+ spin_lock_irqsave(&usb9pfs->lock, flags);
+ if (usb9pfs->client)
+ usb9pfs->client->status = Disconnected;
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
usb9pfs_clear_tx(usb9pfs);
+
+ if (usb9pfs->in_ep->enabled)
+ disable_usb9pfs(usb9pfs);
+
+ reinit_completion(&usb9pfs->send);
}
static struct usb_function *usb9pfs_alloc(struct usb_function_instance *fi)
diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c
index b0d0094ec8e2..9fa63e3ebf67 100644
--- a/net/9p/trans_virtio.c
+++ b/net/9p/trans_virtio.c
@@ -633,6 +633,11 @@ static int p9_virtio_probe(struct virtio_device *vdev)
err = -EINVAL;
goto out_free_vq;
}
+ if (tag_len > NAME_MAX) {
+ dev_err(&vdev->dev, "mount tag too long (%u bytes)\n", tag_len);
+ err = -EINVAL;
+ goto out_free_vq;
+ }
tag = kzalloc(tag_len + 1, GFP_KERNEL);
if (!tag) {
err = -ENOMEM;
diff --git a/net/9p/trans_xen.c b/net/9p/trans_xen.c
index f9fb2db7a066..71a4d276321e 100644
--- a/net/9p/trans_xen.c
+++ b/net/9p/trans_xen.c
@@ -200,9 +200,19 @@ static void p9_xen_response(struct work_struct *work)
masked_prod, &masked_cons,
XEN_9PFS_RING_SIZE(ring));
+ if (h.size < sizeof(h) ||
+ h.size > xen_9pfs_queued(prod, cons,
+ XEN_9PFS_RING_SIZE(ring))) {
+ dev_warn_ratelimited(&priv->dev->dev,
+ "bad response size %u from backend\n", h.size);
+ break;
+ }
+
req = p9_tag_lookup(priv->client, h.tag);
if (!req || req->status != REQ_STATUS_SENT) {
dev_warn(&priv->dev->dev, "Wrong req tag=%x\n", h.tag);
+ if (req)
+ p9_req_put(priv->client, req);
cons += h.size;
virt_mb();
ring->intf->in_cons = cons;
diff --git a/net/batman-adv/bat_iv_ogm.c b/net/batman-adv/bat_iv_ogm.c
index 53fbdbbe8f4f..1c2ab79adabc 100644
--- a/net/batman-adv/bat_iv_ogm.c
+++ b/net/batman-adv/bat_iv_ogm.c
@@ -1909,6 +1909,11 @@ static void batadv_iv_send_outstanding_bat_ogm_packet(struct work_struct *work)
goto out;
}
+ /* Mark aggregate as full before forcing emit. */
+ spin_lock_bh(&bat_priv->forw_bat_list_lock);
+ forw_packet->num_packets = BATADV_MAX_AGGREGATION_PACKETS;
+ spin_unlock_bh(&bat_priv->forw_bat_list_lock);
+
batadv_iv_ogm_emit(forw_packet);
/* we have to have at least one packet in the queue to determine the
diff --git a/net/batman-adv/distributed-arp-table.c b/net/batman-adv/distributed-arp-table.c
index 011cfdc47fab..86791ddc148f 100644
--- a/net/batman-adv/distributed-arp-table.c
+++ b/net/batman-adv/distributed-arp-table.c
@@ -653,6 +653,7 @@ static void batadv_choose_next_candidate(struct batadv_priv *bat_priv,
* NOT_FOUND
*/
cands[select].type = BATADV_DAT_CANDIDATE_NOT_FOUND;
+ cands[select].orig_node = NULL;
/* iterate over the originator list and find the node with the closest
* dat_address which has not been selected yet
diff --git a/net/batman-adv/main.c b/net/batman-adv/main.c
index d89d44706269..5d5a0be323e6 100644
--- a/net/batman-adv/main.c
+++ b/net/batman-adv/main.c
@@ -294,6 +294,11 @@ void batadv_mesh_free(struct net_device *mesh_iface)
WRITE_ONCE(bat_priv->mesh_state, BATADV_MESH_DEACTIVATING);
+ /* the originator purge worker accesses (among others) the TT tables
+ * and must therefore be stopped before any of them are freed
+ */
+ disable_delayed_work_sync(&bat_priv->orig_work);
+
batadv_purge_outstanding_packets(bat_priv, NULL);
batadv_tp_stop_all(bat_priv);
@@ -483,8 +488,10 @@ int batadv_batman_skb_recv(struct sk_buff *skb, struct net_device *dev,
* shut down otherwise the packet may trigger de-reference errors
* further down in the receive path.
*/
- if (!kref_get_unless_zero(&hard_iface->refcount))
- goto err_out;
+ if (!kref_get_unless_zero(&hard_iface->refcount)) {
+ kfree_skb(skb);
+ return NET_RX_DROP;
+ }
skb = skb_share_check(skb, GFP_ATOMIC);
@@ -540,7 +547,6 @@ int batadv_batman_skb_recv(struct sk_buff *skb, struct net_device *dev,
kfree_skb(skb);
err_put:
batadv_hardif_put(hard_iface);
-err_out:
return NET_RX_DROP;
}
diff --git a/net/batman-adv/multicast_forw.c b/net/batman-adv/multicast_forw.c
index ac97cd8b81dc..9ffed1f237b6 100644
--- a/net/batman-adv/multicast_forw.c
+++ b/net/batman-adv/multicast_forw.c
@@ -950,8 +950,11 @@ static int batadv_mcast_forw_packet(struct batadv_priv *bat_priv,
dest = (u8 *)mcast_tracker + sizeof(*mcast_tracker);
- /* check if full tracker tvlv is within skb length */
+ /* check if full tracker tvlv, including padding, is within skb length */
tvlv_len = sizeof(*mcast_tracker) + ETH_ALEN * num_dests;
+ if (!(num_dests % 2))
+ tvlv_len += 2;
+
if (tvlv_len > skb_network_header_len(skb))
return -EINVAL;
diff --git a/net/batman-adv/tp_meter.c b/net/batman-adv/tp_meter.c
index 022eb13e8ec4..56feb02d2b7e 100644
--- a/net/batman-adv/tp_meter.c
+++ b/net/batman-adv/tp_meter.c
@@ -221,7 +221,7 @@ static void batadv_tp_batctl_notify(enum batadv_tp_meter_reason reason,
unsigned long start_time, u64 total_sent,
u32 cookie)
{
- u32 total_bytes;
+ u64 total_bytes;
u32 test_time;
u8 result;
diff --git a/net/batman-adv/translation-table.c b/net/batman-adv/translation-table.c
index 68f79853032e..ed8dcb8a66b1 100644
--- a/net/batman-adv/translation-table.c
+++ b/net/batman-adv/translation-table.c
@@ -418,6 +418,9 @@ batadv_tt_orig_list_entry_put(struct batadv_tt_orig_list_entry *orig_entry)
* @bat_priv: the bat priv with all the mesh interface information
* @tt_local_entry: the TT entry involved in the event
* @event_flags: flags to store in the event structure
+ *
+ * Only a single change per client is queued. It describes the latest state of
+ * the client and a new event therefore replaces an already queued one.
*/
static void batadv_tt_local_event(struct batadv_priv *bat_priv,
struct batadv_tt_local_entry *tt_local_entry,
@@ -429,14 +432,12 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv,
struct batadv_tt_change_node *entry;
struct batadv_tt_change_node *safe;
bool del_op_requested;
- bool del_op_entry;
size_t changes;
tt_change_node = kmem_cache_alloc(batadv_tt_change_cache, GFP_ATOMIC);
if (!tt_change_node)
return;
- tt_change_node->change.flags = flags;
memset(tt_change_node->change.reserved, 0,
sizeof(tt_change_node->change.reserved));
ether_addr_copy(tt_change_node->change.addr, common->addr);
@@ -444,6 +445,15 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv,
del_op_requested = flags & BATADV_TT_CLIENT_DEL;
+ /* the ROAM flag is only meaningful for the receiver in combination
+ * with a DEL. An ADD with ROAM would be interpreted as a roaming
+ * advertisement
+ */
+ if (!del_op_requested)
+ flags &= ~BATADV_TT_CLIENT_ROAM;
+
+ tt_change_node->change.flags = flags;
+
/* check for ADD+DEL, DEL+ADD, ADD+ADD or DEL+DEL events */
spin_lock_bh(&bat_priv->tt.changes_list_lock);
changes = READ_ONCE(bat_priv->tt.local_changes);
@@ -455,25 +465,12 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv,
if (entry->change.vid != tt_change_node->change.vid)
continue;
- del_op_entry = entry->change.flags & BATADV_TT_CLIENT_DEL;
- if (del_op_requested != del_op_entry) {
- /* DEL+ADD in the same orig interval have no effect and
- * can be removed to avoid silly behaviour on the
- * receiver side. The other way around (ADD+DEL) can
- * happen in case of roaming of a client still in the
- * NEW state. Roaming of NEW clients is now possible due
- * to automatically recognition of "temporary" clients
- */
- list_del(&entry->list);
- kmem_cache_free(batadv_tt_change_cache, entry);
- changes--;
- } else {
- /* this is a second add or del in the same originator
- * interval. It could mean that flags have been changed
- * (e.g. double add): update them
- */
- entry->change.flags = flags;
- }
+ /* the other nodes may know the client with different
+ * flags. A DEL+ADD must therefore still announce the
+ * current flags and an ADD+DEL must still remove the
+ * client. ADD+ADD and DEL+DEL only update the flags
+ */
+ entry->change.flags = flags;
kmem_cache_free(batadv_tt_change_cache, tt_change_node);
goto update_changes;
@@ -624,6 +621,7 @@ bool batadv_tt_local_add(struct net_device *mesh_iface, const u8 *addr,
bool iif_is_wifi = false;
struct hlist_head *head;
int packet_size_max;
+ bool added = false;
bool ret = false;
u8 remote_flags;
int hash_added;
@@ -656,7 +654,7 @@ bool batadv_tt_local_add(struct net_device *mesh_iface, const u8 *addr,
* flag can be reset like it was never enqueued
*/
tt_local->common.flags &= ~BATADV_TT_CLIENT_PENDING;
- goto add_event;
+ added = true;
}
if (tt_local->common.flags & BATADV_TT_CLIENT_ROAM) {
@@ -737,14 +735,17 @@ bool batadv_tt_local_add(struct net_device *mesh_iface, const u8 *addr,
goto out;
}
-add_event:
- batadv_tt_local_event(bat_priv, tt_local, BATADV_NO_FLAGS);
+ added = true;
check_roaming:
+ if (added)
+ batadv_tt_local_event(bat_priv, tt_local, BATADV_NO_FLAGS);
+
/* Check whether it is a roaming, but don't do anything if the roaming
* process has already been handled
*/
- if (tt_global && !(tt_global->common.flags & BATADV_TT_CLIENT_ROAM)) {
+ if (tt_global &&
+ (roamed_back || !(tt_global->common.flags & BATADV_TT_CLIENT_ROAM))) {
/* These node are probably going to update their tt table */
head = &tt_global->orig_list;
rcu_read_lock();
@@ -1006,22 +1007,24 @@ batadv_tt_prepare_tvlv_local_data(struct batadv_priv *bat_priv,
* batadv_tt_tvlv_container_update() - update the translation table tvlv
* container after local tt changes have been committed
* @bat_priv: the bat priv with all the mesh interface information
+ * @changes_list: list of committed changes which will be consumed
+ * @tt_diff_entries_num: number of entries in @changes_list
*/
-static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv)
+static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv,
+ struct list_head *changes_list,
+ int tt_diff_entries_num)
{
struct batadv_tvlv_tt_change *tt_change;
struct batadv_tt_change_node *entry;
struct batadv_tvlv_tt_data *tt_data;
struct batadv_tt_change_node *safe;
int tt_diff_entries_count = 0;
- int tt_diff_entries_num = 0;
bool drop_changes = false;
size_t tt_extra_len = 0;
int tt_change_len = 0;
int tt_diff_len;
u16 tvlv_len;
- tt_diff_entries_num = READ_ONCE(bat_priv->tt.local_changes);
tt_diff_len = batadv_tt_len(tt_diff_entries_num);
/* if we have too many changes for one packet don't send any
@@ -1039,19 +1042,39 @@ static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv)
tvlv_len = batadv_tt_prepare_tvlv_local_data(bat_priv, &tt_data,
&tt_change, &tt_diff_len);
- if (!tvlv_len)
+ if (!tvlv_len) {
+ /* the changes are lost and the other nodes have to request the
+ * full table. Don't answer TT requests for the new TTVN with
+ * the stale changeset of the previous TTVN. Without changes, the
+ * TTVN was not incremented and the last changeset is still valid
+ */
+ if (drop_changes || tt_diff_entries_num > 0) {
+ list_for_each_entry_safe(entry, safe, changes_list,
+ list) {
+ list_del(&entry->list);
+ kmem_cache_free(batadv_tt_change_cache, entry);
+ }
+
+ spin_lock_bh(&bat_priv->tt.last_changeset_lock);
+ kfree(bat_priv->tt.last_changeset);
+ bat_priv->tt.last_changeset = NULL;
+ bat_priv->tt.last_changeset_len = 0;
+ spin_unlock_bh(&bat_priv->tt.last_changeset_lock);
+ }
+
+ /* announce the already committed TTVN in the next OGM interval
+ * instead of waiting for BATADV_TT_OGM_APPEND_MAX idle rounds
+ */
+ atomic_set(&bat_priv->tt.ogm_append_cnt, 0);
return;
+ }
tt_data->flags = BATADV_TT_OGM_DIFF;
if (!drop_changes && tt_diff_len == 0)
goto container_register;
- spin_lock_bh(&bat_priv->tt.changes_list_lock);
- WRITE_ONCE(bat_priv->tt.local_changes, 0);
-
- list_for_each_entry_safe(entry, safe, &bat_priv->tt.changes_list,
- list) {
+ list_for_each_entry_safe(entry, safe, changes_list, list) {
if (tt_diff_entries_count < tt_diff_entries_num) {
memcpy(tt_change + tt_diff_entries_count,
&entry->change,
@@ -1061,7 +1084,6 @@ static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv)
list_del(&entry->list);
kmem_cache_free(batadv_tt_change_cache, entry);
}
- spin_unlock_bh(&bat_priv->tt.changes_list_lock);
tt_extra_len = batadv_tt_len(tt_diff_entries_num -
tt_diff_entries_count);
@@ -2830,8 +2852,12 @@ static bool batadv_tt_local_valid(const void *entry_ptr,
if (tt_common_entry->flags & BATADV_TT_CLIENT_NEW)
return false;
+ /* the entry is (still) announced. A ROAM flag of an entry which is
+ * pending to be removed must not be transmitted. Otherwise, the
+ * receiver would handle it like a roaming advertisement
+ */
if (flags)
- *flags = tt_common_entry->flags;
+ *flags = tt_common_entry->flags & ~BATADV_TT_CLIENT_ROAM;
return true;
}
@@ -3408,21 +3434,30 @@ static void _batadv_tt_update_changes(struct batadv_priv *bat_priv,
u16 tt_num_changes, u8 ttvn)
{
int roams;
+ u8 flags;
int i;
for (i = 0; i < tt_num_changes; i++) {
- if ((tt_change + i)->flags & BATADV_TT_CLIENT_DEL) {
- roams = (tt_change + i)->flags & BATADV_TT_CLIENT_ROAM;
+ flags = (tt_change + i)->flags;
+
+ if (flags & BATADV_TT_CLIENT_DEL) {
+ roams = flags & BATADV_TT_CLIENT_ROAM;
batadv_tt_global_del(bat_priv, orig_node,
(tt_change + i)->addr,
ntohs((tt_change + i)->vid),
"tt removed by changes",
roams);
} else {
+ /* an announced client cannot have roamed away from
+ * orig_node. Only a ROAMING_ADV is allowed to add a
+ * global entry with the ROAM flag
+ */
+ flags &= ~BATADV_TT_CLIENT_ROAM;
+
if (!batadv_tt_global_add(bat_priv, orig_node,
(tt_change + i)->addr,
ntohs((tt_change + i)->vid),
- (tt_change + i)->flags, ttvn))
+ flags, ttvn))
/* In case of problem while storing a
* global_entry, we stop the updating
* procedure without committing the
@@ -3883,11 +3918,25 @@ static void batadv_tt_local_purge_pending_clients(struct batadv_priv *bat_priv)
*/
static void batadv_tt_local_commit_changes_nolock(struct batadv_priv *bat_priv)
{
+ LIST_HEAD(changes_list);
+ int changes_num;
+
lockdep_assert_held(&bat_priv->tt.commit_lock);
- if (READ_ONCE(bat_priv->tt.local_changes) == 0) {
+ /* atomically get the list of changes. The TTVN decision and the
+ * announced changeset must cover the same events. Events queued
+ * afterwards stay in the tt.changes_list for the next TT round
+ */
+ spin_lock_bh(&bat_priv->tt.changes_list_lock);
+ list_splice_init(&bat_priv->tt.changes_list, &changes_list);
+ changes_num = READ_ONCE(bat_priv->tt.local_changes);
+ WRITE_ONCE(bat_priv->tt.local_changes, 0);
+ spin_unlock_bh(&bat_priv->tt.changes_list_lock);
+
+ if (changes_num == 0) {
if (!batadv_atomic_dec_not_zero(&bat_priv->tt.ogm_append_cnt))
- batadv_tt_tvlv_container_update(bat_priv);
+ batadv_tt_tvlv_container_update(bat_priv, &changes_list,
+ changes_num);
return;
}
@@ -3904,7 +3953,7 @@ static void batadv_tt_local_commit_changes_nolock(struct batadv_priv *bat_priv)
/* reset the sending counter */
atomic_set(&bat_priv->tt.ogm_append_cnt, BATADV_TT_OGM_APPEND_MAX);
- batadv_tt_tvlv_container_update(bat_priv);
+ batadv_tt_tvlv_container_update(bat_priv, &changes_list, changes_num);
}
/**
diff --git a/net/batman-adv/types.h b/net/batman-adv/types.h
index 9bdc5a3e799e..db0d4f680e65 100644
--- a/net/batman-adv/types.h
+++ b/net/batman-adv/types.h
@@ -2019,7 +2019,13 @@ struct batadv_forw_packet {
/** @direct_link_flags: direct link flags for aggregated OGM packets */
DECLARE_BITMAP(direct_link_flags, BATADV_MAX_AGGREGATION_PACKETS);
- /** @num_packets: counter for aggregated OGMv1 packets */
+ /**
+ * @num_packets: counter for aggregated OGMv1 packets. Set to
+ * BATADV_MAX_AGGREGATION_PACKETS (independent of the real number of
+ * aggregated packets) by the send worker to prevent further
+ * aggregation while the packet is emitted. It must therefore not be
+ * used to iterate over the aggregated packets
+ */
u8 num_packets;
/** @delayed_work: work queue callback item for packet sending */
diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c
index a92a81846e9d..5b99da7028b9 100644
--- a/net/bluetooth/hci_sync.c
+++ b/net/bluetooth/hci_sync.c
@@ -2687,7 +2687,7 @@ static int hci_pause_advertising_sync(struct hci_dev *hdev)
static int hci_resume_advertising_sync(struct hci_dev *hdev)
{
struct adv_info *adv, *tmp;
- int err;
+ int err = 0;
/* If advertising has not been paused there is nothing to do. */
if (!hdev->advertising_paused)
@@ -2720,13 +2720,19 @@ static int hci_resume_advertising_sync(struct hci_dev *hdev)
*/
if (hci_dev_test_and_clear_flag(hdev, HCI_LE_ADV_0))
err = hci_enable_ext_advertising_sync(hdev, 0x00);
- } else {
+ } else if (hdev->cur_adv_instance) {
/* Schedule for most recent instance to be restarted and begin
* the software rotation loop
*/
err = hci_schedule_adv_instance_sync(hdev,
hdev->cur_adv_instance,
true);
+ } else if (hci_dev_test_flag(hdev, HCI_ADVERTISING) &&
+ !hci_dev_test_flag(hdev, HCI_LE_ADV)) {
+ /* hci_schedule_adv_instance_sync() rejects instance 0x00 while
+ * HCI_ADVERTISING is set, so enable it directly.
+ */
+ err = hci_start_adv_sync(hdev, 0x00);
}
hdev->advertising_paused = false;
@@ -6272,6 +6278,7 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
u8 filter_policy = 0x00;
/* Default is to enable duplicates filter */
u8 filter_dup = LE_SCAN_FILTER_DUP_ENABLE;
+ bool paused;
int err;
bt_dev_dbg(hdev, "");
@@ -6295,6 +6302,12 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
if (err)
goto failed;
+ /* LE Set Random Address is disallowed while advertising is enabled. */
+ paused = !hdev->advertising_paused;
+ err = hci_pause_advertising_sync(hdev);
+ if (err)
+ goto failed;
+
/* All active scans will be done with either a resolvable private
* address (when privacy feature has been enabled) or non-resolvable
* private address.
@@ -6304,6 +6317,9 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
if (err < 0)
own_addr_type = ADDR_LE_DEV_PUBLIC;
+ if (paused)
+ hci_resume_advertising_sync(hdev);
+
if (hci_is_adv_monitoring(hdev) ||
(hci_test_quirk(hdev, HCI_QUIRK_STRICT_DUPLICATE_FILTER) &&
hdev->discovery.result_filtering)) {
@@ -6327,9 +6343,8 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
return err;
failed:
- /* Resume advertising if it was paused */
- if (ll_privacy_capable(hdev))
- hci_resume_advertising_sync(hdev);
+ /* No-op when advertising was not paused. */
+ hci_resume_advertising_sync(hdev);
/* Resume passive scanning */
hci_update_passive_scan_sync(hdev);
diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
index b86999899f66..47f0449588ea 100644
--- a/net/bluetooth/l2cap_core.c
+++ b/net/bluetooth/l2cap_core.c
@@ -158,6 +158,28 @@ static struct l2cap_chan *__l2cap_get_chan_by_ident(struct l2cap_conn *conn,
return NULL;
}
+/* Our connection request has been sent and is still outstanding */
+static bool l2cap_chan_connect_pending(struct l2cap_chan *chan)
+{
+ return chan->state == BT_CONNECT &&
+ test_bit(CONF_CONNECT_PEND, &chan->conf_state);
+}
+
+/* Other channels may hold the same ident for a response they still have
+ * to send, so only look at channels waiting for a connection response.
+ */
+static struct l2cap_chan *__l2cap_get_connect_chan_by_ident(struct l2cap_conn *conn,
+ u8 ident)
+{
+ struct l2cap_chan *c;
+
+ list_for_each_entry(c, &conn->chan_l, list) {
+ if (c->ident == ident && l2cap_chan_connect_pending(c))
+ return c;
+ }
+ return NULL;
+}
+
static struct l2cap_chan *__l2cap_global_chan_by_addr(__le16 psm, bdaddr_t *src,
u8 src_type)
{
@@ -4298,7 +4320,7 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn,
if (!chan)
return -EBADSLT;
} else {
- chan = __l2cap_get_chan_by_ident(conn, cmd->ident);
+ chan = __l2cap_get_connect_chan_by_ident(conn, cmd->ident);
if (!chan)
return -EBADSLT;
}
@@ -4311,6 +4333,13 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn,
l2cap_chan_lock(chan);
+ /* Only accept a response matching a connection request we sent and
+ * that is still outstanding. A channel waiting in BT_CONNECT for
+ * security or the feature mask has not sent one yet.
+ */
+ if (!l2cap_chan_connect_pending(chan) || chan->ident != cmd->ident)
+ goto done;
+
switch (result) {
case L2CAP_CR_SUCCESS:
if (__l2cap_get_chan_by_dcid(conn, dcid)) {
@@ -4340,6 +4369,7 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn,
break;
}
+done:
l2cap_chan_unlock(chan);
l2cap_chan_put(chan);
@@ -4472,7 +4502,13 @@ static inline int l2cap_config_req(struct l2cap_conn *conn,
goto unlock;
}
- if (!test_and_set_bit(CONF_REQ_SENT, &chan->conf_state)) {
+ /* A channel in BT_CONNECT2 has not passed security or userspace
+ * accept yet. Answer the peer's request, but hold our own until
+ * the channel reaches BT_CONFIG: the CONNECT2 -> CONFIG sites send
+ * it. Connected channels may also need to reconfigure.
+ */
+ if ((chan->state == BT_CONFIG || chan->state == BT_CONNECTED) &&
+ !test_and_set_bit(CONF_REQ_SENT, &chan->conf_state)) {
u8 buf[64];
l2cap_send_cmd(conn, l2cap_get_ident(conn), L2CAP_CONF_REQ,
l2cap_build_conf_req(chan, buf, sizeof(buf)), buf);
@@ -4520,6 +4556,10 @@ static inline int l2cap_config_rsp(struct l2cap_conn *conn,
if (!chan)
return 0;
+ /* Connected channels may also receive reconfiguration responses. */
+ if (chan->state != BT_CONFIG && chan->state != BT_CONNECTED)
+ goto done;
+
switch (result) {
case L2CAP_CONF_SUCCESS:
l2cap_conf_rfc_get(chan, rsp->data, len);
@@ -4587,6 +4627,9 @@ static inline int l2cap_config_rsp(struct l2cap_conn *conn,
if (test_bit(CONF_OUTPUT_DONE, &chan->conf_state)) {
set_default_fcs(chan);
+ if (chan->state != BT_CONFIG)
+ goto done;
+
if (chan->mode == L2CAP_MODE_ERTM ||
chan->mode == L2CAP_MODE_STREAMING)
err = l2cap_ertm_init(chan);
diff --git a/net/can/af_can.c b/net/can/af_can.c
index 34fe3b28d576..167b2dc40eeb 100644
--- a/net/can/af_can.c
+++ b/net/can/af_can.c
@@ -226,7 +226,7 @@ int can_send(struct sk_buff *skb, int loop)
goto inval_skb;
}
- if (unlikely(skb->dev->type != ARPHRD_CAN)) {
+ if (unlikely(!can_get_ml_priv(skb->dev))) {
err = -EPERM;
goto inval_skb;
}
@@ -452,7 +452,7 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
/* insert new receiver (dev,canid,mask) -> (func,data) */
- if (dev && (dev->type != ARPHRD_CAN || !can_get_ml_priv(dev)))
+ if (dev && !can_get_ml_priv(dev))
return -ENODEV;
if (dev && !net_eq(net, dev_net(dev)))
@@ -462,9 +462,10 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
if (!rcv)
return -ENOMEM;
- spin_lock_bh(&net->can.rcvlists_lock);
-
dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
+
+ spin_lock_bh(&dev_rcv_lists->lock);
+
rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists);
rcv->can_id = can_id;
@@ -478,6 +479,9 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
hlist_add_head_rcu(&rcv->list, rcv_list);
dev_rcv_lists->entries++;
+ spin_unlock_bh(&dev_rcv_lists->lock);
+
+ spin_lock_bh(&net->can.rcvlists_lock);
rcv_lists_stats->rcv_entries++;
rcv_lists_stats->rcv_entries_max = max(rcv_lists_stats->rcv_entries_max,
rcv_lists_stats->rcv_entries);
@@ -509,6 +513,14 @@ static void can_rx_delete_receiver(struct rcu_head *rp)
*
* Description:
* Removes subscription entry depending on given (subscription) values.
+ * The receive lists of a CAN device are protected by a per-device lock,
+ * so the subscription is found even when the device has been moved to
+ * another network namespace after can_rx_register(). The caller has to
+ * ensure that @dev stays valid, e.g. by holding a reference to @dev or
+ * RTNL. @net should be the namespace that has been used for
+ * can_rx_register(). It selects the 'any' CAN device lists for
+ * @dev == NULL and is otherwise only used for the receive list
+ * statistics, which become imprecise when another namespace is passed.
*/
void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
canid_t mask, void (*func)(struct sk_buff *, void *),
@@ -519,15 +531,13 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats;
struct can_dev_rcv_lists *dev_rcv_lists;
- if (dev && dev->type != ARPHRD_CAN)
+ if (dev && !can_get_ml_priv(dev))
return;
- if (dev && !net_eq(net, dev_net(dev)))
- return;
+ dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
- spin_lock_bh(&net->can.rcvlists_lock);
+ spin_lock_bh(&dev_rcv_lists->lock);
- dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists);
/* Search the receiver list for the item to delete. This should
@@ -547,26 +557,26 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
* a warning here.
*/
if (!rcv) {
+ spin_unlock_bh(&dev_rcv_lists->lock);
pr_warn("can: receive list entry not found for dev %s, id %03X, mask %03X\n",
DNAME(dev), can_id, mask);
- goto out;
+ return;
}
hlist_del_rcu(&rcv->list);
dev_rcv_lists->entries--;
+ spin_unlock_bh(&dev_rcv_lists->lock);
+
+ spin_lock_bh(&net->can.rcvlists_lock);
if (rcv_lists_stats->rcv_entries > 0)
rcv_lists_stats->rcv_entries--;
-
- out:
spin_unlock_bh(&net->can.rcvlists_lock);
/* schedule the receiver item for deletion */
- if (rcv) {
- if (rcv->sk)
- sock_hold(rcv->sk);
- call_rcu(&rcv->rcu, can_rx_delete_receiver);
- }
+ if (rcv->sk)
+ sock_hold(rcv->sk);
+ call_rcu(&rcv->rcu, can_rx_delete_receiver);
}
EXPORT_SYMBOL(can_rx_unregister);
@@ -684,8 +694,7 @@ static int can_rcv(struct sk_buff *skb, struct net_device *dev,
{
struct can_skb_ext *csx = can_skb_ext_find(skb);
- if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) ||
- !csx || !can_is_can_skb(skb))) {
+ if (unlikely(!can_get_ml_priv(dev) || !csx || !can_is_can_skb(skb))) {
pr_warn_once("PF_CAN: dropped non conform CAN skbuff: dev type %d, len %d\n",
dev->type, skb->len);
@@ -710,8 +719,7 @@ static int canfd_rcv(struct sk_buff *skb, struct net_device *dev,
{
struct can_skb_ext *csx = can_skb_ext_find(skb);
- if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) ||
- !csx || !can_is_canfd_skb(skb))) {
+ if (unlikely(!can_get_ml_priv(dev) || !csx || !can_is_canfd_skb(skb))) {
pr_warn_once("PF_CAN: dropped non conform CAN FD skbuff: dev type %d, len %d\n",
dev->type, skb->len);
@@ -736,8 +744,7 @@ static int canxl_rcv(struct sk_buff *skb, struct net_device *dev,
{
struct can_skb_ext *csx = can_skb_ext_find(skb);
- if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) ||
- !csx || !can_is_canxl_skb(skb))) {
+ if (unlikely(!can_get_ml_priv(dev) || !csx || !can_is_canxl_skb(skb))) {
pr_warn_once("PF_CAN: dropped non conform CAN XL skbuff: dev type %d, len %d\n",
dev->type, skb->len);
@@ -826,6 +833,7 @@ static int can_pernet_init(struct net *net)
net->can.rx_alldev_list = kzalloc_obj(*net->can.rx_alldev_list);
if (!net->can.rx_alldev_list)
goto out;
+ spin_lock_init(&net->can.rx_alldev_list->lock);
net->can.pkg_stats = kzalloc_obj(*net->can.pkg_stats);
if (!net->can.pkg_stats)
goto out_free_rx_alldev_list;
diff --git a/net/can/bcm.c b/net/can/bcm.c
index 3d637a1e0ac1..9288b96724da 100644
--- a/net/can/bcm.c
+++ b/net/can/bcm.c
@@ -54,6 +54,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/skb.h>
#include <linux/can/bcm.h>
@@ -963,7 +964,7 @@ static void bcm_remove_op(struct bcm_op *op)
static void bcm_rx_unreg(struct net_device *dev, struct bcm_op *op)
{
if (op->rx_reg_dev == dev) {
- can_rx_unregister(dev_net(dev), dev, op->can_id,
+ can_rx_unregister(sock_net(op->sk), dev, op->can_id,
REGMASK(op->can_id), bcm_rx_handler, op);
/* mark as removed subscription */
@@ -989,21 +990,10 @@ static int bcm_delete_rx_op(struct list_head *ops, struct bcm_msg_head *mh,
/* disable automatic timer on frame reception */
op->flags |= RX_NO_AUTOTIMER;
- /*
- * Don't care if we're bound or not (due to netdev
- * problems) can_rx_unregister() is always a save
- * thing to do here.
- */
if (op->ifindex) {
- /*
- * Only remove subscriptions that had not
- * been removed due to NETDEV_UNREGISTER
- * in bcm_notifier()
- *
- * op->rx_reg_dev is a tracked reference taken
- * when the subscription was registered, so it
- * stays valid here even if a concurrent
- * NETDEV_UNREGISTER already unlisted the dev.
+ /* Only remove subscriptions that had not been
+ * removed in bcm_notify(). op->rx_reg_dev keeps
+ * the device's receive lists accessible.
*/
if (op->rx_reg_dev)
bcm_rx_unreg(op->rx_reg_dev, op);
@@ -1719,7 +1709,7 @@ static int bcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
goto out_release;
}
- if (dev->type != ARPHRD_CAN) {
+ if (!can_get_ml_priv(dev)) {
dev_put(dev);
ret = -ENODEV;
goto out_release;
@@ -1867,7 +1857,7 @@ static int bcm_notifier(struct notifier_block *nb, unsigned long msg,
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN)
return NOTIFY_DONE;
@@ -1935,6 +1925,7 @@ static int bcm_release(struct socket *sock)
list_del(&bo->notifier);
spin_unlock(&bcm_notifier_lock);
+ /* bcm_notify() can not access this socket anymore */
lock_sock(sk);
#if IS_ENABLED(CONFIG_PROC_FS)
@@ -1948,29 +1939,20 @@ static int bcm_release(struct socket *sock)
bcm_remove_op(op);
}
- list_for_each_entry_safe(op, next, &bo->rx_ops, list) {
- /*
- * Don't care if we're bound or not (due to netdev problems)
- * can_rx_unregister() is always a save thing to do here.
- */
+ list_for_each_entry(op, &bo->rx_ops, list) {
if (op->ifindex) {
- /*
- * Only remove subscriptions that had not
- * been removed due to NETDEV_UNREGISTER
- * in bcm_notifier()
- *
- * op->rx_reg_dev is a tracked reference taken
- * when the subscription was registered, so it
- * stays valid here even if a concurrent
- * NETDEV_UNREGISTER already unlisted the device.
+ /* Only remove subscriptions that had not been removed
+ * in bcm_notify(). op->rx_reg_dev keeps the device's
+ * receive lists accessible.
*/
if (op->rx_reg_dev)
bcm_rx_unreg(op->rx_reg_dev, op);
- } else
+ } else {
can_rx_unregister(net, NULL, op->can_id,
REGMASK(op->can_id),
bcm_rx_handler, op);
-
+ }
+ cond_resched();
}
synchronize_rcu();
@@ -2024,7 +2006,7 @@ static int bcm_connect(struct socket *sock, struct sockaddr_unsized *uaddr, int
ret = -ENODEV;
goto fail;
}
- if (dev->type != ARPHRD_CAN) {
+ if (!can_get_ml_priv(dev)) {
dev_put(dev);
ret = -ENODEV;
goto fail;
diff --git a/net/can/gw.c b/net/can/gw.c
index 5793cb2420ed..54bb5bd3242a 100644
--- a/net/can/gw.c
+++ b/net/can/gw.c
@@ -52,6 +52,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/skb.h>
#include <linux/can/gw.h>
@@ -612,7 +613,7 @@ static int cgw_notifier(struct notifier_block *nb,
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
struct net *net = dev_net(dev);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg == NETDEV_UNREGISTER) {
@@ -1163,7 +1164,7 @@ static int cgw_create_job(struct sk_buff *skb, struct nlmsghdr *nlh,
if (!gwj->src.dev)
goto out;
- if (gwj->src.dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(gwj->src.dev))
goto out;
gwj->dst.dev = __dev_get_by_index(net, gwj->ccgw.dst_idx);
@@ -1171,7 +1172,7 @@ static int cgw_create_job(struct sk_buff *skb, struct nlmsghdr *nlh,
if (!gwj->dst.dev)
goto out;
- if (gwj->dst.dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(gwj->dst.dev))
goto out;
/* is sending the skb back to the incoming interface intended? */
diff --git a/net/can/isotp.c b/net/can/isotp.c
index f5dc9d04bd68..5e2de051fc5e 100644
--- a/net/can/isotp.c
+++ b/net/can/isotp.c
@@ -65,6 +65,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/skb.h>
#include <linux/can/isotp.h>
@@ -755,6 +756,14 @@ static void isotp_rcv(struct sk_buff *skb, void *data)
if (skb->len != so->ll.mtu)
return;
+ /* check for correct CAN CC/FD frame content */
+ if (so->ll.mtu == CAN_MTU) {
+ if (!can_is_can_skb(skb))
+ return;
+ } else if (!can_is_canfd_skb(skb)) {
+ return;
+ }
+
cf = (struct canfd_frame *)skb->data;
/* if enabled: check reception of my configured extended address */
@@ -1123,7 +1132,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
int off;
int err;
- if (!so->bound || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN)
+ if (!READ_ONCE(so->bound) || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN)
return -EADDRNOTAVAIL;
/* claim the socket under so->rx_lock: this serializes the claim
@@ -1168,7 +1177,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
/* so->bound is only checked once above - a wakeup may have
* unbound/rebound the socket meanwhile => recheck
*/
- if (!so->bound) {
+ if (!READ_ONCE(so->bound)) {
err = -EADDRNOTAVAIL;
goto err_out_drop;
}
@@ -1414,7 +1423,7 @@ static int isotp_recvmsg(struct socket *sock, struct msghdr *msg, size_t size,
if (flags & ~(MSG_DONTWAIT | MSG_TRUNC | MSG_PEEK | MSG_CMSG_COMPAT))
return -EINVAL;
- if (!so->bound)
+ if (!READ_ONCE(so->bound))
return -EADDRNOTAVAIL;
skb = skb_recv_datagram(sk, flags, &ret);
@@ -1488,11 +1497,11 @@ static int isotp_release(struct socket *sock)
list_del(&so->notifier);
spin_unlock(&isotp_notifier_lock);
- rtnl_lock();
lock_sock(sk);
/* remove current filters & unregister
- * tracked reference so->dev is taken at bind() time with rtnl_lock
+ * the tracked reference so->dev keeps the device's receive lists
+ * accessible, even after a namespace change
*/
if (so->bound && so->dev) {
if (isotp_register_rxid(so))
@@ -1507,11 +1516,9 @@ static int isotp_release(struct socket *sock)
}
so->ifindex = 0;
- so->bound = 0;
+ WRITE_ONCE(so->bound, 0);
so->dev = NULL;
- rtnl_unlock();
-
/* Always wait for a grace period before touching the timers below.
* A concurrent NETDEV_UNREGISTER may have already unregistered our
* filters and cleared so->bound in isotp_notify() without waiting
@@ -1541,7 +1548,6 @@ static int isotp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int l
struct sock *sk = sock->sk;
struct isotp_sock *so = isotp_sk(sk);
struct net *net = sock_net(sk);
- int ifindex;
struct net_device *dev;
canid_t tx_id = addr->can_addr.tp.tx_id;
canid_t rx_id = addr->can_addr.tp.rx_id;
@@ -1579,7 +1585,7 @@ static int isotp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int l
if (!addr->can_ifindex)
return -ENODEV;
- rtnl_lock();
+ /* lock_sock() serializes the binding with isotp_notify() */
lock_sock(sk);
if (so->bound) {
@@ -1605,61 +1611,83 @@ static int isotp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int l
goto out;
}
- dev = dev_get_by_index(net, addr->can_ifindex);
- if (!dev) {
- err = -ENODEV;
- goto out;
- }
- if (dev->type != ARPHRD_CAN) {
+ /* Publish the device inside the RCU section of the lookup: When it
+ * is unlisted concurrently, either isotp_notify() sees so->dev after
+ * synchronize_net() or the lookup fails. Check the device before
+ * publishing it.
+ */
+ rcu_read_lock();
+ dev = dev_get_by_index_rcu(net, addr->can_ifindex);
+ if (!dev || !can_get_ml_priv(dev))
err = -ENODEV;
- goto out_put_dev;
- }
- if (READ_ONCE(dev->mtu) < so->ll.mtu) {
+ else if (READ_ONCE(dev->mtu) < so->ll.mtu)
err = -EINVAL;
- goto out_put_dev;
+ if (!err) {
+ dev_hold(dev);
+ WRITE_ONCE(so->dev, dev);
}
- if (!(dev->flags & IFF_UP))
- notify_enetdown = 1;
-
- ifindex = dev->ifindex;
+ rcu_read_unlock();
+ if (err)
+ goto out;
- if (isotp_register_rxid(so))
- can_rx_register(net, dev, rx_id, SINGLE_MASK(rx_id),
- isotp_rcv, sk, "isotp", sk);
+ /* switch to new settings */
+ so->ifindex = dev->ifindex;
+ so->rxid = rx_id;
+ so->txid = tx_id;
/* no consecutive frame echo skb in flight */
WRITE_ONCE(so->cfecho, 0);
- /* register for echo skb's */
- can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id),
- isotp_rcv_echo, sk, "isotpe", sk);
+ /* register for echo skb's first (no echo traffic at this point) */
+ err = can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id),
+ isotp_rcv_echo, sk, "isotpe", sk);
+ if (err)
+ goto out_put_dev;
- /* switch to new settings */
- so->ifindex = ifindex;
- so->rxid = rx_id;
- so->txid = tx_id;
- so->bound = 1;
+ if (isotp_register_rxid(so)) {
+ err = can_rx_register(net, dev, rx_id, SINGLE_MASK(rx_id),
+ isotp_rcv, sk, "isotp", sk);
+ if (err) {
+ /* unregister echo skb subscription */
+ can_rx_unregister(net, dev, tx_id, SINGLE_MASK(tx_id),
+ isotp_rcv_echo, sk);
+ goto out_put_dev;
+ }
+ }
+
+ WRITE_ONCE(so->bound, 1);
/* bind() ok -> hold a reference for so->dev so that isotp_release()
* can safely reach the device later, even if a concurrent
* NETDEV_UNREGISTER has already unlisted it by ifindex.
*/
- so->dev = dev;
netdev_hold(so->dev, &so->dev_tracker, GFP_KERNEL);
+ /* Either isotp_notify() sees so->dev on NETDEV_DOWN or we see the
+ * cleared IFF_UP, pairs with isotp_notify()
+ */
+ smp_mb();
+ if (!(READ_ONCE(dev->flags) & IFF_UP))
+ notify_enetdown = 1;
+
out_put_dev:
- /* remove potential reference from dev_get_by_index() */
+ /* revert so->ifindex and so->dev on failed bind() */
+ if (err) {
+ so->ifindex = 0;
+ WRITE_ONCE(so->dev, NULL);
+ }
+
+ /* remove reference from the device lookup */
dev_put(dev);
out:
- release_sock(sk);
- rtnl_unlock();
-
if (notify_enetdown) {
sk->sk_err = ENETDOWN;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
}
+ release_sock(sk);
+
return err;
}
@@ -1681,8 +1709,8 @@ static int isotp_getname(struct socket *sock, struct sockaddr *uaddr, int peer)
return ISOTP_MIN_NAMELEN;
}
-static int isotp_setsockopt_locked(struct socket *sock, int level, int optname,
- sockptr_t optval, unsigned int optlen)
+static int isotp_setsockopt_locked(struct socket *sock, int optname,
+ const void *val)
{
struct sock *sk = sock->sk;
struct isotp_sock *so = isotp_sk(sk);
@@ -1693,11 +1721,7 @@ static int isotp_setsockopt_locked(struct socket *sock, int level, int optname,
switch (optname) {
case CAN_ISOTP_OPTS:
- if (optlen != sizeof(struct can_isotp_options))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->opt, optval, optlen))
- return -EFAULT;
+ memcpy(&so->opt, val, sizeof(so->opt));
/* no separate rx_ext_address is given => use ext_address */
if (!(so->opt.flags & CAN_ISOTP_RX_EXT_ADDR))
@@ -1722,55 +1746,37 @@ static int isotp_setsockopt_locked(struct socket *sock, int level, int optname,
break;
case CAN_ISOTP_RECV_FC:
- if (optlen != sizeof(struct can_isotp_fc_options))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->rxfc, optval, optlen))
- return -EFAULT;
+ memcpy(&so->rxfc, val, sizeof(so->rxfc));
break;
case CAN_ISOTP_TX_STMIN:
- if (optlen != sizeof(u32))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->force_tx_stmin, optval, optlen))
- return -EFAULT;
+ so->force_tx_stmin = *(const u32 *)val;
break;
case CAN_ISOTP_RX_STMIN:
- if (optlen != sizeof(u32))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->force_rx_stmin, optval, optlen))
- return -EFAULT;
+ so->force_rx_stmin = *(const u32 *)val;
break;
- case CAN_ISOTP_LL_OPTS:
- if (optlen == sizeof(struct can_isotp_ll_options)) {
- struct can_isotp_ll_options ll;
-
- if (copy_from_sockptr(&ll, optval, optlen))
- return -EFAULT;
+ case CAN_ISOTP_LL_OPTS: {
+ const struct can_isotp_ll_options *ll = val;
- /* check for correct ISO 11898-1 DLC data length */
- if (ll.tx_dl != padlen(ll.tx_dl))
- return -EINVAL;
+ /* check for correct ISO 11898-1 DLC data length */
+ if (ll->tx_dl != padlen(ll->tx_dl))
+ return -EINVAL;
- if (ll.mtu != CAN_MTU && ll.mtu != CANFD_MTU)
- return -EINVAL;
+ if (ll->mtu != CAN_MTU && ll->mtu != CANFD_MTU)
+ return -EINVAL;
- if (ll.mtu == CAN_MTU &&
- (ll.tx_dl > CAN_MAX_DLEN || ll.tx_flags != 0))
- return -EINVAL;
+ if (ll->mtu == CAN_MTU &&
+ (ll->tx_dl > CAN_MAX_DLEN || ll->tx_flags != 0))
+ return -EINVAL;
- memcpy(&so->ll, &ll, sizeof(ll));
+ memcpy(&so->ll, ll, sizeof(*ll));
- /* set ll_dl for tx path to similar place as for rx */
- so->tx.ll_dl = ll.tx_dl;
- } else {
- return -EINVAL;
- }
+ /* set ll_dl for tx path to similar place as for rx */
+ so->tx.ll_dl = ll->tx_dl;
break;
+ }
default:
ret = -ENOPROTOOPT;
@@ -1784,13 +1790,49 @@ static int isotp_setsockopt(struct socket *sock, int level, int optname,
{
struct sock *sk = sock->sk;
+ union {
+ struct can_isotp_options opt;
+ struct can_isotp_fc_options rxfc;
+ struct can_isotp_ll_options ll;
+ u32 stmin;
+ } val;
+ unsigned int size;
int ret;
if (level != SOL_CAN_ISOTP)
return -EINVAL;
+ /* copy the option values from user space before lock_sock() */
+ switch (optname) {
+ case CAN_ISOTP_OPTS:
+ size = sizeof(val.opt);
+ break;
+
+ case CAN_ISOTP_RECV_FC:
+ size = sizeof(val.rxfc);
+ break;
+
+ case CAN_ISOTP_TX_STMIN:
+ case CAN_ISOTP_RX_STMIN:
+ size = sizeof(val.stmin);
+ break;
+
+ case CAN_ISOTP_LL_OPTS:
+ size = sizeof(val.ll);
+ break;
+
+ default:
+ return -ENOPROTOOPT;
+ }
+
+ if (optlen != size)
+ return -EINVAL;
+
+ if (copy_from_sockptr(&val, optval, size))
+ return -EFAULT;
+
lock_sock(sk);
- ret = isotp_setsockopt_locked(sock, level, optname, optval, optlen);
+ ret = isotp_setsockopt_locked(sock, optname, &val);
release_sock(sk);
return ret;
}
@@ -1855,12 +1897,21 @@ static void isotp_notify(struct isotp_sock *so, unsigned long msg,
if (!net_eq(dev_net(dev), sock_net(sk)))
return;
- if (so->dev != dev)
+ if (msg == NETDEV_DOWN)
+ smp_mb(); /* pairs with isotp_bind(), see raw_notify() */
+
+ /* lockless fast path, see raw_notify() */
+ if (READ_ONCE(so->dev) != dev)
return;
switch (msg) {
case NETDEV_UNREGISTER:
lock_sock(sk);
+ if (so->dev != dev) {
+ release_sock(sk);
+ break;
+ }
+
/* remove current filters & unregister */
if (so->bound) {
if (isotp_register_rxid(so))
@@ -1875,16 +1926,19 @@ static void isotp_notify(struct isotp_sock *so, unsigned long msg,
}
so->ifindex = 0;
- so->bound = 0;
- so->dev = NULL;
- release_sock(sk);
+ WRITE_ONCE(so->bound, 0);
+ WRITE_ONCE(so->dev, NULL);
sk->sk_err = ENODEV;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
+ release_sock(sk);
break;
case NETDEV_DOWN:
+ if (!READ_ONCE(so->bound))
+ break;
+
sk->sk_err = ENETDOWN;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
@@ -1897,7 +1951,7 @@ static int isotp_notifier(struct notifier_block *nb, unsigned long msg,
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN)
return NOTIFY_DONE;
diff --git a/net/can/proc.c b/net/can/proc.c
index de4d05ae3459..fb144d2adc35 100644
--- a/net/can/proc.c
+++ b/net/can/proc.c
@@ -76,16 +76,39 @@ static const char rx_list_name[][8] = {
* af_can statistics stuff
*/
+static void can_reset_pkg_stats(struct can_pkg_stats *pkg_stats)
+{
+ atomic_long_set(&pkg_stats->rx_frames, 0);
+ atomic_long_set(&pkg_stats->tx_frames, 0);
+ atomic_long_set(&pkg_stats->matches, 0);
+
+ pkg_stats->total_rx_rate = 0;
+ pkg_stats->total_tx_rate = 0;
+ pkg_stats->total_rx_match_ratio = 0;
+
+ pkg_stats->current_rx_rate = 0;
+ pkg_stats->current_tx_rate = 0;
+ pkg_stats->current_rx_match_ratio = 0;
+
+ pkg_stats->max_rx_rate = 0;
+ pkg_stats->max_tx_rate = 0;
+ pkg_stats->max_rx_match_ratio = 0;
+
+ atomic_long_set(&pkg_stats->rx_frames_delta, 0);
+ atomic_long_set(&pkg_stats->tx_frames_delta, 0);
+ atomic_long_set(&pkg_stats->matches_delta, 0);
+}
+
static void can_init_stats(struct net *net)
{
struct can_pkg_stats *pkg_stats = net->can.pkg_stats;
struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats;
/*
- * This memset function is called from a timer context (when
+ * This stats reset is called from a timer context (when
* can_stattimer is active which is the default) OR in a process
* context (reading the proc_fs when can_stattimer is disabled).
*/
- memset(pkg_stats, 0, sizeof(struct can_pkg_stats));
+ can_reset_pkg_stats(pkg_stats);
pkg_stats->jiffies_init = jiffies;
rcv_lists_stats->stats_reset++;
@@ -121,29 +144,23 @@ void can_stat_update(struct timer_list *t)
long rx_frames = atomic_long_read(&pkg_stats->rx_frames);
long tx_frames = atomic_long_read(&pkg_stats->tx_frames);
long matches = atomic_long_read(&pkg_stats->matches);
- long rx_frames_delta = atomic_long_read(&pkg_stats->rx_frames_delta);
- long tx_frames_delta = atomic_long_read(&pkg_stats->tx_frames_delta);
- long matches_delta = atomic_long_read(&pkg_stats->matches_delta);
+ /* fetch and clear values for 'current rate' calculation */
+ long rx_frames_delta = atomic_long_xchg(&pkg_stats->rx_frames_delta, 0);
+ long tx_frames_delta = atomic_long_xchg(&pkg_stats->tx_frames_delta, 0);
+ long matches_delta = atomic_long_xchg(&pkg_stats->matches_delta, 0);
- /* restart counting in timer context on user request */
- if (user_reset)
- can_init_stats(net);
-
- /* restart counting on jiffies overflow */
- if (j < pkg_stats->jiffies_init)
- can_init_stats(net);
-
- /* prevent overflow in calc_rate() */
- if (rx_frames > (LONG_MAX / HZ))
- can_init_stats(net);
-
- /* prevent overflow in calc_rate() */
- if (tx_frames > (LONG_MAX / HZ))
- can_init_stats(net);
-
- /* matches overflow - very improbable */
- if (matches > (LONG_MAX / 100))
+ /*
+ * Restart counting in timer context on user request, on jiffies
+ * overflow or to prevent overflows in calc_rate() and in the
+ * match ratio calculation. The local snapshots are stale after
+ * the reset, so skip the calculation for this interval.
+ */
+ if (user_reset || j < pkg_stats->jiffies_init ||
+ rx_frames > (LONG_MAX / HZ) || tx_frames > (LONG_MAX / HZ) ||
+ matches > (LONG_MAX / 100)) {
can_init_stats(net);
+ goto out;
+ }
/* calc total values */
if (rx_frames)
@@ -172,11 +189,7 @@ void can_stat_update(struct timer_list *t)
if (pkg_stats->max_rx_match_ratio < pkg_stats->current_rx_match_ratio)
pkg_stats->max_rx_match_ratio = pkg_stats->current_rx_match_ratio;
- /* clear values for 'current rate' calculation */
- atomic_long_set(&pkg_stats->tx_frames_delta, 0);
- atomic_long_set(&pkg_stats->rx_frames_delta, 0);
- atomic_long_set(&pkg_stats->matches_delta, 0);
-
+out:
/* restart timer (one second) */
mod_timer(&net->can.stattimer, round_jiffies(jiffies + HZ));
}
diff --git a/net/can/raw.c b/net/can/raw.c
index 0acd4f6c6dd6..0926d5686e41 100644
--- a/net/can/raw.c
+++ b/net/can/raw.c
@@ -85,6 +85,7 @@ struct raw_sock {
struct sock sk;
struct net_device *dev;
netdevice_tracker dev_tracker;
+ struct net_device *bind_dev; /* hint for raw_notify(), no reference */
struct list_head notifier;
int ifindex;
unsigned int bound:1;
@@ -224,6 +225,7 @@ static int raw_enable_filters(struct net *net, struct net_device *dev,
int i;
for (i = 0; i < count; i++) {
+ cond_resched();
err = can_rx_register(net, dev, filter[i].can_id,
filter[i].can_mask,
raw_rcv, sk, "raw", sk);
@@ -258,9 +260,11 @@ static void raw_disable_filters(struct net *net, struct net_device *dev,
{
int i;
- for (i = 0; i < count; i++)
+ for (i = 0; i < count; i++) {
can_rx_unregister(net, dev, filter[i].can_id,
filter[i].can_mask, raw_rcv, sk);
+ cond_resched();
+ }
}
static inline void raw_disable_errfilter(struct net *net,
@@ -305,16 +309,31 @@ static void raw_notify(struct raw_sock *ro, unsigned long msg,
struct net_device *dev)
{
struct sock *sk = &ro->sk;
+ struct net_device *bind_dev;
if (!net_eq(dev_net(dev), sock_net(sk)))
return;
- if (ro->dev != dev)
+ if (msg == NETDEV_DOWN)
+ smp_mb(); /* IFF_UP is cleared before, pairs with raw_bind() */
+
+ /* Lockless fast path, ro->bind_dev covers a concurrent bind().
+ * Read it before ro->dev, pairs with smp_store_release() in
+ * raw_bind(). A stale ro->bind_dev only costs a lock_sock().
+ */
+ bind_dev = smp_load_acquire(&ro->bind_dev);
+ if (READ_ONCE(ro->dev) != dev &&
+ (msg != NETDEV_UNREGISTER || bind_dev != dev))
return;
switch (msg) {
case NETDEV_UNREGISTER:
lock_sock(sk);
+ if (ro->dev != dev) {
+ release_sock(sk);
+ break;
+ }
+
/* remove current filters & unregister */
if (ro->bound) {
raw_disable_allfilters(dev_net(dev), dev, sk);
@@ -326,13 +345,13 @@ static void raw_notify(struct raw_sock *ro, unsigned long msg,
ro->ifindex = 0;
ro->bound = 0;
- ro->dev = NULL;
+ WRITE_ONCE(ro->dev, NULL);
ro->count = 0;
- release_sock(sk);
sk->sk_err = ENODEV;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
+ release_sock(sk);
break;
case NETDEV_DOWN:
@@ -348,7 +367,7 @@ static int raw_notifier(struct notifier_block *nb, unsigned long msg,
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN)
return NOTIFY_DONE;
@@ -381,6 +400,7 @@ static int raw_init(struct sock *sk)
ro->bound = 0;
ro->ifindex = 0;
ro->dev = NULL;
+ ro->bind_dev = NULL;
/* set default filter to single entry dfilter */
ro->dfilter.can_id = 0;
@@ -431,13 +451,15 @@ static int raw_release(struct socket *sock)
list_del(&ro->notifier);
spin_unlock(&raw_notifier_lock);
- rtnl_lock();
lock_sock(sk);
- /* remove current filters & unregister */
+ /* remove current filters & unregister
+ * the tracked reference ro->dev keeps the device's receive lists
+ * accessible, even after a namespace change
+ */
if (ro->bound) {
if (ro->dev) {
- raw_disable_allfilters(dev_net(ro->dev), ro->dev, sk);
+ raw_disable_allfilters(net, ro->dev, sk);
netdev_put(ro->dev, &ro->dev_tracker);
} else {
raw_disable_allfilters(net, NULL, sk);
@@ -456,7 +478,6 @@ static int raw_release(struct socket *sock)
sock->sk = NULL;
release_sock(sk);
- rtnl_unlock();
sock_prot_inuse_add(net, sk->sk_prot, -1);
sock_put(sk);
@@ -479,26 +500,34 @@ static int raw_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int len
if (addr->can_family != AF_CAN)
return -EINVAL;
- rtnl_lock();
+ /* lock_sock() serializes the binding with raw_notify() */
lock_sock(sk);
if (ro->bound && addr->can_ifindex == ro->ifindex)
goto out;
if (addr->can_ifindex) {
- dev = dev_get_by_index(sock_net(sk), addr->can_ifindex);
+ /* Publish the device inside the RCU section of the lookup:
+ * When it is unlisted concurrently, either raw_notify() sees
+ * ro->bind_dev after synchronize_net() or the lookup fails.
+ */
+ rcu_read_lock();
+ dev = dev_get_by_index_rcu(sock_net(sk), addr->can_ifindex);
+ if (dev) {
+ dev_hold(dev);
+ /* pairs with smp_load_acquire() in raw_notify() */
+ smp_store_release(&ro->bind_dev, dev);
+ }
+ rcu_read_unlock();
if (!dev) {
err = -ENODEV;
goto out;
}
- if (dev->type != ARPHRD_CAN) {
+ if (!can_get_ml_priv(dev)) {
err = -ENODEV;
goto out_put_dev;
}
- if (!(dev->flags & IFF_UP))
- notify_enetdown = 1;
-
ifindex = dev->ifindex;
/* filters set by default/setsockopt */
@@ -517,7 +546,7 @@ static int raw_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int len
if (ro->bound) {
/* unregister old filters */
if (ro->dev) {
- raw_disable_allfilters(dev_net(ro->dev),
+ raw_disable_allfilters(sock_net(sk),
ro->dev, sk);
/* drop reference to old ro->dev */
netdev_put(ro->dev, &ro->dev_tracker);
@@ -528,24 +557,32 @@ static int raw_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int len
ro->ifindex = ifindex;
ro->bound = 1;
/* bind() ok -> hold a reference for new ro->dev */
- ro->dev = dev;
+ WRITE_ONCE(ro->dev, dev);
if (ro->dev)
netdev_hold(ro->dev, &ro->dev_tracker, GFP_KERNEL);
+
+ if (dev) {
+ /* Either raw_notify() sees ro->dev on NETDEV_DOWN or
+ * we see the cleared IFF_UP, pairs with raw_notify()
+ */
+ smp_mb();
+ if (!(READ_ONCE(dev->flags) & IFF_UP))
+ notify_enetdown = 1;
+ }
}
out_put_dev:
- /* remove potential reference from dev_get_by_index() */
+ /* remove reference from the device lookup */
dev_put(dev);
out:
- release_sock(sk);
- rtnl_unlock();
-
if (notify_enetdown) {
sk->sk_err = ENETDOWN;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
}
+ release_sock(sk);
+
return err;
}
@@ -567,58 +604,36 @@ static int raw_getname(struct socket *sock, struct sockaddr *uaddr,
}
static int raw_setsockopt_locked(struct socket *sock, int optname,
- sockptr_t optval, unsigned int optlen)
+ const void *val, struct can_filter *filter,
+ int count)
{
struct sock *sk = sock->sk;
struct raw_sock *ro = raw_sk(sk);
- struct can_filter *filter = NULL; /* dyn. alloc'ed filters */
- struct can_filter sfilter; /* single filter */
struct net_device *dev = NULL;
- can_err_mask_t err_mask = 0;
- int count = 0;
+ can_err_mask_t err_mask;
int flag;
int err = 0;
switch (optname) {
case CAN_RAW_FILTER:
- if (optlen % sizeof(struct can_filter) != 0)
- return -EINVAL;
-
- if (optlen > CAN_RAW_FILTER_MAX * sizeof(struct can_filter))
- return -EINVAL;
-
- count = optlen / sizeof(struct can_filter);
-
- if (count > 1) {
- /* filter does not fit into dfilter => alloc space */
- filter = memdup_sockptr(optval, optlen);
- if (IS_ERR(filter))
- return PTR_ERR(filter);
- } else if (count == 1) {
- if (copy_from_sockptr(&sfilter, optval, sizeof(sfilter)))
- return -EFAULT;
- }
-
+ /* @filter has been copied by raw_setsockopt(). It is linked
+ * to the socket on success and freed by the caller on error.
+ *
+ * The reg_state check is only a shortcut: On a concurrent
+ * unregistration or namespace change raw_notify() waits for
+ * lock_sock() and removes the filters set here.
+ */
dev = ro->dev;
- if (ro->bound && dev && dev->reg_state != NETREG_REGISTERED) {
- if (count > 1)
- kfree(filter);
+ if (ro->bound && dev &&
+ READ_ONCE(dev->reg_state) != NETREG_REGISTERED)
return -ENODEV;
- }
if (ro->bound) {
/* (try to) register the new filters */
- if (count == 1)
- err = raw_enable_filters(sock_net(sk), dev, sk,
- &sfilter, 1);
- else
- err = raw_enable_filters(sock_net(sk), dev, sk,
- filter, count);
- if (err) {
- if (count > 1)
- kfree(filter);
+ err = raw_enable_filters(sock_net(sk), dev, sk, filter,
+ count);
+ if (err)
return err;
- }
/* remove old filter registrations */
raw_disable_filters(sock_net(sk), dev, sk, ro->filter,
@@ -632,7 +647,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
/* link new filters to the socket */
if (count == 1) {
/* copy filter data for single filter */
- ro->dfilter = sfilter;
+ ro->dfilter = *filter;
filter = &ro->dfilter;
}
ro->filter = filter;
@@ -640,16 +655,12 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_ERR_FILTER:
- if (optlen != sizeof(err_mask))
- return -EINVAL;
-
- if (copy_from_sockptr(&err_mask, optval, optlen))
- return -EFAULT;
-
- err_mask &= CAN_ERR_MASK;
+ err_mask = *(const can_err_mask_t *)val & CAN_ERR_MASK;
+ /* lockless shortcut, see CAN_RAW_FILTER */
dev = ro->dev;
- if (ro->bound && dev && dev->reg_state != NETREG_REGISTERED)
+ if (ro->bound && dev &&
+ READ_ONCE(dev->reg_state) != NETREG_REGISTERED)
return -ENODEV;
/* remove current error mask */
@@ -671,31 +682,15 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_LOOPBACK:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
-
- ro->loopback = !!flag;
+ ro->loopback = !!*(const int *)val;
break;
case CAN_RAW_RECV_OWN_MSGS:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
-
- ro->recv_own_msgs = !!flag;
+ ro->recv_own_msgs = !!*(const int *)val;
break;
case CAN_RAW_FD_FRAMES:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
+ flag = *(const int *)val;
/* Enabling CAN XL includes CAN FD */
if (ro->xl_frames && !flag)
@@ -705,13 +700,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_XL_FRAMES:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
-
- ro->xl_frames = !!flag;
+ ro->xl_frames = !!*(const int *)val;
/* Enabling CAN XL includes CAN FD */
if (ro->xl_frames)
@@ -719,11 +708,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_XL_VCID_OPTS:
- if (optlen != sizeof(ro->raw_vcid_opts))
- return -EINVAL;
-
- if (copy_from_sockptr(&ro->raw_vcid_opts, optval, optlen))
- return -EFAULT;
+ memcpy(&ro->raw_vcid_opts, val, sizeof(ro->raw_vcid_opts));
/* prepare 32 bit values for handling in hot path */
ro->tx_vcid_shifted = ro->raw_vcid_opts.tx_vcid << CANXL_VCID_OFFSET;
@@ -732,13 +717,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_JOIN_FILTERS:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
-
- ro->join_filters = !!flag;
+ ro->join_filters = !!*(const int *)val;
break;
default:
@@ -751,18 +730,83 @@ static int raw_setsockopt(struct socket *sock, int level, int optname,
sockptr_t optval, unsigned int optlen)
{
struct sock *sk = sock->sk;
+ struct can_filter *filter = NULL; /* dyn. alloc'ed filters */
+ struct can_filter sfilter; /* single filter */
+ union {
+ can_err_mask_t err_mask;
+ int flag;
+ struct can_raw_vcid_options vcid_opts;
+ } val;
+ unsigned int size = 0;
+ int count = 0;
int err;
if (level != SOL_CAN_RAW)
return -EINVAL;
- rtnl_lock();
+ /* Copy the option values from user space before lock_sock(), so a
+ * stalled copy can not block raw_notify() waiting for lock_sock()
+ * under RTNL.
+ */
+ switch (optname) {
+ case CAN_RAW_FILTER:
+ if (optlen % sizeof(struct can_filter) != 0)
+ return -EINVAL;
+
+ if (optlen > CAN_RAW_FILTER_MAX * sizeof(struct can_filter))
+ return -EINVAL;
+
+ count = optlen / sizeof(struct can_filter);
+
+ if (count > 1) {
+ /* filter does not fit into dfilter => alloc space */
+ filter = memdup_sockptr(optval, optlen);
+ if (IS_ERR(filter))
+ return PTR_ERR(filter);
+ } else if (count == 1) {
+ if (copy_from_sockptr(&sfilter, optval, sizeof(sfilter)))
+ return -EFAULT;
+ filter = &sfilter;
+ }
+ break;
+
+ case CAN_RAW_ERR_FILTER:
+ size = sizeof(val.err_mask);
+ break;
+
+ case CAN_RAW_XL_VCID_OPTS:
+ size = sizeof(val.vcid_opts);
+ break;
+
+ case CAN_RAW_LOOPBACK:
+ case CAN_RAW_RECV_OWN_MSGS:
+ case CAN_RAW_FD_FRAMES:
+ case CAN_RAW_XL_FRAMES:
+ case CAN_RAW_JOIN_FILTERS:
+ size = sizeof(val.flag);
+ break;
+
+ default:
+ return -ENOPROTOOPT;
+ }
+
+ if (size) {
+ if (optlen != size)
+ return -EINVAL;
+
+ if (copy_from_sockptr(&val, optval, size))
+ return -EFAULT;
+ }
+
lock_sock(sk);
- err = raw_setsockopt_locked(sock, optname, optval, optlen);
+ err = raw_setsockopt_locked(sock, optname, &val, filter, count);
release_sock(sk);
- rtnl_unlock();
+
+ /* on error free the filters allocated for CAN_RAW_FILTER above */
+ if (err && count > 1)
+ kfree(filter);
return err;
}
@@ -784,6 +828,7 @@ static int raw_getsockopt(struct socket *sock, int level, int optname,
switch (optname) {
case CAN_RAW_FILTER: {
+ struct can_filter *filter = NULL;
int err = 0;
lock_sock(sk);
@@ -798,15 +843,22 @@ static int raw_getsockopt(struct socket *sock, int level, int optname,
} else {
if (len > fsize)
len = fsize;
- if (copy_to_iter(ro->filter, len,
- &opt->iter_out) != len)
- err = -EFAULT;
+ /* copy to user space after release_sock() */
+ filter = kmemdup(ro->filter, len, GFP_KERNEL);
+ if (!filter)
+ err = -ENOMEM;
}
} else {
len = 0;
}
release_sock(sk);
+ if (filter) {
+ if (copy_to_iter(filter, len, &opt->iter_out) != len)
+ err = -EFAULT;
+ kfree(filter);
+ }
+
if (!err)
opt->optlen = len;
return err;
diff --git a/net/core/dev.c b/net/core/dev.c
index cc7b32ab5608..f803d8ba866f 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4204,7 +4204,9 @@ static enum skb_drop_reason qdisc_pkt_len_segs_init(struct sk_buff *skb)
if (likely(shinfo->gso_type & (SKB_GSO_TCPV4 | SKB_GSO_TCPV6))) {
const struct tcphdr *th;
- if (!pskb_may_pull(skb, hdr_len + sizeof(struct tcphdr)))
+ /* A header in front of skb->data gives a negative offset. */
+ if ((int)hdr_len < 0 ||
+ !pskb_may_pull(skb, hdr_len + sizeof(struct tcphdr)))
return SKB_DROP_REASON_SKB_BAD_GSO;
th = (const struct tcphdr *)(skb->data + hdr_len);
diff --git a/net/core/filter.c b/net/core/filter.c
index 70dc621672f2..d3e5a2843d5e 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -2271,9 +2271,13 @@ static int bpf_out_neigh_v6(struct net *net, struct sk_buff *skb,
static int __bpf_redirect_neigh_v6(struct sk_buff *skb, struct net_device *dev,
struct bpf_nh_params *nh)
{
- const struct ipv6hdr *ip6h = ipv6_hdr(skb);
struct net *net = dev_net(dev);
int err, ret = NET_XMIT_DROP;
+ const struct ipv6hdr *ip6h;
+
+ if (unlikely(!pskb_may_pull(skb, sizeof(*ip6h))))
+ goto out_drop;
+ ip6h = ipv6_hdr(skb);
if (!nh) {
struct dst_entry *dst;
@@ -2380,9 +2384,13 @@ static int bpf_out_neigh_v4(struct net *net, struct sk_buff *skb,
static int __bpf_redirect_neigh_v4(struct sk_buff *skb, struct net_device *dev,
struct bpf_nh_params *nh)
{
- const struct iphdr *ip4h = ip_hdr(skb);
struct net *net = dev_net(dev);
int err, ret = NET_XMIT_DROP;
+ const struct iphdr *ip4h;
+
+ if (unlikely(!pskb_may_pull(skb, sizeof(*ip4h))))
+ goto out_drop;
+ ip4h = ip_hdr(skb);
if (!nh) {
struct flowi4 fl4 = {
@@ -2432,15 +2440,14 @@ static int __bpf_redirect_neigh_v4(struct sk_buff *skb, struct net_device *dev,
static int __bpf_redirect_neigh(struct sk_buff *skb, struct net_device *dev,
struct bpf_nh_params *nh)
{
- struct ethhdr *ethh = eth_hdr(skb);
-
- if (unlikely(skb->mac_header >= skb->network_header))
+ if (unlikely(skb_network_offset(skb) < ETH_HLEN ||
+ !pskb_may_pull(skb, ETH_HLEN)))
goto out;
bpf_push_mac_rcsum(skb);
- if (is_multicast_ether_addr(ethh->h_dest))
+ if (is_multicast_ether_addr(eth_hdr(skb)->h_dest))
goto out;
- skb_pull(skb, sizeof(*ethh));
+ skb_pull(skb, ETH_HLEN);
skb_unset_mac_header(skb);
skb_reset_network_header(skb);
@@ -7989,7 +7996,7 @@ BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags)
if (sk_unhashed(sk))
return -EOPNOTSUPP;
if (sk_is_refcounted(sk) &&
- unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
return -ENOENT;
skb_orphan(skb);
diff --git a/net/core/sock.c b/net/core/sock.c
index e8551df8330f..1108543f9fec 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -3262,7 +3262,7 @@ void __release_sock(struct sock *sk)
spin_unlock_bh(&sk->sk_lock.slock);
- while (1) {
+ do {
next = skb->next;
prefetch(next);
DEBUG_NET_WARN_ON_ONCE(skb_dst_is_noref(skb));
@@ -3270,12 +3270,10 @@ void __release_sock(struct sock *sk)
sk_backlog_rcv(sk, skb);
skb = next;
- if (!skb)
- break;
if (!(++nb & 15))
cond_resched();
- }
+ } while (skb);
spin_lock_bh(&sk->sk_lock.slock);
}
diff --git a/net/ipv4/cipso_ipv4.c b/net/ipv4/cipso_ipv4.c
index 1aacbbeffbc6..e65a3f29d2ca 100644
--- a/net/ipv4/cipso_ipv4.c
+++ b/net/ipv4/cipso_ipv4.c
@@ -2013,6 +2013,10 @@ static int cipso_v4_delopt(struct ip_options_rcu __rcu **opt_ptr)
cipso_ptr = &opt->opt.__data[cipso_off];
cipso_len = cipso_ptr[1];
+ optlen_new = cipso_v4_get_actual_opt_len(opt->opt.__data,
+ opt->opt.optlen);
+ optlen_new -= cipso_len;
+
if (opt->opt.srr > opt->opt.cipso)
opt->opt.srr -= cipso_len;
if (opt->opt.rr > opt->opt.cipso)
@@ -2026,11 +2030,12 @@ static int cipso_v4_delopt(struct ip_options_rcu __rcu **opt_ptr)
memmove(cipso_ptr, cipso_ptr + cipso_len,
opt->opt.optlen - cipso_off - cipso_len);
- optlen_new = cipso_v4_get_actual_opt_len(opt->opt.__data,
- opt->opt.optlen);
hdr_delta = opt->opt.optlen;
opt->opt.optlen = (optlen_new + 3) & ~3;
hdr_delta -= opt->opt.optlen;
+
+ memset(&opt->opt.__data[optlen_new], IPOPT_END,
+ opt->opt.optlen - optlen_new);
} else {
/* only the cipso option was present on the socket so we can
* remove the entire option struct */
diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c
index 8e891c8141f8..a1fe2531cb59 100644
--- a/net/ipv4/inet_connection_sock.c
+++ b/net/ipv4/inet_connection_sock.c
@@ -1060,8 +1060,8 @@ static void reqsk_timer_handler(struct timer_list *t)
* hold another count to prevent use-after-free and
* call reqsk_put() just before return.
*/
- refcount_set(&nreq->rsk_refcnt, 2 + 1);
timer_setup(&nreq->rsk_timer, reqsk_timer_handler, TIMER_PINNED);
+ refcount_set_release(&nreq->rsk_refcnt, 2 + 1);
reqsk_queue_migrated(&inet_csk(nsk)->icsk_accept_queue, req);
req = nreq;
@@ -1445,7 +1445,7 @@ struct sock *inet_csk_complete_hashdance(struct sock *sk, struct sock *child,
goto child_put;
}
- refcount_set(&nreq->rsk_refcnt, 1);
+ refcount_set_release(&nreq->rsk_refcnt, 1);
if (inet_csk_reqsk_queue_add(sk, nreq, child)) {
__NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMIGRATEREQSUCCESS);
reqsk_migrate_reset(req);
@@ -1498,7 +1498,7 @@ void inet_csk_listen_stop(struct sock *sk)
if (nsk) {
nreq = inet_reqsk_clone(req, nsk);
if (nreq) {
- refcount_set(&nreq->rsk_refcnt, 1);
+ refcount_set_release(&nreq->rsk_refcnt, 1);
rcu_read_lock();
if (inet_csk_reqsk_queue_add(nsk, nreq, child)) {
diff --git a/net/ipv4/inet_fragment.c b/net/ipv4/inet_fragment.c
index c18da91c1f03..0b8d22ffafca 100644
--- a/net/ipv4/inet_fragment.c
+++ b/net/ipv4/inet_fragment.c
@@ -336,6 +336,7 @@ void inet_frag_queue_flush(struct inet_frag_queue *q,
q->rb_fragments = RB_ROOT;
q->fragments_tail = NULL;
q->last_run_head = NULL;
+ q->truesize = 0;
}
EXPORT_SYMBOL(inet_frag_queue_flush);
@@ -446,6 +447,9 @@ int inet_frag_queue_insert(struct inet_frag_queue *q, struct sk_buff *skb,
if (last && !secpath_equal(last, skb))
return IPFRAG_MIXED_SECPATH;
+ if (skb->truesize > INET_FRAG_MAX_TRUESIZE - q->truesize)
+ return IPFRAG_TOOBIG;
+
/* An IP fragment is never a GSO packet, but an untrusted source
* (virtio_net_hdr) may have attached GSO metadata to it. Do not let
* that reach the reassembled skb, whose head keeps the first
@@ -510,6 +514,7 @@ int inet_frag_queue_insert(struct inet_frag_queue *q, struct sk_buff *skb,
if (offset)
nf_reset_ct(skb);
+ q->truesize += skb->truesize;
return IPFRAG_OK;
}
EXPORT_SYMBOL(inet_frag_queue_insert);
diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c
index 58d54965036a..7ffa9171d6de 100644
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -550,7 +550,7 @@ struct sock *__inet_lookup_established(const struct net *net,
if (sk->sk_hash != hash)
continue;
if (likely(inet_match(net, sk, acookie, ports, dif, sdif))) {
- if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
goto out;
if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) ||
!inet_match(net, sk, acookie, ports, dif,
diff --git a/net/ipv4/inet_timewait_sock.c b/net/ipv4/inet_timewait_sock.c
index d4c781a0667f..74ff40d42e56 100644
--- a/net/ipv4/inet_timewait_sock.c
+++ b/net/ipv4/inet_timewait_sock.c
@@ -141,7 +141,7 @@ void inet_twsk_hashdance_schedule(struct inet_timewait_sock *tw,
* Also note that after this point, we lost our implicit reference
* so we are not allowed to use tw anymore.
*/
- refcount_set(&tw->tw_refcnt, 3);
+ refcount_set_release(&tw->tw_refcnt, 3);
/* Ensure tw_refcnt has been set before tw is published.
* smp_wmb() provides the necessary memory barrier to enforce this
@@ -328,7 +328,7 @@ void inet_twsk_purge(struct inet_hashinfo *hashinfo)
if (check_net(sock_net(sk)))
continue;
- if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
continue;
if (check_net(sock_net(sk))) {
diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c
index 644030561537..4b2d2713653b 100644
--- a/net/ipv4/syncookies.c
+++ b/net/ipv4/syncookies.c
@@ -205,7 +205,7 @@ struct sock *tcp_get_cookie_sock(struct sock *sk, struct sk_buff *skb,
child = icsk->icsk_af_ops->syn_recv_sock(sk, skb, req, dst,
NULL, &own_req, NULL);
if (child) {
- refcount_set(&req->rsk_refcnt, 1);
+ refcount_set_release(&req->rsk_refcnt, 1);
sock_rps_save_rxhash(child, skb);
if (rsk_drop_req(req)) {
diff --git a/net/ipv4/tcp_fastopen.c b/net/ipv4/tcp_fastopen.c
index 471c78be5513..e40477b6b554 100644
--- a/net/ipv4/tcp_fastopen.c
+++ b/net/ipv4/tcp_fastopen.c
@@ -364,7 +364,7 @@ static struct sock *tcp_fastopen_create_child(struct sock *sk,
tcp_reset_xmit_timer(child, ICSK_TIME_RETRANS,
req->timeout, false);
- refcount_set(&req->rsk_refcnt, 2);
+ refcount_set_release(&req->rsk_refcnt, 2);
sk_mark_napi_id_set(child, skb);
diff --git a/net/ipv4/xfrm4_output.c b/net/ipv4/xfrm4_output.c
index 0ae67d537499..fffa3c4f456e 100644
--- a/net/ipv4/xfrm4_output.c
+++ b/net/ipv4/xfrm4_output.c
@@ -38,9 +38,12 @@ int xfrm4_output(struct net *net, struct sock *sk, struct sk_buff *skb)
void xfrm4_local_error(struct sk_buff *skb, u32 mtu)
{
+ struct sock *sk = skb_to_full_sk(skb);
struct iphdr *hdr;
+ if (!sk)
+ return;
+
hdr = skb->encapsulation ? inner_ip_hdr(skb) : ip_hdr(skb);
- ip_local_error(skb->sk, EMSGSIZE, hdr->daddr,
- inet_sk(skb->sk)->inet_dport, mtu);
+ ip_local_error(sk, EMSGSIZE, hdr->daddr, inet_sk(sk)->inet_dport, mtu);
}
diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c
index a2d3fb187f76..3350330c3f1c 100644
--- a/net/ipv6/addrconf.c
+++ b/net/ipv6/addrconf.c
@@ -1484,7 +1484,8 @@ static int ipv6_create_tempaddr(struct inet6_ifaddr *ifp, bool block)
age = (now - tmp_tstamp + ADDRCONF_TIMER_FUZZ_MINUS) / HZ;
if (cfg.preferred_lft <= regen_advance + age) {
cfg.preferred_lft = regen_advance + age + 1;
- if (cfg.preferred_lft > cfg.valid_lft ||
+ if (cfg.preferred_lft <= regen_advance + age ||
+ cfg.preferred_lft > cfg.valid_lft ||
cfg.preferred_lft > if_public_preferred_lft) {
in6_ifa_put(ifp);
in6_dev_put(idev);
diff --git a/net/ipv6/datagram.c b/net/ipv6/datagram.c
index 38d7b4845281..9672ac979f47 100644
--- a/net/ipv6/datagram.c
+++ b/net/ipv6/datagram.c
@@ -149,6 +149,7 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr_unsized *uaddr,
__be32 fl6_flowlabel = 0;
__be32 old_fl6_flowlabel;
__be16 old_dport;
+ int old_bound_dev_if;
int addr_type;
int err;
@@ -219,6 +220,8 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr_unsized *uaddr,
goto out;
}
+ old_bound_dev_if = sk->sk_bound_dev_if;
+
if (__ipv6_addr_needs_scope_id(addr_type)) {
if (addr_len >= sizeof(struct sockaddr_in6) &&
usin->sin6_scope_id) {
@@ -261,6 +264,7 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr_unsized *uaddr,
sk->sk_v6_daddr = old_daddr;
np->flow_label = old_fl6_flowlabel;
inet->inet_dport = old_dport;
+ WRITE_ONCE(sk->sk_bound_dev_if, old_bound_dev_if);
goto out;
}
diff --git a/net/ipv6/inet6_hashtables.c b/net/ipv6/inet6_hashtables.c
index aa5fa1dd062f..1eea584d80df 100644
--- a/net/ipv6/inet6_hashtables.c
+++ b/net/ipv6/inet6_hashtables.c
@@ -105,7 +105,7 @@ struct sock *__inet6_lookup_established(const struct net *net,
continue;
if (!inet6_match(net, sk, saddr, daddr, ports, dif, sdif))
continue;
- if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
goto out;
if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) ||
diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c
index b965258cf9e5..bdea825434ea 100644
--- a/net/ipv6/raw.c
+++ b/net/ipv6/raw.c
@@ -654,6 +654,9 @@ static int rawv6_send_hdrinc(struct sock *sk, struct msghdr *msg, int length,
if (unlikely(!skb))
return 0;
+ /* l3mdev_ip6_out() may have replaced the skb dst. */
+ rt = dst_rt6_info(skb_dst(skb));
+
/* Acquire rcu_read_lock() in case we need to use rt->rt6i_idev
* in the error path. Since skb has been freed, the dst could
* have been queued for deletion.
diff --git a/net/l2tp/l2tp_core.c b/net/l2tp/l2tp_core.c
index 7e042a3e648c..0cd3e38bf662 100644
--- a/net/l2tp/l2tp_core.c
+++ b/net/l2tp/l2tp_core.c
@@ -315,10 +315,19 @@ EXPORT_SYMBOL_GPL(l2tp_v2_session_get);
struct l2tp_session *l2tp_session_get(const struct net *net, struct sock *sk, int pver,
u32 tunnel_id, u32 session_id)
{
+ struct l2tp_session *session;
+
if (pver == L2TP_HDR_VER_2)
return l2tp_v2_session_get(net, tunnel_id, session_id);
- else
- return l2tp_v3_session_get(net, sk, session_id);
+
+ /* l2tp_v3_session_get() doesn't check sk if the ID hasn't collided */
+ session = l2tp_v3_session_get(net, sk, session_id);
+ if (session && session->tunnel->sock != sk) {
+ l2tp_session_put(session);
+ return NULL;
+ }
+
+ return session;
}
EXPORT_SYMBOL_GPL(l2tp_session_get);
diff --git a/net/mac80211/tdls.c b/net/mac80211/tdls.c
index f663d28d9209..71db07aa301b 100644
--- a/net/mac80211/tdls.c
+++ b/net/mac80211/tdls.c
@@ -2021,7 +2021,7 @@ void
ieee80211_process_tdls_channel_switch(struct ieee80211_sub_if_data *sdata,
struct sk_buff *skb)
{
- struct ieee80211_tdls_data *tf = (void *)skb->data;
+ struct ieee80211_tdls_data *tf;
struct wiphy *wiphy = sdata->local->hw.wiphy;
lockdep_assert_wiphy(wiphy);
@@ -2037,6 +2037,7 @@ ieee80211_process_tdls_channel_switch(struct ieee80211_sub_if_data *sdata,
* The packet/size was already validated by mac80211 Rx path, only look
* at the action type.
*/
+ tf = (void *)skb->data;
switch (tf->action_code) {
case WLAN_TDLS_CHANNEL_SWITCH_REQUEST:
ieee80211_process_tdls_channel_switch_req(sdata, skb);
diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
index 5714634c8103..27bae2a4fa86 100644
--- a/net/mac80211/tx.c
+++ b/net/mac80211/tx.c
@@ -1130,6 +1130,7 @@ static bool ieee80211_tx_prep_agg(struct ieee80211_tx_data *tx,
struct sk_buff *purge_skb = NULL;
if (test_bit(HT_AGG_STATE_OPERATIONAL, &tid_tx->state)) {
+ info->flags |= IEEE80211_TX_CTL_AMPDU;
reset_agg_timer = true;
} else if (test_bit(HT_AGG_STATE_WANT_START, &tid_tx->state)) {
/*
@@ -1161,6 +1162,7 @@ static bool ieee80211_tx_prep_agg(struct ieee80211_tx_data *tx,
if (!tid_tx) {
/* do nothing, let packet pass through */
} else if (test_bit(HT_AGG_STATE_OPERATIONAL, &tid_tx->state)) {
+ info->flags |= IEEE80211_TX_CTL_AMPDU;
reset_agg_timer = true;
} else {
queued = true;
diff --git a/net/mptcp/pm_kernel.c b/net/mptcp/pm_kernel.c
index 1a7750813235..f7396628490e 100644
--- a/net/mptcp/pm_kernel.c
+++ b/net/mptcp/pm_kernel.c
@@ -906,6 +906,7 @@ static int mptcp_pm_nl_create_listen_socket(struct sock *sk,
* old status is known (TCP_CLOSE).
*/
inet_sk_state_store(newsk, TCP_LISTEN);
+ sock_set_flag(newsk, SOCK_RCU_FREE);
lock_sock(ssk);
WRITE_ONCE(mptcp_subflow_ctx(ssk)->pm_listener, true);
err = __inet_listen_sk(ssk, backlog);
diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c
index 2c72cb50fbf3..0b67b46ef872 100644
--- a/net/mptcp/subflow.c
+++ b/net/mptcp/subflow.c
@@ -379,6 +379,10 @@ static void subflow_prep_synack(const struct sock *sk, struct request_sock *req,
struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
struct inet_request_sock *ireq = inet_rsk(req);
+ /* MP_JOIN cannot use TFO, do not send a cookie in the SYN/ACK */
+ if (foc && mptcp_subflow_rsk(req)->mp_join)
+ foc->len = -1;
+
/* clear tstamp_ok, as needed depending on cookie */
if (foc && foc->len > -1)
ireq->tstamp_ok = 0;
@@ -887,6 +891,13 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
fallback = true;
} else if (subflow_req->mp_join) {
+ /* a SYN skb here comes from TFO, which MP_JOIN cannot use: just
+ * fall back to the regular path.
+ */
+ if ((TCP_SKB_CB(skb)->tcp_flags & (TCPHDR_SYN | TCPHDR_ACK)) ==
+ TCPHDR_SYN)
+ return NULL;
+
mptcp_get_options(skb, &mp_opt);
if (!(mp_opt.suboptions & OPTION_MPTCP_MPJ_ACK))
fallback = true;
diff --git a/net/qrtr/af_qrtr.c b/net/qrtr/af_qrtr.c
index e7b3647424b8..45cb8e8c36ca 100644
--- a/net/qrtr/af_qrtr.c
+++ b/net/qrtr/af_qrtr.c
@@ -122,6 +122,7 @@ static DEFINE_XARRAY_ALLOC(qrtr_ports);
* @qrtr_tx_flow: xarray of qrtr_tx_flow, keyed by node << 32 | port
* @qrtr_tx_lock: lock for qrtr_tx_flow inserts
* @hello_sent: hello packet send successful
+ * @hello_rcvd: remote has sent a hello packet
* @rx_queue: receive queue
* @item: list item for broadcast list
* @say_hello: delayed work for sending hello packet
@@ -136,6 +137,7 @@ struct qrtr_node {
struct mutex qrtr_tx_lock; /* for qrtr_tx_flow */
bool hello_sent;
+ bool hello_rcvd;
struct sk_buff_head rx_queue;
struct list_head item;
@@ -541,6 +543,11 @@ int qrtr_endpoint_post(struct qrtr_endpoint *ep, const void *data, size_t len)
qrtr_node_assign(node, le32_to_cpu(pkt->server.node));
}
+ if (cb->type == QRTR_TYPE_HELLO) {
+ /* Pairs with smp_load_acquire() in qrtr_hello_work() */
+ smp_store_release(&node->hello_rcvd, true);
+ }
+
if (cb->type == QRTR_TYPE_RESUME_TX) {
qrtr_tx_resume(node, skb);
} else {
@@ -599,6 +606,8 @@ static void qrtr_hello_work(struct work_struct *work)
struct qrtr_node *node;
struct qrtr_sock *ctrl;
struct sk_buff *skb;
+ bool hello_sent;
+ int rc = 0;
node = container_of(to_delayed_work(work), struct qrtr_node, say_hello);
@@ -609,17 +618,45 @@ static void qrtr_hello_work(struct work_struct *work)
return;
}
- skb = qrtr_alloc_ctrl_packet(&pkt, GFP_KERNEL);
- if (!skb) {
- qrtr_port_put(ctrl);
- schedule_delayed_work(&node->say_hello, msecs_to_jiffies(100));
- return;
+ mutex_lock(&node->ep_lock);
+ hello_sent = node->hello_sent;
+ mutex_unlock(&node->ep_lock);
+
+ from.sq_node = qrtr_local_nid;
+ if (!hello_sent) {
+ skb = qrtr_alloc_ctrl_packet(&pkt, GFP_KERNEL);
+ if (!skb)
+ goto retry;
+
+ pkt->cmd = cpu_to_le32(QRTR_TYPE_HELLO);
+ to.sq_node = node->nid;
+ if (qrtr_node_enqueue(node, skb, QRTR_TYPE_HELLO, &from, &to))
+ goto out;
}
+ /* Pairs with smp_store_release() in qrtr_endpoint_post() */
+ if (!smp_load_acquire(&node->hello_rcvd))
+ goto out;
+
+ skb = qrtr_alloc_ctrl_packet(&pkt, GFP_KERNEL);
+ if (!skb)
+ goto retry;
+
pkt->cmd = cpu_to_le32(QRTR_TYPE_HELLO);
- from.sq_node = qrtr_local_nid;
- to.sq_node = node->nid;
- qrtr_node_enqueue(node, skb, QRTR_TYPE_HELLO, &from, &to);
+ to.sq_node = READ_ONCE(node->nid);
+
+ mutex_lock(&node->ep_lock);
+ if (node->ep)
+ rc = qrtr_local_enqueue(NULL, skb, QRTR_TYPE_HELLO, &to, &from);
+ else
+ kfree_skb(skb);
+ mutex_unlock(&node->ep_lock);
+ if (!rc)
+ goto out;
+
+retry:
+ schedule_delayed_work(&node->say_hello, msecs_to_jiffies(100));
+out:
qrtr_port_put(ctrl);
}
diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
index 3e8ee4a691c3..d2e0d20f3ba1 100644
--- a/net/sched/sch_generic.c
+++ b/net/sched/sch_generic.c
@@ -96,9 +96,34 @@ static void qdisc_maybe_clear_missed(struct Qdisc *q,
#define SKB_XOFF_MAGIC ((struct sk_buff *)1UL)
+/* Re-cap a stale queue_mapping; an ONETXQUEUE qdisc pins q->dev_queue. */
+static u16 qdisc_cap_skb_tx_queue(struct Qdisc *q, struct sk_buff *skb)
+{
+ u16 queue = skb_get_queue_mapping(skb);
+ struct net_device *dev = qdisc_dev(q);
+ u16 capped;
+
+ if (q->flags & TCQ_F_ONETXQUEUE)
+ capped = q->dev_queue - netdev_get_tx_queue(dev, 0);
+ else
+ capped = netdev_cap_txqueue(dev, queue);
+
+ if (unlikely(capped != queue))
+ skb_set_queue_mapping(skb, capped);
+
+ return capped;
+}
+
+static struct netdev_queue *skb_cap_tx_queue(struct Qdisc *q,
+ struct sk_buff *skb)
+{
+ return netdev_get_tx_queue(qdisc_dev(q),
+ qdisc_cap_skb_tx_queue(q, skb));
+}
+
static inline struct sk_buff *__skb_dequeue_bad_txq(struct Qdisc *q)
{
- const struct netdev_queue *txq = q->dev_queue;
+ const struct netdev_queue *txq;
spinlock_t *lock = NULL;
struct sk_buff *skb;
@@ -110,7 +135,8 @@ static inline struct sk_buff *__skb_dequeue_bad_txq(struct Qdisc *q)
skb = skb_peek(&q->skb_bad_txq);
if (skb) {
/* check the reason of requeuing without tx lock first */
- txq = skb_get_tx_queue(txq->dev, skb);
+ /* re-cap: real_num_tx_queues may have shrunk */
+ txq = skb_cap_tx_queue(q, skb);
if (!netif_xmit_frozen_or_stopped(txq)) {
skb = __skb_dequeue(&q->skb_bad_txq);
if (qdisc_is_percpu_stats(q)) {
@@ -224,6 +250,8 @@ static void try_bulk_dequeue_skb(struct Qdisc *q,
if (!nskb)
break;
+ /* Every element of an ONETXQUEUE qdisc is pinned to q->dev_queue. */
+ qdisc_cap_skb_tx_queue(q, nskb);
bytelimit -= nskb->len; /* covers GSO len */
skb->next = nskb;
skb = nskb;
@@ -241,7 +269,6 @@ static void try_bulk_dequeue_skb_slow(struct Qdisc *q,
struct sk_buff *skb,
int *packets)
{
- int mapping = skb_get_queue_mapping(skb);
struct sk_buff *nskb;
int cnt = 0;
@@ -249,7 +276,9 @@ static void try_bulk_dequeue_skb_slow(struct Qdisc *q,
nskb = q->dequeue(q);
if (!nskb)
break;
- if (unlikely(skb_get_queue_mapping(nskb) != mapping)) {
+ /* A tc BPF store may have changed the mapping; cap it. */
+ if (unlikely(qdisc_cap_skb_tx_queue(q, nskb) !=
+ skb_get_queue_mapping(skb))) {
qdisc_enqueue_skb_bad_txq(q, nskb);
break;
}
@@ -294,7 +323,7 @@ static struct sk_buff *dequeue_skb(struct Qdisc *q, bool *validate,
if (xfrm_offload(skb))
*validate = true;
/* check the reason of requeuing without tx lock first */
- txq = skb_get_tx_queue(txq->dev, skb);
+ txq = skb_cap_tx_queue(q, skb);
if (!netif_xmit_frozen_or_stopped(txq)) {
skb = __skb_dequeue(&q->gso_skb);
if (qdisc_is_percpu_stats(q)) {
@@ -330,6 +359,8 @@ static struct sk_buff *dequeue_skb(struct Qdisc *q, bool *validate,
skb = q->dequeue(q);
if (skb) {
bulk:
+ /* cap the head; the bulk helpers cap every appended element */
+ qdisc_cap_skb_tx_queue(q, skb);
if (qdisc_may_bulk(q))
try_bulk_dequeue_skb(q, skb, txq, packets, budget);
else
diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c
index 6f3b7273cb16..e900890e9c54 100644
--- a/net/sched/sch_qfq.c
+++ b/net/sched/sch_qfq.c
@@ -517,11 +517,14 @@ static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
sch_tree_lock(sch);
qfq_init_agg(q, new_agg, lmax, weight);
}
+ if (existing && new_agg == cl->agg)
+ goto unlock;
if (existing)
qfq_deact_rm_from_agg(q, cl);
else
qdisc_class_hash_insert(&q->clhash, &cl->common);
qfq_add_to_agg(q, new_agg, cl);
+unlock:
sch_tree_unlock(sch);
qdisc_class_hash_grow(sch, &q->clhash);
diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index 39ac5b97aa3a..0eb0803047a1 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -632,6 +632,11 @@ static int taprio_enqueue(struct sk_buff *skb, struct Qdisc *sch,
int queue;
queue = skb_get_queue_mapping(skb);
+ if (unlikely(queue >= qdisc_dev(sch)->real_num_tx_queues)) {
+ /* A tc BPF program may have stored an out-of-range mapping. */
+ queue = netdev_cap_txqueue(qdisc_dev(sch), queue);
+ skb_set_queue_mapping(skb, queue);
+ }
child = q->qdiscs[queue];
if (unlikely(child == &noop_qdisc))
diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c
index 84a4c97d0f75..0896ec6c75ce 100644
--- a/net/sctp/sm_make_chunk.c
+++ b/net/sctp/sm_make_chunk.c
@@ -94,7 +94,7 @@ static void sctp_control_release_owner(struct sk_buff *skb)
}
}
-static void sctp_control_set_owner_w(struct sctp_chunk *chunk)
+void sctp_control_set_owner_w(struct sctp_chunk *chunk)
{
struct sctp_association *asoc = chunk->asoc;
struct sk_buff *skb = chunk->skb;
@@ -107,7 +107,8 @@ static void sctp_control_set_owner_w(struct sctp_chunk *chunk)
* For now don't do anything for now.
*/
if (chunk->auth) {
- chunk->shkey = asoc->shkey;
+ if (!chunk->shkey)
+ chunk->shkey = asoc->shkey;
sctp_auth_shkey_hold(chunk->shkey);
}
skb->sk = asoc ? asoc->base.sk : NULL;
@@ -136,6 +137,7 @@ int sctp_init_cause(struct sctp_chunk *chunk, __be16 cause_code,
size_t paylen)
{
struct sctp_errhdr err;
+ int chunklen, padlen;
__u16 len;
/* Cause code constants are now defined in network order. */
@@ -143,7 +145,13 @@ int sctp_init_cause(struct sctp_chunk *chunk, __be16 cause_code,
len = sizeof(err) + paylen;
err.length = htons(len);
- if (skb_tailroom(chunk->skb) < len)
+ /* sctp_addto_chunk() pads the chunk up to a 4-byte boundary before
+ * appending the cause, so account for that padding here as well.
+ */
+ chunklen = ntohs(chunk->chunk_hdr->length);
+ padlen = SCTP_PAD4(chunklen) - chunklen;
+
+ if (skb_tailroom(chunk->skb) < padlen + len)
return -ENOSPC;
chunk->subh.err_hdr = sctp_addto_chunk(chunk, sizeof(err), &err);
diff --git a/net/sctp/socket.c b/net/sctp/socket.c
index c616f1609d0c..b5a57f61ee6d 100644
--- a/net/sctp/socket.c
+++ b/net/sctp/socket.c
@@ -148,25 +148,47 @@ static inline void sctp_set_owner_w(struct sctp_chunk *chunk)
static void sctp_clear_owner_w(struct sctp_chunk *chunk)
{
+ /* Keep the shkey alive until the new owner takes its reference. */
+ if (chunk->shkey)
+ sctp_auth_shkey_hold(chunk->shkey);
skb_orphan(chunk->skb);
}
+static void sctp_set_owner_w_migrate(struct sctp_chunk *chunk)
+{
+ struct sctp_shared_key *shkey = chunk->shkey;
+
+ if (chunk->msg)
+ sctp_set_owner_w(chunk);
+ else
+ sctp_control_set_owner_w(chunk);
+
+ if (shkey)
+ sctp_auth_shkey_release(shkey);
+}
+
+static void sctp_process_tx_chunk(struct sctp_association *asoc,
+ struct sctp_chunk *chunk, bool clear,
+ void (*cb)(struct sctp_chunk *))
+{
+ if ((clear && asoc->base.sk == chunk->skb->sk) ||
+ (!clear && asoc->base.sk != chunk->skb->sk))
+ cb(chunk);
+}
+
#define traverse_and_process() \
do { \
msg = chunk->msg; \
if (msg == prev_msg) \
continue; \
- list_for_each_entry(c, &msg->chunks, frag_list) { \
- if ((clear && asoc->base.sk == c->skb->sk) || \
- (!clear && asoc->base.sk != c->skb->sk)) \
- cb(c); \
- } \
+ list_for_each_entry(c, &msg->chunks, frag_list) \
+ sctp_process_tx_chunk(asoc, c, clear, cb); \
prev_msg = msg; \
} while (0)
-static void sctp_for_each_tx_datachunk(struct sctp_association *asoc,
- bool clear,
- void (*cb)(struct sctp_chunk *))
+static void sctp_for_each_tx_chunk(struct sctp_association *asoc,
+ bool clear,
+ void (*cb)(struct sctp_chunk *))
{
struct sctp_datamsg *msg, *prev_msg = NULL;
@@ -189,6 +211,21 @@ static void sctp_for_each_tx_datachunk(struct sctp_association *asoc,
list_for_each_entry(chunk, &q->out_chunk_list, list)
traverse_and_process();
+
+ list_for_each_entry(chunk, &q->control_chunk_list, list)
+ sctp_process_tx_chunk(asoc, chunk, clear, cb);
+
+ list_for_each_entry(chunk, &asoc->asconf_ack_list, transmitted_list)
+ sctp_process_tx_chunk(asoc, chunk, clear, cb);
+
+ list_for_each_entry(chunk, &asoc->addip_chunk_list, list)
+ sctp_process_tx_chunk(asoc, chunk, clear, cb);
+
+ if (asoc->strreset_chunk)
+ sctp_process_tx_chunk(asoc, asoc->strreset_chunk, clear, cb);
+
+ if (asoc->addip_last_asconf)
+ sctp_process_tx_chunk(asoc, asoc->addip_last_asconf, clear, cb);
}
static void sctp_for_each_rx_skb(struct sctp_association *asoc, struct sock *sk,
@@ -9647,9 +9684,9 @@ static int sctp_sock_migrate(struct sock *oldsk, struct sock *newsk,
* paths won't try to lock it and then oldsk.
*/
lock_sock_nested(newsk, SINGLE_DEPTH_NESTING);
- sctp_for_each_tx_datachunk(assoc, true, sctp_clear_owner_w);
+ sctp_for_each_tx_chunk(assoc, true, sctp_clear_owner_w);
sctp_assoc_migrate(assoc, newsk);
- sctp_for_each_tx_datachunk(assoc, false, sctp_set_owner_w);
+ sctp_for_each_tx_chunk(assoc, false, sctp_set_owner_w_migrate);
/* If the association on the newsk is already closed before accept()
* is called, set RCV_SHUTDOWN flag.
diff --git a/net/smc/smc_rx.c b/net/smc/smc_rx.c
index 197fddc6271f..63c178c10bf1 100644
--- a/net/smc/smc_rx.c
+++ b/net/smc/smc_rx.c
@@ -286,6 +286,27 @@ int smc_rx_wait(struct smc_sock *smc, long *timeo, size_t peeked,
return rc;
}
+/* Wait for the pipe to release the spliced data. Unlike smc_rx_wait()
+ * it does not end on shutdown or sk_err, the data is already in the
+ * RMB; smc_conn_kill() wakes it and the caller checks conn->killed.
+ */
+static void smc_rx_wait_splice(struct smc_sock *smc, long *timeo)
+{
+ DEFINE_WAIT_FUNC(wait, woken_wake_function);
+ struct smc_connection *conn = &smc->conn;
+ struct sock *sk = &smc->sk;
+
+ sk_set_bit(SOCKWQ_ASYNC_WAITDATA, sk);
+ add_wait_queue(sk_sleep(sk), &wait);
+ /* pairs with the smp_mb() in wq_has_sleeper(): the release drops
+ * splice_pending, then looks for a sleeper
+ */
+ smp_mb();
+ sk_wait_event(sk, timeo, !atomic_read(&conn->splice_pending), &wait);
+ remove_wait_queue(sk_sleep(sk), &wait);
+ sk_clear_bit(SOCKWQ_ASYNC_WAITDATA, sk);
+}
+
static int smc_rx_recv_urg(struct smc_sock *smc, struct msghdr *msg, int len,
int flags)
{
@@ -443,7 +464,23 @@ int smc_rx_recvmsg(struct smc_sock *smc, struct msghdr *msg,
/* could be just 1 byte, even after waiting on data above */
readable = smc_rx_data_available(conn, peeked_bytes);
splbytes = atomic_read(&conn->splice_pending);
- if (!readable || (msg && splbytes)) {
+ if (msg && splbytes) {
+ /* a pipe holds bytes ahead of these; the copy path
+ * skips the timeout and signal checks
+ */
+ if (!read_remaining)
+ break;
+ if (!timeo || signal_pending(current)) {
+ if (!read_done)
+ read_done = timeo ?
+ sock_intr_errno(timeo) :
+ -EAGAIN;
+ break;
+ }
+ smc_rx_wait_splice(smc, &timeo);
+ continue;
+ }
+ if (!readable) {
if (splbytes)
func = smc_rx_data_available_and_no_splice_pend;
else
diff --git a/net/sunrpc/rpcb_clnt.c b/net/sunrpc/rpcb_clnt.c
index 4c0b7fefee4e..84c06f99e1c7 100644
--- a/net/sunrpc/rpcb_clnt.c
+++ b/net/sunrpc/rpcb_clnt.c
@@ -400,7 +400,8 @@ static struct rpc_clnt *rpcb_create(struct net *net, const char *nodename,
return rpc_create(&args);
}
-static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, struct rpc_message *msg, bool is_set)
+static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt,
+ struct rpc_message *msg, bool is_set)
{
int flags = RPC_TASK_NOCONNECT;
int error, result = 0;
@@ -410,8 +411,23 @@ static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, stru
msg->rpc_resp = &result;
error = rpc_call_sync(clnt, msg, flags);
- if (error < 0)
- return error;
+ if (error < 0) {
+ switch (error) {
+ /* rpcbind answered; the reply itself carries the error */
+ case -EPROTONOSUPPORT:
+ case -EPFNOSUPPORT:
+ case -EOPNOTSUPP:
+ case -EACCES:
+ /* the call never made it onto the wire */
+ case -ENOMEM:
+ case -EMSGSIZE:
+ /* the caller is going away; this says nothing about rpcbind */
+ case -ERESTARTSYS:
+ return error;
+ }
+ /* anything else, we assume that rpcbind isn't functional */
+ return -EIO;
+ }
if (!result)
return -EACCES;
diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c
index 4f402bbf97ba..ca6f90653327 100644
--- a/net/sunrpc/svc.c
+++ b/net/sunrpc/svc.c
@@ -1179,10 +1179,40 @@ int svc_generic_rpcbind_set(struct net *net,
error = svc_rpcbind_set_version(net, progp, version,
family, proto, port);
+ /* -EIO means no answer, not a refusal, so vs_rpcb_optnl must keep it. */
+ if (error == -EIO)
+ return error;
+
return (vers->vs_rpcb_optnl) ? 0 : error;
}
EXPORT_SYMBOL_GPL(svc_generic_rpcbind_set);
+/**
+ * svc_rpcb_failure_count - local rpcbind calls for @serv that got no answer
+ * @serv: RPC service to query
+ *
+ * svc_register() adds one for each of its calls that got no answer. A reply
+ * that refuses one entry does not count, because rpcbind answered and the
+ * next entry may still succeed.
+ *
+ * The count is kept per serv rather than per net. The local rpcbind client
+ * is per-net and lockd shares it, but a count that another service can move
+ * says nothing about this serv's own calls.
+ *
+ * This is for callers that cannot see the svc_register() return, because a
+ * transport class sits in between. Such a caller reads the count before it
+ * starts and compares as it goes, so there is no state to reset between
+ * operations. The count never resets, and callers must not attach meaning
+ * to the value itself.
+ *
+ * Return: the number of unanswered calls since this serv was created.
+ */
+unsigned int svc_rpcb_failure_count(struct svc_serv *serv)
+{
+ return atomic_read(&serv->sv_rpcb_failures);
+}
+EXPORT_SYMBOL_GPL(svc_rpcb_failure_count);
+
/**
* svc_register - register an RPC service with the local portmapper
* @serv: svc_serv struct for the service to register
@@ -1193,10 +1223,11 @@ EXPORT_SYMBOL_GPL(svc_generic_rpcbind_set);
*
* Service is registered for any address in the passed-in protocol family
*/
-int svc_register(const struct svc_serv *serv, struct net *net,
+int svc_register(struct svc_serv *serv, struct net *net,
const int family, const unsigned short proto,
const unsigned short port)
{
+ bool noanswer = false;
unsigned int p, i;
int error = 0;
@@ -1208,10 +1239,16 @@ int svc_register(const struct svc_serv *serv, struct net *net,
struct svc_program *progp = &serv->sv_programs[p];
for (i = 0; i < progp->pg_nvers; i++) {
+ const struct svc_version *vers = progp->pg_vers[i];
int ret;
ret = progp->pg_rpcbind_set(net, progp, i,
family, proto, port);
+ if (ret == -EIO) {
+ noanswer = true;
+ if (vers && vers->vs_rpcb_optnl)
+ ret = 0;
+ }
if (ret < 0) {
printk(KERN_WARNING "svc: failed to register "
"%sv%u RPC service (errno %d).\n",
@@ -1223,6 +1260,9 @@ int svc_register(const struct svc_serv *serv, struct net *net,
}
}
+ if (noanswer)
+ atomic_inc(&serv->sv_rpcb_failures);
+
return error;
}
diff --git a/net/wireless/reg.c b/net/wireless/reg.c
index 11665e0a7efc..44226687349e 100644
--- a/net/wireless/reg.c
+++ b/net/wireless/reg.c
@@ -679,6 +679,9 @@ static bool valid_rule(const u8 *data, unsigned int size, u16 rule_ptr)
/* mandatory fields */
if (rule->len < offsetofend(struct fwdb_rule, max_bw))
return false;
+ /* the rule must lie entirely within the database */
+ if ((u8 *)rule + rule->len > data + size)
+ return false;
if (rule->len >= offsetofend(struct fwdb_rule, wmm_ptr)) {
u32 wmm_ptr = be16_to_cpu(rule->wmm_ptr) << 2;
struct fwdb_wmm_rule *wmm;
diff --git a/net/xfrm/espintcp.c b/net/xfrm/espintcp.c
index 3e72b9f067b9..4052ae5bbb2e 100644
--- a/net/xfrm/espintcp.c
+++ b/net/xfrm/espintcp.c
@@ -434,7 +434,9 @@ static void espintcp_destruct(struct sock *sk)
bool tcp_is_ulp_esp(struct sock *sk)
{
- return sk->sk_prot == &espintcp_prot || sk->sk_prot == &espintcp6_prot;
+ const struct proto *prot = READ_ONCE(sk->sk_prot);
+
+ return prot == &espintcp_prot || prot == &espintcp6_prot;
}
EXPORT_SYMBOL_GPL(tcp_is_ulp_esp);
@@ -466,34 +468,41 @@ static int espintcp_init_sk(struct sock *sk)
__sk_dst_reset(sk);
- strp_check_rcv(&ctx->strp);
skb_queue_head_init(&ctx->ike_queue);
skb_queue_head_init(&ctx->out_queue);
+ INIT_WORK(&ctx->work, espintcp_tx_work);
+
+ ctx->saved_data_ready = sk->sk_data_ready;
+ ctx->saved_write_space = sk->sk_write_space;
+ ctx->saved_destruct = sk->sk_destruct;
+
+ /* avoid using task_frag */
+ sk->sk_allocation = GFP_ATOMIC;
+ sk->sk_use_task_frag = false;
+
+ rcu_assign_pointer(icsk->icsk_ulp_data, ctx);
+
+ /* pairs with smp_rmb() in espintcp_getctx() */
+ smp_wmb();
+
+ WRITE_ONCE(sk->sk_data_ready, espintcp_data_ready);
+ WRITE_ONCE(sk->sk_write_space, espintcp_write_space);
+ sk->sk_destruct = espintcp_destruct;
if (sk->sk_family == AF_INET) {
- sk->sk_prot = &espintcp_prot;
- sk->sk_socket->ops = &espintcp_ops;
+ WRITE_ONCE(sk->sk_prot, &espintcp_prot);
+ WRITE_ONCE(sk->sk_socket->ops, &espintcp_ops);
} else {
mutex_lock(&tcpv6_prot_mutex);
if (!espintcp6_prot.recvmsg)
build_protos(&espintcp6_prot, &espintcp6_ops, sk->sk_prot, sk->sk_socket->ops);
mutex_unlock(&tcpv6_prot_mutex);
- sk->sk_prot = &espintcp6_prot;
- sk->sk_socket->ops = &espintcp6_ops;
+ WRITE_ONCE(sk->sk_prot, &espintcp6_prot);
+ WRITE_ONCE(sk->sk_socket->ops, &espintcp6_ops);
}
- ctx->saved_data_ready = sk->sk_data_ready;
- ctx->saved_write_space = sk->sk_write_space;
- ctx->saved_destruct = sk->sk_destruct;
- sk->sk_data_ready = espintcp_data_ready;
- sk->sk_write_space = espintcp_write_space;
- sk->sk_destruct = espintcp_destruct;
- rcu_assign_pointer(icsk->icsk_ulp_data, ctx);
- INIT_WORK(&ctx->work, espintcp_tx_work);
- /* avoid using task_frag */
- sk->sk_allocation = GFP_ATOMIC;
- sk->sk_use_task_frag = false;
+ strp_check_rcv(&ctx->strp);
return 0;
diff --git a/net/xfrm/xfrm_interface_core.c b/net/xfrm/xfrm_interface_core.c
index 688306bf62c5..9835fbe63ccb 100644
--- a/net/xfrm/xfrm_interface_core.c
+++ b/net/xfrm/xfrm_interface_core.c
@@ -531,8 +531,13 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev)
memset(&fl, 0, sizeof(fl));
+ if (!pskb_inet_may_pull(skb))
+ goto tx_err;
+
switch (skb->protocol) {
case htons(ETH_P_IPV6):
+ if (ipv6_hdr(skb)->version != 6)
+ goto tx_err;
memset(IP6CB(skb), 0, sizeof(*IP6CB(skb)));
xfrm_decode_session(dev_net(dev), skb, &fl, AF_INET6);
if (!dst) {
@@ -548,6 +553,9 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev)
}
break;
case htons(ETH_P_IP):
+ if (ip_hdr(skb)->version != 4 || ip_hdr(skb)->ihl < 5 ||
+ !pskb_network_may_pull(skb, ip_hdr(skb)->ihl * 4))
+ goto tx_err;
memset(IPCB(skb), 0, sizeof(*IPCB(skb)));
xfrm_decode_session(dev_net(dev), skb, &fl, AF_INET);
if (!dst) {
diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
index 513c9f228334..f6f40ba713d5 100644
--- a/net/xfrm/xfrm_policy.c
+++ b/net/xfrm/xfrm_policy.c
@@ -2157,6 +2157,7 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type,
struct xfrm_pol_inexact_bin *bin;
struct xfrm_policy *pol, *ret;
struct hlist_head *chain;
+ unsigned int inexact_sequence;
unsigned int sequence;
int err;
@@ -2191,12 +2192,18 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type,
goto skip_inexact;
bin = xfrm_policy_inexact_lookup_rcu(net, type, family, dir, if_id);
- if (!bin || !xfrm_policy_find_inexact_candidates(&cand, bin, saddr,
- daddr))
+ if (!bin)
+ goto skip_inexact;
+
+ inexact_sequence = read_seqcount_begin(&bin->count);
+ if (!xfrm_policy_find_inexact_candidates(&cand, bin, saddr, daddr))
goto skip_inexact;
pol = xfrm_policy_eval_candidates(&cand, ret, fl, type,
family, if_id);
+ if (read_seqcount_retry(&bin->count, inexact_sequence))
+ goto retry;
+
if (pol) {
ret = pol;
if (IS_ERR(pol))
diff --git a/rust/kernel/cpufreq.rs b/rust/kernel/cpufreq.rs
index b75d692846a2..f31671129c98 100644
--- a/rust/kernel/cpufreq.rs
+++ b/rust/kernel/cpufreq.rs
@@ -696,8 +696,8 @@ fn clear_data<T: ForeignOwnable>(&mut self) -> Option<T> {
impl<'a> PolicyCpu<'a> {
fn from_cpu(cpu: CpuId) -> Result<Self> {
// SAFETY: It is safe to call `cpufreq_cpu_get` for any valid CPU.
- let ptr =
- NonNull::new(unsafe { bindings::cpufreq_cpu_get(u32::from(cpu)) }).ok_or(ENODEV)?;
+ let ptr = unsafe { bindings::cpufreq_cpu_get(u32::from(cpu)) };
+ let ptr = NonNull::new(ptr).ok_or(ENODEV)?;
Ok(Self(
// SAFETY: `ptr` is non-NULL and `cpufreq_cpu_get()` took a reference on it, so it is
diff --git a/sound/core/hwdep.c b/sound/core/hwdep.c
index c6de5345e946..fb9034897c56 100644
--- a/sound/core/hwdep.c
+++ b/sound/core/hwdep.c
@@ -212,6 +212,7 @@ static int snd_hwdep_dsp_load(struct snd_hwdep *hw,
return -ENXIO;
if (info->index >= 32)
return -EINVAL;
+ guard(mutex)(&hw->dsp_mutex);
/* check whether the dsp was already loaded */
if (hw->dsp_loaded & (1u << info->index))
return -EBUSY;
@@ -380,6 +381,7 @@ int snd_hwdep_new(struct snd_card *card, char *id, int device,
init_waitqueue_head(&hwdep->open_wait);
mutex_init(&hwdep->open_mutex);
+ mutex_init(&hwdep->dsp_mutex);
hwdep->card = card;
hwdep->device = device;
if (id)
diff --git a/sound/core/ump_convert.c b/sound/core/ump_convert.c
index 0a7972d91ec5..ecbd894f4c83 100644
--- a/sound/core/ump_convert.c
+++ b/sound/core/ump_convert.c
@@ -69,7 +69,7 @@ static u32 upscale_14_to_32bit(u16 src)
static int cvt_ump_system_to_legacy(u32 data, unsigned char *buf)
{
buf[0] = ump_message_status_channel(data);
- switch (ump_message_status_code(data)) {
+ switch (buf[0]) {
case UMP_SYSTEM_STATUS_MIDI_TIME_CODE:
case UMP_SYSTEM_STATUS_SONG_SELECT:
buf[1] = (data >> 8) & 0x7f;
diff --git a/sound/hda/codecs/realtek/alc269.c b/sound/hda/codecs/realtek/alc269.c
index 68650004ce0f..49c4d01759b0 100644
--- a/sound/hda/codecs/realtek/alc269.c
+++ b/sound/hda/codecs/realtek/alc269.c
@@ -3537,6 +3537,27 @@ static void alc233_fixup_no_audio_jack(struct hda_codec *codec,
alc_process_coef_fw(codec, alc233_fixup_no_audio_jack_coefs);
}
+static const struct coef_fw alc256_asus_speaker_coefs[] = {
+ WRITE_COEF(0x10, 0x7f20), WRITE_COEF(0x16, 0x0c50), WRITE_COEF(0x35, 0x8d6a),
+ WRITE_COEF(0x37, 0xfe06), WRITE_COEF(0x57, 0x7f7f),
+ {}
+};
+
+static void alc256_fixup_asus_speaker_coefs(struct hda_codec *codec,
+ const struct hda_fixup *fix,
+ int action)
+{
+ /*
+ * The internal speakers of the ASUS ExpertBook PM3606CHA stay silent
+ * with the default COEF values. These are the values set by the
+ * Windows driver. They are applied on every init so that they are
+ * restored after resume, too.
+ */
+ if (action != HDA_FIXUP_ACT_INIT)
+ return;
+ alc_process_coef_fw(codec, alc256_asus_speaker_coefs);
+}
+
static void alc256_fixup_mic_no_presence_and_resume(struct hda_codec *codec,
const struct hda_fixup *fix,
int action)
@@ -4391,6 +4412,8 @@ enum {
ALC285_LENOVO_DAC_RENAME,
ALC287_FIXUP_YOGA9_SPEAKER2_TO_DAC1,
ALC256_FIXUP_IPASON_SMARTBOOK_S1,
+ ALC256_FIXUP_ASUS_SPEAKER_COEFS,
+ ALC245_FIXUP_MINISFORUM_V3_MIC_NO_PRESENCE,
};
/* A special fixup for Lenovo C940 and Yoga Duet 7;
@@ -7208,6 +7231,21 @@ static const struct hda_fixup alc269_fixups[] = {
{ }
},
},
+ [ALC256_FIXUP_ASUS_SPEAKER_COEFS] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc256_fixup_asus_speaker_coefs,
+ .chained = true,
+ .chain_id = ALC256_FIXUP_ASUS_MIC_NO_PRESENCE,
+ },
+ [ALC245_FIXUP_MINISFORUM_V3_MIC_NO_PRESENCE] = {
+ .type = HDA_FIXUP_PINS,
+ .v.pins = (const struct hda_pintbl[]) {
+ { 0x19, 0x04a19150 }, /* headset mic jack, no presence detect */
+ { }
+ },
+ .chained = true,
+ .chain_id = ALC245_FIXUP_BASS_HP_DAC
+ },
};
static const struct hda_quirk alc269_fixup_tbl[] = {
@@ -7335,6 +7373,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1028, 0x097e, "Dell Precision", ALC289_FIXUP_DUAL_SPK),
SND_PCI_QUIRK(0x1028, 0x098d, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1028, 0x09bf, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1028, 0x09d1, "Alienware m17 R3", ALC289_FIXUP_DUAL_SPK),
SND_PCI_QUIRK(0x1028, 0x0a2e, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC),
SND_PCI_QUIRK(0x1028, 0x0a30, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC),
SND_PCI_QUIRK(0x1028, 0x0a38, "Dell Latitude 7520", ALC269_FIXUP_DELL4_MIC_NO_PRESENCE_QUIET),
@@ -7605,7 +7644,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x103c, 0x8a2b, "HP Envy 15", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8a2c, "HP Envy 16", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8a2d, "HP Envy 16", ALC287_FIXUP_CS35L41_I2C_2),
- SND_PCI_QUIRK(0x103c, 0x8a2e, "HP Envy 16", ALC287_FIXUP_CS35L41_I2C_2),
+ SND_PCI_QUIRK(0x103c, 0x8a2e, "HP ENVY Laptop 17-cr0xxx", ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS),
SND_PCI_QUIRK(0x103c, 0x8a30, "HP Envy 17", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8a31, "HP ENVY x360 2-in-1 Laptop 15-ey0xxx", ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS),
SND_PCI_QUIRK(0x103c, 0x8a34, "HP Pavilion x360 2-in-1 Laptop 14-ek0xxx", ALC245_FIXUP_HP_MUTE_LED_COEFBIT),
@@ -7990,6 +8029,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1043, 0x31e1, "ASUS B5605CCA", ALC294_FIXUP_ASUS_CS35L41_SPI_2),
SND_PCI_QUIRK(0x1043, 0x31f1, "ASUS B3605CCA", ALC294_FIXUP_ASUS_CS35L41_SPI_2),
SND_PCI_QUIRK(0x1043, 0x3391, "ASUS PM3606CKA", ALC287_FIXUP_CS35L41_I2C_2),
+ SND_PCI_QUIRK(0x1043, 0x3501, "ASUS PM3606CHA", ALC256_FIXUP_ASUS_SPEAKER_COEFS),
SND_PCI_QUIRK(0x1043, 0x3601, "ASUS PM5406CGA", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x1043, 0x3611, "ASUS PM5606CGA", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x1043, 0x3701, "ASUS P5406CCA", ALC245_FIXUP_CS35L41_SPI_2),
@@ -8445,7 +8485,8 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1ee7, 0x2081, "HONOR MRB-XXX M1020", ALC256_FIXUP_HONOR_MRB_XXX_M1020_AUDIO),
SND_PCI_QUIRK(0x1f4c, 0xb020, "Minisforum AI X1 Pro",
ALC245_FIXUP_MINISFORUM_JACK_DETECT),
- SND_PCI_QUIRK(0x1f4c, 0xe001, "Minisforum V3 (SE)", ALC245_FIXUP_BASS_HP_DAC),
+ SND_PCI_QUIRK(0x1f4c, 0xe001, "Minisforum V3 (SE)",
+ ALC245_FIXUP_MINISFORUM_V3_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1f66, 0x0105, "Ayaneo Portable Game Player", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x2014, 0x800a, "Positivo ARN50", ALC269_FIXUP_LIMIT_INT_MIC_BOOST),
SND_PCI_QUIRK(0x2039, 0x0001, "Inspur S14-G1", ALC295_FIXUP_CHROME_BOOK),
diff --git a/sound/hda/codecs/side-codecs/tas2781_hda.c b/sound/hda/codecs/side-codecs/tas2781_hda.c
index e0da299df780..9162173ccb10 100644
--- a/sound/hda/codecs/side-codecs/tas2781_hda.c
+++ b/sound/hda/codecs/side-codecs/tas2781_hda.c
@@ -50,6 +50,31 @@ static void cali_cnv(unsigned char *data, unsigned int base, int offset)
(u32 *)®_data, TASDEV_CALIB_N);
}
+/*
+ * V1 data without CRC is only trusted if every expected device has a
+ * populated, self-consistent entry: R0 (Q27) * 1/R0 (Q31) must be
+ * close to 1.0 in Q58.
+ */
+static bool tas2781_v1_nocrc_valid(const unsigned int *v, int ndev)
+{
+ const u64 one = 1ULL << 58;
+ int i;
+
+ if (ndev <= 0 || ndev > 4)
+ return false;
+
+ for (i = 0; i < ndev; i++) {
+ const unsigned int *e = &v[i * 5];
+ u64 prod = (u64)e[0] * e[1];
+
+ if (!e[0] || !e[1] ||
+ prod < one - one / 10 || prod > one + one / 10)
+ return false;
+ }
+
+ return true;
+}
+
static void tas2781_apply_calib(struct tasdevice_priv *p)
{
struct calidata *cali_data = &p->cali_data;
@@ -145,7 +170,7 @@ static void tas2781_apply_calib(struct tasdevice_priv *p)
* but leave TimeStamp and CRC zeroed.
*/
if (!tmp_val[20] && !tmp_val[21] &&
- tmp_val[0] && tmp_val[1]) {
+ tas2781_v1_nocrc_valid(tmp_val, p->ndev)) {
dev_warn(p->dev,
"%s: V1 data without CRC, accepting\n",
__func__);
diff --git a/sound/pci/cs46xx/dsp_spos.c b/sound/pci/cs46xx/dsp_spos.c
index a77e6f5a14ad..57b0a6321e0b 100644
--- a/sound/pci/cs46xx/dsp_spos.c
+++ b/sound/pci/cs46xx/dsp_spos.c
@@ -1104,6 +1104,8 @@ int cs46xx_dsp_scb_and_task_init (struct snd_cs46xx *chip)
null_scb.entry_point = null_algorithm->address;
ins->the_null_scb = cs46xx_dsp_create_scb(chip, "nullSCB", (u32 *)&null_scb, NULL_SCB_ADDR);
+ if (!ins->the_null_scb)
+ return -ENOMEM;
ins->the_null_scb->task_entry = null_algorithm;
ins->the_null_scb->sub_list_ptr = ins->the_null_scb;
ins->the_null_scb->next_scb_ptr = ins->the_null_scb;
diff --git a/sound/pci/cs46xx/dsp_spos_scb_lib.c b/sound/pci/cs46xx/dsp_spos_scb_lib.c
index fd19365026b4..c5208e42cde9 100644
--- a/sound/pci/cs46xx/dsp_spos_scb_lib.c
+++ b/sound/pci/cs46xx/dsp_spos_scb_lib.c
@@ -274,7 +274,8 @@ _dsp_create_generic_scb (struct snd_cs46xx *chip, char * name, u32 * scb_data, u
dev_dbg(chip->card->dev, "dsp_spos: creating SCB <%s>\n", name);
scb = cs46xx_dsp_create_scb(chip,name,scb_data,dest);
-
+ if (!scb)
+ return NULL;
scb->sub_list_ptr = ins->the_null_scb;
scb->next_scb_ptr = ins->the_null_scb;
diff --git a/sound/soc/amd/yc/acp6x-mach.c b/sound/soc/amd/yc/acp6x-mach.c
index 1be8504571ad..96b0bfeddc06 100644
--- a/sound/soc/amd/yc/acp6x-mach.c
+++ b/sound/soc/amd/yc/acp6x-mach.c
@@ -745,6 +745,13 @@ static const struct dmi_system_id yc_acp_quirk_table[] = {
DMI_MATCH(DMI_BOARD_NAME, "8BD6"),
}
},
+ {
+ .driver_data = &acp6x_card,
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "HP"),
+ DMI_MATCH(DMI_BOARD_NAME, "8EE3"),
+ }
+ },
{
.driver_data = &acp6x_card,
.matches = {
diff --git a/sound/soc/codecs/cs35l56-sdw.c b/sound/soc/codecs/cs35l56-sdw.c
index e10f04d677f4..a49ad69644b5 100644
--- a/sound/soc/codecs/cs35l56-sdw.c
+++ b/sound/soc/codecs/cs35l56-sdw.c
@@ -9,6 +9,7 @@
#include <linux/device.h>
#include <linux/err.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/pm_runtime.h>
#include <linux/regmap.h>
#include <linux/soundwire/sdw.h>
@@ -36,6 +37,9 @@
#define CS35L56_LATE_READ_POLL_US 10
#define CS35L56_LATE_READ_TIMEOUT_US 1000
+static DEFINE_MUTEX(cs35l56_fw_idle_wait_lock);
+static bool cs35l56_fw_idle_wait_pending;
+
static int cs35l56_sdw_poll_mem_status(struct sdw_slave *peripheral,
unsigned int mask,
unsigned int match)
@@ -387,14 +391,34 @@ static int __maybe_unused cs35l56_sdw_runtime_resume(struct device *dev)
return 0;
}
+static int cs35l56_sdw_system_suspend_prepare(struct device *dev)
+{
+ cs35l56_fw_idle_wait_pending = true;
+
+ return 0;
+}
+
static int __maybe_unused cs35l56_sdw_system_suspend(struct device *dev)
{
struct cs35l56_private *cs35l56 = dev_get_drvdata(dev);
+ int ret;
if (cs35l56->sdw_attached)
cs35l56_mask_soundwire_interrupts(cs35l56);
- return cs35l56_system_suspend(dev);
+ ret = cs35l56_system_suspend(dev);
+ if (ret < 0)
+ return ret;
+
+ scoped_guard(mutex, &cs35l56_fw_idle_wait_lock) {
+ if (cs35l56_fw_idle_wait_pending && cs35l56->needs_wait_for_fw_idle) {
+ dev_dbg(cs35l56->base.dev, "Wait for FW timer expiry\n");
+ msleep(CS35L56_FW_REQ_ACTIVE_TIMEOUT_MS + 10);
+ cs35l56_fw_idle_wait_pending = false;
+ }
+ }
+
+ return 0;
}
static int cs35l56_sdw_probe(struct sdw_slave *peripheral, const struct sdw_device_id *id)
@@ -460,6 +484,7 @@ static void cs35l56_sdw_remove(struct sdw_slave *peripheral)
}
static const struct dev_pm_ops cs35l56_sdw_pm = {
+ .prepare = cs35l56_sdw_system_suspend_prepare,
SET_RUNTIME_PM_OPS(cs35l56_sdw_runtime_suspend, cs35l56_sdw_runtime_resume, NULL)
SYSTEM_SLEEP_PM_OPS(cs35l56_sdw_system_suspend, cs35l56_system_resume)
LATE_SYSTEM_SLEEP_PM_OPS(cs35l56_system_suspend_late, cs35l56_system_resume_early)
diff --git a/sound/soc/codecs/cs35l56-shared.c b/sound/soc/codecs/cs35l56-shared.c
index 7b3e37d462d6..b37f65cd4d33 100644
--- a/sound/soc/codecs/cs35l56-shared.c
+++ b/sound/soc/codecs/cs35l56-shared.c
@@ -1379,6 +1379,44 @@ void cs35l56_warn_if_firmware_missing(struct cs35l56_base *cs35l56_base)
}
EXPORT_SYMBOL_NS_GPL(cs35l56_warn_if_firmware_missing, "SND_SOC_CS35L56_SHARED");
+bool cs35l56_needs_wait_for_firmware_timer_expiry(struct cs35l56_base *cs35l56_base)
+{
+ unsigned int fw_ver;
+ bool prot_sts;
+ int ret;
+
+ switch (cs35l56_base->type) {
+ case 0x54:
+ case 0x56:
+ case 0x57:
+ switch (cs35l56_base->rev) {
+ case 0xb0:
+ ret = cs35l56_read_prot_status(cs35l56_base, &prot_sts, &fw_ver);
+ if (ret)
+ return true;
+
+ if ((fw_ver & CS35L56_FW_MAIN_VERSION_MASK) < 0x30d07)
+ return true;
+
+ return false;
+ case 0xb2:
+ ret = cs35l56_read_prot_status(cs35l56_base, &prot_sts, &fw_ver);
+ if (ret)
+ return true;
+
+ if ((fw_ver & CS35L56_FW_MAIN_VERSION_MASK) < 0x40710)
+ return true;
+
+ return false;
+ default:
+ return false;
+ }
+ default:
+ return false;
+ }
+}
+EXPORT_SYMBOL_NS_GPL(cs35l56_needs_wait_for_firmware_timer_expiry, "SND_SOC_CS35L56_SHARED");
+
void cs35l56_log_tuning(struct cs35l56_base *cs35l56_base, struct cs_dsp *cs_dsp)
{
__be32 pid, sid, tid;
diff --git a/sound/soc/codecs/cs35l56.c b/sound/soc/codecs/cs35l56.c
index 279db3f13f1e..573cc71a1910 100644
--- a/sound/soc/codecs/cs35l56.c
+++ b/sound/soc/codecs/cs35l56.c
@@ -1693,6 +1693,10 @@ int cs35l56_system_resume(struct device *dev)
dev_dbg(dev, "system_resume\n");
+ /* Assume the firmware reset to default until we can check it */
+ if (cs35l56->base.init_done)
+ cs35l56->needs_wait_for_fw_idle = true;
+
/*
* We might have done a hard reset or the CS35L56 was power-cycled
* so wait for control port to be ready.
@@ -1707,6 +1711,16 @@ int cs35l56_system_resume(struct device *dev)
if (ret)
return ret;
+ if (cs35l56->sdw_peripheral && cs35l56->base.init_done) {
+ ret = pm_runtime_resume_and_get(cs35l56->base.dev);
+ if (ret < 0)
+ return ret;
+
+ cs35l56->needs_wait_for_fw_idle =
+ cs35l56_needs_wait_for_firmware_timer_expiry(&cs35l56->base);
+ pm_runtime_put_autosuspend(cs35l56->base.dev);
+ }
+
/* Firmware won't have been loaded if the component hasn't probed */
if (!cs35l56->component)
return 0;
@@ -2231,6 +2245,9 @@ int cs35l56_init(struct cs35l56_private *cs35l56)
if (ret)
return dev_err_probe(cs35l56->base.dev, ret, "Failed to write ASP1_CONTROL3\n");
+ cs35l56->needs_wait_for_fw_idle =
+ cs35l56_needs_wait_for_firmware_timer_expiry(&cs35l56->base);
+
cs35l56->base.init_done = true;
complete_all(&cs35l56->init_completion);
diff --git a/sound/soc/codecs/cs35l56.h b/sound/soc/codecs/cs35l56.h
index f7cf8aa653e2..b46f410f0e09 100644
--- a/sound/soc/codecs/cs35l56.h
+++ b/sound/soc/codecs/cs35l56.h
@@ -43,6 +43,7 @@ struct cs35l56_private {
bool soft_resetting;
bool sdw_attached;
bool component_registered;
+ bool needs_wait_for_fw_idle;
struct completion init_completion;
int speaker_id;
diff --git a/sound/soc/codecs/hdac_hda.c b/sound/soc/codecs/hdac_hda.c
index 40d602f79314..9e45b20940f2 100644
--- a/sound/soc/codecs/hdac_hda.c
+++ b/sound/soc/codecs/hdac_hda.c
@@ -633,12 +633,35 @@ static const struct snd_soc_component_driver hdac_hda_hdmi_codec = {
.endianness = 1,
};
+static void hdac_hda_restore_dev_type(void *data)
+{
+ struct hdac_device *hdev = data;
+
+ hdev->type = HDA_DEV_LEGACY;
+}
+
static int hdac_hda_dev_probe(struct hdac_device *hdev)
{
- struct hdac_hda_priv *hda_pvt = dev_get_drvdata(&hdev->dev);
+ struct hdac_hda_priv *hda_pvt;
struct hdac_ext_link *hlink;
int ret;
+ hda_pvt = devm_kzalloc(&hdev->dev, sizeof(*hda_pvt), GFP_KERNEL);
+ if (!hda_pvt)
+ return -ENOMEM;
+
+ hda_pvt->codec = dev_to_hda_codec(&hdev->dev);
+ hda_pvt->dev_index = hdev->addr;
+ /* Intel iDisp (HDMI/DP) codec */
+ if (((hdev->vendor_id >> 16) & 0xFFFF) == 0x8086)
+ hda_pvt->need_display_power = true;
+ dev_set_drvdata(&hdev->dev, hda_pvt);
+
+ /* Allow the legacy codec driver to match again after unbind */
+ ret = devm_add_action(&hdev->dev, hdac_hda_restore_dev_type, hdev);
+ if (ret)
+ return ret;
+
/* hold the ref while we probe */
hlink = snd_hdac_ext_bus_get_hlink_by_name(hdev->bus, dev_name(&hdev->dev));
if (!hlink) {
diff --git a/sound/soc/codecs/tas2770.c b/sound/soc/codecs/tas2770.c
index e79754edac5c..355fcbfd01c7 100644
--- a/sound/soc/codecs/tas2770.c
+++ b/sound/soc/codecs/tas2770.c
@@ -513,7 +513,7 @@ static int tas2770_set_dai_tdm_idle(struct snd_soc_dai *dai,
ret = snd_soc_component_update_bits(component, TAS2770_DIN_PD,
TAS2770_DIN_PD_SDOUT,
TAS2770_DIN_PD_SDOUT);
- if (ret)
+ if (ret < 0)
return ret;
break;
@@ -521,12 +521,12 @@ static int tas2770_set_dai_tdm_idle(struct snd_soc_dai *dai,
ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4,
TAS2770_TDM_CFG_REG4_TX_KEEPER,
TAS2770_TDM_CFG_REG4_TX_KEEPER);
- if (ret)
+ if (ret < 0)
return ret;
ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4,
TAS2770_TDM_CFG_REG4_TX_FILL, 0);
- if (ret)
+ if (ret < 0)
return ret;
break;
@@ -534,25 +534,25 @@ static int tas2770_set_dai_tdm_idle(struct snd_soc_dai *dai,
ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4,
TAS2770_TDM_CFG_REG4_TX_KEEPER,
TAS2770_TDM_CFG_REG4_TX_KEEPER);
- if (ret)
+ if (ret < 0)
return ret;
ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4,
TAS2770_TDM_CFG_REG4_TX_FILL,
TAS2770_TDM_CFG_REG4_TX_FILL);
- if (ret)
+ if (ret < 0)
return ret;
break;
case SND_SOC_DAI_TDM_IDLE_OFF:
ret = snd_soc_component_update_bits(component, TAS2770_DIN_PD,
TAS2770_DIN_PD_SDOUT, 0);
- if (ret)
+ if (ret < 0)
return ret;
ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4,
TAS2770_TDM_CFG_REG4_TX_KEEPER, 0);
- if (ret)
+ if (ret < 0)
return ret;
break;
diff --git a/sound/soc/qcom/qdsp6/q6apm.c b/sound/soc/qcom/qdsp6/q6apm.c
index a79bbb34c777..8a756249e588 100644
--- a/sound/soc/qcom/qdsp6/q6apm.c
+++ b/sound/soc/qcom/qdsp6/q6apm.c
@@ -884,8 +884,6 @@ static int apm_probe(gpr_device_t *gdev)
if (!apm)
return -ENOMEM;
- dev_set_drvdata(dev, apm);
-
mutex_init(&apm->lock);
apm->dev = dev;
apm->gdev = gdev;
@@ -899,6 +897,7 @@ static int apm_probe(gpr_device_t *gdev)
idr_init(&apm->modules_idr);
+ dev_set_drvdata(dev, apm);
g_apm = apm;
q6apm_get_apm_state(apm);
@@ -942,6 +941,9 @@ static int apm_callback(const struct gpr_resp_pkt *data, void *priv, int op)
const struct gpr_hdr *hdr = &data->hdr;
int graph_id, is_pos_buf;
+ if (!apm)
+ return 0;
+
result = data->payload;
switch (hdr->opcode) {
diff --git a/sound/soc/qcom/qdsp6/q6prm.c b/sound/soc/qcom/qdsp6/q6prm.c
index 1f3ce4cc0837..bcc73127d54d 100644
--- a/sound/soc/qcom/qdsp6/q6prm.c
+++ b/sound/soc/qcom/qdsp6/q6prm.c
@@ -192,6 +192,9 @@ static int prm_callback(const struct gpr_resp_pkt *data, void *priv, int op)
const struct gpr_ibasic_rsp_result_t *result;
const struct gpr_hdr *hdr = &data->hdr;
+ if (!prm)
+ return 0;
+
switch (hdr->opcode) {
case PRM_CMD_RSP_REQUEST_HW_RSC:
case PRM_CMD_RSP_RELEASE_HW_RSC:
diff --git a/sound/soc/sof/intel/hda-codec.c b/sound/soc/sof/intel/hda-codec.c
index fd371850b0d6..32fcc1cd3f92 100644
--- a/sound/soc/sof/intel/hda-codec.c
+++ b/sound/soc/sof/intel/hda-codec.c
@@ -15,8 +15,6 @@
#include "hda.h"
#if IS_ENABLED(CONFIG_SND_SOC_SOF_HDA_AUDIO_CODEC)
-#include "../../codecs/hdac_hda.h"
-
#define CODEC_PROBE_RETRIES 3
#define IDISP_VID_INTEL 0x80860000
@@ -148,7 +146,6 @@ static struct hda_codec *hda_codec_device_init(struct hdac_bus *bus, int addr, i
/* probe individual codec */
static int hda_codec_probe(struct snd_sof_dev *sdev, int address)
{
- struct hdac_hda_priv *hda_priv;
struct hda_bus *hbus = sof_to_hbus(sdev);
struct hda_codec *codec;
u32 hda_cmd = (address << 28) | (AC_NODE_ROOT << 20) |
@@ -168,27 +165,15 @@ static int hda_codec_probe(struct snd_sof_dev *sdev, int address)
dev_dbg(sdev->dev, "HDA codec #%d probed OK: response: %x\n",
address, resp);
- hda_priv = devm_kzalloc(sdev->dev, sizeof(*hda_priv), GFP_KERNEL);
- if (!hda_priv)
- return -ENOMEM;
-
codec = hda_codec_device_init(&hbus->core, address, HDA_DEV_LEGACY);
ret = PTR_ERR_OR_ZERO(codec);
if (ret < 0)
return ret;
- hda_priv->codec = codec;
- hda_priv->dev_index = address;
- dev_set_drvdata(&codec->core.dev, hda_priv);
-
- if ((resp & 0xFFFF0000) == IDISP_VID_INTEL) {
- if (!hbus->core.audio_component) {
- dev_dbg(sdev->dev,
- "iDisp hw present but no driver\n");
- ret = -ENOENT;
- goto out;
- }
- hda_priv->need_display_power = true;
+ if ((resp & 0xFFFF0000) == IDISP_VID_INTEL && !hbus->core.audio_component) {
+ dev_dbg(sdev->dev, "iDisp hw present but no driver\n");
+ ret = -ENOENT;
+ goto out;
}
if (is_generic_config(hbus))
diff --git a/sound/soc/sof/intel/hda-sdw-bpt.c b/sound/soc/sof/intel/hda-sdw-bpt.c
index ba9fbd662b85..9d97d8676251 100644
--- a/sound/soc/sof/intel/hda-sdw-bpt.c
+++ b/sound/soc/sof/intel/hda-sdw-bpt.c
@@ -456,6 +456,12 @@ int hda_sdw_bpt_close(struct device *dev, int link_id, struct hdac_ext_stream *b
int ret;
int ret1;
+ if (bpt_rx_stream && hdac_stream(bpt_rx_stream)->running)
+ hda_sdw_bpt_dma_disable(dev, bpt_rx_stream);
+
+ if (bpt_tx_stream && hdac_stream(bpt_tx_stream)->running)
+ hda_sdw_bpt_dma_disable(dev, bpt_tx_stream);
+
/*
* In the case of SoundWire we need to reset the PCMSyCM registers.
* Need to continue depreparing the DMA buffers even if this fails.
diff --git a/sound/usb/bcd2000/bcd2000.c b/sound/usb/bcd2000/bcd2000.c
index 2bd49bf82748..0fd086cac955 100644
--- a/sound/usb/bcd2000/bcd2000.c
+++ b/sound/usb/bcd2000/bcd2000.c
@@ -183,10 +183,24 @@ static int bcd2000_midi_output_open(struct snd_rawmidi_substream *substream)
static int bcd2000_midi_output_close(struct snd_rawmidi_substream *substream)
{
struct bcd2000 *bcd2k = substream->rmidi->private_data;
+ struct urb *urb = NULL;
- if (bcd2k->midi_out_active && bcd2k->midi_out_urb) {
- usb_kill_urb(bcd2k->midi_out_urb);
- bcd2k->midi_out_active = 0;
+ /*
+ * Reference the URB under the lock so that it stays valid until
+ * usb_kill_urb() returns, even if a disconnect frees it in the
+ * meantime. usb_kill_urb() cannot run under the lock, as the
+ * completion handler takes the same lock.
+ */
+ scoped_guard(spinlock_irqsave, &bcd2k->midi_lock) {
+ if (bcd2k->midi_out_active && bcd2k->midi_out_urb) {
+ urb = usb_get_urb(bcd2k->midi_out_urb);
+ bcd2k->midi_out_active = 0;
+ }
+ }
+
+ if (urb) {
+ usb_kill_urb(urb);
+ usb_put_urb(urb);
}
return 0;
diff --git a/sound/usb/caiaq/input.c b/sound/usb/caiaq/input.c
index 8d924330c54c..309a361ca9f7 100644
--- a/sound/usb/caiaq/input.c
+++ b/sound/usb/caiaq/input.c
@@ -840,6 +840,8 @@ int snd_usb_caiaq_input_init(struct snd_usb_caiaqdev *cdev)
return 0;
exit_free_idev:
+ usb_free_urb(cdev->ep4_in_urb);
+ cdev->ep4_in_urb = NULL;
input_free_device(input);
cdev->input_dev = NULL;
return ret;
diff --git a/sound/usb/line6/capture.c b/sound/usb/line6/capture.c
index ceee51eeda0d..32922b7f2ffc 100644
--- a/sound/usb/line6/capture.c
+++ b/sound/usb/line6/capture.c
@@ -94,6 +94,18 @@ void line6_capture_copy(struct snd_line6_pcm *line6pcm, char *fbuf, int fsize)
if (runtime == NULL)
return;
+ /*
+ * snd_line6_capture_open() constrains the buffer to hold at least
+ * two packets, so this cannot happen for a conforming device. Drop
+ * the packet rather than overrun the single wrap below.
+ */
+ if (unlikely(frames > runtime->buffer_size)) {
+ dev_err_ratelimited(line6pcm->line6->ifcdev,
+ "packet of %d frames exceeds %lu-frame buffer, dropping\n",
+ frames, runtime->buffer_size);
+ return;
+ }
+
if (line6pcm->in.pos_done + frames > runtime->buffer_size) {
/*
The transferred area goes over buffer boundary,
@@ -231,6 +243,23 @@ static int snd_line6_capture_open(struct snd_pcm_substream *substream)
if (err < 0)
return err;
+ /*
+ * line6_capture_copy() wraps the ring buffer once per packet and
+ * advances pos_done by the packet, which only works when the buffer
+ * holds more than one packet: with exactly one, pos_done gains a
+ * buffer and loses it again on every packet. The packet size comes
+ * from the device, so bound the buffer by two packets here. The
+ * impulse-response path copies a playback URB into this ring, so
+ * take the larger of the two endpoints.
+ */
+ err = snd_pcm_hw_constraint_minmax(runtime,
+ SNDRV_PCM_HW_PARAM_BUFFER_BYTES,
+ 2 * max(line6pcm->max_packet_size_in,
+ line6pcm->max_packet_size_out),
+ UINT_MAX);
+ if (err < 0)
+ return err;
+
line6_pcm_acquire(line6pcm, LINE6_STREAM_CAPTURE_HELPER, false);
runtime->hw = line6pcm->properties->capture_hw;
diff --git a/sound/usb/midi.c b/sound/usb/midi.c
index 7e5b1b13360f..1099e5b81411 100644
--- a/sound/usb/midi.c
+++ b/sound/usb/midi.c
@@ -113,6 +113,7 @@ struct snd_usb_midi {
unsigned int opened[2];
unsigned char disconnected;
unsigned char input_running;
+ unsigned char keep_input_running;
struct snd_kcontrol *roland_load_ctl;
};
@@ -1149,41 +1150,42 @@ static int substream_open(struct snd_rawmidi_substream *substream, int dir,
int open)
{
struct snd_usb_midi *umidi = substream->rmidi->private_data;
- struct snd_kcontrol *ctl;
+ struct snd_ctl_elem_id ctl_id;
+ bool activate_ctl = false;
+ bool active;
guard(rwsem_read)(&umidi->disc_rwsem);
if (umidi->disconnected)
return open ? -ENODEV : 0;
- guard(mutex)(&umidi->mutex);
- if (open) {
- if (!umidi->opened[0] && !umidi->opened[1]) {
- if (umidi->roland_load_ctl) {
- ctl = umidi->roland_load_ctl;
- ctl->vd[0].access |=
- SNDRV_CTL_ELEM_ACCESS_INACTIVE;
- snd_ctl_notify(umidi->card,
- SNDRV_CTL_EVENT_MASK_INFO, &ctl->id);
- update_roland_altsetting(umidi);
+ scoped_guard(mutex, &umidi->mutex) {
+ if (open) {
+ if (!umidi->opened[0] && !umidi->opened[1]) {
+ if (umidi->roland_load_ctl) {
+ ctl_id = umidi->roland_load_ctl->id;
+ activate_ctl = true;
+ active = false;
+ update_roland_altsetting(umidi);
+ }
}
- }
- umidi->opened[dir]++;
- if (umidi->opened[1])
- snd_usbmidi_input_start(&umidi->list);
- } else {
- umidi->opened[dir]--;
- if (!umidi->opened[1])
- snd_usbmidi_input_stop(&umidi->list);
- if (!umidi->opened[0] && !umidi->opened[1]) {
- if (umidi->roland_load_ctl) {
- ctl = umidi->roland_load_ctl;
- ctl->vd[0].access &=
- ~SNDRV_CTL_ELEM_ACCESS_INACTIVE;
- snd_ctl_notify(umidi->card,
- SNDRV_CTL_EVENT_MASK_INFO, &ctl->id);
+ umidi->opened[dir]++;
+ if (umidi->opened[1])
+ snd_usbmidi_input_start(&umidi->list);
+ } else {
+ umidi->opened[dir]--;
+ if (!umidi->opened[1] && !umidi->keep_input_running)
+ snd_usbmidi_input_stop(&umidi->list);
+ if (!umidi->opened[0] && !umidi->opened[1]) {
+ if (umidi->roland_load_ctl) {
+ ctl_id = umidi->roland_load_ctl->id;
+ activate_ctl = true;
+ active = true;
+ }
}
}
}
+ if (activate_ctl)
+ snd_ctl_activate_id(umidi->card, &ctl_id, active);
return 0;
}
@@ -2262,6 +2264,10 @@ static int snd_usbmidi_detect_roland(struct snd_usb_midi *umidi,
struct usb_host_interface *hostif;
u8 *cs_desc;
+ /* TD-11 becomes unusable if input isn't serviced continuously */
+ if (umidi->usb_id == USB_ID(0x0582, 0x0151))
+ umidi->keep_input_running = 1;
+
intf = umidi->iface;
if (!intf)
return -ENOENT;
@@ -2469,7 +2475,7 @@ void snd_usbmidi_input_start(struct list_head *p)
int i;
umidi = list_entry(p, struct snd_usb_midi, list);
- if (umidi->input_running || !umidi->opened[1])
+ if (umidi->input_running || (!umidi->opened[1] && !umidi->keep_input_running))
return;
for (i = 0; i < MIDI_MAX_ENDPOINTS; ++i)
snd_usbmidi_input_start_ep(umidi, umidi->endpoints[i].in);
@@ -2644,6 +2650,10 @@ int __snd_usbmidi_create(struct snd_card *card,
if (err < 0)
goto exit;
+ /* start input for devices that need continuous servicing */
+ if (umidi->keep_input_running)
+ snd_usbmidi_input_start(&umidi->list);
+
usb_autopm_get_interface_no_resume(umidi->iface);
list_add_tail(&umidi->list, midi_list);
diff --git a/sound/usb/qcom/qc_audio_offload.c b/sound/usb/qcom/qc_audio_offload.c
index e4bfd43a2488..5ee568e0454c 100644
--- a/sound/usb/qcom/qc_audio_offload.c
+++ b/sound/usb/qcom/qc_audio_offload.c
@@ -1969,8 +1969,13 @@ static int qc_usb_audio_probe(struct auxiliary_device *auxdev,
QMI_UAUDIO_STREAM_REQ_MSG_V01_MAX_MSG_LEN,
&uaudio_svc_ops_options,
&uaudio_stream_req_handlers);
+ if (ret)
+ goto free_qmi_hdl;
+
ret = qmi_add_server(svc->uaudio_svc_hdl, UAUDIO_STREAM_SERVICE_ID_V01,
UAUDIO_STREAM_SERVICE_VERS_V01, 0);
+ if (ret)
+ goto release_qmi_hdl;
uaudio_svc = svc;
@@ -1987,7 +1992,9 @@ static int qc_usb_audio_probe(struct auxiliary_device *auxdev,
release_qmi:
qc_usb_audio_cleanup_qmi_dev();
+release_qmi_hdl:
qmi_handle_release(svc->uaudio_svc_hdl);
+free_qmi_hdl:
kfree(svc->uaudio_svc_hdl);
free_svc:
kfree(svc);
diff --git a/sound/usb/usx2y/us122l.c b/sound/usb/usx2y/us122l.c
index a5db0d044ef9..9b4ec9bdc138 100644
--- a/sound/usb/usx2y/us122l.c
+++ b/sound/usb/usx2y/us122l.c
@@ -129,19 +129,26 @@ static const struct vm_operations_struct usb_stream_hwdep_vm_ops = {
static int usb_stream_hwdep_open(struct snd_hwdep *hw, struct file *file)
{
struct us122l *us122l = hw->private_data;
- struct usb_interface *iface;
+ struct usb_interface *iface, *iface0 = NULL;
if (hw->used >= 2)
return -EBUSY;
- if (!us122l->first)
- us122l->first = file;
-
if (us122l->is_us144) {
- iface = usb_ifnum_to_if(us122l->dev, 0);
- usb_autopm_get_interface(iface);
+ iface0 = usb_ifnum_to_if(us122l->dev, 0);
+ if (!iface0)
+ return -ENODEV;
}
+
iface = usb_ifnum_to_if(us122l->dev, 1);
+ if (!iface)
+ return -ENODEV;
+
+ if (!us122l->first)
+ us122l->first = file;
+
+ if (iface0)
+ usb_autopm_get_interface(iface0);
usb_autopm_get_interface(iface);
return 0;
}
@@ -153,10 +160,12 @@ static int usb_stream_hwdep_release(struct snd_hwdep *hw, struct file *file)
if (us122l->is_us144) {
iface = usb_ifnum_to_if(us122l->dev, 0);
- usb_autopm_put_interface(iface);
+ if (iface)
+ usb_autopm_put_interface(iface);
}
iface = usb_ifnum_to_if(us122l->dev, 1);
- usb_autopm_put_interface(iface);
+ if (iface)
+ usb_autopm_put_interface(iface);
if (us122l->first == file)
us122l->first = NULL;
guard(mutex)(&us122l->mutex);
@@ -179,6 +188,8 @@ static int usb_stream_hwdep_mmap(struct snd_hwdep *hw,
offset = area->vm_pgoff << PAGE_SHIFT;
guard(mutex)(&us122l->mutex);
s = us122l->sk.s;
+ if (!s)
+ return -ENODEV;
read = offset < s->read_size;
if (read) {
if (area->vm_flags & VM_WRITE)
diff --git a/sound/usb/usx2y/usX2Yhwdep.c b/sound/usb/usx2y/usX2Yhwdep.c
index 7f153a2fcdd8..3800c5b5887c 100644
--- a/sound/usb/usx2y/usX2Yhwdep.c
+++ b/sound/usb/usx2y/usX2Yhwdep.c
@@ -179,6 +179,9 @@ static int snd_usx2y_hwdep_dsp_load(struct snd_hwdep *hw,
int lret, err;
char *buf;
+ if (priv->chip_status & USX2Y_STAT_CHIP_HUP)
+ return -ENODEV;
+
buf = memdup_user(dsp->image, dsp->length);
if (IS_ERR(buf))
return PTR_ERR(buf);
@@ -201,11 +204,13 @@ static int snd_usx2y_hwdep_dsp_load(struct snd_hwdep *hw,
err = usx2y_in04_init(priv);
if (err) {
dev_err(&dev->dev, "usx2y_in04_init error\n");
+ priv->chip_status |= USX2Y_STAT_CHIP_HUP;
return err;
}
err = usx2y_create_alsa_devices(hw->card);
if (err) {
dev_err(&dev->dev, "usx2y_create_alsa_devices error %i\n", err);
+ priv->chip_status |= USX2Y_STAT_CHIP_HUP;
return err;
}
priv->chip_status |= USX2Y_STAT_CHIP_INIT;
diff --git a/sound/usb/usx2y/usb_stream.c b/sound/usb/usx2y/usb_stream.c
index 3122cf653273..7d621280fba3 100644
--- a/sound/usb/usx2y/usb_stream.c
+++ b/sound/usb/usx2y/usb_stream.c
@@ -682,6 +682,10 @@ int usb_stream_start(struct usb_stream_kernel *sk)
s->sync_packet = -1;
s->inpacket_head = -1;
sk->iso_frame_balance = 0;
+ sk->idle_inurb = sk->inurb[USB_STREAM_NURBS - 2];
+ sk->idle_outurb = sk->outurb[USB_STREAM_NURBS - 2];
+ sk->completed_inurb = sk->inurb[USB_STREAM_NURBS - 1];
+ sk->completed_outurb = sk->outurb[USB_STREAM_NURBS - 1];
++try;
for (u = 0; u < 2; u++) {
struct urb *inurb = sk->inurb[u];
@@ -739,11 +743,6 @@ int usb_stream_start(struct usb_stream_kernel *sk)
return -EFAULT;
}
- sk->idle_inurb = sk->inurb[USB_STREAM_NURBS - 2];
- sk->idle_outurb = sk->outurb[USB_STREAM_NURBS - 2];
- sk->completed_inurb = sk->inurb[USB_STREAM_NURBS - 1];
- sk->completed_outurb = sk->outurb[USB_STREAM_NURBS - 1];
-
/* wait, check */
{
int wait_ms = 3000;
diff --git a/sound/usb/usx2y/usbusx2yaudio.c b/sound/usb/usx2y/usbusx2yaudio.c
index a3a0bc13632b..798c9bb5386a 100644
--- a/sound/usb/usx2y/usbusx2yaudio.c
+++ b/sound/usb/usx2y/usbusx2yaudio.c
@@ -51,6 +51,18 @@ static int usx2y_urb_capt_retire(struct snd_usx2y_substream *subs)
dev_dbg(&usx2y->dev->dev, "%s: 0 == len ERROR!\n", __func__);
continue;
}
+ /*
+ * snd_usx2y_pcm_open() constrains the buffer to hold at least
+ * two URBs, so this cannot happen for a device that keeps its
+ * packets at 50 frames. Drop the packet rather than exceed the
+ * single wrap below.
+ */
+ if (unlikely(len > runtime->buffer_size)) {
+ dev_err_ratelimited(&usx2y->dev->dev,
+ "%s: packet of %i frames exceeds %lu-frame buffer, dropping\n",
+ __func__, len, runtime->buffer_size);
+ continue;
+ }
/* copy a data chunk */
if ((hwptr_done + len) > runtime->buffer_size) {
@@ -106,14 +118,31 @@ static int usx2y_urb_play_prepare(struct snd_usx2y_substream *subs,
__func__, counts);
return -EPIPE;
}
+ if (counts * usx2y->stride > subs->maxpacksize) {
+ dev_err_ratelimited(&usx2y->dev->dev,
+ "%s: packet of %i bytes exceeds maxpacksize %u\n",
+ __func__, counts * usx2y->stride,
+ subs->maxpacksize);
+ return -EPIPE;
+ }
/* set up descriptor */
urb->iso_frame_desc[pack].offset = pack ?
urb->iso_frame_desc[pack - 1].offset +
urb->iso_frame_desc[pack - 1].length :
0;
- urb->iso_frame_desc[pack].length = cap_urb->iso_frame_desc[pack].actual_length;
+ urb->iso_frame_desc[pack].length = counts * usx2y->stride;
}
if (atomic_read(&subs->state) >= STATE_PRERUNNING) {
+ /*
+ * The copy below wraps once, which needs count <= buffer_size.
+ * snd_usx2y_pcm_open() guarantees it; guard anyway.
+ */
+ if (unlikely(count > runtime->buffer_size)) {
+ dev_err_ratelimited(&usx2y->dev->dev,
+ "%s: URB of %i frames exceeds %lu-frame buffer, stopping\n",
+ __func__, count, runtime->buffer_size);
+ return -EPIPE;
+ }
if (subs->hwptr + count > runtime->buffer_size) {
/* err, the transferred area goes over buffer boundary.
* copy the data to the temp buffer.
@@ -888,6 +917,18 @@ static int snd_usx2y_pcm_open(struct snd_pcm_substream *substream)
runtime->private_data = subs;
subs->pcm_substream = substream;
snd_pcm_hw_constraint_minmax(runtime, SNDRV_PCM_HW_PARAM_PERIOD_TIME, 1000, 200000);
+ /*
+ * Capture wraps the ring buffer once per packet and playback once
+ * per URB of nr_of_packs() packets of at most 50 frames (see
+ * usx2y_urb_play_prepare()). Playback needs the buffer to hold more
+ * than one URB and capture more than one packet; use the URB bound
+ * for both, and two URBs rather than one, because the position only
+ * advances when a URB is less than the whole buffer. The bound is in
+ * frames and does not depend on the altsetting, which is selected
+ * in prepare.
+ */
+ snd_pcm_hw_constraint_minmax(runtime, SNDRV_PCM_HW_PARAM_BUFFER_SIZE,
+ 2 * nr_of_packs() * 50, UINT_MAX);
return 0;
}
diff --git a/tools/perf/tests/tool_pmu.c b/tools/perf/tests/tool_pmu.c
index c6c5ebf0e935..667160f9e2cd 100644
--- a/tools/perf/tests/tool_pmu.c
+++ b/tools/perf/tests/tool_pmu.c
@@ -1,8 +1,16 @@
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+#include <errno.h>
+#include <fcntl.h>
+#include <string.h>
+#include <unistd.h>
+#include <internal/xyarray.h>
+#include "cpumap.h"
#include "debug.h"
#include "evlist.h"
+#include "evsel.h"
#include "parse-events.h"
#include "tests.h"
+#include "thread_map.h"
#include "tool_pmu.h"
static int do_test(enum tool_pmu_event ev, bool with_pmu)
@@ -100,9 +108,80 @@ static int test__tool_pmu_with_pmu(struct test_suite *test __maybe_unused,
return TEST_OK;
}
+static int test__tool_pmu_incremental_open_unwind(struct test_suite *test __maybe_unused,
+ int subtest __maybe_unused)
+{
+ struct evlist *evlist = evlist__new();
+ struct parse_events_error err;
+ struct perf_cpu_map *cpus = NULL;
+ struct perf_thread_map *threads = NULL;
+ struct evsel *evsel;
+ int ret = TEST_FAIL, fd0 = -1;
+
+ if (!evlist)
+ return TEST_FAIL;
+
+ parse_events_error__init(&err);
+ if (parse_events(evlist, "tool/user_time/", &err)) {
+ parse_events_error__exit(&err);
+ evlist__put(evlist);
+ return TEST_FAIL;
+ }
+ parse_events_error__exit(&err);
+
+ evsel = evlist__first(evlist);
+ cpus = perf_cpu_map__new("0,1");
+ threads = thread_map__new_by_tid(getpid());
+ if (!cpus || !threads)
+ goto out;
+
+ /* Step 1: Open CPU index 0 successfully */
+ if (evsel__open_per_cpu_and_thread(evsel, cpus, 0, threads) < 0) {
+ pr_debug("Failed to open CPU index 0\n");
+ goto out;
+ }
+
+ fd0 = (*(int *)xyarray__entry(evsel->core.fd, 0, 0));
+ if (fd0 < 0 || fcntl(fd0, F_GETFD) < 0) {
+ pr_debug("CPU index 0 FD is invalid (%d)\n", fd0);
+ goto out;
+ }
+
+ /* Step 2: Intentionally trigger failure on CPU index 1 */
+ evsel->core.attr.sample_period = 1; /* Not supported for tool PMU -> -EINVAL */
+ if (evsel__open_per_cpu_and_thread(evsel, cpus, 1, threads) >= 0) {
+ pr_debug("Unexpected success opening CPU index 1 with sample_period=1\n");
+ goto out;
+ }
+
+ /*
+ * Step 3: Check that CPU index 0's FD was NOT destroyed by CPU index 1's unwind.
+ */
+ if ((*(int *)xyarray__entry(evsel->core.fd, 0, 0)) != fd0) {
+ pr_debug("FAILED: CPU 0 FD overwritten: FD(evsel, 0, 0)=%d, expected %d\n",
+ (*(int *)xyarray__entry(evsel->core.fd, 0, 0)), fd0);
+ goto out;
+ }
+
+ if (fcntl(fd0, F_GETFD) < 0) {
+ pr_debug("FAILED: CPU 0 FD %d was closed by error unwind! errno=%d (%s)\n",
+ fd0, errno, strerror(errno));
+ goto out;
+ }
+
+ ret = TEST_OK;
+out:
+ evsel__close(evsel);
+ perf_cpu_map__put(cpus);
+ perf_thread_map__put(threads);
+ evlist__put(evlist);
+ return ret;
+}
+
static struct test_case tests__tool_pmu[] = {
TEST_CASE("Parsing without PMU name", tool_pmu_without_pmu),
TEST_CASE("Parsing with PMU name", tool_pmu_with_pmu),
+ TEST_CASE("Incremental open error unwind boundary", tool_pmu_incremental_open_unwind),
{ .name = NULL, }
};
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index d4cb455f4a7d..5db43520f25e 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -2974,6 +2974,19 @@ static bool evsel__detect_missing_features(struct evsel *evsel, struct perf_cpu
return false;
}
+static void evsel__close_fds(struct evsel *evsel, int start_cpu_map_idx,
+ int cpu_map_idx, int thread, int nthreads)
+{
+ do {
+ while (--thread >= 0) {
+ if (FD(evsel, cpu_map_idx, thread) >= 0)
+ close(FD(evsel, cpu_map_idx, thread));
+ FD(evsel, cpu_map_idx, thread) = -1;
+ }
+ thread = nthreads;
+ } while (--cpu_map_idx >= start_cpu_map_idx);
+}
+
static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,
struct perf_thread_map *threads,
int start_cpu_map_idx, int end_cpu_map_idx)
@@ -3109,8 +3122,14 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,
if (err == -EMFILE && rlimit__increase_nofile(&set_rlimit))
goto retry_open;
- if (err == -EINVAL && evsel__detect_missing_features(evsel, cpu))
+ if (err == -EINVAL && evsel__detect_missing_features(evsel, cpu)) {
+ /*
+ * The restart opens the events again, so close what this
+ * attempt opened before its entries are overwritten.
+ */
+ evsel__close_fds(evsel, start_cpu_map_idx, idx, thread, nthreads);
goto fallback_missing_features;
+ }
if (evsel__precise_ip_fallback(evsel))
goto retry_open;
@@ -3120,14 +3139,7 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,
threads->err_thread = thread;
old_errno = errno;
- do {
- while (--thread >= 0) {
- if (FD(evsel, idx, thread) >= 0)
- close(FD(evsel, idx, thread));
- FD(evsel, idx, thread) = -1;
- }
- thread = nthreads;
- } while (--idx >= 0);
+ evsel__close_fds(evsel, start_cpu_map_idx, idx, thread, nthreads);
errno = old_errno;
out:
if (err)
diff --git a/tools/perf/util/hwmon_pmu.c b/tools/perf/util/hwmon_pmu.c
index ed544dca70c3..96aeb379e169 100644
--- a/tools/perf/util/hwmon_pmu.c
+++ b/tools/perf/util/hwmon_pmu.c
@@ -803,7 +803,7 @@ int evsel__hwmon_pmu_open(struct evsel *evsel,
FD(evsel, idx, thread) = -1;
}
thread = nthreads;
- } while (--idx >= 0);
+ } while (--idx >= start_cpu_map_idx);
close(dir);
return err;
}
diff --git a/tools/perf/util/tool_pmu.c b/tools/perf/util/tool_pmu.c
index 5c30854b4644..d2f7591747d9 100644
--- a/tools/perf/util/tool_pmu.c
+++ b/tools/perf/util/tool_pmu.c
@@ -349,7 +349,7 @@ int evsel__tool_pmu_open(struct evsel *evsel,
FD(evsel, idx, thread) = -1;
}
thread = nthreads;
- } while (--idx >= 0);
+ } while (--idx >= start_cpu_map_idx);
errno = old_errno;
return err;
}
diff --git a/tools/testing/selftests/filesystems/fscontext_ns/fscontext_ns_test.c b/tools/testing/selftests/filesystems/fscontext_ns/fscontext_ns_test.c
index 0f30a3e6e197..535bf1f4e1a3 100644
--- a/tools/testing/selftests/filesystems/fscontext_ns/fscontext_ns_test.c
+++ b/tools/testing/selftests/filesystems/fscontext_ns/fscontext_ns_test.c
@@ -193,7 +193,7 @@ FIXTURE_VARIANT_ADD(fscontext_ns, binfmt_misc) {
FIXTURE_VARIANT_ADD(fscontext_ns, overlay) {
.fsname = "overlay",
- .expected_errno = EIO,
+ .expected_errno = EINVAL,
};
TEST_F(fscontext_ns, create_from_descendant_userns)
diff --git a/tools/testing/selftests/net/fib_nexthops.sh b/tools/testing/selftests/net/fib_nexthops.sh
index 431d7bed7622..adbec51917eb 100755
--- a/tools/testing/selftests/net/fib_nexthops.sh
+++ b/tools/testing/selftests/net/fib_nexthops.sh
@@ -425,6 +425,14 @@ stop_ip_monitor()
return $rc
}
+fdb_entry_aged()
+{
+ local mac=$1
+ local dev=$2
+
+ ! $BRIDGE fdb get to $mac dev $dev self &> /dev/null
+}
+
check_nexthop_fdb_support()
{
$IP nexthop help 2>&1 | grep -q fdb
@@ -538,14 +546,18 @@ ipv6_fdb_grp_fcnal()
log_test $? 0 "Fdb mac add with nexthop group and static state"
run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic"
- log_test $? 255 "Fdb mac add with nexthop group and dynamic state"
+ log_test $? 0 "Fdb mac add with nexthop group and dynamic state"
+
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self dynamic extern_learn"
+ log_test $? 0 "Fdb mac add with nexthop group and extern_learn"
- run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self"
- run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic"
- log_test $? 255 "Fdb mac replace with nexthop group and dynamic state"
+ run_cmd "$IP link set dev vx10 up type vxlan ageing 1"
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:18 dev vx10 dst 2001:db8:91::11 self dynamic"
+ slowwait 20 fdb_entry_aged 02:02:00:00:00:18 vx10
+ log_test $? 0 "Fdb mac without nexthop group aged out"
- run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic"
- log_test $? 255 "Fdb mac append with nexthop group and dynamic state"
+ run_cmd "$BRIDGE fdb get to 02:02:00:00:00:16 dev vx10 self"
+ log_test $? 0 "Fdb mac with nexthop group not aged out"
run_cmd "$IP -6 ro add 2001:db8:101::1/128 nhid 66"
log_test $? 2 "Route add with fdb nexthop"
@@ -688,14 +700,18 @@ ipv4_fdb_grp_fcnal()
log_test $? 0 "Fdb mac add with nexthop group and static state"
run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic"
- log_test $? 255 "Fdb mac add with nexthop group and dynamic state"
+ log_test $? 0 "Fdb mac add with nexthop group and dynamic state"
+
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self dynamic extern_learn"
+ log_test $? 0 "Fdb mac add with nexthop group and extern_learn"
- run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self"
- run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic"
- log_test $? 255 "Fdb mac replace with nexthop group and dynamic state"
+ run_cmd "$IP link set dev vx10 up type vxlan ageing 1"
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:18 dev vx10 dst 10.0.0.3 self dynamic"
+ slowwait 20 fdb_entry_aged 02:02:00:00:00:18 vx10
+ log_test $? 0 "Fdb mac without nexthop group aged out"
- run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic"
- log_test $? 255 "Fdb mac append with nexthop group and dynamic state"
+ run_cmd "$BRIDGE fdb get to 02:02:00:00:00:16 dev vx10 self"
+ log_test $? 0 "Fdb mac with nexthop group not aged out"
run_cmd "$IP ro add 172.16.0.0/22 nhid 16"
log_test $? 2 "Route add with fdb nexthop"
prev parent reply other threads:[~2026-11-17 10:56 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-17 10:45 Linux 7.3.5 Greg Kroah-Hartman
2026-11-17 10:45 ` Greg Kroah-Hartman [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2026111729-armory-overjoyed-70fd@gregkh \
--to=gregkh@linuxfoundation.org \
--cc=akpm@linux-foundation.org \
--cc=jslaby@suse.cz \
--cc=linux-kernel@vger.kernel.org \
--cc=lwn@lwn.net \
--cc=stable@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.