* Re: Linux 6.18.64
2026-11-17 10:45 Linux 6.18.64 Greg Kroah-Hartman
@ 2026-11-17 10:45 ` Greg Kroah-Hartman
0 siblings, 0 replies; 2+ messages in thread
From: Greg Kroah-Hartman @ 2026-11-17 10:45 UTC (permalink / raw)
To: linux-kernel, akpm, torvalds, stable; +Cc: lwn, jslaby, Greg Kroah-Hartman
diff --git a/Documentation/admin-guide/kernel-parameters.rst b/Documentation/admin-guide/kernel-parameters.rst
index 7bf8cc7df6b5..02a725536cc5 100644
--- a/Documentation/admin-guide/kernel-parameters.rst
+++ b/Documentation/admin-guide/kernel-parameters.rst
@@ -110,102 +110,7 @@ The parameters listed below are only valid if certain kernel build options
were enabled and if respective hardware is present. This list should be kept
in alphabetical order. The text in square brackets at the beginning
of each description states the restrictions within which a parameter
-is applicable::
-
- ACPI ACPI support is enabled.
- AGP AGP (Accelerated Graphics Port) is enabled.
- ALSA ALSA sound support is enabled.
- APIC APIC support is enabled.
- APM Advanced Power Management support is enabled.
- APPARMOR AppArmor support is enabled.
- ARM ARM architecture is enabled.
- ARM64 ARM64 architecture is enabled.
- AX25 Appropriate AX.25 support is enabled.
- CLK Common clock infrastructure is enabled.
- CMA Contiguous Memory Area support is enabled.
- DRM Direct Rendering Management support is enabled.
- DYNAMIC_DEBUG Build in debug messages and enable them at runtime
- EARLY Parameter processed too early to be embedded in initrd.
- EDD BIOS Enhanced Disk Drive Services (EDD) is enabled
- EFI EFI Partitioning (GPT) is enabled
- EVM Extended Verification Module
- FB The frame buffer device is enabled.
- FTRACE Function tracing enabled.
- GCOV GCOV profiling is enabled.
- HIBERNATION HIBERNATION is enabled.
- HW Appropriate hardware is enabled.
- HYPER_V HYPERV support is enabled.
- IMA Integrity measurement architecture is enabled.
- IP_PNP IP DHCP, BOOTP, or RARP is enabled.
- IPV6 IPv6 support is enabled.
- ISAPNP ISA PnP code is enabled.
- ISDN Appropriate ISDN support is enabled.
- ISOL CPU Isolation is enabled.
- JOY Appropriate joystick support is enabled.
- KGDB Kernel debugger support is enabled.
- KVM Kernel Virtual Machine support is enabled.
- LIBATA Libata driver is enabled
- LOONGARCH LoongArch architecture is enabled.
- LOOP Loopback device support is enabled.
- LP Printer support is enabled.
- M68k M68k architecture is enabled.
- These options have more detailed description inside of
- Documentation/arch/m68k/kernel-options.rst.
- MDA MDA console support is enabled.
- MIPS MIPS architecture is enabled.
- MOUSE Appropriate mouse support is enabled.
- MSI Message Signaled Interrupts (PCI).
- MTD MTD (Memory Technology Device) support is enabled.
- NET Appropriate network support is enabled.
- NFS Appropriate NFS support is enabled.
- NUMA NUMA support is enabled.
- OF Devicetree is enabled.
- PARISC The PA-RISC architecture is enabled.
- PCI PCI bus support is enabled.
- PCIE PCI Express support is enabled.
- PCMCIA The PCMCIA subsystem is enabled.
- PNP Plug & Play support is enabled.
- PPC PowerPC architecture is enabled.
- PPT Parallel port support is enabled.
- PS2 Appropriate PS/2 support is enabled.
- PV_OPS A paravirtualized kernel is enabled.
- RAM RAM disk support is enabled.
- RDT Intel Resource Director Technology.
- RISCV RISCV architecture is enabled.
- S390 S390 architecture is enabled.
- SCSI Appropriate SCSI support is enabled.
- A lot of drivers have their options described inside
- the Documentation/scsi/ sub-directory.
- SDW SoundWire support is enabled.
- SECURITY Different security models are enabled.
- SELINUX SELinux support is enabled.
- SERIAL Serial support is enabled.
- SH SuperH architecture is enabled.
- SMP The kernel is an SMP kernel.
- SPARC Sparc architecture is enabled.
- SUSPEND System suspend states are enabled.
- SWSUSP Software suspend (hibernation) is enabled.
- TPM TPM drivers are enabled.
- UMS USB Mass Storage support is enabled.
- USB USB support is enabled.
- USBHID USB Human Interface Device support is enabled.
- V4L Video For Linux support is enabled.
- VGA The VGA console has been enabled.
- VMMIO Driver for memory mapped virtio devices is enabled.
- VT Virtual terminal support is enabled.
- WDT Watchdog support is enabled.
- X86-32 X86-32, aka i386 architecture is enabled.
- X86-64 X86-64 architecture is enabled.
- X86 Either 32-bit or 64-bit x86 (same as X86-32+X86-64)
- X86_UV SGI UV support is enabled.
- XEN Xen support is enabled
- XTENSA xtensa architecture is enabled.
-
-In addition, the following text indicates that the option::
-
- BOOT Is a boot loader parameter.
- BUGS= Relates to possible processor bugs on the said processor.
- KNL Is a kernel start-up parameter.
+is applicable.
Parameters denoted with BOOT are actually interpreted by the boot
loader, and have no meaning to the kernel directly.
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 362ff3225220..e9ffe9c877b1 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1,3 +1,101 @@
+ ACPI ACPI support is enabled.
+ AGP AGP (Accelerated Graphics Port) is enabled.
+ ALSA ALSA sound support is enabled.
+ APIC APIC support is enabled.
+ APM Advanced Power Management support is enabled.
+ APPARMOR AppArmor support is enabled.
+ ARM ARM architecture is enabled.
+ ARM64 ARM64 architecture is enabled.
+ AX25 Appropriate AX.25 support is enabled.
+ CLK Common clock infrastructure is enabled.
+ CMA Contiguous Memory Area support is enabled.
+ DRM Direct Rendering Management support is enabled.
+ DYNAMIC_DEBUG Build in debug messages and enable them at runtime
+ EARLY Parameter processed too early to be embedded in initrd.
+ EDD BIOS Enhanced Disk Drive Services (EDD) is enabled
+ EFI EFI Partitioning (GPT) is enabled
+ EVM Extended Verification Module
+ FB The frame buffer device is enabled.
+ FTRACE Function tracing enabled.
+ GCOV GCOV profiling is enabled.
+ HIBERNATION HIBERNATION is enabled.
+ HW Appropriate hardware is enabled.
+ HYPER_V HYPERV support is enabled.
+ IMA Integrity measurement architecture is enabled.
+ IP_PNP IP DHCP, BOOTP, or RARP is enabled.
+ IPV6 IPv6 support is enabled.
+ ISAPNP ISA PnP code is enabled.
+ ISDN Appropriate ISDN support is enabled.
+ ISOL CPU Isolation is enabled.
+ JOY Appropriate joystick support is enabled.
+ KGDB Kernel debugger support is enabled.
+ KVM Kernel Virtual Machine support is enabled.
+ LIBATA Libata driver is enabled
+ LOONGARCH LoongArch architecture is enabled.
+ LOOP Loopback device support is enabled.
+ LP Printer support is enabled.
+ M68k M68k architecture is enabled.
+ These options have more detailed description inside of
+ Documentation/arch/m68k/kernel-options.rst.
+ MDA MDA console support is enabled.
+ MIPS MIPS architecture is enabled.
+ MOUSE Appropriate mouse support is enabled.
+ MSI Message Signaled Interrupts (PCI).
+ MTD MTD (Memory Technology Device) support is enabled.
+ NET Appropriate network support is enabled.
+ NFS Appropriate NFS support is enabled.
+ NUMA NUMA support is enabled.
+ OF Devicetree is enabled.
+ PARISC The PA-RISC architecture is enabled.
+ PCI PCI bus support is enabled.
+ PCIE PCI Express support is enabled.
+ PCMCIA The PCMCIA subsystem is enabled.
+ PNP Plug & Play support is enabled.
+ PPC PowerPC architecture is enabled.
+ PPT Parallel port support is enabled.
+ PS2 Appropriate PS/2 support is enabled.
+ PV_OPS A paravirtualized kernel is enabled.
+ RAM RAM disk support is enabled.
+ RDT Intel Resource Director Technology.
+ RISCV RISCV architecture is enabled.
+ S390 S390 architecture is enabled.
+ SCSI Appropriate SCSI support is enabled.
+ A lot of drivers have their options described inside
+ the Documentation/scsi/ sub-directory.
+ SDW SoundWire support is enabled.
+ SECURITY Different security models are enabled.
+ SELINUX SELinux support is enabled.
+ SERIAL Serial support is enabled.
+ SH SuperH architecture is enabled.
+ SMP The kernel is an SMP kernel.
+ SPARC Sparc architecture is enabled.
+ SUSPEND System suspend states are enabled.
+ SWSUSP Software suspend (hibernation) is enabled.
+ TPM TPM drivers are enabled.
+ UMS USB Mass Storage support is enabled.
+ USB USB support is enabled.
+ USBHID USB Human Interface Device support is enabled.
+ V4L Video For Linux support is enabled.
+ VGA The VGA console has been enabled.
+ VMMIO Driver for memory mapped virtio devices is enabled.
+ VT Virtual terminal support is enabled.
+ WDT Watchdog support is enabled.
+ X86-32 X86-32, aka i386 architecture is enabled.
+ X86-64 X86-64 architecture is enabled.
+ X86 Either 32-bit or 64-bit x86 (same as X86-32+X86-64)
+ X86_UV SGI UV support is enabled.
+ XEN Xen support is enabled
+ XTENSA xtensa architecture is enabled.
+
+In addition, the following text indicates that the option
+
+ BOOT Is a boot loader parameter.
+ BUGS= Relates to possible processor bugs on the said processor.
+ KNL Is a kernel start-up parameter.
+
+
+Kernel parameters
+
accept_memory= [MM]
Format: { eager | lazy }
default: lazy
@@ -4553,6 +4651,17 @@
This can be set from sysctl after boot.
See Documentation/admin-guide/sysctl/vm.rst for details.
+
+ nvme_tcp.tls_handshake_timeout=
+ [NVME] NVMe/TCP host TLS handshake timeout in seconds.
+ Valid values range from 0 to the smaller of
+ UINT_MAX / 1000 and MAX_JIFFY_OFFSET / HZ. Default: 10
+
+ nvmet_tcp.tls_handshake_timeout=
+ [NVME] NVMe/TCP target TLS handshake timeout in seconds.
+ Valid values range from 0 to the smaller of
+ UINT_MAX / 1000 and MAX_JIFFY_OFFSET / HZ. Default: 10
+
ohci1394_dma=early [HW,EARLY] enable debugging via the ohci1394 driver.
See Documentation/core-api/debugging-via-ohci1394.rst for more
info.
diff --git a/Documentation/filesystems/fsverity.rst b/Documentation/filesystems/fsverity.rst
index 412cf11e3298..fd17bbf5795c 100644
--- a/Documentation/filesystems/fsverity.rst
+++ b/Documentation/filesystems/fsverity.rst
@@ -170,6 +170,7 @@ fatal signal), no changes are made to the file.
FS_IOC_ENABLE_VERITY can fail with the following errors:
- ``EACCES``: the process does not have write access to the file
+- ``EBADF``: the file descriptor is not open for reading
- ``EBADMSG``: the builtin signature is malformed
- ``EBUSY``: this ioctl is already running on the file
- ``EEXIST``: the file already has verity enabled
@@ -232,6 +233,7 @@ regardless of the size of the file.
FS_IOC_MEASURE_VERITY can fail with the following errors:
+- ``EBADF``: the file descriptor is not open for reading
- ``EFAULT``: the caller provided inaccessible memory
- ``ENODATA``: the file is not a verity file
- ``ENOTTY``: this type of filesystem does not implement fs-verity
@@ -310,6 +312,7 @@ blocks from disk without actually verifying the path to the root node.
FS_IOC_READ_VERITY_METADATA can fail with the following errors:
+- ``EBADF``: the file descriptor is not open for reading
- ``EFAULT``: the caller provided inaccessible memory
- ``EINTR``: the ioctl was interrupted before any data was read
- ``EINVAL``: reserved fields were set, or ``offset + length``
diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst
index c493fcbe0df4..2d6202807fd1 100644
--- a/Documentation/filesystems/overlayfs.rst
+++ b/Documentation/filesystems/overlayfs.rst
@@ -716,7 +716,8 @@ When decoding an overlay file handle, the following steps are followed:
1. Find underlying layer by UUID and path type information.
2. Decode the underlying filesystem file handle to underlying dentry.
- 3. For a lower file handle, lookup the handle in index directory by name.
+ 3. For a lower file handle, encode the decoded underlying inode again and
+ lookup that handle in index directory by name.
4. If a whiteout is found in index, return ESTALE. This represents an
overlay object that was deleted after its file handle was encoded.
5. For a non-directory, instantiate a disconnected overlay dentry from the
diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index b7f35b07876a..e48298f3babb 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -260,12 +260,12 @@ with the following files:
"mbm_assign_mode":
The supported counter assignment modes. The enclosed brackets indicate which mode
is enabled. The MBM events associated with counters may reset when "mbm_assign_mode"
- is changed.
+ is changed. See architecture specific notes below.
::
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
- [mbm_event]
- default
+ [default]
+ mbm_event
"mbm_event":
@@ -283,20 +283,17 @@ with the following files:
Moving to mbm_event counter assignment mode requires users to assign the counters
to the events. Otherwise, the MBM event counters will return 'Unassigned' when read.
- The mode is beneficial for AMD platforms that support more CTRL_MON
- and MON groups than available hardware counters. By default, this
- feature is enabled on AMD platforms with the ABMC (Assignable Bandwidth
- Monitoring Counters) capability, ensuring counters remain assigned even
- when the corresponding RMID is not actively used by any processor.
+ It is intended for deployments that need to manage counter assignment on
+ platforms where the number of monitoring groups exceeds the available
+ hardware counters. Hardware counters are scarce on these platforms, so
+ mbm_event mode suits workflows that monitor a subset of groups at a time and
+ rotate assignments as needed. The mbm_event mode ensures counters remain
+ assigned even when the corresponding RMID is not actively used.
"default":
In default mode, resctrl assumes there is a hardware counter for each
- event within every CTRL_MON and MON group. On AMD platforms, it is
- recommended to use the mbm_event mode, if supported, to prevent reset of MBM
- events between reads resulting from hardware re-allocating counters. This can
- result in misleading values or display "Unavailable" if no counter is assigned
- to the event.
+ event within every CTRL_MON and MON group.
* To enable "mbm_event" counter assignment mode:
::
@@ -308,6 +305,24 @@ with the following files:
# echo "default" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
+ In default mode on AMD platforms that support more CTRL_MON and MON groups
+ than hardware counters, hardware dynamically shares a smaller pool of
+ counters among RMIDs. One counter from this pool is used per monitoring
+ group and counts every MBM event of that group's RMID. The size of the pool
+ is not enumerated to software (unlike "num_mbm_cntrs"), and "num_rmids" may
+ be much larger. For example, 64 counters in this pool monitor 64 groups.
+ Newer hardware may provide a larger pool.
+
+ On such platforms, while the number of monitoring groups does not exceed
+ that pool, a counter stays attached to each RMID and readings remain
+ accurate. Creating more groups than the pool can cause hardware to
+ re-allocate those counters between successive reads of an event. Bandwidth
+ values may then be misleading, or a read may return "Unavailable" if no
+ counter is allocated to the RMID. There is no user-visible indication when
+ this begins. Users who need stable readings beyond that pool should switch
+ to mbm_event mode, if supported, and assign counters to the groups of
+ interest (rotating assignments as needed).
+
"num_mbm_cntrs":
The maximum number of counters (total of available and assigned counters) in
each domain when the system supports mbm_event mode.
@@ -1639,17 +1654,19 @@ View the llc occupancy snapshot::
Examples on working with mbm_assign_mode
========================================
-a. Check if MBM counter assignment mode is supported.
+a. Check if MBM counter assignment mode is supported. Enable "mbm_event"
+mode. Switching the mbm_assign_mode may reset all the MBM counters (and
+thus all MBM events) of all the resctrl groups.
::
- # mount -t resctrl resctrl /sys/fs/resctrl/
-
+ # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
+ [default]
+ mbm_event
+ # echo "mbm_event" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
[mbm_event]
default
-The "mbm_event" mode is detected and enabled.
-
b. Check how many assignable counters are supported.
::
@@ -1660,36 +1677,39 @@ c. Check how many assignable counters are available for assignment in each domai
::
# cat /sys/fs/resctrl/info/L3_MON/available_mbm_cntrs
- 0=30;1=30
+ 0=32;1=32
-d. To list the default group's assign states.
+d. To list the default group's assign states. Counters are unassigned when the mode
+is switched to "mbm_event".
::
# cat /sys/fs/resctrl/mbm_L3_assignments
- mbm_total_bytes:0=e;1=e
- mbm_local_bytes:0=e;1=e
+ mbm_total_bytes:0=_;1=_
+ mbm_local_bytes:0=_;1=_
-e. To unassign the counter associated with the mbm_total_bytes event on domain 0.
+e. To assign the counter associated with the mbm_total_bytes event on domain 0
+in exclusive mode.
::
- # echo "mbm_total_bytes:0=_" > /sys/fs/resctrl/mbm_L3_assignments
+ # echo "mbm_total_bytes:0=e" > /sys/fs/resctrl/mbm_L3_assignments
# cat /sys/fs/resctrl/mbm_L3_assignments
- mbm_total_bytes:0=_;1=e
- mbm_local_bytes:0=e;1=e
+ mbm_total_bytes:0=e;1=_
+ mbm_local_bytes:0=_;1=_
f. To unassign the counter associated with the mbm_total_bytes event on all domains.
::
# echo "mbm_total_bytes:*=_" > /sys/fs/resctrl/mbm_L3_assignments
- # cat /sys/fs/resctrl/mbm_L3_assignment
+ # cat /sys/fs/resctrl/mbm_L3_assignments
mbm_total_bytes:0=_;1=_
- mbm_local_bytes:0=e;1=e
+ mbm_local_bytes:0=_;1=_
-g. To assign a counter associated with the mbm_total_bytes event on all domains in
-exclusive mode.
+g. To assign a counter associated with the mbm_total_bytes and mbm_local_bytes
+events on all domains in exclusive mode.
::
# echo "mbm_total_bytes:*=e" > /sys/fs/resctrl/mbm_L3_assignments
+ # echo "mbm_local_bytes:*=e" > /sys/fs/resctrl/mbm_L3_assignments
# cat /sys/fs/resctrl/mbm_L3_assignments
mbm_total_bytes:0=e;1=e
mbm_local_bytes:0=e;1=e
@@ -1746,13 +1766,8 @@ the MBM counters (and thus all MBM events) of all the resctrl groups.
# echo "default" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
- mbm_event
[default]
-
-m. Unmount the resctrl filesystem.
-::
-
- # umount /sys/fs/resctrl/
+ mbm_event
Intel RDT Errata
================
diff --git a/Documentation/hwmon/coretemp.rst b/Documentation/hwmon/coretemp.rst
index c609329e3bc4..6089bee1159f 100644
--- a/Documentation/hwmon/coretemp.rst
+++ b/Documentation/hwmon/coretemp.rst
@@ -47,6 +47,9 @@ All Sysfs entries are named with their core_id (represented here by 'X').
================= ========================================================
tempX_input Core temperature (in millidegrees Celsius).
+ Reading fails with -ENODATA while the CPU reports the
+ temperature as invalid. Some CPUs do this when the
+ temperature is too low to be measured.
tempX_max All cooling devices should be turned on (on Core2).
tempX_crit Maximum junction temperature (in millidegrees Celsius).
tempX_crit_alarm Set when Out-of-spec bit is set, never clears.
diff --git a/MAINTAINERS b/MAINTAINERS
index f5ea8588244c..5821d97ae330 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9606,7 +9606,7 @@ S: Supported
F: Documentation/filesystems/nfs/exporting.rst
F: fs/exportfs/
F: fs/fhandle.c
-F: include/linux/exportfs.h
+F: include/linux/exportfs*.h
FILESYSTEMS [IDMAPPED MOUNTS]
M: Christian Brauner <brauner@kernel.org>
diff --git a/Makefile b/Makefile
index 124ecd981272..2a51de155f3e 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
VERSION = 6
PATCHLEVEL = 18
-SUBLEVEL = 63
+SUBLEVEL = 64
EXTRAVERSION =
NAME = Baby Opossum Posse
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 2e567a7d89c9..66b7284ce751 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -222,6 +222,7 @@ config ARM64
select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
select HAVE_BUILDTIME_MCOUNT_SORT
select HAVE_EFFICIENT_UNALIGNED_ACCESS
+ select HAVE_EXIT_THREAD
select HAVE_GUP_FAST
select HAVE_FTRACE_GRAPH_FUNC
select HAVE_FUNCTION_TRACER
diff --git a/arch/arm64/include/asm/sdei.h b/arch/arm64/include/asm/sdei.h
index 484cb6972e99..3b689185f371 100644
--- a/arch/arm64/include/asm/sdei.h
+++ b/arch/arm64/include/asm/sdei.h
@@ -44,7 +44,8 @@ asmlinkage unsigned long __sdei_handler(struct pt_regs *regs,
struct sdei_registered_event *arg);
unsigned long do_sdei_event(struct pt_regs *regs,
- struct sdei_registered_event *arg);
+ struct sdei_registered_event *arg,
+ unsigned long vbar);
unsigned long sdei_arch_get_entry_point(int conduit);
#define sdei_arch_get_entry_point(x) sdei_arch_get_entry_point(x)
diff --git a/arch/arm64/include/asm/suspend.h b/arch/arm64/include/asm/suspend.h
index eb60c9735553..2cc5fb405769 100644
--- a/arch/arm64/include/asm/suspend.h
+++ b/arch/arm64/include/asm/suspend.h
@@ -2,7 +2,7 @@
#ifndef __ASM_SUSPEND_H
#define __ASM_SUSPEND_H
-#define NR_CTX_REGS 14
+#define NR_CTX_REGS 15
#define NR_CALLEE_SAVED_REGS 12
/*
diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
index 5b97dfcf796d..4334bade249b 100644
--- a/arch/arm64/kernel/entry-common.c
+++ b/arch/arm64/kernel/entry-common.c
@@ -30,6 +30,7 @@
#include <asm/stacktrace.h>
#include <asm/sysreg.h>
#include <asm/system_misc.h>
+#include <asm/vectors.h>
/*
* Handle IRQ/context state management when entering from kernel mode.
@@ -993,6 +994,7 @@ asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs)
asmlinkage noinstr unsigned long
__sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
{
+ unsigned long elr, spsr, esr, far, vbar;
irqentry_state_t state;
unsigned long ret;
@@ -1018,10 +1020,46 @@ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
else if (cpu_has_pan())
set_pstate_pan(0);
+ /*
+ * Nothing saved the interrupted context's exception registers on the
+ * way in, and it may be in the middle of exception entry or return
+ * (SDEI ignores DAIF). Save them before anything can take an exception
+ * and overwrite them.
+ */
+ elr = read_sysreg(elr_el1);
+ spsr = read_sysreg(spsr_el1);
+ esr = read_sysreg(esr_el1);
+ far = read_sysreg(far_el1);
+
+ /*
+ * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to
+ * EL0 until the next exception from EL0 maps the kernel. They only
+ * handle exceptions from EL0: an exception taken here would run their
+ * empty current-EL half and never come back. Switch to the kernel
+ * vectors. The IRQ-like resume keeps using the interrupted VBAR, so it
+ * still goes through the trampoline.
+ */
+ vbar = read_sysreg(vbar_el1);
+ if (arm64_kernel_unmapped_at_el0() &&
+ vbar == (unsigned long)raw_cpu_read(this_cpu_vector)) {
+ write_sysreg((unsigned long)vectors, vbar_el1);
+ isb();
+ }
+
state = irqentry_nmi_enter(regs);
- ret = do_sdei_event(regs, arg);
+ ret = do_sdei_event(regs, arg, vbar);
irqentry_nmi_exit(regs, state);
+ /*
+ * Put them back unconditionally. An unchanged ELR does not rule out a
+ * nested exception: one may have returned to the same address.
+ */
+ write_sysreg(elr, elr_el1);
+ write_sysreg(spsr, spsr_el1);
+ write_sysreg(esr, esr_el1);
+ write_sysreg(far, far_el1);
+ write_sysreg(vbar, vbar_el1);
+
return ret;
}
#endif /* CONFIG_ARM_SDE_INTERFACE */
diff --git a/arch/arm64/kernel/machine_kexec_file.c b/arch/arm64/kernel/machine_kexec_file.c
index 410060ebd86d..2034ca8287a2 100644
--- a/arch/arm64/kernel/machine_kexec_file.c
+++ b/arch/arm64/kernel/machine_kexec_file.c
@@ -128,6 +128,8 @@ int load_other_segments(struct kimage *image,
vfree(headers);
goto out_err;
}
+
+ vfree(image->elf_headers);
image->elf_headers = headers;
image->elf_load_addr = kbuf.mem;
image->elf_headers_sz = headers_sz;
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index 4c328b7c79ba..b3cde5f60b00 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -319,9 +319,18 @@ static int copy_thread_gcs(struct task_struct *p,
return 0;
}
+static void gcs_reset_thread(struct task_struct *task)
+{
+ task->thread.gcs_base = 0;
+ task->thread.gcs_size = 0;
+ task->thread.gcspr_el0 = 0;
+}
+
#else
static void flush_gcs(void) { }
+static void gcs_reset_thread(struct task_struct *task) { }
+
static int copy_thread_gcs(struct task_struct *p,
const struct kernel_clone_args *args)
{
@@ -340,6 +349,12 @@ void flush_thread(void)
flush_gcs();
}
+void exit_thread(struct task_struct *tsk)
+{
+ /* Failed fork after copy_thread(); deactivate_mm() does not run. */
+ gcs_free(tsk);
+}
+
#ifdef CONFIG_ARM64_ERRATUM_4193714
static void arch_dup_tlbbatch_mask(struct task_struct *dst)
@@ -537,6 +552,8 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
p->thread.cpu_context.x19 = (unsigned long)args->fn;
p->thread.cpu_context.x20 = (unsigned long)args->fn_arg;
+ /* Kernel threads sharing the parent's mm must not free its GCS. */
+ gcs_reset_thread(p);
if (system_supports_poe())
p->thread.por_el0 = POR_EL0_INIT;
diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c
index 95169f7b6531..839b0723f5db 100644
--- a/arch/arm64/kernel/sdei.c
+++ b/arch/arm64/kernel/sdei.c
@@ -199,21 +199,49 @@ unsigned long sdei_arch_get_entry_point(int conduit)
return 0;
}
+/*
+ * Exceptions an SDEI handler may take without harm once __sdei_handler() has
+ * restored the interrupted context's exception registers. Their handlers are
+ * safe in NMI context:
+ * - EL1 data aborts on translation, permission and access-flag faults:
+ * faulthandler_disabled() is true in NMI context, so do_page_fault() never
+ * takes the mmap lock. This covers the _nofault() accessors used by the
+ * generic dump_stack()/show_regs() code (e.g. print_worker_info()).
+ * - BRK: kprobes, WARN/BUG, kCFI and UBSAN traps, all NMI-capable.
+ * Anything else taken at EL1 is fatal on its own, or unexpected here.
+ */
+static bool sdei_nested_exception_ok(unsigned long esr)
+{
+ switch (ESR_ELx_EC(esr)) {
+ case ESR_ELx_EC_DABT_CUR:
+ return esr_fsc_is_translation_fault(esr) ||
+ esr_fsc_is_permission_fault(esr) ||
+ esr_fsc_is_access_flag_fault(esr);
+ case ESR_ELx_EC_BRK64:
+ return true;
+ default:
+ return false;
+ }
+}
+
/*
* do_sdei_event() returns one of:
* SDEI_EV_HANDLED - success, return to the interrupted context.
* SDEI_EV_FAILED - failure, return this error code to firmare.
* virtual-address - success, return to this address.
+ *
+ * vbar is the interrupted context's VBAR_EL1, for the IRQ-like resume.
*/
unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
- struct sdei_registered_event *arg)
+ struct sdei_registered_event *arg,
+ unsigned long vbar)
{
u32 mode;
int i, err = 0;
int clobbered_registers = 4;
u64 elr = read_sysreg(elr_el1);
+ u64 nested_elr, nested_esr, nested_far;
u32 kernel_mode = read_sysreg(CurrentEL) | 1; /* +SPSel */
- unsigned long vbar = read_sysreg(vbar_el1);
if (arm64_kernel_unmapped_at_el0())
clobbered_registers++;
@@ -225,18 +253,27 @@ unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
}
err = sdei_event_handler(regs, arg);
- if (err)
- return SDEI_EV_FAILED;
- if (elr != read_sysreg(elr_el1)) {
- /*
- * We took a synchronous exception from the SDEI handler.
- * This could deadlock, and if you interrupt KVM it will
- * hyp-panic instead.
- */
- pr_warn("unsafe: exception during handler\n");
+ /*
+ * A synchronous exception taken by the handler has overwritten the
+ * exception registers; __sdei_handler() puts them back. Only the last
+ * one's syndrome survives; report it if it is not one known to be
+ * harmless.
+ *
+ * If the event interrupted a KVM world switch, VBAR points at the KVM
+ * hyp vectors and any such exception is a hyp panic instead.
+ */
+ nested_elr = read_sysreg(elr_el1);
+ nested_esr = read_sysreg(esr_el1);
+ nested_far = read_sysreg(far_el1);
+ if (nested_elr != elr && !sdei_nested_exception_ok(nested_esr)) {
+ pr_warn_once("unexpected exception during handler: ESR %#llx FAR %#llx near %pS\n",
+ nested_esr, nested_far, (void *)nested_elr);
}
+ if (err)
+ return SDEI_EV_FAILED;
+
mode = regs->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK);
/*
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index 202f677e2b0f..415891ffa62e 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -449,9 +449,7 @@ int counters_read_on_cpu(int cpu, smp_call_func_t func, u64 *val)
if (WARN_ON_ONCE(irqs_disabled()))
return -EPERM;
- smp_call_function_single(cpu, func, val, 1);
-
- return 0;
+ return smp_call_function_single(cpu, func, val, 1);
}
/*
@@ -478,10 +476,19 @@ bool cpc_ffh_supported(void)
return true;
}
+static bool cpc_ffh_reg_valid(const struct cpc_reg *reg)
+{
+ return reg->bit_width && reg->bit_width <= 64 &&
+ reg->bit_offset <= 64 - reg->bit_width;
+}
+
int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val)
{
int ret = -EOPNOTSUPP;
+ if (!cpc_ffh_reg_valid(reg))
+ return -EINVAL;
+
switch ((u64)reg->address) {
case 0x0:
ret = counters_read_on_cpu(cpu, cpu_read_corecnt, val);
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 3f675875abea..e3348cfe74db 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -502,17 +502,11 @@ static void timer_save_state(struct arch_timer_context *ctx)
goto out;
switch (index) {
- u64 cval;
-
case TIMER_VTIMER:
case TIMER_HVTIMER:
timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTV_CTL));
- cval = read_sysreg_el0(SYS_CNTV_CVAL);
-
- if (has_broken_cntvoff())
- cval -= timer_get_offset(ctx);
-
- timer_set_cval(ctx, cval);
+ if (!has_broken_cntvoff() || !timer_get_offset(ctx))
+ timer_set_cval(ctx, read_sysreg_el0(SYS_CNTV_CVAL));
/* Disable the timer */
write_sysreg_el0(0, SYS_CNTV_CTL);
@@ -538,11 +532,12 @@ static void timer_save_state(struct arch_timer_context *ctx)
case TIMER_PTIMER:
case TIMER_HPTIMER:
timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTP_CTL));
- cval = read_sysreg_el0(SYS_CNTP_CVAL);
-
- cval -= timer_get_offset(ctx);
-
- timer_set_cval(ctx, cval);
+ /*
+ * With an offset, memory already holds the guest's CVAL (the
+ * trap handler or __deactivate_traps() wrote it).
+ */
+ if (!timer_get_offset(ctx))
+ timer_set_cval(ctx, read_sysreg_el0(SYS_CNTP_CVAL));
/* Disable the timer */
write_sysreg_el0(0, SYS_CNTP_CTL);
@@ -621,7 +616,8 @@ static void timer_restore_state(struct arch_timer_context *ctx)
offset = timer_get_offset(ctx);
if (has_broken_cntvoff()) {
set_cntvoff(0);
- cval += offset;
+ if (offset)
+ cval = timer_apply_offset(cval, offset, kvm_phys_timer_read());
} else {
set_cntvoff(offset);
}
@@ -634,7 +630,8 @@ static void timer_restore_state(struct arch_timer_context *ctx)
cval = timer_get_cval(ctx);
offset = timer_get_offset(ctx);
set_cntpoff(offset);
- cval += offset;
+ if (offset)
+ cval = timer_apply_offset(cval, offset, kvm_phys_timer_read());
write_sysreg_el0(cval, SYS_CNTP_CVAL);
isb();
write_sysreg_el0(timer_get_ctl(ctx), SYS_CNTP_CTL);
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 4e0db01a528a..1cf6b6038200 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1225,11 +1225,11 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
*/
preempt_disable();
- kvm_nested_flush_hwstate(vcpu);
-
if (kvm_vcpu_has_pmu(vcpu))
kvm_pmu_flush_hwstate(vcpu);
+ kvm_nested_flush_hwstate(vcpu);
+
local_irq_disable();
/*
diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
index 6e7547b7ca43..1363758035bd 100644
--- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c
+++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
@@ -385,8 +385,34 @@ static void handle___pkvm_host_mkyoung_guest(struct kvm_cpu_context *host_ctxt)
static void handle___kvm_adjust_pc(struct kvm_cpu_context *host_ctxt)
{
DECLARE_REG(struct kvm_vcpu *, vcpu, host_ctxt, 1);
+ struct pkvm_hyp_vcpu *hyp_vcpu;
+ struct kvm_vcpu *host_vcpu;
+
+ host_vcpu = kern_hyp_va(vcpu);
+ if (unlikely(is_protected_kvm_enabled())) {
+ hyp_vcpu = pkvm_get_loaded_hyp_vcpu();
+
+ if (!hyp_vcpu || hyp_vcpu->host_vcpu != host_vcpu)
+ host_vcpu = NULL;
+ }
+
+ if (host_vcpu) {
+ __kvm_adjust_pc(host_vcpu);
+ return;
+ }
+
+ /*
+ * With no hyp vCPU loaded for it, the host vCPU may be unpinned,
+ * and so unmapped at EL2: its first run pins it. A pin fails only
+ * for memory the host isn't sharing, a bad pointer, so the request
+ * is dropped.
+ */
+ host_vcpu = kern_hyp_va(vcpu);
+ if (hyp_pin_shared_mem(host_vcpu, host_vcpu + 1))
+ return;
- __kvm_adjust_pc(kern_hyp_va(vcpu));
+ __kvm_adjust_pc(host_vcpu);
+ hyp_unpin_shared_mem(host_vcpu, host_vcpu + 1);
}
static void handle___kvm_flush_vm_context(struct kvm_cpu_context *host_ctxt)
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 552f39d913be..7a69bc0b6ea8 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -169,7 +169,8 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu)
offset = read_sysreg_s(SYS_CNTPOFF_EL2);
if (map.direct_ptimer && offset) {
- write_sysreg_el0(val + offset, SYS_CNTP_CVAL);
+ val = timer_apply_offset(val, offset, arch_timer_read_cntpct_el0());
+ write_sysreg_el0(val, SYS_CNTP_CVAL);
isb();
}
}
@@ -290,10 +291,10 @@ static bool kvm_hyp_handle_timer(struct kvm_vcpu *vcpu, u64 *exit_code)
break;
case SYS_CNTP_CVAL_EL0:
if (vcpu_el2_e2h_is_set(vcpu)) {
- val = read_sysreg_el0(SYS_CNTP_CVAL);
-
- if (!has_cntpoff())
- val -= timer_get_offset(vcpu_hptimer(vcpu));
+ if (!has_cntpoff() && timer_get_offset(vcpu_hptimer(vcpu)))
+ val = __vcpu_sys_reg(vcpu, CNTHP_CVAL_EL2);
+ else
+ val = read_sysreg_el0(SYS_CNTP_CVAL);
} else {
val = __vcpu_sys_reg(vcpu, CNTP_CVAL_EL0);
}
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index af50fed5b597..e8029ac82da7 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1536,6 +1536,8 @@ static int gmem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
if (ret)
return ret;
+ fault_ipa = ALIGN_DOWN(fault_ipa, PAGE_SIZE);
+
if (nested)
gfn = kvm_s2_trans_output(nested) >> PAGE_SHIFT;
else
@@ -1552,7 +1554,7 @@ static int gmem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
ret = kvm_gmem_get_pfn(kvm, memslot, gfn, &pfn, &page, NULL);
if (ret) {
- kvm_prepare_memory_fault_exit(vcpu, fault_ipa, PAGE_SIZE,
+ kvm_prepare_memory_fault_exit(vcpu, gfn_to_gpa(gfn), PAGE_SIZE,
write_fault, exec_fault, false);
return ret;
}
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 92853a343c42..9e1ce558fb7e 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -16,6 +16,7 @@
#include <asm/sysreg.h>
#include "sys_regs.h"
+#include "vgic/vgic.h"
struct vncr_tlb {
/* The guest's VNCR_EL2 */
@@ -1879,6 +1880,9 @@ void check_nested_vcpu_requests(struct kvm_vcpu *vcpu)
*
* What follows is some slop to (1) react to exception routing / masking and (2)
* preserve the pending SError state across translation regimes.
+ *
+ * Also generate the maintenance interrupt state, and finally check that if
+ * we need to take an interrupt targetting the guest hypervisor.
*/
void kvm_nested_flush_hwstate(struct kvm_vcpu *vcpu)
{
@@ -1887,6 +1891,12 @@ void kvm_nested_flush_hwstate(struct kvm_vcpu *vcpu)
if (unlikely(vcpu_test_and_clear_flag(vcpu, NESTED_SERROR_PENDING)))
kvm_inject_serror_esr(vcpu, vcpu_get_vsesr(vcpu));
+
+ if (!vgic_state_is_nested(vcpu))
+ vgic_v3_nested_update_mi(vcpu);
+
+ if (vgic_state_is_nested(vcpu) && kvm_vgic_vcpu_pending_irq(vcpu))
+ kvm_make_request(KVM_REQ_GUEST_HYP_IRQ_PENDING, vcpu);
}
void kvm_nested_sync_hwstate(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/kvm/ptdump.c b/arch/arm64/kvm/ptdump.c
index dc5acfb00af9..8e4966194be4 100644
--- a/arch/arm64/kvm/ptdump.c
+++ b/arch/arm64/kvm/ptdump.c
@@ -97,22 +97,35 @@ static struct kvm_ptdump_guest_state *kvm_ptdump_parser_create(struct kvm *kvm)
{
struct kvm_ptdump_guest_state *st;
struct kvm_s2_mmu *mmu = &kvm->arch.mmu;
- struct kvm_pgtable *pgtable = mmu->pgt;
+ s8 start_level;
+ u32 ia_bits;
int ret;
+ /*
+ * We only need the pgt start level and IPA size to initialize the ptdump,
+ * get them while holding the mmu_lock. It's fine if the pgt gets freed
+ * afterwards, we'll check again when doing the actual dump.
+ */
+ scoped_guard(read_lock, &kvm->mmu_lock) {
+ if (!mmu->pgt)
+ return ERR_PTR(-ENOENT);
+ start_level = mmu->pgt->start_level;
+ ia_bits = mmu->pgt->ia_bits;
+ }
+
st = kzalloc(sizeof(struct kvm_ptdump_guest_state), GFP_KERNEL_ACCOUNT);
if (!st)
return ERR_PTR(-ENOMEM);
- ret = kvm_ptdump_build_levels(&st->level[0], pgtable->start_level);
+ ret = kvm_ptdump_build_levels(&st->level[0], start_level);
if (ret) {
kfree(st);
return ERR_PTR(ret);
}
st->ipa_marker[0].name = "Guest IPA";
- st->ipa_marker[1].start_address = BIT(pgtable->ia_bits);
- st->range[0].end = BIT(pgtable->ia_bits);
+ st->ipa_marker[1].start_address = BIT(ia_bits);
+ st->range[0].end = BIT(ia_bits);
st->kvm = kvm;
st->parser_state = (struct ptdump_pg_state) {
@@ -141,9 +154,11 @@ static int kvm_ptdump_guest_show(struct seq_file *m, void *unused)
parser_state->seq = m;
- write_lock(&kvm->mmu_lock);
+ guard(write_lock)(&kvm->mmu_lock);
+ if (!mmu->pgt)
+ return 0;
+
ret = kvm_pgtable_walk(mmu->pgt, 0, BIT(mmu->pgt->ia_bits), &walker);
- write_unlock(&kvm->mmu_lock);
return ret;
}
@@ -193,17 +208,27 @@ static const struct file_operations kvm_ptdump_guest_fops = {
static int kvm_pgtable_range_show(struct seq_file *m, void *unused)
{
- struct kvm_pgtable *pgtable = m->private;
+ struct kvm_s2_mmu *mmu = m->private;
+ struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
+
+ guard(read_lock)(&kvm->mmu_lock);
+
+ if (mmu->pgt)
+ seq_printf(m, "%2u\n", mmu->pgt->ia_bits);
- seq_printf(m, "%2u\n", pgtable->ia_bits);
return 0;
}
static int kvm_pgtable_levels_show(struct seq_file *m, void *unused)
{
- struct kvm_pgtable *pgtable = m->private;
+ struct kvm_s2_mmu *mmu = m->private;
+ struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
+
+ guard(read_lock)(&kvm->mmu_lock);
+
+ if (mmu->pgt)
+ seq_printf(m, "%1d\n", KVM_PGTABLE_MAX_LEVELS - mmu->pgt->start_level);
- seq_printf(m, "%1d\n", KVM_PGTABLE_MAX_LEVELS - pgtable->start_level);
return 0;
}
@@ -211,15 +236,12 @@ static int kvm_pgtable_debugfs_open(struct inode *m, struct file *file,
int (*show)(struct seq_file *, void *))
{
struct kvm *kvm = m->i_private;
- struct kvm_pgtable *pgtable;
int ret;
if (!kvm_get_kvm_safe(kvm))
return -ENOENT;
- pgtable = kvm->arch.mmu.pgt;
-
- ret = single_open(file, show, pgtable);
+ ret = single_open(file, show, &kvm->arch.mmu);
if (ret < 0)
kvm_put_kvm(kvm);
return ret;
diff --git a/arch/arm64/kvm/vgic/vgic-mmio-v3.c b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
index 79e018cec0f9..5ba3bb576bb1 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
@@ -303,9 +303,19 @@ static void vgic_mmio_write_v3r_ctlr(struct kvm_vcpu *vcpu,
if (ctlr != GICR_CTLR_ENABLE_LPIS)
return;
+ /*
+ * Yes, disabling LPIs is painful, since it can be done from
+ * a *remote* vcpu! So let's not take any chance, and make
+ * sure that the target vCPU has written their LRs back to the
+ * irq structures, and release any reference they would have.
+ *
+ * If it hurts, don't do it.
+ */
+ kvm_arm_halt_guest(vcpu->kvm);
vgic_flush_pending_lpis(vcpu);
vgic_its_invalidate_all_caches(vcpu->kvm);
atomic_set_release(&vgic_cpu->ctlr, 0);
+ kvm_arm_resume_guest(vcpu->kvm);
} else {
ctlr = atomic_cmpxchg_acquire(&vgic_cpu->ctlr, 0,
GICR_CTLR_ENABLE_LPIS);
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index 7c8ec4482415..23720e1131b4 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -969,7 +969,7 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
*
* We do that by placing a request to ourselves which will
* abort the entry procedure and inject the exception at the
- * beginning of the run loop.
+ * beginning of the run loop (see kvm_nested_flush_hwstate()).
*
* - Otherwise, do exactly *NOTHING*. The guest state is
* already loaded, and we can carry on with running it.
@@ -977,15 +977,8 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
* If we have NV, but are not in a nested state, compute the
* maintenance interrupt state, as it may fire.
*/
- if (vgic_state_is_nested(vcpu)) {
- if (kvm_vgic_vcpu_pending_irq(vcpu))
- kvm_make_request(KVM_REQ_GUEST_HYP_IRQ_PENDING, vcpu);
-
+ if (vgic_state_is_nested(vcpu))
return;
- }
-
- if (vcpu_has_nv(vcpu))
- vgic_v3_nested_update_mi(vcpu);
/*
* If there are no virtual interrupts active or pending for this
diff --git a/arch/arm64/mm/mmap.c b/arch/arm64/mm/mmap.c
index 92b2f5097a96..dabc61000ecc 100644
--- a/arch/arm64/mm/mmap.c
+++ b/arch/arm64/mm/mmap.c
@@ -61,7 +61,9 @@ int valid_phys_addr_range(phys_addr_t addr, size_t size)
*/
int valid_mmap_phys_addr_range(unsigned long pfn, size_t size)
{
- return !(((pfn << PAGE_SHIFT) + size) & ~PHYS_MASK);
+ const phys_addr_t limit = PHYS_MASK + 1;
+
+ return size <= limit && pfn <= (limit - size) >> PAGE_SHIFT;
}
static int __init adjust_protection_map(void)
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 17df19647556..5e7bc1a8d9ca 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -103,6 +103,10 @@ SYM_FUNC_START(cpu_do_suspend)
alternative_if ARM64_HAS_TCR2
mrs x2, REG_TCR2_EL1
str x2, [x0, #104]
+alternative_else_nop_endif
+alternative_if ARM64_SME
+ mrs_s x2, SYS_TPIDR2_EL0
+ str x2, [x0, #112]
alternative_else_nop_endif
ret
SYM_FUNC_END(cpu_do_suspend)
@@ -142,6 +146,10 @@ alternative_if ARM64_HAS_TCR2
ldr x2, [x0, #104]
msr REG_TCR2_EL1, x2
alternative_else_nop_endif
+alternative_if ARM64_SME
+ ldr x2, [x0, #112]
+ msr_s SYS_TPIDR2_EL0, x2
+alternative_else_nop_endif
msr sctlr_el1, x12
set_this_cpu_offset x13
diff --git a/arch/loongarch/include/asm/hw_breakpoint.h b/arch/loongarch/include/asm/hw_breakpoint.h
index d202052df8a1..f4478936787d 100644
--- a/arch/loongarch/include/asm/hw_breakpoint.h
+++ b/arch/loongarch/include/asm/hw_breakpoint.h
@@ -121,12 +121,12 @@ bool watchpoint_handler(struct pt_regs *regs);
#ifdef CONFIG_HAVE_HW_BREAKPOINT
extern void ptrace_hw_copy_thread(struct task_struct *task);
-extern void hw_breakpoint_thread_switch(struct task_struct *next);
+extern void hw_breakpoint_thread_switch(struct task_struct *prev, struct task_struct *next);
#else
static inline void ptrace_hw_copy_thread(struct task_struct *task)
{
}
-static inline void hw_breakpoint_thread_switch(struct task_struct *next)
+static inline void hw_breakpoint_thread_switch(struct task_struct *prev, struct task_struct *next)
{
}
#endif
diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h
index 642dc9adddce..9a85227ed84c 100644
--- a/arch/loongarch/include/asm/loongarch.h
+++ b/arch/loongarch/include/asm/loongarch.h
@@ -1093,6 +1093,9 @@
#define CSR_FWPS_SKIP_SHIFT 16
#define CSR_FWPS_SKIP (_ULCAST_(1) << CSR_FWPS_SKIP_SHIFT)
+#define CSR_MWPS_SKIP_SHIFT 16
+#define CSR_MWPS_SKIP (_ULCAST_(1) << CSR_MWPS_SKIP_SHIFT)
+
/*
* CSR_ECFG IM
*/
diff --git a/arch/loongarch/include/asm/processor.h b/arch/loongarch/include/asm/processor.h
index c3bc44b5f5b3..81af48cbee7b 100644
--- a/arch/loongarch/include/asm/processor.h
+++ b/arch/loongarch/include/asm/processor.h
@@ -138,6 +138,7 @@ struct thread_struct {
struct loongarch_lbt lbt; /* Also conditionally copied */
/* Hardware breakpoints pinned to this task. */
+ bool hbp_break_skip, hbp_watch_skip;
struct perf_event *hbp_break[LOONGARCH_MAX_BRP];
struct perf_event *hbp_watch[LOONGARCH_MAX_WRP];
};
diff --git a/arch/loongarch/include/asm/switch_to.h b/arch/loongarch/include/asm/switch_to.h
index 5b225aff3ba2..3b662b18e19a 100644
--- a/arch/loongarch/include/asm/switch_to.h
+++ b/arch/loongarch/include/asm/switch_to.h
@@ -36,7 +36,7 @@ extern asmlinkage struct task_struct *__switch_to(struct task_struct *prev,
do { \
lose_fpu_inatomic(1, prev); \
lose_lbt_inatomic(1, prev); \
- hw_breakpoint_thread_switch(next); \
+ hw_breakpoint_thread_switch(prev, next); \
(last) = __switch_to(prev, next, task_thread_info(next), \
__builtin_return_address(0), __builtin_frame_address(0)); \
} while (0)
diff --git a/arch/loongarch/kernel/hw_breakpoint.c b/arch/loongarch/kernel/hw_breakpoint.c
index 1a05350159f7..df149ff06dd5 100644
--- a/arch/loongarch/kernel/hw_breakpoint.c
+++ b/arch/loongarch/kernel/hw_breakpoint.c
@@ -156,6 +156,9 @@ static int hw_breakpoint_slot_setup(struct perf_event **slots, int max_slots,
void ptrace_hw_copy_thread(struct task_struct *tsk)
{
+ tsk->thread.hbp_break_skip = 0;
+ tsk->thread.hbp_watch_skip = 0;
+
memset(tsk->thread.hbp_break, 0, sizeof(tsk->thread.hbp_break));
memset(tsk->thread.hbp_watch, 0, sizeof(tsk->thread.hbp_watch));
}
@@ -485,6 +488,7 @@ NOKPROBE_SYMBOL(update_bp_registers);
bool breakpoint_handler(struct pt_regs *regs)
{
int i;
+ unsigned int clear_mask = 0;
bool need_sigtrap = false;
struct perf_event *bp, **slots;
@@ -500,11 +504,13 @@ bool breakpoint_handler(struct pt_regs *regs)
if (bp->attr.sigtrap)
need_sigtrap = true;
- csr_write32(0x1 << i, LOONGARCH_CSR_FWPS);
- update_bp_registers(regs, 0, 0);
+ clear_mask |= (0x1 << i);
}
}
+ if (clear_mask)
+ csr_write32(clear_mask | CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS);
+
return need_sigtrap;
}
NOKPROBE_SYMBOL(breakpoint_handler);
@@ -512,6 +518,7 @@ NOKPROBE_SYMBOL(breakpoint_handler);
bool watchpoint_handler(struct pt_regs *regs)
{
int i;
+ unsigned int clear_mask = 0;
bool need_sigtrap = false;
struct perf_event *wp, **slots;
@@ -527,11 +534,13 @@ bool watchpoint_handler(struct pt_regs *regs)
if (wp->attr.sigtrap)
need_sigtrap = true;
- csr_write32(0x1 << i, LOONGARCH_CSR_MWPS);
- update_bp_registers(regs, 0, 1);
+ clear_mask |= (0x1 << i);
}
}
+ if (clear_mask)
+ csr_write32(clear_mask | CSR_MWPS_SKIP, LOONGARCH_CSR_MWPS);
+
return need_sigtrap;
}
NOKPROBE_SYMBOL(watchpoint_handler);
@@ -555,7 +564,7 @@ static int __init arch_hw_breakpoint_init(void)
}
arch_initcall(arch_hw_breakpoint_init);
-void hw_breakpoint_thread_switch(struct task_struct *next)
+void hw_breakpoint_thread_switch(struct task_struct *prev, struct task_struct *next)
{
u64 addr, mask;
struct pt_regs *regs = task_pt_regs(next);
@@ -567,6 +576,26 @@ void hw_breakpoint_thread_switch(struct task_struct *next)
csr_write32(CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS);
regs->csr_prmd |= CSR_PRMD_PWE;
} else {
+ unsigned int fwps = csr_read32(LOONGARCH_CSR_FWPS);
+ unsigned int mwps = csr_read32(LOONGARCH_CSR_MWPS);
+
+ prev->thread.hbp_break_skip = !!(fwps & CSR_FWPS_SKIP);
+ prev->thread.hbp_watch_skip = !!(mwps & CSR_MWPS_SKIP);
+
+ if (!next->thread.hbp_break_skip)
+ csr_write32(0, LOONGARCH_CSR_FWPS);
+ else {
+ csr_write32(CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS);
+ next->thread.hbp_break_skip = 0;
+ }
+
+ if (!next->thread.hbp_watch_skip)
+ csr_write32(0, LOONGARCH_CSR_MWPS);
+ else {
+ csr_write32(CSR_MWPS_SKIP, LOONGARCH_CSR_MWPS);
+ next->thread.hbp_watch_skip = 0;
+ }
+
/* Update breakpoints */
update_bp_registers(regs, 1, 0);
/* Update watchpoints */
diff --git a/arch/loongarch/kernel/numa.c b/arch/loongarch/kernel/numa.c
index 8b89898e20df..69ea85c4f232 100644
--- a/arch/loongarch/kernel/numa.c
+++ b/arch/loongarch/kernel/numa.c
@@ -216,6 +216,8 @@ static int __init fake_numa_init(void)
phys_addr_t start = memblock_start_of_DRAM();
phys_addr_t end = memblock_end_of_DRAM() - 1;
+ set_cpuid_to_node(0, 0);
+ early_numa_add_cpu(0, 0);
node_set(0, numa_nodes_parsed);
pr_info("Faking a node at [mem %pap-%pap]\n", &start, &end);
@@ -245,8 +247,10 @@ int __init init_numa_memory(void)
return -EINVAL;
for_each_node_mask(node, node_possible_map) {
- node_mem_init(node);
- node_set_online(node);
+ if (!cpumask_empty(&cpus_on_node[node])) {
+ node_mem_init(node);
+ node_set_online(node);
+ }
}
max_pfn = PFN_DOWN(memblock_end_of_DRAM());
max_low_pfn = min(PFN_DOWN(HIGHMEM_START), max_pfn);
diff --git a/arch/m68k/68000/entry.S b/arch/m68k/68000/entry.S
index 72e95663b62f..0d3cbededc43 100644
--- a/arch/m68k/68000/entry.S
+++ b/arch/m68k/68000/entry.S
@@ -18,6 +18,13 @@
.text
+/* get thread_info pointer into a2 */
+ .macro getthreadinfo
+ movel %sp,%d1
+ andl #-THREAD_SIZE,%d1
+ movel %d1,%a2
+ .endm
+
.globl system_call
.globl resume
.globl ret_from_exception
@@ -70,10 +77,10 @@ ENTRY(system_call)
movel %sp@(PT_OFF_ORIG_D0),%d0
- movel %sp,%d1 /* get thread_info pointer */
- andl #-THREAD_SIZE,%d1
- movel %d1,%a2
- btst #(TIF_SYSCALL_TRACE%8),%a2@(TINFO_FLAGS+(31-TIF_SYSCALL_TRACE)/8)
+ /* Doing a trace ? */
+ getthreadinfo
+ moveb %a2@(TINFO_FLAGS+2),%d1
+ andl #((_TIF_SYSCALL_TRACE + _TIF_SECCOMP) >> 8),%d1
jne do_trace
cmpl #NR_syscalls,%d0
jcc badsys
@@ -96,16 +103,15 @@ Luser_return:
/* heavy interrupt load*/
andw #ALLOWINT,%sr
- movel %sp,%d1 /* get thread_info pointer */
- andl #-THREAD_SIZE,%d1
- movel %d1,%a2
+ getthreadinfo
1:
- move %a2@(TINFO_FLAGS),%d1 /* thread_info->flags */
+ /* check if any of the flags are set */
+ moveb %a2@(TINFO_FLAGS + 3),%d1 /* thread_info->flags (low 8 bits) */
jne Lwork_to_do
RESTORE_ALL
Lwork_to_do:
- movel %a2@(TINFO_FLAGS),%d1 /* thread_info->flags */
+ /* check if reschedule needs to be called */
btst #TIF_NEED_RESCHED,%d1
jne reschedule
diff --git a/arch/m68k/coldfire/entry.S b/arch/m68k/coldfire/entry.S
index 4ea08336e2fb..d658abdc5207 100644
--- a/arch/m68k/coldfire/entry.S
+++ b/arch/m68k/coldfire/entry.S
@@ -72,7 +72,8 @@ ENTRY(system_call)
movel %d2,%a0
movel %a0@,%a1 /* save top of frame */
movel %sp,%a1@(TASK_THREAD+THREAD_ESP0)
- btst #(TIF_SYSCALL_TRACE%8),%a0@(TINFO_FLAGS+(31-TIF_SYSCALL_TRACE)/8)
+ moveb %a0@(TINFO_FLAGS+2),%d2
+ andl #((_TIF_SYSCALL_TRACE + _TIF_SECCOMP) >> 8),%d2
bnes 1f
movel %d3,%a0
diff --git a/arch/mips/mm/cex-gen.S b/arch/mips/mm/cex-gen.S
index e528583d1331..6989fe274aab 100644
--- a/arch/mips/mm/cex-gen.S
+++ b/arch/mips/mm/cex-gen.S
@@ -8,6 +8,7 @@
*
* Cache error handler
*/
+#include <asm/addrspace.h>
#include <asm/asm.h>
#include <asm/regdef.h>
#include <asm/mipsregs.h>
@@ -37,6 +38,21 @@
nop
nop
+#if defined(CONFIG_64BIT) && VMLINUX_LOAD_ADDRESS >= XKPHYS && \
+ VMLINUX_LOAD_ADDRESS < XKSEG
+ /*
+ * An XKPHYS kernel may lie outside the 256MB region a j from the
+ * vector reaches, so jump to the handler's uncached XKPHYS address.
+ */
+ PTR_LA k0, cache_parity_error
+ PTR_LI k1, TO_PHYS_MASK
+ and k0, k0, k1
+ PTR_LI k1, UNCAC_BASE
+ or k0, k0, k1
+ jr k0
+ nop
+#else
j cache_parity_error
nop
+#endif
END(except_vec2_generic)
diff --git a/arch/mips/pci/ops-mace.c b/arch/mips/pci/ops-mace.c
index 951d8070fb48..8827e5f2ea46 100644
--- a/arch/mips/pci/ops-mace.c
+++ b/arch/mips/pci/ops-mace.c
@@ -35,15 +35,27 @@ static inline int mkaddr(struct pci_bus *bus, unsigned int devfn,
(reg & 0xfc);
}
+/*
+ * On CRIME 1.1, a MACE PCI register read followed by two register writes
+ * can hang the machine under DMA load, so every register write in this
+ * file is followed by a read. Config cycles to empty slots end in a
+ * master abort; ack it with interrupts still off so the bridge error
+ * interrupt does not report it.
+ */
+static void mace_pci_ack_abort(void)
+{
+ u32 error = mace->pci.error;
+
+ if (error & MACEPCI_ERROR_MASTER_ABORT) {
+ mace->pci.error = error & ~MACEPCI_ERROR_MASTER_ABORT;
+ mace->pci.error;
+ }
+}
static int
mace_pci_read_config(struct pci_bus *bus, unsigned int devfn,
int reg, int size, u32 *val)
{
- u32 control = mace->pci.control;
-
- /* disable master aborts interrupts during config read */
- mace->pci.control = control & ~MACEPCI_CONTROL_MAR_INT;
mace->pci.config_addr = mkaddr(bus, devfn, reg);
switch (size) {
case 1:
@@ -56,9 +68,7 @@ mace_pci_read_config(struct pci_bus *bus, unsigned int devfn,
*val = mace->pci.config_data.l;
break;
}
- /* ack possible master abort */
- mace->pci.error &= ~MACEPCI_ERROR_MASTER_ABORT;
- mace->pci.control = control;
+ mace_pci_ack_abort();
/*
* someone forgot to set the ultra bit for the onboard
* scsi chips; we fake it here
@@ -77,6 +87,7 @@ mace_pci_write_config(struct pci_bus *bus, unsigned int devfn,
int reg, int size, u32 val)
{
mace->pci.config_addr = mkaddr(bus, devfn, reg);
+ mace->pci.error;
switch (size) {
case 1:
mace->pci.config_data.b[(reg & 3) ^ 3] = val;
@@ -88,6 +99,7 @@ mace_pci_write_config(struct pci_bus *bus, unsigned int devfn,
mace->pci.config_data.l = val;
break;
}
+ mace_pci_ack_abort();
DPRINTK("write%d: reg=%08x,val=%02x\n", size * 8, reg, val);
diff --git a/arch/mips/pci/pci-ip32.c b/arch/mips/pci/pci-ip32.c
index 7ae89d0c7099..41ec8453c865 100644
--- a/arch/mips/pci/pci-ip32.c
+++ b/arch/mips/pci/pci-ip32.c
@@ -77,6 +77,8 @@ static irqreturn_t macepci_error(int irq, void *dev)
}
mace->pci.error = flags;
+ /* follow the write with a read, see mace_pci_ack_abort() */
+ mace->pci.error;
return IRQ_HANDLED;
}
@@ -125,10 +127,13 @@ static int __init mace_init(void)
{
PCIBIOS_MIN_IO = 0x1000;
- /* Clear any outstanding errors and enable interrupts */
+ /*
+ * Clear any outstanding errors and enable interrupts. PCI interrupts
+ * are masked in CRIME, so the per-slot enables stay on.
+ */
mace->pci.error_addr = 0;
mace->pci.error = 0;
- mace->pci.control = 0xff008500;
+ mace->pci.control = 0xff0085ff;
printk("MACE PCI rev %d\n", mace->pci.rev);
diff --git a/arch/mips/sgi-ip32/ip32-irq.c b/arch/mips/sgi-ip32/ip32-irq.c
index 29d04468a06b..b9c72af481ed 100644
--- a/arch/mips/sgi-ip32/ip32-irq.c
+++ b/arch/mips/sgi-ip32/ip32-irq.c
@@ -160,38 +160,6 @@ static struct irq_chip crime_edge_interrupt = {
.irq_unmask = crime_enable_irq,
};
-/*
- * This is for MACE PCI interrupts. We can decrease bus traffic by masking
- * as close to the source as possible. This also means we can take the
- * next chunk of the CRIME register in one piece.
- */
-
-static unsigned long macepci_mask;
-
-static void enable_macepci_irq(struct irq_data *d)
-{
- macepci_mask |= MACEPCI_CONTROL_INT(d->irq - MACEPCI_SCSI0_IRQ);
- mace->pci.control = macepci_mask;
- crime_mask |= 1 << (d->irq - CRIME_IRQ_BASE);
- crime->imask = crime_mask;
-}
-
-static void disable_macepci_irq(struct irq_data *d)
-{
- crime_mask &= ~(1 << (d->irq - CRIME_IRQ_BASE));
- crime->imask = crime_mask;
- flush_crime_bus();
- macepci_mask &= ~MACEPCI_CONTROL_INT(d->irq - MACEPCI_SCSI0_IRQ);
- mace->pci.control = macepci_mask;
- flush_mace_bus();
-}
-
-static struct irq_chip ip32_macepci_interrupt = {
- .name = "IP32 MACE PCI",
- .irq_mask = disable_macepci_irq,
- .irq_unmask = enable_macepci_irq,
-};
-
/* This is used for MACE ISA interrupts. That means bits 4-6 in the
* CRIME register.
*/
@@ -299,8 +267,8 @@ static struct irq_chip ip32_maceisa_edge_interrupt = {
.irq_unmask = enable_maceisa_irq,
};
-/* This is used for regular non-ISA, non-PCI MACE interrupts. That means
- * bits 0-3 and 7 in the CRIME register.
+/* This is used for regular non-ISA MACE interrupts. That means bits 0-3
+ * and 7-15 in the CRIME register.
*/
static void enable_mace_irq(struct irq_data *d)
@@ -436,20 +404,13 @@ void __init arch_init_irq(void)
mips_cpu_irq_init();
for (irq = CRIME_IRQ_BASE; irq <= IP32_IRQ_MAX; irq++) {
switch (irq) {
- case MACE_VID_IN1_IRQ ... MACE_PCI_BRIDGE_IRQ:
+ case MACE_VID_IN1_IRQ ... MACEPCI_SHARED2_IRQ:
irq_set_chip_and_handler_name(irq,
&ip32_mace_interrupt,
handle_level_irq,
"level");
break;
- case MACEPCI_SCSI0_IRQ ... MACEPCI_SHARED2_IRQ:
- irq_set_chip_and_handler_name(irq,
- &ip32_macepci_interrupt,
- handle_level_irq,
- "level");
- break;
-
case CRIME_CPUERR_IRQ:
case CRIME_MEMERR_IRQ:
irq_set_chip_and_handler_name(irq,
diff --git a/arch/openrisc/kernel/or32_ksyms.c b/arch/openrisc/kernel/or32_ksyms.c
index 212e5f85004c..c63d7f25bc57 100644
--- a/arch/openrisc/kernel/or32_ksyms.c
+++ b/arch/openrisc/kernel/or32_ksyms.c
@@ -35,6 +35,7 @@ DECLARE_EXPORT(__divsi3);
DECLARE_EXPORT(__umodsi3);
DECLARE_EXPORT(__modsi3);
DECLARE_EXPORT(__muldi3);
+DECLARE_EXPORT(__mulsi3);
DECLARE_EXPORT(__ashrdi3);
DECLARE_EXPORT(__ashldi3);
DECLARE_EXPORT(__lshrdi3);
diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h
index 8334cd667bba..ae2c92a8b6e7 100644
--- a/arch/powerpc/net/bpf_jit.h
+++ b/arch/powerpc/net/bpf_jit.h
@@ -14,6 +14,13 @@
#include <asm/ppc-opcode.h>
#include <linux/build_bug.h>
+/*
+ * We need at least 2 passes for proper code generation, and may need
+ * additional passes if code size changes between passes.
+ */
+#define CODEGEN_MIN_PASSES 2
+#define CODEGEN_MAX_PASSES 3
+
#ifdef CONFIG_PPC64_ELF_ABI_V1
#define FUNCTION_DESCR_SIZE 24
#else
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index cedab25aa3a8..829b60ae485f 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -99,11 +99,10 @@ void bpf_jit_build_fentry_stubs(u32 *image, struct codegen_context *ctx)
int bpf_jit_emit_exit_insn(u32 *image, struct codegen_context *ctx, int tmp_reg, long exit_addr)
{
- if (!exit_addr || is_offset_in_branch_range(exit_addr - (ctx->idx * 4))) {
+ if (exit_addr && is_offset_in_branch_range(exit_addr - (long)(ctx->idx * 4))) {
PPC_JMP(exit_addr);
- } else if (ctx->alt_exit_addr) {
- if (WARN_ON(!is_offset_in_branch_range((long)ctx->alt_exit_addr - (ctx->idx * 4))))
- return -1;
+ } else if (ctx->alt_exit_addr && is_offset_in_branch_range(
+ (long)(ctx->alt_exit_addr) - (long)(ctx->idx * 4))) {
PPC_JMP(ctx->alt_exit_addr);
} else {
ctx->alt_exit_addr = ctx->idx * 4;
@@ -223,6 +222,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
*/
if (cgctx.seen & SEEN_TAILCALL || !is_offset_in_branch_range((long)cgctx.idx * 4)) {
cgctx.idx = 0;
+ cgctx.alt_exit_addr = 0;
if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
fp = org_fp;
goto out_addrs;
@@ -259,10 +259,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
code_base = (u32 *)(image + FUNCTION_DESCR_SIZE);
fcode_base = (u32 *)(fimage + FUNCTION_DESCR_SIZE);
- /* Code generation passes 1-2 */
- for (pass = 1; pass < 3; pass++) {
+ /* Code generation passes 1-2+, loop until program size converges. */
+ for (pass = 1; pass <= CODEGEN_MAX_PASSES; pass++) {
+ u32 prev_proglen = proglen;
+
/* Now build the prologue, body code & epilogue for real. */
cgctx.idx = 0;
+ cgctx.exentry_idx = 0;
cgctx.alt_exit_addr = 0;
bpf_jit_build_prologue(code_base, &cgctx);
if (bpf_jit_build_body(fp, code_base, fcode_base, &cgctx, addrs, pass,
@@ -272,11 +275,27 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
fp = org_fp;
goto out_addrs;
}
+ addrs[fp->len] = cgctx.idx * 4;
bpf_jit_build_epilogue(code_base, &cgctx);
+ proglen = cgctx.idx * 4;
+
if (bpf_jit_enable > 1)
pr_info("Pass %d: shrink = %d, seen = 0x%x\n", pass,
- proglen - (cgctx.idx * 4), cgctx.seen);
+ prev_proglen - proglen, cgctx.seen);
+
+ /* Check if program size has converged, but ensure minimum passes */
+ if (pass >= CODEGEN_MIN_PASSES && proglen == prev_proglen)
+ break;
+
+ if (pass == CODEGEN_MAX_PASSES && proglen != prev_proglen) {
+ pr_err("BPF JIT: Program did not converge after %d passes\n",
+ CODEGEN_MAX_PASSES);
+ bpf_arch_text_copy(&fhdr->size, &hdr->size, sizeof(hdr->size));
+ bpf_jit_binary_pack_free(fhdr, hdr);
+ fp = org_fp;
+ goto out_addrs;
+ }
}
if (bpf_jit_enable > 1)
@@ -336,7 +355,7 @@ int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, int pass
u32 *fixup;
/* Populate extable entries only in the last pass */
- if (pass != 2)
+ if (pass < CODEGEN_MIN_PASSES)
return 0;
if (!fp->aux->extable ||
diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 4bcb6f222065..aad5200b7bc1 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -69,6 +69,8 @@ static struct iommu_table *iommu_pseries_alloc_table(int node)
return tbl;
}
+static phys_addr_t pseries_ddw_max_ram __ro_after_init;
+
#ifdef CONFIG_IOMMU_API
static struct iommu_table_group_ops spapr_tce_table_group_ops;
#endif
@@ -1284,15 +1286,19 @@ struct failed_ddw_pdn {
static LIST_HEAD(failed_ddw_pdn_list);
-static phys_addr_t ddw_memory_hotplug_max(void)
+static phys_addr_t __init ddw_memory_hotplug_max(void)
{
- resource_size_t max_addr;
+ resource_size_t max_addr = memory_hotplug_max();
+ struct device_node *memory;
-#if defined(CONFIG_NUMA) && defined(CONFIG_MEMORY_HOTPLUG)
- max_addr = hot_add_drconf_memory_max();
-#else
- max_addr = memblock_end_of_DRAM();
-#endif
+ for_each_node_by_type(memory, "memory") {
+ struct resource res;
+
+ if (of_address_to_resource(memory, 0, &res))
+ continue;
+
+ max_addr = max_t(resource_size_t, max_addr, res.end + 1);
+ }
return max_addr;
}
@@ -1447,7 +1453,7 @@ static struct property *ddw_property_create(const char *propname, u32 liobn, u64
static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mask)
{
int len = 0, ret;
- int max_ram_len = order_base_2(ddw_memory_hotplug_max());
+ int max_ram_len = order_base_2(pseries_ddw_max_ram);
struct ddw_query_response query;
struct ddw_create_response create;
int page_shift;
@@ -1617,12 +1623,12 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas
if (default_win_removed && pmem_present && !direct_mapping) {
/* DDW is big enough to be split */
if ((1ULL << ddw_sz) >=
- MIN_DDW_VPMEM_DMA_WINDOW + (1ULL << max_ram_len)) {
+ MIN_DDW_VPMEM_DMA_WINDOW + pseries_ddw_max_ram) {
direct_mapping = true;
/* offset of the Dynamic part of DDW */
- dynamic_offset = 1ULL << max_ram_len;
+ dynamic_offset = pseries_ddw_max_ram;
}
/* DDW will at least have dynamic allocation */
@@ -1669,7 +1675,7 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas
if (direct_mapping) {
/* DDW maps the whole partition, so enable direct DMA mapping */
- ret = walk_system_ram_range(0, ddw_memory_hotplug_max() >> PAGE_SHIFT,
+ ret = walk_system_ram_range(0, pseries_ddw_max_ram >> PAGE_SHIFT,
win64->value, tce_setrange_multi_pSeriesLP_walk);
if (ret) {
dev_info(&dev->dev, "failed to map DMA window for %pOF: %d\n",
@@ -1774,8 +1780,7 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas
* to directly map both RAM and vPMEM, then we need to set DMA limit.
*/
if (pmem_present && direct_mapping && len != MAX_PHYSMEM_BITS)
- dev->dev.bus_dma_limit = dev->dev.archdata.dma_offset +
- (1ULL << max_ram_len);
+ dev->dev.bus_dma_limit = dev->dev.archdata.dma_offset + pseries_ddw_max_ram;
dev_info(&dev->dev, "lsa_required: %x, lsa_enabled: %x, direct mapping: %x\n",
limited_addr_req, limited_addr_enabled, direct_mapping);
@@ -2422,23 +2427,35 @@ static int iommu_mem_notifier(struct notifier_block *nb, unsigned long action,
{
struct dma_win *window;
struct memory_notify *arg = data;
+ unsigned long limit = arg->nr_pages;
+ unsigned long max_ram_pages = pseries_ddw_max_ram >> PAGE_SHIFT;
int ret = 0;
/* This notifier can get called when onlining persistent memory as well.
* TCEs are not pre-mapped for persistent memory. Persistent memory will
- * always be above ddw_memory_hotplug_max()
+ * always be above pseries_ddw_max_ram
*/
+ if (arg->start_pfn >= max_ram_pages)
+ return NOTIFY_OK;
+
+ /* RAM is being DLPAR'ed. The range should never exceed max ram.
+ * Just in case, clamp the range and throw a warning.
+ */
+ if (arg->start_pfn + limit > max_ram_pages) {
+ limit = max_ram_pages - arg->start_pfn;
+ WARN_ONCE(1, "Limiting Page Range %lx - %lx to Max Mem Pages: %lx\n",
+ arg->start_pfn, arg->start_pfn + arg->nr_pages,
+ max_ram_pages);
+ }
switch (action) {
case MEM_GOING_ONLINE:
spin_lock(&dma_win_list_lock);
list_for_each_entry(window, &dma_win_list, list) {
- if (window->direct && (arg->start_pfn << PAGE_SHIFT) <
- ddw_memory_hotplug_max()) {
+ if (window->direct) {
ret |= tce_setrange_multi_pSeriesLP(arg->start_pfn,
- arg->nr_pages, window->prop);
+ limit, window->prop);
}
- /* XXX log error */
}
spin_unlock(&dma_win_list_lock);
break;
@@ -2446,12 +2463,10 @@ static int iommu_mem_notifier(struct notifier_block *nb, unsigned long action,
case MEM_OFFLINE:
spin_lock(&dma_win_list_lock);
list_for_each_entry(window, &dma_win_list, list) {
- if (window->direct && (arg->start_pfn << PAGE_SHIFT) <
- ddw_memory_hotplug_max()) {
+ if (window->direct) {
ret |= tce_clearrange_multi_pSeriesLP(arg->start_pfn,
- arg->nr_pages, window->prop);
+ limit, window->prop);
}
- /* XXX log error */
}
spin_unlock(&dma_win_list_lock);
break;
@@ -2535,6 +2550,14 @@ void __init iommu_init_early_pSeries(void)
register_memory_notifier(&iommu_mem_nb);
set_pci_dma_ops(&dma_iommu_ops);
+
+ /* During init determine the max memory an LPAR can have and set it. This
+ * will be used for pre-mapping RAM in DDW. memblock_end_of_DRAM() can
+ * change during the running of LPAR - daxctl can add pmemory as
+ * "system-ram". This memory range should not be pre-mapped in DDW since
+ * the address of pmemory can be much higher than the DDW size.
+ */
+ pseries_ddw_max_ram = ddw_memory_hotplug_max();
}
static int __init disable_multitce(char *str)
diff --git a/arch/s390/pci/pci_irq.c b/arch/s390/pci/pci_irq.c
index e73be96ce5fe..836db17f4e8f 100644
--- a/arch/s390/pci/pci_irq.c
+++ b/arch/s390/pci/pci_irq.c
@@ -450,15 +450,10 @@ static int __init zpci_directed_irq_init(void)
if (!zpci_sbv)
return -ENOMEM;
- iib.diib.isc = PCI_ISC;
- iib.diib.nr_cpus = num_possible_cpus();
- iib.diib.disb_addr = virt_to_phys(zpci_sbv->vector);
- zpci_set_irq_ctrl(SIC_IRQ_MODE_DIRECT, 0, &iib);
-
zpci_ibv = kcalloc(num_possible_cpus(), sizeof(*zpci_ibv),
GFP_KERNEL);
if (!zpci_ibv)
- return -ENOMEM;
+ goto out_free_sbv;
for_each_possible_cpu(cpu) {
/*
@@ -470,13 +465,30 @@ static int __init zpci_directed_irq_init(void)
AIRQ_IV_CACHELINE |
(!cpu ? AIRQ_IV_ALLOC : 0), NULL);
if (!zpci_ibv[cpu])
- return -ENOMEM;
+ goto out_free_dibv;
}
+
+ iib.diib.isc = PCI_ISC;
+ iib.diib.nr_cpus = num_possible_cpus();
+ iib.diib.disb_addr = virt_to_phys(zpci_sbv->vector);
+ zpci_set_irq_ctrl(SIC_IRQ_MODE_DIRECT, 0, &iib);
on_each_cpu(cpu_enable_directed_irq, NULL, 1);
zpci_irq_chip.irq_set_affinity = zpci_set_irq_affinity;
return 0;
+
+out_free_dibv:
+ for_each_possible_cpu(cpu) {
+ if (zpci_ibv[cpu])
+ airq_iv_release(zpci_ibv[cpu]);
+ }
+ kfree(zpci_ibv);
+ zpci_ibv = NULL;
+out_free_sbv:
+ airq_iv_release(zpci_sbv);
+ zpci_sbv = NULL;
+ return -ENOMEM;
}
static int __init zpci_floating_irq_init(void)
diff --git a/arch/x86/boot/startup/sme.c b/arch/x86/boot/startup/sme.c
index e7ea65f3f1d6..1528199e3a3f 100644
--- a/arch/x86/boot/startup/sme.c
+++ b/arch/x86/boot/startup/sme.c
@@ -97,7 +97,7 @@ static void __init sme_clear_pgd(struct sme_populate_pgd_data *ppd)
pgd_t *pgd_p;
pgd_start = ppd->vaddr & PGDIR_MASK;
- pgd_end = ppd->vaddr_end & PGDIR_MASK;
+ pgd_end = (ppd->vaddr_end - 1) & PGDIR_MASK;
pgd_size = (((pgd_end - pgd_start) / PGDIR_SIZE) + 1) * sizeof(pgd_t);
diff --git a/arch/x86/include/asm/vmware.h b/arch/x86/include/asm/vmware.h
index 4220dae14a2d..fb011cef01e7 100644
--- a/arch/x86/include/asm/vmware.h
+++ b/arch/x86/include/asm/vmware.h
@@ -276,20 +276,22 @@ unsigned long vmware_hypercall_hb_out(unsigned long cmd, unsigned long in2,
asm_inline volatile (
UNWIND_HINT_SAVE
+ "mov %[in6], %%" _ASM_AX "\n\t"
"push %%" _ASM_BP "\n\t"
UNWIND_HINT_UNDEFINED
- "mov %[in6], %%" _ASM_BP "\n\t"
+ "mov %%" _ASM_AX ", %%" _ASM_BP "\n\t"
+ "mov %[magic], %%eax\n\t"
"rep outsb\n\t"
"pop %%" _ASM_BP "\n\t"
UNWIND_HINT_RESTORE
- : "=a" (out0), "=b" (*out1)
- : "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (cmd),
+ : "=&a" (out0), "=b" (*out1)
+ : "b" (cmd),
"c" (in2),
"d" (in3 | VMWARE_HYPERVISOR_PORT_HB),
"S" (in4),
"D" (in5),
- [in6] VMW_BP_CONSTRAINT (in6)
+ [in6] VMW_BP_CONSTRAINT(in6),
+ [magic] "i" (VMWARE_HYPERVISOR_MAGIC)
: "cc", "memory");
return out0;
}
@@ -304,20 +306,22 @@ unsigned long vmware_hypercall_hb_in(unsigned long cmd, unsigned long in2,
asm_inline volatile (
UNWIND_HINT_SAVE
+ "mov %[in6], %%" _ASM_AX "\n\t"
"push %%" _ASM_BP "\n\t"
UNWIND_HINT_UNDEFINED
- "mov %[in6], %%" _ASM_BP "\n\t"
+ "mov %%" _ASM_AX ", %%" _ASM_BP "\n\t"
+ "mov %[magic], %%eax\n\t"
"rep insb\n\t"
"pop %%" _ASM_BP "\n\t"
UNWIND_HINT_RESTORE
- : "=a" (out0), "=b" (*out1)
- : "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (cmd),
+ : "=&a" (out0), "=b" (*out1)
+ : "b" (cmd),
"c" (in2),
"d" (in3 | VMWARE_HYPERVISOR_PORT_HB),
"S" (in4),
"D" (in5),
- [in6] VMW_BP_CONSTRAINT (in6)
+ [in6] VMW_BP_CONSTRAINT(in6),
+ [magic] "i" (VMWARE_HYPERVISOR_MAGIC)
: "cc", "memory");
return out0;
}
diff --git a/arch/x86/include/asm/xor_avx.h b/arch/x86/include/asm/xor_avx.h
index 7f81dd5897f4..c6450acd055d 100644
--- a/arch/x86/include/asm/xor_avx.h
+++ b/arch/x86/include/asm/xor_avx.h
@@ -50,6 +50,7 @@ do { \
p1 = (unsigned long *)((uintptr_t)p1 + 512);
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -81,6 +82,7 @@ do { \
p2 = (unsigned long *)((uintptr_t)p2 + 512);
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -116,6 +118,7 @@ do { \
p3 = (unsigned long *)((uintptr_t)p3 + 512);
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -155,6 +158,7 @@ do { \
p4 = (unsigned long *)((uintptr_t)p4 + 512);
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
diff --git a/arch/x86/kernel/acpi/cppc.c b/arch/x86/kernel/acpi/cppc.c
index be4c5e9e5ff6..ca5287ccd896 100644
--- a/arch/x86/kernel/acpi/cppc.c
+++ b/arch/x86/kernel/acpi/cppc.c
@@ -6,6 +6,8 @@
#include <linux/bitfield.h>
+#include <linux/limits.h>
+
#include <acpi/cppc_acpi.h>
#include <asm/msr.h>
#include <asm/processor.h>
@@ -45,10 +47,20 @@ bool cpc_ffh_supported(void)
return true;
}
+static bool cpc_ffh_reg_valid(const struct cpc_reg *reg)
+{
+ return reg->address <= U32_MAX && reg->bit_width &&
+ reg->bit_width <= 64 &&
+ reg->bit_offset <= 64 - reg->bit_width;
+}
+
int cpc_read_ffh(int cpunum, struct cpc_reg *reg, u64 *val)
{
int err;
+ if (!cpc_ffh_reg_valid(reg))
+ return -EINVAL;
+
err = rdmsrq_safe_on_cpu(cpunum, reg->address, val);
if (!err) {
u64 mask = GENMASK_ULL(reg->bit_offset + reg->bit_width - 1,
@@ -65,6 +77,9 @@ int cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val)
u64 rd_val;
int err;
+ if (!cpc_ffh_reg_valid(reg))
+ return -EINVAL;
+
err = rdmsrq_safe_on_cpu(cpunum, reg->address, &rd_val);
if (!err) {
u64 mask = GENMASK_ULL(reg->bit_offset + reg->bit_width - 1,
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index bd274c2b4c6a..ef55c76a3906 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -524,6 +524,7 @@ static void bsp_init_amd(struct cpuinfo_x86 *c)
case 0x50 ... 0x5f:
case 0x80 ... 0xaf:
case 0xc0 ... 0xcf:
+ case 0xd8 ... 0xef:
setup_force_cpu_cap(X86_FEATURE_ZEN6);
break;
default:
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 90d0488d54c5..cbf5c0ddd3b3 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -2233,14 +2233,12 @@ static void __init bhi_select_mitigation(void)
if (bhi_mitigation != BHI_MITIGATION_AUTO)
return;
- if (cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST)) {
- if (cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL))
- bhi_mitigation = BHI_MITIGATION_ON;
- else
- bhi_mitigation = BHI_MITIGATION_VMEXIT_ONLY;
- } else {
+ if (cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL))
+ bhi_mitigation = BHI_MITIGATION_ON;
+ else if (cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST))
+ bhi_mitigation = BHI_MITIGATION_VMEXIT_ONLY;
+ else
bhi_mitigation = BHI_MITIGATION_OFF;
- }
}
static void __init bhi_update_mitigation(void)
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 6c9e75440c10..6dd132414a95 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -201,8 +201,8 @@ union l3_qos_abmc_cfg {
bw_src :12,
reserved1: 3,
is_clos : 1,
- cntr_id : 5,
- reserved : 9,
+ cntr_id :12,
+ reserved : 2,
cntr_en : 1,
cfg_en : 1;
} split;
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 9fc3adb4b232..d16253607ac4 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -476,8 +476,8 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)
rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL))) {
r->mon.mbm_cntr_assignable = true;
cpuid_count(0x80000020, 5, &eax, &ebx, &ecx, &edx);
- r->mon.num_mbm_cntrs = (ebx & GENMASK(15, 0)) + 1;
- hw_res->mbm_cntr_assign_enabled = true;
+ /* cntr_id is 12 bits and can only encode 4096 counters. */
+ r->mon.num_mbm_cntrs = min((ebx & GENMASK(15, 0)) + 1, BIT(12));
}
r->mon_capable = true;
diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c
index be93ec7255bf..eece49c490e0 100644
--- a/arch/x86/kernel/nmi.c
+++ b/arch/x86/kernel/nmi.c
@@ -114,6 +114,14 @@ static void nmi_check_duration(struct nmiaction *action, u64 duration)
{
int remainder_ns, decimal_msecs;
+ /* sched_clock() went backwards? */
+ if ((s64)duration < 0) {
+ pr_warn_once("NMI handler (%ps): sched_clock() went backwards by %lld ns on CPU%d\n",
+ action->handler, -(s64)duration,
+ smp_processor_id());
+ return;
+ }
+
if (duration < nmi_longest_ns || duration < action->max_duration)
return;
diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c
index 0a2bbd674a6d..c1e20abc7fe5 100644
--- a/arch/x86/kernel/x86_init.c
+++ b/arch/x86/kernel/x86_init.c
@@ -36,7 +36,7 @@ static void iommu_shutdown_noop(void) { }
bool __init bool_x86_init_noop(void) { return false; }
void x86_op_int_noop(int cpu) { }
int set_rtc_noop(const struct timespec64 *now) { return -EINVAL; }
-void get_rtc_noop(struct timespec64 *now) { }
+void get_rtc_noop(struct timespec64 *now) { *now = (struct timespec64){0}; }
static __initconst const struct of_device_id of_cmos_match[] = {
{ .compatible = "motorola,mc146818" },
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 24f22c10e677..3b1fb5696ed6 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -2279,6 +2279,16 @@ static struct kvm_mmu_page *kvm_mmu_find_shadow_page(struct kvm *kvm,
continue;
}
+ /*
+ * Note! Unlike fully direct MMUs, direct SPs in indirect MMUs
+ * can have an arbitrary number of parents, e.g if the guest is
+ * mapping a huge PTE at many points in its page tables.
+ */
+ if (pte_list_count(&sp->parent_ptes) >= RMAP_RECYCLE_THRESHOLD) {
+ kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list);
+ break;
+ }
+
/* unsync and write-flooding only apply to indirect SPs. */
if (sp->role.direct)
goto out;
diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h
index 4e84b5a1eeaa..ec265eb196da 100644
--- a/arch/x86/kvm/mmu/paging_tmpl.h
+++ b/arch/x86/kvm/mmu/paging_tmpl.h
@@ -321,6 +321,13 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,
gpa_t real_gpa;
gfn_t gfn;
+ /*
+ * KVM might sleep/resched when retrying the walk if the updating A/D
+ * fails because the gPTE changed between reading the gPTE and trying
+ * the CMPXHG to set A/D bits.
+ */
+ might_sleep();
+
trace_kvm_mmu_pagetable_walk(addr, access);
retry_walk:
walker->level = mmu->cpu_role.base.level;
@@ -471,8 +478,10 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,
addr, write_fault);
if (unlikely(ret < 0))
goto error;
- else if (ret)
+ else if (ret) {
+ cond_resched();
goto retry_walk;
+ }
}
return 1;
diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index 064a2bb9cf8e..49cc457a4c37 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -1880,6 +1880,7 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
if (!(save->cr0 & X86_CR0_PG) ||
!(save->cr0 & X86_CR0_PE) ||
(save->rflags & X86_EFLAGS_VM) ||
+ ((save->efer & EFER_LMA) && !(save->efer & EFER_LME)) ||
!__nested_vmcb_check_save(vcpu, &save_cached))
goto out_free;
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index ab1558f1f9f8..ffca5fef5b5e 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -5136,6 +5136,7 @@ static void nested_vmx_restore_host_state(struct kvm_vcpu *vcpu)
__func__, j, h.index, h.value);
goto vmabort;
}
+ break;
}
}
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index dcbfcd79791c..43d6b20dfbb1 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -899,28 +899,37 @@ static __always_inline bool tdx_is_exit_reason_valid(u64 vp_enter_ret)
}
}
-static __always_inline u32 tdx_to_vmx_exit_reason(struct kvm_vcpu *vcpu)
+static __always_inline union vmx_exit_reason tdx_to_vmx_exit_reason(struct kvm_vcpu *vcpu)
{
struct vcpu_tdx *tdx = to_tdx(vcpu);
- u32 exit_reason;
+ union vmx_exit_reason exit_reason;
+ /*
+ * Return the synthesized invalid Exit Reason, as the TDX module
+ * never attempted to run the vCPU, i.e. the Exit Reason is undefined,
+ * but this is NOT a failed VM-Enter.
+ */
if (!tdx_is_exit_reason_valid(tdx->vp_enter_ret))
- return -1u;
+ return (union vmx_exit_reason) {
+ .basic = EXIT_REASON_UNDEFINED,
+ };
- exit_reason = tdx->vp_enter_ret;
+ exit_reason.full = (u32)tdx->vp_enter_ret;
- switch (exit_reason) {
+ switch (exit_reason.basic) {
case EXIT_REASON_TDCALL:
if (tdvmcall_exit_type(vcpu))
- return EXIT_REASON_VMCALL;
-
- return tdcall_to_vmx_exit_reason(vcpu);
+ exit_reason.basic = EXIT_REASON_VMCALL;
+ else
+ exit_reason.basic = tdcall_to_vmx_exit_reason(vcpu);
+ break;
case EXIT_REASON_EPT_MISCONFIG:
/*
* Defer KVM_BUG_ON() until tdx_handle_exit() because this is in
* non-instrumentable code with interrupts disabled.
*/
- return -1u;
+ exit_reason.basic = EXIT_REASON_UNDEFINED;
+ break;
default:
break;
}
@@ -937,7 +946,7 @@ static noinstr void tdx_vcpu_enter_exit(struct kvm_vcpu *vcpu)
tdx->vp_enter_ret = tdh_vp_enter(&tdx->vp, &tdx->vp_enter_args);
- vt->exit_reason.full = tdx_to_vmx_exit_reason(vcpu);
+ vt->exit_reason = tdx_to_vmx_exit_reason(vcpu);
vt->exit_qualification = tdx->vp_enter_args.rcx;
tdx->ext_exit_qualification = tdx->vp_enter_args.rdx;
@@ -951,8 +960,7 @@ static noinstr void tdx_vcpu_enter_exit(struct kvm_vcpu *vcpu)
static bool tdx_failed_vmentry(struct kvm_vcpu *vcpu)
{
- return vmx_get_exit_reason(vcpu).failed_vmentry &&
- vmx_get_exit_reason(vcpu).full != -1u;
+ return vmx_get_exit_reason(vcpu).failed_vmentry;
}
static fastpath_t tdx_exit_handlers_fastpath(struct kvm_vcpu *vcpu)
@@ -1041,8 +1049,16 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
* allowing vCPU entry to avoid contention with tdh_vp_enter() and
* TDCALLs.
*/
- if (unlikely(READ_ONCE(to_kvm_tdx(vcpu->kvm)->wait_for_sept_zap)))
+ if (unlikely(READ_ONCE(to_kvm_tdx(vcpu->kvm)->wait_for_sept_zap))) {
+ /*
+ * The vCPU never entered the guest, but this looks like a
+ * handled exit to the caller. Synthesize an invalid exit
+ * reason so the previous exit's stale value isn't consumed
+ * a second time.
+ */
+ vt->exit_reason.full = EXIT_REASON_UNDEFINED;
return EXIT_FASTPATH_EXIT_HANDLED;
+ }
trace_kvm_entry(vcpu, run_flags & KVM_RUN_FORCE_IMMEDIATE_EXIT);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 6576c692a94e..3ca396316d3c 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -9218,10 +9218,12 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
} else if (vcpu->arch.complete_userspace_io) {
writeback = false;
r = 0;
- } else if (r == EMULATION_RESTART)
+ } else if (r == EMULATION_RESTART) {
+ cond_resched();
goto restart;
- else
+ } else {
r = 1;
+ }
writeback:
if (writeback) {
@@ -12276,18 +12278,23 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
kvm_x86_call(set_efer)(vcpu, 0);
kvm_x86_call(update_exception_bitmap)(vcpu);
+ if (old_cr0 & X86_CR0_PG)
+ kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu);
+
/*
- * On the standard CR0/CR4/EFER modification paths, there are several
- * complex conditions determining whether the MMU has to be reset and/or
- * which PCIDs have to be flushed. However, CR0.WP and the paging-related
- * bits in CR4 and EFER are irrelevant if CR0.PG was '0'; and a reset+flush
- * is needed anyway if CR0.PG was '1' (which can only happen for INIT, as
- * CR0 will be '0' prior to RESET). So we only need to check CR0.PG here.
+ * Unconditionally reset the MMU, even though super strictly speaking,
+ * a MMU reset is only necessary if and only one of the page role bits
+ * is changing. All of the CR0/CR4/EFER bits effectively get covered
+ * by checking CR0.PG=1, as CR0.WP and the paging-related bits in CR4
+ * and EFER are irrelevant if CR0.PG was '0'. Subtly, if the vCPU was
+ * in guest mode (L2) or SMM (although emulating INIT in SMM _should_
+ * be impossible), an MMU reset is also necessary as "guest mode" and
+ * "smm" are both encoded in the shadow page role. As a result, the
+ * only scenarios where an MMU reset is *not* needed is if the vCPU is
+ * in Real Mode or Unpaged Protected Mode, and micro-optimizing those
+ * paths are not worth the risk of running with a stale MMU context.
*/
- if (old_cr0 & X86_CR0_PG) {
- kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu);
- kvm_mmu_reset_context(vcpu);
- }
+ kvm_mmu_reset_context(vcpu);
/*
* Intel's SDM states that all TLB entries are flushed on INIT. AMD's
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 5d221709353e..41a81d58cf96 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -319,13 +319,19 @@ static u16 allocate_global_asid(void)
lockdep_assert_held(&global_asid_lock);
- /* The previous allocation hit the edge of available address space */
- if (last_global_asid >= MAX_ASID_AVAILABLE - 1)
- reset_global_asid_space();
-
asid = find_next_zero_bit(global_asid_used, MAX_ASID_AVAILABLE, last_global_asid);
- if (asid >= MAX_ASID_AVAILABLE && !global_asid_available) {
+ /*
+ * Nothing is free between last_global_asid and MAX_ASID_AVAILABLE.
+ * Flush lazy-freed ASIDs and reset the search space.
+ */
+ if (asid >= MAX_ASID_AVAILABLE) {
+ reset_global_asid_space();
+ asid = find_next_zero_bit(global_asid_used, MAX_ASID_AVAILABLE,
+ last_global_asid);
+ }
+
+ if (asid >= MAX_ASID_AVAILABLE) {
/* This should never happen. */
VM_WARN_ONCE(1, "Unable to allocate global ASID despite %d available\n",
global_asid_available);
diff --git a/block/blk-iocost.c b/block/blk-iocost.c
index 5c7cdb493768..8c0214d48e3e 100644
--- a/block/blk-iocost.c
+++ b/block/blk-iocost.c
@@ -2818,6 +2818,9 @@ static void ioc_rqos_done(struct rq_qos *rqos, struct request *rq)
if (!ioc->enabled || !rq->alloc_time_ns || !rq->start_time_ns)
return;
+ if (unlikely(rq->rq_flags & RQF_FLUSH_SEQ))
+ return;
+
switch (req_op(rq)) {
case REQ_OP_READ:
pidx = QOS_RLAT;
diff --git a/block/blk-mq.c b/block/blk-mq.c
index cced68a0e3c5..82df2f6073d8 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -3223,16 +3223,15 @@ void blk_mq_submit_bio(struct bio *bio)
blk_mq_bio_to_request(rq, bio, nr_segs);
+ if (bio_zone_write_plugging(bio))
+ blk_zone_write_plug_init_request(rq);
+
ret = blk_crypto_rq_get_keyslot(rq);
if (ret != BLK_STS_OK) {
- bio_endio_status(bio, ret);
- blk_mq_free_request(rq);
+ blk_mq_end_request(rq, ret);
return;
}
- if (bio_zone_write_plugging(bio))
- blk_zone_write_plug_init_request(rq);
-
if (op_is_flush(bio->bi_opf) && blk_insert_flush(rq))
return;
@@ -3496,8 +3495,10 @@ void blk_mq_free_rqs(struct blk_mq_tag_set *set, struct blk_mq_tags *tags,
if (blk_mq_is_shared_tags(set->flags))
drv_tags = set->shared_tags;
- else
+ else if (hctx_idx < set->nr_hw_queues)
drv_tags = set->tags[hctx_idx];
+ else
+ drv_tags = NULL;
if (tags->static_rqs && set->ops->exit_request) {
int i;
diff --git a/block/bsg.c b/block/bsg.c
index 72157a59b788..7231b1de663f 100644
--- a/block/bsg.c
+++ b/block/bsg.c
@@ -130,7 +130,10 @@ static long bsg_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
case SG_SET_TIMEOUT:
if (get_user(val, intp))
return -EFAULT;
- bd->timeout = clock_t_to_jiffies(val);
+ if (val < 0)
+ return -EIO;
+ bd->timeout = min(clock_t_to_jiffies(val),
+ min(UINT_MAX, MAX_JIFFY_OFFSET));
return 0;
case SG_GET_TIMEOUT:
return jiffies_to_clock_t(bd->timeout);
diff --git a/block/ioctl.c b/block/ioctl.c
index 801f904b5af4..c038a4b27b57 100644
--- a/block/ioctl.c
+++ b/block/ioctl.c
@@ -742,8 +742,8 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg)
return compat_put_ulong(argp, bdev_nr_sectors(bdev));
/* The data is compatible, but the command number is different */
- case BLKBSZGET_32: /* get the logical block size (cf. BLKSSZGET) */
- return put_int(argp, bdev_logical_block_size(bdev));
+ case BLKBSZGET_32: /* get the soft block size (cf. BLKSSZGET) */
+ return put_int(argp, block_size(bdev));
case BLKBSZSET_32:
return blkdev_bszset(file, mode, argp);
case BLKGETSIZE64_32:
diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c
index 1a77aba0475b..f301bee64331 100644
--- a/drivers/acpi/apei/ghes.c
+++ b/drivers/acpi/apei/ghes.c
@@ -608,7 +608,7 @@ static bool ghes_handle_arm_hw_error(struct acpi_hest_generic_data *gdata,
* and don't filter out 'corrected' error here.
*/
if (is_cache && has_pa) {
- queued = ghes_do_memory_failure(err_info->physical_fault_addr, flags);
+ queued |= ghes_do_memory_failure(err_info->physical_fault_addr, flags);
p += err_info->length;
continue;
}
@@ -925,11 +925,11 @@ static void ghes_do_proc(struct ghes *ghes,
atomic_notifier_call_chain(&ghes_report_chain, sev, mem_err);
arch_apei_report_mem_error(sev, mem_err);
- queued = ghes_handle_memory_failure(gdata, sev, sync);
+ queued |= ghes_handle_memory_failure(gdata, sev, sync);
} else if (guid_equal(sec_type, &CPER_SEC_PCIE)) {
ghes_handle_aer(gdata);
} else if (guid_equal(sec_type, &CPER_SEC_PROC_ARM)) {
- queued = ghes_handle_arm_hw_error(gdata, sev, sync);
+ queued |= ghes_handle_arm_hw_error(gdata, sev, sync);
} else if (guid_equal(sec_type, &CPER_SEC_CXL_PROT_ERR)) {
struct cxl_cper_sec_prot_err *prot_err = acpi_hest_get_payload(gdata);
diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c
index 531777557b47..2668004e89ec 100644
--- a/drivers/acpi/cppc_acpi.c
+++ b/drivers/acpi/cppc_acpi.c
@@ -36,6 +36,7 @@
#include <linux/delay.h>
#include <linux/iopoll.h>
#include <linux/ktime.h>
+#include <linux/mutex.h>
#include <linux/rwsem.h>
#include <linux/wait.h>
#include <linux/topology.h>
@@ -93,6 +94,16 @@ static DEFINE_PER_CPU(int, cpu_pcc_subspace_idx);
*/
static DEFINE_PER_CPU(struct cpc_desc *, cpc_desc_ptr);
+/* Protect immutable capability queries against descriptor removal. */
+static DEFINE_MUTEX(cpc_desc_lock);
+
+static void cpc_set_desc(unsigned int cpu, struct cpc_desc *desc)
+{
+ mutex_lock(&cpc_desc_lock);
+ per_cpu(cpc_desc_ptr, cpu) = desc;
+ mutex_unlock(&cpc_desc_lock);
+}
+
/* pcc mapped address + header size + offset within PCC subspace */
#define GET_PCC_VADDR(offs, pcc_ss_id) (pcc_data[pcc_ss_id]->pcc_channel->shmem + \
0x8 + (offs))
@@ -899,12 +910,12 @@ int acpi_cppc_processor_probe(struct acpi_processor *pr)
}
/* Plug PSD data into this CPU's CPC descriptor. */
- per_cpu(cpc_desc_ptr, pr->id) = cpc_ptr;
+ cpc_set_desc(pr->id, cpc_ptr);
ret = kobject_init_and_add(&cpc_ptr->kobj, &cppc_ktype, &cpu_dev->kobj,
"acpi_cppc");
if (ret) {
- per_cpu(cpc_desc_ptr, pr->id) = NULL;
+ cpc_set_desc(pr->id, NULL);
kobject_put(&cpc_ptr->kobj);
goto out_free;
}
@@ -956,6 +967,8 @@ void acpi_cppc_processor_exit(struct acpi_processor *pr)
if (!cpc_ptr)
return;
+ cpc_set_desc(pr->id, NULL);
+
/* Free all the mapped sys mem areas for this CPU */
for (i = 2; i < cpc_ptr->num_entries; i++) {
addr = cpc_ptr->cpc_regs[i-2].sys_mem_vaddr;
@@ -1017,6 +1030,9 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val)
}
*val = 0;
+ if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
+ return cpc_read_ffh(cpu, reg, val);
+
size = GET_BIT_WIDTH(reg);
if (IS_ENABLED(CONFIG_HAS_IOPORT) &&
@@ -1044,8 +1060,6 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val)
}
else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY)
vaddr = reg_res->sys_mem_vaddr;
- else if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
- return cpc_read_ffh(cpu, reg, val);
else
return acpi_os_read_memory((acpi_physical_address)reg->address,
val, size);
@@ -1091,6 +1105,9 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val)
struct cpc_desc *cpc_desc;
unsigned long flags;
+ if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
+ return cpc_write_ffh(cpu, reg, val);
+
size = GET_BIT_WIDTH(reg);
if (IS_ENABLED(CONFIG_HAS_IOPORT) &&
@@ -1116,8 +1133,6 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val)
}
else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY)
vaddr = reg_res->sys_mem_vaddr;
- else if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
- return cpc_write_ffh(cpu, reg, val);
else
return acpi_os_write_memory((acpi_physical_address)reg->address,
val, size);
@@ -1271,6 +1286,11 @@ static int cppc_set_reg_val(int cpu, enum cppc_regs reg_idx, u64 val)
reg = &cpc_desc->cpc_regs[reg_idx];
+ /* Integer 1 describes autonomous selection that is always enabled. */
+ if (reg_idx == AUTO_SEL_ENABLE && reg->type == ACPI_TYPE_INTEGER &&
+ reg->cpc_entry.int_value == 1)
+ return val == 1 ? 0 : -EOPNOTSUPP;
+
/* if a register is writeable, it must be a buffer and not null */
if ((reg->type != ACPI_TYPE_BUFFER) || IS_NULL_REG(®->cpc_entry.reg)) {
pr_debug("CPC register is not supported\n");
@@ -1720,6 +1740,32 @@ int cppc_get_auto_sel(int cpu, bool *enable)
}
EXPORT_SYMBOL_GPL(cppc_get_auto_sel);
+/**
+ * cppc_auto_sel_is_immutable - Check for always-enabled autonomous selection.
+ * @cpu: CPU whose _CPC descriptor to check.
+ *
+ * Context: Process context.
+ * Return: true for Integer 1, false for a register or an absent descriptor.
+ */
+bool cppc_auto_sel_is_immutable(int cpu)
+{
+ struct cpc_desc *cpc_desc;
+ struct cpc_register_resource *reg;
+ bool immutable = false;
+
+ mutex_lock(&cpc_desc_lock);
+ cpc_desc = per_cpu(cpc_desc_ptr, cpu);
+ if (cpc_desc) {
+ reg = &cpc_desc->cpc_regs[AUTO_SEL_ENABLE];
+ immutable = reg->type == ACPI_TYPE_INTEGER &&
+ reg->cpc_entry.int_value == 1;
+ }
+ mutex_unlock(&cpc_desc_lock);
+
+ return immutable;
+}
+EXPORT_SYMBOL_GPL(cppc_auto_sel_is_immutable);
+
/**
* cppc_set_auto_sel - Write autonomous selection register.
* @cpu : CPU to which to write register.
diff --git a/drivers/acpi/riscv/cppc.c b/drivers/acpi/riscv/cppc.c
index 42c1a9052470..4ea4ddedd91f 100644
--- a/drivers/acpi/riscv/cppc.c
+++ b/drivers/acpi/riscv/cppc.c
@@ -97,6 +97,7 @@ bool cpc_ffh_supported(void)
int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val)
{
struct sbi_cppc_data data;
+ int ret;
if (WARN_ON_ONCE(irqs_disabled()))
return -EPERM;
@@ -107,19 +108,27 @@ int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val)
data.reg = FFH_CPPC_SBI_REG(reg->address);
- smp_call_function_single(cpu, sbi_cppc_read, &data, 1);
+ ret = smp_call_function_single(cpu, sbi_cppc_read, &data, 1);
+ if (ret)
+ return ret;
+ if (data.ret.error)
+ return sbi_err_map_linux_errno(data.ret.error);
*val = data.ret.value;
- return (data.ret.error) ? sbi_err_map_linux_errno(data.ret.error) : 0;
+ return 0;
} else if (FFH_CPPC_TYPE(reg->address) == FFH_CPPC_CSR) {
data.reg = FFH_CPPC_CSR_NUM(reg->address);
- smp_call_function_single(cpu, cppc_ffh_csr_read, &data, 1);
+ ret = smp_call_function_single(cpu, cppc_ffh_csr_read, &data, 1);
+ if (ret)
+ return ret;
+ if (data.ret.error)
+ return data.ret.error;
*val = data.ret.value;
- return data.ret.error;
+ return 0;
}
return -EINVAL;
@@ -128,6 +137,7 @@ int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val)
int cpc_write_ffh(int cpu, struct cpc_reg *reg, u64 val)
{
struct sbi_cppc_data data;
+ int ret;
if (WARN_ON_ONCE(irqs_disabled()))
return -EPERM;
@@ -139,14 +149,18 @@ int cpc_write_ffh(int cpu, struct cpc_reg *reg, u64 val)
data.reg = FFH_CPPC_SBI_REG(reg->address);
data.val = val;
- smp_call_function_single(cpu, sbi_cppc_write, &data, 1);
+ ret = smp_call_function_single(cpu, sbi_cppc_write, &data, 1);
+ if (ret)
+ return ret;
return (data.ret.error) ? sbi_err_map_linux_errno(data.ret.error) : 0;
} else if (FFH_CPPC_TYPE(reg->address) == FFH_CPPC_CSR) {
data.reg = FFH_CPPC_CSR_NUM(reg->address);
data.val = val;
- smp_call_function_single(cpu, cppc_ffh_csr_write, &data, 1);
+ ret = smp_call_function_single(cpu, cppc_ffh_csr_write, &data, 1);
+ if (ret)
+ return ret;
return data.ret.error;
}
diff --git a/drivers/ata/Kconfig b/drivers/ata/Kconfig
index 120a2b7067fc..ff81f707d0b2 100644
--- a/drivers/ata/Kconfig
+++ b/drivers/ata/Kconfig
@@ -329,13 +329,6 @@ config SATA_AHCI_SEATTLE
If unsure, say N
-config SATA_INIC162X
- tristate "Initio 162x SATA support (Very Experimental)"
- depends on PCI
- select SATA_HOST
- help
- This option enables support for Initio 162x Serial ATA.
-
config SATA_ACARD_AHCI
tristate "ACard AHCI variant (ATP 8620)"
depends on PCI
diff --git a/drivers/ata/Makefile b/drivers/ata/Makefile
index 20e6645ab737..81e04e96b807 100644
--- a/drivers/ata/Makefile
+++ b/drivers/ata/Makefile
@@ -9,7 +9,6 @@ obj-$(CONFIG_SATA_AHCI_SEATTLE) += ahci_seattle.o libahci.o libahci_platform.o
obj-$(CONFIG_SATA_AHCI_PLATFORM) += ahci_platform.o libahci.o libahci_platform.o
obj-$(CONFIG_SATA_FSL) += sata_fsl.o
obj-$(CONFIG_SATA_GEMINI) += sata_gemini.o
-obj-$(CONFIG_SATA_INIC162X) += sata_inic162x.o
obj-$(CONFIG_SATA_SIL24) += sata_sil24.o
obj-$(CONFIG_SATA_DWC) += sata_dwc_460ex.o
obj-$(CONFIG_SATA_HIGHBANK) += sata_highbank.o libahci.o
diff --git a/drivers/ata/libata-eh.c b/drivers/ata/libata-eh.c
index f21efb0f6419..9d35b4f8d392 100644
--- a/drivers/ata/libata-eh.c
+++ b/drivers/ata/libata-eh.c
@@ -2812,10 +2812,9 @@ static bool ata_eh_followup_srst_needed(struct ata_link *link, int rc)
return false;
}
-int ata_eh_reset(struct ata_link *link, int classify,
+int ata_eh_reset(struct ata_port *ap, struct ata_link *link, int classify,
struct ata_reset_operations *reset_ops)
{
- struct ata_port *ap = link->ap;
struct ata_link *slave = ap->slave_link;
struct ata_eh_context *ehc = &link->eh_context;
struct ata_eh_context *sehc = slave ? &slave->eh_context : NULL;
@@ -3863,7 +3862,8 @@ int ata_eh_recover(struct ata_port *ap, struct ata_reset_operations *reset_ops,
if (!(ehc->i.action & ATA_EH_RESET))
continue;
- rc = ata_eh_reset(link, ata_link_nr_vacant(link), reset_ops);
+ rc = ata_eh_reset(ap, link, ata_link_nr_vacant(link),
+ reset_ops);
if (rc) {
ata_link_err(link, "reset failed, giving up\n");
goto out;
diff --git a/drivers/ata/libata-pmp.c b/drivers/ata/libata-pmp.c
index 299b1823e0d0..c3b3af28d17c 100644
--- a/drivers/ata/libata-pmp.c
+++ b/drivers/ata/libata-pmp.c
@@ -482,6 +482,20 @@ static void sata_pmp_quirks(struct ata_port *ap)
if (link->pmp == 4)
link->flags |= ATA_LFLAG_DISABLED;
}
+ } else if (vendor == 0x1b4b && devid == 0x9705) {
+ /*
+ * Marvell 88SM9705 quirks
+ *
+ * 1-to-5 PMP with an extra SEMB port. It reports 6 ports and
+ * port 5 is the SEMB, which never answers SRST: EH spends about
+ * a minute per round retrying it, and disk discovery waits for
+ * that. Seen behind a Marvell 88SE9235 in the Synology DX1215
+ * and DX1222.
+ */
+ ata_for_each_link(link, ap, EDGE) {
+ if (link->pmp == 5)
+ link->flags |= ATA_LFLAG_DISABLED;
+ }
}
}
@@ -782,7 +796,7 @@ static int sata_pmp_eh_recover_pmp(struct ata_port *ap,
struct ata_link *tlink;
/* reset */
- rc = ata_eh_reset(link, 0, reset_ops);
+ rc = ata_eh_reset(ap, link, 0, reset_ops);
if (rc) {
ata_link_err(link, "failed to reset PMP, giving up\n");
goto fail;
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 8b9530c50531..0988bf574b81 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1977,6 +1977,18 @@ static int ata_scsi_translate(struct ata_device *dev, struct scsi_cmnd *cmd,
if (xlat_func(qc))
goto done;
+ /*
+ * A data-out request may contain pages pinned without write access.
+ * Reject a translated data-in command before issuing it, since its
+ * completion handler may write to the buffer even if the device fails.
+ */
+ if (unlikely(cmd->sc_data_direction == DMA_TO_DEVICE &&
+ ata_is_data(qc->tf.protocol) &&
+ !(qc->tf.flags & ATA_TFLAG_WRITE))) {
+ ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0x4b, 0x0a);
+ goto done;
+ }
+
return ata_scsi_qc_issue(ap, qc);
done:
@@ -2008,6 +2020,12 @@ static void ata_scsi_rbuf_fill(struct ata_device *dev, struct scsi_cmnd *cmd,
unsigned long flags;
unsigned int len;
+ /* A data-out request may contain pages pinned without write access. */
+ if (cmd->sc_data_direction == DMA_TO_DEVICE) {
+ ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0x4b, 0x0a);
+ return;
+ }
+
spin_lock_irqsave(&ata_scsi_rbuf_lock, flags);
memset(ata_scsi_rbuf, 0, ATA_SCSI_RBUF_SIZE);
@@ -3065,6 +3083,13 @@ static unsigned int atapi_xlat(struct ata_queued_cmd *qc)
qc->complete_fn = atapi_qc_complete;
+ /* atapi_fixup_inquiry() writes to the buffer on successful completion. */
+ if (scmd->sc_data_direction == DMA_TO_DEVICE &&
+ scmd->cmnd[0] == INQUIRY) {
+ ata_scsi_set_sense(dev, scmd, ABORTED_COMMAND, 0x4b, 0x0a);
+ return 1;
+ }
+
qc->tf.flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_DEVICE;
if (scmd->sc_data_direction == DMA_TO_DEVICE) {
qc->tf.flags |= ATA_TFLAG_WRITE;
diff --git a/drivers/ata/libata.h b/drivers/ata/libata.h
index 9f96ca2acfd9..506a649b73d5 100644
--- a/drivers/ata/libata.h
+++ b/drivers/ata/libata.h
@@ -192,8 +192,8 @@ extern void ata_eh_done(struct ata_link *link, struct ata_device *dev,
extern void ata_eh_autopsy(struct ata_port *ap);
const char *ata_get_cmd_name(u8 command);
extern void ata_eh_report(struct ata_port *ap);
-extern int ata_eh_reset(struct ata_link *link, int classify,
- struct ata_reset_operations *reset_ops);
+extern int ata_eh_reset(struct ata_port *ap, struct ata_link *link,
+ int classify, struct ata_reset_operations *reset_ops);
extern int ata_eh_recover(struct ata_port *ap,
struct ata_reset_operations *reset_ops,
struct ata_link **r_failed_disk);
diff --git a/drivers/ata/sata_inic162x.c b/drivers/ata/sata_inic162x.c
deleted file mode 100644
index 46a8c20daf18..000000000000
--- a/drivers/ata/sata_inic162x.c
+++ /dev/null
@@ -1,902 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * sata_inic162x.c - Driver for Initio 162x SATA controllers
- *
- * Copyright 2006 SUSE Linux Products GmbH
- * Copyright 2006 Tejun Heo <teheo@novell.com>
- *
- * **** WARNING ****
- *
- * This driver never worked properly and unfortunately data corruption is
- * relatively common. There isn't anyone working on the driver and there's
- * no support from the vendor. Do not use this driver in any production
- * environment.
- *
- * http://thread.gmane.org/gmane.linux.debian.devel.bugs.rc/378525/focus=54491
- * https://bugzilla.kernel.org/show_bug.cgi?id=60565
- *
- * *****************
- *
- * This controller is eccentric and easily locks up if something isn't
- * right. Documentation is available at initio's website but it only
- * documents registers (not programming model).
- *
- * This driver has interesting history. The first version was written
- * from the documentation and a 2.4 IDE driver posted on a Taiwan
- * company, which didn't use any IDMA features and couldn't handle
- * LBA48. The resulting driver couldn't handle LBA48 devices either
- * making it pretty useless.
- *
- * After a while, initio picked the driver up, renamed it to
- * sata_initio162x, updated it to use IDMA for ATA DMA commands and
- * posted it on their website. It only used ATA_PROT_DMA for IDMA and
- * attaching both devices and issuing IDMA and !IDMA commands
- * simultaneously broke it due to PIRQ masking interaction but it did
- * show how to use the IDMA (ADMA + some initio specific twists)
- * engine.
- *
- * Then, I picked up their changes again and here's the usable driver
- * which uses IDMA for everything. Everything works now including
- * LBA48, CD/DVD burning, suspend/resume and hotplug. There are some
- * issues tho. Result Tf is not resported properly, NCQ isn't
- * supported yet and CD/DVD writing works with DMA assisted PIO
- * protocol (which, for native SATA devices, shouldn't cause any
- * noticeable difference).
- *
- * Anyways, so, here's finally a working driver for inic162x. Enjoy!
- *
- * initio: If you guys wanna improve the driver regarding result TF
- * access and other stuff, please feel free to contact me. I'll be
- * happy to assist.
- */
-
-#include <linux/gfp.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/pci.h>
-#include <scsi/scsi_host.h>
-#include <linux/libata.h>
-#include <linux/blkdev.h>
-#include <scsi/scsi_device.h>
-
-#define DRV_NAME "sata_inic162x"
-#define DRV_VERSION "0.4"
-
-enum {
- MMIO_BAR_PCI = 5,
- MMIO_BAR_CARDBUS = 1,
-
- NR_PORTS = 2,
-
- IDMA_CPB_TBL_SIZE = 4 * 32,
-
- INIC_DMA_BOUNDARY = 0xffffff,
-
- HOST_ACTRL = 0x08,
- HOST_CTL = 0x7c,
- HOST_STAT = 0x7e,
- HOST_IRQ_STAT = 0xbc,
- HOST_IRQ_MASK = 0xbe,
-
- PORT_SIZE = 0x40,
-
- /* registers for ATA TF operation */
- PORT_TF_DATA = 0x00,
- PORT_TF_FEATURE = 0x01,
- PORT_TF_NSECT = 0x02,
- PORT_TF_LBAL = 0x03,
- PORT_TF_LBAM = 0x04,
- PORT_TF_LBAH = 0x05,
- PORT_TF_DEVICE = 0x06,
- PORT_TF_COMMAND = 0x07,
- PORT_TF_ALT_STAT = 0x08,
- PORT_IRQ_STAT = 0x09,
- PORT_IRQ_MASK = 0x0a,
- PORT_PRD_CTL = 0x0b,
- PORT_PRD_ADDR = 0x0c,
- PORT_PRD_XFERLEN = 0x10,
- PORT_CPB_CPBLAR = 0x18,
- PORT_CPB_PTQFIFO = 0x1c,
-
- /* IDMA register */
- PORT_IDMA_CTL = 0x14,
- PORT_IDMA_STAT = 0x16,
-
- PORT_RPQ_FIFO = 0x1e,
- PORT_RPQ_CNT = 0x1f,
-
- PORT_SCR = 0x20,
-
- /* HOST_CTL bits */
- HCTL_LEDEN = (1 << 3), /* enable LED operation */
- HCTL_IRQOFF = (1 << 8), /* global IRQ off */
- HCTL_FTHD0 = (1 << 10), /* fifo threshold 0 */
- HCTL_FTHD1 = (1 << 11), /* fifo threshold 1*/
- HCTL_PWRDWN = (1 << 12), /* power down PHYs */
- HCTL_SOFTRST = (1 << 13), /* global reset (no phy reset) */
- HCTL_RPGSEL = (1 << 15), /* register page select */
-
- HCTL_KNOWN_BITS = HCTL_IRQOFF | HCTL_PWRDWN | HCTL_SOFTRST |
- HCTL_RPGSEL,
-
- /* HOST_IRQ_(STAT|MASK) bits */
- HIRQ_PORT0 = (1 << 0),
- HIRQ_PORT1 = (1 << 1),
- HIRQ_SOFT = (1 << 14),
- HIRQ_GLOBAL = (1 << 15), /* STAT only */
-
- /* PORT_IRQ_(STAT|MASK) bits */
- PIRQ_OFFLINE = (1 << 0), /* device unplugged */
- PIRQ_ONLINE = (1 << 1), /* device plugged */
- PIRQ_COMPLETE = (1 << 2), /* completion interrupt */
- PIRQ_FATAL = (1 << 3), /* fatal error */
- PIRQ_ATA = (1 << 4), /* ATA interrupt */
- PIRQ_REPLY = (1 << 5), /* reply FIFO not empty */
- PIRQ_PENDING = (1 << 7), /* port IRQ pending (STAT only) */
-
- PIRQ_ERR = PIRQ_OFFLINE | PIRQ_ONLINE | PIRQ_FATAL,
- PIRQ_MASK_DEFAULT = PIRQ_REPLY | PIRQ_ATA,
- PIRQ_MASK_FREEZE = 0xff,
-
- /* PORT_PRD_CTL bits */
- PRD_CTL_START = (1 << 0),
- PRD_CTL_WR = (1 << 3),
- PRD_CTL_DMAEN = (1 << 7), /* DMA enable */
-
- /* PORT_IDMA_CTL bits */
- IDMA_CTL_RST_ATA = (1 << 2), /* hardreset ATA bus */
- IDMA_CTL_RST_IDMA = (1 << 5), /* reset IDMA machinery */
- IDMA_CTL_GO = (1 << 7), /* IDMA mode go */
- IDMA_CTL_ATA_NIEN = (1 << 8), /* ATA IRQ disable */
-
- /* PORT_IDMA_STAT bits */
- IDMA_STAT_PERR = (1 << 0), /* PCI ERROR MODE */
- IDMA_STAT_CPBERR = (1 << 1), /* ADMA CPB error */
- IDMA_STAT_LGCY = (1 << 3), /* ADMA legacy */
- IDMA_STAT_UIRQ = (1 << 4), /* ADMA unsolicited irq */
- IDMA_STAT_STPD = (1 << 5), /* ADMA stopped */
- IDMA_STAT_PSD = (1 << 6), /* ADMA pause */
- IDMA_STAT_DONE = (1 << 7), /* ADMA done */
-
- IDMA_STAT_ERR = IDMA_STAT_PERR | IDMA_STAT_CPBERR,
-
- /* CPB Control Flags*/
- CPB_CTL_VALID = (1 << 0), /* CPB valid */
- CPB_CTL_QUEUED = (1 << 1), /* queued command */
- CPB_CTL_DATA = (1 << 2), /* data, rsvd in datasheet */
- CPB_CTL_IEN = (1 << 3), /* PCI interrupt enable */
- CPB_CTL_DEVDIR = (1 << 4), /* device direction control */
-
- /* CPB Response Flags */
- CPB_RESP_DONE = (1 << 0), /* ATA command complete */
- CPB_RESP_REL = (1 << 1), /* ATA release */
- CPB_RESP_IGNORED = (1 << 2), /* CPB ignored */
- CPB_RESP_ATA_ERR = (1 << 3), /* ATA command error */
- CPB_RESP_SPURIOUS = (1 << 4), /* ATA spurious interrupt error */
- CPB_RESP_UNDERFLOW = (1 << 5), /* APRD deficiency length error */
- CPB_RESP_OVERFLOW = (1 << 6), /* APRD exccess length error */
- CPB_RESP_CPB_ERR = (1 << 7), /* CPB error flag */
-
- /* PRD Control Flags */
- PRD_DRAIN = (1 << 1), /* ignore data excess */
- PRD_CDB = (1 << 2), /* atapi packet command pointer */
- PRD_DIRECT_INTR = (1 << 3), /* direct interrupt */
- PRD_DMA = (1 << 4), /* data transfer method */
- PRD_WRITE = (1 << 5), /* data dir, rsvd in datasheet */
- PRD_IOM = (1 << 6), /* io/memory transfer */
- PRD_END = (1 << 7), /* APRD chain end */
-};
-
-/* Comman Parameter Block */
-struct inic_cpb {
- u8 resp_flags; /* Response Flags */
- u8 error; /* ATA Error */
- u8 status; /* ATA Status */
- u8 ctl_flags; /* Control Flags */
- __le32 len; /* Total Transfer Length */
- __le32 prd; /* First PRD pointer */
- u8 rsvd[4];
- /* 16 bytes */
- u8 feature; /* ATA Feature */
- u8 hob_feature; /* ATA Ex. Feature */
- u8 device; /* ATA Device/Head */
- u8 mirctl; /* Mirror Control */
- u8 nsect; /* ATA Sector Count */
- u8 hob_nsect; /* ATA Ex. Sector Count */
- u8 lbal; /* ATA Sector Number */
- u8 hob_lbal; /* ATA Ex. Sector Number */
- u8 lbam; /* ATA Cylinder Low */
- u8 hob_lbam; /* ATA Ex. Cylinder Low */
- u8 lbah; /* ATA Cylinder High */
- u8 hob_lbah; /* ATA Ex. Cylinder High */
- u8 command; /* ATA Command */
- u8 ctl; /* ATA Control */
- u8 slave_error; /* Slave ATA Error */
- u8 slave_status; /* Slave ATA Status */
- /* 32 bytes */
-} __packed;
-
-/* Physical Region Descriptor */
-struct inic_prd {
- __le32 mad; /* Physical Memory Address */
- __le16 len; /* Transfer Length */
- u8 rsvd;
- u8 flags; /* Control Flags */
-} __packed;
-
-struct inic_pkt {
- struct inic_cpb cpb;
- struct inic_prd prd[LIBATA_MAX_PRD + 1]; /* + 1 for cdb */
- u8 cdb[ATAPI_CDB_LEN];
-} __packed;
-
-struct inic_host_priv {
- void __iomem *mmio_base;
- u16 cached_hctl;
-};
-
-struct inic_port_priv {
- struct inic_pkt *pkt;
- dma_addr_t pkt_dma;
- u32 *cpb_tbl;
- dma_addr_t cpb_tbl_dma;
-};
-
-static const struct scsi_host_template inic_sht = {
- ATA_BASE_SHT(DRV_NAME),
- .sg_tablesize = LIBATA_MAX_PRD, /* maybe it can be larger? */
-
- /*
- * This controller is braindamaged. dma_boundary is 0xffff like others
- * but it will lock up the whole machine HARD if 65536 byte PRD entry
- * is fed. Reduce maximum segment size.
- */
- .dma_boundary = INIC_DMA_BOUNDARY,
- .max_segment_size = 65536 - 512,
-};
-
-static const int scr_map[] = {
- [SCR_STATUS] = 0,
- [SCR_ERROR] = 1,
- [SCR_CONTROL] = 2,
-};
-
-static void __iomem *inic_port_base(struct ata_port *ap)
-{
- struct inic_host_priv *hpriv = ap->host->private_data;
-
- return hpriv->mmio_base + ap->port_no * PORT_SIZE;
-}
-
-static void inic_reset_port(void __iomem *port_base)
-{
- void __iomem *idma_ctl = port_base + PORT_IDMA_CTL;
-
- /* stop IDMA engine */
- readw(idma_ctl); /* flush */
- msleep(1);
-
- /* mask IRQ and assert reset */
- writew(IDMA_CTL_RST_IDMA, idma_ctl);
- readw(idma_ctl); /* flush */
- msleep(1);
-
- /* release reset */
- writew(0, idma_ctl);
-
- /* clear irq */
- writeb(0xff, port_base + PORT_IRQ_STAT);
-}
-
-static int inic_scr_read(struct ata_link *link, unsigned sc_reg, u32 *val)
-{
- void __iomem *scr_addr = inic_port_base(link->ap) + PORT_SCR;
-
- if (unlikely(sc_reg >= ARRAY_SIZE(scr_map)))
- return -EINVAL;
-
- *val = readl(scr_addr + scr_map[sc_reg] * 4);
-
- /* this controller has stuck DIAG.N, ignore it */
- if (sc_reg == SCR_ERROR)
- *val &= ~SERR_PHYRDY_CHG;
- return 0;
-}
-
-static int inic_scr_write(struct ata_link *link, unsigned sc_reg, u32 val)
-{
- void __iomem *scr_addr = inic_port_base(link->ap) + PORT_SCR;
-
- if (unlikely(sc_reg >= ARRAY_SIZE(scr_map)))
- return -EINVAL;
-
- writel(val, scr_addr + scr_map[sc_reg] * 4);
- return 0;
-}
-
-static void inic_stop_idma(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- readb(port_base + PORT_RPQ_FIFO);
- readb(port_base + PORT_RPQ_CNT);
- writew(0, port_base + PORT_IDMA_CTL);
-}
-
-static void inic_host_err_intr(struct ata_port *ap, u8 irq_stat, u16 idma_stat)
-{
- struct ata_eh_info *ehi = &ap->link.eh_info;
- struct inic_port_priv *pp = ap->private_data;
- struct inic_cpb *cpb = &pp->pkt->cpb;
- bool freeze = false;
-
- ata_ehi_clear_desc(ehi);
- ata_ehi_push_desc(ehi, "irq_stat=0x%x idma_stat=0x%x",
- irq_stat, idma_stat);
-
- inic_stop_idma(ap);
-
- if (irq_stat & (PIRQ_OFFLINE | PIRQ_ONLINE)) {
- ata_ehi_push_desc(ehi, "hotplug");
- ata_ehi_hotplugged(ehi);
- freeze = true;
- }
-
- if (idma_stat & IDMA_STAT_PERR) {
- ata_ehi_push_desc(ehi, "PCI error");
- freeze = true;
- }
-
- if (idma_stat & IDMA_STAT_CPBERR) {
- ata_ehi_push_desc(ehi, "CPB error");
-
- if (cpb->resp_flags & CPB_RESP_IGNORED) {
- __ata_ehi_push_desc(ehi, " ignored");
- ehi->err_mask |= AC_ERR_INVALID;
- freeze = true;
- }
-
- if (cpb->resp_flags & CPB_RESP_ATA_ERR)
- ehi->err_mask |= AC_ERR_DEV;
-
- if (cpb->resp_flags & CPB_RESP_SPURIOUS) {
- __ata_ehi_push_desc(ehi, " spurious-intr");
- ehi->err_mask |= AC_ERR_HSM;
- freeze = true;
- }
-
- if (cpb->resp_flags &
- (CPB_RESP_UNDERFLOW | CPB_RESP_OVERFLOW)) {
- __ata_ehi_push_desc(ehi, " data-over/underflow");
- ehi->err_mask |= AC_ERR_HSM;
- freeze = true;
- }
- }
-
- if (freeze)
- ata_port_freeze(ap);
- else
- ata_port_abort(ap);
-}
-
-static void inic_host_intr(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
- struct ata_queued_cmd *qc = ata_qc_from_tag(ap, ap->link.active_tag);
- u8 irq_stat;
- u16 idma_stat;
-
- /* read and clear IRQ status */
- irq_stat = readb(port_base + PORT_IRQ_STAT);
- writeb(irq_stat, port_base + PORT_IRQ_STAT);
- idma_stat = readw(port_base + PORT_IDMA_STAT);
-
- if (unlikely((irq_stat & PIRQ_ERR) || (idma_stat & IDMA_STAT_ERR)))
- inic_host_err_intr(ap, irq_stat, idma_stat);
-
- if (unlikely(!qc))
- goto spurious;
-
- if (likely(idma_stat & IDMA_STAT_DONE)) {
- inic_stop_idma(ap);
-
- /* Depending on circumstances, device error
- * isn't reported by IDMA, check it explicitly.
- */
- if (unlikely(readb(port_base + PORT_TF_COMMAND) &
- (ATA_DF | ATA_ERR)))
- qc->err_mask |= AC_ERR_DEV;
-
- ata_qc_complete(qc);
- return;
- }
-
- spurious:
- ata_port_warn(ap, "unhandled interrupt: cmd=0x%x irq_stat=0x%x idma_stat=0x%x\n",
- qc ? qc->tf.command : 0xff, irq_stat, idma_stat);
-}
-
-static irqreturn_t inic_interrupt(int irq, void *dev_instance)
-{
- struct ata_host *host = dev_instance;
- struct inic_host_priv *hpriv = host->private_data;
- u16 host_irq_stat;
- int i, handled = 0;
-
- host_irq_stat = readw(hpriv->mmio_base + HOST_IRQ_STAT);
-
- if (unlikely(!(host_irq_stat & HIRQ_GLOBAL)))
- goto out;
-
- spin_lock(&host->lock);
-
- for (i = 0; i < NR_PORTS; i++)
- if (host_irq_stat & (HIRQ_PORT0 << i)) {
- inic_host_intr(host->ports[i]);
- handled++;
- }
-
- spin_unlock(&host->lock);
-
- out:
- return IRQ_RETVAL(handled);
-}
-
-static int inic_check_atapi_dma(struct ata_queued_cmd *qc)
-{
- /* For some reason ATAPI_PROT_DMA doesn't work for some
- * commands including writes and other misc ops. Use PIO
- * protocol instead, which BTW is driven by the DMA engine
- * anyway, so it shouldn't make much difference for native
- * SATA devices.
- */
- if (atapi_cmd_type(qc->cdb[0]) == READ)
- return 0;
- return 1;
-}
-
-static void inic_fill_sg(struct inic_prd *prd, struct ata_queued_cmd *qc)
-{
- struct scatterlist *sg;
- unsigned int si;
- u8 flags = 0;
-
- if (qc->tf.flags & ATA_TFLAG_WRITE)
- flags |= PRD_WRITE;
-
- if (ata_is_dma(qc->tf.protocol))
- flags |= PRD_DMA;
-
- for_each_sg(qc->sg, sg, qc->n_elem, si) {
- prd->mad = cpu_to_le32(sg_dma_address(sg));
- prd->len = cpu_to_le16(sg_dma_len(sg));
- prd->flags = flags;
- prd++;
- }
-
- WARN_ON(!si);
- prd[-1].flags |= PRD_END;
-}
-
-static enum ata_completion_errors inic_qc_prep(struct ata_queued_cmd *qc)
-{
- struct inic_port_priv *pp = qc->ap->private_data;
- struct inic_pkt *pkt = pp->pkt;
- struct inic_cpb *cpb = &pkt->cpb;
- struct inic_prd *prd = pkt->prd;
- bool is_atapi = ata_is_atapi(qc->tf.protocol);
- bool is_data = ata_is_data(qc->tf.protocol);
- unsigned int cdb_len = 0;
-
- if (is_atapi)
- cdb_len = qc->dev->cdb_len;
-
- /* prepare packet, based on initio driver */
- memset(pkt, 0, sizeof(struct inic_pkt));
-
- cpb->ctl_flags = CPB_CTL_VALID | CPB_CTL_IEN;
- if (is_atapi || is_data)
- cpb->ctl_flags |= CPB_CTL_DATA;
-
- cpb->len = cpu_to_le32(qc->nbytes + cdb_len);
- cpb->prd = cpu_to_le32(pp->pkt_dma + offsetof(struct inic_pkt, prd));
-
- cpb->device = qc->tf.device;
- cpb->feature = qc->tf.feature;
- cpb->nsect = qc->tf.nsect;
- cpb->lbal = qc->tf.lbal;
- cpb->lbam = qc->tf.lbam;
- cpb->lbah = qc->tf.lbah;
-
- if (qc->tf.flags & ATA_TFLAG_LBA48) {
- cpb->hob_feature = qc->tf.hob_feature;
- cpb->hob_nsect = qc->tf.hob_nsect;
- cpb->hob_lbal = qc->tf.hob_lbal;
- cpb->hob_lbam = qc->tf.hob_lbam;
- cpb->hob_lbah = qc->tf.hob_lbah;
- }
-
- cpb->command = qc->tf.command;
- /* don't load ctl - dunno why. it's like that in the initio driver */
-
- /* setup PRD for CDB */
- if (is_atapi) {
- memcpy(pkt->cdb, qc->cdb, ATAPI_CDB_LEN);
- prd->mad = cpu_to_le32(pp->pkt_dma +
- offsetof(struct inic_pkt, cdb));
- prd->len = cpu_to_le16(cdb_len);
- prd->flags = PRD_CDB | PRD_WRITE;
- if (!is_data)
- prd->flags |= PRD_END;
- prd++;
- }
-
- /* setup sg table */
- if (is_data)
- inic_fill_sg(prd, qc);
-
- pp->cpb_tbl[0] = pp->pkt_dma;
-
- return AC_ERR_OK;
-}
-
-static unsigned int inic_qc_issue(struct ata_queued_cmd *qc)
-{
- struct ata_port *ap = qc->ap;
- void __iomem *port_base = inic_port_base(ap);
-
- /* fire up the ADMA engine */
- writew(HCTL_FTHD0 | HCTL_LEDEN, port_base + HOST_CTL);
- writew(IDMA_CTL_GO, port_base + PORT_IDMA_CTL);
- writeb(0, port_base + PORT_CPB_PTQFIFO);
-
- return 0;
-}
-
-static void inic_tf_read(struct ata_port *ap, struct ata_taskfile *tf)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- tf->error = readb(port_base + PORT_TF_FEATURE);
- tf->nsect = readb(port_base + PORT_TF_NSECT);
- tf->lbal = readb(port_base + PORT_TF_LBAL);
- tf->lbam = readb(port_base + PORT_TF_LBAM);
- tf->lbah = readb(port_base + PORT_TF_LBAH);
- tf->device = readb(port_base + PORT_TF_DEVICE);
- tf->status = readb(port_base + PORT_TF_COMMAND);
-}
-
-static void inic_qc_fill_rtf(struct ata_queued_cmd *qc)
-{
- struct ata_taskfile *rtf = &qc->result_tf;
- struct ata_taskfile tf;
-
- /* FIXME: Except for status and error, result TF access
- * doesn't work. I tried reading from BAR0/2, CPB and BAR5.
- * None works regardless of which command interface is used.
- * For now return true iff status indicates device error.
- * This means that we're reporting bogus sector for RW
- * failures. Eeekk....
- */
- inic_tf_read(qc->ap, &tf);
-
- if (tf.status & ATA_ERR) {
- rtf->status = tf.status;
- rtf->error = tf.error;
- }
-}
-
-static void inic_freeze(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- writeb(PIRQ_MASK_FREEZE, port_base + PORT_IRQ_MASK);
- writeb(0xff, port_base + PORT_IRQ_STAT);
-}
-
-static void inic_thaw(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- writeb(0xff, port_base + PORT_IRQ_STAT);
- writeb(PIRQ_MASK_DEFAULT, port_base + PORT_IRQ_MASK);
-}
-
-static int inic_check_ready(struct ata_link *link)
-{
- void __iomem *port_base = inic_port_base(link->ap);
-
- return ata_check_ready(readb(port_base + PORT_TF_COMMAND));
-}
-
-/*
- * SRST and SControl hardreset don't give valid signature on this
- * controller. Only controller specific hardreset mechanism works.
- */
-static int inic_hardreset(struct ata_link *link, unsigned int *class,
- unsigned long deadline)
-{
- struct ata_port *ap = link->ap;
- void __iomem *port_base = inic_port_base(ap);
- void __iomem *idma_ctl = port_base + PORT_IDMA_CTL;
- const unsigned int *timing = sata_ehc_deb_timing(&link->eh_context);
- int rc;
-
- /* hammer it into sane state */
- inic_reset_port(port_base);
-
- writew(IDMA_CTL_RST_ATA, idma_ctl);
- readw(idma_ctl); /* flush */
- ata_msleep(ap, 1);
- writew(0, idma_ctl);
-
- rc = sata_link_resume(link, timing, deadline);
- if (rc) {
- ata_link_warn(link,
- "failed to resume link after reset (errno=%d)\n",
- rc);
- return rc;
- }
-
- *class = ATA_DEV_NONE;
- if (ata_link_online(link)) {
- struct ata_taskfile tf;
-
- /* wait for link to become ready */
- rc = ata_wait_after_reset(link, deadline, inic_check_ready);
- /* link occupied, -ENODEV too is an error */
- if (rc) {
- ata_link_warn(link,
- "device not ready after hardreset (errno=%d)\n",
- rc);
- return rc;
- }
-
- inic_tf_read(ap, &tf);
- *class = ata_port_classify(ap, &tf);
- }
-
- return 0;
-}
-
-static void inic_error_handler(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- inic_reset_port(port_base);
- ata_std_error_handler(ap);
-}
-
-static void inic_post_internal_cmd(struct ata_queued_cmd *qc)
-{
- /* make DMA engine forget about the failed command */
- if (qc->flags & ATA_QCFLAG_EH)
- inic_reset_port(inic_port_base(qc->ap));
-}
-
-static void init_port(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
- struct inic_port_priv *pp = ap->private_data;
-
- /* clear packet and CPB table */
- memset(pp->pkt, 0, sizeof(struct inic_pkt));
- memset(pp->cpb_tbl, 0, IDMA_CPB_TBL_SIZE);
-
- /* setup CPB lookup table addresses */
- writel(pp->cpb_tbl_dma, port_base + PORT_CPB_CPBLAR);
-}
-
-static int inic_port_resume(struct ata_port *ap)
-{
- init_port(ap);
- return 0;
-}
-
-static int inic_port_start(struct ata_port *ap)
-{
- struct device *dev = ap->host->dev;
- struct inic_port_priv *pp;
-
- /* alloc and initialize private data */
- pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
- if (!pp)
- return -ENOMEM;
- ap->private_data = pp;
-
- /* Alloc resources */
- pp->pkt = dmam_alloc_coherent(dev, sizeof(struct inic_pkt),
- &pp->pkt_dma, GFP_KERNEL);
- if (!pp->pkt)
- return -ENOMEM;
-
- pp->cpb_tbl = dmam_alloc_coherent(dev, IDMA_CPB_TBL_SIZE,
- &pp->cpb_tbl_dma, GFP_KERNEL);
- if (!pp->cpb_tbl)
- return -ENOMEM;
-
- init_port(ap);
-
- return 0;
-}
-
-static struct ata_port_operations inic_port_ops = {
- .inherits = &sata_port_ops,
-
- .check_atapi_dma = inic_check_atapi_dma,
- .qc_prep = inic_qc_prep,
- .qc_issue = inic_qc_issue,
- .qc_fill_rtf = inic_qc_fill_rtf,
-
- .freeze = inic_freeze,
- .thaw = inic_thaw,
- .reset.hardreset = inic_hardreset,
- .error_handler = inic_error_handler,
- .post_internal_cmd = inic_post_internal_cmd,
-
- .scr_read = inic_scr_read,
- .scr_write = inic_scr_write,
-
- .port_resume = inic_port_resume,
- .port_start = inic_port_start,
-};
-
-static const struct ata_port_info inic_port_info = {
- .flags = ATA_FLAG_SATA | ATA_FLAG_PIO_DMA,
- .pio_mask = ATA_PIO4,
- .mwdma_mask = ATA_MWDMA2,
- .udma_mask = ATA_UDMA6,
- .port_ops = &inic_port_ops
-};
-
-static int init_controller(void __iomem *mmio_base, u16 hctl)
-{
- int i;
- u16 val;
-
- hctl &= ~HCTL_KNOWN_BITS;
-
- /* Soft reset whole controller. Spec says reset duration is 3
- * PCI clocks, be generous and give it 10ms.
- */
- writew(hctl | HCTL_SOFTRST, mmio_base + HOST_CTL);
- readw(mmio_base + HOST_CTL); /* flush */
-
- for (i = 0; i < 10; i++) {
- msleep(1);
- val = readw(mmio_base + HOST_CTL);
- if (!(val & HCTL_SOFTRST))
- break;
- }
-
- if (val & HCTL_SOFTRST)
- return -EIO;
-
- /* mask all interrupts and reset ports */
- for (i = 0; i < NR_PORTS; i++) {
- void __iomem *port_base = mmio_base + i * PORT_SIZE;
-
- writeb(0xff, port_base + PORT_IRQ_MASK);
- inic_reset_port(port_base);
- }
-
- /* port IRQ is masked now, unmask global IRQ */
- writew(hctl & ~HCTL_IRQOFF, mmio_base + HOST_CTL);
- val = readw(mmio_base + HOST_IRQ_MASK);
- val &= ~(HIRQ_PORT0 | HIRQ_PORT1);
- writew(val, mmio_base + HOST_IRQ_MASK);
-
- return 0;
-}
-
-#ifdef CONFIG_PM_SLEEP
-static int inic_pci_device_resume(struct pci_dev *pdev)
-{
- struct ata_host *host = pci_get_drvdata(pdev);
- struct inic_host_priv *hpriv = host->private_data;
- int rc;
-
- rc = ata_pci_device_do_resume(pdev);
- if (rc)
- return rc;
-
- if (pdev->dev.power.power_state.event == PM_EVENT_SUSPEND) {
- rc = init_controller(hpriv->mmio_base, hpriv->cached_hctl);
- if (rc)
- return rc;
- }
-
- ata_host_resume(host);
-
- return 0;
-}
-#endif
-
-static int inic_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
-{
- const struct ata_port_info *ppi[] = { &inic_port_info, NULL };
- struct ata_host *host;
- struct inic_host_priv *hpriv;
- void __iomem * const *iomap;
- int mmio_bar;
- int i, rc;
-
- ata_print_version_once(&pdev->dev, DRV_VERSION);
-
- dev_alert(&pdev->dev, "inic162x support is broken with common data corruption issues and will be disabled by default, contact linux-ide@vger.kernel.org if in production use\n");
-
- /* alloc host */
- host = ata_host_alloc_pinfo(&pdev->dev, ppi, NR_PORTS);
- hpriv = devm_kzalloc(&pdev->dev, sizeof(*hpriv), GFP_KERNEL);
- if (!host || !hpriv)
- return -ENOMEM;
-
- host->private_data = hpriv;
-
- /* Acquire resources and fill host. Note that PCI and cardbus
- * use different BARs.
- */
- rc = pcim_enable_device(pdev);
- if (rc)
- return rc;
-
- if (pci_resource_flags(pdev, MMIO_BAR_PCI) & IORESOURCE_MEM)
- mmio_bar = MMIO_BAR_PCI;
- else
- mmio_bar = MMIO_BAR_CARDBUS;
-
- rc = pcim_iomap_regions(pdev, 1 << mmio_bar, DRV_NAME);
- if (rc)
- return rc;
- host->iomap = iomap = pcim_iomap_table(pdev);
- hpriv->mmio_base = iomap[mmio_bar];
- hpriv->cached_hctl = readw(hpriv->mmio_base + HOST_CTL);
-
- for (i = 0; i < NR_PORTS; i++) {
- struct ata_port *ap = host->ports[i];
-
- ata_port_pbar_desc(ap, mmio_bar, -1, "mmio");
- ata_port_pbar_desc(ap, mmio_bar, i * PORT_SIZE, "port");
- }
-
- /* Set dma_mask. This devices doesn't support 64bit addressing. */
- rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
- if (rc) {
- dev_err(&pdev->dev, "32-bit DMA enable failed\n");
- return rc;
- }
-
- rc = init_controller(hpriv->mmio_base, hpriv->cached_hctl);
- if (rc) {
- dev_err(&pdev->dev, "failed to initialize controller\n");
- return rc;
- }
-
- pci_set_master(pdev);
- return ata_host_activate(host, pdev->irq, inic_interrupt, IRQF_SHARED,
- &inic_sht);
-}
-
-static const struct pci_device_id inic_pci_tbl[] = {
- { PCI_VDEVICE(INIT, 0x1622), },
- { },
-};
-
-static struct pci_driver inic_pci_driver = {
- .name = DRV_NAME,
- .id_table = inic_pci_tbl,
-#ifdef CONFIG_PM_SLEEP
- .suspend = ata_pci_device_suspend,
- .resume = inic_pci_device_resume,
-#endif
- .probe = inic_init_one,
- .remove = ata_pci_remove_one,
-};
-
-module_pci_driver(inic_pci_driver);
-
-MODULE_AUTHOR("Tejun Heo");
-MODULE_DESCRIPTION("low-level driver for Initio 162x SATA");
-MODULE_LICENSE("GPL v2");
-MODULE_DEVICE_TABLE(pci, inic_pci_tbl);
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/base/devtmpfs.c b/drivers/base/devtmpfs.c
index 9d4e46ad8352..645c9f2674e9 100644
--- a/drivers/base/devtmpfs.c
+++ b/drivers/base/devtmpfs.c
@@ -73,39 +73,90 @@ static struct file_system_type internal_fs_type = {
.kill_sb = kill_litter_super,
};
-/* Simply take a ref on the existing mount */
+struct devtmpfs_context {
+ struct fs_context *fc;
+};
+
+static void devtmpfs_free(struct fs_context *fc)
+{
+ struct devtmpfs_context *ctx = fc->fs_private;
+
+ if (ctx) {
+ put_fs_context(ctx->fc);
+ kfree(ctx);
+ }
+}
+
+static int devtmpfs_parse_param(struct fs_context *fc, struct fs_parameter *param)
+{
+ struct devtmpfs_context *ctx = fc->fs_private;
+
+ return ctx->fc->ops->parse_param(ctx->fc, param);
+}
+
+static int devtmpfs_parse_monolithic(struct fs_context *fc, void *data)
+{
+ struct devtmpfs_context *ctx = fc->fs_private;
+
+ if (ctx->fc->ops->parse_monolithic)
+ return ctx->fc->ops->parse_monolithic(ctx->fc, data);
+ return generic_parse_monolithic(ctx->fc, data);
+}
+
static int devtmpfs_get_tree(struct fs_context *fc)
{
+ struct devtmpfs_context *ctx = fc->fs_private;
struct super_block *sb = mnt->mnt_sb;
+ int err;
atomic_inc(&sb->s_active);
down_write(&sb->s_umount);
+
+ if (ctx->fc->ops->reconfigure) {
+ err = ctx->fc->ops->reconfigure(ctx->fc);
+ if (err) {
+ deactivate_locked_super(sb);
+ return err;
+ }
+ }
+
fc->root = dget(sb->s_root);
return 0;
}
-/* Ops are filled in during init depending on underlying shmem or ramfs type */
-struct fs_context_operations devtmpfs_context_ops = {};
+static const struct fs_context_operations devtmpfs_context_ops = {
+ .free = devtmpfs_free,
+ .parse_param = devtmpfs_parse_param,
+ .parse_monolithic = devtmpfs_parse_monolithic,
+ .get_tree = devtmpfs_get_tree,
+};
-/* Call the underlying initialization and set to our ops */
static int devtmpfs_init_fs_context(struct fs_context *fc)
{
- int ret;
-#ifdef CONFIG_TMPFS
- ret = shmem_init_fs_context(fc);
-#else
- ret = ramfs_init_fs_context(fc);
-#endif
- if (ret < 0)
- return ret;
+ struct devtmpfs_context *ctx;
+ int err;
+
+ ctx = kzalloc_obj(struct devtmpfs_context);
+ if (!ctx)
+ return -ENOMEM;
+
+ /* Each mount will reconfigure the shared superblock w/ new options */
+ ctx->fc = fs_context_for_reconfigure(mnt->mnt_root,
+ mnt->mnt_sb->s_flags, MS_RMT_MASK);
+ if (IS_ERR(ctx->fc)) {
+ err = PTR_ERR(ctx->fc);
+ kfree(ctx);
+ return err;
+ }
+ fc->fs_private = ctx;
fc->ops = &devtmpfs_context_ops;
return 0;
}
static struct file_system_type dev_fs_type = {
- .name = "devtmpfs",
+ .name = "devtmpfs",
.init_fs_context = devtmpfs_init_fs_context,
};
@@ -443,31 +494,6 @@ static int __ref devtmpfsd(void *p)
return 0;
}
-/*
- * Get the underlying (shmem/ramfs) context ops to build ours
- */
-static int devtmpfs_configure_context(void)
-{
- struct fs_context *fc;
-
- fc = fs_context_for_reconfigure(mnt->mnt_root, mnt->mnt_sb->s_flags,
- MS_RMT_MASK);
- if (IS_ERR(fc))
- return PTR_ERR(fc);
-
- /* Set up devtmpfs_context_ops based on underlying type */
- devtmpfs_context_ops.free = fc->ops->free;
- devtmpfs_context_ops.dup = fc->ops->dup;
- devtmpfs_context_ops.parse_param = fc->ops->parse_param;
- devtmpfs_context_ops.parse_monolithic = fc->ops->parse_monolithic;
- devtmpfs_context_ops.get_tree = &devtmpfs_get_tree;
- devtmpfs_context_ops.reconfigure = fc->ops->reconfigure;
-
- put_fs_context(fc);
-
- return 0;
-}
-
/*
* Create devtmpfs instance, driver-core devices will add their device
* nodes here.
@@ -483,12 +509,6 @@ int __init devtmpfs_init(void)
return PTR_ERR(mnt);
}
- err = devtmpfs_configure_context();
- if (err) {
- pr_err("unable to configure devtmpfs type %d\n", err);
- return err;
- }
-
err = register_filesystem(&dev_fs_type);
if (err) {
pr_err("unable to register devtmpfs type %d\n", err);
diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
index 8c33a5f86c3d..197825f44b1d 100644
--- a/drivers/block/ublk_drv.c
+++ b/drivers/block/ublk_drv.c
@@ -233,6 +233,11 @@ struct ublk_device {
u32 nr_io_ready;
bool unprivileged_daemons;
struct mutex cancel_mutex;
+ /*
+ * A cancel started in this FETCH round. Set by ublk_set_canceling(),
+ * cleared only by ublk_reset_ch_dev() when a new round starts. While
+ * it is set, no queue clears its ->canceling.
+ */
bool canceling;
pid_t ublksrv_tgid;
struct delayed_work exit_work;
@@ -1637,6 +1642,11 @@ static void ublk_reset_ch_dev(struct ublk_device *ub)
for (i = 0; i < ub->dev_info.nr_hw_queues; i++)
ublk_queue_reinit(ub, ublk_get_queue(ub, i));
+ /* a new FETCH round starts, the queues stay canceling until ready */
+ mutex_lock(&ub->cancel_mutex);
+ ub->canceling = false;
+ mutex_unlock(&ub->cancel_mutex);
+
/* set to NULL, otherwise new tasks cannot mmap io_cmd_buf */
ub->mm = NULL;
ub->nr_io_ready = 0;
@@ -1993,8 +2003,10 @@ static void ublk_cancel_cmd(struct ublk_queue *ubq, unsigned tag,
io->flags |= UBLK_IO_FLAG_CANCELED;
spin_unlock(&ubq->cancel_lock);
+ /* dependency ordered against smp_wmb() in ublk_prep_cancel() */
if (!done)
- io_uring_cmd_done(io->cmd, UBLK_IO_RES_ABORT, issue_flags);
+ io_uring_cmd_done(READ_ONCE(io->cmd), UBLK_IO_RES_ABORT,
+ issue_flags);
}
/*
@@ -2152,8 +2164,15 @@ static void ublk_reset_io_flags(struct ublk_device *ub)
for (i = 0; i < ub->dev_info.nr_hw_queues; i++)
ublk_get_queue(ub, i)->fail_io = false;
+ /*
+ * A cancel in this FETCH round took a command which still counts as
+ * ready, so the queues have to stay canceling. ub->canceling is set
+ * under cancel_mutex before any command is taken: either we see it
+ * here, or the cancel marks these queues again later.
+ */
mutex_lock(&ub->cancel_mutex);
- ublk_set_canceling(ub, false);
+ if (!ub->canceling)
+ ublk_set_canceling(ub, false);
mutex_unlock(&ub->cancel_mutex);
}
@@ -2173,9 +2192,8 @@ static void ublk_mark_io_ready(struct ublk_device *ub, struct ublk_queue *ubq,
ub->nr_io_ready++;
if (ublk_dev_ready(ub)) {
/*
- * now we are ready for handling ublk io request, clear
- * device-level canceling flag and wake ublk_dev_ready()
- * waiters
+ * Reset queue flags unless this round saw a cancel, and
+ * wake ublk_dev_ready() waiters.
*/
ublk_reset_io_flags(ub);
wake_up_var(&ub->nr_io_ready);
@@ -2258,6 +2276,12 @@ ublk_fill_io_cmd(struct ublk_io *io, struct io_uring_cmd *cmd)
return req;
}
+/*
+ * Call before ublk_fill_io_cmd() publishes @cmd in io->cmd: a control-path
+ * cancel may complete any command found there, and io_uring_cmd_done() only
+ * takes it off the cancelable list if it is marked already. The handlers
+ * hold uring_lock, so marking takes no lock.
+ */
static inline void ublk_prep_cancel(struct io_uring_cmd *cmd,
unsigned int issue_flags,
struct ublk_queue *ubq, unsigned int tag)
@@ -2271,6 +2295,8 @@ static inline void ublk_prep_cancel(struct io_uring_cmd *cmd,
pdu->ubq = ubq;
pdu->tag = tag;
io_uring_cmd_mark_cancelable(cmd, issue_flags);
+ /* pairs with the cancel loading cmd from io->cmd, then cmd->flags */
+ smp_wmb();
}
static void ublk_io_release(void *priv)
@@ -2511,11 +2537,11 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
ret = ublk_check_fetch_buf(ub, addr);
if (ret)
goto out;
+ /* before ublk_fetch() publishes io->cmd, see ublk_prep_cancel() */
+ ublk_prep_cancel(cmd, issue_flags, ubq, tag);
ret = ublk_fetch(cmd, ub, ubq, io, addr);
if (ret)
- goto out;
-
- ublk_prep_cancel(cmd, issue_flags, ubq, tag);
+ goto out_done;
return -EIOCBQUEUED;
}
@@ -2559,6 +2585,7 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
if (ret)
goto out;
io->res = result;
+ ublk_prep_cancel(cmd, issue_flags, ubq, tag);
req = ublk_fill_io_cmd(io, cmd);
ublk_apply_io_buf(ub, io, cmd, addr, &auto_buf, &buf_idx);
if (buf_idx != UBLK_INVALID_BUF_IDX)
@@ -2578,19 +2605,24 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
* uring_cmd active first and prepare for handling new requeued
* request
*/
+ ublk_prep_cancel(cmd, issue_flags, ubq, tag);
req = ublk_fill_io_cmd(io, cmd);
io->buf.addr = addr;
if (likely(ublk_get_data(ubq, io, req))) {
__ublk_prep_compl_io_cmd(io, req);
- return UBLK_IO_RES_OK;
+ ret = UBLK_IO_RES_OK;
+ goto out_done;
}
break;
default:
goto out;
}
- ublk_prep_cancel(cmd, issue_flags, ubq, tag);
return -EIOCBQUEUED;
+ out_done:
+ /* marked cancelable: complete through io_uring_cmd_done() */
+ io_uring_cmd_done(cmd, ret, issue_flags);
+ return -EIOCBQUEUED;
out:
pr_devel("%s: complete: cmd op %d, tag %d ret %x io_flags %x\n",
__func__, cmd_op, tag, ret, io ? io->flags : 0);
@@ -2996,9 +3028,10 @@ static bool ublk_validate_user_pid(struct ublk_device *ub, pid_t ublksrv_pid)
/*
* Wait until all queues have fetched their I/O commands, and return with
- * ub->mutex held and readiness guaranteed: then every queue's ->canceling
- * is cleared. Ready may regress between wakeup and mutex_lock() (daemon
- * death), so re-check it under the mutex and wait again.
+ * ub->mutex held and readiness guaranteed. The queues stay canceling if
+ * this round saw a cancel, see ublk_reset_io_flags(). Ready may regress
+ * between wakeup and mutex_lock() (daemon death), so re-check it under
+ * the mutex and wait again.
*/
static int ublk_wait_dev_ready_and_lock(struct ublk_device *ub)
{
@@ -3872,6 +3905,7 @@ static int ublk_ctrl_uring_cmd_permission(struct ublk_device *ub,
case UBLK_CMD_STOP_DEV:
case UBLK_CMD_ADD_DEV:
case UBLK_CMD_DEL_DEV:
+ case UBLK_CMD_DEL_DEV_ASYNC:
case UBLK_CMD_SET_PARAMS:
case UBLK_CMD_START_USER_RECOVERY:
case UBLK_CMD_END_USER_RECOVERY:
diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
index e577b02cf400..84dc751266e9 100644
--- a/drivers/block/virtio_blk.c
+++ b/drivers/block/virtio_blk.c
@@ -1588,7 +1588,7 @@ static int virtblk_freeze_priv(struct virtio_device *vdev)
/* Ensure no requests in virtqueues before deleting vqs. */
memflags = blk_mq_freeze_queue(q);
- blk_mq_quiesce_queue_nowait(q);
+ blk_mq_quiesce_queue(q);
blk_mq_unfreeze_queue(q, memflags);
/* Ensure we don't receive any more interrupts */
diff --git a/drivers/bluetooth/btqca.c b/drivers/bluetooth/btqca.c
index bdbb5801b240..4215522e62a2 100644
--- a/drivers/bluetooth/btqca.c
+++ b/drivers/bluetooth/btqca.c
@@ -443,7 +443,8 @@ static int qca_tlv_check_data(struct hci_dev *hdev,
tlv_nvm->data[0] |= 0x80;
/* UART Baud Rate */
- if (soc_type >= QCA_WCN3991)
+ if (soc_type == QCA_WCN3950 ||
+ soc_type >= QCA_WCN3991)
tlv_nvm->data[1] = nvm_baud_rate;
else
tlv_nvm->data[2] = nvm_baud_rate;
diff --git a/drivers/char/hw_random/virtio-rng.c b/drivers/char/hw_random/virtio-rng.c
index 2933f8cb769e..fcb60d4deaac 100644
--- a/drivers/char/hw_random/virtio-rng.c
+++ b/drivers/char/hw_random/virtio-rng.c
@@ -187,6 +187,7 @@ static int probe_common(struct virtio_device *vdev)
err_find:
ida_free(&rng_index_ida, index);
err_ida:
+ vdev->priv = NULL;
kfree(vi);
return err;
}
@@ -195,6 +196,11 @@ static void remove_common(struct virtio_device *vdev)
{
struct virtrng_info *vi = vdev->priv;
+ if (!vi) {
+ virtio_reset_device(vdev);
+ return;
+ }
+
vi->hwrng_removed = true;
vi->data_avail = 0;
vi->data_idx = 0;
@@ -204,6 +210,7 @@ static void remove_common(struct virtio_device *vdev)
virtio_reset_device(vdev);
vdev->config->del_vqs(vdev);
ida_free(&rng_index_ida, vi->index);
+ vdev->priv = NULL;
kfree(vi);
}
diff --git a/drivers/char/tpm/tpm2-cmd.c b/drivers/char/tpm/tpm2-cmd.c
index c5465834396a..85496fc5306b 100644
--- a/drivers/char/tpm/tpm2-cmd.c
+++ b/drivers/char/tpm/tpm2-cmd.c
@@ -487,9 +487,10 @@ int tpm2_probe(struct tpm_chip *chip)
out = (struct tpm_header *)buf.data;
if (be16_to_cpu(out->tag) == TPM2_ST_NO_SESSIONS)
chip->flags |= TPM_CHIP_FLAG_TPM2;
+ rc = 0;
}
tpm_buf_destroy(&buf);
- return 0;
+ return rc;
}
EXPORT_SYMBOL_GPL(tpm2_probe);
diff --git a/drivers/clk/nuvoton/clk-ma35d1.c b/drivers/clk/nuvoton/clk-ma35d1.c
index 9f65d0623517..c914079cee2d 100644
--- a/drivers/clk/nuvoton/clk-ma35d1.c
+++ b/drivers/clk/nuvoton/clk-ma35d1.c
@@ -17,6 +17,7 @@
static DEFINE_SPINLOCK(ma35d1_lock);
#define PLL_MAX_NUM 5
+#define MA35D1_CLK_COUNT 238
/* Clock Control Registers Offset */
#define REG_CLK_PWRCTL 0x00
@@ -467,12 +468,12 @@ static int ma35d1_clocks_probe(struct platform_device *pdev)
int ret;
ma35d1_hw_data = devm_kzalloc(dev,
- struct_size(ma35d1_hw_data, hws, CLK_MAX_IDX),
+ struct_size(ma35d1_hw_data, hws, MA35D1_CLK_COUNT),
GFP_KERNEL);
if (!ma35d1_hw_data)
return -ENOMEM;
- ma35d1_hw_data->num = CLK_MAX_IDX;
+ ma35d1_hw_data->num = MA35D1_CLK_COUNT;
hws = ma35d1_hw_data->hws;
clk_base = devm_platform_ioremap_resource(pdev, 0);
@@ -543,6 +544,8 @@ static int ma35d1_clocks_probe(struct platform_device *pdev)
hws[HCLK3] = ma35d1_clk_fixed_factor(dev, "hclk3", "sysclk1_mux", 1, 2);
hws[PCLK3] = ma35d1_clk_fixed_factor(dev, "pclk3", "sysclk1_mux", 1, 2);
hws[PCLK4] = ma35d1_clk_fixed_factor(dev, "pclk4", "sysclk1_mux", 1, 2);
+ hws[PCLK3_DIV4096] = ma35d1_clk_fixed_factor(dev, "pclk3_div4096", "pclk3", 1, 4096);
+ hws[PCLK4_DIV4096] = ma35d1_clk_fixed_factor(dev, "pclk4_div4096", "pclk4", 1, 4096);
hws[USBPHY0] = ma35d1_clk_fixed("usbphy0", 480000000);
hws[USBPHY1] = ma35d1_clk_fixed("usbphy1", 480000000);
diff --git a/drivers/cpufreq/amd-pstate.c b/drivers/cpufreq/amd-pstate.c
index 0b06bbd7215f..c5047b56be50 100644
--- a/drivers/cpufreq/amd-pstate.c
+++ b/drivers/cpufreq/amd-pstate.c
@@ -1653,7 +1653,7 @@ EXPORT_SYMBOL_GPL(amd_pstate_get_status);
int amd_pstate_update_status(const char *buf, size_t size)
{
- int mode_idx;
+ int cpu, mode_idx, ret = 0;
if (size > strlen("passive") || size < strlen("active"))
return -EINVAL;
@@ -1663,12 +1663,27 @@ int amd_pstate_update_status(const char *buf, size_t size)
if (mode_idx < 0 || mode_idx >= AMD_PSTATE_MAX)
return -EINVAL;
- if (mode_state_machine[cppc_state][mode_idx]) {
- guard(mutex)(&amd_pstate_driver_lock);
- return mode_state_machine[cppc_state][mode_idx](mode_idx);
+ guard(mutex)(&amd_pstate_driver_lock);
+
+ if (!mode_state_machine[cppc_state][mode_idx])
+ return 0;
+
+ if (mode_idx == AMD_PSTATE_PASSIVE &&
+ !cpu_feature_enabled(X86_FEATURE_CPPC)) {
+ cpus_read_lock();
+ /* Check offline CPUs too, before changing or removing the driver. */
+ for_each_present_cpu(cpu) {
+ if (cppc_auto_sel_is_immutable(cpu)) {
+ ret = -EOPNOTSUPP;
+ break;
+ }
+ }
+ cpus_read_unlock();
+ if (ret)
+ return ret;
}
- return 0;
+ return mode_state_machine[cppc_state][mode_idx](mode_idx);
}
EXPORT_SYMBOL_GPL(amd_pstate_update_status);
diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c
index ff8aac02994a..2cda2e05ab42 100644
--- a/drivers/crypto/omap-sham.c
+++ b/drivers/crypto/omap-sham.c
@@ -2041,6 +2041,20 @@ static struct attribute *omap_sham_attrs[] = {
};
ATTRIBUTE_GROUPS(omap_sham);
+static void omap_sham_unregister_algs(const struct omap_sham_pdata *pdata)
+{
+ struct omap_sham_algs_info *alg_info;
+ int i;
+
+ for (i = pdata->algs_info_size - 1; i >= 0; i--) {
+ alg_info = &pdata->algs_info[i];
+
+ crypto_engine_unregister_ahashes(alg_info->algs_list,
+ alg_info->registered);
+ alg_info->registered = 0;
+ }
+}
+
static int omap_sham_probe(struct platform_device *pdev)
{
struct omap_sham_dev *dd;
@@ -2158,10 +2172,7 @@ static int omap_sham_probe(struct platform_device *pdev)
return 0;
err_algs:
- for (i = dd->pdata->algs_info_size - 1; i >= 0; i--)
- for (j = dd->pdata->algs_info[i].registered - 1; j >= 0; j--)
- crypto_engine_unregister_ahash(
- &dd->pdata->algs_info[i].algs_list[j]);
+ omap_sham_unregister_algs(dd->pdata);
err_engine_start:
crypto_engine_exit(dd->engine);
err_engine:
@@ -2182,19 +2193,14 @@ static int omap_sham_probe(struct platform_device *pdev)
static void omap_sham_remove(struct platform_device *pdev)
{
struct omap_sham_dev *dd;
- int i, j;
dd = platform_get_drvdata(pdev);
spin_lock_bh(&sham.lock);
list_del(&dd->list);
spin_unlock_bh(&sham.lock);
- for (i = dd->pdata->algs_info_size - 1; i >= 0; i--)
- for (j = dd->pdata->algs_info[i].registered - 1; j >= 0; j--) {
- crypto_engine_unregister_ahash(
- &dd->pdata->algs_info[i].algs_list[j]);
- dd->pdata->algs_info[i].registered--;
- }
+ omap_sham_unregister_algs(dd->pdata);
+ crypto_engine_exit(dd->engine);
cancel_work_sync(&dd->done_task);
pm_runtime_dont_use_autosuspend(&pdev->dev);
pm_runtime_disable(&pdev->dev);
diff --git a/drivers/crypto/sa2ul.c b/drivers/crypto/sa2ul.c
index 4a766e435197..563e0f0d574a 100644
--- a/drivers/crypto/sa2ul.c
+++ b/drivers/crypto/sa2ul.c
@@ -125,9 +125,7 @@ struct sa_cmdl_cfg {
* @mci_enc: Mode Control Instruction for Encryption algorithm
* @mci_dec: Mode Control Instruction for Decryption
* @inv_key: Whether the encryption algorithm demands key inversion
- * @ctx: Pointer to the algorithm context
* @keyed_mac: Whether the authentication algorithm has key
- * @prep_iopad: Function pointer to generate intermediate ipad/opad
*/
struct algo_data {
struct sa_eng_info enc_eng;
@@ -141,10 +139,7 @@ struct algo_data {
u8 *mci_enc;
u8 *mci_dec;
bool inv_key;
- struct sa_tfm_ctx *ctx;
bool keyed_mac;
- void (*prep_iopad)(struct algo_data *algo, const u8 *key,
- u16 key_sz, __be32 *ipad, __be32 *opad);
};
/**
@@ -381,86 +376,36 @@ static void sa_swiz_128(u8 *in, u16 len)
}
}
-/* Prepare the ipad and opad from key as per SHA algorithm step 1*/
-static void prepare_kipad(u8 *k_ipad, const u8 *key, u16 key_sz)
+static void sa_prepare_iopads(struct algo_data *data, const u8 *key, u16 key_sz,
+ __be32 *ipad, __be32 *opad)
{
- int i;
-
- for (i = 0; i < key_sz; i++)
- k_ipad[i] = key[i] ^ 0x36;
-
- /* Instead of XOR with 0 */
- for (; i < SHA1_BLOCK_SIZE; i++)
- k_ipad[i] = 0x36;
-}
-
-static void prepare_kopad(u8 *k_opad, const u8 *key, u16 key_sz)
-{
- int i;
-
- for (i = 0; i < key_sz; i++)
- k_opad[i] = key[i] ^ 0x5c;
-
- /* Instead of XOR with 0 */
- for (; i < SHA1_BLOCK_SIZE; i++)
- k_opad[i] = 0x5c;
-}
-
-static void sa_export_shash(void *state, struct shash_desc *hash,
- int digest_size, __be32 *out)
-{
- struct sha1_state *sha1;
- struct sha256_state *sha256;
- u32 *result;
-
- switch (digest_size) {
- case SHA1_DIGEST_SIZE:
- sha1 = state;
- result = sha1->state;
+ union {
+ struct hmac_sha1_key sha1;
+ struct hmac_sha256_key sha256;
+ } k;
+
+ switch (data->aalg_id) {
+ case SA_AALG_ID_HMAC_SHA1:
+ hmac_sha1_preparekey(&k.sha1, key, key_sz);
+ cpu_to_be32_array(ipad, k.sha1.istate.h,
+ ARRAY_SIZE(k.sha1.istate.h));
+ cpu_to_be32_array(opad, k.sha1.ostate.h,
+ ARRAY_SIZE(k.sha1.ostate.h));
break;
- case SHA256_DIGEST_SIZE:
- sha256 = state;
- result = sha256->state;
+ case SA_AALG_ID_HMAC_SHA2_256:
+ hmac_sha256_preparekey(&k.sha256, key, key_sz);
+ cpu_to_be32_array(ipad, k.sha256.key.istate.h,
+ ARRAY_SIZE(k.sha256.key.istate.h));
+ cpu_to_be32_array(opad, k.sha256.key.ostate.h,
+ ARRAY_SIZE(k.sha256.key.ostate.h));
break;
default:
- dev_err(sa_k3_dev, "%s: bad digest_size=%d\n", __func__,
- digest_size);
+ dev_err(sa_k3_dev, "%s: bad aalg_id=%d\n", __func__,
+ data->aalg_id);
return;
}
- crypto_shash_export(hash, state);
-
- cpu_to_be32_array(out, result, digest_size / 4);
-}
-
-static void sa_prepare_iopads(struct algo_data *data, const u8 *key,
- u16 key_sz, __be32 *ipad, __be32 *opad)
-{
- SHASH_DESC_ON_STACK(shash, data->ctx->shash);
- int block_size = crypto_shash_blocksize(data->ctx->shash);
- int digest_size = crypto_shash_digestsize(data->ctx->shash);
- union {
- struct sha1_state sha1;
- struct sha256_state sha256;
- u8 k_pad[SHA1_BLOCK_SIZE];
- } sha;
-
- shash->tfm = data->ctx->shash;
-
- prepare_kipad(sha.k_pad, key, key_sz);
-
- crypto_shash_init(shash);
- crypto_shash_update(shash, sha.k_pad, block_size);
- sa_export_shash(&sha, shash, digest_size, ipad);
-
- prepare_kopad(sha.k_pad, key, key_sz);
-
- crypto_shash_init(shash);
- crypto_shash_update(shash, sha.k_pad, block_size);
-
- sa_export_shash(&sha, shash, digest_size, opad);
-
- memzero_explicit(&sha, sizeof(sha));
+ memzero_explicit(&k, sizeof(k));
}
/* Derive the inverse key used in AES-CBC decryption operation */
@@ -544,7 +489,7 @@ static void sa_set_sc_auth(struct algo_data *ad, const u8 *key, u16 key_sz,
/* Copy the keys or ipad/opad */
if (ad->keyed_mac)
- ad->prep_iopad(ad, key, key_sz, ipad, opad);
+ sa_prepare_iopads(ad, key, key_sz, ipad, opad);
else {
/* basic hash */
sc_buf[1] |= SA_BASIC_HASH;
@@ -1697,8 +1642,7 @@ static void sa_aead_dma_in_callback(void *data)
aead_request_complete(req, err);
}
-static int sa_cra_init_aead(struct crypto_aead *tfm, const char *hash,
- const char *fallback)
+static int sa_cra_init_aead(struct crypto_aead *tfm, const char *fallback)
{
struct sa_tfm_ctx *ctx = crypto_aead_ctx(tfm);
struct sa_crypto_data *data = dev_get_drvdata(sa_k3_dev);
@@ -1707,12 +1651,6 @@ static int sa_cra_init_aead(struct crypto_aead *tfm, const char *hash,
memzero_explicit(ctx, sizeof(*ctx));
ctx->dev_data = data;
- ctx->shash = crypto_alloc_shash(hash, 0, CRYPTO_ALG_NEED_FALLBACK);
- if (IS_ERR(ctx->shash)) {
- dev_err(sa_k3_dev, "base driver %s couldn't be loaded\n", hash);
- return PTR_ERR(ctx->shash);
- }
-
ctx->fallback.aead = crypto_alloc_aead(fallback, 0,
CRYPTO_ALG_NEED_FALLBACK);
@@ -1744,14 +1682,12 @@ static int sa_cra_init_aead(struct crypto_aead *tfm, const char *hash,
static int sa_cra_init_aead_sha1(struct crypto_aead *tfm)
{
- return sa_cra_init_aead(tfm, "sha1",
- "authenc(hmac(sha1),cbc(aes))");
+ return sa_cra_init_aead(tfm, "authenc(hmac(sha1),cbc(aes))");
}
static int sa_cra_init_aead_sha256(struct crypto_aead *tfm)
{
- return sa_cra_init_aead(tfm, "sha256",
- "authenc(hmac(sha256),cbc(aes))");
+ return sa_cra_init_aead(tfm, "authenc(hmac(sha256),cbc(aes))");
}
static void sa_exit_tfm_aead(struct crypto_aead *tfm)
@@ -1759,7 +1695,6 @@ static void sa_exit_tfm_aead(struct crypto_aead *tfm)
struct sa_tfm_ctx *ctx = crypto_aead_ctx(tfm);
struct sa_crypto_data *data = dev_get_drvdata(sa_k3_dev);
- crypto_free_shash(ctx->shash);
crypto_free_aead(ctx->fallback.aead);
sa_free_ctx_info(&ctx->enc, data);
@@ -1785,7 +1720,6 @@ static int sa_aead_setkey(struct crypto_aead *authenc,
if (key_idx >= 3)
return -EINVAL;
- ad->ctx = ctx;
ad->enc_eng.eng_id = SA_ENG_ID_EM1;
ad->enc_eng.sc_size = SA_CTX_ENC_TYPE1_SZ;
ad->auth_eng.eng_id = SA_ENG_ID_AM1;
@@ -1795,7 +1729,6 @@ static int sa_aead_setkey(struct crypto_aead *authenc,
ad->inv_key = true;
ad->keyed_mac = true;
ad->ealg_id = SA_EALG_ID_AES_CBC;
- ad->prep_iopad = sa_prepare_iopads;
memset(&cfg, 0, sizeof(cfg));
cfg.enc = true;
diff --git a/drivers/cxl/core/features.c b/drivers/cxl/core/features.c
index bb73f205f16c..0ea5a31275d3 100644
--- a/drivers/cxl/core/features.c
+++ b/drivers/cxl/core/features.c
@@ -97,6 +97,7 @@ get_supported_features(struct cxl_features_state *cxlfs)
kvmalloc(struct_size(entries, ent, count), GFP_KERNEL);
if (!entries)
return NULL;
+ entries->num_features = count;
struct cxl_mbox_get_sup_feats_out *mbox_out __free(kvfree) =
kvmalloc(cxl_mbox->payload_size, GFP_KERNEL);
@@ -174,7 +175,6 @@ get_supported_features(struct cxl_features_state *cxlfs)
start += num_entries;
} while (remain_feats);
- entries->num_features = count;
entries->num_user_features = user_feats;
return no_free_ptr(entries);
diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index 76731576ebfc..161a56169f87 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -934,11 +934,11 @@ void cxl_event_trace_record(const struct cxl_memdev *cxlmd,
if (evt->gen_media.media_hdr.descriptor &
CXL_GMER_EVT_DESC_THRESHOLD_EVENT)
- WARN_ON_ONCE((evt->gen_media.media_hdr.type &
+ WARN_ON_ONCE((evt->gen_media.media_hdr.type ==
CXL_GMER_MEM_EVT_TYPE_AP_CME_COUNTER_EXPIRE) &&
!get_unaligned_le24(evt->gen_media.cme_count));
else
- WARN_ON_ONCE(evt->gen_media.media_hdr.type &
+ WARN_ON_ONCE(evt->gen_media.media_hdr.type ==
CXL_GMER_MEM_EVT_TYPE_AP_CME_COUNTER_EXPIRE);
trace_cxl_general_media(cxlmd, type, cxlr, hpa,
@@ -949,11 +949,11 @@ void cxl_event_trace_record(const struct cxl_memdev *cxlmd,
if (evt->dram.media_hdr.descriptor &
CXL_GMER_EVT_DESC_THRESHOLD_EVENT)
- WARN_ON_ONCE((evt->dram.media_hdr.type &
+ WARN_ON_ONCE((evt->dram.media_hdr.type ==
CXL_DER_MEM_EVT_TYPE_AP_CME_COUNTER_EXPIRE) &&
!get_unaligned_le24(evt->dram.cvme_count));
else
- WARN_ON_ONCE(evt->dram.media_hdr.type &
+ WARN_ON_ONCE(evt->dram.media_hdr.type ==
CXL_DER_MEM_EVT_TYPE_AP_CME_COUNTER_EXPIRE);
trace_cxl_dram(cxlmd, type, cxlr, hpa, hpa_alias,
diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
index 599aa9d9eddc..01655eeb43a8 100644
--- a/drivers/cxl/core/port.c
+++ b/drivers/cxl/core/port.c
@@ -1073,11 +1073,15 @@ static int add_dport(struct cxl_port *port, struct cxl_dport *dport)
return -EBUSY;
}
+ /* Arrange for dport_dev to be valid through remove_dport() */
+ struct device *dev __free(put_device) = get_device(dport->dport_dev);
+
rc = xa_insert(&port->dports, (unsigned long)dport->dport_dev, dport,
GFP_KERNEL);
if (rc)
return rc;
+ retain_and_null_ptr(dev);
port->nr_dports++;
return 0;
}
@@ -1101,11 +1105,41 @@ static void cond_cxl_root_unlock(struct cxl_port *port)
device_unlock(&port->dev);
}
+static int clear_decoder_target(struct device *dev, void *data)
+{
+ struct cxl_dport *dport = data;
+ struct cxl_switch_decoder *cxlsd;
+
+ if (!is_switch_decoder(dev))
+ return 0;
+
+ cxlsd = to_cxl_switch_decoder(dev);
+ guard(rwsem_write)(&cxl_rwsem.region);
+
+ /*
+ * A dport can occupy more than one position of an interleave, so
+ * scan the whole target list rather than stopping at the first hit.
+ */
+ for (int i = 0; i < cxlsd->nr_targets; i++) {
+ if (cxlsd->target[i] != dport)
+ continue;
+ cxlsd->target[i] = NULL;
+ dev_dbg(dev, "dport%d removed from target list, index %d\n",
+ dport->port_id, i);
+ }
+
+ return 0;
+}
+
static void cxl_dport_remove(void *data)
{
struct cxl_dport *dport = data;
struct cxl_port *port = dport->port;
+ /* counterpart of cxl_port_update_decoder_targets() */
+ device_for_each_child(&port->dev, dport, clear_decoder_target);
+
+ port->nr_dports--;
xa_erase(&port->dports, (unsigned long) dport->dport_dev);
put_device(dport->dport_dev);
}
@@ -1188,21 +1222,6 @@ __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev,
if (rc)
return ERR_PTR(rc);
- /*
- * Setup port register if this is the first dport showed up. Having
- * a dport also means that there is at least 1 active link.
- */
- if (port->nr_dports == 1 &&
- port->component_reg_phys != CXL_RESOURCE_NONE) {
- rc = cxl_port_setup_regs(port, port->component_reg_phys);
- if (rc) {
- xa_erase(&port->dports, (unsigned long)dport->dport_dev);
- return ERR_PTR(rc);
- }
- port->component_reg_phys = CXL_RESOURCE_NONE;
- }
-
- get_device(dport_dev);
rc = devm_add_action_or_reset(host, cxl_dport_remove, dport);
if (rc)
return ERR_PTR(rc);
@@ -1629,7 +1648,16 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port,
cxl_switch_parse_cdat(new_dport);
- if (ida_is_empty(&port->decoder_ida)) {
+ if (port->nr_dports == 1) {
+ /*
+ * Some host bridges are known to not have component regsisters
+ * available until a root port has trained CXL. Perform that
+ * setup now.
+ */
+ rc = cxl_port_setup_regs(port, port->component_reg_phys);
+ if (rc)
+ return ERR_PTR(rc);
+
rc = devm_cxl_switch_port_decoders_setup(port);
if (rc)
return ERR_PTR(rc);
diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c
index 781cea16196f..9f514f9201bc 100644
--- a/drivers/dma/qcom/bam_dma.c
+++ b/drivers/dma/qcom/bam_dma.c
@@ -1322,8 +1322,8 @@ static int bam_dma_probe(struct platform_device *pdev)
for (i = 0; i < bdev->num_channels; i++)
bam_channel_init(bdev, &bdev->channels[i], i);
- ret = devm_request_irq(bdev->dev, bdev->irq, bam_dma_irq,
- IRQF_TRIGGER_HIGH, "bam_dma", bdev);
+ ret = request_irq(bdev->irq, bam_dma_irq, IRQF_TRIGGER_NONE,
+ "bam_dma", bdev);
if (ret)
goto err_bam_channel_exit;
@@ -1356,7 +1356,7 @@ static int bam_dma_probe(struct platform_device *pdev)
ret = dma_async_device_register(&bdev->common);
if (ret) {
dev_err(bdev->dev, "failed to register dma async device\n");
- goto err_bam_channel_exit;
+ goto err_free_irq;
}
ret = of_dma_controller_register(pdev->dev.of_node, bam_dma_xlate,
@@ -1375,6 +1375,8 @@ static int bam_dma_probe(struct platform_device *pdev)
err_unregister_dma:
dma_async_device_unregister(&bdev->common);
+err_free_irq:
+ free_irq(bdev->irq, bdev);
err_bam_channel_exit:
for (i = 0; i < bdev->num_channels; i++)
tasklet_kill(&bdev->channels[i].vc.task);
@@ -1400,7 +1402,7 @@ static void bam_dma_remove(struct platform_device *pdev)
/* mask all interrupts for this execution environment */
writel_relaxed(0, bam_addr(bdev, 0, BAM_IRQ_SRCS_MSK_EE));
- devm_free_irq(bdev->dev, bdev->irq, bdev);
+ free_irq(bdev->irq, bdev);
for (i = 0; i < bdev->num_channels; i++) {
bam_dma_terminate_all(&bdev->channels[i].vc.chan);
diff --git a/drivers/gpio/gpio-pca953x.c b/drivers/gpio/gpio-pca953x.c
index 7ebc4f9662c9..99b7bc262958 100644
--- a/drivers/gpio/gpio-pca953x.c
+++ b/drivers/gpio/gpio-pca953x.c
@@ -839,13 +839,18 @@ static int pca953x_irq_set_wake(struct irq_data *d, unsigned int on)
{
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
struct pca953x_chip *chip = gpiochip_get_data(gc);
+ int ret;
+
+ ret = irq_set_irq_wake(chip->client->irq, on);
+ if (ret)
+ return ret;
if (on)
atomic_inc(&chip->wakeup_path);
else
atomic_dec(&chip->wakeup_path);
- return irq_set_irq_wake(chip->client->irq, on);
+ return 0;
}
static void pca953x_irq_bus_lock(struct irq_data *d)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
index 5a04c20a5fd1..4dc44acc1b77 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
@@ -400,7 +400,11 @@ static int amdgpu_atif_query_backlight_caps(struct amdgpu_atif *atif)
characteristics.max_input_signal;
atif->backlight_caps.ac_level = characteristics.ac_level;
atif->backlight_caps.dc_level = characteristics.dc_level;
- atif->backlight_caps.data_points = characteristics.number_of_points;
+ if (characteristics.number_of_points > ATIF_QBTC_MAX_DATA_POINTS)
+ pr_warn(FW_BUG "ATIF: backlight curve has %u points, using the first %u\n",
+ characteristics.number_of_points, ATIF_QBTC_MAX_DATA_POINTS);
+ atif->backlight_caps.data_points = min_t(u8, characteristics.number_of_points,
+ ATIF_QBTC_MAX_DATA_POINTS);
memcpy(atif->backlight_caps.luminance_data,
characteristics.data_points,
sizeof(atif->backlight_caps.luminance_data));
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index 21dbc8675f4e..7aa5b22ed87a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -330,7 +330,8 @@ int amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(struct kgd_mem *mem,
void **kptr, uint64_t *size);
void amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(struct kgd_mem *mem);
-int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart);
+int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart,
+ struct amdkfd_process_info *process_info);
int amdgpu_amdkfd_gpuvm_restore_process_bos(void *process_info,
struct dma_fence __rcu **ef);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index 6de7dfa30ce5..c57d9e4837d6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -2244,11 +2244,13 @@ int amdgpu_amdkfd_gpuvm_sync_memory(
* amdgpu_amdkfd_map_gtt_bo_to_gart - Map BO to GART and increment reference count
* @bo: Buffer object to be mapped
* @bo_gart: Return bo reference
+ * @process_info: KFD process info of the queue the BO is for
*
* Before return, bo reference count is incremented. To release the reference and unpin/
* unmap the BO, call amdgpu_amdkfd_free_gtt_mem.
*/
-int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart)
+int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart,
+ struct amdkfd_process_info *process_info)
{
int ret;
@@ -2270,8 +2272,8 @@ int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo
goto err_map_bo_gart_failed;
}
- amdgpu_amdkfd_remove_eviction_fence(
- bo, bo->vm_bo->vm->process_info->eviction_fence);
+ /* bo->vm_bo can be the VM of any file which opened the BO */
+ amdgpu_amdkfd_remove_eviction_fence(bo, process_info->eviction_fence);
amdgpu_bo_unreserve(bo);
@@ -2448,6 +2450,9 @@ static int import_obj_create(struct amdgpu_device *adev,
err_remove_mem:
remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info);
+ amdgpu_sync_free(&(*mem)->sync);
+ mutex_destroy(&(*mem)->lock);
+ dma_buf_put(dma_buf);
drm_vma_node_revoke(&obj->vma_node, drm_priv);
err_free_mem:
kfree(*mem);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index efcd2322afa6..0f6505d4c958 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -161,7 +161,12 @@ static int amdgpu_cs_p1_user_fence(struct amdgpu_cs_parser *p,
if (size != PAGE_SIZE || data->offset > (size - 8))
return -EINVAL;
- if (amdgpu_ttm_tt_get_usermm(p->uf_bo->tbo.ttm))
+ if (amdgpu_ttm_tt_is_userptr(p->uf_bo->tbo.ttm))
+ return -EINVAL;
+
+ /* The GPU writes the user fence through the GART */
+ if (p->uf_bo->preferred_domains != AMDGPU_GEM_DOMAIN_GTT &&
+ p->uf_bo->allowed_domains != AMDGPU_GEM_DOMAIN_GTT)
return -EINVAL;
*offset = data->offset;
@@ -205,6 +210,9 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p,
int ret;
int i;
+ if (cs->in.flags)
+ return -EINVAL;
+
chunk_array = memdup_array_user(u64_to_user_ptr(cs->in.chunks),
cs->in.num_chunks,
sizeof(uint64_t));
@@ -1088,7 +1096,7 @@ static int amdgpu_cs_patch_ibs(struct amdgpu_cs_parser *p,
if (r)
return r;
- kptr += va_start - (m->start * AMDGPU_GPU_PAGE_SIZE);
+ kptr += va_start - (m->start * AMDGPU_GPU_PAGE_SIZE) + m->offset;
if (ring->funcs->parse_cs) {
memcpy(ib->ptr, kptr, ib->length_dw * 4);
@@ -1325,10 +1333,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
fence = &p->jobs[i]->base.s_fence->scheduled;
dma_fence_get(fence);
r = drm_sched_job_add_dependency(&leader->base, fence);
- if (r) {
- dma_fence_put(fence);
+ if (r)
return r;
- }
}
if (p->gang_size > 1) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 2e0998de67ed..b152beddbc3f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -6667,8 +6667,10 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
amdgpu_device_halt_activities(adev, job, reset_context, &device_list,
hive, need_emergency_restart);
- if (need_emergency_restart)
- goto skip_sched_resume;
+ if (need_emergency_restart) {
+ amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart);
+ goto reset_unlock;
+ }
/*
* Must check guilty signal here since after this point all old
* HW fences are force signaled.
@@ -6678,18 +6680,17 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
if (job && dma_fence_is_signaled(&job->hw_fence.base)) {
job_signaled = true;
dev_info(adev->dev, "Guilty job already signaled, skipping HW reset");
- goto skip_hw_reset;
+ goto post_resume;
}
r = amdgpu_device_asic_reset(adev, &device_list, reset_context);
if (r)
goto reset_unlock;
-skip_hw_reset:
+post_resume:
+ amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart);
r = amdgpu_device_sched_resume(&device_list, reset_context, job_signaled);
if (r)
goto reset_unlock;
-skip_sched_resume:
- amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart);
reset_unlock:
amdgpu_device_recovery_put_reset_lock(adev, &device_list);
end_reset:
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
index efa6214eee31..640f62734291 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
@@ -533,6 +533,11 @@ amdgpu_dma_buf_move_notify(struct dma_buf_attachment *attach)
continue;
}
+ if (!amdgpu_vm_ready(vm)) {
+ dma_resv_unlock(resv);
+ continue;
+ }
+
/* Reserve fences for two SDMA page table updates */
r = dma_resv_reserve_fences(resv, 2);
if (!r)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
index d6ae9974c952..4f31021b2152 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
@@ -447,10 +447,8 @@ int amdgpu_sync_push_to_job(struct amdgpu_sync *sync, struct amdgpu_job *job)
dma_fence_get(f);
r = drm_sched_job_add_dependency(&job->base, f);
- if (r) {
- dma_fence_put(f);
+ if (r)
return r;
- }
}
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
index 902d0a538e41..dd72891624eb 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
@@ -234,7 +234,6 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p,
dma_fence_get(fence);
r = drm_sched_job_add_dependency(&p->job->base, fence);
if (r) {
- dma_fence_put(fence);
dma_resv_iter_end(&cursor);
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
index 72ca6538b2e4..08535245e5fd 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
@@ -1735,8 +1735,9 @@ static void dce_v10_0_afmt_setmode(struct drm_encoder *encoder,
WREG32(mmAFMT_RAMP_CONTROL2 + dig->afmt->offset, 0x00000001);
WREG32(mmAFMT_RAMP_CONTROL3 + dig->afmt->offset, 0x00000001);
- /* enable audio after to setting up hw */
- dce_v10_0_audio_enable(adev, dig->afmt->pin, true);
+ /* Only enable audio if the display supports it */
+ if (connector->display_info.has_audio)
+ dce_v10_0_audio_enable(adev, dig->afmt->pin, true);
}
static void dce_v10_0_afmt_enable(struct drm_encoder *encoder, bool enable)
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
index acc887a58518..24462e6f221b 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
@@ -1777,8 +1777,9 @@ static void dce_v6_0_afmt_setmode(struct drm_encoder *encoder,
dce_v6_0_audio_dp_enable(encoder, 1);
}
- /* enable audio after setting up hw */
- dce_v6_0_audio_enable(adev, dig->afmt->pin, true);
+ /* Only enable audio if the display supports it */
+ if (connector->display_info.has_audio)
+ dce_v6_0_audio_enable(adev, dig->afmt->pin, true);
}
static void dce_v6_0_afmt_enable(struct drm_encoder *encoder, bool enable)
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
index 2ccd6aad8dd6..03d6b6f54c0f 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
@@ -1682,8 +1682,9 @@ static void dce_v8_0_afmt_setmode(struct drm_encoder *encoder,
WREG32(mmAFMT_RAMP_CONTROL2 + offset, 0x00000001);
WREG32(mmAFMT_RAMP_CONTROL3 + offset, 0x00000001);
- /* enable audio after setting up hw */
- dce_v8_0_audio_enable(adev, dig->afmt->pin, true);
+ /* Only enable audio if the display supports it */
+ if (connector->display_info.has_audio)
+ dce_v8_0_audio_enable(adev, dig->afmt->pin, true);
}
static void dce_v8_0_afmt_enable(struct drm_encoder *encoder, bool enable)
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
index 06fde713b922..69c9af88704c 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
@@ -3976,6 +3976,7 @@ static void gfx_v12_0_update_spm_vmid(struct amdgpu_device *adev,
{
u32 reg, data;
+ amdgpu_gfx_off_ctrl(adev, false);
reg = SOC15_REG_OFFSET(GC, 0, regRLC_SPM_MC_CNTL);
if (amdgpu_sriov_is_pp_one_vf(adev))
data = RREG32_NO_KIQ(reg);
@@ -3989,6 +3990,7 @@ static void gfx_v12_0_update_spm_vmid(struct amdgpu_device *adev,
WREG32_SOC15_NO_KIQ(GC, 0, regRLC_SPM_MC_CNTL, data);
else
WREG32_SOC15(GC, 0, regRLC_SPM_MC_CNTL, data);
+ amdgpu_gfx_off_ctrl(adev, true);
if (ring
&& amdgpu_sriov_is_pp_one_vf(adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
index 333e9c30c091..535c84e55558 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
@@ -168,8 +168,19 @@ static int ih_v6_0_toggle_ring_interrupts(struct amdgpu_device *adev,
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0));
if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
- if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ /*
+ * PSP programming of IH_RB_CNTL is slow enough for MSI to
+ * arrive before this call returns. Publish ih->enabled first
+ * so amdgpu_ih_process() consumes that IV and rearms.
+ */
+ if (enable) {
+ ih->enabled = true;
+ smp_wmb();
+ }
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) {
+ ih->enabled = false;
return -ETIMEDOUT;
+ }
} else {
WREG32(ih_regs->ih_rb_cntl, tmp);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
index d5cc32fa7848..f73b41b4be7e 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
@@ -207,8 +207,10 @@ static int mes_v11_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes,
r = amdgpu_fence_wait_polling(ring,
seq - ring->fence_drv.num_fences_mask,
timeout);
- if (r < 1)
+ if (r < 1) {
+ r = -ETIMEDOUT;
goto error_undo;
+ }
api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off);
api_status->api_completion_fence_addr = status_gpu_addr;
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
index 7d7bcf1c69f2..5e438d5896df 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
@@ -192,8 +192,10 @@ static int mes_v12_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes,
r = amdgpu_fence_wait_polling(ring,
seq - ring->fence_drv.num_fences_mask,
timeout);
- if (r < 1)
+ if (r < 1) {
+ r = -ETIMEDOUT;
goto error_undo;
+ }
api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off);
api_status->api_completion_fence_addr = status_gpu_addr;
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
index 4cd325149b63..bed0eb4b6ec3 100644
--- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
@@ -161,13 +161,52 @@ static int navi10_ih_toggle_ring_interrupts(struct amdgpu_device *adev,
tmp = RREG32(ih_regs->ih_rb_cntl);
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_ENABLE, (enable ? 1 : 0));
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_GPU_TS_ENABLE, 1);
+
+ if (enable) {
+ /* Unset the CLEAR_OVERFLOW bit to make sure the next step
+ * is switching the bit from 0 to 1
+ */
+ tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 0);
+ if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ return -ETIMEDOUT;
+ } else {
+ WREG32_NO_KIQ(ih_regs->ih_rb_cntl, tmp);
+ }
+
+ /* Clear RB_OVERFLOW bit */
+ tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 1);
+ if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ return -ETIMEDOUT;
+ } else {
+ WREG32_NO_KIQ(ih_regs->ih_rb_cntl, tmp);
+ }
+
+ /* Unset the CLEAR_OVERFLOW bit immediately so new overflows
+ * can be detected.
+ */
+ tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 0);
+ }
+
/* enable_intr field is only valid in ring0 */
if (ih == &adev->irq.ih)
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0));
if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
- if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ /*
+ * PSP programming of IH_RB_CNTL is slow enough for MSI to
+ * arrive before this call returns. Publish ih->enabled first
+ * so amdgpu_ih_process() consumes that IV and rearms.
+ */
+ if (enable) {
+ ih->enabled = true;
+ smp_wmb();
+ }
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) {
+ ih->enabled = false;
return -ETIMEDOUT;
+ }
} else {
WREG32(ih_regs->ih_rb_cntl, tmp);
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
index 99a47f2a45e7..6f3cfb9245c4 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
@@ -286,7 +286,8 @@ static int init_user_queue(struct process_queue_manager *pqm,
}
retval = amdgpu_amdkfd_map_gtt_bo_to_gart(q_properties->wptr_bo,
- &(*q)->wptr_bo_gart);
+ &(*q)->wptr_bo_gart,
+ pqm->process->kgd_process_info);
if (retval) {
pr_err("Failed to map wptr bo to GART\n");
goto free_gang_ctx_bo;
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c
index 306016c1f109..8f53a1f228f5 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c
@@ -535,6 +535,9 @@ static void dcn401_update_clocks_update_dtb_dto(struct clk_mgr_internal *clk_mgr
otg_master = resource_get_otg_master_for_stream(
&context->res_ctx, context->streams[i]);
ASSERT(otg_master);
+ if (!otg_master)
+ continue;
+
ASSERT(otg_master->clock_source);
ASSERT(otg_master->clock_source->funcs->program_pix_clk);
ASSERT(otg_master->stream_res.pix_clk_params.controller_id >= CONTROLLER_ID_D0);
diff --git a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c
index eb262ce42e2d..9154fea65f58 100644
--- a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c
+++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c
@@ -2483,12 +2483,8 @@ bool dp_is_sink_present(struct dc_link *link)
* which indicates we need additional delay
*/
- if (dal_ddc_open(ddc, GPIO_MODE_INPUT,
- GPIO_DDC_CONFIG_TYPE_MODE_I2C) != GPIO_RESULT_OK) {
- dal_ddc_close(ddc);
-
+ if (dal_ddc_open(ddc, GPIO_MODE_INPUT, GPIO_DDC_CONFIG_TYPE_MODE_I2C) != GPIO_RESULT_OK)
return present;
- }
/*
* Read GPIO: DP sink is present if both clock and data pins are zero
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c b/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c
index e7a90a437fff..06edf6613659 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c
@@ -452,6 +452,13 @@ void optc2_lock_doublebuffer_disable(struct timing_generator *optc)
void optc2_setup_manual_trigger(struct timing_generator *optc)
{
struct optc *optc1 = DCN10TG_FROM_TG(optc);
+ uint32_t v_total, v_total_max;
+
+ REG_GET(OTG_V_TOTAL_MAX, OTG_V_TOTAL_MAX, &v_total_max);
+ if (!v_total_max) {
+ REG_GET(OTG_V_TOTAL, OTG_V_TOTAL, &v_total);
+ optc->funcs->set_vtotal_min_max(optc, v_total, v_total);
+ }
/* Set the min/max selectors unconditionally so that
* DMCUB fw may change OTG timings when necessary
diff --git a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
index e65577298685..61cb8d051917 100644
--- a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
@@ -1777,8 +1777,21 @@ static bool dml1_validate(struct dc *dc, struct dc_state *context, enum dc_valid
out = dcn32_internal_validate_bw(dc, context, pipes, &pipe_cnt, &vlevel, validate_mode);
DC_FP_END();
- if (pipe_cnt == 0)
+ if (pipe_cnt == 0) {
+ /* The context is a copy of the previous state, so without this the
+ * clocks the last display needed stay applied with all displays off.
+ */
+ context->bw_ctx.bw.dcn.clk.socclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dppclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dcfclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dcfclk_deep_sleep_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dramclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.fclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.p_state_change_support = true;
+ context->bw_ctx.bw.dcn.clk.fclk_p_state_change_support = true;
+ context->bw_ctx.bw.dcn.clk.num_ways = 0;
goto validate_out;
+ }
if (!out)
goto validate_fail;
diff --git a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
index 1aae46d703ba..4a957f87c07c 100644
--- a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
+++ b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
@@ -1265,10 +1265,8 @@ void mod_freesync_handle_v_update(struct mod_freesync *mod_freesync,
* If VRR is inactive, set vtotal min and max to nominal vtotal
*/
if (in_out_vrr->state == VRR_STATE_INACTIVE) {
- in_out_vrr->adjust.v_total_min =
- mod_freesync_calc_v_total_from_refresh(stream,
- in_out_vrr->max_refresh_in_uhz);
- in_out_vrr->adjust.v_total_max = in_out_vrr->adjust.v_total_min;
+ in_out_vrr->adjust.v_total_min = stream->timing.v_total;
+ in_out_vrr->adjust.v_total_max = stream->timing.v_total;
return;
}
}
diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
index e00beb901d90..0557fbf9649c 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
+++ b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
@@ -370,6 +370,8 @@ struct smu_table_context {
void *boot_overdrive_table;
void *user_overdrive_table;
+ uint32_t user_od_feature_mask;
+
uint32_t gpu_metrics_table_size;
void *gpu_metrics_table;
};
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
index e463a39efab2..9d2d630bba24 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
@@ -1643,7 +1643,6 @@ static int smu_v13_0_0_od_edit_dpm_table(struct smu_context *smu,
struct amdgpu_device *adev = smu->adev;
uint32_t offset_of_voltageoffset;
int32_t minimum, maximum;
- uint32_t feature_ctrlmask;
int i, ret = 0;
switch (type) {
@@ -1965,21 +1964,16 @@ static int smu_v13_0_0_od_edit_dpm_table(struct smu_context *smu,
if (ret)
return ret;
} else {
- feature_ctrlmask = od_table->OverDriveTable.FeatureCtrlMask;
memcpy(od_table,
table_context->boot_overdrive_table,
sizeof(OverDriveTableExternal_t));
- od_table->OverDriveTable.FeatureCtrlMask = feature_ctrlmask;
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
+ table_context->user_od_feature_mask = 0;
}
fallthrough;
case PP_OD_COMMIT_DPM_TABLE:
- /*
- * The member below instructs PMFW the settings focused in
- * this single operation.
- * `uint32_t FeatureCtrlMask;`
- * It does not contain actual informations about user's custom
- * settings. Thus we do not cache it.
- */
+ table_context->user_od_feature_mask =
+ od_table->OverDriveTable.FeatureCtrlMask;
offset_of_voltageoffset = offsetof(OverDriveTable_t, VoltageOffsetPerZoneBoundary);
if (memcmp((u8 *)od_table + offset_of_voltageoffset,
table_context->user_overdrive_table + offset_of_voltageoffset,
@@ -1992,7 +1986,6 @@ static int smu_v13_0_0_od_edit_dpm_table(struct smu_context *smu,
return ret;
}
- od_table->OverDriveTable.FeatureCtrlMask = 0;
memcpy(table_context->user_overdrive_table + offset_of_voltageoffset,
(u8 *)od_table + offset_of_voltageoffset,
sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset);
@@ -2271,6 +2264,7 @@ static int smu_v13_0_0_set_default_od_settings(struct smu_context *smu)
boot_od_table,
sizeof(OverDriveTableExternal_t));
smu->user_dpm_profile.user_od = false;
+ smu->smu_table.user_od_feature_mask = 0;
} else if (smu->user_dpm_profile.user_od) {
memcpy(&user_od_table_bak,
user_od_table,
@@ -2320,12 +2314,8 @@ static int smu_v13_0_0_restore_user_od_settings(struct smu_context *smu)
OverDriveTableExternal_t *od_table = table_context->overdrive_table;
OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table;
int res;
-
- user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) |
- BIT(PP_OD_FEATURE_UCLK_BIT) |
- BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) |
- BIT(PP_OD_FEATURE_FAN_CURVE_BIT) |
- BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
+ user_od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
res = smu_v13_0_0_upload_overdrive_table(smu, user_od_table);
user_od_table->OverDriveTable.FeatureCtrlMask = 0;
if (res == 0)
@@ -3130,8 +3120,10 @@ static int smu_v13_0_0_set_power_limit(struct smu_context *smu,
if (limit <= msg_limit) {
if (smu->current_power_limit > msg_limit) {
od_table->OverDriveTable.Ppt = 0;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
-
+ table_context->user_od_feature_mask &=
+ ~(1U << PP_OD_FEATURE_PPT_BIT);
+ od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
ret = smu_v13_0_0_upload_overdrive_table(smu, od_table);
if (ret) {
dev_err(smu->adev->dev, "Failed to upload overdrive table!\n");
@@ -3145,8 +3137,9 @@ static int smu_v13_0_0_set_power_limit(struct smu_context *smu,
return ret;
od_table->OverDriveTable.Ppt = (limit * 100) / msg_limit - 100;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
-
+ table_context->user_od_feature_mask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
ret = smu_v13_0_0_upload_overdrive_table(smu, od_table);
if (ret) {
dev_err(smu->adev->dev, "Failed to upload overdrive table!\n");
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
index 0af98b5bf0ac..9947b3f6bb5f 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
@@ -1631,7 +1631,6 @@ static int smu_v13_0_7_od_edit_dpm_table(struct smu_context *smu,
struct amdgpu_device *adev = smu->adev;
uint32_t offset_of_voltageoffset;
int32_t minimum, maximum;
- uint32_t feature_ctrlmask;
int i, ret = 0;
switch (type) {
@@ -1953,22 +1952,17 @@ static int smu_v13_0_7_od_edit_dpm_table(struct smu_context *smu,
if (ret)
return ret;
} else {
- feature_ctrlmask = od_table->OverDriveTable.FeatureCtrlMask;
memcpy(od_table,
table_context->boot_overdrive_table,
sizeof(OverDriveTableExternal_t));
- od_table->OverDriveTable.FeatureCtrlMask = feature_ctrlmask;
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
+ table_context->user_od_feature_mask = 0;
}
fallthrough;
case PP_OD_COMMIT_DPM_TABLE:
- /*
- * The member below instructs PMFW the settings focused in
- * this single operation.
- * `uint32_t FeatureCtrlMask;`
- * It does not contain actual informations about user's custom
- * settings. Thus we do not cache it.
- */
+ table_context->user_od_feature_mask =
+ od_table->OverDriveTable.FeatureCtrlMask;
offset_of_voltageoffset = offsetof(OverDriveTable_t, VoltageOffsetPerZoneBoundary);
if (memcmp((u8 *)od_table + offset_of_voltageoffset,
table_context->user_overdrive_table + offset_of_voltageoffset,
@@ -1981,7 +1975,6 @@ static int smu_v13_0_7_od_edit_dpm_table(struct smu_context *smu,
return ret;
}
- od_table->OverDriveTable.FeatureCtrlMask = 0;
memcpy(table_context->user_overdrive_table + offset_of_voltageoffset,
(u8 *)od_table + offset_of_voltageoffset,
sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset);
@@ -2254,6 +2247,7 @@ static int smu_v13_0_7_set_default_od_settings(struct smu_context *smu)
boot_od_table,
sizeof(OverDriveTableExternal_t));
smu->user_dpm_profile.user_od = false;
+ smu->smu_table.user_od_feature_mask = 0;
} else if (smu->user_dpm_profile.user_od) {
memcpy(&user_od_table_bak,
user_od_table,
@@ -2304,11 +2298,8 @@ static int smu_v13_0_7_restore_user_od_settings(struct smu_context *smu)
OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table;
int res;
- user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) |
- BIT(PP_OD_FEATURE_UCLK_BIT) |
- BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) |
- BIT(PP_OD_FEATURE_FAN_CURVE_BIT) |
- BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
+ user_od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
res = smu_v13_0_7_upload_overdrive_table(smu, user_od_table);
user_od_table->OverDriveTable.FeatureCtrlMask = 0;
if (res == 0)
@@ -2726,7 +2717,10 @@ static int smu_v13_0_7_set_power_limit(struct smu_context *smu,
if (limit <= msg_limit) {
if (smu->current_power_limit > msg_limit) {
od_table->OverDriveTable.Ppt = 0;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ table_context->user_od_feature_mask &=
+ ~(1U << PP_OD_FEATURE_PPT_BIT);
+ od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
ret = smu_v13_0_7_upload_overdrive_table(smu, od_table);
if (ret) {
@@ -2741,7 +2735,9 @@ static int smu_v13_0_7_set_power_limit(struct smu_context *smu,
return ret;
od_table->OverDriveTable.Ppt = (limit * 100) / msg_limit - 100;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ table_context->user_od_feature_mask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
ret = smu_v13_0_7_upload_overdrive_table(smu, od_table);
if (ret) {
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c
index f8761a64a6e1..88bbc30b5ec9 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c
@@ -2391,6 +2391,7 @@ static int smu_v14_0_2_set_default_od_settings(struct smu_context *smu)
boot_od_table,
sizeof(OverDriveTableExternal_t));
smu->user_dpm_profile.user_od = false;
+ smu->smu_table.user_od_feature_mask = 0;
} else if (smu->user_dpm_profile.user_od) {
memcpy(&user_od_table_bak,
user_od_table,
@@ -2439,11 +2440,8 @@ static int smu_v14_0_2_restore_user_od_settings(struct smu_context *smu)
OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table;
int res;
- user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) |
- BIT(PP_OD_FEATURE_UCLK_BIT) |
- BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) |
- BIT(PP_OD_FEATURE_FAN_CURVE_BIT) |
- BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
+ user_od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
res = smu_v14_0_2_upload_overdrive_table(smu, user_od_table);
user_od_table->OverDriveTable.FeatureCtrlMask = 0;
if (res == 0)
@@ -2528,7 +2526,6 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu,
struct amdgpu_device *adev = smu->adev;
uint32_t offset_of_voltageoffset;
int32_t minimum, maximum;
- uint32_t feature_ctrlmask;
int i, ret = 0;
switch (type) {
@@ -2817,21 +2814,26 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu,
if (ret)
return ret;
} else {
- feature_ctrlmask = od_table->OverDriveTable.FeatureCtrlMask;
memcpy(od_table,
table_context->boot_overdrive_table,
sizeof(OverDriveTableExternal_t));
- od_table->OverDriveTable.FeatureCtrlMask = feature_ctrlmask;
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
+ table_context->user_od_feature_mask = 0;
}
fallthrough;
case PP_OD_COMMIT_DPM_TABLE:
/*
- * The member below instructs PMFW the settings focused in
- * this single operation.
- * `uint32_t FeatureCtrlMask;`
- * It does not contain actual informations about user's custom
- * settings. Thus we do not cache it.
+ * PMFW interprets FeatureCtrlMask as the complete set of OD
+ * features that are currently overridden, not as a delta
+ * describing this single operation. Any feature whose bit is
+ * clear gets reset to its default on import. od_table's mask
+ * has accumulated every feature the user touched since the
+ * last full restore, so record it and keep restating it on
+ * subsequent uploads.
*/
+ table_context->user_od_feature_mask =
+ od_table->OverDriveTable.FeatureCtrlMask;
+
offset_of_voltageoffset = offsetof(OverDriveTable_t, VoltageOffsetPerZoneBoundary);
if (memcmp((u8 *)od_table + offset_of_voltageoffset,
table_context->user_overdrive_table + offset_of_voltageoffset,
@@ -2844,7 +2846,6 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu,
return ret;
}
- od_table->OverDriveTable.FeatureCtrlMask = 0;
memcpy(table_context->user_overdrive_table + offset_of_voltageoffset,
(u8 *)od_table + offset_of_voltageoffset,
sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset);
@@ -2882,7 +2883,10 @@ static int smu_v14_0_2_set_power_limit(struct smu_context *smu,
if (limit <= msg_limit) {
if (smu->current_power_limit > msg_limit) {
od_table->OverDriveTable.Ppt = 0;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ table_context->user_od_feature_mask &=
+ ~(1U << PP_OD_FEATURE_PPT_BIT);
+ od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
ret = smu_v14_0_2_upload_overdrive_table(smu, od_table);
if (ret) {
@@ -2897,7 +2901,9 @@ static int smu_v14_0_2_set_power_limit(struct smu_context *smu,
return ret;
od_table->OverDriveTable.Ppt = (limit * 100) / msg_limit - 100;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ table_context->user_od_feature_mask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
ret = smu_v14_0_2_upload_overdrive_table(smu, od_table);
if (ret) {
diff --git a/drivers/gpu/drm/bridge/nwl-dsi.c b/drivers/gpu/drm/bridge/nwl-dsi.c
index 2f7429b24fc2..38bc83e7af47 100644
--- a/drivers/gpu/drm/bridge/nwl-dsi.c
+++ b/drivers/gpu/drm/bridge/nwl-dsi.c
@@ -110,6 +110,8 @@ struct nwl_dsi {
int error;
struct nwl_dsi_transfer *xfer;
+
+ unsigned int endpoint;
};
static const struct regmap_config nwl_dsi_regmap_config = {
@@ -821,10 +823,13 @@ static int nwl_dsi_bridge_atomic_check(struct drm_bridge *bridge,
struct drm_connector_state *conn_state)
{
struct drm_display_mode *adjusted_mode = &crtc_state->adjusted_mode;
+ struct nwl_dsi *dsi = bridge_to_dsi(bridge);
/* At least LCDIF + NWL needs active high sync */
- adjusted_mode->flags |= (DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC);
- adjusted_mode->flags &= ~(DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC);
+ if (dsi->endpoint == NWL_DSI_ENDPOINT_LCDIF) {
+ adjusted_mode->flags |= (DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC);
+ adjusted_mode->flags &= ~(DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC);
+ }
/*
* Do a full modeset if crtc_state->active is changed to be true.
@@ -1088,13 +1093,12 @@ static int nwl_dsi_parse_dt(struct nwl_dsi *dsi)
static int nwl_dsi_select_input(struct nwl_dsi *dsi)
{
struct device_node *remote;
- u32 use_dcss = 1;
int ret;
remote = of_graph_get_remote_node(dsi->dev->of_node, 0,
NWL_DSI_ENDPOINT_LCDIF);
if (remote) {
- use_dcss = 0;
+ dsi->endpoint = NWL_DSI_ENDPOINT_LCDIF;
} else {
remote = of_graph_get_remote_node(dsi->dev->of_node, 0,
NWL_DSI_ENDPOINT_DCSS);
@@ -1103,11 +1107,12 @@ static int nwl_dsi_select_input(struct nwl_dsi *dsi)
"No valid input endpoint found\n");
return -EINVAL;
}
+ dsi->endpoint = NWL_DSI_ENDPOINT_DCSS;
}
DRM_DEV_INFO(dsi->dev, "Using %s as input source\n",
- (use_dcss) ? "DCSS" : "LCDIF");
- ret = mux_control_try_select(dsi->mux, use_dcss);
+ (dsi->endpoint == NWL_DSI_ENDPOINT_DCSS) ? "DCSS" : "LCDIF");
+ ret = mux_control_try_select(dsi->mux, dsi->endpoint);
if (ret < 0)
DRM_DEV_ERROR(dsi->dev, "Failed to select input: %d\n", ret);
diff --git a/drivers/gpu/drm/display/drm_dp_helper.c b/drivers/gpu/drm/display/drm_dp_helper.c
index 4fc371f0fa12..8ef5adecace1 100644
--- a/drivers/gpu/drm/display/drm_dp_helper.c
+++ b/drivers/gpu/drm/display/drm_dp_helper.c
@@ -2183,7 +2183,8 @@ static int drm_dp_i2c_xfer(struct i2c_adapter *adapter, struct i2c_msg *msgs,
msg.buffer = msgs[i].buf + j;
msg.size = min(transfer_size, msgs[i].len - j);
- if (j + msg.size == msgs[i].len && aux->no_zero_sized)
+ if (aux->no_zero_sized && i == num - 1 &&
+ j + msg.size == msgs[i].len)
msg.request &= ~DP_AUX_I2C_MOT;
err = drm_dp_i2c_drain_msg(aux, &msg);
diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c
index 46655339003d..4ba190287c2a 100644
--- a/drivers/gpu/drm/drm_crtc.c
+++ b/drivers/gpu/drm/drm_crtc.c
@@ -476,14 +476,23 @@ EXPORT_SYMBOL(__drmm_crtc_alloc_with_planes);
* drm_crtc_cleanup - Clean up the core crtc usage
* @crtc: CRTC to cleanup
*
- * This function cleans up @crtc and removes it from the DRM mode setting
- * core. Note that the function does *not* free the crtc structure itself,
- * this is the responsibility of the caller.
+ * This function cleans up @crtc and removes it from the DRM mode setting core,
+ * after first waiting an RCU grace period to ensure @crtc->dev can safely be
+ * dereferenced by our dma_fence_ops.
+ *
+ * Note that the function does *not* free the crtc structure itself, this is the
+ * responsibility of the caller.
*/
void drm_crtc_cleanup(struct drm_crtc *crtc)
{
struct drm_device *dev = crtc->dev;
+ /* Ensure our dma_fence_ops remain valid for an RCU grace period after
+ * the fence is signaled. This is necessary because our dma_fence_ops
+ * dereference crtc->dev.
+ */
+ synchronize_rcu();
+
/* Note that the crtc_list is considered to be static; should we
* remove the drm_crtc at runtime we would have to decrement all
* the indices on the drm_crtc after us in the crtc_list.
diff --git a/drivers/gpu/drm/drm_panic.c b/drivers/gpu/drm/drm_panic.c
index 9c4ac2febc93..17b795a18d2c 100644
--- a/drivers/gpu/drm/drm_panic.c
+++ b/drivers/gpu/drm/drm_panic.c
@@ -562,7 +562,8 @@ static void draw_panic_static_kmsg(struct drm_scanout_buffer *sb)
const struct font_desc *font = get_default_font(sb->width, sb->height, NULL, NULL);
struct drm_rect r_screen = DRM_RECT_INIT(0, 0, sb->width, sb->height);
struct kmsg_dump_iter iter;
- char kmsg_buf[512];
+ /* the size printk formats one record into, too big for the stack */
+ static char kmsg_buf[2048];
size_t kmsg_len;
struct drm_panic_line line;
int yoffset;
@@ -580,6 +581,10 @@ static void draw_panic_static_kmsg(struct drm_scanout_buffer *sb)
char *start;
char *end;
+ /* a record that does not fit comes back empty, every time */
+ if (!kmsg_len)
+ break;
+
/* ignore terminating NUL and newline */
start = kmsg_buf + kmsg_len - 2;
end = kmsg_buf + kmsg_len - 1;
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
index 88385dc3b30d..4f22e373ed34 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
@@ -230,6 +230,17 @@ u16 etnaviv_buffer_config_pta(struct etnaviv_gpu *gpu, unsigned short id)
return buffer->user_size / 8;
}
+static void etnaviv_cmd_flush_ts(struct etnaviv_cmdbuf *buffer, bool has_blt)
+{
+ CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE, VIVS_TS_FLUSH_CACHE_FLUSH);
+
+ if (has_blt) {
+ CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1);
+ CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1);
+ CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
+ }
+}
+
void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
{
struct etnaviv_cmdbuf *buffer = &gpu->buffer;
@@ -253,7 +264,7 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
unsigned int dwords = 7;
if (has_blt)
- dwords += 10;
+ dwords += 11;
link_target = etnaviv_buffer_reserve(gpu, buffer, dwords);
@@ -266,16 +277,8 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
}
CMD_LOAD_STATE(buffer, VIVS_GL_FLUSH_CACHE, flush);
- if (gpu->exec_state == ETNA_PIPE_3D) {
- if (has_blt) {
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
- } else {
- CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE,
- VIVS_TS_FLUSH_CACHE_FLUSH);
- }
- }
+ if (gpu->exec_state == ETNA_PIPE_3D)
+ etnaviv_cmd_flush_ts(buffer, has_blt);
CMD_SEM(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
CMD_STALL(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
if (has_blt) {
@@ -462,12 +465,12 @@ void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, u32 exec_state,
return_dwords = 7;
/*
- * When the BLT engine is present we need 6 more dwords in the return
+ * When the BLT engine is present we need 7 more dwords in the return
* target: 3 enable/flush/disable + 4 enable/semaphore stall/disable,
- * but we don't need the normal TS flush state.
+ * on top of the normal TS flush state.
*/
if (has_blt)
- return_dwords += 6;
+ return_dwords += 7;
return_target = etnaviv_buffer_reserve(gpu, buffer, return_dwords);
CMD_LINK(cmdbuf, return_dwords, return_target);
@@ -484,14 +487,7 @@ void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, u32 exec_state,
VIVS_GL_FLUSH_CACHE_DEPTH |
VIVS_GL_FLUSH_CACHE_COLOR |
VIVS_GL_FLUSH_CACHE_SHADER_L1);
- if (has_blt) {
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
- } else {
- CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE,
- VIVS_TS_FLUSH_CACHE_FLUSH);
- }
+ etnaviv_cmd_flush_ts(buffer, has_blt);
}
CMD_SEM(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
CMD_STALL(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
diff --git a/drivers/gpu/drm/i915/display/intel_display_device.c b/drivers/gpu/drm/i915/display/intel_display_device.c
index f3f1f25b0f38..d94f07bda5ee 100644
--- a/drivers/gpu/drm/i915/display/intel_display_device.c
+++ b/drivers/gpu/drm/i915/display/intel_display_device.c
@@ -1767,7 +1767,10 @@ static void __intel_display_device_info_runtime_init(struct intel_display *displ
if (display->platform.alderlake_s && IS_DISPLAY_STEP(display, STEP_A0, STEP_A2))
for_each_pipe(display, pipe)
display_runtime->num_scalers[pipe] = 0;
- else if (DISPLAY_VER(display) >= 11) {
+ else if (DISPLAY_VERx100(display) == 3002) {
+ for_each_pipe(display, pipe)
+ display_runtime->num_scalers[pipe] = 1;
+ } else if (DISPLAY_VER(display) >= 11) {
for_each_pipe(display, pipe)
display_runtime->num_scalers[pipe] = 2;
} else if (DISPLAY_VER(display) >= 9) {
diff --git a/drivers/gpu/drm/i915/display/intel_plane.c b/drivers/gpu/drm/i915/display/intel_plane.c
index 91deeaec6110..bbf85f47a73f 100644
--- a/drivers/gpu/drm/i915/display/intel_plane.c
+++ b/drivers/gpu/drm/i915/display/intel_plane.c
@@ -139,6 +139,7 @@ intel_plane_duplicate_state(struct drm_plane *plane)
intel_state->dpt_vma = NULL;
intel_state->flags = 0;
intel_state->damage = DRM_RECT_INIT(0, 0, 0, 0);
+ intel_state->psr2_sel_fetch_area = DRM_RECT_INIT(0, 0, 0, 0);
/* add reference to fb */
if (intel_state->hw.fb)
@@ -1698,6 +1699,16 @@ static int intel_add_affected_planes(struct intel_atomic_state *state)
ret = intel_joiner_add_affected_planes(state, intel_crtc_joined_pipe_mask(crtc_state));
if (ret)
return ret;
+
+ /*
+ * Add affected plane here to ensure UV and Y plane linking is
+ * in place if UV plane is added as a part of selective update.
+ */
+ if (crtc_state->enable_psr2_sel_fetch) {
+ ret = intel_plane_add_affected(state, crtc);
+ if (ret)
+ return ret;
+ }
}
return 0;
diff --git a/drivers/gpu/drm/i915/display/intel_psr.c b/drivers/gpu/drm/i915/display/intel_psr.c
index 5f6ccd953c1e..ebade0f4e70b 100644
--- a/drivers/gpu/drm/i915/display/intel_psr.c
+++ b/drivers/gpu/drm/i915/display/intel_psr.c
@@ -2592,23 +2592,25 @@ static u32 psr2_pipe_srcsz_early_tpt_calc(struct intel_crtc_state *crtc_state,
}
static void clip_area_update(struct drm_rect *overlap_damage_area,
- struct drm_rect *damage_area,
- struct drm_rect *display_area)
+ const struct drm_rect *damage_area,
+ const struct drm_rect *display_area)
{
- if (!drm_rect_intersect(damage_area, display_area))
+ struct drm_rect inter = *damage_area;
+
+ if (!drm_rect_intersect(&inter, display_area))
return;
if (overlap_damage_area->y1 == -1) {
- overlap_damage_area->y1 = damage_area->y1;
- overlap_damage_area->y2 = damage_area->y2;
+ overlap_damage_area->y1 = inter.y1;
+ overlap_damage_area->y2 = inter.y2;
return;
}
- if (damage_area->y1 < overlap_damage_area->y1)
- overlap_damage_area->y1 = damage_area->y1;
+ if (inter.y1 < overlap_damage_area->y1)
+ overlap_damage_area->y1 = inter.y1;
- if (damage_area->y2 > overlap_damage_area->y2)
- overlap_damage_area->y2 = damage_area->y2;
+ if (inter.y2 > overlap_damage_area->y2)
+ overlap_damage_area->y2 = inter.y2;
}
static bool intel_psr2_sel_fetch_pipe_alignment(struct intel_crtc_state *crtc_state)
@@ -2779,7 +2781,7 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
.y2 = crtc_state->hw.adjusted_mode.crtc_vdisplay,
};
bool full_update = false, su_area_changed;
- int i, ret;
+ int i;
/*
* Selective fetch is not always usable, for instance it is dropped
@@ -2797,16 +2799,16 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
if (!crtc_state->enable_psr2_sel_fetch)
return 0;
- if (!psr2_sel_fetch_pipe_state_supported(crtc_state)) {
- full_update = true;
- goto skip_sel_fetch_set_loop;
- }
-
crtc_state->psr2_su_area.x1 = 0;
crtc_state->psr2_su_area.y1 = -1;
crtc_state->psr2_su_area.x2 = drm_rect_width(&display_area);
crtc_state->psr2_su_area.y2 = -1;
+ if (!psr2_sel_fetch_pipe_state_supported(crtc_state)) {
+ full_update = true;
+ goto skip_su_area_calc;
+ }
+
/*
* Calculate minimal selective fetch area of each plane and calculate
* the pipe damaged area.
@@ -2890,23 +2892,19 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
}
if (full_update)
- goto skip_sel_fetch_set_loop;
+ goto skip_su_area_calc;
intel_psr_apply_su_area_workarounds(crtc_state);
- ret = drm_atomic_add_affected_planes(&state->base, &crtc->base);
- if (ret)
- return ret;
-
do {
bool cursor_in_su_area = false;
/*
* Adjust su area to cover cursor fully as necessary
- * (early transport). This needs to be done after
- * drm_atomic_add_affected_planes to ensure visible
- * cursor is added into affected planes even when
- * cursor is not updated by itself.
+ * (early transport). A visible cursor is in the state
+ * even when it is not updated by itself as all planes
+ * of the pipe are added into the state before the plane
+ * checks when selective fetch is enabled.
*/
intel_psr2_sel_fetch_et_alignment(state, crtc, &display_area,
&cursor_in_su_area);
@@ -2926,6 +2924,11 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
break;
} while (su_area_changed);
+skip_su_area_calc:
+ if (full_update)
+ clip_area_update(&crtc_state->psr2_su_area, &display_area,
+ &display_area);
+
/*
* Now that we have the pipe damaged area check if it intersect with
* every plane, if it does set the plane selective fetch area.
@@ -2941,28 +2944,14 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
inter = crtc_state->psr2_su_area;
sel_fetch_area = &new_plane_state->psr2_sel_fetch_area;
- if (!drm_rect_intersect(&inter, &new_plane_state->uapi.dst)) {
- sel_fetch_area->y1 = -1;
- sel_fetch_area->y2 = -1;
- /*
- * if plane sel fetch was previously enabled ->
- * disable it
- */
- if (drm_rect_height(&old_plane_state->psr2_sel_fetch_area) > 0)
- crtc_state->update_planes |= BIT(plane->id);
-
- continue;
- }
-
- if (!psr2_sel_fetch_plane_state_supported(new_plane_state)) {
- full_update = true;
- break;
+ if (drm_rect_intersect(&inter, &new_plane_state->uapi.dst)) {
+ sel_fetch_area->y1 = inter.y1 - new_plane_state->uapi.dst.y1;
+ sel_fetch_area->y2 = inter.y2 - new_plane_state->uapi.dst.y1;
}
- sel_fetch_area = &new_plane_state->psr2_sel_fetch_area;
- sel_fetch_area->y1 = inter.y1 - new_plane_state->uapi.dst.y1;
- sel_fetch_area->y2 = inter.y2 - new_plane_state->uapi.dst.y1;
- crtc_state->update_planes |= BIT(plane->id);
+ if (!drm_rect_equals(&old_plane_state->psr2_sel_fetch_area,
+ sel_fetch_area))
+ crtc_state->update_planes |= BIT(plane->id);
/*
* Sel_fetch_area is calculated for UV plane. Use
@@ -2970,24 +2959,25 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
*/
if (linked) {
struct intel_plane_state *linked_new_plane_state;
+ struct intel_plane_state *linked_old_plane_state;
struct drm_rect *linked_sel_fetch_area;
- linked_new_plane_state = intel_atomic_get_plane_state(state, linked);
- if (IS_ERR(linked_new_plane_state))
- return PTR_ERR(linked_new_plane_state);
+ linked_new_plane_state = intel_atomic_get_new_plane_state(state, linked);
+ if (!linked_new_plane_state)
+ return -EINVAL;
+
+ linked_old_plane_state = intel_atomic_get_old_plane_state(state, linked);
linked_sel_fetch_area = &linked_new_plane_state->psr2_sel_fetch_area;
linked_sel_fetch_area->y1 = sel_fetch_area->y1;
linked_sel_fetch_area->y2 = sel_fetch_area->y2;
- crtc_state->update_planes |= BIT(linked->id);
+
+ if (!drm_rect_equals(&linked_old_plane_state->psr2_sel_fetch_area,
+ linked_sel_fetch_area))
+ crtc_state->update_planes |= BIT(linked->id);
}
}
-skip_sel_fetch_set_loop:
- if (full_update)
- clip_area_update(&crtc_state->psr2_su_area, &display_area,
- &display_area);
-
psr2_man_trk_ctl_calc(crtc_state, full_update);
crtc_state->pipe_srcsz_early_tpt =
psr2_pipe_srcsz_early_tpt_calc(crtc_state, full_update);
diff --git a/drivers/gpu/drm/i915/display/intel_psr_regs.h b/drivers/gpu/drm/i915/display/intel_psr_regs.h
index 8afbf5a38335..1e85860eefb8 100644
--- a/drivers/gpu/drm/i915/display/intel_psr_regs.h
+++ b/drivers/gpu/drm/i915/display/intel_psr_regs.h
@@ -254,12 +254,12 @@
#define _LNL_SFF_CTL_A 0x60918
#define _LNL_SFF_CTL_B 0x61918
#define LNL_SFF_CTL(tran) _MMIO_TRANS(tran, _LNL_SFF_CTL_A, _LNL_SFF_CTL_B)
-#define LNL_SFF_CTL_SF_SINGLE_FULL_FRAME REG_BIT(1)
+#define LNL_SFF_CTL_SF_SINGLE_FULL_FRAME REG_BIT(0)
#define _LNL_CFF_CTL_A 0x6091c
#define _LNL_CFF_CTL_B 0x6191c
#define LNL_CFF_CTL(tran) _MMIO_TRANS(tran, _LNL_CFF_CTL_A, _LNL_CFF_CTL_B)
-#define LNL_CFF_CTL_SF_CONTINUOUS_FULL_FRAME REG_BIT(1)
+#define LNL_CFF_CTL_SF_CONTINUOUS_FULL_FRAME REG_BIT(0)
/* PSR2 Early transport */
#define _PIPE_SRCSZ_ERLY_TPT_A 0x70074
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index 61f1b8f172a7..e05b1e98c9b2 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -2789,7 +2789,6 @@ add_timeline_fence_array(struct i915_execbuffer *eb,
u64 __user *user_values;
struct eb_fence *f;
u64 nfences;
- int err = 0;
nfences = timeline_fences->fence_count;
if (!nfences)
@@ -2827,6 +2826,7 @@ add_timeline_fence_array(struct i915_execbuffer *eb,
struct drm_syncobj *syncobj;
struct dma_fence *fence = NULL;
u64 point;
+ int err = 0;
if (__copy_from_user(&user_fence,
user_fences++,
diff --git a/drivers/gpu/drm/loongson/lsdc_gem.c b/drivers/gpu/drm/loongson/lsdc_gem.c
index a720d8f53209..5eb6b4f92302 100644
--- a/drivers/gpu/drm/loongson/lsdc_gem.c
+++ b/drivers/gpu/drm/loongson/lsdc_gem.c
@@ -155,14 +155,18 @@ struct drm_gem_object *lsdc_gem_object_create(struct drm_device *ddev,
return ERR_PTR(ret);
}
+ gobj = &lbo->tbo.base;
+ gobj->funcs = &lsdc_gem_object_funcs;
+
if (!sg) {
/* VRAM is filled with random data */
- lsdc_bo_clear(lbo);
+ ret = lsdc_bo_clear(lbo);
+ if (ret) {
+ drm_gem_object_put(gobj);
+ return ERR_PTR(ret);
+ }
}
- gobj = &lbo->tbo.base;
- gobj->funcs = &lsdc_gem_object_funcs;
-
/* tracking the BOs we created */
mutex_lock(&ldev->gem.mutex);
list_add_tail(&lbo->list, &ldev->gem.objects);
diff --git a/drivers/gpu/drm/loongson/lsdc_plane.c b/drivers/gpu/drm/loongson/lsdc_plane.c
index aa9a97f9c4dc..df941f13a969 100644
--- a/drivers/gpu/drm/loongson/lsdc_plane.c
+++ b/drivers/gpu/drm/loongson/lsdc_plane.c
@@ -228,7 +228,7 @@ static void lsdc_cursor_plane_atomic_async_update(struct drm_plane *plane,
new_state = drm_atomic_get_new_plane_state(state, plane);
- new_fb = plane->state->fb;
+ new_fb = new_state->fb;
plane->state->crtc_x = new_state->crtc_x;
plane->state->crtc_y = new_state->crtc_y;
diff --git a/drivers/gpu/drm/loongson/lsdc_ttm.c b/drivers/gpu/drm/loongson/lsdc_ttm.c
index 2e42c6970c9f..17b1998254e2 100644
--- a/drivers/gpu/drm/loongson/lsdc_ttm.c
+++ b/drivers/gpu/drm/loongson/lsdc_ttm.c
@@ -387,9 +387,13 @@ void lsdc_bo_kunmap(struct lsdc_bo *lbo)
ttm_bo_kunmap(&lbo->kmap);
}
-void lsdc_bo_clear(struct lsdc_bo *lbo)
+int lsdc_bo_clear(struct lsdc_bo *lbo)
{
- lsdc_bo_kmap(lbo);
+ int ret;
+
+ ret = lsdc_bo_kmap(lbo);
+ if (ret)
+ return ret;
if (lbo->is_iomem)
memset_io((void __iomem *)lbo->kptr, 0, lbo->size);
@@ -397,6 +401,8 @@ void lsdc_bo_clear(struct lsdc_bo *lbo)
memset(lbo->kptr, 0, lbo->size);
lsdc_bo_kunmap(lbo);
+
+ return 0;
}
int lsdc_bo_evict_vram(struct drm_device *ddev)
@@ -474,10 +480,8 @@ struct lsdc_bo *lsdc_bo_create(struct drm_device *ddev,
ret = ttm_bo_init_validate(bdev, tbo, bo_type, &lbo->placement, 0,
false, sg, resv, lsdc_bo_destroy);
- if (ret) {
- kfree(lbo);
+ if (ret)
return ERR_PTR(ret);
- }
return lbo;
}
diff --git a/drivers/gpu/drm/loongson/lsdc_ttm.h b/drivers/gpu/drm/loongson/lsdc_ttm.h
index 843e1475064e..df47d9bb0d03 100644
--- a/drivers/gpu/drm/loongson/lsdc_ttm.h
+++ b/drivers/gpu/drm/loongson/lsdc_ttm.h
@@ -89,7 +89,7 @@ size_t lsdc_bo_size(struct lsdc_bo *lbo);
int lsdc_bo_kmap(struct lsdc_bo *lbo);
void lsdc_bo_kunmap(struct lsdc_bo *lbo);
-void lsdc_bo_clear(struct lsdc_bo *lbo);
+int lsdc_bo_clear(struct lsdc_bo *lbo);
int lsdc_bo_evict_vram(struct drm_device *ddev);
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
index b7045d1c8415..8c9e3427bfc0 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
@@ -157,6 +157,7 @@ nvkm_subdev_init_(struct nvkm_subdev *subdev)
ret = subdev->func->init(subdev);
if (ret) {
nvkm_error(subdev, "init failed, %d\n", ret);
+ subdev->use.enabled = false;
return ret;
}
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
index 5368e705e7fd..186e09ea0cb1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
@@ -27,6 +27,7 @@
#include <subdev/bios/init.h>
#include <subdev/bios/pll.h>
#include <subdev/clk/pll.h>
+#include <subdev/vga.h>
int
gf100_devinit_pll_set(struct nvkm_devinit *init, u32 type, u32 freq)
@@ -99,6 +100,25 @@ gf100_devinit_preinit(struct nvkm_devinit *base)
* can use it as a reliable way to know whether we should run devinit.
*/
base->post = ((nvkm_rd32(device, 0x2240c) & BIT(1)) == 0);
+
+ /*
+ * However some VBIOS init sequences miss this bit. So fall back to
+ * the nv50 method of checking if the bit is not set. Running the
+ * VBIOS multiple times may have detrimental effects.
+ */
+ if (base->post) {
+ u64 disable = nvkm_devinit_disable(base);
+ /* magic to detect whether or not x86 vbios code has
+ * executed the devinit scripts to initialise the
+ * board. only works if there's a display engine.
+ */
+ if (!(disable & (1ULL << NVKM_ENGINE_DISP))) {
+ if (nvkm_rdvgac(device, 0, 0x00) ||
+ nvkm_rdvgac(device, 0, 0x1a)) {
+ base->post = false;
+ }
+ }
+ }
}
static const struct nvkm_devinit_func
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index b499f6635cd2..92df68f5523b 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -222,6 +222,8 @@ v3d_clean_caches(struct v3d_dev *v3d)
struct drm_device *dev = &v3d->drm;
int core = 0;
+ guard(mutex)(&v3d->cache_clean_lock);
+
trace_v3d_cache_clean_begin(dev);
/* GFXH-1897: Ensure pending flushes complete before writing L2TCACTL */
@@ -238,7 +240,6 @@ v3d_clean_caches(struct v3d_dev *v3d)
DRM_ERROR("Timeout waiting for TMU write combiner flush\n");
}
- mutex_lock(&v3d->cache_clean_lock);
V3D_CORE_WRITE(core, V3D_CTL_L2TCACTL,
V3D_L2TCACTL_L2TFLS |
V3D_SET_FIELD(V3D_L2TCACTL_FLM_CLEAN, V3D_L2TCACTL_FLM));
@@ -248,8 +249,6 @@ v3d_clean_caches(struct v3d_dev *v3d)
DRM_ERROR("Timeout waiting for L2T clean\n");
}
- mutex_unlock(&v3d->cache_clean_lock);
-
trace_v3d_cache_clean_end(dev);
}
diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c
index 9b9838696579..5cdfc107a012 100644
--- a/drivers/gpu/drm/v3d/v3d_sched.c
+++ b/drivers/gpu/drm/v3d/v3d_sched.c
@@ -403,6 +403,8 @@ v3d_csd_job_run(struct drm_sched_job *sched_job)
dma_fence_put(job->base.irq_fence);
job->base.irq_fence = dma_fence_get(fence);
+ job->timedout_batches = job->args.cfg[4];
+
trace_v3d_submit_csd(dev, to_v3d_fence(fence)->seqno);
v3d_job_start_stats(&job->base, V3D_CSD);
diff --git a/drivers/gpu/drm/vc4/vc4_gem.c b/drivers/gpu/drm/vc4/vc4_gem.c
index 5eadf11f289a..4fa6e71140a8 100644
--- a/drivers/gpu/drm/vc4/vc4_gem.c
+++ b/drivers/gpu/drm/vc4/vc4_gem.c
@@ -639,13 +639,14 @@ vc4_queue_submit(struct drm_device *dev, struct vc4_exec_info *exec,
fence->seqno = exec->seqno;
exec->fence = &fence->base;
+ /* The job can complete and drop exec->fence as soon as job_lock is
+ * released, so hold our own reference.
+ */
if (out_sync)
- drm_syncobj_replace_fence(out_sync, exec->fence);
+ dma_fence_get(&fence->base);
vc4_attach_fences(exec);
- drm_exec_fini(exec_ctx);
-
list_add_tail(&exec->head, &vc4->bin_job_list);
/* If no bin job was executing and if the render job (if any) has the
@@ -662,6 +663,13 @@ vc4_queue_submit(struct drm_device *dev, struct vc4_exec_info *exec,
spin_unlock_irqrestore(&vc4->job_lock, irqflags);
+ drm_exec_fini(exec_ctx);
+
+ if (out_sync) {
+ drm_syncobj_replace_fence(out_sync, &fence->base);
+ dma_fence_put(&fence->base);
+ }
+
return 0;
}
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi.c b/drivers/gpu/drm/vc4/vc4_hdmi.c
index 2f854386c338..8f5d43946583 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi.c
+++ b/drivers/gpu/drm/vc4/vc4_hdmi.c
@@ -764,8 +764,7 @@ static void vc4_hdmi_disable_scrambling(struct drm_encoder *encoder)
vc4_hdmi->scdc_enabled = false;
- if (delayed_work_pending(&vc4_hdmi->scrambling_work))
- cancel_delayed_work_sync(&vc4_hdmi->scrambling_work);
+ cancel_delayed_work_sync(&vc4_hdmi->scrambling_work);
if (!drm_dev_enter(drm, &idx))
return;
@@ -840,8 +839,6 @@ static void vc4_hdmi_encoder_post_crtc_disable(struct drm_encoder *encoder,
spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
- vc4_hdmi_disable_scrambling(encoder);
-
drm_dev_exit(idx);
out:
@@ -870,6 +867,8 @@ static void vc4_hdmi_encoder_post_crtc_powerdown(struct drm_encoder *encoder,
if (vc4_hdmi->variant->phy_disable)
vc4_hdmi->variant->phy_disable(vc4_hdmi);
+ vc4_hdmi_disable_scrambling(encoder);
+
clk_disable_unprepare(vc4_hdmi->pixel_bvb_clock);
clk_disable_unprepare(vc4_hdmi->pixel_clock);
@@ -1479,6 +1478,8 @@ static void vc4_hdmi_encoder_pre_crtc_configure(struct drm_encoder *encoder,
goto err_disable_pixel_clock;
}
+ vc4_hdmi_enable_scrambling(encoder);
+
if (vc4_hdmi->variant->phy_init)
vc4_hdmi->variant->phy_init(vc4_hdmi, conn_state);
@@ -1613,7 +1614,6 @@ static void vc4_hdmi_encoder_post_crtc_enable(struct drm_encoder *encoder,
}
vc4_hdmi_recenter_fifo(vc4_hdmi);
- vc4_hdmi_enable_scrambling(encoder);
drm_dev_exit(idx);
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi_phy.c b/drivers/gpu/drm/vc4/vc4_hdmi_phy.c
index 56e6a35da357..5040f014bf7e 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi_phy.c
+++ b/drivers/gpu/drm/vc4/vc4_hdmi_phy.c
@@ -1197,4 +1197,9 @@ void vc6_hdmi_phy_init(struct vc4_hdmi *vc4_hdmi,
void vc6_hdmi_phy_disable(struct vc4_hdmi *vc4_hdmi)
{
+ unsigned long flags;
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+ vc6_hdmi_reset_phy(vc4_hdmi);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
diff --git a/drivers/hwmon/aquacomputer_d5next.c b/drivers/hwmon/aquacomputer_d5next.c
index 0dcb8a3a691d..95ec74c6e67e 100644
--- a/drivers/hwmon/aquacomputer_d5next.c
+++ b/drivers/hwmon/aquacomputer_d5next.c
@@ -19,6 +19,7 @@
#include <linux/hwmon.h>
#include <linux/jiffies.h>
#include <linux/ktime.h>
+#include <linux/limits.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/seq_file.h>
@@ -133,6 +134,7 @@ static u8 aquaero_secondary_ctrl_report[] = {
#define AQUAERO_FAN_CURRENT_OFFSET 0x06
#define AQUAERO_FAN_POWER_OFFSET 0x08
#define AQUAERO_FAN_SPEED_OFFSET 0x00
+#define AQUAERO_MIN_STATUS_REPORT_SIZE (0x18B + AQUAERO_FAN_POWER_OFFSET + sizeof(u16))
static u16 aquaero_sensor_fan_offsets[] = { 0x167, 0x173, 0x17f, 0x18B };
/* Control report offsets for the Aquaero fan controllers */
@@ -156,6 +158,8 @@ static u16 aquaero_ctrl_fan_offsets[] = { 0x20c, 0x220, 0x234, 0x248 };
#define D5NEXT_5V_VOLTAGE 0x39
#define D5NEXT_12V_VOLTAGE 0x37
#define D5NEXT_VIRTUAL_SENSORS_START 0x3f
+#define D5NEXT_MIN_STATUS_REPORT_SIZE (D5NEXT_PUMP_OFFSET + AQC_FAN_SPEED_OFFSET + \
+ sizeof(u16))
static u16 d5next_sensor_fan_offsets[] = { D5NEXT_PUMP_OFFSET, D5NEXT_FAN_OFFSET };
/* Control report offsets for the D5 Next pump */
@@ -180,11 +184,14 @@ static u16 d5next_ctrl_fan_offsets[] = { 0x97, 0x42 }; /* Pump and fan speed (fr
#define AQUASTREAMULT_FAN_CURRENT_OFFSET 0x00
#define AQUASTREAMULT_FAN_POWER_OFFSET 0x04
#define AQUASTREAMULT_FAN_SPEED_OFFSET 0x06
+#define AQUASTREAMULT_MIN_STATUS_REPORT_SIZE (AQUASTREAMULT_PRESSURE_OFFSET + sizeof(u16))
static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET };
/* Spec and sensor report offset for the Farbwerk RGB controller */
#define FARBWERK_NUM_SENSORS 4
#define FARBWERK_SENSOR_START 0x2f
+#define FARBWERK_MIN_STATUS_REPORT_SIZE (FARBWERK_SENSOR_START + \
+ FARBWERK_NUM_SENSORS * AQC_SENSOR_SIZE)
/* Specs of the Farbwerk 360 RGB controller */
#define FARBWERK360_NUM_SENSORS 4
@@ -194,6 +201,8 @@ static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET };
/* Sensor report offsets for the Farbwerk 360 */
#define FARBWERK360_SENSOR_START 0x32
#define FARBWERK360_VIRTUAL_SENSORS_START 0x3a
+#define FARBWERK360_MIN_STATUS_REPORT_SIZE (FARBWERK360_VIRTUAL_SENSORS_START + \
+ FARBWERK360_NUM_VIRTUAL_SENSORS * AQC_SENSOR_SIZE)
/* Control report offsets for the Farbwerk 360 */
#define FARBWERK360_TEMP_CTRL_OFFSET 0x8
@@ -210,6 +219,7 @@ static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET };
#define OCTO_SENSOR_START 0x3D
#define OCTO_VIRTUAL_SENSORS_START 0x45
#define OCTO_FLOW_SENSOR_OFFSET 0x7B
+#define OCTO_MIN_STATUS_REPORT_SIZE (0xD8 + AQC_FAN_SPEED_OFFSET + sizeof(u16))
static u16 octo_sensor_fan_offsets[] = { 0x7D, 0x8A, 0x97, 0xA4, 0xB1, 0xBE, 0xCB, 0xD8 };
/* Control report offsets for the Octo */
@@ -230,6 +240,7 @@ static u16 octo_ctrl_fan_offsets[] = { 0x5B, 0xB0, 0x105, 0x15A, 0x1AF, 0x204, 0
#define QUADRO_SENSOR_START 0x34
#define QUADRO_VIRTUAL_SENSORS_START 0x3c
#define QUADRO_FLOW_SENSOR_OFFSET 0x6e
+#define QUADRO_MIN_STATUS_REPORT_SIZE (0x97 + AQC_FAN_SPEED_OFFSET + sizeof(u16))
static u16 quadro_sensor_fan_offsets[] = { 0x70, 0x7D, 0x8A, 0x97 };
/* Control report offsets for the Quadro */
@@ -249,6 +260,7 @@ static u16 quadro_ctrl_fan_offsets[] = { 0x37, 0x8c, 0xe1, 0x136 }; /* Fan speed
#define HIGHFLOWNEXT_CONDUCTIVITY 95
#define HIGHFLOWNEXT_5V_VOLTAGE 97
#define HIGHFLOWNEXT_5V_VOLTAGE_USB 99
+#define HIGHFLOWNEXT_MIN_STATUS_REPORT_SIZE (HIGHFLOWNEXT_5V_VOLTAGE_USB + sizeof(u16))
/* Specs of the Leakshield */
#define LEAKSHIELD_NUM_SENSORS 2
@@ -264,6 +276,7 @@ static u16 quadro_ctrl_fan_offsets[] = { 0x37, 0x8c, 0xe1, 0x136 }; /* Fan speed
#define LEAKSHIELD_FLOW_IN 111
#define LEAKSHIELD_RESERVOIR_VOLUME 313
#define LEAKSHIELD_RESERVOIR_FILLED 311
+#define LEAKSHIELD_MIN_STATUS_REPORT_SIZE (LEAKSHIELD_RESERVOIR_VOLUME + sizeof(u16))
/* Specs of the Aquastream XT pump */
#define AQUASTREAMXT_SERIAL_START 0x3a
@@ -556,6 +569,7 @@ struct aqc_data {
const char *name;
int status_report_id; /* Used for legacy devices, report is stored in buffer */
+ u16 min_report_size;
int ctrl_report_id;
int secondary_ctrl_report_id;
int secondary_ctrl_report_size;
@@ -601,9 +615,9 @@ struct aqc_data {
u32 speed_input_min[1];
u32 speed_input_target[1];
u32 speed_input_max[1];
- u32 power_input[8];
+ long power_input[8];
u16 voltage_input[8];
- u16 current_input[8];
+ u32 current_input[8];
/* Label values */
const char *const *temp_label;
@@ -996,7 +1010,7 @@ static int aqc_legacy_read(struct aqc_data *priv)
/* Calculation derived from linear regression */
sensor_value = get_unaligned_le16(priv->buffer + AQUASTREAMXT_PUMP_CURR_OFFSET);
- priv->current_input[0] = DIV_ROUND_CLOSEST(sensor_value * 176, 100) - 52;
+ priv->current_input[0] = max(DIV_ROUND_CLOSEST(sensor_value * 176, 100) - 52, 0);
sensor_value = get_unaligned_le16(priv->buffer + AQUASTREAMXT_PUMP_VOLTAGE_OFFSET);
priv->voltage_input[0] = DIV_ROUND_CLOSEST(sensor_value * 1000, 61);
@@ -1029,6 +1043,7 @@ static int aqc_legacy_read(struct aqc_data *priv)
static int aqc_read(struct device *dev, enum hwmon_sensor_types type, u32 attr,
int channel, long *val)
{
+ long power;
int ret;
struct aqc_data *priv = dev_get_drvdata(dev);
@@ -1093,7 +1108,10 @@ static int aqc_read(struct device *dev, enum hwmon_sensor_types type, u32 attr,
}
break;
case hwmon_power:
- *val = priv->power_input[channel];
+ power = READ_ONCE(priv->power_input[channel]);
+ if (power == -ENODATA)
+ return -ENODATA;
+ *val = power;
break;
case hwmon_pwm:
switch (priv->kind) {
@@ -1356,6 +1374,8 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
return 0;
priv = hid_get_drvdata(hdev);
+ if (!priv->min_report_size || size < priv->min_report_size)
+ return 0;
/* Info provided with every report */
priv->serial_number[0] = get_unaligned_be16(data + priv->serial_number_start_offset);
@@ -1391,9 +1411,9 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
priv->speed_input[i] =
get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
priv->fan_structure->speed);
- priv->power_input[i] =
- get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
- priv->fan_structure->power) * 10000;
+ WRITE_ONCE(priv->power_input[i],
+ get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
+ priv->fan_structure->power) * 10000);
priv->voltage_input[i] =
get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
priv->fan_structure->voltage) * 10;
@@ -1433,7 +1453,8 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
priv->speed_input[2] = get_unaligned_be16(data + AQUASTREAMULT_PRESSURE_OFFSET);
priv->speed_input[3] = get_unaligned_be16(data + AQUASTREAMULT_FLOW_SENSOR_OFFSET);
- priv->power_input[1] = get_unaligned_be16(data + AQUASTREAMULT_PUMP_POWER) * 10000;
+ WRITE_ONCE(priv->power_input[1],
+ get_unaligned_be16(data + AQUASTREAMULT_PUMP_POWER) * 10000);
priv->voltage_input[1] = get_unaligned_be16(data + AQUASTREAMULT_PUMP_VOLTAGE) * 10;
@@ -1446,10 +1467,11 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
case highflownext:
/* If external temp sensor is not connected, its power reading is also N/A */
if (priv->temp_input[1] == -ENODATA)
- priv->power_input[0] = -ENODATA;
+ WRITE_ONCE(priv->power_input[0], -ENODATA);
else
- priv->power_input[0] =
- get_unaligned_be16(data + HIGHFLOWNEXT_POWER) * 1000000;
+ WRITE_ONCE(priv->power_input[0],
+ min_t(u64, get_unaligned_be16(data + HIGHFLOWNEXT_POWER) *
+ 1000000ULL, LONG_MAX));
priv->voltage_input[0] = get_unaligned_be16(data + HIGHFLOWNEXT_5V_VOLTAGE) * 10;
priv->voltage_input[1] =
@@ -1541,6 +1563,7 @@ static void aqc_debugfs_init(struct aqc_data *priv)
static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
{
struct aqc_data *priv;
+ u16 min_report_size = 0;
int ret;
priv = devm_kzalloc(&hdev->dev, sizeof(*priv), GFP_KERNEL);
@@ -1577,7 +1600,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
* they present. The two other devices have the type of the second element in
* their respective collections set to 1, while the real device has it set to 0.
*/
- if (hdev->collection[1].type != 0) {
+ if (hdev->maxcollection < 2 || hdev->collection[1].type != 0) {
ret = -ENODEV;
goto fail_and_close;
}
@@ -1608,6 +1631,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_fan_power;
priv->voltage_label = label_fan_voltage;
priv->current_label = label_fan_current;
+ min_report_size = AQUAERO_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_D5NEXT:
priv->kind = d5next;
@@ -1633,6 +1657,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_d5next_power;
priv->voltage_label = label_d5next_voltages;
priv->current_label = label_d5next_current;
+ min_report_size = D5NEXT_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_FARBWERK:
priv->kind = farbwerk;
@@ -1643,6 +1668,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->temp_sensor_start_offset = FARBWERK_SENSOR_START;
priv->temp_label = label_temp_sensors;
+ min_report_size = FARBWERK_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_FARBWERK360:
priv->kind = farbwerk360;
@@ -1659,6 +1685,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->temp_label = label_temp_sensors;
priv->virtual_temp_label = label_virtual_temp_sensors;
+ min_report_size = FARBWERK360_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_OCTO:
priv->kind = octo;
@@ -1688,6 +1715,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_fan_power;
priv->voltage_label = label_fan_voltage;
priv->current_label = label_fan_current;
+ min_report_size = OCTO_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_QUADRO:
priv->kind = quadro;
@@ -1717,6 +1745,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_fan_power;
priv->voltage_label = label_fan_voltage;
priv->current_label = label_fan_current;
+ min_report_size = QUADRO_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_HIGHFLOWNEXT:
priv->kind = highflownext;
@@ -1734,6 +1763,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->speed_label = label_highflownext_fan_speed;
priv->power_label = label_highflownext_power;
priv->voltage_label = label_highflownext_voltage;
+ min_report_size = HIGHFLOWNEXT_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_LEAKSHIELD:
/*
@@ -1753,6 +1783,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->temp_label = label_leakshield_temp_sensors;
priv->speed_label = label_leakshield_fan_speed;
+ min_report_size = LEAKSHIELD_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_AQUASTREAMXT:
priv->kind = aquastreamxt;
@@ -1783,6 +1814,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_aquastreamult_power;
priv->voltage_label = label_aquastreamult_voltages;
priv->current_label = label_aquastreamult_current;
+ min_report_size = AQUASTREAMULT_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_POWERADJUST3:
priv->kind = poweradjust3;
@@ -1856,6 +1888,8 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
break;
}
+ priv->min_report_size = min_report_size;
+
if (priv->buffer_size != 0) {
priv->checksum_start = 0x01;
priv->checksum_length = priv->buffer_size - 3;
diff --git a/drivers/hwmon/asus_wmi_sensors.c b/drivers/hwmon/asus_wmi_sensors.c
index c2dd7ff882f2..502996e97cf1 100644
--- a/drivers/hwmon/asus_wmi_sensors.c
+++ b/drivers/hwmon/asus_wmi_sensors.c
@@ -164,6 +164,7 @@ struct asus_wmi_wmi_info {
struct asus_wmi_sensors {
struct asus_wmi_wmi_info wmi;
+ struct hwmon_chip_info chip_info;
/* lock access to internal cache */
struct mutex lock;
};
@@ -496,11 +497,6 @@ static const struct hwmon_ops asus_wmi_hwmon_ops = {
.read_string = asus_wmi_hwmon_read_string,
};
-static struct hwmon_chip_info asus_wmi_chip_info = {
- .ops = &asus_wmi_hwmon_ops,
- .info = NULL,
-};
-
static int asus_wmi_configure_sensor_setup(struct device *dev,
struct asus_wmi_sensors *sensor_data)
{
@@ -508,7 +504,7 @@ static int asus_wmi_configure_sensor_setup(struct device *dev,
struct hwmon_channel_info *asus_wmi_hwmon_chan;
int nr_count[hwmon_max] = {}, nr_types = 0;
struct asus_wmi_sensor_info *temp_sensor;
- const struct hwmon_chip_info *chip_info;
+ struct hwmon_chip_info *chip_info;
enum hwmon_sensor_types type;
struct device *hwdev;
int i, idx;
@@ -549,8 +545,9 @@ static int asus_wmi_configure_sensor_setup(struct device *dev,
if (!ptr_asus_wmi_ci)
return -ENOMEM;
- asus_wmi_chip_info.info = ptr_asus_wmi_ci;
- chip_info = &asus_wmi_chip_info;
+ chip_info = &sensor_data->chip_info;
+ chip_info->ops = &asus_wmi_hwmon_ops;
+ chip_info->info = ptr_asus_wmi_ci;
sensor_data->wmi.info_by_id = devm_kcalloc(dev, sensor_data->wmi.sensor_count,
sizeof(*sensor_data->wmi.info_by_id),
diff --git a/drivers/hwmon/chipcap2.c b/drivers/hwmon/chipcap2.c
index 0e182454779e..5f4b0fd535af 100644
--- a/drivers/hwmon/chipcap2.c
+++ b/drivers/hwmon/chipcap2.c
@@ -21,6 +21,7 @@
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/module.h>
+#include <linux/unaligned.h>
#include <linux/regulator/consumer.h>
#define CC2_START_CM 0xA0
@@ -88,6 +89,7 @@ struct cc2_data {
int irq_low;
int irq_high;
bool process_irqs;
+ bool regulator_enabled;
};
/* %RH as a per cent mille from a register value */
@@ -117,8 +119,10 @@ static int cc2_enable(struct cc2_data *data)
int ret;
/* exclusive regulator, check in case a disable failed */
- if (regulator_is_enabled(data->regulator))
+ if (data->regulator_enabled) {
+ data->process_irqs = true;
return 0;
+ }
/* clear any pending completion */
try_wait_for_completion(&data->complete);
@@ -127,6 +131,8 @@ static int cc2_enable(struct cc2_data *data)
if (ret < 0)
return ret;
+ data->regulator_enabled = true;
+
usleep_range(CC2_STARTUP_TIME_US, CC2_STARTUP_TIME_US + 125);
data->process_irqs = true;
@@ -138,15 +144,27 @@ static void cc2_disable(struct cc2_data *data)
{
int err;
+ /* let the alarms already raised by the last measurement be processed */
+ if (data->irq_low > 0)
+ synchronize_irq(data->irq_low);
+
+ if (data->irq_high > 0)
+ synchronize_irq(data->irq_high);
+
/* ignore alarms triggered by voltage toggling when powering up */
data->process_irqs = false;
/* exclusive regulator, check in case an enable failed */
- if (regulator_is_enabled(data->regulator)) {
- err = regulator_disable(data->regulator);
- if (err)
- dev_dbg(&data->client->dev, "Failed to disable device");
+ if (!data->regulator_enabled)
+ return;
+
+ err = regulator_disable(data->regulator);
+ if (err) {
+ dev_dbg(&data->client->dev, "Failed to disable device");
+ return;
}
+
+ data->regulator_enabled = false;
}
static int cc2_cmd_response_diagnostic(struct device *dev, u8 status)
@@ -165,13 +183,13 @@ static int cc2_cmd_response_diagnostic(struct device *dev, u8 status)
case CC2_RESPONSE_BUSY:
return -EBUSY;
case CC2_RESPONSE_NACK:
- if (resp & CC2_ERR_CORR_EEPROM)
+ if (status & CC2_ERR_CORR_EEPROM)
dev_dbg(dev, "Command failed: corrected EEPROM\n");
- if (resp & CC2_ERR_UNCORR_EEPROM)
+ if (status & CC2_ERR_UNCORR_EEPROM)
dev_dbg(dev, "Command failed: uncorrected EEPROM\n");
- if (resp & CC2_ERR_RAM_PARITY)
+ if (status & CC2_ERR_RAM_PARITY)
dev_dbg(dev, "Command failed: RAM parity\n");
- if (resp & CC2_ERR_RAM_PARITY)
+ if (status & CC2_ERR_CONFIG_LOAD)
dev_dbg(dev, "Command failed: configuration error\n");
return -ENODATA;
default:
@@ -224,7 +242,7 @@ static int cc2_command_mode_start(struct cc2_data *data)
2 * CC2_RESP_START_CM_US);
}
ret = cc2_read_command_status(data->client);
- if (ret != -ETIMEDOUT || i == CC2_CM_RETRIES)
+ if (ret != -ETIMEDOUT || i == CC2_CM_RETRIES - 1)
break;
/* command window missed, prepare for a retry */
@@ -258,8 +276,7 @@ static int cc2_write_reg(struct cc2_data *data, u8 reg, u16 val)
if (ret < 0)
goto disable;
- cpu_to_be16s(&val);
- ret = i2c_smbus_write_word_data(data->client, reg, val);
+ ret = i2c_smbus_write_word_swapped(data->client, reg, val);
if (ret < 0)
goto disable;
@@ -309,7 +326,7 @@ static int cc2_read_reg(struct cc2_data *data, u8 reg, u16 *val)
if (ret != CC2_EEPROM_DATA_LEN)
return ret < 0 ? ret : -EIO;
- *val = be16_to_cpup((__be16 *)&buf[1]);
+ *val = get_unaligned_be16(&buf[1]);
return cc2_read_command_status(data->client);
}
@@ -349,10 +366,10 @@ static int cc2_data_fetch(struct i2c_client *client,
switch (type) {
case hwmon_humidity:
- *val = cc2_rh_convert(be16_to_cpup((__be16 *)&data[0]));
+ *val = cc2_rh_convert(get_unaligned_be16(&data[0]));
break;
case hwmon_temp:
- *val = cc2_temp_convert(be16_to_cpup((__be16 *)&data[2]));
+ *val = cc2_temp_convert(get_unaligned_be16(&data[2]));
break;
default:
return -EINVAL;
@@ -624,6 +641,8 @@ static int cc2_request_ready_irq(struct cc2_data *data, struct device *dev)
int ret = 0;
data->irq_ready = fwnode_irq_get_byname(dev_fwnode(dev), "ready");
+ if (data->irq_ready == -EPROBE_DEFER)
+ return data->irq_ready;
if (data->irq_ready > 0) {
init_completion(&data->complete);
ret = devm_request_threaded_irq(dev, data->irq_ready, NULL,
@@ -641,6 +660,8 @@ static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev)
int ret = 0;
data->irq_low = fwnode_irq_get_byname(dev_fwnode(dev), "low");
+ if (data->irq_low == -EPROBE_DEFER)
+ return data->irq_low;
if (data->irq_low > 0) {
ret = devm_request_threaded_irq(dev, data->irq_low, NULL,
cc2_low_interrupt,
@@ -654,6 +675,8 @@ static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev)
}
data->irq_high = fwnode_irq_get_byname(dev_fwnode(dev), "high");
+ if (data->irq_high == -EPROBE_DEFER)
+ return data->irq_high;
if (data->irq_high > 0) {
ret = devm_request_threaded_irq(dev, data->irq_high, NULL,
cc2_high_interrupt,
diff --git a/drivers/hwmon/coretemp.c b/drivers/hwmon/coretemp.c
index c9b7e435eda8..25a9cf8feb31 100644
--- a/drivers/hwmon/coretemp.c
+++ b/drivers/hwmon/coretemp.c
@@ -61,6 +61,8 @@ enum coretemp_attr_index {
/*
* Per-Core Temperature Data
+ * @temp_valid: Whether the CPU reported @temp as valid. @temp is not
+ * reported when this is false.
* @tjmax: The static tjmax value when tjmax cannot be retrieved from
* IA32_TEMPERATURE_TARGET MSR.
* @last_updated: The time when the current temperature value was updated
@@ -73,6 +75,7 @@ enum coretemp_attr_index {
*/
struct temp_data {
int temp;
+ bool temp_valid;
int tjmax;
unsigned long last_updated;
unsigned int cpu;
@@ -391,7 +394,8 @@ static ssize_t show_temp(struct device *dev,
{
u32 eax, edx;
struct temp_data *tdata = container_of(devattr, struct temp_data, sd_attrs[ATTR_TEMP]);
- int tjmax;
+ int tjmax, temp;
+ bool valid;
mutex_lock(&tdata->update_lock);
@@ -400,17 +404,22 @@ static ssize_t show_temp(struct device *dev,
if (time_after(jiffies, tdata->last_updated + HZ)) {
rdmsr_on_cpu(tdata->cpu, tdata->status_reg, &eax, &edx);
/*
- * Ignore the valid bit. In all observed cases the register
- * value is either low or zero if the valid bit is 0.
- * Return it instead of reporting an error which doesn't
- * really help at all.
+ * Bit 31 indicates whether the reading is valid. Some CPUs
+ * clear it while the temperature is too low to be measured.
+ * Such a reading is not a temperature. Do not report it.
*/
+ tdata->temp_valid = (eax >> 31) & 1;
tdata->temp = tjmax - ((eax >> 16) & 0xff) * 1000;
tdata->last_updated = jiffies;
}
+ valid = tdata->temp_valid;
+ temp = tdata->temp;
mutex_unlock(&tdata->update_lock);
- return sprintf(buf, "%d\n", tdata->temp);
+
+ if (!valid)
+ return -ENODATA;
+ return sprintf(buf, "%d\n", temp);
}
static int create_core_attrs(struct temp_data *tdata, struct device *dev)
@@ -686,6 +695,9 @@ static int coretemp_cpu_online(unsigned int cpu)
if (!housekeeping_cpu(cpu, HK_TYPE_MISC))
return 0;
+ if (!pdev)
+ return 0;
+
pdata = platform_get_drvdata(pdev);
if (!pdata->hwmon_dev) {
struct device *hwmon;
@@ -737,6 +749,9 @@ static int coretemp_cpu_offline(unsigned int cpu)
return 0;
/* If the physical CPU device does not exist, just return */
+ if (!pdev)
+ return 0;
+
pd = platform_get_drvdata(pdev);
if (!pd->hwmon_dev)
return 0;
diff --git a/drivers/hwmon/corsair-cpro.c b/drivers/hwmon/corsair-cpro.c
index 17bb080eb9bb..1c152a6db380 100644
--- a/drivers/hwmon/corsair-cpro.c
+++ b/drivers/hwmon/corsair-cpro.c
@@ -215,10 +215,11 @@ static int set_target(struct ccp_device *ccp, int channel, long val)
int ret;
val = clamp_val(val, 0, 0xFFFF);
- ccp->target[channel] = val;
mutex_lock(&ccp->mutex);
ret = send_usb_cmd(ccp, CTL_SET_FAN_TARGET, channel, val >> 8, val);
+ if (!ret)
+ ccp->target[channel] = val;
mutex_unlock(&ccp->mutex);
return ret;
diff --git a/drivers/hwmon/gpio-fan.c b/drivers/hwmon/gpio-fan.c
index 1dfc02df9bbb..34986266180e 100644
--- a/drivers/hwmon/gpio-fan.c
+++ b/drivers/hwmon/gpio-fan.c
@@ -150,7 +150,10 @@ static int set_fan_speed(struct gpio_fan_data *fan_data, int speed_index)
__set_fan_ctrl(fan_data, fan_data->speed[speed_index].ctrl_val);
- if (fan_data->speed_index > 0 && speed_index == 0) {
+ /* The GPIOs have changed, and a PM put drops the reference on error. */
+ fan_data->speed_index = speed_index;
+
+ if (speed_index == 0) {
int ret;
ret = pm_runtime_put_sync(fan_data->dev);
@@ -158,8 +161,6 @@ static int set_fan_speed(struct gpio_fan_data *fan_data, int speed_index)
return ret;
}
- fan_data->speed_index = speed_index;
-
return 0;
}
diff --git a/drivers/hwmon/hp-wmi-sensors.c b/drivers/hwmon/hp-wmi-sensors.c
index cfa0d6ad0fb3..12950666c981 100644
--- a/drivers/hwmon/hp-wmi-sensors.c
+++ b/drivers/hwmon/hp-wmi-sensors.c
@@ -390,6 +390,7 @@ struct hp_wmi_info {
*/
struct hp_wmi_sensors {
struct wmi_device *wdev;
+ struct hwmon_chip_info chip_info;
struct hp_wmi_info **info_map[hwmon_max];
u8 channel_count[hwmon_max];
bool has_intrusion;
@@ -738,15 +739,21 @@ static long scale_numeric_sensor(const struct hp_wmi_numeric_sensor *nsensor)
val = current_reading;
- for (; unit_modifier < target_modifier; unit_modifier++)
+ for (; val && unit_modifier < target_modifier; unit_modifier++)
val = DIV_ROUND_CLOSEST(val, 10);
- for (; unit_modifier > target_modifier; unit_modifier--) {
- if (val > LONG_MAX / 10) {
- val = LONG_MAX;
- break;
+ if (val) {
+ for (; unit_modifier > target_modifier; unit_modifier--) {
+ if (val > LONG_MAX / 10) {
+ val = LONG_MAX;
+ break;
+ }
+ if (val < LONG_MIN / 10) {
+ val = LONG_MIN;
+ break;
+ }
+ val *= 10;
}
- val *= 10;
}
if (sensor_type == HP_WMI_TYPE_TEMPERATURE) {
@@ -1531,11 +1538,6 @@ static const struct hwmon_ops hp_wmi_hwmon_ops = {
.write = hp_wmi_hwmon_write,
};
-static struct hwmon_chip_info hp_wmi_chip_info = {
- .ops = &hp_wmi_hwmon_ops,
- .info = NULL,
-};
-
static struct hp_wmi_info *match_fan_event(struct hp_wmi_sensors *state,
const char *event_description)
{
@@ -1962,7 +1964,8 @@ static int make_chip_info(struct hp_wmi_sensors *state, bool has_events)
if (!ptr_channel_info)
return -ENOMEM;
- hp_wmi_chip_info.info = ptr_channel_info;
+ state->chip_info.ops = &hp_wmi_hwmon_ops;
+ state->chip_info.info = ptr_channel_info;
for (type = hwmon_chip; type < hwmon_max; type++) {
count = channel_count[type];
@@ -2053,7 +2056,7 @@ static int hp_wmi_sensors_init(struct hp_wmi_sensors *state)
return err;
hwdev = devm_hwmon_device_register_with_info(dev, "hp_wmi_sensors",
- state, &hp_wmi_chip_info,
+ state, &state->chip_info,
NULL);
return PTR_ERR_OR_ZERO(hwdev);
}
diff --git a/drivers/hwmon/lm70.c b/drivers/hwmon/lm70.c
index 0d5a250cb672..ee06c5641870 100644
--- a/drivers/hwmon/lm70.c
+++ b/drivers/hwmon/lm70.c
@@ -97,21 +97,21 @@ static ssize_t temp1_input_show(struct device *dev,
*/
switch (p_lm70->chip) {
case LM70_CHIP_LM70:
- val = ((int)raw / 32) * 250;
+ val = ((int)raw >> 5) * 250;
break;
case LM70_CHIP_TMP121:
case LM70_CHIP_TMP122:
case LM70_CHIP_LM74:
- val = ((int)raw / 8) * 625 / 10;
+ val = ((int)raw >> 3) * 625 / 10;
break;
case LM70_CHIP_LM71:
- val = ((int)raw / 4) * 3125 / 100;
+ val = ((int)raw >> 2) * 3125 / 100;
break;
case LM70_CHIP_TMP125:
- val = (sign_extend32(raw, 14) / 32) * 250;
+ val = (sign_extend32(raw, 14) >> 5) * 250;
break;
}
diff --git a/drivers/hwmon/lm87.c b/drivers/hwmon/lm87.c
index d2d970e73c61..bdd633feb2ba 100644
--- a/drivers/hwmon/lm87.c
+++ b/drivers/hwmon/lm87.c
@@ -600,13 +600,16 @@ static DEVICE_ATTR_RO(cpu0_vid);
static ssize_t vrm_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- struct lm87_data *data = dev_get_drvdata(dev);
+ struct i2c_client *client = dev_get_drvdata(dev);
+ struct lm87_data *data = i2c_get_clientdata(client);
+
return sprintf(buf, "%d\n", data->vrm);
}
static ssize_t vrm_store(struct device *dev, struct device_attribute *attr,
const char *buf, size_t count)
{
- struct lm87_data *data = dev_get_drvdata(dev);
+ struct i2c_client *client = dev_get_drvdata(dev);
+ struct lm87_data *data = i2c_get_clientdata(client);
unsigned long val;
int err;
diff --git a/drivers/hwmon/nct6683.c b/drivers/hwmon/nct6683.c
index 4a8380414038..05fc440f417c 100644
--- a/drivers/hwmon/nct6683.c
+++ b/drivers/hwmon/nct6683.c
@@ -918,7 +918,7 @@ static ssize_t
show_pwm(struct device *dev, struct device_attribute *attr, char *buf)
{
struct nct6683_data *data = nct6683_update_device(dev);
- struct sensor_device_attribute_2 *sattr = to_sensor_dev_attr_2(attr);
+ struct sensor_device_attribute *sattr = to_sensor_dev_attr(attr);
int index = sattr->index;
return sprintf(buf, "%d\n", data->pwm[index]);
@@ -928,7 +928,7 @@ static ssize_t
store_pwm(struct device *dev, struct device_attribute *attr, const char *buf,
size_t count)
{
- struct sensor_device_attribute_2 *sattr = to_sensor_dev_attr_2(attr);
+ struct sensor_device_attribute *sattr = to_sensor_dev_attr(attr);
struct nct6683_data *data = dev_get_drvdata(dev);
int index = sattr->index;
unsigned long val;
diff --git a/drivers/hwmon/nct6775-platform.c b/drivers/hwmon/nct6775-platform.c
index 0b529a542b0d..1975399ac440 100644
--- a/drivers/hwmon/nct6775-platform.c
+++ b/drivers/hwmon/nct6775-platform.c
@@ -1159,6 +1159,7 @@ static const char * const asus_wmi_boards[] = {
"Pro A520M-C",
"Pro A520M-C II",
"Pro B550M-C",
+ "Pro WS W480-ACE",
"Pro WS X570-ACE",
"ProArt B550-CREATOR",
"ProArt X570-CREATOR WIFI",
@@ -1258,6 +1259,7 @@ static const char * const asus_wmi_boards[] = {
"TUF Z390-PRO GAMING",
"TUF Z390M-PRO GAMING",
"TUF Z390M-PRO GAMING (WI-FI)",
+ "W480/SYS",
"WS Z390 PRO",
"Z490-GUNDAM (WI-FI)",
};
@@ -1270,6 +1272,7 @@ static const char * const asus_msi_boards[] = {
"EX-B760M-V5 D4",
"EX-H510M-V3",
"EX-H610M-V3 D4",
+ "G15CE",
"G15CF",
"PRIME A620M-A",
"PRIME B560-PLUS",
@@ -1320,6 +1323,8 @@ static const char * const asus_msi_boards[] = {
"PRIME X670-P",
"PRIME X670-P WIFI",
"PRIME X670E-PRO WIFI",
+ "PRIME X870-P",
+ "PRIME X870-P WIFI",
"PRIME Z590-A",
"PRIME Z590-P",
"PRIME Z590-P WIFI",
@@ -1362,11 +1367,18 @@ static const char * const asus_msi_boards[] = {
"ProArt B660-CREATOR D4",
"ProArt B760-CREATOR D4",
"ProArt X670E-CREATOR WIFI",
+ "ProArt X870E-CREATOR WIFI",
"ProArt Z690-CREATOR WIFI",
"ProArt Z790-CREATOR WIFI",
"ROG CROSSHAIR X670E EXTREME",
"ROG CROSSHAIR X670E GENE",
"ROG CROSSHAIR X670E HERO",
+ "ROG CROSSHAIR X870E APEX",
+ "ROG CROSSHAIR X870E DARK HERO",
+ "ROG CROSSHAIR X870E EXTREME",
+ "ROG CROSSHAIR X870E GLACIAL",
+ "ROG CROSSHAIR X870E HERO",
+ "ROG CROSSHAIR X870E HERO BTF",
"ROG MAXIMUS XIII APEX",
"ROG MAXIMUS XIII EXTREME",
"ROG MAXIMUS XIII EXTREME GLACIAL",
@@ -1404,6 +1416,12 @@ static const char * const asus_msi_boards[] = {
"ROG STRIX X670E-E GAMING WIFI",
"ROG STRIX X670E-F GAMING WIFI",
"ROG STRIX X670E-I GAMING WIFI",
+ "ROG STRIX X870-A GAMING WIFI",
+ "ROG STRIX X870-F GAMING WIFI",
+ "ROG STRIX X870-I GAMING WIFI",
+ "ROG STRIX X870E-E GAMING WIFI",
+ "ROG STRIX X870E-E GAMING WIFI7 R2",
+ "ROG STRIX X870E-H GAMING WIFI7",
"ROG STRIX Z590-A GAMING WIFI",
"ROG STRIX Z590-A GAMING WIFI II",
"ROG STRIX Z590-E GAMING WIFI",
@@ -1450,6 +1468,9 @@ static const char * const asus_msi_boards[] = {
"TUF GAMING H770-PRO WIFI",
"TUF GAMING X670E-PLUS",
"TUF GAMING X670E-PLUS WIFI",
+ "TUF GAMING X870-PLUS WIFI",
+ "TUF GAMING X870-PRO WIFI7 W NEO",
+ "TUF GAMING X870E-PLUS WIFI7",
"TUF GAMING Z590-PLUS",
"TUF GAMING Z590-PLUS WIFI",
"TUF GAMING Z690-PLUS",
@@ -1459,6 +1480,9 @@ static const char * const asus_msi_boards[] = {
"TUF GAMING Z790-PLUS D4",
"TUF GAMING Z790-PLUS WIFI",
"TUF GAMING Z790-PLUS WIFI D4",
+ "X870 AYW GAMING WIFI W",
+ "X870 MAX GAMING WIFI7",
+ "X870 MAX GAMING WIFI7 W",
"Z590 WIFI GUNDAM EDITION",
};
diff --git a/drivers/hwmon/peci/dimmtemp.c b/drivers/hwmon/peci/dimmtemp.c
index fbe82d9852e0..d327e38d59e1 100644
--- a/drivers/hwmon/peci/dimmtemp.c
+++ b/drivers/hwmon/peci/dimmtemp.c
@@ -4,7 +4,6 @@
#include <linux/auxiliary_bus.h>
#include <linux/bitfield.h>
#include <linux/bitops.h>
-#include <linux/devm-helpers.h>
#include <linux/hwmon.h>
#include <linux/jiffies.h>
#include <linux/module.h>
@@ -331,11 +330,13 @@ static int create_dimm_temp_info(struct peci_dimmtemp *priv)
if (ret == -ENODEV) {
dev_dbg(priv->dev, "No DIMMs found\n");
return 0;
- } else if (ret) {
+ }
+ if (ret == -EAGAIN) {
schedule_delayed_work(&priv->detect_work, DIMM_MASK_CHECK_DELAY_JIFFIES);
dev_dbg(priv->dev, "Deferred populating DIMM temp info\n");
- return ret;
}
+ if (ret)
+ return ret;
channels = priv->gen_info->chan_rank_max * priv->gen_info->dimm_idx_max;
@@ -405,10 +406,8 @@ static int peci_dimmtemp_probe(struct auxiliary_device *adev, const struct auxil
"Unexpected PECI revision %#x, some features may be unavailable\n",
peci_dev->info.peci_revision);
- ret = devm_delayed_work_autocancel(priv->dev, &priv->detect_work,
- create_dimm_temp_info_delayed);
- if (ret)
- return ret;
+ auxiliary_set_drvdata(adev, priv);
+ INIT_DELAYED_WORK(&priv->detect_work, create_dimm_temp_info_delayed);
ret = create_dimm_temp_info(priv);
if (ret && ret != -EAGAIN) {
@@ -419,6 +418,13 @@ static int peci_dimmtemp_probe(struct auxiliary_device *adev, const struct auxil
return 0;
}
+static void peci_dimmtemp_remove(struct auxiliary_device *adev)
+{
+ struct peci_dimmtemp *priv = auxiliary_get_drvdata(adev);
+
+ cancel_delayed_work_sync(&priv->detect_work);
+}
+
static int
read_thresholds_hsx(struct peci_dimmtemp *priv, int dimm_order, int chan_rank, u32 *data)
{
@@ -655,6 +661,7 @@ MODULE_DEVICE_TABLE(auxiliary, peci_dimmtemp_ids);
static struct auxiliary_driver peci_dimmtemp_driver = {
.probe = peci_dimmtemp_probe,
+ .remove = peci_dimmtemp_remove,
.id_table = peci_dimmtemp_ids,
};
diff --git a/drivers/hwmon/pmbus/pmbus_core.c b/drivers/hwmon/pmbus/pmbus_core.c
index aa578d2723f2..acab5fdf108e 100644
--- a/drivers/hwmon/pmbus/pmbus_core.c
+++ b/drivers/hwmon/pmbus/pmbus_core.c
@@ -1401,11 +1401,14 @@ static int pmbus_thermal_add_sensor(struct pmbus_data *pmbus_data,
tzd = devm_thermal_of_zone_register(dev, index, tdata,
&pmbus_thermal_ops);
/*
- * If CONFIG_THERMAL_OF is disabled, this returns -ENODEV,
- * so ignore that error but forward any other error.
+ * If CONFIG_THERMAL_OF is disabled, devm_thermal_of_zone_register()
+ * returns -ENOTSUPP. If no thermal zone is configured for the sensor,
+ * it returns -ENODEV. Ignore those errors but forward any other error.
*/
- if (IS_ERR(tzd) && (PTR_ERR(tzd) != -ENODEV))
+ if (IS_ERR(tzd) && PTR_ERR(tzd) != -ENODEV && PTR_ERR(tzd) != -ENOTSUPP) {
+ devm_kfree(dev, tdata);
return PTR_ERR(tzd);
+ }
return 0;
}
@@ -1420,10 +1423,11 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data,
{
struct sensor_device_attribute *a;
struct pmbus_sensor *sensor;
+ int err;
sensor = devm_kzalloc(data->dev, sizeof(*sensor), GFP_KERNEL);
if (!sensor)
- return NULL;
+ return ERR_PTR(-ENOMEM);
a = &sensor->attribute;
if (type)
@@ -1448,14 +1452,17 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data,
pmbus_set_sensor, -1);
if (pmbus_add_attribute(data, &a->dev_attr.attr))
- return NULL;
+ return ERR_PTR(-ENOMEM);
sensor->next = data->sensors;
data->sensors = sensor;
/* temperature sensors with _input values are registered with thermal */
- if (class == PSC_TEMPERATURE && strcmp(type, "input") == 0)
- pmbus_thermal_add_sensor(data, sensor, seq);
+ if (class == PSC_TEMPERATURE && strcmp(type, "input") == 0) {
+ err = pmbus_thermal_add_sensor(data, sensor, seq);
+ if (err)
+ return ERR_PTR(err);
+ }
return sensor;
}
@@ -1556,8 +1563,8 @@ static int pmbus_add_limit_attrs(struct i2c_client *client,
page, 0xff, l->reg, attr->class,
attr->update || l->update,
l->readonly, l->writeonly, true);
- if (!curr)
- return -ENOMEM;
+ if (IS_ERR(curr))
+ return PTR_ERR(curr);
if (l->sbit && (info->func[page] & attr->sfunc)) {
ret = pmbus_add_boolean(data, name,
l->alarm, index,
@@ -1596,8 +1603,8 @@ static int pmbus_add_sensor_attrs_one(struct i2c_client *client,
}
base = pmbus_add_sensor(data, name, "input", index, page, phase,
attr->reg, attr->class, true, true, false, true);
- if (!base)
- return -ENOMEM;
+ if (IS_ERR(base))
+ return PTR_ERR(base);
/* No limit and alarm attributes for phase specific sensors */
if (attr->sfunc && phase == 0xff) {
ret = pmbus_add_limit_attrs(client, data, info, name,
@@ -2270,8 +2277,8 @@ static int pmbus_add_fan_ctrl(struct i2c_client *client,
0xff, PMBUS_VIRT_FAN_TARGET_1 + id, PSC_FAN,
false, false, false, true);
- if (!sensor)
- return -ENOMEM;
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
if (!((data->info->func[page] & PMBUS_HAVE_PWM12) ||
(data->info->func[page] & PMBUS_HAVE_PWM34)))
@@ -2281,15 +2288,15 @@ static int pmbus_add_fan_ctrl(struct i2c_client *client,
0xff, PMBUS_VIRT_PWM_1 + id, PSC_PWM,
false, false, false, true);
- if (!sensor)
- return -ENOMEM;
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
sensor = pmbus_add_sensor(data, "pwm", "enable", index, page,
0xff, PMBUS_VIRT_PWM_ENABLE_1 + id, PSC_PWM,
true, false, false, false);
- if (!sensor)
- return -ENOMEM;
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
return 0;
}
@@ -2306,6 +2313,7 @@ static int pmbus_add_fan_attributes(struct i2c_client *client,
int f;
for (f = 0; f < ARRAY_SIZE(pmbus_fan_registers); f++) {
+ struct pmbus_sensor *sensor;
int regval;
if (!(info->func[page] & pmbus_fan_flags[f]))
@@ -2326,10 +2334,11 @@ static int pmbus_add_fan_attributes(struct i2c_client *client,
(!(regval & (PB_FAN_1_INSTALLED >> ((f & 1) * 4)))))
continue;
- if (pmbus_add_sensor(data, "fan", "input", index,
- page, 0xff, pmbus_fan_registers[f],
- PSC_FAN, true, true, false, true) == NULL)
- return -ENOMEM;
+ sensor = pmbus_add_sensor(data, "fan", "input", index,
+ page, 0xff, pmbus_fan_registers[f],
+ PSC_FAN, true, true, false, true);
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
/* Fan control */
if (pmbus_check_word_register(client, page,
diff --git a/drivers/hwmon/pmbus/q54sj108a2.c b/drivers/hwmon/pmbus/q54sj108a2.c
index 7b0d292a425a..1f2811354aac 100644
--- a/drivers/hwmon/pmbus/q54sj108a2.c
+++ b/drivers/hwmon/pmbus/q54sj108a2.c
@@ -343,6 +343,8 @@ static int q54sj108a2_probe(struct i2c_client *client)
psu->client = client;
debugfs = pmbus_get_debugfs_dir(client);
+ if (!debugfs)
+ return 0;
q54sj108a2_dir = debugfs_create_dir(client->name, debugfs);
diff --git a/drivers/hwmon/sht4x.c b/drivers/hwmon/sht4x.c
index 83dc3ca67046..53c8feb7e291 100644
--- a/drivers/hwmon/sht4x.c
+++ b/drivers/hwmon/sht4x.c
@@ -99,7 +99,9 @@ static int sht4x_read_values(struct sht4x_data *data)
msleep(jiffies_to_msecs(data->heating_complete - curr_jiffies));
if (data->data_pending &&
- time_before64(get_jiffies_64(), data->heating_complete + data->update_interval)) {
+ time_before64(get_jiffies_64(),
+ data->heating_complete +
+ msecs_to_jiffies(data->update_interval))) {
data->data_pending = false;
} else {
next_update = data->last_updated +
diff --git a/drivers/hwmon/tmp102.c b/drivers/hwmon/tmp102.c
index 376e0eac8cc1..62a7bee9e4b9 100644
--- a/drivers/hwmon/tmp102.c
+++ b/drivers/hwmon/tmp102.c
@@ -56,7 +56,7 @@ struct tmp102 {
const char *label;
struct regmap *regmap;
u16 config_orig;
- unsigned long ready_time;
+ u64 ready_time;
};
/* convert left adjusted 13-bit TMP102 register value to milliCelsius */
@@ -91,7 +91,7 @@ static int tmp102_read(struct device *dev, enum hwmon_sensor_types type,
switch (attr) {
case hwmon_temp_input:
/* Is it too early to return a conversion ? */
- if (time_before(jiffies, tmp102->ready_time)) {
+ if (time_before64(get_jiffies_64(), tmp102->ready_time)) {
dev_dbg(dev, "%s: Conversion not ready yet..\n", __func__);
return -EAGAIN;
}
@@ -270,7 +270,7 @@ static int tmp102_probe(struct i2c_client *client)
* Mark that we are not ready with data until the first
* conversion is complete
*/
- tmp102->ready_time = jiffies + msecs_to_jiffies(CONVERSION_TIME_MS);
+ tmp102->ready_time = get_jiffies_64() + msecs_to_jiffies(CONVERSION_TIME_MS);
hwmon_dev = devm_hwmon_device_register_with_info(dev, client->name,
tmp102,
@@ -303,7 +303,7 @@ static int tmp102_resume(struct device *dev)
err = regmap_update_bits(tmp102->regmap, TMP102_CONF_REG,
TMP102_CONF_SD, 0);
- tmp102->ready_time = jiffies + msecs_to_jiffies(CONVERSION_TIME_MS);
+ tmp102->ready_time = get_jiffies_64() + msecs_to_jiffies(CONVERSION_TIME_MS);
return err;
}
diff --git a/drivers/hwmon/tmp108.c b/drivers/hwmon/tmp108.c
index a971ff628435..b298fae9296c 100644
--- a/drivers/hwmon/tmp108.c
+++ b/drivers/hwmon/tmp108.c
@@ -74,7 +74,7 @@
struct tmp108 {
struct regmap *regmap;
u16 orig_config;
- unsigned long ready_time;
+ u64 ready_time;
};
/* convert 12-bit TMP108 register value to milliCelsius */
@@ -125,7 +125,7 @@ static int tmp108_read(struct device *dev, enum hwmon_sensor_types type,
switch (attr) {
case hwmon_temp_input:
/* Is it too early to return a conversion ? */
- if (time_before(jiffies, tmp108->ready_time)) {
+ if (time_before64(get_jiffies_64(), tmp108->ready_time)) {
dev_dbg(dev, "%s: Conversion not ready yet..\n",
__func__);
return -EAGAIN;
@@ -362,7 +362,7 @@ static int tmp108_common_probe(struct device *dev, struct regmap *regmap, char *
return err;
}
- tmp108->ready_time = jiffies;
+ tmp108->ready_time = get_jiffies_64();
if ((tmp108->orig_config & TMP108_CONF_MODE_MASK) ==
TMP108_MODE_SHUTDOWN)
tmp108->ready_time +=
@@ -413,7 +413,7 @@ static int tmp108_resume(struct device *dev)
err = regmap_update_bits(tmp108->regmap, TMP108_REG_CONF,
TMP108_CONF_MODE_MASK, TMP108_MODE_CONTINUOUS);
- tmp108->ready_time = jiffies +
+ tmp108->ready_time = get_jiffies_64() +
msecs_to_jiffies(TMP108_CONVERSION_TIME_MS);
return err;
}
diff --git a/drivers/iio/adc/ad4080.c b/drivers/iio/adc/ad4080.c
index e15310fcd21a..584f273c6bb3 100644
--- a/drivers/iio/adc/ad4080.c
+++ b/drivers/iio/adc/ad4080.c
@@ -455,7 +455,8 @@ static int ad4080_setup(struct iio_dev *indio_dev)
return ret;
ret = regmap_write(st->regmap, AD4080_REG_INTERFACE_CONFIG_A,
- AD4080_INTERFACE_CONFIG_A_SDO_ENABLE);
+ AD4080_INTERFACE_CONFIG_A_SDO_ENABLE |
+ AD4080_INTERFACE_CONFIG_A_ADDR_ASC);
if (ret)
return ret;
diff --git a/drivers/iio/adc/ad7192.c b/drivers/iio/adc/ad7192.c
index 530e1d307860..394137a127ab 100644
--- a/drivers/iio/adc/ad7192.c
+++ b/drivers/iio/adc/ad7192.c
@@ -1403,7 +1403,7 @@ static int ad7192_probe(struct spi_device *spi)
st->chip_info = spi_get_device_match_data(spi);
if (!st->chip_info)
- return -ENODEV;
+ return -ENODATA;
indio_dev->name = st->chip_info->name;
indio_dev->modes = INDIO_DIRECT_MODE;
diff --git a/drivers/iio/dac/mcp4821.c b/drivers/iio/dac/mcp4821.c
index 748bdca9a964..daaebf175fed 100644
--- a/drivers/iio/dac/mcp4821.c
+++ b/drivers/iio/dac/mcp4821.c
@@ -172,9 +172,13 @@ static const struct iio_info mcp4821_info = {
static int mcp4821_probe(struct spi_device *spi)
{
+ const struct mcp4821_chip_info *info;
struct iio_dev *indio_dev;
struct mcp4821_state *state;
- const struct mcp4821_chip_info *info;
+
+ info = spi_get_device_match_data(spi);
+ if (!info)
+ return -ENODATA;
indio_dev = devm_iio_device_alloc(&spi->dev, sizeof(*state));
if (indio_dev == NULL)
@@ -183,7 +187,6 @@ static int mcp4821_probe(struct spi_device *spi)
state = iio_priv(indio_dev);
state->spi = spi;
- info = spi_get_device_match_data(spi);
indio_dev->name = info->name;
indio_dev->info = &mcp4821_info;
indio_dev->modes = INDIO_DIRECT_MODE;
diff --git a/drivers/iio/light/adux1020.c b/drivers/iio/light/adux1020.c
index 66ff9c5fb66a..df8a85826d19 100644
--- a/drivers/iio/light/adux1020.c
+++ b/drivers/iio/light/adux1020.c
@@ -808,10 +808,8 @@ static int adux1020_probe(struct i2c_client *client)
NULL, adux1020_interrupt_handler,
IRQF_TRIGGER_HIGH | IRQF_ONESHOT,
ADUX1020_DRV_NAME, indio_dev);
- if (ret) {
- dev_err(&client->dev, "irq request error %d\n", -ret);
+ if (ret)
return ret;
- }
}
return devm_iio_device_register(&client->dev, indio_dev);
diff --git a/drivers/iio/light/apds9160.c b/drivers/iio/light/apds9160.c
index 9b8af11b7b67..fc8a9fbd0974 100644
--- a/drivers/iio/light/apds9160.c
+++ b/drivers/iio/light/apds9160.c
@@ -1545,11 +1545,8 @@ static int apds9160_probe(struct i2c_client *client)
apds9160_irq_handler,
IRQF_ONESHOT, "apds9160_event",
indio_dev);
- if (ret) {
- return dev_err_probe(dev, ret,
- "request irq (%d) failed\n",
- client->irq);
- }
+ if (ret)
+ return ret;
} else {
indio_dev->info = &apds9160_info_no_events;
indio_dev->channels = apds9160_channels_without_events;
diff --git a/drivers/iio/light/apds9300.c b/drivers/iio/light/apds9300.c
index 05ba21675063..7053b310cb92 100644
--- a/drivers/iio/light/apds9300.c
+++ b/drivers/iio/light/apds9300.c
@@ -432,10 +432,8 @@ static int apds9300_probe(struct i2c_client *client)
NULL, apds9300_interrupt_handler,
IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
"apds9300_event", indio_dev);
- if (ret) {
- dev_err(&client->dev, "irq request error %d\n", -ret);
+ if (ret)
goto err;
- }
}
ret = iio_device_register(indio_dev);
diff --git a/drivers/iio/light/apds9306.c b/drivers/iio/light/apds9306.c
index fc07518faec7..dbcd8939e071 100644
--- a/drivers/iio/light/apds9306.c
+++ b/drivers/iio/light/apds9306.c
@@ -1286,8 +1286,7 @@ static int apds9306_probe(struct i2c_client *client)
apds9306_irq_handler, IRQF_ONESHOT,
"apds9306_event", indio_dev);
if (ret)
- return dev_err_probe(dev, ret,
- "failed to assign interrupt.\n");
+ return ret;
} else {
indio_dev->info = &apds9306_info_no_events;
indio_dev->channels = apds9306_channels_without_events;
diff --git a/drivers/iio/light/apds9960.c b/drivers/iio/light/apds9960.c
index 79b202c59a0f..ddcc94d398cd 100644
--- a/drivers/iio/light/apds9960.c
+++ b/drivers/iio/light/apds9960.c
@@ -1098,10 +1098,8 @@ static int apds9960_probe(struct i2c_client *client)
IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
"apds9960_event",
indio_dev);
- if (ret) {
- dev_err(&client->dev, "request irq (%d) failed\n", client->irq);
+ if (ret)
goto error_power_down;
- }
ret = iio_device_register(indio_dev);
if (ret)
diff --git a/drivers/iio/light/bh1745.c b/drivers/iio/light/bh1745.c
index 10b00344bbed..31e94813b111 100644
--- a/drivers/iio/light/bh1745.c
+++ b/drivers/iio/light/bh1745.c
@@ -784,8 +784,7 @@ static int bh1745_setup_triggered_buffer(struct iio_dev *indio_dev,
IRQF_ONESHOT,
"bh1745_interrupt", indio_dev);
if (ret)
- return dev_err_probe(dev, ret,
- "Request for IRQ failed\n");
+ return ret;
}
return 0;
diff --git a/drivers/iio/light/cm3605.c b/drivers/iio/light/cm3605.c
index 0c17378e27d1..1690e3ce4283 100644
--- a/drivers/iio/light/cm3605.c
+++ b/drivers/iio/light/cm3605.c
@@ -233,10 +233,8 @@ static int cm3605_probe(struct platform_device *pdev)
ret = devm_request_threaded_irq(dev, irq, cm3605_prox_irq,
NULL, 0, "cm3605", indio_dev);
- if (ret) {
- dev_err(dev, "unable to request IRQ\n");
+ if (ret)
goto out_disable_aset;
- }
/* Just name the trigger the same as the driver */
led_trigger_register_simple("cm3605", &cm3605->led);
diff --git a/drivers/iio/light/gp2ap002.c b/drivers/iio/light/gp2ap002.c
index bb17ef2a829b..0a8eee74c494 100644
--- a/drivers/iio/light/gp2ap002.c
+++ b/drivers/iio/light/gp2ap002.c
@@ -580,10 +580,8 @@ static int gp2ap002_probe(struct i2c_client *client)
ret = devm_request_threaded_irq(dev, client->irq, NULL,
gp2ap002_prox_irq, IRQF_ONESHOT,
"gp2ap002", indio_dev);
- if (ret) {
- dev_err(dev, "unable to request IRQ\n");
+ if (ret)
goto out_put_pm;
- }
gp2ap002->irq = client->irq;
/*
diff --git a/drivers/iio/light/isl29028.c b/drivers/iio/light/isl29028.c
index f89c5d9ee524..408b32d096a8 100644
--- a/drivers/iio/light/isl29028.c
+++ b/drivers/iio/light/isl29028.c
@@ -333,16 +333,6 @@ static int isl29028_ir_get(struct isl29028_chip *chip, int *ir_data)
return isl29028_read_als_ir(chip, ir_data);
}
-static int isl29028_set_pm_runtime_busy(struct isl29028_chip *chip, bool on)
-{
- struct device *dev = regmap_get_device(chip->regmap);
-
- if (on)
- return pm_runtime_resume_and_get(dev);
-
- return pm_runtime_put_autosuspend(dev);
-}
-
/* Channel IO */
static int isl29028_write_raw(struct iio_dev *indio_dev,
struct iio_chan_spec const *chan,
@@ -352,8 +342,9 @@ static int isl29028_write_raw(struct iio_dev *indio_dev,
struct device *dev = regmap_get_device(chip->regmap);
int ret;
- ret = isl29028_set_pm_runtime_busy(chip, true);
- if (ret < 0)
+ PM_RUNTIME_ACQUIRE_AUTOSUSPEND(dev, pm);
+ ret = PM_RUNTIME_ACQUIRE_ERR(&pm);
+ if (ret)
return ret;
mutex_lock(&chip->lock);
@@ -402,14 +393,7 @@ static int isl29028_write_raw(struct iio_dev *indio_dev,
mutex_unlock(&chip->lock);
- if (ret < 0)
- return ret;
-
- ret = isl29028_set_pm_runtime_busy(chip, false);
- if (ret < 0)
- return ret;
-
- return 0;
+ return ret;
}
static int isl29028_read_raw(struct iio_dev *indio_dev,
@@ -418,10 +402,11 @@ static int isl29028_read_raw(struct iio_dev *indio_dev,
{
struct isl29028_chip *chip = iio_priv(indio_dev);
struct device *dev = regmap_get_device(chip->regmap);
- int ret, pm_ret;
+ int ret;
- ret = isl29028_set_pm_runtime_busy(chip, true);
- if (ret < 0)
+ PM_RUNTIME_ACQUIRE_AUTOSUSPEND(dev, pm);
+ ret = PM_RUNTIME_ACQUIRE_ERR(&pm);
+ if (ret)
return ret;
mutex_lock(&chip->lock);
@@ -471,18 +456,6 @@ static int isl29028_read_raw(struct iio_dev *indio_dev,
mutex_unlock(&chip->lock);
- if (ret < 0)
- return ret;
-
- /**
- * Preserve the ret variable if the call to
- * isl29028_set_pm_runtime_busy() is successful so the reading
- * (if applicable) is returned to user space.
- */
- pm_ret = isl29028_set_pm_runtime_busy(chip, false);
- if (pm_ret < 0)
- return pm_ret;
-
return ret;
}
diff --git a/drivers/iio/light/ltr390.c b/drivers/iio/light/ltr390.c
index a2b804e9089a..d40bd98dfd11 100644
--- a/drivers/iio/light/ltr390.c
+++ b/drivers/iio/light/ltr390.c
@@ -838,8 +838,7 @@ static int ltr390_probe(struct i2c_client *client)
"ltr390_thresh_event",
indio_dev);
if (ret)
- return dev_err_probe(dev, ret,
- "request irq (%d) failed\n", client->irq);
+ return ret;
}
ret = ltr390_pm_init(data);
diff --git a/drivers/iio/light/ltr501.c b/drivers/iio/light/ltr501.c
index 022e0693983b..a38cf1490e13 100644
--- a/drivers/iio/light/ltr501.c
+++ b/drivers/iio/light/ltr501.c
@@ -1538,11 +1538,8 @@ static int ltr501_probe(struct i2c_client *client)
IRQF_ONESHOT,
"ltr501_thresh_event",
indio_dev);
- if (ret) {
- dev_err(&client->dev, "request irq (%d) failed\n",
- client->irq);
- return ret;
- }
+ if (ret)
+ goto powerdown_on_error;
} else {
indio_dev->info = data->chip_info->info_no_irq;
}
diff --git a/drivers/iio/light/opt4060.c b/drivers/iio/light/opt4060.c
index 9be752535d75..70aa8b46784a 100644
--- a/drivers/iio/light/opt4060.c
+++ b/drivers/iio/light/opt4060.c
@@ -1223,7 +1223,7 @@ static int opt4060_setup_trigger(struct opt4060_chip *chip, struct iio_dev *idev
IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
name, idev);
if (ret)
- return dev_err_probe(chip->dev, ret, "Could not request IRQ\n");
+ return ret;
init_completion(&chip->completion);
diff --git a/drivers/iio/light/rpr0521.c b/drivers/iio/light/rpr0521.c
index 9341c1d58cbe..13e498492092 100644
--- a/drivers/iio/light/rpr0521.c
+++ b/drivers/iio/light/rpr0521.c
@@ -988,11 +988,8 @@ static int rpr0521_probe(struct i2c_client *client)
rpr0521_drdy_irq_handler, rpr0521_drdy_irq_thread,
IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
"rpr0521_event", indio_dev);
- if (ret < 0) {
- dev_err(&client->dev, "request irq %d for trigger0 failed\n",
- client->irq);
+ if (ret)
goto err_pm_disable;
- }
ret = devm_iio_trigger_register(indio_dev->dev.parent,
data->drdy_trigger0);
diff --git a/drivers/iio/light/si1145.c b/drivers/iio/light/si1145.c
index ef0abc4499b7..a36d1d0346a8 100644
--- a/drivers/iio/light/si1145.c
+++ b/drivers/iio/light/si1145.c
@@ -1251,10 +1251,8 @@ static int si1145_probe_trigger(struct iio_dev *indio_dev)
IRQF_TRIGGER_FALLING | IRQF_NO_THREAD,
"si1145_irq",
trig);
- if (ret < 0) {
- dev_err(&client->dev, "irq request failed\n");
+ if (ret)
return ret;
- }
ret = devm_iio_trigger_register(&client->dev, trig);
if (ret)
diff --git a/drivers/iio/light/st_uvis25_core.c b/drivers/iio/light/st_uvis25_core.c
index bcd729a9924e..94d5261b2663 100644
--- a/drivers/iio/light/st_uvis25_core.c
+++ b/drivers/iio/light/st_uvis25_core.c
@@ -196,11 +196,8 @@ static int st_uvis25_allocate_trigger(struct iio_dev *iio_dev)
st_uvis25_trigger_handler_thread,
irq_type | IRQF_ONESHOT,
iio_dev->name, hw);
- if (err) {
- dev_err(dev, "failed to request trigger irq %d\n",
- hw->irq);
+ if (err)
return err;
- }
hw->trig = devm_iio_trigger_alloc(dev, "%s-trigger",
iio_dev->name);
diff --git a/drivers/iio/light/stk3310.c b/drivers/iio/light/stk3310.c
index 3be693421886..ca1cb5dd776a 100644
--- a/drivers/iio/light/stk3310.c
+++ b/drivers/iio/light/stk3310.c
@@ -661,11 +661,8 @@ static int stk3310_probe(struct i2c_client *client)
IRQF_TRIGGER_FALLING |
IRQF_ONESHOT,
"stk3310_event", indio_dev);
- if (ret < 0) {
- dev_err(&client->dev, "request irq %d failed\n",
- client->irq);
+ if (ret)
goto err_standby;
- }
}
ret = iio_device_register(indio_dev);
diff --git a/drivers/iio/light/tsl2563.c b/drivers/iio/light/tsl2563.c
index f2af1cd7c2d1..4e3daae227ae 100644
--- a/drivers/iio/light/tsl2563.c
+++ b/drivers/iio/light/tsl2563.c
@@ -755,7 +755,7 @@ static int tsl2563_probe(struct i2c_client *client)
"tsl2563_event",
indio_dev);
if (err)
- return dev_err_probe(dev, err, "irq request error\n");
+ return err;
}
err = tsl2563_configure(chip);
diff --git a/drivers/iio/light/tsl2772.c b/drivers/iio/light/tsl2772.c
index ffbde1540114..e54265f605dd 100644
--- a/drivers/iio/light/tsl2772.c
+++ b/drivers/iio/light/tsl2772.c
@@ -1837,11 +1837,8 @@ static int tsl2772_probe(struct i2c_client *clientp)
IRQF_ONESHOT,
"TSL2772_event",
indio_dev);
- if (ret) {
- dev_err(&clientp->dev,
- "%s: irq request failed\n", __func__);
+ if (ret)
return ret;
- }
} else {
indio_dev->channels = chip->chip_info->channel_without_events;
}
diff --git a/drivers/iio/light/veml6030.c b/drivers/iio/light/veml6030.c
index da8c32cabfd6..ad3bf9ab07e7 100644
--- a/drivers/iio/light/veml6030.c
+++ b/drivers/iio/light/veml6030.c
@@ -925,9 +925,7 @@ static int veml6030_set_info(struct iio_dev *indio_dev)
IRQF_TRIGGER_LOW | IRQF_ONESHOT,
indio_dev->name, indio_dev);
if (ret < 0)
- return dev_err_probe(&client->dev, ret,
- "irq %d request failed\n",
- client->irq);
+ return ret;
indio_dev->info = &veml6030_info;
} else {
diff --git a/drivers/iio/light/vl6180.c b/drivers/iio/light/vl6180.c
index c1314b144367..127904704df3 100644
--- a/drivers/iio/light/vl6180.c
+++ b/drivers/iio/light/vl6180.c
@@ -722,7 +722,7 @@ static int vl6180_probe(struct i2c_client *client)
IRQF_ONESHOT,
indio_dev->name, indio_dev);
if (ret)
- return dev_err_probe(&client->dev, ret, "devm_request_irq error\n");
+ return ret;
init_completion(&data->completion);
diff --git a/drivers/iio/pressure/dps310.c b/drivers/iio/pressure/dps310.c
index 41b2d0b014e1..a617f26eeb30 100644
--- a/drivers/iio/pressure/dps310.c
+++ b/drivers/iio/pressure/dps310.c
@@ -50,9 +50,9 @@
#define DPS310_CFG_REG 0x09
#define DPS310_INT_HL BIT(7)
#define DPS310_TMP_SHIFT_EN BIT(3)
-#define DPS310_PRS_SHIFT_EN BIT(4)
-#define DPS310_FIFO_EN BIT(5)
-#define DPS310_SPI_EN BIT(6)
+#define DPS310_PRS_SHIFT_EN BIT(2)
+#define DPS310_FIFO_EN BIT(1)
+#define DPS310_SPI_EN BIT(0)
#define DPS310_RESET 0x0c
#define DPS310_RESET_MAGIC 0x09
#define DPS310_COEF_BASE 0x10
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index 16c26d75f46f..c4694ff364db 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -1736,6 +1736,21 @@ static int __init init_iommu_from_acpi(struct amd_iommu *iommu,
return 0;
}
+/*
+ * Apply workaround for Zen1 to Zen5(BRH) generation of processors.
+ * References: CVE-2024-21963
+ */
+static bool __init amd_iommu_force_max_dev_table(void)
+{
+ if (boot_cpu_data.x86 >= 0x17 && boot_cpu_data.x86 <= 0x1a) {
+ if (cpu_feature_enabled(X86_FEATURE_ZEN6))
+ return false;
+ else
+ return true;
+ }
+ return false;
+}
+
/* Allocate PCI segment data structure */
static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id,
struct acpi_table_header *ivrs_base)
@@ -1758,9 +1773,15 @@ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id,
pci_seg->last_bdf = last_bdf;
DUMP_printk("PCI segment : 0x%0x, last bdf : 0x%04x\n", id, last_bdf);
- pci_seg->dev_table_size =
- max(roundup_pow_of_two((last_bdf + 1) * DEV_TABLE_ENTRY_SIZE),
- SZ_4K);
+ if (amd_iommu_force_max_dev_table()) {
+ pr_info("Force device table to maximum size\n");
+ pci_seg->dev_table_size = SZ_2M;
+ } else {
+ pci_seg->dev_table_size =
+ max(roundup_pow_of_two((last_bdf + 1) *
+ DEV_TABLE_ENTRY_SIZE),
+ SZ_4K);
+ }
pci_seg->id = id;
init_llist_head(&pci_seg->dev_data_list);
@@ -2721,6 +2742,15 @@ static int __init init_unity_map_range(struct ivmd_header *m,
return 0;
}
+static bool ivrs_block_type_ivmd(u8 type)
+{
+ if (type == ACPI_IVMD_TYPE ||
+ type == ACPI_IVMD_TYPE_ALL ||
+ type == ACPI_IVMD_TYPE_RANGE)
+ return true;
+ return false;
+}
+
/* iterates over all memory definitions we find in the ACPI table */
static int __init init_memory_definitions(struct acpi_table_header *table)
{
@@ -2732,10 +2762,11 @@ static int __init init_memory_definitions(struct acpi_table_header *table)
while (p < end) {
m = (struct ivmd_header *)p;
+ p += m->length;
+ if (!ivrs_block_type_ivmd(m->type))
+ continue;
if (m->flags & (IVMD_FLAG_UNITY_MAP | IVMD_FLAG_EXCL_RANGE))
init_unity_map_range(m, table);
-
- p += m->length;
}
return 0;
diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c
index 77891d4d65fb..a6face8a46f5 100644
--- a/drivers/iommu/iommufd/io_pagetable.c
+++ b/drivers/iommu/iommufd/io_pagetable.c
@@ -552,7 +552,7 @@ iommu_read_and_clear_dirty(struct iommu_domain *domain,
arg.iopt = iopt;
arg.domain = domain;
arg.dirty = &dirty;
- iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty);
+ ret = iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty);
if (!(flags & IOMMU_DIRTY_NO_CLEAR))
iommu_iotlb_sync(domain, &gather);
diff --git a/drivers/iommu/iommufd/iova_bitmap.c b/drivers/iommu/iommufd/iova_bitmap.c
index b5b67a9d3fb3..3a7eca3c7d98 100644
--- a/drivers/iommu/iommufd/iova_bitmap.c
+++ b/drivers/iommu/iommufd/iova_bitmap.c
@@ -116,6 +116,8 @@ struct iova_bitmap {
/* length of the IOVA range for the whole bitmap */
size_t length;
+
+ int err;
};
/*
@@ -384,7 +386,9 @@ static int iova_bitmap_advance_to(struct iova_bitmap *bitmap,
int iova_bitmap_for_each(struct iova_bitmap *bitmap, void *opaque,
iova_bitmap_fn_t fn)
{
- return fn(bitmap, bitmap->iova, bitmap->length, opaque);
+ int ret = fn(bitmap, bitmap->iova, bitmap->length, opaque);
+
+ return ret ? ret : bitmap->err;
}
EXPORT_SYMBOL_NS_GPL(iova_bitmap_for_each, "IOMMUFD");
@@ -404,6 +408,18 @@ void iova_bitmap_set(struct iova_bitmap *bitmap,
struct iova_bitmap_map *mapped = &bitmap->mapped;
unsigned long cur_bit, last_bit, last_page_idx;
+ /*
+ * If the range cannot be recorded then the system call returns failure.
+ * This is to prevent loosing dirties if userspace is expecting to step
+ * the bitmap through the IOVA space.
+ */
+ if (!length || length > bitmap->length || iova < bitmap->iova ||
+ iova - bitmap->iova > bitmap->length - length) {
+ if (!bitmap->err)
+ bitmap->err = -EPROTO;
+ return;
+ }
+
update_indexes:
if (unlikely(!iova_bitmap_mapped_range(mapped, iova, length))) {
/*
diff --git a/drivers/iommu/virtio-iommu.c b/drivers/iommu/virtio-iommu.c
index d314fa5cd847..3aa54bd71672 100644
--- a/drivers/iommu/virtio-iommu.c
+++ b/drivers/iommu/virtio-iommu.c
@@ -1227,12 +1227,12 @@ static int viommu_probe(struct virtio_device *vdev)
/* Populate the event queue with buffers */
ret = viommu_fill_evtq(viommu);
if (ret)
- goto err_free_vqs;
+ goto err_reset_vdev;
ret = iommu_device_sysfs_add(&viommu->iommu, dev, NULL, "%s",
virtio_bus_name(vdev));
if (ret)
- goto err_free_vqs;
+ goto err_reset_vdev;
vdev->priv = viommu;
@@ -1244,6 +1244,8 @@ static int viommu_probe(struct virtio_device *vdev)
return 0;
+err_reset_vdev:
+ virtio_reset_device(vdev);
err_free_vqs:
vdev->config->del_vqs(vdev);
diff --git a/drivers/irqchip/irq-renesas-rzv2h.c b/drivers/irqchip/irq-renesas-rzv2h.c
index c9f9099a5129..927b5ffa0c7c 100644
--- a/drivers/irqchip/irq-renesas-rzv2h.c
+++ b/drivers/irqchip/irq-renesas-rzv2h.c
@@ -328,9 +328,9 @@ static void rzv2h_clear_tint_int(struct rzv2h_icu_priv *priv, unsigned int hwirq
static int rzv2h_tint_set_type(struct irq_data *d, unsigned int type)
{
u32 titsr, titsr_k, titsel_n, tien;
+ u32 titsr_cur, tssr_cur, tien_cur;
struct rzv2h_icu_priv *priv;
u32 tssr, tssr_k, tssel_n;
- u32 titsr_cur, tssr_cur;
unsigned int hwirq;
u32 tint, sense;
int tint_nr;
@@ -386,6 +386,8 @@ static int rzv2h_tint_set_type(struct irq_data *d, unsigned int type)
if (tssr_cur == tint && titsr_cur == sense)
return 0;
+ tien_cur = tssr & tien;
+
tssr &= ~(ICU_TSSR_TSSEL_MASK(tssel_n, priv->info->field_width) | tien);
tssr |= ICU_TSSR_TSSEL_PREP(tint, tssel_n, priv->info->field_width);
@@ -398,7 +400,7 @@ static int rzv2h_tint_set_type(struct irq_data *d, unsigned int type)
rzv2h_clear_tint_int(priv, hwirq);
- writel_relaxed(tssr | tien, priv->base + priv->info->t_offs + ICU_TSSR(tssr_k));
+ writel_relaxed(tssr | tien_cur, priv->base + priv->info->t_offs + ICU_TSSR(tssr_k));
return 0;
}
diff --git a/drivers/media/common/cypress_firmware.c b/drivers/media/common/cypress_firmware.c
index cdc7050ed3ac..3468eff508b0 100644
--- a/drivers/media/common/cypress_firmware.c
+++ b/drivers/media/common/cypress_firmware.c
@@ -59,6 +59,8 @@ static int cypress_get_hexline(const struct firmware *fw,
if (hx->type == 0x04) {
/* b[4] and b[5] are the Extended linear address record data
* field */
+ if (hx->len != 2)
+ return -EINVAL;
hx->addr |= (b[4] << 24) | (b[5] << 16);
}
diff --git a/drivers/media/i2c/adv748x/adv748x-core.c b/drivers/media/i2c/adv748x/adv748x-core.c
index 3eb6d5e8f082..70594d001804 100644
--- a/drivers/media/i2c/adv748x/adv748x-core.c
+++ b/drivers/media/i2c/adv748x/adv748x-core.c
@@ -675,9 +675,6 @@ static int adv748x_parse_dt(struct adv748x_state *state)
continue;
}
- of_node_get(ep_np);
- state->endpoints[ep.port] = ep_np;
-
/*
* At least one input endpoint and one output endpoint shall
* be defined.
@@ -689,8 +686,13 @@ static int adv748x_parse_dt(struct adv748x_state *state)
/* Store number of CSI-2 lanes used for TXA and TXB. */
ret = adv748x_parse_csi2_lanes(state, ep.port, ep_np);
- if (ret)
+ if (ret) {
+ of_node_put(ep_np);
return ret;
+ }
+
+ of_node_get(ep_np);
+ state->endpoints[ep.port] = ep_np;
}
return in_found && out_found ? 0 : -ENODEV;
@@ -739,7 +741,7 @@ static int adv748x_probe(struct i2c_client *client)
ret = adv748x_parse_dt(state);
if (ret) {
adv_err(state, "Failed to parse device tree");
- goto err_free_mutex;
+ goto err_cleanup_dt;
}
/* Configure IO Regmap region */
@@ -809,7 +811,6 @@ static int adv748x_probe(struct i2c_client *client)
adv748x_unregister_clients(state);
err_cleanup_dt:
adv748x_dt_cleanup(state);
-err_free_mutex:
mutex_destroy(&state->mutex);
return ret;
diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 00aec91e3d49..e399a87f9926 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -68,6 +68,7 @@
#define IMX219_EXPOSURE_STEP 1
#define IMX219_EXPOSURE_DEFAULT 0x640
#define IMX219_EXPOSURE_MAX 65535
+#define IMX219_EXPOSURE_OFFSET 4
/* V_TIMING internal */
#define IMX219_REG_FRM_LENGTH_A CCI_REG16(0x0160)
@@ -77,6 +78,7 @@
#define IMX219_LLP_MIN 0x0d78
#define IMX219_BINNED_LLP_MIN 0x0de8
#define IMX219_LLP_MAX 0x7ff0
+#define IMX219_LLP_STEP 8
#define IMX219_REG_X_ADD_STA_A CCI_REG16(0x0164)
#define IMX219_REG_X_ADD_END_A CCI_REG16(0x0166)
@@ -324,13 +326,13 @@ static const struct imx219_mode supported_modes[] = {
/* 2x2 binned 60fps mode */
.width = 1640,
.height = 1232,
- .fll_def = 1707,
+ .fll_def = 1706,
},
{
/* 640x480 60fps mode */
.width = 640,
.height = 480,
- .fll_def = 1707,
+ .fll_def = 1706,
},
};
@@ -460,13 +462,17 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
int exposure_max, exposure_def;
/* Update max exposure while meeting expected vblanking */
- exposure_max = format->height + ctrl->val - 4;
+ exposure_max = format->height + ctrl->val -
+ IMX219_EXPOSURE_OFFSET * rate_factor;
exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
- exposure_max : IMX219_EXPOSURE_DEFAULT;
- __v4l2_ctrl_modify_range(imx219->exposure,
- imx219->exposure->minimum,
- exposure_max, imx219->exposure->step,
- exposure_def);
+ exposure_max : IMX219_EXPOSURE_DEFAULT;
+ ret = __v4l2_ctrl_modify_range(imx219->exposure,
+ imx219->exposure->minimum,
+ exposure_max,
+ rate_factor, exposure_def);
+ if (ret)
+ return ret;
+
}
/*
@@ -583,11 +589,12 @@ static int imx219_init_controls(struct imx219 *imx219)
imx219->hblank = v4l2_ctrl_new_std(ctrl_hdlr, &imx219_ctrl_ops,
V4L2_CID_HBLANK,
IMX219_LLP_MIN - mode->width,
- IMX219_LLP_MAX - mode->width, 1,
+ IMX219_LLP_MAX - mode->width,
+ IMX219_LLP_STEP,
IMX219_LLP_MIN - mode->width);
- exposure_max = mode->fll_def - 4;
+ exposure_max = mode->fll_def - IMX219_EXPOSURE_OFFSET;
exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
- exposure_max : IMX219_EXPOSURE_DEFAULT;
+ exposure_max : IMX219_EXPOSURE_DEFAULT;
imx219->exposure = v4l2_ctrl_new_std(ctrl_hdlr, &imx219_ctrl_ops,
V4L2_CID_EXPOSURE,
IMX219_EXPOSURE_MIN, exposure_max,
@@ -858,6 +865,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
struct v4l2_rect *crop;
u8 bin_h, bin_v, binning;
u32 prev_line_len;
+ int ret;
format = v4l2_subdev_state_get_format(state, 0);
prev_line_len = format->width + imx219->hblank->val;
@@ -887,25 +895,36 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
+ int rate_factor = imx219_get_rate_factor(state);
int exposure_max;
int exposure_def;
int hblank, llp_min;
int pixel_rate;
/* Update limits and set FPS to default */
- __v4l2_ctrl_modify_range(imx219->vblank, IMX219_VBLANK_MIN,
- IMX219_FLL_MAX - mode->height, 1,
+ ret = __v4l2_ctrl_modify_range(imx219->vblank, IMX219_VBLANK_MIN,
+ IMX219_FLL_MAX - mode->height,
+ rate_factor,
+ mode->fll_def - mode->height);
+ if (ret)
+ return ret;
+
+ ret = __v4l2_ctrl_s_ctrl(imx219->vblank,
mode->fll_def - mode->height);
- __v4l2_ctrl_s_ctrl(imx219->vblank,
- mode->fll_def - mode->height);
+ if (ret)
+ return ret;
+
/* Update max exposure while meeting expected vblanking */
- exposure_max = mode->fll_def - 4;
+ exposure_max = mode->fll_def -
+ IMX219_EXPOSURE_OFFSET * rate_factor;
exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
- exposure_max : IMX219_EXPOSURE_DEFAULT;
- __v4l2_ctrl_modify_range(imx219->exposure,
- imx219->exposure->minimum,
- exposure_max, imx219->exposure->step,
- exposure_def);
+ exposure_max : IMX219_EXPOSURE_DEFAULT;
+ ret = __v4l2_ctrl_modify_range(imx219->exposure,
+ imx219->exposure->minimum,
+ exposure_max,
+ rate_factor, exposure_def);
+ if (ret)
+ return ret;
/*
* With analog binning the default minimum line length of 3448
@@ -916,9 +935,13 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
imx219_get_binning(state, &bin_h, &bin_v);
llp_min = (bin_h & bin_v) == IMX219_BINNING_X2_ANALOG ?
IMX219_BINNED_LLP_MIN : IMX219_LLP_MIN;
- __v4l2_ctrl_modify_range(imx219->hblank, llp_min - mode->width,
- IMX219_LLP_MAX - mode->width, 1,
- llp_min - mode->width);
+ ret = __v4l2_ctrl_modify_range(imx219->hblank,
+ llp_min - mode->width,
+ IMX219_LLP_MAX - mode->width,
+ IMX219_LLP_STEP,
+ llp_min - mode->width);
+ if (ret)
+ return ret;
/*
* Retain PPL setting from previous mode so that the
* line time does not change on a mode change.
@@ -927,13 +950,16 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
* mode width subtracted.
*/
hblank = prev_line_len - mode->width;
- __v4l2_ctrl_s_ctrl(imx219->hblank, hblank);
+ ret = __v4l2_ctrl_s_ctrl(imx219->hblank, hblank);
+ if (ret)
+ return ret;
/* Scale the pixel rate based on the mode specific factor */
- pixel_rate = imx219_get_pixel_rate(imx219) *
- imx219_get_rate_factor(state);
- __v4l2_ctrl_modify_range(imx219->pixel_rate, pixel_rate,
- pixel_rate, 1, pixel_rate);
+ pixel_rate = imx219_get_pixel_rate(imx219) * rate_factor;
+ ret = __v4l2_ctrl_modify_range(imx219->pixel_rate, pixel_rate,
+ pixel_rate, 1, pixel_rate);
+ if (ret)
+ return ret;
}
return 0;
@@ -982,9 +1008,7 @@ static int imx219_init_state(struct v4l2_subdev *sd,
},
};
- imx219_set_pad_format(sd, state, &fmt);
-
- return 0;
+ return imx219_set_pad_format(sd, state, &fmt);
}
static const struct v4l2_subdev_video_ops imx219_video_ops = {
diff --git a/drivers/media/i2c/og01a1b.c b/drivers/media/i2c/og01a1b.c
index 7b892b26203c..68ebf4487678 100644
--- a/drivers/media/i2c/og01a1b.c
+++ b/drivers/media/i2c/og01a1b.c
@@ -442,9 +442,6 @@ struct og01a1b {
/* Current mode */
const struct og01a1b_mode *cur_mode;
-
- /* To serialize asynchronus callbacks */
- struct mutex mutex;
};
static u64 to_pixel_rate(u32 f_index)
@@ -616,7 +613,6 @@ static int og01a1b_init_controls(struct og01a1b *og01a1b)
if (ret)
return ret;
- ctrl_hdlr->lock = &og01a1b->mutex;
og01a1b->link_freq = v4l2_ctrl_new_int_menu(ctrl_hdlr,
&og01a1b_ctrl_ops,
V4L2_CID_LINK_FREQ,
@@ -686,74 +682,66 @@ static void og01a1b_update_pad_format(const struct og01a1b_mode *mode,
fmt->field = V4L2_FIELD_NONE;
}
-static int og01a1b_start_streaming(struct og01a1b *og01a1b)
+static int og01a1b_enable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
{
+ struct og01a1b *og01a1b = to_og01a1b(sd);
+ unsigned int link_freq_index = og01a1b->cur_mode->link_freq_index;
const struct og01a1b_reg_list *reg_list;
- int link_freq_index, ret;
+ int ret;
- link_freq_index = og01a1b->cur_mode->link_freq_index;
- reg_list = &link_freq_configs[link_freq_index].reg_list;
+ ret = pm_runtime_resume_and_get(og01a1b->dev);
+ if (ret)
+ return ret;
+ reg_list = &link_freq_configs[link_freq_index].reg_list;
ret = og01a1b_write_reg_list(og01a1b, reg_list);
if (ret) {
- dev_err(og01a1b->dev, "failed to set plls");
- return ret;
+ dev_err(og01a1b->dev, "failed to set plls: %d\n", ret);
+ goto error;
}
reg_list = &og01a1b->cur_mode->reg_list;
ret = og01a1b_write_reg_list(og01a1b, reg_list);
if (ret) {
- dev_err(og01a1b->dev, "failed to set mode");
- return ret;
+ dev_err(og01a1b->dev, "failed to set mode: %d\n", ret);
+ goto error;
}
ret = __v4l2_ctrl_handler_setup(og01a1b->sd.ctrl_handler);
if (ret)
- return ret;
+ goto error;
ret = og01a1b_write_reg(og01a1b, OG01A1B_REG_MODE_SELECT,
OG01A1B_REG_VALUE_08BIT,
OG01A1B_MODE_STREAMING);
if (ret) {
- dev_err(og01a1b->dev, "failed to set stream");
- return ret;
+ dev_err(og01a1b->dev, "failed to start streaming: %d\n", ret);
+ goto error;
}
return 0;
-}
-static void og01a1b_stop_streaming(struct og01a1b *og01a1b)
-{
- if (og01a1b_write_reg(og01a1b, OG01A1B_REG_MODE_SELECT,
- OG01A1B_REG_VALUE_08BIT, OG01A1B_MODE_STANDBY))
- dev_err(og01a1b->dev, "failed to set stream");
+error:
+ pm_runtime_put_autosuspend(og01a1b->dev);
+
+ return ret;
}
-static int og01a1b_set_stream(struct v4l2_subdev *sd, int enable)
+static int og01a1b_disable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
{
struct og01a1b *og01a1b = to_og01a1b(sd);
- int ret = 0;
-
- mutex_lock(&og01a1b->mutex);
- if (enable) {
- ret = pm_runtime_resume_and_get(og01a1b->dev);
- if (ret) {
- mutex_unlock(&og01a1b->mutex);
- return ret;
- }
+ int ret;
- ret = og01a1b_start_streaming(og01a1b);
- if (ret) {
- enable = 0;
- og01a1b_stop_streaming(og01a1b);
- pm_runtime_put(og01a1b->dev);
- }
- } else {
- og01a1b_stop_streaming(og01a1b);
- pm_runtime_put(og01a1b->dev);
- }
+ ret = og01a1b_write_reg(og01a1b, OG01A1B_REG_MODE_SELECT,
+ OG01A1B_REG_VALUE_08BIT, OG01A1B_MODE_STANDBY);
+ if (ret)
+ dev_err(og01a1b->dev, "failed to stop streaming: %d\n", ret);
- mutex_unlock(&og01a1b->mutex);
+ pm_runtime_put_autosuspend(og01a1b->dev);
return ret;
}
@@ -771,7 +759,6 @@ static int og01a1b_set_format(struct v4l2_subdev *sd,
height, fmt->format.width,
fmt->format.height);
- mutex_lock(&og01a1b->mutex);
og01a1b_update_pad_format(mode, &fmt->format);
if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
*v4l2_subdev_state_get_format(sd_state, fmt->pad) = fmt->format;
@@ -794,8 +781,6 @@ static int og01a1b_set_format(struct v4l2_subdev *sd,
h_blank);
}
- mutex_unlock(&og01a1b->mutex);
-
return 0;
}
@@ -805,15 +790,12 @@ static int og01a1b_get_format(struct v4l2_subdev *sd,
{
struct og01a1b *og01a1b = to_og01a1b(sd);
- mutex_lock(&og01a1b->mutex);
if (fmt->which == V4L2_SUBDEV_FORMAT_TRY)
fmt->format = *v4l2_subdev_state_get_format(sd_state,
fmt->pad);
else
og01a1b_update_pad_format(og01a1b->cur_mode, &fmt->format);
- mutex_unlock(&og01a1b->mutex);
-
return 0;
}
@@ -849,18 +831,14 @@ static int og01a1b_enum_frame_size(struct v4l2_subdev *sd,
static int og01a1b_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
{
- struct og01a1b *og01a1b = to_og01a1b(sd);
-
- mutex_lock(&og01a1b->mutex);
og01a1b_update_pad_format(&supported_modes[0],
v4l2_subdev_state_get_format(fh->state, 0));
- mutex_unlock(&og01a1b->mutex);
return 0;
}
static const struct v4l2_subdev_video_ops og01a1b_video_ops = {
- .s_stream = og01a1b_set_stream,
+ .s_stream = v4l2_subdev_s_stream_helper,
};
static const struct v4l2_subdev_pad_ops og01a1b_pad_ops = {
@@ -868,6 +846,8 @@ static const struct v4l2_subdev_pad_ops og01a1b_pad_ops = {
.get_fmt = og01a1b_get_format,
.enum_mbus_code = og01a1b_enum_mbus_code,
.enum_frame_size = og01a1b_enum_frame_size,
+ .enable_streams = og01a1b_enable_streams,
+ .disable_streams = og01a1b_disable_streams,
};
static const struct v4l2_subdev_ops og01a1b_subdev_ops = {
@@ -1046,7 +1026,11 @@ static void og01a1b_remove(struct i2c_client *client)
media_entity_cleanup(&sd->entity);
v4l2_ctrl_handler_free(sd->ctrl_handler);
pm_runtime_disable(og01a1b->dev);
- mutex_destroy(&og01a1b->mutex);
+
+ if (!pm_runtime_status_suspended(og01a1b->dev)) {
+ og01a1b_power_off(og01a1b->dev);
+ pm_runtime_set_suspended(og01a1b->dev);
+ }
}
static int og01a1b_probe(struct i2c_client *client)
@@ -1135,7 +1119,6 @@ static int og01a1b_probe(struct i2c_client *client)
goto power_off;
}
- mutex_init(&og01a1b->mutex);
og01a1b->cur_mode = &supported_modes[0];
ret = og01a1b_init_controls(og01a1b);
if (ret) {
@@ -1143,6 +1126,7 @@ static int og01a1b_probe(struct i2c_client *client)
goto probe_error_v4l2_ctrl_handler_free;
}
+ og01a1b->sd.state_lock = og01a1b->ctrl_handler.lock;
og01a1b->sd.internal_ops = &og01a1b_internal_ops;
og01a1b->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
og01a1b->sd.entity.ops = &og01a1b_subdev_entity_ops;
@@ -1183,7 +1167,6 @@ static int og01a1b_probe(struct i2c_client *client)
probe_error_v4l2_ctrl_handler_free:
v4l2_ctrl_handler_free(og01a1b->sd.ctrl_handler);
- mutex_destroy(&og01a1b->mutex);
power_off:
og01a1b_power_off(og01a1b->dev);
diff --git a/drivers/media/i2c/ov5647.c b/drivers/media/i2c/ov5647.c
index 5fb10e02ba6e..36460440059a 100644
--- a/drivers/media/i2c/ov5647.c
+++ b/drivers/media/i2c/ov5647.c
@@ -1480,6 +1480,10 @@ static void ov5647_remove(struct i2c_client *client)
v4l2_ctrl_handler_free(&sensor->ctrls);
v4l2_device_unregister_subdev(sd);
pm_runtime_disable(&client->dev);
+ if (!pm_runtime_status_suspended(&client->dev)) {
+ ov5647_power_off(&client->dev);
+ pm_runtime_set_suspended(&client->dev);
+ }
mutex_destroy(&sensor->lock);
}
diff --git a/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c b/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c
index 62ed92ff1d30..9f5bd9c4ddf9 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c
@@ -32,7 +32,7 @@ static int handle_proxy_response(struct ipu6_isys *isys, unsigned int req_id)
struct ipu6_fw_isys_proxy_resp_info_abi *resp;
int ret;
- resp = ipu6_recv_get_token(isys->fwcom, IPU6_BASE_PROXY_RECV_QUEUES);
+ resp = ipu6_recv_get_token(isys->fwctx, IPU6_BASE_PROXY_RECV_QUEUES);
if (!resp)
return 1;
@@ -42,7 +42,7 @@ static int handle_proxy_response(struct ipu6_isys *isys, unsigned int req_id)
ret = req_id == resp->request_id ? 0 : -EIO;
- ipu6_recv_put_token(isys->fwcom, IPU6_BASE_PROXY_RECV_QUEUES);
+ ipu6_recv_put_token(isys->fwctx, IPU6_BASE_PROXY_RECV_QUEUES);
return ret;
}
@@ -52,7 +52,7 @@ int ipu6_fw_isys_send_proxy_token(struct ipu6_isys *isys,
unsigned int index,
unsigned int offset, u32 value)
{
- struct ipu6_fw_com_context *ctx = isys->fwcom;
+ struct ipu6_fw_com_context *ctx = isys->fwctx;
struct device *dev = &isys->adev->auxdev.dev;
struct ipu6_fw_proxy_send_queue_token *token;
unsigned int timeout = 1000;
@@ -96,7 +96,7 @@ int ipu6_fw_isys_complex_cmd(struct ipu6_isys *isys,
dma_addr_t dma_mapped_buf,
size_t size, u16 send_type)
{
- struct ipu6_fw_com_context *ctx = isys->fwcom;
+ struct ipu6_fw_com_context *ctx = isys->fwctx;
struct device *dev = &isys->adev->auxdev.dev;
struct ipu6_fw_send_queue_token *token;
@@ -138,7 +138,7 @@ int ipu6_fw_isys_close(struct ipu6_isys *isys)
struct device *dev = &isys->adev->auxdev.dev;
int retry = IPU6_ISYS_CLOSE_RETRY;
unsigned long flags;
- void *fwcom;
+ void *fwctx;
int ret;
/*
@@ -148,9 +148,9 @@ int ipu6_fw_isys_close(struct ipu6_isys *isys)
* spinlock to wait the interrupt handler to be finished
*/
spin_lock_irqsave(&isys->power_lock, flags);
- ret = ipu6_fw_com_close(isys->fwcom);
- fwcom = isys->fwcom;
- isys->fwcom = NULL;
+ ret = ipu6_fw_com_close(isys->fwctx);
+ fwctx = isys->fwctx;
+ isys->fwctx = NULL;
spin_unlock_irqrestore(&isys->power_lock, flags);
if (ret)
dev_err(dev, "Device close failure: %d\n", ret);
@@ -158,14 +158,14 @@ int ipu6_fw_isys_close(struct ipu6_isys *isys)
/* release probably fails if the close failed. Let's try still */
do {
usleep_range(400, 500);
- ret = ipu6_fw_com_release(fwcom, 0);
+ ret = ipu6_fw_com_release(fwctx, 0);
retry--;
} while (ret && retry);
if (ret) {
dev_err(dev, "Device release time out %d\n", ret);
spin_lock_irqsave(&isys->power_lock, flags);
- isys->fwcom = fwcom;
+ isys->fwctx = fwctx;
spin_unlock_irqrestore(&isys->power_lock, flags);
}
@@ -176,11 +176,11 @@ void ipu6_fw_isys_cleanup(struct ipu6_isys *isys)
{
int ret;
- ret = ipu6_fw_com_release(isys->fwcom, 1);
+ ret = ipu6_fw_com_release(isys->fwctx, 1);
if (ret < 0)
dev_warn(&isys->adev->auxdev.dev,
"Device busy, fw_com release failed.");
- isys->fwcom = NULL;
+ isys->fwctx = NULL;
}
static void start_sp(struct ipu6_bus_device *adev)
@@ -212,13 +212,12 @@ static int query_sp(struct ipu6_bus_device *adev)
}
static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
- struct ipu6_fw_com_cfg *fwcom,
+ struct ipu6_fw_com_cfg *fwcom_cfg,
unsigned int num_streams)
{
unsigned int max_send_queues, max_sram_blocks, max_devq_size;
struct ipu6_fw_syscom_queue_config *input_queue_cfg;
struct ipu6_fw_syscom_queue_config *output_queue_cfg;
- struct device *dev = &isys->adev->auxdev.dev;
int type_proxy = IPU6_FW_ISYS_QUEUE_TYPE_PROXY;
int type_dev = IPU6_FW_ISYS_QUEUE_TYPE_DEV;
int type_msg = IPU6_FW_ISYS_QUEUE_TYPE_MSG;
@@ -228,7 +227,6 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
struct ipu6_fw_isys_fw_config *isys_fw_cfg;
u32 num_in_message_queues;
unsigned int max_streams;
- unsigned int size;
unsigned int i;
max_streams = isys->pdata->ipdata->max_streams;
@@ -236,7 +234,7 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
max_sram_blocks = isys->pdata->ipdata->max_sram_blocks;
max_devq_size = isys->pdata->ipdata->max_devq_size;
num_in_message_queues = clamp(num_streams, 1U, max_streams);
- isys_fw_cfg = devm_kzalloc(dev, sizeof(*isys_fw_cfg), GFP_KERNEL);
+ isys_fw_cfg = kzalloc_obj(*isys_fw_cfg);
if (!isys_fw_cfg)
return -ENOMEM;
@@ -248,24 +246,25 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
isys_fw_cfg->num_recv_queues[type_dev] = 0;
isys_fw_cfg->num_recv_queues[type_msg] = 1;
- size = sizeof(*input_queue_cfg) * max_send_queues;
- input_queue_cfg = devm_kzalloc(dev, size, GFP_KERNEL);
+ input_queue_cfg = kzalloc_objs(*input_queue_cfg, max_send_queues);
if (!input_queue_cfg)
- return -ENOMEM;
+ goto err_free_fw_cfg;
- size = sizeof(*output_queue_cfg) * IPU6_N_MAX_RECV_QUEUES;
- output_queue_cfg = devm_kzalloc(dev, size, GFP_KERNEL);
+ output_queue_cfg = kzalloc_objs(*output_queue_cfg,
+ IPU6_N_MAX_RECV_QUEUES);
if (!output_queue_cfg)
- return -ENOMEM;
+ goto err_free_input_queue_cfg;
- fwcom->input = input_queue_cfg;
- fwcom->output = output_queue_cfg;
+ fwcom_cfg->input = input_queue_cfg;
+ fwcom_cfg->output = output_queue_cfg;
- fwcom->num_input_queues = isys_fw_cfg->num_send_queues[type_proxy] +
+ fwcom_cfg->num_input_queues =
+ isys_fw_cfg->num_send_queues[type_proxy] +
isys_fw_cfg->num_send_queues[type_dev] +
isys_fw_cfg->num_send_queues[type_msg];
- fwcom->num_output_queues = isys_fw_cfg->num_recv_queues[type_proxy] +
+ fwcom_cfg->num_output_queues =
+ isys_fw_cfg->num_recv_queues[type_proxy] +
isys_fw_cfg->num_recv_queues[type_dev] +
isys_fw_cfg->num_recv_queues[type_msg];
@@ -280,7 +279,7 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
isys_fw_cfg->buffer_partition.num_gda_pages[i] = 0;
}
- /* FW assumes proxy interface at fwcom queue 0 */
+ /* FW assumes proxy interface at fwcom_cfg queue 0 */
for (i = 0; i < isys_fw_cfg->num_send_queues[type_proxy]; i++) {
input_queue_cfg[i].token_size =
sizeof(struct ipu6_fw_proxy_send_queue_token);
@@ -314,34 +313,51 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
IPU6_ISYS_SIZE_RECV_QUEUE;
}
- fwcom->dmem_addr = isys->pdata->ipdata->hw_variant.dmem_offset;
- fwcom->specific_addr = isys_fw_cfg;
- fwcom->specific_size = sizeof(*isys_fw_cfg);
+ fwcom_cfg->dmem_addr = isys->pdata->ipdata->hw_variant.dmem_offset;
+ fwcom_cfg->specific_addr = isys_fw_cfg;
+ fwcom_cfg->specific_size = sizeof(*isys_fw_cfg);
return 0;
+
+err_free_input_queue_cfg:
+ kfree(input_queue_cfg);
+err_free_fw_cfg:
+ kfree(isys_fw_cfg);
+
+ return -ENOMEM;
+}
+
+static void ipu6_isys_fwcom_cfg_free(struct ipu6_fw_com_cfg *fwcom_cfg)
+{
+ kfree(fwcom_cfg->specific_addr);
+ kfree(fwcom_cfg->output);
+ kfree(fwcom_cfg->input);
}
int ipu6_fw_isys_init(struct ipu6_isys *isys, unsigned int num_streams)
{
struct device *dev = &isys->adev->auxdev.dev;
int retry = IPU6_ISYS_OPEN_RETRY;
- struct ipu6_fw_com_cfg fwcom = {
+ struct ipu6_fw_com_cfg fwcom_cfg = {
.cell_start = start_sp,
.cell_ready = query_sp,
.buttress_boot_offset = SYSCOM_BUTTRESS_FW_PARAMS_ISYS_OFFSET,
};
int ret;
- ipu6_isys_fwcom_cfg_init(isys, &fwcom, num_streams);
+ ret = ipu6_isys_fwcom_cfg_init(isys, &fwcom_cfg, num_streams);
+ if (ret)
+ return ret;
- isys->fwcom = ipu6_fw_com_prepare(&fwcom, isys->adev,
+ isys->fwctx = ipu6_fw_com_prepare(&fwcom_cfg, isys->adev,
isys->pdata->base);
- if (!isys->fwcom) {
+ ipu6_isys_fwcom_cfg_free(&fwcom_cfg);
+ if (!isys->fwctx) {
dev_err(dev, "isys fw com prepare failed\n");
return -EIO;
}
- ret = ipu6_fw_com_open(isys->fwcom);
+ ret = ipu6_fw_com_open(isys->fwctx);
if (ret) {
dev_err(dev, "isys fw com open failed %d\n", ret);
return ret;
@@ -349,7 +365,7 @@ int ipu6_fw_isys_init(struct ipu6_isys *isys, unsigned int num_streams)
do {
usleep_range(400, 500);
- if (ipu6_fw_com_ready(isys->fwcom))
+ if (ipu6_fw_com_ready(isys->fwctx))
break;
retry--;
} while (retry > 0);
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
index b96ae563b82f..d9b67502dbd2 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
@@ -1116,7 +1116,7 @@ int ipu6_isys_fw_open(struct ipu6_isys *isys)
*/
ipu6_cleanup_fw_msg_bufs(isys);
- if (isys->fwcom) {
+ if (isys->fwctx) {
/*
* Something went wrong in previous shutdown. As we are now
* restarting isys we can safely delete old context.
@@ -1149,7 +1149,7 @@ void ipu6_isys_fw_close(struct ipu6_isys *isys)
isys->ref_count--;
if (!isys->ref_count) {
ipu6_fw_isys_close(isys);
- if (isys->fwcom) {
+ if (isys->fwctx) {
isys->need_reset = true;
dev_warn(&isys->adev->auxdev.dev,
"failed to close fw isys\n");
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys.c b/drivers/media/pci/intel/ipu6/ipu6-isys.c
index 9a9e245edc1a..a0f209b8141d 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys.c
@@ -1227,10 +1227,10 @@ static int isys_isr_one(struct ipu6_bus_device *adev)
u32 index;
u64 ts;
- if (!isys->fwcom)
+ if (!isys->fwctx)
return 1;
- resp = ipu6_fw_isys_get_resp(isys->fwcom, IPU6_BASE_MSG_RECV_QUEUES);
+ resp = ipu6_fw_isys_get_resp(isys->fwctx, IPU6_BASE_MSG_RECV_QUEUES);
if (!resp)
return 1;
@@ -1339,7 +1339,7 @@ static int isys_isr_one(struct ipu6_bus_device *adev)
ipu6_isys_put_stream(stream);
leave:
- ipu6_fw_isys_put_resp(isys->fwcom, IPU6_BASE_MSG_RECV_QUEUES);
+ ipu6_fw_isys_put_resp(isys->fwctx, IPU6_BASE_MSG_RECV_QUEUES);
return 0;
}
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys.h b/drivers/media/pci/intel/ipu6/ipu6-isys.h
index 0e2c8b71edfc..ed38decdd525 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys.h
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys.h
@@ -114,8 +114,7 @@ struct sensor_async_sd {
* @csi2_rx_ctrl_cached: cached shared value between all CSI2 receivers
* @streams_lock: serialise access to streams
* @streams: streams per firmware stream ID
- * @fwcom: fw communication layer private pointer
- * or optional external library private pointer
+ * @fwctx: fw communication layer context pointer
* @phy_termcal_val: the termination calibration value, only used for DWC PHY
* @need_reset: Isys requires d0i0->i3 transition
* @ref_count: total number of callers fw open
@@ -136,7 +135,7 @@ struct ipu6_isys {
spinlock_t streams_lock;
struct ipu6_isys_stream streams[IPU6_ISYS_MAX_STREAMS];
int streams_ref_count[IPU6_ISYS_MAX_STREAMS];
- void *fwcom;
+ void *fwctx;
u32 phy_termcal_val;
bool need_reset;
bool icache_prefetch;
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 8e52b40e2423..f4eebd757549 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1581,7 +1581,7 @@ static irqreturn_t mtk_jpeg_enc_done(struct mtk_jpeg_dev *jpeg)
v4l2_m2m_buf_done(src_buf, buf_state);
v4l2_m2m_buf_done(dst_buf, buf_state);
v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
- pm_runtime_put(ctx->jpeg->dev);
+ pm_runtime_put(jpeg->dev);
return IRQ_HANDLED;
}
@@ -1869,7 +1869,7 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
v4l2_m2m_buf_done(src_buf, buf_state);
v4l2_m2m_buf_done(dst_buf, buf_state);
v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
- pm_runtime_put(ctx->jpeg->dev);
+ pm_runtime_put(jpeg->dev);
return IRQ_HANDLED;
}
diff --git a/drivers/media/platform/microchip/microchip-isc-base.c b/drivers/media/platform/microchip/microchip-isc-base.c
index 21d108eb93c7..963d5fd53b58 100644
--- a/drivers/media/platform/microchip/microchip-isc-base.c
+++ b/drivers/media/platform/microchip/microchip-isc-base.c
@@ -8,6 +8,7 @@
* Author: Eugen Hristev <eugen.hristev@microchip.com>
*
*/
+#include <linux/bitfield.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/math64.h>
@@ -62,17 +63,17 @@ static inline void isc_update_awb_ctrls(struct isc_device *isc)
/* In here we set our actual hw pipeline config */
regmap_write(isc->regmap, ISC_WB_O_RGR,
- ((ctrls->offset[ISC_HIS_CFG_MODE_R])) |
- ((ctrls->offset[ISC_HIS_CFG_MODE_GR]) << 16));
+ FIELD_PREP(ISC_WB_O_LO, ctrls->offset[ISC_HIS_CFG_MODE_R]) |
+ FIELD_PREP(ISC_WB_O_HI, ctrls->offset[ISC_HIS_CFG_MODE_GR]));
regmap_write(isc->regmap, ISC_WB_O_BGB,
- ((ctrls->offset[ISC_HIS_CFG_MODE_B])) |
- ((ctrls->offset[ISC_HIS_CFG_MODE_GB]) << 16));
+ FIELD_PREP(ISC_WB_O_LO, ctrls->offset[ISC_HIS_CFG_MODE_B]) |
+ FIELD_PREP(ISC_WB_O_HI, ctrls->offset[ISC_HIS_CFG_MODE_GB]));
regmap_write(isc->regmap, ISC_WB_G_RGR,
- ctrls->gain[ISC_HIS_CFG_MODE_R] |
- (ctrls->gain[ISC_HIS_CFG_MODE_GR] << 16));
+ FIELD_PREP(ISC_WB_G_LO, ctrls->gain[ISC_HIS_CFG_MODE_R]) |
+ FIELD_PREP(ISC_WB_G_HI, ctrls->gain[ISC_HIS_CFG_MODE_GR]));
regmap_write(isc->regmap, ISC_WB_G_BGB,
- ctrls->gain[ISC_HIS_CFG_MODE_B] |
- (ctrls->gain[ISC_HIS_CFG_MODE_GB] << 16));
+ FIELD_PREP(ISC_WB_G_LO, ctrls->gain[ISC_HIS_CFG_MODE_B]) |
+ FIELD_PREP(ISC_WB_G_HI, ctrls->gain[ISC_HIS_CFG_MODE_GB]));
}
static inline void isc_reset_awb_ctrls(struct isc_device *isc)
@@ -291,7 +292,8 @@ static int isc_configure(struct isc_device *isc)
struct isc_subdev_entity *subdev = isc->current_subdev;
int ret;
- pfe_cfg0 = isc->config.sd_format->pfe_cfg0_bps;
+ pfe_cfg0 = FIELD_PREP(ISC_PFE_CFG0_BPS_MASK,
+ isc->config.sd_format->pfe_cfg0_bps);
pipeline = isc->config.bits_pipeline;
dcfg = isc->config.dcfg_imode | isc->dcfg;
diff --git a/drivers/media/platform/microchip/microchip-isc-regs.h b/drivers/media/platform/microchip/microchip-isc-regs.h
index e77e1d9a1db8..fe145b142b82 100644
--- a/drivers/media/platform/microchip/microchip-isc-regs.h
+++ b/drivers/media/platform/microchip/microchip-isc-regs.h
@@ -31,11 +31,11 @@
#define ISC_PFE_CFG0_MODE_PROGRESSIVE (0x0 << 4)
#define ISC_PFE_CFG0_MODE_MASK GENMASK(6, 4)
-#define ISC_PFE_CFG0_BPS_EIGHT (0x4 << 28)
-#define ISC_PFG_CFG0_BPS_NINE (0x3 << 28)
-#define ISC_PFG_CFG0_BPS_TEN (0x2 << 28)
-#define ISC_PFG_CFG0_BPS_ELEVEN (0x1 << 28)
-#define ISC_PFG_CFG0_BPS_TWELVE (0x0 << 28)
+#define ISC_PFE_CFG0_BPS_EIGHT 0x4
+#define ISC_PFE_CFG0_BPS_NINE 0x3
+#define ISC_PFE_CFG0_BPS_TEN 0x2
+#define ISC_PFE_CFG0_BPS_ELEVEN 0x1
+#define ISC_PFE_CFG0_BPS_TWELVE 0x0
#define ISC_PFE_CFG0_BPS_MASK GENMASK(30, 28)
#define ISC_PFE_CFG0_COLEN BIT(12)
@@ -149,6 +149,12 @@
/* ISC White Balance Gain for B, GB Register */
#define ISC_WB_G_BGB 0x0000006c
+/* Each WB offset/gain register packs two 13-bit fields, low and high */
+#define ISC_WB_O_LO GENMASK(12, 0) /* R or B offset [12:0] */
+#define ISC_WB_O_HI GENMASK(28, 16) /* GR or GB offset [28:16] */
+#define ISC_WB_G_LO GENMASK(12, 0) /* R or B gain [12:0] */
+#define ISC_WB_G_HI GENMASK(28, 16) /* GR or GB gain [28:16] */
+
/* ISC Color Filter Array Control Register */
#define ISC_CFA_CTRL 0x00000070
diff --git a/drivers/media/platform/microchip/microchip-isc.h b/drivers/media/platform/microchip/microchip-isc.h
index f5e322c2e36b..d7bcd74efff9 100644
--- a/drivers/media/platform/microchip/microchip-isc.h
+++ b/drivers/media/platform/microchip/microchip-isc.h
@@ -62,7 +62,9 @@ struct isc_subdev_entity {
* @mbus_code: V4L2 media bus format code.
* @cfa_baycfg: If this format is RAW BAYER, indicate the type of bayer.
this is either BGBG, RGRG, etc.
- * @pfe_cfg0_bps: Number of hardware data lines connected to the ISC
+ * @pfe_cfg0_bps: ISC_PFE_CFG0 BPS field value (e.g. ISC_PFE_CFG0_BPS_EIGHT),
+ written into PFE_CFG0 with FIELD_PREP(ISC_PFE_CFG0_BPS_MASK)
+ at configure time.
* @raw: If the format is raw bayer.
*/
diff --git a/drivers/media/platform/microchip/microchip-sama5d2-isc.c b/drivers/media/platform/microchip/microchip-sama5d2-isc.c
index 5bd109e93a8f..5e41eee45dbd 100644
--- a/drivers/media/platform/microchip/microchip-sama5d2-isc.c
+++ b/drivers/media/platform/microchip/microchip-sama5d2-isc.c
@@ -147,49 +147,49 @@ static struct isc_format sama5d2_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_SBGGR10,
.mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
- .cfa_baycfg = ISC_BAY_CFG_RGRG,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
+ .cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG10,
.mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG10,
.mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB10,
.mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
.fourcc = V4L2_PIX_FMT_SBGGR12,
.mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG12,
.mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG12,
.mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB12,
.mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
@@ -210,7 +210,7 @@ static struct isc_format sama5d2_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_Y10,
.mbus_code = MEDIA_BUS_FMT_Y10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
},
};
diff --git a/drivers/media/platform/microchip/microchip-sama7g5-isc.c b/drivers/media/platform/microchip/microchip-sama7g5-isc.c
index d978ce448a4d..9e78ea07b013 100644
--- a/drivers/media/platform/microchip/microchip-sama7g5-isc.c
+++ b/drivers/media/platform/microchip/microchip-sama7g5-isc.c
@@ -156,49 +156,49 @@ static struct isc_format sama7g5_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_SBGGR10,
.mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
- .cfa_baycfg = ISC_BAY_CFG_RGRG,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
+ .cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG10,
.mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG10,
.mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB10,
.mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
.fourcc = V4L2_PIX_FMT_SBGGR12,
.mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG12,
.mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG12,
.mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB12,
.mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
@@ -224,7 +224,7 @@ static struct isc_format sama7g5_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_Y10,
.mbus_code = MEDIA_BUS_FMT_Y10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
},
};
diff --git a/drivers/media/platform/qcom/iris/iris_core.h b/drivers/media/platform/qcom/iris/iris_core.h
index fb194c967ad4..530fe14c3a47 100644
--- a/drivers/media/platform/qcom/iris/iris_core.h
+++ b/drivers/media/platform/qcom/iris/iris_core.h
@@ -7,6 +7,7 @@
#define __IRIS_CORE_H__
#include <linux/types.h>
+#include <linux/dma-mapping.h>
#include <linux/pm_domain.h>
#include <media/v4l2-device.h>
@@ -25,6 +26,9 @@ struct icc_info {
#define IRIS_FW_VERSION_LENGTH 128
#define IFACEQ_CORE_PKT_SIZE (1024 * 4)
+#define IRIS_NP_RESERVE_IOVA_START 0x0
+#define IRIS_NP_RESERVE_IOVA_SIZE 0x25800000
+
enum domain_type {
ENCODER = BIT(0),
DECODER = BIT(1),
@@ -73,6 +77,7 @@ enum domain_type {
* @instances: a list_head of all instances
* @inst_fw_caps_dec: an array of supported instance capabilities by decoder
* @inst_fw_caps_enc: an array of supported instance capabilities by encoder
+ * @iova_state: a pointer to an array of dma_iova_state entries reserved for restricted IOVA region
*/
struct iris_core {
@@ -117,6 +122,7 @@ struct iris_core {
/* encoder and decoder have overlapping caps, so two different arrays are required */
struct platform_inst_fw_cap inst_fw_caps_dec[INST_FW_CAP_MAX];
struct platform_inst_fw_cap inst_fw_caps_enc[INST_FW_CAP_MAX];
+ struct dma_iova_state *iova_state;
};
int iris_core_init(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/iris/iris_probe.c b/drivers/media/platform/qcom/iris/iris_probe.c
index cdeb78ba605a..b6251493bd9c 100644
--- a/drivers/media/platform/qcom/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/iris/iris_probe.c
@@ -153,6 +153,57 @@ static int iris_init_resources(struct iris_core *core)
return iris_init_resets(core);
}
+static void iris_unreserve_iova_region(struct device *dev, struct dma_iova_state *iova_state)
+{
+ unsigned int i;
+
+ for (i = 0; dma_iova_size(&iova_state[i]); i++)
+ dma_iova_free(dev, &iova_state[i]);
+}
+
+static int iris_reserve_iova_region(struct device *dev, struct dma_iova_state **iova_state,
+ unsigned long start, unsigned long size)
+{
+ unsigned long dma_limit = dev->bus_dma_limit;
+ unsigned long end, rem, chunk;
+ struct dma_iova_state *state;
+ unsigned int count = 0;
+ int ret = -ENOMEM;
+
+ state = devm_kcalloc(dev, BITS_PER_TYPE(dma_addr_t) + 1, sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return ret;
+
+ end = start + size;
+ rem = end - max(start, PAGE_SIZE);
+ dev->bus_dma_limit = end - 1;
+
+ while (rem) {
+ chunk = min(end & -end, (u64)1 << (fls64(rem) - 1));
+
+ if (!dma_iova_try_alloc(dev, &state[count], 0, chunk))
+ goto err_free_iova;
+
+ if (state[count].addr != end - chunk || state[count].__size != chunk)
+ goto err_free_iova;
+
+ rem -= chunk;
+ end -= chunk;
+ count++;
+ }
+
+ *iova_state = state;
+ dev->bus_dma_limit = dma_limit;
+
+ return 0;
+
+err_free_iova:
+ iris_unreserve_iova_region(dev, state);
+ dev->bus_dma_limit = dma_limit;
+
+ return ret;
+}
+
static int iris_register_video_device(struct iris_core *core, enum domain_type type)
{
struct video_device *vdev;
@@ -210,6 +261,8 @@ static void iris_remove(struct platform_device *pdev)
v4l2_device_unregister(&core->v4l2_dev);
+ iris_unreserve_iova_region(core->dev, core->iova_state);
+
mutex_destroy(&core->lock);
}
@@ -272,10 +325,15 @@ static int iris_probe(struct platform_device *pdev)
iris_session_init_caps(core);
- ret = v4l2_device_register(dev, &core->v4l2_dev);
+ ret = iris_reserve_iova_region(dev, &core->iova_state, IRIS_NP_RESERVE_IOVA_START,
+ IRIS_NP_RESERVE_IOVA_SIZE);
if (ret)
return ret;
+ ret = v4l2_device_register(dev, &core->v4l2_dev);
+ if (ret)
+ goto err_unresv_iova_region;
+
ret = iris_register_video_device(core, DECODER);
if (ret)
goto err_v4l2_unreg;
@@ -309,6 +367,8 @@ static int iris_probe(struct platform_device *pdev)
video_unregister_device(core->vdev_dec);
err_v4l2_unreg:
v4l2_device_unregister(&core->v4l2_dev);
+err_unresv_iova_region:
+ iris_unreserve_iova_region(dev, core->iova_state);
return ret;
}
diff --git a/drivers/media/platform/qcom/iris/iris_vpu_common.c b/drivers/media/platform/qcom/iris/iris_vpu_common.c
index 6839e293d9b5..a20565abef6a 100644
--- a/drivers/media/platform/qcom/iris/iris_vpu_common.c
+++ b/drivers/media/platform/qcom/iris/iris_vpu_common.c
@@ -260,8 +260,8 @@ int iris_vpu_power_off_controller(struct iris_core *core)
void iris_vpu_power_off_hw(struct iris_core *core)
{
dev_pm_genpd_set_hwmode(core->pmdomain_tbl->pd_devs[IRIS_HW_POWER_DOMAIN], false);
- iris_disable_power_domains(core, core->pmdomain_tbl->pd_devs[IRIS_HW_POWER_DOMAIN]);
iris_disable_unprepare_clock(core, IRIS_HW_CLK);
+ iris_disable_power_domains(core, core->pmdomain_tbl->pd_devs[IRIS_HW_POWER_DOMAIN]);
}
void iris_vpu_power_off(struct iris_core *core)
diff --git a/drivers/media/platform/rockchip/rga/rga.c b/drivers/media/platform/rockchip/rga/rga.c
index 776046de979a..f956f2fb222a 100644
--- a/drivers/media/platform/rockchip/rga/rga.c
+++ b/drivers/media/platform/rockchip/rga/rga.c
@@ -702,48 +702,10 @@ static const struct video_device rga_videodev = {
.device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING,
};
-static int rga_enable_clocks(struct rockchip_rga *rga)
-{
- int ret;
-
- ret = clk_prepare_enable(rga->sclk);
- if (ret) {
- dev_err(rga->dev, "Cannot enable rga sclk: %d\n", ret);
- return ret;
- }
-
- ret = clk_prepare_enable(rga->aclk);
- if (ret) {
- dev_err(rga->dev, "Cannot enable rga aclk: %d\n", ret);
- goto err_disable_sclk;
- }
-
- ret = clk_prepare_enable(rga->hclk);
- if (ret) {
- dev_err(rga->dev, "Cannot enable rga hclk: %d\n", ret);
- goto err_disable_aclk;
- }
-
- return 0;
-
-err_disable_aclk:
- clk_disable_unprepare(rga->aclk);
-err_disable_sclk:
- clk_disable_unprepare(rga->sclk);
-
- return ret;
-}
-
-static void rga_disable_clocks(struct rockchip_rga *rga)
-{
- clk_disable_unprepare(rga->sclk);
- clk_disable_unprepare(rga->hclk);
- clk_disable_unprepare(rga->aclk);
-}
-
static int rga_parse_dt(struct rockchip_rga *rga)
{
struct reset_control *core_rst, *axi_rst, *ahb_rst;
+ int ret;
core_rst = devm_reset_control_get(rga->dev, "core");
if (IS_ERR(core_rst)) {
@@ -775,23 +737,12 @@ static int rga_parse_dt(struct rockchip_rga *rga)
udelay(1);
reset_control_deassert(ahb_rst);
- rga->sclk = devm_clk_get(rga->dev, "sclk");
- if (IS_ERR(rga->sclk)) {
- dev_err(rga->dev, "failed to get sclk clock\n");
- return PTR_ERR(rga->sclk);
- }
-
- rga->aclk = devm_clk_get(rga->dev, "aclk");
- if (IS_ERR(rga->aclk)) {
- dev_err(rga->dev, "failed to get aclk clock\n");
- return PTR_ERR(rga->aclk);
- }
-
- rga->hclk = devm_clk_get(rga->dev, "hclk");
- if (IS_ERR(rga->hclk)) {
- dev_err(rga->dev, "failed to get hclk clock\n");
- return PTR_ERR(rga->hclk);
+ ret = devm_clk_bulk_get_all(rga->dev, &rga->clks);
+ if (ret < 0) {
+ dev_err(rga->dev, "failed to get clocks\n");
+ return ret;
}
+ rga->num_clks = ret;
return 0;
}
@@ -831,6 +782,7 @@ static int rga_probe(struct platform_device *pdev)
ret = irq;
goto err_put_clk;
}
+ rga->irq = irq;
ret = devm_request_irq(rga->dev, irq, rga_isr, 0,
dev_name(rga->dev), rga);
@@ -928,8 +880,9 @@ static void rga_remove(struct platform_device *pdev)
v4l2_info(&rga->v4l2_dev, "Removing\n");
- v4l2_m2m_release(rga->m2m_dev);
video_unregister_device(rga->vfd);
+ devm_free_irq(rga->dev, rga->irq, rga);
+ v4l2_m2m_release(rga->m2m_dev);
v4l2_device_unregister(&rga->v4l2_dev);
pm_runtime_disable(rga->dev);
@@ -939,7 +892,7 @@ static int __maybe_unused rga_runtime_suspend(struct device *dev)
{
struct rockchip_rga *rga = dev_get_drvdata(dev);
- rga_disable_clocks(rga);
+ clk_bulk_disable_unprepare(rga->num_clks, rga->clks);
return 0;
}
@@ -948,7 +901,7 @@ static int __maybe_unused rga_runtime_resume(struct device *dev)
{
struct rockchip_rga *rga = dev_get_drvdata(dev);
- return rga_enable_clocks(rga);
+ return clk_bulk_prepare_enable(rga->num_clks, rga->clks);
}
static const struct dev_pm_ops rga_pm = {
diff --git a/drivers/media/platform/rockchip/rga/rga.h b/drivers/media/platform/rockchip/rga/rga.h
index 72a28b120fab..0a15ec9aa8aa 100644
--- a/drivers/media/platform/rockchip/rga/rga.h
+++ b/drivers/media/platform/rockchip/rga/rga.h
@@ -6,6 +6,7 @@
#ifndef __RGA_H__
#define __RGA_H__
+#include <linux/clk.h>
#include <linux/platform_device.h>
#include <media/videobuf2-v4l2.h>
#include <media/v4l2-ctrls.h>
@@ -81,9 +82,9 @@ struct rockchip_rga {
struct device *dev;
struct regmap *grf;
void __iomem *regs;
- struct clk *sclk;
- struct clk *aclk;
- struct clk *hclk;
+ int irq;
+ struct clk_bulk_data *clks;
+ int num_clks;
struct rockchip_rga_version version;
/* vfd lock */
diff --git a/drivers/media/rc/redrat3.c b/drivers/media/rc/redrat3.c
index c9a5c030f772..080207aa888c 100644
--- a/drivers/media/rc/redrat3.c
+++ b/drivers/media/rc/redrat3.c
@@ -773,17 +773,9 @@ static int redrat3_transmit_ir(struct rc_dev *rcdev, unsigned *txbuf,
u8 curlencheck = 0;
unsigned i, sendbuf_len;
- if (rr3->transmitting) {
- dev_warn(dev, "%s: transmitter already in use\n", __func__);
- return -EAGAIN;
- }
-
if (count > RR3_MAX_SIG_SIZE - RR3_TX_TRAILER_LEN)
return -EINVAL;
- /* rr3 will disable rc detector on transmit */
- rr3->transmitting = true;
-
sample_lens = kcalloc(RR3_DRIVER_MAXLENS,
sizeof(*sample_lens),
GFP_KERNEL);
@@ -796,6 +788,9 @@ static int redrat3_transmit_ir(struct rc_dev *rcdev, unsigned *txbuf,
goto out;
}
+ /* rr3 will disable rc detector on transmit */
+ rr3->transmitting = true;
+
for (i = 0; i < count; i++) {
cur_sample_len = redrat3_us_to_len(txbuf[i]);
if (cur_sample_len > 0xffff) {
diff --git a/drivers/media/usb/dvb-usb/dvb-usb-firmware.c b/drivers/media/usb/dvb-usb/dvb-usb-firmware.c
index 0fb3fa6100e4..675d9d1d4f47 100644
--- a/drivers/media/usb/dvb-usb/dvb-usb-firmware.c
+++ b/drivers/media/usb/dvb-usb/dvb-usb-firmware.c
@@ -141,6 +141,8 @@ int dvb_usb_get_hexline(const struct firmware *fw, struct hexline *hx,
if (hx->type == 0x04) {
/* b[4] and b[5] are the Extended linear address record data field */
+ if (hx->len != 2)
+ return -EINVAL;
hx->addr |= (b[4] << 24) | (b[5] << 16);
/* hx->len -= 2;
data_offs += 2; */
diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index aa3e8d295e0f..511912074ba0 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -2847,6 +2847,26 @@ static const struct usb_device_id uvc_ids[] = {
.bInterfaceSubClass = 1,
.bInterfaceProtocol = 0,
.driver_info = (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
+ /* Avermedia GC515 */
+ { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
+ | USB_DEVICE_ID_MATCH_INT_INFO,
+ .idVendor = 0x07ca,
+ .idProduct = 0x1515,
+ .bInterfaceClass = USB_CLASS_VIDEO,
+ .bInterfaceSubClass = 1,
+ .bInterfaceProtocol = UVC_PC_PROTOCOL_15,
+ .driver_info = (kernel_ulong_t)&(const struct uvc_device_info){
+ .uvc_version = 0x0110,
+ } },
+ /* AVerMedia Live Gamer HD 2 capture card */
+ { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
+ | USB_DEVICE_ID_MATCH_INT_INFO,
+ .idVendor = 0x07ca,
+ .idProduct = 0x1570,
+ .bInterfaceClass = USB_CLASS_VIDEO,
+ .bInterfaceSubClass = 1,
+ .bInterfaceProtocol = 0,
+ .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_DISABLE_AUTOSUSPEND) },
/* ViMicro Vega */
{ .match_flags = USB_DEVICE_ID_MATCH_DEVICE
| USB_DEVICE_ID_MATCH_INT_INFO,
diff --git a/drivers/mfd/ab8500-core.c b/drivers/mfd/ab8500-core.c
index 86fa99022cb3..44fc2ce83cdd 100644
--- a/drivers/mfd/ab8500-core.c
+++ b/drivers/mfd/ab8500-core.c
@@ -1194,39 +1194,41 @@ static int ab8500_probe(struct platform_device *pdev)
return ret;
if (is_ab9540(ab8500))
- ret = mfd_add_devices(ab8500->dev, 0, ab9540_devs,
- ARRAY_SIZE(ab9540_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab9540_devs,
+ ARRAY_SIZE(ab9540_devs), NULL,
+ 0, ab8500->domain);
else if (is_ab8540(ab8500)) {
- ret = mfd_add_devices(ab8500->dev, 0, ab8540_devs,
- ARRAY_SIZE(ab8540_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab8540_devs,
+ ARRAY_SIZE(ab8540_devs), NULL,
+ 0, ab8500->domain);
if (ret)
return ret;
if (is_ab8540_1p2_or_earlier(ab8500))
- ret = mfd_add_devices(ab8500->dev, 0, ab8540_cut1_devs,
- ARRAY_SIZE(ab8540_cut1_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0,
+ ab8540_cut1_devs,
+ ARRAY_SIZE(ab8540_cut1_devs),
+ NULL, 0, ab8500->domain);
else /* ab8540 >= cut2 */
- ret = mfd_add_devices(ab8500->dev, 0, ab8540_cut2_devs,
- ARRAY_SIZE(ab8540_cut2_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0,
+ ab8540_cut2_devs,
+ ARRAY_SIZE(ab8540_cut2_devs),
+ NULL, 0, ab8500->domain);
} else if (is_ab8505(ab8500))
- ret = mfd_add_devices(ab8500->dev, 0, ab8505_devs,
- ARRAY_SIZE(ab8505_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab8505_devs,
+ ARRAY_SIZE(ab8505_devs), NULL,
+ 0, ab8500->domain);
else
- ret = mfd_add_devices(ab8500->dev, 0, ab8500_devs,
- ARRAY_SIZE(ab8500_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab8500_devs,
+ ARRAY_SIZE(ab8500_devs), NULL,
+ 0, ab8500->domain);
if (ret)
return ret;
/* Add battery management devices */
- ret = mfd_add_devices(ab8500->dev, 0, ab8500_bm_devs,
- ARRAY_SIZE(ab8500_bm_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab8500_bm_devs,
+ ARRAY_SIZE(ab8500_bm_devs), NULL,
+ 0, ab8500->domain);
if (ret)
dev_err(ab8500->dev, "error adding bm devices\n");
diff --git a/drivers/mfd/twl-core.c b/drivers/mfd/twl-core.c
index f89eda4a17fe..128cd3f6c60c 100644
--- a/drivers/mfd/twl-core.c
+++ b/drivers/mfd/twl-core.c
@@ -719,6 +719,11 @@ static const struct mfd_cell twl6032_cells[] = {
{ .name = "twl6032-clk" },
};
+static void twl_unregister_device(void *data)
+{
+ platform_device_unregister(data);
+}
+
/* NOTE: This driver only handles a single twl4030/tps659x0 chip */
static int
twl_probe(struct i2c_client *client)
@@ -754,17 +759,22 @@ twl_probe(struct i2c_client *client)
return status;
}
- if (i2c_check_functionality(client->adapter, I2C_FUNC_I2C) == 0) {
+ status = devm_add_action_or_reset(&client->dev, twl_unregister_device,
+ pdev);
+ if (status)
+ return status;
+
+ if (!i2c_check_functionality(client->adapter, I2C_FUNC_I2C)) {
dev_dbg(&client->dev, "can't talk I2C?\n");
status = -EIO;
- goto free;
+ goto out;
}
twl_priv = devm_kzalloc(&client->dev, sizeof(struct twl_private),
GFP_KERNEL);
if (!twl_priv) {
status = -ENOMEM;
- goto free;
+ goto out;
}
if ((id->driver_data) & TWL6030_CLASS) {
@@ -784,7 +794,7 @@ twl_probe(struct i2c_client *client)
GFP_KERNEL);
if (!twl_priv->twl_modules) {
status = -ENOMEM;
- goto free;
+ goto out;
}
for (i = 0; i < num_slaves; i++) {
@@ -880,7 +890,7 @@ twl_probe(struct i2c_client *client)
status = devm_mfd_add_devices(&client->dev, PLATFORM_DEVID_NONE,
cells, num_cells, NULL, 0, NULL);
if (status < 0)
- goto free;
+ goto out;
if (of_device_is_system_power_controller(node)) {
if (!pm_power_off)
@@ -896,10 +906,7 @@ twl_probe(struct i2c_client *client)
fail:
if (status < 0)
twl_remove(client);
-free:
- if (status < 0)
- platform_device_unregister(pdev);
-
+out:
return status;
}
diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c
index 29f9003999a0..9d596f4fd4dc 100644
--- a/drivers/misc/fastrpc.c
+++ b/drivers/misc/fastrpc.c
@@ -106,7 +106,7 @@
#define miscdev_to_fdevice(d) container_of(d, struct fastrpc_device, miscdev)
struct fastrpc_phy_page {
- u64 addr; /* physical address */
+ dma_addr_t addr; /* dma address */
u64 size; /* size of contiguous region */
};
@@ -171,7 +171,7 @@ struct fastrpc_msg {
u64 ctx; /* invoke caller context */
u32 handle; /* handle to invoke */
u32 sc; /* scalars structure describing the data */
- u64 addr; /* physical address */
+ dma_addr_t addr; /* dma address */
u64 size; /* size of contiguous region */
};
@@ -194,7 +194,7 @@ struct fastrpc_buf {
struct dma_buf *dmabuf;
struct device *dev;
void *virt;
- u64 phys;
+ dma_addr_t dma_addr;
u64 size;
/* Lock for dma buf attachments */
struct mutex lock;
@@ -217,7 +217,7 @@ struct fastrpc_map {
struct dma_buf *buf;
struct sg_table *table;
struct dma_buf_attachment *attach;
- u64 phys;
+ dma_addr_t dma_addr;
u64 size;
void *va;
u64 len;
@@ -322,11 +322,12 @@ static void fastrpc_free_map(struct kref *ref)
perm.vmid = QCOM_SCM_VMID_HLOS;
perm.perm = QCOM_SCM_PERM_RWX;
- err = qcom_scm_assign_mem(map->phys, map->len,
+ err = qcom_scm_assign_mem(map->dma_addr, map->len,
&src_perms, &perm, 1);
if (err) {
- dev_err(map->fl->sctx->dev, "Failed to assign memory phys 0x%llx size 0x%llx err %d\n",
- map->phys, map->len, err);
+ dev_err(map->fl->sctx->dev,
+ "Failed to assign memory dma_addr %pad size 0x%llx err %d\n",
+ &map->dma_addr, map->len, err);
return;
}
}
@@ -397,7 +398,7 @@ static int fastrpc_map_lookup(struct fastrpc_user *fl, int fd,
static void fastrpc_buf_free(struct fastrpc_buf *buf)
{
dma_free_coherent(buf->dev, buf->size, buf->virt,
- FASTRPC_PHYS(buf->phys));
+ FASTRPC_PHYS(buf->dma_addr));
kfree(buf);
}
@@ -416,12 +417,12 @@ static int __fastrpc_buf_alloc(struct fastrpc_user *fl, struct device *dev,
buf->fl = fl;
buf->virt = NULL;
- buf->phys = 0;
+ buf->dma_addr = 0;
buf->size = size;
buf->dev = dev;
buf->raddr = 0;
- buf->virt = dma_alloc_coherent(dev, buf->size, (dma_addr_t *)&buf->phys,
+ buf->virt = dma_alloc_coherent(dev, buf->size, &buf->dma_addr,
GFP_KERNEL);
if (!buf->virt) {
mutex_destroy(&buf->lock);
@@ -447,7 +448,7 @@ static int fastrpc_buf_alloc(struct fastrpc_user *fl, struct device *dev,
buf = *obuf;
if (fl->sctx && fl->sctx->sid)
- buf->phys += ((u64)fl->sctx->sid << 32);
+ buf->dma_addr += ((u64)fl->sctx->sid << 32);
return 0;
}
@@ -740,7 +741,7 @@ static int fastrpc_dma_buf_attach(struct dma_buf *dmabuf,
return -ENOMEM;
ret = dma_get_sgtable(buffer->dev, &a->sgt, buffer->virt,
- FASTRPC_PHYS(buffer->phys), buffer->size);
+ FASTRPC_PHYS(buffer->dma_addr), buffer->size);
if (ret < 0) {
dev_err(buffer->dev, "failed to get scatterlist from DMA API\n");
kfree(a);
@@ -789,7 +790,7 @@ static int fastrpc_mmap(struct dma_buf *dmabuf,
dma_resv_assert_held(dmabuf->resv);
return dma_mmap_coherent(buf->dev, vma, buf->virt,
- FASTRPC_PHYS(buf->phys), size);
+ FASTRPC_PHYS(buf->dma_addr), size);
}
static const struct dma_buf_ops fastrpc_dma_buf_ops = {
@@ -841,10 +842,10 @@ static int fastrpc_map_attach(struct fastrpc_user *fl, int fd,
map->table = table;
if (attr & FASTRPC_ATTR_SECUREMAP) {
- map->phys = sg_phys(map->table->sgl);
+ map->dma_addr = sg_phys(map->table->sgl);
} else {
- map->phys = sg_dma_address(map->table->sgl);
- map->phys += ((u64)fl->sctx->sid << 32);
+ map->dma_addr = sg_dma_address(map->table->sgl);
+ map->dma_addr += ((u64)fl->sctx->sid << 32);
}
for_each_sg(map->table->sgl, sgl, map->table->nents,
sgl_index)
@@ -853,7 +854,7 @@ static int fastrpc_map_attach(struct fastrpc_user *fl, int fd,
dev_dbg(sess->dev, "Bad size passed len 0x%llx map size 0x%llx\n",
len, map->size);
err = -EINVAL;
- goto map_err;
+ goto get_err;
}
map->va = sg_virt(map->table->sgl);
map->len = len;
@@ -871,11 +872,12 @@ static int fastrpc_map_attach(struct fastrpc_user *fl, int fd,
dst_perms[1].vmid = fl->cctx->vmperms[0].vmid;
dst_perms[1].perm = QCOM_SCM_PERM_RWX;
map->attr = attr;
- err = qcom_scm_assign_mem(map->phys, (u64)map->len, &src_perms, dst_perms, 2);
+ err = qcom_scm_assign_mem(map->dma_addr, (u64)map->len, &src_perms, dst_perms, 2);
if (err) {
- dev_err(sess->dev, "Failed to assign memory with phys 0x%llx size 0x%llx err %d\n",
- map->phys, map->len, err);
- goto map_err;
+ dev_err(sess->dev,
+ "Failed to assign memory with dma_addr %pad size 0x%llx err %d\n",
+ &map->dma_addr, map->len, err);
+ goto get_err;
}
}
spin_lock(&fl->lock);
@@ -1056,7 +1058,7 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx)
struct vm_area_struct *vma = NULL;
rpra[i].buf.pv = (u64) ctx->args[i].ptr;
- pages[i].addr = ctx->maps[i]->phys;
+ pages[i].addr = ctx->maps[i]->dma_addr;
mmap_read_lock(current->mm);
vma = vma_lookup(current->mm, ctx->args[i].ptr);
@@ -1083,7 +1085,7 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx)
goto bail;
rpra[i].buf.pv = args - ctx->olaps[oix].offset;
- pages[i].addr = ctx->buf->phys -
+ pages[i].addr = ctx->buf->dma_addr -
ctx->olaps[oix].offset +
(pkt_size - rlen);
pages[i].addr = pages[i].addr & PAGE_MASK;
@@ -1115,7 +1117,7 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx)
list[i].num = ctx->args[i].length ? 1 : 0;
list[i].pgidx = i;
if (ctx->maps[i]) {
- pages[i].addr = ctx->maps[i]->phys;
+ pages[i].addr = ctx->maps[i]->dma_addr;
pages[i].size = ctx->maps[i]->size;
}
rpra[i].dma.fd = ctx->args[i].fd;
@@ -1197,7 +1199,7 @@ static int fastrpc_invoke_send(struct fastrpc_session_ctx *sctx,
msg->ctx = ctx->ctxid | fl->pd;
msg->handle = handle;
msg->sc = ctx->sc;
- msg->addr = ctx->buf ? ctx->buf->phys : 0;
+ msg->addr = ctx->buf ? ctx->buf->dma_addr : 0;
msg->size = roundup(ctx->msg_sz, PAGE_SIZE);
fastrpc_context_get(ctx);
@@ -1357,13 +1359,15 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl,
if (fl->cctx->vmcount) {
u64 src_perms = BIT(QCOM_SCM_VMID_HLOS);
- err = qcom_scm_assign_mem(fl->cctx->remote_heap->phys,
+ err = qcom_scm_assign_mem(fl->cctx->remote_heap->dma_addr,
(u64)fl->cctx->remote_heap->size,
&src_perms,
fl->cctx->vmperms, fl->cctx->vmcount);
if (err) {
- dev_err(fl->sctx->dev, "Failed to assign memory with phys 0x%llx size 0x%llx err %d\n",
- fl->cctx->remote_heap->phys, fl->cctx->remote_heap->size, err);
+ dev_err(fl->sctx->dev,
+ "Failed to assign memory with dma_addr %pad size 0x%llx err %d\n",
+ &fl->cctx->remote_heap->dma_addr,
+ fl->cctx->remote_heap->size, err);
goto err_map;
}
scm_done = true;
@@ -1381,7 +1385,7 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl,
args[1].length = inbuf.namelen;
args[1].fd = -1;
- pages[0].addr = fl->cctx->remote_heap->phys;
+ pages[0].addr = fl->cctx->remote_heap->dma_addr;
pages[0].size = fl->cctx->remote_heap->size;
args[2].ptr = (u64)(uintptr_t) pages;
@@ -1410,12 +1414,12 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl,
dst_perms.vmid = QCOM_SCM_VMID_HLOS;
dst_perms.perm = QCOM_SCM_PERM_RWX;
- err = qcom_scm_assign_mem(fl->cctx->remote_heap->phys,
+ err = qcom_scm_assign_mem(fl->cctx->remote_heap->dma_addr,
(u64)fl->cctx->remote_heap->size,
&src_perms, &dst_perms, 1);
if (err)
- dev_err(fl->sctx->dev, "Failed to assign memory phys 0x%llx size 0x%llx err %d\n",
- fl->cctx->remote_heap->phys, fl->cctx->remote_heap->size, err);
+ dev_err(fl->sctx->dev, "Failed to assign memory dma_addr %pad size 0x%llx err %d\n",
+ &fl->cctx->remote_heap->dma_addr, fl->cctx->remote_heap->size, err);
}
err_map:
fastrpc_buf_free(fl->cctx->remote_heap);
@@ -1505,7 +1509,7 @@ static int fastrpc_init_create_process(struct fastrpc_user *fl,
args[2].length = inbuf.filelen;
args[2].fd = init.filefd;
- pages[0].addr = imem->phys;
+ pages[0].addr = imem->dma_addr;
pages[0].size = imem->size;
args[3].ptr = (u64)(uintptr_t) pages;
@@ -1945,7 +1949,7 @@ static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp)
args[0].ptr = (u64) (uintptr_t) &req_msg;
args[0].length = sizeof(req_msg);
- pages.addr = buf->phys;
+ pages.addr = buf->dma_addr;
pages.size = buf->size;
args[1].ptr = (u64) (uintptr_t) &pages;
@@ -1973,11 +1977,12 @@ static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp)
if (req.flags == ADSP_MMAP_REMOTE_HEAP_ADDR && fl->cctx->vmcount) {
u64 src_perms = BIT(QCOM_SCM_VMID_HLOS);
- err = qcom_scm_assign_mem(buf->phys, (u64)buf->size,
+ err = qcom_scm_assign_mem(buf->dma_addr, (u64)buf->size,
&src_perms, fl->cctx->vmperms, fl->cctx->vmcount);
if (err) {
- dev_err(fl->sctx->dev, "Failed to assign memory phys 0x%llx size 0x%llx err %d",
- buf->phys, buf->size, err);
+ dev_err(fl->sctx->dev,
+ "Failed to assign memory dma_addr %pad size 0x%llx err %d",
+ &buf->dma_addr, buf->size, err);
goto err_assign;
}
}
@@ -2091,7 +2096,7 @@ static int fastrpc_req_mem_map(struct fastrpc_user *fl, char __user *argp)
args[0].ptr = (u64) (uintptr_t) &req_msg;
args[0].length = sizeof(req_msg);
- pages.addr = map->phys;
+ pages.addr = map->dma_addr;
pages.size = map->len;
args[1].ptr = (u64) (uintptr_t) &pages;
diff --git a/drivers/mtd/nand/raw/Kconfig b/drivers/mtd/nand/raw/Kconfig
index 7408f34f0c68..2aba02ebe03e 100644
--- a/drivers/mtd/nand/raw/Kconfig
+++ b/drivers/mtd/nand/raw/Kconfig
@@ -71,7 +71,8 @@ config MTD_NAND_AU1550
config MTD_NAND_NDFC
tristate "IBM/MCC 4xx NAND controller"
- depends on 4xx
+ depends on 44x || COMPILE_TEST
+ depends on OF
select MTD_NAND_ECC_SW_HAMMING
select MTD_NAND_ECC_SW_HAMMING_SMC
help
diff --git a/drivers/mtd/nand/raw/ndfc.c b/drivers/mtd/nand/raw/ndfc.c
index 13365128194d..6f67bfe32a09 100644
--- a/drivers/mtd/nand/raw/ndfc.c
+++ b/drivers/mtd/nand/raw/ndfc.c
@@ -44,13 +44,13 @@ static void ndfc_select_chip(struct nand_chip *nchip, int chip)
uint32_t ccr;
struct ndfc_controller *ndfc = nand_get_controller_data(nchip);
- ccr = in_be32(ndfc->ndfcbase + NDFC_CCR);
+ ccr = ioread32be(ndfc->ndfcbase + NDFC_CCR);
if (chip >= 0) {
ccr &= ~NDFC_CCR_BS_MASK;
ccr |= NDFC_CCR_BS(chip + ndfc->chip_select);
} else
ccr |= NDFC_CCR_RESET_CE;
- out_be32(ndfc->ndfcbase + NDFC_CCR, ccr);
+ iowrite32be(ccr, ndfc->ndfcbase + NDFC_CCR);
}
static void ndfc_hwcontrol(struct nand_chip *chip, int cmd, unsigned int ctrl)
@@ -70,7 +70,7 @@ static int ndfc_ready(struct nand_chip *chip)
{
struct ndfc_controller *ndfc = nand_get_controller_data(chip);
- return in_be32(ndfc->ndfcbase + NDFC_STAT) & NDFC_STAT_IS_READY;
+ return ioread32be(ndfc->ndfcbase + NDFC_STAT) & NDFC_STAT_IS_READY;
}
static void ndfc_enable_hwecc(struct nand_chip *chip, int mode)
@@ -78,9 +78,9 @@ static void ndfc_enable_hwecc(struct nand_chip *chip, int mode)
uint32_t ccr;
struct ndfc_controller *ndfc = nand_get_controller_data(chip);
- ccr = in_be32(ndfc->ndfcbase + NDFC_CCR);
+ ccr = ioread32be(ndfc->ndfcbase + NDFC_CCR);
ccr |= NDFC_CCR_RESET_ECC;
- out_be32(ndfc->ndfcbase + NDFC_CCR, ccr);
+ iowrite32be(ccr, ndfc->ndfcbase + NDFC_CCR);
wmb();
}
@@ -92,7 +92,7 @@ static int ndfc_calculate_ecc(struct nand_chip *chip,
uint8_t *p = (uint8_t *)&ecc;
wmb();
- ecc = in_be32(ndfc->ndfcbase + NDFC_ECC);
+ ecc = ioread32be(ndfc->ndfcbase + NDFC_ECC);
/* The NDFC uses Smart Media (SMC) bytes order */
ecc_code[0] = p[1];
ecc_code[1] = p[2];
@@ -114,7 +114,7 @@ static void ndfc_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
uint32_t *p = (uint32_t *) buf;
for(;len > 0; len -= 4)
- *p++ = in_be32(ndfc->ndfcbase + NDFC_DATA);
+ *p++ = ioread32be(ndfc->ndfcbase + NDFC_DATA);
}
static void ndfc_write_buf(struct nand_chip *chip, const uint8_t *buf, int len)
@@ -123,7 +123,7 @@ static void ndfc_write_buf(struct nand_chip *chip, const uint8_t *buf, int len)
uint32_t *p = (uint32_t *) buf;
for(;len > 0; len -= 4)
- out_be32(ndfc->ndfcbase + NDFC_DATA, *p++);
+ iowrite32be(*p++, ndfc->ndfcbase + NDFC_DATA);
}
/*
@@ -185,10 +185,15 @@ static int ndfc_chip_init(struct ndfc_controller *ndfc,
static int ndfc_probe(struct platform_device *ofdev)
{
struct ndfc_controller *ndfc;
+ void __iomem *ndfcbase;
const __be32 *reg;
u32 ccr;
u32 cs;
- int err, len;
+ int len = 0;
+
+ ndfcbase = devm_platform_ioremap_resource(ofdev, 0);
+ if (IS_ERR(ndfcbase))
+ return PTR_ERR(ndfcbase);
/* Read the reg property to get the chip select */
reg = of_get_property(ofdev->dev.of_node, "reg", &len);
@@ -210,11 +215,7 @@ static int ndfc_probe(struct platform_device *ofdev)
ndfc->ofdev = ofdev;
dev_set_drvdata(&ofdev->dev, ndfc);
- ndfc->ndfcbase = of_iomap(ofdev->dev.of_node, 0);
- if (!ndfc->ndfcbase) {
- dev_err(&ofdev->dev, "failed to get memory\n");
- return -EIO;
- }
+ ndfc->ndfcbase = ndfcbase;
ccr = NDFC_CCR_BS(ndfc->chip_select);
@@ -223,22 +224,16 @@ static int ndfc_probe(struct platform_device *ofdev)
if (reg)
ccr |= be32_to_cpup(reg);
- out_be32(ndfc->ndfcbase + NDFC_CCR, ccr);
+ iowrite32be(ccr, ndfc->ndfcbase + NDFC_CCR);
/* Set the bank settings if given */
reg = of_get_property(ofdev->dev.of_node, "bank-settings", NULL);
if (reg) {
int offset = NDFC_BCFG0 + (ndfc->chip_select << 2);
- out_be32(ndfc->ndfcbase + offset, be32_to_cpup(reg));
+ iowrite32be(be32_to_cpup(reg), ndfc->ndfcbase + offset);
}
- err = ndfc_chip_init(ndfc, ofdev->dev.of_node);
- if (err) {
- iounmap(ndfc->ndfcbase);
- return err;
- }
-
- return 0;
+ return ndfc_chip_init(ndfc, ofdev->dev.of_node);
}
static void ndfc_remove(struct platform_device *ofdev)
diff --git a/drivers/mtd/spi-nor/macronix.c b/drivers/mtd/spi-nor/macronix.c
index e97f5cbd9aad..c854a1163a47 100644
--- a/drivers/mtd/spi-nor/macronix.c
+++ b/drivers/mtd/spi-nor/macronix.c
@@ -132,7 +132,9 @@ static const struct flash_info macronix_nor_parts[] = {
}, {
/* MX25L12805D */
.id = SNOR_ID(0xc2, 0x20, 0x18),
+ .size = SZ_16M,
.flags = SPI_NOR_HAS_LOCK | SPI_NOR_4BIT_BP,
+ .no_sfdp_flags = SECT_4K,
}, {
/* MX25L25635E, MX25L25645G */
.id = SNOR_ID(0xc2, 0x20, 0x19),
diff --git a/drivers/net/can/dev/netlink.c b/drivers/net/can/dev/netlink.c
index fcfff221d1cc..f4ab494b9151 100644
--- a/drivers/net/can/dev/netlink.c
+++ b/drivers/net/can/dev/netlink.c
@@ -586,7 +586,7 @@ static size_t can_get_size(const struct net_device *dev)
size += nla_total_size(sizeof(u32)); /* IFLA_CAN_STATE */
size += nla_total_size(sizeof(struct can_ctrlmode)); /* IFLA_CAN_CTRLMODE */
size += nla_total_size(sizeof(u32)); /* IFLA_CAN_RESTART_MS */
- if (priv->do_get_berr_counter) /* IFLA_CAN_BERR_COUNTER */
+ if (priv->do_get_berr_counter && netif_running(dev)) /* IFLA_CAN_BERR_COUNTER */
size += nla_total_size(sizeof(struct can_berr_counter));
if (priv->termination_const) {
size += nla_total_size(sizeof(priv->termination)); /* IFLA_CAN_TERMINATION */
@@ -737,7 +737,7 @@ static int can_fill_info(struct sk_buff *skb, const struct net_device *dev)
nla_put(skb, IFLA_CAN_CTRLMODE, sizeof(cm), &cm) ||
nla_put_u32(skb, IFLA_CAN_RESTART_MS, priv->restart_ms) ||
- (priv->do_get_berr_counter &&
+ (priv->do_get_berr_counter && netif_running(dev) &&
!priv->do_get_berr_counter(dev, &bec) &&
nla_put(skb, IFLA_CAN_BERR_COUNTER, sizeof(bec), &bec)) ||
diff --git a/drivers/net/can/dev/skb.c b/drivers/net/can/dev/skb.c
index 3ebd4f779b9b..6fb1f9b2a9e2 100644
--- a/drivers/net/can/dev/skb.c
+++ b/drivers/net/can/dev/skb.c
@@ -52,6 +52,7 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev,
if (idx >= priv->echo_skb_max) {
netdev_err(dev, "%s: BUG! Trying to access can_priv::echo_skb out of bounds (%u/max %u)\n",
__func__, idx, priv->echo_skb_max);
+ dev_kfree_skb_any(skb);
return -EINVAL;
}
diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
index f92a3b404ec1..7823b4b52c16 100644
--- a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
+++ b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
@@ -755,7 +755,7 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb)
u16 rx_msg_type, rx_msg_size;
size_t rx_msg_min_size;
- if (msg_end - msg_ptr < sizeof(*rx_msg)) {
+ if (msg_end - msg_ptr < sizeof(rx_msg->size)) {
err = -EBADMSG;
break;
}
@@ -766,6 +766,11 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb)
break;
}
+ if (msg_end - msg_ptr < sizeof(*rx_msg)) {
+ err = -EBADMSG;
+ break;
+ }
+
rx_msg_size = le16_to_cpu(rx_msg->size);
rx_msg_type = le16_to_cpu(rx_msg->type);
diff --git a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
index e58f1d2c26bd..eddd8406867f 100644
--- a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
+++ b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
@@ -1934,6 +1934,39 @@ static int atl1c_clean_rx(struct napi_struct *napi, int budget)
break;
}
atl1c_clean_rfd(rfd_ring, rrs, rfd_num);
+
+ /* A link partner whose l2mtu is larger than ours can hand us a
+ * frame that does not fit the buffer the hardware was given:
+ * the descriptor reports the untruncated length while at most
+ * buffer_info->length bytes were written. skb_put() of such a
+ * length would pull in whatever the tail of the buffer holds
+ * and, past the end of the skb, trips skb_over_panic and takes
+ * the host down. Bound it by what the device actually wrote
+ * rather than by skb_tailroom(): the skb carries 64 bytes of
+ * slack for the DMA erratum, so the two are not the same.
+ *
+ * Frames shorter than an Ethernet header plus its FCS are
+ * rejected by the same test; before this, a length below
+ * ETH_FCS_LEN underflowed in skb_put().
+ */
+ if (unlikely(length < ETH_HLEN + ETH_FCS_LEN ||
+ length > buffer_info->length)) {
+ dev_core_stats_rx_dropped_inc(netdev);
+ if (netif_msg_rx_err(adapter))
+ dev_warn_ratelimited(&pdev->dev,
+ "dropped rx frame: %u bytes, buffer %u\n",
+ length, buffer_info->length);
+ kfree_skb_reason(skb, SKB_DROP_REASON_PKT_TOO_BIG);
+ /* atl1c_clean_rfd() has already recycled the RFD, so
+ * count the drop as work: the refill at the end of the
+ * poll only runs when work_done is non-zero, and
+ * sustained oversized traffic would otherwise drain the
+ * ring and stop RX for good.
+ */
+ work_done++;
+ continue;
+ }
+
skb_put(skb, length - ETH_FCS_LEN);
skb->protocol = eth_type_trans(skb, netdev);
atl1c_rx_checksum(adapter, skb, rrs);
diff --git a/drivers/net/ethernet/broadcom/bgmac.c b/drivers/net/ethernet/broadcom/bgmac.c
index 3e9c57196a39..1835266fdd09 100644
--- a/drivers/net/ethernet/broadcom/bgmac.c
+++ b/drivers/net/ethernet/broadcom/bgmac.c
@@ -1183,9 +1183,6 @@ static int bgmac_open(struct net_device *net_dev)
if (err)
return err;
- /* Specs say about reclaiming rings here, but we do that in DMA init */
- bgmac_chip_init(bgmac);
-
err = request_irq(bgmac->irq, bgmac_interrupt, IRQF_SHARED,
net_dev->name, net_dev);
if (err < 0) {
@@ -1195,6 +1192,9 @@ static int bgmac_open(struct net_device *net_dev)
}
napi_enable(&bgmac->napi);
+ /* Specs say about reclaiming rings here, but we do that in DMA init */
+ bgmac_chip_init(bgmac);
+
phy_start(net_dev->phydev);
netif_start_queue(net_dev);
@@ -1608,10 +1608,10 @@ int bgmac_enet_resume(struct bgmac *bgmac)
if (rc)
return rc;
- bgmac_chip_init(bgmac);
-
napi_enable(&bgmac->napi);
+ bgmac_chip_init(bgmac);
+
netif_tx_lock(bgmac->net_dev);
netif_device_attach(bgmac->net_dev);
netif_tx_unlock(bgmac->net_dev);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 602b97d00e9a..8ef1d9d2f135 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -13000,6 +13000,21 @@ static int bnxt_tx_nr_rings_per_tc(struct bnxt *bp)
return bp->num_tc ? bp->tx_nr_rings / bp->num_tc : bp->tx_nr_rings;
}
+static void bnxt_set_xdp_tx_rings(struct bnxt *bp)
+{
+ bp->tx_nr_rings_xdp = bp->tx_nr_rings_per_tc;
+ bp->tx_nr_rings += bp->tx_nr_rings_xdp;
+}
+
+static void bnxt_adj_tx_rings(struct bnxt *bp)
+{
+ /* Make adjustments if reserved TX rings are less than requested */
+ bp->tx_nr_rings -= bp->tx_nr_rings_xdp;
+ bp->tx_nr_rings_per_tc = bnxt_tx_nr_rings_per_tc(bp);
+ if (bp->tx_nr_rings_xdp)
+ bnxt_set_xdp_tx_rings(bp);
+}
+
static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)
{
int rc = 0;
@@ -13017,13 +13032,7 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)
if (rc)
return rc;
- /* Make adjustments if reserved TX rings are less than requested */
- bp->tx_nr_rings -= bp->tx_nr_rings_xdp;
- bp->tx_nr_rings_per_tc = bnxt_tx_nr_rings_per_tc(bp);
- if (bp->tx_nr_rings_xdp) {
- bp->tx_nr_rings_xdp = bp->tx_nr_rings_per_tc;
- bp->tx_nr_rings += bp->tx_nr_rings_xdp;
- }
+ bnxt_adj_tx_rings(bp);
rc = bnxt_alloc_mem(bp, irq_re_init);
if (rc) {
netdev_err(bp->dev, "bnxt_alloc_mem err: %x\n", rc);
@@ -15515,11 +15524,19 @@ static int bnxt_change_mtu(struct net_device *dev, int new_mtu)
return 0;
}
+void bnxt_set_cp_rings(struct bnxt *bp, bool sh)
+{
+ int tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings);
+
+ bp->cp_nr_rings = sh ? max_t(int, tx_cp, bp->rx_nr_rings) :
+ tx_cp + bp->rx_nr_rings;
+}
+
int bnxt_setup_mq_tc(struct net_device *dev, u8 tc)
{
struct bnxt *bp = netdev_priv(dev);
bool sh = false;
- int rc, tx_cp;
+ int rc;
if (tc > bp->max_tc) {
netdev_err(dev, "Too many traffic classes requested: %d. Max supported is %d.\n",
@@ -15552,9 +15569,7 @@ int bnxt_setup_mq_tc(struct net_device *dev, u8 tc)
bp->num_tc = 0;
}
bp->tx_nr_rings += bp->tx_nr_rings_xdp;
- tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings);
- bp->cp_nr_rings = sh ? max_t(int, tx_cp, bp->rx_nr_rings) :
- tx_cp + bp->rx_nr_rings;
+ bnxt_set_cp_rings(bp, sh);
if (netif_running(bp->dev))
return bnxt_open_nic(bp, true, false);
@@ -16616,6 +16631,15 @@ static void bnxt_trim_dflt_sh_rings(struct bnxt *bp)
bp->tx_nr_rings = bnxt_tx_nr_rings(bp);
}
+static void bnxt_adj_dflt_rings(struct bnxt *bp, bool sh)
+{
+ if (sh)
+ bnxt_trim_dflt_sh_rings(bp);
+ else
+ bp->cp_nr_rings = bp->tx_nr_rings_per_tc + bp->rx_nr_rings;
+ bp->tx_nr_rings = bnxt_tx_nr_rings(bp);
+}
+
static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)
{
int dflt_rings, max_rx_rings, max_tx_rings, rc;
@@ -16641,11 +16665,8 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)
return rc;
bp->rx_nr_rings = min_t(int, dflt_rings, max_rx_rings);
bp->tx_nr_rings_per_tc = min_t(int, dflt_rings, max_tx_rings);
- if (sh)
- bnxt_trim_dflt_sh_rings(bp);
- else
- bp->cp_nr_rings = bp->tx_nr_rings_per_tc + bp->rx_nr_rings;
- bp->tx_nr_rings = bnxt_tx_nr_rings(bp);
+
+ bnxt_adj_dflt_rings(bp, sh);
avail_msix = bnxt_get_max_func_irqs(bp) - bp->cp_nr_rings;
if (avail_msix >= BNXT_MIN_ROCE_CP_RINGS) {
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 1d17ffb7415e..668d929ddb7d 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -2987,6 +2987,7 @@ int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs,
int tx_xdp);
int bnxt_fw_init_one(struct bnxt *bp);
bool bnxt_hwrm_reset_permitted(struct bnxt *bp);
+void bnxt_set_cp_rings(struct bnxt *bp, bool sh);
int bnxt_setup_mq_tc(struct net_device *dev, u8 tc);
struct bnxt_ntuple_filter *bnxt_lookup_ntp_filter_from_idx(struct bnxt *bp,
struct bnxt_ntuple_filter *fltr, u32 idx);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index 7f60d7b9f557..2dc9708fb63f 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -940,7 +940,6 @@ static int bnxt_set_channels(struct net_device *dev,
bool sh = false;
int tx_xdp = 0;
int rc = 0;
- int tx_cp;
if (channel->other_count)
return -EINVAL;
@@ -1008,9 +1007,7 @@ static int bnxt_set_channels(struct net_device *dev,
if (tcs > 1)
bp->tx_nr_rings = bp->tx_nr_rings_per_tc * tcs + tx_xdp;
- tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings);
- bp->cp_nr_rings = sh ? max_t(int, tx_cp, bp->rx_nr_rings) :
- tx_cp + bp->rx_nr_rings;
+ bnxt_set_cp_rings(bp, sh);
/* After changing number of rx channels, update NTUPLE feature. */
netdev_update_features(dev);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
index 85cbeb35681c..2db8cfcf922f 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
@@ -381,11 +381,31 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames,
return nxmit;
}
+static void bnxt_xdp_apply_cfg(struct bnxt *bp, int tx_xdp)
+{
+ struct net_device *dev = bp->dev;
+ int tc = bp->num_tc ? : 1;
+
+ if (bp->xdp_prog) {
+ bnxt_set_rx_skb_mode(bp, true);
+ xdp_features_set_redirect_target_locked(dev, true);
+ } else {
+ xdp_features_clear_redirect_target_locked(dev);
+ bnxt_set_rx_skb_mode(bp, false);
+ }
+ bp->tx_nr_rings_xdp = tx_xdp;
+ bp->tx_nr_rings = bp->tx_nr_rings_per_tc * tc + tx_xdp;
+ bnxt_set_cp_rings(bp, true);
+ bnxt_set_tpa_flags(bp);
+ bnxt_set_ring_params(bp);
+}
+
static int bnxt_xdp_set(struct bnxt *bp, struct bpf_prog *prog)
{
struct net_device *dev = bp->dev;
- int tx_xdp = 0, tx_cp, rc, tc;
+ int tx_xdp = 0, rc, tc;
struct bpf_prog *old;
+ int old_tx_xdp;
netdev_assert_locked(dev);
@@ -418,26 +438,22 @@ static int bnxt_xdp_set(struct bnxt *bp, struct bpf_prog *prog)
if (netif_running(dev))
bnxt_close_nic(bp, true, false);
+ old_tx_xdp = bp->tx_nr_rings_xdp;
old = xchg(&bp->xdp_prog, prog);
- if (old)
- bpf_prog_put(old);
-
- if (prog) {
- bnxt_set_rx_skb_mode(bp, true);
- xdp_features_set_redirect_target_locked(dev, true);
- } else {
- xdp_features_clear_redirect_target_locked(dev);
- bnxt_set_rx_skb_mode(bp, false);
+ bnxt_xdp_apply_cfg(bp, tx_xdp);
+
+ if (netif_running(dev)) {
+ rc = bnxt_open_nic(bp, true, false);
+ /* a detach cannot be undone, only an attach */
+ if (rc && prog) {
+ WRITE_ONCE(bp->xdp_prog, old);
+ bnxt_xdp_apply_cfg(bp, old_tx_xdp);
+ return rc;
+ }
}
- bp->tx_nr_rings_xdp = tx_xdp;
- bp->tx_nr_rings = bp->tx_nr_rings_per_tc * tc + tx_xdp;
- tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings);
- bp->cp_nr_rings = max_t(int, tx_cp, bp->rx_nr_rings);
- bnxt_set_tpa_flags(bp);
- bnxt_set_ring_params(bp);
- if (netif_running(dev))
- return bnxt_open_nic(bp, true, false);
+ if (old)
+ bpf_prog_put(old);
return 0;
}
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index 50eb40de7b22..897bb86bedb1 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -18201,41 +18201,17 @@ static int tg3_resume(struct device *device)
static SIMPLE_DEV_PM_OPS(tg3_pm_ops, tg3_suspend, tg3_resume);
-/* Systems where ACPI _PTS (Prepare To Sleep) S5 will result in a fatal
- * PCIe AER event on the tg3 device if the tg3 device is not, or cannot
- * be, powered down.
+/* Systems where powering down a tg3 device on restarts triggers
+ * NMIs and fatal hangs during reboot if the network adapters were
+ * initialized under SNP.
*/
-static const struct dmi_system_id tg3_restart_aer_quirk_table[] = {
- {
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
- DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R440"),
- },
- },
- {
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
- DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R540"),
- },
- },
- {
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
- DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R640"),
- },
- },
+static const struct dmi_system_id tg3_restart_snp_quirk_table[] = {
{
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R650"),
},
},
- {
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
- DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R740"),
- },
- },
{
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
@@ -18259,14 +18235,18 @@ static void tg3_shutdown(struct pci_dev *pdev)
if (netif_running(dev))
dev_close(dev);
- if (system_state == SYSTEM_POWER_OFF)
+ if (!dmi_first_match(tg3_restart_snp_quirk_table) ||
+ system_state == SYSTEM_POWER_OFF) {
tg3_power_down(tp);
- else if (system_state == SYSTEM_RESTART &&
- dmi_first_match(tg3_restart_aer_quirk_table) &&
+ } else if (system_state == SYSTEM_RESTART &&
pdev->current_state != PCI_D3cold &&
pdev->current_state != PCI_UNKNOWN) {
/* Disable PCIe AER on the tg3 to avoid a fatal
* error during this system restart.
+ * Needed since on restarts for systems in
+ * tg3_restart_snp_quirk_table skipping on powering down the
+ * tg3 device causes a fatal AER error during ACPI _PTS
+ * (Prepare To Sleep).
*/
pcie_capability_clear_word(pdev, PCI_EXP_DEVCTL,
PCI_EXP_DEVCTL_CERE |
diff --git a/drivers/net/ethernet/jme.c b/drivers/net/ethernet/jme.c
index d8be0e4dcb07..6cfcfda05554 100644
--- a/drivers/net/ethernet/jme.c
+++ b/drivers/net/ethernet/jme.c
@@ -1502,7 +1502,8 @@ jme_intr_msi(struct jme_adapter *jme, u32 intrstat)
* all other events are ignored
*/
jwrite32(jme, JME_IEVE, intrstat);
- schedule_work(&jme->linkch_task);
+ if (!test_bit(JME_FLAG_WOL_QUIESCE, &jme->flags))
+ schedule_work(&jme->linkch_task);
goto out_reenable;
}
@@ -1546,7 +1547,8 @@ jme_intr_msi(struct jme_adapter *jme, u32 intrstat)
/*
* Re-enable interrupt
*/
- jwrite32f(jme, JME_IENS, INTR_ENABLE);
+ if (!test_bit(JME_FLAG_WOL_QUIESCE, &jme->flags))
+ jwrite32f(jme, JME_IENS, INTR_ENABLE);
}
static irqreturn_t
@@ -1892,6 +1894,7 @@ jme_powersave_phy(struct jme_adapter *jme)
jme_set_100m_half(jme);
if (jme->reg_pmcs & (PMCS_LFEN | PMCS_LREN))
jme_wait_link(jme);
+ jme_mac_rxclk_on(jme);
jme_clear_pm_enable_wol(jme);
} else {
jme_phy_off(jme);
@@ -3158,6 +3161,21 @@ jme_shutdown(struct pci_dev *pdev)
struct net_device *netdev = pci_get_drvdata(pdev);
struct jme_adapter *jme = netdev_priv(netdev);
+ if (netif_running(netdev)) {
+ set_bit(JME_FLAG_WOL_QUIESCE, &jme->flags);
+ netif_device_detach(netdev);
+ netif_stop_queue(netdev);
+ jme_stop_irq(jme);
+ synchronize_irq(pdev->irq);
+ jme_stop_irq(jme);
+ cancel_work_sync(&jme->linkch_task);
+ tasklet_disable(&jme->pcc_task);
+ napi_disable(&jme->napi);
+ tasklet_disable(&jme->txclean_task);
+ tasklet_disable(&jme->rxclean_task);
+ tasklet_disable(&jme->rxempty_task);
+ }
+
jme_powersave_phy(jme);
pci_pme_active(pdev, true);
}
@@ -3172,11 +3190,16 @@ jme_suspend(struct device *dev)
if (!netif_running(netdev))
return 0;
- atomic_dec(&jme->link_changing);
-
+ set_bit(JME_FLAG_WOL_QUIESCE, &jme->flags);
netif_device_detach(netdev);
netif_stop_queue(netdev);
jme_stop_irq(jme);
+ synchronize_irq(jme->pdev->irq);
+ jme_stop_irq(jme);
+ cancel_work_sync(&jme->linkch_task);
+ tasklet_disable(&jme->pcc_task);
+ napi_disable(&jme->napi);
+ atomic_dec(&jme->link_changing);
tasklet_disable(&jme->txclean_task);
tasklet_disable(&jme->rxclean_task);
@@ -3215,6 +3238,7 @@ jme_resume(struct device *dev)
return 0;
jme_clear_pm_disable_wol(jme);
+ jme_mac_rxclk_off(jme);
jme_phy_on(jme);
if (test_bit(JME_FLAG_SSET, &jme->flags))
jme_set_link_ksettings(netdev, &jme->old_cmd);
@@ -3226,8 +3250,11 @@ jme_resume(struct device *dev)
atomic_inc(&jme->link_changing);
- jme_reset_link(jme);
+ napi_enable(&jme->napi);
+ tasklet_enable(&jme->pcc_task);
+ clear_bit(JME_FLAG_WOL_QUIESCE, &jme->flags);
+ jme_reset_link(jme);
jme_start_irq(jme);
return 0;
diff --git a/drivers/net/ethernet/jme.h b/drivers/net/ethernet/jme.h
index 860494ff3714..ec5dfddf7322 100644
--- a/drivers/net/ethernet/jme.h
+++ b/drivers/net/ethernet/jme.h
@@ -450,6 +450,7 @@ enum jme_flags_bits {
JME_FLAG_SSET = 2,
JME_FLAG_POLL = 5,
JME_FLAG_SHUTDOWN = 6,
+ JME_FLAG_WOL_QUIESCE = 7,
};
#define TX_TIMEOUT (5 * HZ)
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.c b/drivers/net/ethernet/mellanox/mlxsw/core.c
index 83c7cf3bbea3..bd15c14bf5c8 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/core.c
@@ -2001,7 +2001,7 @@ mlxsw_core_health_fw_fatal_test(struct devlink_health_reporter *reporter,
struct netlink_ext_ack *extack)
{
struct mlxsw_core *mlxsw_core = devlink_health_reporter_priv(reporter);
- char mfgd_pl[MLXSW_REG_MFGD_LEN];
+ char mfgd_pl[MLXSW_REG_MFGD_LEN] = {0};
int err;
/* Read the register first to make sure no other bits are changed. */
@@ -2022,7 +2022,7 @@ mlxsw_core_health_fw_fatal_ops = {
static int mlxsw_core_health_fw_fatal_config(struct mlxsw_core *mlxsw_core,
bool enable)
{
- char mfgd_pl[MLXSW_REG_MFGD_LEN];
+ char mfgd_pl[MLXSW_REG_MFGD_LEN] = {0};
int err;
/* Read the register first to make sure no other bits are changed. */
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c b/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c
index 9c12e1feb643..e39a8b0a5709 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c
@@ -120,7 +120,7 @@ static ssize_t mlxsw_hwmon_temp_rst_store(struct device *dev,
container_of(attr, struct mlxsw_hwmon_attr, dev_attr);
struct mlxsw_hwmon_dev *mlxsw_hwmon_dev = mlxsw_hwmon_attr->mlxsw_hwmon_dev;
struct mlxsw_hwmon *mlxsw_hwmon = mlxsw_hwmon_dev->hwmon;
- char mtmp_pl[MLXSW_REG_MTMP_LEN];
+ char mtmp_pl[MLXSW_REG_MTMP_LEN] = {0};
unsigned long val;
int index;
int err;
@@ -177,7 +177,7 @@ static ssize_t mlxsw_hwmon_fan_fault_show(struct device *dev,
container_of(attr, struct mlxsw_hwmon_attr, dev_attr);
struct mlxsw_hwmon_dev *mlxsw_hwmon_dev = mlxsw_hwmon_attr->mlxsw_hwmon_dev;
struct mlxsw_hwmon *mlxsw_hwmon = mlxsw_hwmon_dev->hwmon;
- char fore_pl[MLXSW_REG_FORE_LEN];
+ char fore_pl[MLXSW_REG_FORE_LEN] = {0};
bool fault;
int err;
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c
index 32d2e61f2b82..91fb77c8e20d 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c
@@ -1720,7 +1720,7 @@ static int mlxsw_sp2_span_init(struct mlxsw_sp *mlxsw_sp)
static int mlxsw_sp2_span_policer_id_base_set(struct mlxsw_sp *mlxsw_sp,
u16 policer_id_base)
{
- char mogcr_pl[MLXSW_REG_MOGCR_LEN];
+ char mogcr_pl[MLXSW_REG_MOGCR_LEN] = {0};
int err;
err = mlxsw_reg_query(mlxsw_sp->core, MLXSW_REG(mogcr), mogcr_pl);
diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.c b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
index 5bcf1126d627..89fdc30d80a4 100644
--- a/drivers/net/ethernet/ti/am65-cpsw-nuss.c
+++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
@@ -128,6 +128,7 @@
#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_319 BIT(21)
#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_320 BIT(22)
#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO BIT(23)
+#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_UNI_EN BIT(24)
/* The PTP event messages - Sync, Delay_Req, Pdelay_Req, and Pdelay_Resp. */
#define AM65_CPSW_TS_EVENT_MSG_TYPE_BITS (BIT(0) | BIT(1) | BIT(2) | BIT(3))
@@ -1859,7 +1860,8 @@ static int am65_cpsw_nuss_hwtstamp_set(struct net_device *ndev,
AM65_CPSW_PN_TS_CTL_LTYPE2_TS_132 |
AM65_CPSW_PN_TS_CTL_LTYPE2_TS_319 |
AM65_CPSW_PN_TS_CTL_LTYPE2_TS_320 |
- AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO;
+ AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO |
+ AM65_CPSW_PN_TS_CTL_LTYPE2_TS_UNI_EN;
ts_ctrl = AM65_CPSW_TS_EVENT_MSG_TYPE_BITS <<
AM65_CPSW_PN_TS_CTL_MSG_TYPE_EN_SHIFT;
diff --git a/drivers/net/hamradio/6pack.c b/drivers/net/hamradio/6pack.c
index c8b2dc5c1bec..549bd79d7636 100644
--- a/drivers/net/hamradio/6pack.c
+++ b/drivers/net/hamradio/6pack.c
@@ -256,6 +256,12 @@ static int sp_open_dev(struct net_device *dev)
if (sp->tty == NULL)
return -ENODEV;
+
+ /* sp_close() stopped the queue: restart it, or the interface
+ * never transmits again after it is set down and up.
+ */
+ netif_start_queue(dev);
+
return 0;
}
diff --git a/drivers/net/hamradio/bpqether.c b/drivers/net/hamradio/bpqether.c
index 045c5177262e..f30616d915fd 100644
--- a/drivers/net/hamradio/bpqether.c
+++ b/drivers/net/hamradio/bpqether.c
@@ -267,11 +267,10 @@ static netdev_tx_t bpq_xmit(struct sk_buff *skb, struct net_device *dev)
skb->protocol = ax25_type_trans(skb, dev);
skb_reset_network_header(skb);
dev_hard_header(skb, dev, ETH_P_BPQ, bpq->dest_addr, NULL, 0);
- dev->stats.tx_packets++;
- dev->stats.tx_bytes+=skb->len;
-
+ orig_dev->stats.tx_packets++;
+ orig_dev->stats.tx_bytes += skb->len;
+
dev_queue_xmit(skb);
- netif_wake_queue(dev);
return NETDEV_TX_OK;
}
@@ -367,6 +366,7 @@ static void *bpq_seq_start(struct seq_file *seq, loff_t *pos)
list_for_each_entry_rcu(bpqdev, &bpq_devices, bpq_list) {
if (i == *pos)
return bpqdev;
+ i++;
}
return NULL;
}
diff --git a/drivers/net/hamradio/mkiss.c b/drivers/net/hamradio/mkiss.c
index 5f38a002bd9e..8b0e2ca5d0ea 100644
--- a/drivers/net/hamradio/mkiss.c
+++ b/drivers/net/hamradio/mkiss.c
@@ -30,6 +30,7 @@
#include <net/ax25.h>
#define AX_MTU 236
+#define AX_MAX_MTU 65535
/* some arch define END as assembly function ending, just undef it */
#undef END
@@ -355,13 +356,19 @@ static int ax_set_mac_address(struct net_device *dev, void *addr)
/*---------------------------------------------------------------------------*/
-static void ax_changedmtu(struct mkiss *ax)
-{
- struct net_device *dev = ax->dev;
- unsigned char *xbuff, *rbuff, *oxbuff, *orbuff;
- int len;
+/*
+ * Length of the frame buffers for the current MTU.
+ *
+ * ax_encaps() accepts frames of up to dev->mtu + 73 bytes and KISS escaping
+ * can double every one of them. The buffers are allocated AX_BUF_SLACK bytes
+ * larger than this, for the two END delimiters and the two CRC bytes, which
+ * can need escaping as well.
+ */
+#define AX_BUF_SLACK 6
- len = dev->mtu * 2;
+static int ax_buflen(const struct net_device *dev)
+{
+ int len = (dev->mtu + 73) * 2;
/*
* allow for arrival of larger UDP packets, even if we say not to
@@ -371,8 +378,19 @@ static void ax_changedmtu(struct mkiss *ax)
if (len < 576 * 2)
len = 576 * 2;
- xbuff = kmalloc(len + 4, GFP_ATOMIC);
- rbuff = kmalloc(len + 4, GFP_ATOMIC);
+ return len;
+}
+
+static void ax_changedmtu(struct mkiss *ax)
+{
+ struct net_device *dev = ax->dev;
+ unsigned char *xbuff, *rbuff, *oxbuff, *orbuff;
+ int len;
+
+ len = ax_buflen(dev);
+
+ xbuff = kmalloc(len + AX_BUF_SLACK, GFP_ATOMIC);
+ rbuff = kmalloc(len + AX_BUF_SLACK, GFP_ATOMIC);
if (xbuff == NULL || rbuff == NULL) {
printk(KERN_ERR "mkiss: %s: unable to grow ax25 buffers, "
@@ -508,6 +526,14 @@ static void ax_encaps(struct net_device *dev, unsigned char *icp, int len)
set_bit(TTY_DO_WRITE_WAKEUP, &ax->tty->flags);
actual = ax->tty->ops->write(ax->tty, ax->xbuff, count);
+ if (actual < 0) {
+ /* The tty refused the frame: drop it. */
+ clear_bit(TTY_DO_WRITE_WAKEUP, &ax->tty->flags);
+ dev->stats.tx_errors++;
+ ax->xleft = 0;
+ netif_start_queue(dev);
+ return;
+ }
dev->stats.tx_packets++;
dev->stats.tx_bytes += actual;
@@ -563,6 +589,11 @@ static int ax_open_dev(struct net_device *dev)
if (ax->tty == NULL)
return -ENODEV;
+ /* ax_close() stopped the queue: restart it, or the interface
+ * never transmits again after it is set down and up.
+ */
+ netif_start_queue(dev);
+
return 0;
}
@@ -581,20 +612,12 @@ static int ax_open(struct net_device *dev)
* rbuff Receive buffer.
* xbuff Transmit buffer.
*/
- len = dev->mtu * 2;
+ len = ax_buflen(dev);
- /*
- * allow for arrival of larger UDP packets, even if we say not to
- * also fixes a bug in which SunOS sends 512-byte packets even with
- * an MSS of 128
- */
- if (len < 576 * 2)
- len = 576 * 2;
-
- if ((ax->rbuff = kmalloc(len + 4, GFP_KERNEL)) == NULL)
+ if ((ax->rbuff = kmalloc(len + AX_BUF_SLACK, GFP_KERNEL)) == NULL)
goto norbuff;
- if ((ax->xbuff = kmalloc(len + 4, GFP_KERNEL)) == NULL)
+ if ((ax->xbuff = kmalloc(len + AX_BUF_SLACK, GFP_KERNEL)) == NULL)
goto noxbuff;
ax->mtu = dev->mtu + 73;
@@ -640,6 +663,7 @@ static void ax_setup(struct net_device *dev)
{
/* Finish setting up the DEVICE info. */
dev->mtu = AX_MTU;
+ dev->max_mtu = AX_MAX_MTU;
dev->hard_header_len = AX25_MAX_HEADER_LEN;
dev->addr_len = AX25_ADDR_LEN;
dev->type = ARPHRD_AX25;
@@ -928,6 +952,14 @@ static void mkiss_write_wakeup(struct tty_struct *tty)
}
actual = tty->ops->write(tty, ax->xhead, ax->xleft);
+ if (actual < 0) {
+ /* Give up on the rest of this frame. */
+ clear_bit(TTY_DO_WRITE_WAKEUP, &tty->flags);
+ ax->dev->stats.tx_errors++;
+ ax->xleft = 0;
+ netif_wake_queue(ax->dev);
+ goto out;
+ }
ax->xleft -= actual;
ax->xhead += actual;
diff --git a/drivers/net/ppp/pppoe.c b/drivers/net/ppp/pppoe.c
index 7ee63051a161..0bfb048c291a 100644
--- a/drivers/net/ppp/pppoe.c
+++ b/drivers/net/ppp/pppoe.c
@@ -195,18 +195,16 @@ static int __set_item(struct pppoe_net *pn, struct pppox_sock *po)
return 0;
}
-static void __delete_item(struct pppoe_net *pn, __be16 sid,
- char *addr, int ifindex)
+static void __delete_item(struct pppoe_net *pn, struct pppox_sock *po)
{
- int hash = hash_item(sid, addr);
+ int hash = hash_item(po->pppoe_pa.sid, po->pppoe_pa.remote);
struct pppox_sock *ret, __rcu **src;
ret = rcu_dereference_protected(pn->hash_table[hash], lockdep_is_held(&pn->hash_lock));
src = &pn->hash_table[hash];
while (ret) {
- if (cmp_addr(&ret->pppoe_pa, sid, addr) &&
- ret->pppoe_ifindex == ifindex) {
+ if (ret == po) {
struct pppox_sock *next;
next = rcu_dereference_protected(ret->next,
@@ -256,11 +254,10 @@ static inline struct pppox_sock *__get_item_by_addr(struct net *net,
return pppox_sock;
}
-static inline void delete_item(struct pppoe_net *pn, __be16 sid,
- char *addr, int ifindex)
+static void delete_item(struct pppoe_net *pn, struct pppox_sock *po)
{
spin_lock(&pn->hash_lock);
- __delete_item(pn, sid, addr, ifindex);
+ __delete_item(pn, po);
spin_unlock(&pn->hash_lock);
}
@@ -602,8 +599,7 @@ static int pppoe_release(struct socket *sock)
* protect "po" from concurrent updates
* on pppoe_flush_dev
*/
- delete_item(pn, po->pppoe_pa.sid, po->pppoe_pa.remote,
- po->pppoe_ifindex);
+ delete_item(pn, po);
sock_orphan(sk);
sock->sk = NULL;
@@ -653,8 +649,7 @@ static int pppoe_connect(struct socket *sock, struct sockaddr *uservaddr,
if (stage_session(po->pppoe_pa.sid)) {
pppox_unbind_sock(sk);
pn = pppoe_pernet(sock_net(sk));
- delete_item(pn, po->pppoe_pa.sid,
- po->pppoe_pa.remote, po->pppoe_ifindex);
+ delete_item(pn, po);
if (po->pppoe_dev) {
dev_put(po->pppoe_dev);
po->pppoe_dev = NULL;
@@ -705,8 +700,7 @@ static int pppoe_connect(struct socket *sock, struct sockaddr *uservaddr,
error = ppp_register_net_channel(dev_net(dev), &po->chan);
if (error) {
- delete_item(pn, po->pppoe_pa.sid,
- po->pppoe_pa.remote, po->pppoe_ifindex);
+ delete_item(pn, po);
goto err_put;
}
diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 25cdbba5dfa9..c2b8b9d02e35 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -1371,9 +1371,10 @@ static int veth_set_channels(struct net_device *dev,
* to identify the range we have to disable
*/
veth_disable_range_safe(dev, new_rx_count, old_rx_count);
- netif_carrier_on(dev);
- if (peer)
+ if (peer && (peer->flags & IFF_UP)) {
+ netif_carrier_on(dev);
netif_carrier_on(peer);
+ }
}
/* update XDP supported features */
diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c
index 580fdec50bd5..090d50b16381 100644
--- a/drivers/net/vxlan/vxlan_core.c
+++ b/drivers/net/vxlan/vxlan_core.c
@@ -997,12 +997,6 @@ static int vxlan_fdb_update_existing(struct vxlan_dev *vxlan,
return -EOPNOTSUPP;
}
- if (rcu_access_pointer(f->nh) &&
- !(state & (NUD_PERMANENT | NUD_NOARP))) {
- NL_SET_ERR_MSG(extack, "Cannot make a nexthop fdb dynamic");
- return -EOPNOTSUPP;
- }
-
/* Do not allow an externally learned entry to take over an entry added
* by the user.
*/
@@ -1264,11 +1258,6 @@ static int vxlan_fdb_add(struct ndmsg *ndm, struct nlattr *tb[],
if (err)
return err;
- if (nhid && !(ndm->ndm_state & (NUD_PERMANENT | NUD_NOARP))) {
- NL_SET_ERR_MSG(extack, "A nexthop fdb cannot be dynamic");
- return -EINVAL;
- }
-
if (vxlan->default_dst.remote_ip.sa.sa_family != ip.sa.sa_family)
return -EAFNOSUPPORT;
@@ -2889,6 +2878,9 @@ static void vxlan_cleanup(struct timer_list *t)
if (f->flags & NTF_EXT_LEARNED)
continue;
+ if (rcu_access_pointer(f->nh))
+ continue;
+
timeout = READ_ONCE(f->updated) + vxlan->cfg.age_interval * HZ;
if (time_before_eq(timeout, jiffies)) {
spin_lock(&vxlan->hash_lock);
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
index 1fc9ecd86d5d..1ca879ffb602 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
@@ -6481,6 +6481,7 @@ brcmf_bss_connect_done(struct brcmf_cfg80211_info *cfg,
struct brcmf_cfg80211_profile *profile = &ifp->vif->profile;
struct brcmf_cfg80211_connect_info *conn_info = cfg_to_conn(cfg);
struct cfg80211_connect_resp_params conn_params;
+ bool authorized;
brcmf_dbg(TRACE, "Enter\n");
@@ -6505,7 +6506,13 @@ brcmf_bss_connect_done(struct brcmf_cfg80211_info *cfg,
conn_params.req_ie_len = conn_info->req_ie_len;
conn_params.resp_ie = conn_info->resp_ie;
conn_params.resp_ie_len = conn_info->resp_ie_len;
+ authorized = completed &&
+ ((profile->use_fwsup == BRCMF_PROFILE_FWSUP_SAE) ||
+ (profile->use_fwsup == BRCMF_PROFILE_FWSUP_PSK));
cfg80211_connect_done(ndev, &conn_params, GFP_KERNEL);
+ if (authorized)
+ cfg80211_port_authorized(ndev, profile->bssid, NULL, 0,
+ GFP_KERNEL);
brcmf_dbg(CONN, "Report connect result - connection %s\n",
completed ? "succeeded" : "failed");
}
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c
index f850e64108c4..24f083f42bb7 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c
@@ -1639,7 +1639,7 @@ int brcmf_proto_msgbuf_attach(struct brcmf_pub *drvr)
if (!msgbuf->tx_pktids)
goto fail;
msgbuf->rx_pktids = brcmf_msgbuf_init_pktids(NR_RX_PKTIDS,
- DMA_FROM_DEVICE);
+ DMA_BIDIRECTIONAL);
if (!msgbuf->rx_pktids)
goto fail;
diff --git a/drivers/net/wireless/mediatek/mt76/mac80211.c b/drivers/net/wireless/mediatek/mt76/mac80211.c
index 2d8cd627ef4c..53c9cd0cfadb 100644
--- a/drivers/net/wireless/mediatek/mt76/mac80211.c
+++ b/drivers/net/wireless/mediatek/mt76/mac80211.c
@@ -1034,6 +1034,20 @@ void mt76_update_survey(struct mt76_phy *phy)
}
EXPORT_SYMBOL_GPL(mt76_update_survey);
+void mt76_phy_chandef_set(struct mt76_phy *phy,
+ struct cfg80211_chan_def *chandef)
+{
+ if (!chandef->chan)
+ return;
+
+ mt76_update_survey(phy);
+
+ phy->chandef = *chandef;
+ phy->main_chandef = *chandef;
+ phy->chan_state = mt76_channel_state(phy, chandef->chan);
+}
+EXPORT_SYMBOL_GPL(mt76_phy_chandef_set);
+
int __mt76_set_channel(struct mt76_phy *phy, struct cfg80211_chan_def *chandef,
bool offchannel)
{
diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h
index 6ca5f3b2c6ba..9542c4fbdee2 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76.h
@@ -1531,6 +1531,8 @@ void mt76_sta_ps_transition(struct mt76_dev *dev, struct mt76_wcid *wcid,
bool mt76_has_tx_pending(struct mt76_phy *phy);
int mt76_update_channel(struct mt76_phy *phy);
void mt76_update_survey(struct mt76_phy *phy);
+void mt76_phy_chandef_set(struct mt76_phy *phy,
+ struct cfg80211_chan_def *chandef);
void mt76_update_survey_active_time(struct mt76_phy *phy, ktime_t time);
int mt76_get_survey(struct ieee80211_hw *hw, int idx,
struct survey_info *survey);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
index 0d48dbb2354f..2af061a987f2 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
@@ -826,11 +826,14 @@ void mt7921_usb_sdio_tx_complete_skb(struct mt76_dev *mdev,
u16 idx;
idx = le32_get_bits(txwi[1], MT_TXD1_WLAN_IDX);
+
+ rcu_read_lock();
wcid = __mt76_wcid_ptr(mdev, idx);
sta = wcid_to_sta(wcid);
if (sta && likely(e->skb->protocol != cpu_to_be16(ETH_P_PAE)))
mt76_connac2_tx_check_aggr(sta, txwi);
+ rcu_read_unlock();
skb_pull(e->skb, headroom);
mt76_tx_complete_skb(mdev, e->wcid, e->skb);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/main.c b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
index 35ece0f61733..26595de275bf 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
@@ -1405,10 +1405,13 @@ mt7921_change_chanctx(struct ieee80211_hw *hw,
vif = container_of((void *)mvif, struct ieee80211_vif, drv_priv);
mt792x_mutex_acquire(phy->dev);
- if (vif->type == NL80211_IFTYPE_MONITOR)
+ if (vif->type == NL80211_IFTYPE_MONITOR) {
+ mt7921_mcu_set_sniffer(mvif->phy->dev, vif, true);
mt7921_mcu_config_sniffer(mvif, ctx);
- else
+ } else {
mt76_connac_mcu_uni_set_chctx(mvif->phy->mt76, &mvif->bss_conf.mt76, ctx);
+ }
+ mt76_phy_chandef_set(mvif->phy->mt76, &ctx->def);
mt792x_mutex_release(phy->dev);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/mac.c b/drivers/net/wireless/mediatek/mt76/mt7925/mac.c
index ce81ac1c2c63..4e41ef92d623 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/mac.c
@@ -1476,11 +1476,14 @@ void mt7925_usb_sdio_tx_complete_skb(struct mt76_dev *mdev,
u16 idx;
idx = le32_get_bits(txwi[1], MT_TXD1_WLAN_IDX);
+
+ rcu_read_lock();
wcid = __mt76_wcid_ptr(mdev, idx);
sta = wcid_to_sta(wcid);
if (sta && likely(e->skb->protocol != cpu_to_be16(ETH_P_PAE)))
mt7925_tx_check_aggr(sta, e->skb, wcid);
+ rcu_read_unlock();
skb_pull(e->skb, headroom);
mt76_tx_complete_skb(mdev, e->wcid, e->skb);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/main.c b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
index 7c63c7f5caf3..70068c9eb8ea 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
@@ -1864,6 +1864,7 @@ mt7925_change_chanctx(struct ieee80211_hw *hw,
link_conf, ctx);
}
}
+ mt76_phy_chandef_set(mvif->phy->mt76, &ctx->def);
mt792x_mutex_release(phy->dev);
}
@@ -2176,6 +2177,7 @@ static int mt7925_assign_vif_chanctx(struct ieee80211_hw *hw,
mconf->mt76.ctx = ctx;
mctx->bss_conf = mconf;
+ mt76_phy_chandef_set(mvif->phy->mt76, &ctx->def);
mutex_unlock(&dev->mt76.mutex);
return 0;
diff --git a/drivers/net/wireless/mediatek/mt76/mt792x_core.c b/drivers/net/wireless/mediatek/mt76/mt792x_core.c
index 96c50a501c1a..62c40450472a 100644
--- a/drivers/net/wireless/mediatek/mt76/mt792x_core.c
+++ b/drivers/net/wireless/mediatek/mt76/mt792x_core.c
@@ -342,6 +342,7 @@ int mt792x_assign_vif_chanctx(struct ieee80211_hw *hw,
mutex_lock(&dev->mt76.mutex);
mvif->bss_conf.mt76.ctx = ctx;
mctx->bss_conf = &mvif->bss_conf;
+ mt76_phy_chandef_set(&dev->mphy, &ctx->def);
mutex_unlock(&dev->mt76.mutex);
return 0;
@@ -704,7 +705,7 @@ int mt792x_init_wiphy(struct ieee80211_hw *hw)
}
EXPORT_SYMBOL_GPL(mt792x_init_wiphy);
-static u8
+static int
mt792x_get_offload_capability(struct device *dev, const char *fw_wm)
{
const struct mt76_connac2_fw_trailer *hdr;
@@ -766,13 +767,20 @@ mt792x_get_mac80211_ops(struct device *dev,
void *drv_data, u8 *fw_features)
{
struct ieee80211_ops *ops;
+ int ret;
ops = devm_kmemdup(dev, mac80211_ops, sizeof(struct ieee80211_ops),
GFP_KERNEL);
if (!ops)
return NULL;
- *fw_features = mt792x_get_offload_capability(dev, drv_data);
+ ret = mt792x_get_offload_capability(dev, drv_data);
+ if (ret < 0) {
+ dev_warn(dev, "failed to read firmware offload caps: %d\n", ret);
+ *fw_features = 0;
+ } else {
+ *fw_features = ret;
+ }
if (!(*fw_features & MT792x_FW_CAP_CNM)) {
ops->remain_on_channel = NULL;
ops->cancel_remain_on_channel = NULL;
diff --git a/drivers/net/wireless/realtek/rtw88/mac80211.c b/drivers/net/wireless/realtek/rtw88/mac80211.c
index 3b6302f14de0..a832720f441b 100644
--- a/drivers/net/wireless/realtek/rtw88/mac80211.c
+++ b/drivers/net/wireless/realtek/rtw88/mac80211.c
@@ -468,6 +468,7 @@ static int rtw_ops_start_ap(struct ieee80211_hw *hw,
mutex_lock(&rtwdev->mutex);
rtw_write32_set(rtwdev, REG_TCR, BIT_TCR_UPDATE_HGQMD);
+ rtw_write16(rtwdev, REG_ATIMWND, ATIMWND_AP);
rtwdev->ap_active = true;
rtw_store_op_chan(rtwdev, true);
chip->ops->phy_calibration(rtwdev);
@@ -484,6 +485,7 @@ static void rtw_ops_stop_ap(struct ieee80211_hw *hw,
mutex_lock(&rtwdev->mutex);
rtw_write32_clr(rtwdev, REG_TCR, BIT_TCR_UPDATE_HGQMD);
+ rtw_write16(rtwdev, REG_ATIMWND, ATIMWND_DEFAULT);
rtwdev->ap_active = false;
if (!rtw_core_check_sta_active(rtwdev))
rtw_clear_op_chan(rtwdev);
diff --git a/drivers/net/wireless/realtek/rtw88/reg.h b/drivers/net/wireless/realtek/rtw88/reg.h
index 08e9494977e0..0b9ba6f37680 100644
--- a/drivers/net/wireless/realtek/rtw88/reg.h
+++ b/drivers/net/wireless/realtek/rtw88/reg.h
@@ -483,6 +483,8 @@
#define REG_DRVERLYINT 0x0558
#define REG_BCNDMATIM 0x0559
#define REG_ATIMWND 0x055A
+#define ATIMWND_DEFAULT 0x2
+#define ATIMWND_AP 0xa
#define REG_USTIME_TSF 0x055C
#define REG_BCN_MAX_ERR 0x055D
#define REG_RXTSF_OFFSET_CCK 0x055E
diff --git a/drivers/net/wireless/realtek/rtw88/tx.c b/drivers/net/wireless/realtek/rtw88/tx.c
index e73c06b268cf..b46f4fc8384c 100644
--- a/drivers/net/wireless/realtek/rtw88/tx.c
+++ b/drivers/net/wireless/realtek/rtw88/tx.c
@@ -196,7 +196,7 @@ void rtw_tx_report_purge_timer(struct timer_list *t)
void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn)
{
struct rtw_tx_report *tx_report = &rtwdev->tx_report;
- unsigned long timeout = RTW_TX_PROBE_TIMEOUT;
+ unsigned long timeout;
unsigned long flags;
u8 *drv_data;
@@ -208,9 +208,10 @@ void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn)
__skb_queue_tail(&tx_report->queue, skb);
spin_unlock_irqrestore(&tx_report->q_lock, flags);
- if (rtwdev->chip->id == RTW_CHIP_TYPE_8723D &&
- rtwdev->hci.type == RTW_HCI_TYPE_USB)
- timeout = msecs_to_jiffies(2500);
+ if (rtwdev->hci.type == RTW_HCI_TYPE_PCIE)
+ timeout = RTW_TX_PROBE_TIMEOUT_SHORT;
+ else
+ timeout = RTW_TX_PROBE_TIMEOUT_LONG;
mod_timer(&tx_report->purge_timer, jiffies + timeout);
}
diff --git a/drivers/net/wireless/realtek/rtw88/tx.h b/drivers/net/wireless/realtek/rtw88/tx.h
index d34cdeca16f1..168f48e87853 100644
--- a/drivers/net/wireless/realtek/rtw88/tx.h
+++ b/drivers/net/wireless/realtek/rtw88/tx.h
@@ -7,7 +7,8 @@
#define RTK_TX_MAX_AGG_NUM_MASK 0x1f
-#define RTW_TX_PROBE_TIMEOUT msecs_to_jiffies(500)
+#define RTW_TX_PROBE_TIMEOUT_SHORT msecs_to_jiffies(500)
+#define RTW_TX_PROBE_TIMEOUT_LONG msecs_to_jiffies(2500)
struct rtw_tx_desc {
__le32 w0;
diff --git a/drivers/net/wireless/realtek/rtw88/usb.c b/drivers/net/wireless/realtek/rtw88/usb.c
index c7f723c08433..caba2e2cedad 100644
--- a/drivers/net/wireless/realtek/rtw88/usb.c
+++ b/drivers/net/wireless/realtek/rtw88/usb.c
@@ -2,9 +2,11 @@
/* Copyright(c) 2018-2019 Realtek Corporation
*/
+#include <linux/ip.h>
#include <linux/module.h>
-#include <linux/usb.h>
#include <linux/mutex.h>
+#include <linux/udp.h>
+#include <linux/usb.h>
#include "main.h"
#include "debug.h"
#include "mac.h"
@@ -557,9 +559,61 @@ static int rtw_usb_write_data_h2c(struct rtw_dev *rtwdev, u8 *buf, u32 size)
return rtw_usb_write_data(rtwdev, &pkt_info, buf);
}
-static u8 rtw_usb_tx_queue_mapping_to_qsel(struct sk_buff *skb)
+static bool rtw_usb_bmc_needs_dtim(struct sk_buff *skb)
+{
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
+ struct udphdr *udphdr;
+
+ if (info->control.flags & IEEE80211_TX_CTRL_PORT_CTRL_PROTO)
+ return true;
+
+ if (skb->protocol == htons(ETH_P_ARP))
+ return true;
+
+ if (skb->protocol == htons(ETH_P_IP) &&
+ ip_hdr(skb)->protocol == IPPROTO_UDP) {
+ udphdr = udp_hdr(skb);
+
+ return udphdr->dest == htons(67) || udphdr->dest == htons(68);
+ }
+
+ return false;
+}
+
+#define RTW_USB_HIQ_REFILL_INTERVAL (HZ / 10) /* one unit of budget per 100 ms */
+#define RTW_USB_HIQ_BUDGET_MAX 16
+
+static bool rtw_usb_hiq_take_budget(struct rtw_usb *rtwusb)
+{
+ unsigned long flags, elapsed, add;
+ bool ok;
+
+ spin_lock_irqsave(&rtwusb->hiq_lock, flags);
+
+ elapsed = jiffies - rtwusb->hiq_refill;
+ add = elapsed / RTW_USB_HIQ_REFILL_INTERVAL;
+ if (add) {
+ rtwusb->hiq_budget = min_t(unsigned long, rtwusb->hiq_budget + add,
+ RTW_USB_HIQ_BUDGET_MAX);
+
+ /* keep the unfinished part of the interval for the next unit */
+ rtwusb->hiq_refill = jiffies - elapsed % RTW_USB_HIQ_REFILL_INTERVAL;
+ }
+
+ ok = rtwusb->hiq_budget > 0;
+ if (ok)
+ rtwusb->hiq_budget--;
+
+ spin_unlock_irqrestore(&rtwusb->hiq_lock, flags);
+
+ return ok;
+}
+
+static u8 rtw_usb_tx_queue_mapping_to_qsel(struct rtw_usb *rtwusb,
+ struct sk_buff *skb)
{
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
__le16 fc = hdr->frame_control;
u8 qsel;
@@ -567,7 +621,9 @@ static u8 rtw_usb_tx_queue_mapping_to_qsel(struct sk_buff *skb)
qsel = TX_DESC_QSEL_MGMT;
else if (is_broadcast_ether_addr(hdr->addr1) ||
is_multicast_ether_addr(hdr->addr1))
- qsel = TX_DESC_QSEL_HIGH;
+ qsel = (info->flags & IEEE80211_TX_CTL_SEND_AFTER_DTIM) &&
+ rtw_usb_bmc_needs_dtim(skb) && rtw_usb_hiq_take_budget(rtwusb) ?
+ TX_DESC_QSEL_HIGH : skb->priority;
else if (skb_get_queue_mapping(skb) <= IEEE80211_AC_BK)
qsel = skb->priority;
else
@@ -586,7 +642,7 @@ static int rtw_usb_tx_write(struct rtw_dev *rtwdev,
u8 *pkt_desc;
int ep;
- pkt_info->qsel = rtw_usb_tx_queue_mapping_to_qsel(skb);
+ pkt_info->qsel = rtw_usb_tx_queue_mapping_to_qsel(rtwusb, skb);
pkt_desc = skb_push(skb, chip->tx_pkt_desc_sz);
memset(pkt_desc, 0, chip->tx_pkt_desc_sz);
ep = qsel_to_ep(rtwusb, pkt_info->qsel);
@@ -1023,6 +1079,10 @@ static int rtw_usb_init_tx(struct rtw_dev *rtwdev)
struct rtw_usb *rtwusb = rtw_get_usb_priv(rtwdev);
int i;
+ spin_lock_init(&rtwusb->hiq_lock);
+ rtwusb->hiq_budget = RTW_USB_HIQ_BUDGET_MAX;
+ rtwusb->hiq_refill = jiffies;
+
rtwusb->txwq = create_singlethread_workqueue("rtw88_usb: tx wq");
if (!rtwusb->txwq) {
rtw_err(rtwdev, "failed to create TX work queue\n");
diff --git a/drivers/net/wireless/realtek/rtw88/usb.h b/drivers/net/wireless/realtek/rtw88/usb.h
index 9b695b688b24..94ff21dd0846 100644
--- a/drivers/net/wireless/realtek/rtw88/usb.h
+++ b/drivers/net/wireless/realtek/rtw88/usb.h
@@ -75,6 +75,11 @@ struct rtw_usb {
u8 out_ep[RTW_USB_EP_MAX];
int qsel_to_ep[TX_DESC_QSEL_MAX];
+ /* protects hiq_budget and hiq_refill */
+ spinlock_t hiq_lock;
+ u32 hiq_budget;
+ unsigned long hiq_refill;
+
struct workqueue_struct *txwq, *rxwq;
struct sk_buff_head tx_queue[RTW_USB_EP_MAX];
diff --git a/drivers/net/wireless/rsi/rsi_91x_core.c b/drivers/net/wireless/rsi/rsi_91x_core.c
index ead4d4e04328..3281fc94e107 100644
--- a/drivers/net/wireless/rsi/rsi_91x_core.c
+++ b/drivers/net/wireless/rsi/rsi_91x_core.c
@@ -458,7 +458,7 @@ void rsi_core_xmit(struct rsi_common *common, struct sk_buff *skb)
tx_params->sta_id = 0;
}
- if (rsta) {
+ if (rsta && tid < IEEE80211_NUM_TIDS) {
/* Start aggregation if not done for this tid */
if (!rsta->start_tx_aggr[tid]) {
rsta->start_tx_aggr[tid] = true;
diff --git a/drivers/nvme/common/keyring.c b/drivers/nvme/common/keyring.c
index 32d16c53133b..87cb51e1f5a7 100644
--- a/drivers/nvme/common/keyring.c
+++ b/drivers/nvme/common/keyring.c
@@ -35,6 +35,7 @@ struct key *nvme_tls_key_lookup(key_serial_t key_id)
}
if (nvme_tls_psk_revoked(key)) {
pr_err("key id %08x revoked\n", key_id);
+ key_put(key);
return ERR_PTR(-EKEYREVOKED);
}
return key;
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 99a8e2953b02..1ecf26f24b7f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -4628,9 +4628,8 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING_NOIO);
mutex_lock(&ctrl->namespaces_lock);
- list_splice_init_rcu(&ctrl->namespaces, &ns_list, synchronize_rcu);
+ list_splice_init_srcu(&ctrl->namespaces, &ns_list, &ctrl->srcu);
mutex_unlock(&ctrl->namespaces_lock);
- synchronize_srcu(&ctrl->srcu);
list_for_each_entry_safe(ns, next, &ns_list, list)
nvme_ns_remove(ns);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 6e4261b0a082..9245fae71321 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3939,6 +3939,8 @@ static const struct pci_device_id nvme_id_table[] = {
.driver_data = NVME_QUIRK_BOGUS_NID, },
{ PCI_DEVICE(0x1344, 0x6004), /* Micron 4100AT */
.driver_data = NVME_QUIRK_DMAPOOL_ALIGN_512, },
+ { PCI_DEVICE(0x1e0f, 0x0032), /* KIOXIA XG10d */
+ .driver_data = NVME_QUIRK_DMAPOOL_ALIGN_512, },
{ PCI_DEVICE(0x1c5c, 0x1504), /* SK Hynix PC400 */
.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
{ PCI_DEVICE(0x1c5c, 0x174a), /* SK Hynix P31 SSD */
diff --git a/drivers/nvme/host/pr.c b/drivers/nvme/host/pr.c
index fe7dbe264815..d8b6ac4d83bb 100644
--- a/drivers/nvme/host/pr.c
+++ b/drivers/nvme/host/pr.c
@@ -128,8 +128,12 @@ static int nvme_pr_register(struct block_device *bdev, u64 old_key, u64 new_key,
data.crkey = cpu_to_le64(old_key);
data.nrkey = cpu_to_le64(new_key);
- cdw10 = old_key ? NVME_PR_REGISTER_ACT_REPLACE :
- NVME_PR_REGISTER_ACT_REG;
+ if (!new_key)
+ cdw10 = NVME_PR_REGISTER_ACT_UNREG;
+ else if (old_key)
+ cdw10 = NVME_PR_REGISTER_ACT_REPLACE;
+ else
+ cdw10 = NVME_PR_REGISTER_ACT_REG;
cdw10 |= (flags & PR_FL_IGNORE_KEY) ? NVME_PR_IGNORE_KEY : 0;
cdw10 |= NVME_PR_CPTPL_PERSIST;
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 94495b022871..fad2f2f969a0 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -6,6 +6,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/jiffies.h>
#include <linux/slab.h>
#include <linux/err.h>
#include <linux/crc32.h>
@@ -46,13 +47,34 @@ MODULE_PARM_DESC(wq_unbound, "Use unbound workqueue for nvme-tcp IO context (def
/*
* TLS handshake timeout
*/
-static int tls_handshake_timeout = 10;
+static unsigned int tls_handshake_timeout = 10;
#ifdef CONFIG_NVME_TCP_TLS
-module_param(tls_handshake_timeout, int, 0644);
+static int tls_handshake_timeout_set(const char *val,
+ const struct kernel_param *kp)
+{
+ return param_set_uint_minmax(val, kp, 0,
+ min_t(unsigned long, UINT_MAX / MSEC_PER_SEC,
+ MAX_JIFFY_OFFSET / HZ));
+}
+
+static const struct kernel_param_ops tls_handshake_timeout_ops = {
+ .set = tls_handshake_timeout_set,
+ .get = param_get_uint,
+};
+
+module_param_cb(tls_handshake_timeout, &tls_handshake_timeout_ops,
+ &tls_handshake_timeout, 0644);
+__MODULE_PARM_TYPE(tls_handshake_timeout, "uint");
MODULE_PARM_DESC(tls_handshake_timeout,
"nvme TLS handshake timeout in seconds (default 10)");
#endif
+/* read a TLS control record in chunks of this size */
+#define NVME_TCP_TLS_REC_CHUNK 256
+
+/* TLS records read while waiting for the ICResp, the ICResp included */
+#define NVME_TCP_ICRESP_MAX_RECORDS 8
+
static atomic_t nvme_tcp_cpu_queues[NR_CPUS];
#ifdef CONFIG_DEBUG_LOCK_ALLOC
@@ -1535,12 +1557,143 @@ static void nvme_tcp_free_queue(struct nvme_ctrl *nctrl, int qid)
mutex_destroy(&queue->pf_cache_lock);
}
+static int nvme_tcp_recv_tls_alert(struct nvme_tcp_queue *queue,
+ struct msghdr *msg, int len)
+{
+ struct sock *sk = queue->sock->sk;
+ u8 level, description;
+
+ if (len < 2) {
+ dev_err(queue->ctrl->ctrl.device,
+ "queue %d: truncated TLS alert\n",
+ nvme_tcp_queue_id(queue));
+ return -ENOTCONN;
+ }
+
+ /* tls_alert_recv() reads through the iterator */
+ iov_iter_revert(&msg->msg_iter, len);
+ tls_alert_recv(sk, msg, &level, &description);
+
+ if (level == TLS_ALERT_LEVEL_FATAL) {
+ dev_err(queue->ctrl->ctrl.device,
+ "queue %d: fatal TLS alert %u\n",
+ nvme_tcp_queue_id(queue), description);
+ return -ENOTCONN;
+ }
+
+ /* a peer shutting down normally is not worth a warning */
+ if (description == TLS_ALERT_DESC_CLOSE_NOTIFY)
+ dev_dbg(queue->ctrl->ctrl.device,
+ "queue %d: TLS close_notify\n",
+ nvme_tcp_queue_id(queue));
+ else
+ dev_warn(queue->ctrl->ctrl.device,
+ "queue %d: TLS alert %u\n",
+ nvme_tcp_queue_id(queue), description);
+
+ return 0;
+}
+
+/*
+ * Discard a TLS record that arrived in place of the ICResp. A peer may send
+ * a NewSessionTicket as soon as the handshake completes.
+ *
+ * Returns 0 when the record was discarded and the ICResp can be read, or a
+ * negative errno if the connection cannot continue.
+ */
+static int nvme_tcp_skip_tls_record(struct nvme_tcp_queue *queue,
+ struct msghdr *msg, u8 ctype, int len)
+{
+ u8 buf[NVME_TCP_TLS_REC_CHUNK];
+ int ret;
+
+ switch (ctype) {
+ case TLS_RECORD_TYPE_HANDSHAKE:
+ break;
+ case TLS_RECORD_TYPE_ALERT:
+ ret = nvme_tcp_recv_tls_alert(queue, msg, len);
+ return ret ? : -ENOTCONN;
+ default:
+ pr_err("queue %d: unhandled TLS record %u\n",
+ nvme_tcp_queue_id(queue), ctype);
+ return -ENOTCONN;
+ }
+
+ /* the ICResp is in a later record, drain this one first */
+ while (!(msg->msg_flags & MSG_EOR)) {
+ struct kvec iov = {
+ .iov_base = buf,
+ .iov_len = sizeof(buf),
+ };
+ char cbuf[CMSG_LEN(sizeof(char))] = {};
+ /* kTLS fails a control record read without a cmsg buffer */
+ struct msghdr drain = {
+ .msg_control = cbuf,
+ .msg_controllen = sizeof(cbuf),
+ };
+
+ ret = kernel_recvmsg(queue->sock, &drain, &iov, 1, sizeof(buf),
+ 0);
+ if (ret <= 0) {
+ pr_warn("queue %d: failed to drain TLS record: %d\n",
+ nvme_tcp_queue_id(queue), ret);
+ return ret ? : -ECONNRESET;
+ }
+ msg->msg_flags = drain.msg_flags;
+ }
+
+ return 0;
+}
+
+/*
+ * Receive the ICResp. Over TLS, skip the records that a peer may send
+ * ahead of it.
+ *
+ * Returns the length received, or a negative errno.
+ */
+static int nvme_tcp_recv_icresp(struct nvme_tcp_queue *queue,
+ struct nvme_tcp_icresp_pdu *icresp)
+{
+ char cbuf[CMSG_LEN(sizeof(char))];
+ struct msghdr msg;
+ struct kvec iov;
+ u8 ctype;
+ int i, ret;
+
+ for (i = 0; i < NVME_TCP_ICRESP_MAX_RECORDS; i++) {
+ memset(&msg, 0, sizeof(msg));
+ memset(cbuf, 0, sizeof(cbuf));
+ iov.iov_base = icresp;
+ iov.iov_len = sizeof(*icresp);
+ if (nvme_tcp_queue_tls(queue)) {
+ msg.msg_control = cbuf;
+ msg.msg_controllen = sizeof(cbuf);
+ }
+ msg.msg_flags = MSG_WAITALL;
+ ret = kernel_recvmsg(queue->sock, &msg, &iov, 1,
+ iov.iov_len, msg.msg_flags);
+ if (ret <= 0 || !nvme_tcp_queue_tls(queue))
+ return ret;
+
+ ctype = tls_get_record_type(queue->sock->sk,
+ (struct cmsghdr *)cbuf);
+ if (ctype == TLS_RECORD_TYPE_DATA)
+ return ret;
+
+ ret = nvme_tcp_skip_tls_record(queue, &msg, ctype, ret);
+ if (ret < 0)
+ return ret;
+ }
+
+ pr_err("queue %d: too many TLS records before the ICResp\n",
+ nvme_tcp_queue_id(queue));
+ return -EPROTO;
+}
+
static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue)
{
struct nvme_tcp_icreq_pdu *icreq;
struct nvme_tcp_icresp_pdu *icresp;
- char cbuf[CMSG_LEN(sizeof(char))] = {};
- u8 ctype;
struct msghdr msg = {};
struct kvec iov;
bool ctrl_hdgst, ctrl_ddgst;
@@ -1578,16 +1731,7 @@ static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue)
goto free_icresp;
}
- memset(&msg, 0, sizeof(msg));
- iov.iov_base = icresp;
- iov.iov_len = sizeof(*icresp);
- if (nvme_tcp_queue_tls(queue)) {
- msg.msg_control = cbuf;
- msg.msg_controllen = sizeof(cbuf);
- }
- msg.msg_flags = MSG_WAITALL;
- ret = kernel_recvmsg(queue->sock, &msg, &iov, 1,
- iov.iov_len, msg.msg_flags);
+ ret = nvme_tcp_recv_icresp(queue, icresp);
if (ret >= 0 && ret < sizeof(*icresp))
ret = -ECONNRESET;
if (ret < 0) {
@@ -1595,16 +1739,6 @@ static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue)
nvme_tcp_queue_id(queue), ret);
goto free_icresp;
}
- ret = -ENOTCONN;
- if (nvme_tcp_queue_tls(queue)) {
- ctype = tls_get_record_type(queue->sock->sk,
- (struct cmsghdr *)cbuf);
- if (ctype != TLS_RECORD_TYPE_DATA) {
- pr_err("queue %d: unhandled TLS record %d\n",
- nvme_tcp_queue_id(queue), ctype);
- goto free_icresp;
- }
- }
ret = -EINVAL;
if (icresp->hdr.type != nvme_tcp_icresp) {
pr_err("queue %d: bad type returned %d\n",
@@ -1794,8 +1928,10 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl,
{
int qid = nvme_tcp_queue_id(queue);
int ret;
+ long wait_ret;
struct tls_handshake_args args;
- unsigned long tmo = tls_handshake_timeout * HZ;
+ unsigned int timeout = READ_ONCE(tls_handshake_timeout);
+ unsigned long tmo = (unsigned long)timeout * HZ;
key_serial_t keyring = nvme_keyring_id();
dev_dbg(nctrl->device, "queue %d: start TLS with key %x\n",
@@ -1809,7 +1945,7 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl,
if (nctrl->opts->keyring)
keyring = key_serial(nctrl->opts->keyring);
args.ta_keyring = keyring;
- args.ta_timeout_ms = tls_handshake_timeout * 1000;
+ args.ta_timeout_ms = timeout * MSEC_PER_SEC;
queue->tls_err = -EOPNOTSUPP;
init_completion(&queue->tls_complete);
ret = tls_client_hello_psk(&args, GFP_KERNEL);
@@ -1818,10 +1954,10 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl,
qid, ret);
return ret;
}
- ret = wait_for_completion_interruptible_timeout(&queue->tls_complete, tmo);
- if (ret <= 0) {
- if (ret == 0)
- ret = -ETIMEDOUT;
+ wait_ret = wait_for_completion_interruptible_timeout(&queue->tls_complete,
+ tmo);
+ if (wait_ret <= 0) {
+ ret = wait_ret < 0 ? -ERESTARTSYS : -ETIMEDOUT;
dev_err(nctrl->device,
"queue %d: TLS handshake failed, error %d\n",
diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 623b5303de2a..007029eaca39 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -543,6 +543,15 @@ static void nvmet_execute_get_log_page_ana(struct nvmet_req *req)
u16 ngrps = 0;
u16 status;
+ /*
+ * The header and one descriptor per enabled group are the only bytes
+ * written below; zero the whole transfer first so the unwritten tail
+ * does not leak stale kernel memory to the host.
+ */
+ status = nvmet_zero_sgl(req, 0, req->transfer_len);
+ if (status)
+ goto out;
+
status = NVME_SC_INTERNAL;
desc = kmalloc(struct_size(desc, nsids, NVMET_MAX_NAMESPACES),
GFP_KERNEL);
@@ -1228,8 +1237,13 @@ u16 nvmet_set_feat_kato(struct nvmet_req *req)
{
u32 val32 = le32_to_cpu(req->cmd->common.cdw11);
+ if (val32 > NVMET_MAX_KATO * MSEC_PER_SEC) {
+ req->error_loc = offsetof(struct nvme_common_command, cdw11);
+ return NVME_SC_KA_TIMEOUT_INVALID | NVME_STATUS_DNR;
+ }
+
nvmet_stop_keep_alive_timer(req->sq->ctrl);
- req->sq->ctrl->kato = DIV_ROUND_UP(val32, 1000);
+ req->sq->ctrl->kato = DIV_ROUND_UP_ULL(val32, MSEC_PER_SEC);
nvmet_start_keep_alive_timer(req->sq->ctrl);
nvmet_set_result(req, req->sq->ctrl->kato);
diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index e2032b58191c..61b4e90f4e34 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -150,7 +150,7 @@ static ssize_t nvmet_addr_traddr_store(struct config_item *item,
if (nvmet_is_port_enabled(port, __func__))
return -EACCES;
- if (sscanf(page, "%s\n", port->disc_addr.traddr) != 1)
+ if (sscanf(page, "%255s", port->disc_addr.traddr) != 1)
return -EINVAL;
return count;
}
@@ -242,7 +242,7 @@ static ssize_t nvmet_addr_trsvcid_store(struct config_item *item,
if (nvmet_is_port_enabled(port, __func__))
return -EACCES;
- if (sscanf(page, "%s\n", port->disc_addr.trsvcid) != 1)
+ if (sscanf(page, "%31s", port->disc_addr.trsvcid) != 1)
return -EINVAL;
return count;
}
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 9b64c313c703..36b3dab59e21 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1588,6 +1588,12 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
u8 dhchap_status;
int ret;
+ if (kato > NVMET_MAX_KATO * MSEC_PER_SEC) {
+ args->status = NVME_SC_KA_TIMEOUT_INVALID | NVME_STATUS_DNR;
+ args->error_loc = offsetof(struct nvmf_connect_command, kato);
+ return NULL;
+ }
+
args->status = NVME_SC_CONNECT_INVALID_PARAM | NVME_STATUS_DNR;
subsys = nvmet_find_get_subsys(args->port, args->subsysnqn);
if (!subsys) {
@@ -1655,7 +1661,7 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
kato = NVMET_DISC_KATO_MS;
/* keep-alive timeout in seconds */
- ctrl->kato = DIV_ROUND_UP(kato, 1000);
+ ctrl->kato = DIV_ROUND_UP_ULL(kato, MSEC_PER_SEC);
ctrl->err_counter = 0;
spin_lock_init(&ctrl->error_lock);
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 45fc188412d2..6a0d9b37443e 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -7,6 +7,7 @@
#define _NVMET_H
#include <linux/dma-mapping.h>
+#include <linux/jiffies.h>
#include <linux/types.h>
#include <linux/device.h>
#include <linux/kref.h>
@@ -694,6 +695,13 @@ void nvmet_add_async_event(struct nvmet_ctrl *ctrl, u8 event_type,
#define NVMET_KAS 10
#define NVMET_DISC_KATO_MS 120000
+/*
+ * Largest keep-alive timeout in seconds that can be reported back in
+ * milliseconds and converted to jiffies without overflowing.
+ */
+#define NVMET_MAX_KATO \
+ min3(U32_MAX / MSEC_PER_SEC, U32_MAX / HZ, MAX_JIFFY_OFFSET / HZ)
+
int __init nvmet_init_configfs(void);
void __exit nvmet_exit_configfs(void);
diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
index a0ba6641c7ff..ae44a0ff34d7 100644
--- a/drivers/nvme/target/rdma.c
+++ b/drivers/nvme/target/rdma.c
@@ -864,7 +864,8 @@ static u16 nvmet_rdma_map_sgl_inline(struct nvmet_rdma_rsp *rsp)
return NVME_SC_INVALID_FIELD | NVME_STATUS_DNR;
}
- if (off + len > rsp->queue->dev->inline_data_size) {
+ if (off > (u64)rsp->queue->dev->inline_data_size ||
+ len > (u64)rsp->queue->dev->inline_data_size - off) {
pr_err("invalid inline data offset!\n");
return NVME_SC_SGL_INVALID_OFFSET | NVME_STATUS_DNR;
}
diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index b9a904f18489..f478f226e29f 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -6,6 +6,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/jiffies.h>
#include <linux/slab.h>
#include <linux/crc32c.h>
#include <linux/err.h>
@@ -41,8 +42,18 @@ static int param_store_val(const char *str, int *val, int min, int max)
return 0;
}
+#ifdef CONFIG_NVME_TARGET_TCP_TLS
+static unsigned int tls_handshake_timeout = 10;
+#endif
+
static int set_params(const char *str, const struct kernel_param *kp)
{
+#ifdef CONFIG_NVME_TARGET_TCP_TLS
+ if (kp->arg == &tls_handshake_timeout)
+ return param_set_uint_minmax(str, kp, 0,
+ min_t(unsigned long, UINT_MAX / MSEC_PER_SEC,
+ MAX_JIFFY_OFFSET / HZ));
+#endif
return param_store_val(str, kp->arg, 0, INT_MAX);
}
@@ -76,8 +87,14 @@ MODULE_PARM_DESC(idle_poll_period_usecs,
/*
* TLS handshake timeout
*/
-static int tls_handshake_timeout = 10;
-module_param(tls_handshake_timeout, int, 0644);
+static const struct kernel_param_ops tls_handshake_timeout_ops = {
+ .set = set_params,
+ .get = param_get_uint,
+};
+
+module_param_cb(tls_handshake_timeout, &tls_handshake_timeout_ops,
+ &tls_handshake_timeout, 0644);
+__MODULE_PARM_TYPE(tls_handshake_timeout, "uint");
MODULE_PARM_DESC(tls_handshake_timeout,
"nvme TLS handshake timeout in seconds (default 10)");
#endif
@@ -1926,6 +1943,7 @@ static int nvmet_tcp_tls_handshake(struct nvmet_tcp_queue *queue)
{
int ret = -EOPNOTSUPP;
struct tls_handshake_args args;
+ unsigned int timeout = READ_ONCE(tls_handshake_timeout);
if (queue->state != NVMET_TCP_Q_TLS_HANDSHAKE) {
pr_warn("cannot start TLS in state %d\n", queue->state);
@@ -1939,7 +1957,7 @@ static int nvmet_tcp_tls_handshake(struct nvmet_tcp_queue *queue)
args.ta_done = nvmet_tcp_tls_handshake_done;
args.ta_data = queue;
args.ta_keyring = key_serial(queue->port->nport->keyring);
- args.ta_timeout_ms = tls_handshake_timeout * 1000;
+ args.ta_timeout_ms = timeout * MSEC_PER_SEC;
ret = tls_server_hello_psk(&args, GFP_KERNEL);
if (ret) {
@@ -1947,7 +1965,7 @@ static int nvmet_tcp_tls_handshake(struct nvmet_tcp_queue *queue)
pr_err("failed to start TLS, err=%d\n", ret);
} else {
queue_delayed_work(nvmet_wq, &queue->tls_handshake_tmo_work,
- tls_handshake_timeout * HZ);
+ (unsigned long)timeout * HZ);
}
return ret;
}
diff --git a/drivers/nvmem/rockchip-otp.c b/drivers/nvmem/rockchip-otp.c
index d88f12c53242..1475ce11a9c7 100644
--- a/drivers/nvmem/rockchip-otp.c
+++ b/drivers/nvmem/rockchip-otp.c
@@ -12,6 +12,7 @@
#include <linux/io.h>
#include <linux/iopoll.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/nvmem-provider.h>
#include <linux/reset.h>
#include <linux/slab.h>
@@ -77,9 +78,11 @@ struct rockchip_data {
struct rockchip_otp {
struct device *dev;
void __iomem *base;
- struct clk_bulk_data *clks;
struct reset_control *rst;
const struct rockchip_data *data;
+ /* Serializes access to the OTP controller state machine */
+ struct mutex mutex;
+ struct clk_bulk_data clks[];
};
static int rockchip_otp_reset(struct rockchip_otp *otp)
@@ -239,16 +242,21 @@ static int rockchip_otp_read(void *context, unsigned int offset,
if (!otp->data || !otp->data->reg_read)
return -EINVAL;
+ mutex_lock(&otp->mutex);
+
ret = clk_bulk_prepare_enable(otp->data->num_clks, otp->clks);
if (ret < 0) {
dev_err(otp->dev, "failed to prepare/enable clks\n");
- return ret;
+ goto unlock;
}
ret = otp->data->reg_read(context, offset, val, bytes);
clk_bulk_disable_unprepare(otp->data->num_clks, otp->clks);
+unlock:
+ mutex_unlock(&otp->mutex);
+
return ret;
}
@@ -327,23 +335,23 @@ static int rockchip_otp_probe(struct platform_device *pdev)
if (!data)
return dev_err_probe(dev, -EINVAL, "failed to get match data\n");
- otp = devm_kzalloc(&pdev->dev, sizeof(struct rockchip_otp),
+ otp = devm_kzalloc(&pdev->dev, struct_size(otp, clks, data->num_clks),
GFP_KERNEL);
if (!otp)
return -ENOMEM;
otp->data = data;
otp->dev = dev;
+
+ ret = devm_mutex_init(dev, &otp->mutex);
+ if (ret)
+ return ret;
+
otp->base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(otp->base))
return dev_err_probe(dev, PTR_ERR(otp->base),
"failed to ioremap resource\n");
- otp->clks = devm_kcalloc(dev, data->num_clks, sizeof(*otp->clks),
- GFP_KERNEL);
- if (!otp->clks)
- return -ENOMEM;
-
for (i = 0; i < data->num_clks; ++i)
otp->clks[i].id = data->clks[i];
diff --git a/drivers/pci/controller/cadence/pcie-cadence-plat.c b/drivers/pci/controller/cadence/pcie-cadence-plat.c
index 0456845dabb9..c1ec619ab0a4 100644
--- a/drivers/pci/controller/cadence/pcie-cadence-plat.c
+++ b/drivers/pci/controller/cadence/pcie-cadence-plat.c
@@ -14,14 +14,6 @@
#define CDNS_PLAT_CPU_TO_BUS_ADDR 0x0FFFFFFF
-/**
- * struct cdns_plat_pcie - private data for this PCIe platform driver
- * @pcie: Cadence PCIe controller
- */
-struct cdns_plat_pcie {
- struct cdns_pcie *pcie;
-};
-
struct cdns_plat_pcie_of_data {
bool is_rc;
};
@@ -40,9 +32,9 @@ static const struct cdns_pcie_ops cdns_plat_ops = {
static int cdns_plat_pcie_probe(struct platform_device *pdev)
{
const struct cdns_plat_pcie_of_data *data;
- struct cdns_plat_pcie *cdns_plat_pcie;
struct device *dev = &pdev->dev;
struct pci_host_bridge *bridge;
+ struct cdns_pcie *pcie;
struct cdns_pcie_ep *ep;
struct cdns_pcie_rc *rc;
int phy_count;
@@ -56,11 +48,6 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
is_rc = data->is_rc;
pr_debug(" Started %s with is_rc: %d\n", __func__, is_rc);
- cdns_plat_pcie = devm_kzalloc(dev, sizeof(*cdns_plat_pcie), GFP_KERNEL);
- if (!cdns_plat_pcie)
- return -ENOMEM;
-
- platform_set_drvdata(pdev, cdns_plat_pcie);
if (is_rc) {
if (!IS_ENABLED(CONFIG_PCIE_CADENCE_PLAT_HOST))
return -ENODEV;
@@ -72,9 +59,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
rc = pci_host_bridge_priv(bridge);
rc->pcie.dev = dev;
rc->pcie.ops = &cdns_plat_ops;
- cdns_plat_pcie->pcie = &rc->pcie;
+ pcie = &rc->pcie;
- ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie);
+ ret = cdns_pcie_init_phy(dev, pcie);
if (ret) {
dev_err(dev, "failed to init phy\n");
return ret;
@@ -99,9 +86,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
ep->pcie.dev = dev;
ep->pcie.ops = &cdns_plat_ops;
- cdns_plat_pcie->pcie = &ep->pcie;
+ pcie = &ep->pcie;
- ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie);
+ ret = cdns_pcie_init_phy(dev, pcie);
if (ret) {
dev_err(dev, "failed to init phy\n");
return ret;
@@ -119,18 +106,20 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
goto err_init;
}
+ platform_set_drvdata(pdev, pcie);
+
return 0;
err_init:
err_get_sync:
pm_runtime_put_sync(dev);
pm_runtime_disable(dev);
- cdns_pcie_disable_phy(cdns_plat_pcie->pcie);
- phy_count = cdns_plat_pcie->pcie->phy_count;
+ cdns_pcie_disable_phy(pcie);
+ phy_count = pcie->phy_count;
while (phy_count--)
- device_link_del(cdns_plat_pcie->pcie->link[phy_count]);
+ device_link_del(pcie->link[phy_count]);
- return 0;
+ return ret;
}
static void cdns_plat_pcie_shutdown(struct platform_device *pdev)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index de09cc2a6311..4235f1fb7b33 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -540,7 +540,7 @@ static int imx_setup_phy_mpll(struct imx_pcie *imx_pcie)
return 0;
for (i = 0; i < imx_pcie->num_clks; i++)
- if (strncmp(clks[i].id, "pcie_phy", 8) == 0)
+ if (clks[i].id && strncmp(clks[i].id, "pcie_phy", 8) == 0)
phy_rate = clk_get_rate(clks[i].clk);
switch (phy_rate) {
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index a77d7cde4524..f2e2dbae050b 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -568,6 +568,8 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci,
/* The data-less messages only for now */
val |= PCIE_ATU_INHIBIT_PAYLOAD | atu->code;
}
+ if (dw_pcie_ver_is_ge(pci, 460A))
+ val |= PCIE_ATU_DMA_BYPASS;
dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL2, val);
/*
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index aa27211b68dd..c01634c0211f 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -173,6 +173,7 @@
#define PCIE_ATU_ENABLE BIT(31)
#define PCIE_ATU_BAR_MODE_ENABLE BIT(30)
#define PCIE_ATU_CFG_SHIFT_MODE_ENABLE BIT(28)
+#define PCIE_ATU_DMA_BYPASS BIT(27)
#define PCIE_ATU_INHIBIT_PAYLOAD BIT(22)
#define PCIE_ATU_FUNC_NUM_MATCH_EN BIT(19)
#define PCIE_ATU_LOWER_BASE 0x008
diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
index 7e171526b1de..9e5dc1009ed4 100644
--- a/drivers/pci/controller/dwc/pcie-rcar-gen4.c
+++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
@@ -305,6 +305,57 @@ static struct rcar_gen4_pcie *rcar_gen4_pcie_alloc(struct platform_device *pdev)
return rcar;
}
+static int rcar_gen4_pcie_enable_device(struct pci_host_bridge *bridge,
+ struct pci_dev *dev)
+{
+ /*
+ * R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes
+ * Max_Payload_Size (MPS). PCIe specification indicates that the MPS
+ * must not exceed minimum MPS of any element along the packet path.
+ * The controller reports Max_Payload_Size_Supported (MPSS) 256 Bytes
+ * for header type 0 and 128 Bytes for header type 1. The PCIe core
+ * will not allow MPS to be set higher than MPSS; warn here in case
+ * something went very wrong in the core.
+ *
+ * For details, refer to chapter "104.1.1 Features" in either of:
+ * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or
+ * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or
+ * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026.
+ */
+ WARN_ON(pcie_get_mps(dev) > 256);
+
+ /*
+ * R-Car Gen4 Reference Manual, chapter 104.4.8 Usage notes for
+ * MRRS (Max Read Request Size) states:
+ *
+ * Please set "Max Read Request Size" to 128 bytes or 256 bytes.
+ * If "Max Read Request Size" is set to anything other than the
+ * above, the transferred data will not match the expected value.
+ *
+ * This limitation also seems to apply to devices issuing MRd TLPs.
+ * This limitation can be triggered by using non-HMB NVMe SSD with
+ * Max_Read_Request_Size 512 Bytes, for example Crucial P5 Plus.
+ * Any write into the SSD (MRd TLP issued by the SSD) longer than
+ * 256 Bytes wraps around at 256 Byte boundary, and the same data
+ * are written into the SSD starting at offset 0 and at 256 Bytes.
+ *
+ * Limit Max_Read_Request_Size to at most 256 Bytes for each
+ * device connected to this PCIe controller to avoid this behavior.
+ *
+ * For details, refer to aforementioned chapter in either of:
+ * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or
+ * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or
+ * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026.
+ */
+ bridge->no_inc_mrrs = 1;
+ if (pcie_get_readrq(dev) > 256) {
+ pci_info(dev, "Limiting MRRS to 256 bytes\n");
+ pcie_set_readrq(dev, 256);
+ }
+
+ return 0;
+}
+
/* Host mode */
static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp)
{
@@ -313,6 +364,9 @@ static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp)
int ret;
u32 val;
+ if (pp->bridge)
+ pp->bridge->enable_device = rcar_gen4_pcie_enable_device;
+
gpiod_set_value_cansleep(dw->pe_rst, 1);
ret = rcar_gen4_pcie_common_init(rcar);
diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
index 68110855ea1c..39d9844873f5 100644
--- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
+++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
@@ -118,7 +118,7 @@ struct epf_ntb_ctrl {
} __packed;
struct epf_ntb {
- struct ntb_dev ntb;
+ struct ntb_dev *ntb;
struct pci_epf *epf;
struct config_group group;
@@ -144,10 +144,15 @@ struct epf_ntb {
void __iomem *vpci_mw_addr[MAX_MW];
struct delayed_work cmd_handler;
+ struct pci_bus *vpci_bus;
};
#define to_epf_ntb(epf_group) container_of((epf_group), struct epf_ntb, group)
-#define ntb_ndev(__ntb) container_of(__ntb, struct epf_ntb, ntb)
+
+static struct epf_ntb *ntb_ndev(struct ntb_dev *ntb)
+{
+ return ntb->pdev->sysdata;
+}
static struct pci_epf_header epf_ntb_header = {
.vendorid = PCI_ANY_ID,
@@ -173,7 +178,7 @@ static int epf_ntb_link_up(struct epf_ntb *ntb, bool link_up)
else
ntb->reg->link_status &= ~LINK_STATUS_UP;
- ntb_link_event(&ntb->ntb);
+ ntb_link_event(ntb->ntb);
return 0;
}
@@ -261,7 +266,7 @@ static void epf_ntb_cmd_handler(struct work_struct *work)
for (i = 1; i < ntb->db_count; i++) {
if (ntb->epf_db[i]) {
ntb->db |= 1 << (i - 1);
- ntb_db_event(&ntb->ntb, i);
+ ntb_db_event(ntb->ntb, i);
ntb->epf_db[i] = 0;
}
}
@@ -796,7 +801,6 @@ static int epf_ntb_epc_init(struct epf_ntb *ntb)
}
INIT_DELAYED_WORK(&ntb->cmd_handler, epf_ntb_cmd_handler);
- queue_work(kpcintb_workqueue, &ntb->cmd_handler.work);
return 0;
@@ -1100,6 +1104,7 @@ static int vpci_scan_bus(void *sysdata)
pr_err("create pci bus failed\n");
return -EINVAL;
}
+ ndev->vpci_bus = vpci_bus;
pci_bus_add_devices(vpci_bus);
@@ -1144,7 +1149,7 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx,
int ret;
struct device *dev;
- dev = &ntb->ntb.dev;
+ dev = &ndev->dev;
barno = ntb->epf_ntb_bar[BAR_MW1 + idx];
epf_bar = &ntb->epf->bar[barno];
epf_bar->phys_addr = addr;
@@ -1244,7 +1249,7 @@ static int vntb_epf_peer_db_set(struct ntb_dev *ndev, u64 db_bits)
ret = pci_epc_raise_irq(ntb->epf->epc, func_no, vfunc_no,
PCI_IRQ_MSI, interrupt_num + 1);
if (ret)
- dev_err(&ntb->ntb.dev, "Failed to raise IRQ\n");
+ dev_err(&ntb->epf->dev, "Failed to raise IRQ\n");
return ret;
}
@@ -1327,13 +1332,18 @@ static const struct ntb_dev_ops vntb_epf_ops = {
static int pci_vntb_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
- int ret;
struct epf_ntb *ndev = (struct epf_ntb *)pdev->sysdata;
struct device *dev = &pdev->dev;
+ struct ntb_dev *ntb;
+ int ret;
- ndev->ntb.pdev = pdev;
- ndev->ntb.topo = NTB_TOPO_NONE;
- ndev->ntb.ops = &vntb_epf_ops;
+ ntb = devm_kzalloc(dev, sizeof(*ntb), GFP_KERNEL);
+ if (!ntb)
+ return -ENOMEM;
+
+ ntb->pdev = pdev;
+ ntb->topo = NTB_TOPO_NONE;
+ ntb->ops = &vntb_epf_ops;
ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
if (ret) {
@@ -1341,16 +1351,33 @@ static int pci_vntb_probe(struct pci_dev *pdev, const struct pci_device_id *id)
return ret;
}
- ret = ntb_register_device(&ndev->ntb);
+ ret = ntb_register_device(ntb);
if (ret) {
dev_err(dev, "Failed to register NTB device\n");
return ret;
}
+ /* Publish after ntb_register_device() succeeds */
+ smp_store_release(&ndev->ntb, ntb);
+ queue_delayed_work(kpcintb_workqueue, &ndev->cmd_handler, 0);
+
dev_dbg(dev, "PCI Virtual NTB driver loaded\n");
return 0;
}
+static void pci_vntb_remove(struct pci_dev *pdev)
+{
+ struct epf_ntb *ndev = pdev->sysdata;
+ struct ntb_dev *ntb;
+
+ /* Stop the work reader */
+ cancel_delayed_work_sync(&ndev->cmd_handler);
+
+ ntb = xchg(&ndev->ntb, NULL);
+
+ ntb_unregister_device(ntb);
+}
+
static struct pci_device_id pci_vntb_table[] = {
{
PCI_DEVICE(0xffff, 0xffff),
@@ -1362,6 +1389,7 @@ static struct pci_driver vntb_pci_driver = {
.name = "pci-vntb",
.id_table = pci_vntb_table,
.probe = pci_vntb_probe,
+ .remove = pci_vntb_remove,
};
/* ============ PCIe EPF Driver Bind ====================*/
@@ -1442,12 +1470,25 @@ static int epf_ntb_bind(struct pci_epf *epf)
*/
static void epf_ntb_unbind(struct pci_epf *epf)
{
+ struct pci_host_bridge *bridge;
struct epf_ntb *ntb = epf_get_drvdata(epf);
+ pci_unregister_driver(&vntb_pci_driver);
+
+ if (ntb->vpci_bus) {
+ bridge = to_pci_host_bridge(ntb->vpci_bus->bridge);
+
+ pci_lock_rescan_remove();
+ pci_stop_root_bus(ntb->vpci_bus);
+ pci_remove_root_bus(ntb->vpci_bus);
+ ntb->vpci_bus = NULL;
+ pci_unlock_rescan_remove();
+
+ pci_free_host_bridge(bridge);
+ }
+
epf_ntb_epc_cleanup(ntb);
epf_ntb_config_spad_bar_free(ntb);
-
- pci_unregister_driver(&vntb_pci_driver);
}
// EPF driver probe
diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c
index ad3fea4b56f8..9ded8dacb62c 100644
--- a/drivers/pci/pcie/aer.c
+++ b/drivers/pci/pcie/aer.c
@@ -1330,14 +1330,13 @@ static void aer_recover_work_func(struct work_struct *work)
while (kfifo_get(&aer_recover_ring, &entry)) {
pdev = pci_get_domain_bus_and_slot(entry.domain, entry.bus,
entry.devfn);
- if (!pdev) {
+ if (!pdev)
pr_err_ratelimited("%04x:%02x:%02x.%x: no pci_dev found\n",
entry.domain, entry.bus,
PCI_SLOT(entry.devfn),
PCI_FUNC(entry.devfn));
- continue;
- }
- pci_print_aer(pdev, entry.severity, entry.regs);
+ else
+ pci_print_aer(pdev, entry.severity, entry.regs);
/*
* Memory for aer_capability_regs(entry.regs) is being
@@ -1349,13 +1348,15 @@ static void aer_recover_work_func(struct work_struct *work)
ghes_estatus_pool_region_free((unsigned long)entry.regs,
sizeof(struct aer_capability_regs));
- if (entry.severity == AER_NONFATAL)
- pcie_do_recovery(pdev, pci_channel_io_normal,
- aer_root_reset);
- else if (entry.severity == AER_FATAL)
- pcie_do_recovery(pdev, pci_channel_io_frozen,
- aer_root_reset);
- pci_dev_put(pdev);
+ if (pdev) {
+ if (entry.severity == AER_NONFATAL)
+ pcie_do_recovery(pdev, pci_channel_io_normal,
+ aer_root_reset);
+ else if (entry.severity == AER_FATAL)
+ pcie_do_recovery(pdev, pci_channel_io_frozen,
+ aer_root_reset);
+ pci_dev_put(pdev);
+ }
}
}
diff --git a/drivers/platform/x86/amd/pmf/core.c b/drivers/platform/x86/amd/pmf/core.c
index e787480f4df2..c1b369fdf801 100644
--- a/drivers/platform/x86/amd/pmf/core.c
+++ b/drivers/platform/x86/amd/pmf/core.c
@@ -464,6 +464,7 @@ static const struct acpi_device_id amd_pmf_acpi_ids[] = {
{"AMDI0105", 0},
{"AMDI0107", 0},
{"AMDI0108", 0},
+ {"AMDI0112", 0},
{ }
};
MODULE_DEVICE_TABLE(acpi, amd_pmf_acpi_ids);
diff --git a/drivers/platform/x86/asus-wmi.c b/drivers/platform/x86/asus-wmi.c
index c8cc47df9294..75316c268a74 100644
--- a/drivers/platform/x86/asus-wmi.c
+++ b/drivers/platform/x86/asus-wmi.c
@@ -15,6 +15,7 @@
#include <linux/acpi.h>
#include <linux/backlight.h>
+#include <linux/cleanup.h>
#include <linux/debugfs.h>
#include <linux/delay.h>
#include <linux/dmi.h>
@@ -27,6 +28,7 @@
#include <linux/leds.h>
#include <linux/minmax.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/pci.h>
#include <linux/pci_hotplug.h>
#include <linux/platform_data/x86/asus-wmi.h>
@@ -341,6 +343,20 @@ static enum asus_ally_mcu_hack use_ally_mcu_hack = ASUS_WMI_ALLY_MCU_HACK_INIT;
/* WMI ************************************************************************/
+/*
+ * Concurrent evaluations of ASUS WMI methods can re-enter firmware
+ * SMI/EC mailbox handling and corrupt the mailbox.
+ */
+static DEFINE_MUTEX(asus_wmi_eval_lock);
+
+static acpi_status asus_wmi_evaluate_method_locked(u32 method_id,
+ struct acpi_buffer *input,
+ struct acpi_buffer *output)
+{
+ guard(mutex)(&asus_wmi_eval_lock);
+ return wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id, input, output);
+}
+
static int asus_wmi_evaluate_method3(u32 method_id,
u32 arg0, u32 arg1, u32 arg2, u32 *retval)
{
@@ -355,8 +371,7 @@ static int asus_wmi_evaluate_method3(u32 method_id,
union acpi_object *obj;
u32 tmp = 0;
- status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id,
- &input, &output);
+ status = asus_wmi_evaluate_method_locked(method_id, &input, &output);
pr_debug("%s called (0x%08x) with args: 0x%08x, 0x%08x, 0x%08x\n",
__func__, method_id, arg0, arg1, arg2);
@@ -407,8 +422,7 @@ static int asus_wmi_evaluate_method5(u32 method_id,
union acpi_object *obj;
u32 tmp = 0;
- status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id,
- &input, &output);
+ status = asus_wmi_evaluate_method_locked(method_id, &input, &output);
pr_debug("%s called (0x%08x) with args: 0x%08x, 0x%08x, 0x%08x, 0x%08x, 0x%08x\n",
__func__, method_id, arg0, arg1, arg2, arg3, arg4);
@@ -455,8 +469,7 @@ static int asus_wmi_evaluate_method_buf(u32 method_id,
union acpi_object *obj;
int err = 0;
- status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id,
- &input, &output);
+ status = asus_wmi_evaluate_method_locked(method_id, &input, &output);
pr_debug("%s called (0x%08x) with args: 0x%08x, 0x%08x\n",
__func__, method_id, arg0, arg1);
@@ -4656,9 +4669,8 @@ static int show_call(struct seq_file *m, void *data)
union acpi_object *obj;
acpi_status status;
- status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID,
- 0, asus->debug.method_id,
- &input, &output);
+ status = asus_wmi_evaluate_method_locked(asus->debug.method_id,
+ &input, &output);
if (ACPI_FAILURE(status))
return -EIO;
diff --git a/drivers/platform/x86/intel/int3472/discrete.c b/drivers/platform/x86/intel/int3472/discrete.c
index 96c33ba80002..6ebfaa141a42 100644
--- a/drivers/platform/x86/intel/int3472/discrete.c
+++ b/drivers/platform/x86/intel/int3472/discrete.c
@@ -15,6 +15,7 @@
#include <linux/platform_data/x86/int3472.h>
#include <linux/platform_device.h>
#include <linux/string_choices.h>
+#include <linux/time64.h>
#include <linux/uuid.h>
/*
@@ -123,10 +124,31 @@ skl_int3472_gpiod_get_from_temp_lookup(struct int3472_discrete_device *int3472,
return desc;
}
+/*
+ * Other vana-supply users (e.g. ST, Toshiba, Sony sensors) can be added to
+ * this array instead of adding new quirk table entries.
+ */
+static const char * const power_enable_hids_vana[] = {
+ "SONY471A", /* imx471 on Lenovo X9-14 and X9-15 */
+ "TBE20A0", /* imx471 on Lenovo X1 Carbon G14 */
+ NULL
+};
+
+static const char * const power_enable_hids_vdd[] = {
+ "INT33F0", /* mt9m114 */
+ NULL
+};
+
+static const char * const power_enable_hids_enable[] = {
+ "INT347E", /* ov7251 */
+ NULL
+};
+
/**
* struct int3472_gpio_map - Map GPIOs to whatever is expected by the
* sensor driver (as in DT bindings)
- * @hid: The ACPI HID of the device without the instance number e.g. INT347E
+ * @hids: NULL-terminated array of ACPI HIDs of the devices without the
+ * instance number e.g. INT347E
* @type_from: The GPIO type from ACPI ?SDT
* @type_to: The assigned GPIO type, typically same as @type_from
* @enable_time_us: Enable time in usec for GPIOs mapped to regulators
@@ -135,7 +157,7 @@ skl_int3472_gpiod_get_from_temp_lookup(struct int3472_discrete_device *int3472,
* GPIO_ACTIVE_HIGH otherwise
*/
struct int3472_gpio_map {
- const char *hid;
+ const char * const *hids;
u8 type_from;
u8 type_to;
bool polarity_low;
@@ -145,24 +167,24 @@ struct int3472_gpio_map {
static const struct int3472_gpio_map int3472_gpio_map[] = {
{ /* mt9m114 designs declare a powerdown pin which controls the regulators */
- .hid = "INT33F0",
+ .hids = power_enable_hids_vdd,
.type_from = INT3472_GPIO_TYPE_POWERDOWN,
.type_to = INT3472_GPIO_TYPE_POWER_ENABLE,
.con_id = "vdd",
.enable_time_us = GPIO_REGULATOR_ENABLE_TIME,
},
{ /* ov7251 driver / DT-bindings expect "enable" as con_id for reset */
- .hid = "INT347E",
+ .hids = power_enable_hids_enable,
.type_from = INT3472_GPIO_TYPE_RESET,
.type_to = INT3472_GPIO_TYPE_RESET,
.con_id = "enable",
},
- { /* ov08x40's handshake pin needs a 45 ms delay on some HP laptops */
- .hid = "OVTI08F4",
- .type_from = INT3472_GPIO_TYPE_HANDSHAKE,
- .type_to = INT3472_GPIO_TYPE_HANDSHAKE,
- .con_id = "dvdd",
- .enable_time_us = 45 * USEC_PER_MSEC,
+ { /* Sensors which expect "vana" as con_id for power enable */
+ .hids = power_enable_hids_vana,
+ .type_from = INT3472_GPIO_TYPE_POWER_ENABLE,
+ .type_to = INT3472_GPIO_TYPE_POWER_ENABLE,
+ .con_id = "vana",
+ .enable_time_us = GPIO_REGULATOR_ENABLE_TIME,
},
};
@@ -171,6 +193,7 @@ static void int3472_get_con_id_and_polarity(struct int3472_discrete_device *int3
unsigned int *enable_time_us)
{
struct acpi_device *adev = int3472->sensor;
+ const char * const *hids;
unsigned int i;
for (i = 0; i < ARRAY_SIZE(int3472_gpio_map); i++) {
@@ -182,7 +205,11 @@ static void int3472_get_con_id_and_polarity(struct int3472_discrete_device *int3
if (*type != int3472_gpio_map[i].type_from)
continue;
- if (!acpi_dev_hid_uid_match(adev, int3472_gpio_map[i].hid, NULL))
+ for (hids = int3472_gpio_map[i].hids; *hids; hids++) {
+ if (acpi_dev_hid_uid_match(adev, *hids, NULL))
+ break;
+ }
+ if (!*hids)
continue;
dev_dbg(int3472->dev, "mapping type 0x%02x pin to 0x%02x %s\n",
@@ -226,8 +253,8 @@ static void int3472_get_con_id_and_polarity(struct int3472_discrete_device *int3
case INT3472_GPIO_TYPE_HANDSHAKE:
*con_id = "dvdd";
*gpio_flags = GPIO_ACTIVE_HIGH;
- /* Setups using a handshake pin need 25 ms enable delay */
- *enable_time_us = 25 * USEC_PER_MSEC;
+ /* Powering up the sensor through the handshake pin takes up to 200 ms */
+ *enable_time_us = 200 * USEC_PER_MSEC;
break;
default:
*con_id = "unknown";
diff --git a/drivers/platform/x86/lenovo/Kconfig b/drivers/platform/x86/lenovo/Kconfig
index d22b774e0236..587da1c602ca 100644
--- a/drivers/platform/x86/lenovo/Kconfig
+++ b/drivers/platform/x86/lenovo/Kconfig
@@ -233,7 +233,7 @@ config YT2_1380
To compile this driver as a module, choose M here: the module will
be called lenovo-yogabook.
-config LENOVO_WMI_DATA01
+config LENOVO_WMI_CAPDATA
tristate
depends on ACPI_WMI
@@ -264,7 +264,7 @@ config LENOVO_WMI_TUNING
tristate "Lenovo Other Mode WMI Driver"
depends on ACPI_WMI
select FW_ATTR_CLASS
- select LENOVO_WMI_DATA01
+ select LENOVO_WMI_CAPDATA
select LENOVO_WMI_EVENTS
select LENOVO_WMI_HELPERS
help
diff --git a/drivers/platform/x86/lenovo/Makefile b/drivers/platform/x86/lenovo/Makefile
index 7b2128e3a214..91a9370f11b3 100644
--- a/drivers/platform/x86/lenovo/Makefile
+++ b/drivers/platform/x86/lenovo/Makefile
@@ -12,7 +12,7 @@ lenovo-target-$(CONFIG_LENOVO_YMC) += ymc.o
lenovo-target-$(CONFIG_YOGABOOK) += yogabook.o
lenovo-target-$(CONFIG_YT2_1380) += yoga-tab2-pro-1380-fastcharger.o
lenovo-target-$(CONFIG_LENOVO_WMI_CAMERA) += wmi-camera.o
-lenovo-target-$(CONFIG_LENOVO_WMI_DATA01) += wmi-capdata01.o
+lenovo-target-$(CONFIG_LENOVO_WMI_CAPDATA) += wmi-capdata.o
lenovo-target-$(CONFIG_LENOVO_WMI_EVENTS) += wmi-events.o
lenovo-target-$(CONFIG_LENOVO_WMI_HELPERS) += wmi-helpers.o
lenovo-target-$(CONFIG_LENOVO_WMI_GAMEZONE) += wmi-gamezone.o
diff --git a/drivers/platform/x86/lenovo/wmi-capdata.c b/drivers/platform/x86/lenovo/wmi-capdata.c
new file mode 100644
index 000000000000..c7fa1c296956
--- /dev/null
+++ b/drivers/platform/x86/lenovo/wmi-capdata.c
@@ -0,0 +1,310 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Lenovo Capability Data WMI Data Block driver.
+ *
+ * Lenovo Capability Data provides information on tunable attributes used by
+ * the "Other Mode" WMI interface.
+ *
+ * Capability Data 01 includes if the attribute is supported by the hardware,
+ * and the default_value, max_value, min_value, and step increment. Each
+ * attribute has multiple pages, one for each of the thermal modes managed by
+ * the Gamezone interface.
+ *
+ * Copyright (C) 2025 Derek J. Clark <derekjohn.clark@gmail.com>
+ * - Initial implementation (formerly named lenovo-wmi-capdata01)
+ */
+
+#include <linux/acpi.h>
+#include <linux/cleanup.h>
+#include <linux/component.h>
+#include <linux/container_of.h>
+#include <linux/device.h>
+#include <linux/export.h>
+#include <linux/gfp_types.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/mutex_types.h>
+#include <linux/notifier.h>
+#include <linux/overflow.h>
+#include <linux/types.h>
+#include <linux/wmi.h>
+
+#include "wmi-capdata.h"
+
+#define LENOVO_CAPABILITY_DATA_01_GUID "7A8F5407-CB67-4D6E-B547-39B3BE018154"
+
+#define ACPI_AC_CLASS "ac_adapter"
+#define ACPI_AC_NOTIFY_STATUS 0x80
+
+struct lwmi_cd_priv {
+ struct notifier_block acpi_nb; /* ACPI events */
+ struct wmi_device *wdev;
+ struct cd_list *list;
+};
+
+struct cd_list {
+ struct mutex list_mutex; /* list R/W mutex */
+ u8 count;
+ struct capdata01 data[];
+};
+
+/**
+ * lwmi_cd_component_bind() - Bind component to master device.
+ * @cd_dev: Pointer to the lenovo-wmi-capdata driver parent device.
+ * @om_dev: Pointer to the lenovo-wmi-other driver parent device.
+ * @data: cd_list object pointer used to return the capability data.
+ *
+ * On lenovo-wmi-other's master bind, provide a pointer to the local capdata
+ * list. This is used to call lwmi_cd*_get_data to look up attribute data
+ * from the lenovo-wmi-other driver.
+ *
+ * Return: 0
+ */
+static int lwmi_cd_component_bind(struct device *cd_dev,
+ struct device *om_dev, void *data)
+{
+ struct lwmi_cd_priv *priv = dev_get_drvdata(cd_dev);
+ struct cd_list **cd_list = data;
+
+ *cd_list = priv->list;
+
+ return 0;
+}
+
+static const struct component_ops lwmi_cd_component_ops = {
+ .bind = lwmi_cd_component_bind,
+};
+
+/**
+ * lwmi_cd01_get_data - Get the data of the specified attribute
+ * @list: The lenovo-wmi-capdata pointer to its cd_list struct.
+ * @attribute_id: The capdata attribute ID to be found.
+ * @output: Pointer to a capdata01 struct to return the data.
+ *
+ * Retrieves the capability data 01 struct pointer for the given
+ * attribute for its specified thermal mode.
+ *
+ * Return: 0 on success, or -EINVAL.
+ */
+int lwmi_cd01_get_data(struct cd_list *list, u32 attribute_id, struct capdata01 *output)
+{
+ u8 idx;
+
+ guard(mutex)(&list->list_mutex);
+ for (idx = 0; idx < list->count; idx++) {
+ if (list->data[idx].id != attribute_id)
+ continue;
+ memcpy(output, &list->data[idx], sizeof(list->data[idx]));
+ return 0;
+ }
+
+ return -EINVAL;
+}
+EXPORT_SYMBOL_NS_GPL(lwmi_cd01_get_data, "LENOVO_WMI_CAPDATA");
+
+/**
+ * lwmi_cd_cache() - Cache all WMI data block information
+ * @priv: lenovo-wmi-capdata driver data.
+ *
+ * Loop through each WMI data block and cache the data.
+ *
+ * Return: 0 on success, or an error.
+ */
+static int lwmi_cd_cache(struct lwmi_cd_priv *priv)
+{
+ int idx;
+
+ guard(mutex)(&priv->list->list_mutex);
+ for (idx = 0; idx < priv->list->count; idx++) {
+ union acpi_object *ret_obj __free(kfree) = NULL;
+
+ ret_obj = wmidev_block_query(priv->wdev, idx);
+ if (!ret_obj)
+ return -ENODEV;
+
+ if (ret_obj->type != ACPI_TYPE_BUFFER ||
+ ret_obj->buffer.length < sizeof(priv->list->data[idx]))
+ continue;
+
+ memcpy(&priv->list->data[idx], ret_obj->buffer.pointer,
+ ret_obj->buffer.length);
+ }
+
+ return 0;
+}
+
+/**
+ * lwmi_cd_alloc() - Allocate a cd_list struct in drvdata
+ * @priv: lenovo-wmi-capdata driver data.
+ *
+ * Allocate a cd_list struct large enough to contain data from all WMI data
+ * blocks provided by the interface.
+ *
+ * Return: 0 on success, or an error.
+ */
+static int lwmi_cd_alloc(struct lwmi_cd_priv *priv)
+{
+ struct cd_list *list;
+ size_t list_size;
+ int count, ret;
+
+ count = wmidev_instance_count(priv->wdev);
+ list_size = struct_size(list, data, count);
+
+ list = devm_kzalloc(&priv->wdev->dev, list_size, GFP_KERNEL);
+ if (!list)
+ return -ENOMEM;
+
+ ret = devm_mutex_init(&priv->wdev->dev, &list->list_mutex);
+ if (ret)
+ return ret;
+
+ list->count = count;
+ priv->list = list;
+
+ return 0;
+}
+
+/**
+ * lwmi_cd_setup() - Cache all WMI data block information
+ * @priv: lenovo-wmi-capdata driver data.
+ *
+ * Allocate a cd_list struct large enough to contain data from all WMI data
+ * blocks provided by the interface. Then loop through each data block and
+ * cache the data.
+ *
+ * Return: 0 on success, or an error code.
+ */
+static int lwmi_cd_setup(struct lwmi_cd_priv *priv)
+{
+ int ret;
+
+ ret = lwmi_cd_alloc(priv);
+ if (ret)
+ return ret;
+
+ return lwmi_cd_cache(priv);
+}
+
+/**
+ * lwmi_cd01_notifier_call() - Call method for cd01 notifier.
+ * @nb: The cd01 ACPI notifier_block.
+ * @action: Unused.
+ * @data: The ACPI event.
+ *
+ * Refresh cached capdata on AC status change.
+ *
+ * Return: notifier_block status.
+ */
+static int lwmi_cd01_notifier_call(struct notifier_block *nb, unsigned long action,
+ void *data)
+{
+ struct acpi_bus_event *event = data;
+ struct lwmi_cd_priv *priv;
+ int ret;
+
+ if (strcmp(event->device_class, ACPI_AC_CLASS) != 0)
+ return NOTIFY_DONE;
+
+ priv = container_of(nb, struct lwmi_cd_priv, acpi_nb);
+
+ switch (event->type) {
+ case ACPI_AC_NOTIFY_STATUS:
+ ret = lwmi_cd_cache(priv);
+ if (ret) {
+ dev_warn(&priv->wdev->dev,
+ "failed to refresh capdata on AC change: %d\n", ret);
+
+ /*
+ * Our error is non-fatal for others, and we still want
+ * the other event consumers to process it.
+ */
+ return NOTIFY_DONE;
+ }
+
+ return NOTIFY_OK;
+ default:
+ return NOTIFY_DONE;
+ }
+}
+
+/**
+ * lwmi_cd01_unregister() - Unregister the cd01 ACPI notifier_block.
+ * @data: The ACPI event notifier_block to unregister.
+ */
+static void lwmi_cd01_unregister(void *data)
+{
+ struct notifier_block *acpi_nb = data;
+
+ unregister_acpi_notifier(acpi_nb);
+}
+
+static int lwmi_cd_probe(struct wmi_device *wdev, const void *context)
+{
+ struct lwmi_cd_priv *priv;
+ int ret;
+
+ priv = devm_kzalloc(&wdev->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ priv->wdev = wdev;
+ dev_set_drvdata(&wdev->dev, priv);
+
+ ret = lwmi_cd_setup(priv);
+ if (ret)
+ return ret;
+
+ priv->acpi_nb.notifier_call = lwmi_cd01_notifier_call;
+
+ ret = register_acpi_notifier(&priv->acpi_nb);
+ if (ret)
+ return ret;
+
+ ret = devm_add_action_or_reset(&wdev->dev, lwmi_cd01_unregister, &priv->acpi_nb);
+ if (ret)
+ return ret;
+
+ return component_add(&wdev->dev, &lwmi_cd_component_ops);
+}
+
+static void lwmi_cd_remove(struct wmi_device *wdev)
+{
+ component_del(&wdev->dev, &lwmi_cd_component_ops);
+}
+
+static const struct wmi_device_id lwmi_cd_id_table[] = {
+ { LENOVO_CAPABILITY_DATA_01_GUID, NULL },
+ {}
+};
+
+static struct wmi_driver lwmi_cd_driver = {
+ .driver = {
+ .name = "lenovo_wmi_capdata",
+ .probe_type = PROBE_PREFER_ASYNCHRONOUS,
+ },
+ .id_table = lwmi_cd_id_table,
+ .probe = lwmi_cd_probe,
+ .remove = lwmi_cd_remove,
+ .no_singleton = true,
+};
+
+/**
+ * lwmi_cd01_match() - Match rule for the master driver.
+ * @dev: Pointer to the capability data 01 parent device.
+ * @data: Unused void pointer for passing match criteria.
+ *
+ * Return: int.
+ */
+int lwmi_cd01_match(struct device *dev, void *data)
+{
+ return dev->driver == &lwmi_cd_driver.driver;
+}
+EXPORT_SYMBOL_NS_GPL(lwmi_cd01_match, "LENOVO_WMI_CAPDATA");
+
+module_wmi_driver(lwmi_cd_driver);
+
+MODULE_DEVICE_TABLE(wmi, lwmi_cd_id_table);
+MODULE_AUTHOR("Derek J. Clark <derekjohn.clark@gmail.com>");
+MODULE_DESCRIPTION("Lenovo Capability Data WMI Driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/platform/x86/lenovo/wmi-capdata.h b/drivers/platform/x86/lenovo/wmi-capdata.h
new file mode 100644
index 000000000000..2a4746e38ad4
--- /dev/null
+++ b/drivers/platform/x86/lenovo/wmi-capdata.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+/* Copyright (C) 2025 Derek J. Clark <derekjohn.clark@gmail.com> */
+
+#ifndef _LENOVO_WMI_CAPDATA_H_
+#define _LENOVO_WMI_CAPDATA_H_
+
+#include <linux/types.h>
+
+struct device;
+struct cd_list;
+
+struct capdata01 {
+ u32 id;
+ u32 supported;
+ u32 default_value;
+ u32 step;
+ u32 min_value;
+ u32 max_value;
+};
+
+int lwmi_cd01_get_data(struct cd_list *list, u32 attribute_id, struct capdata01 *output);
+int lwmi_cd01_match(struct device *dev, void *data);
+
+#endif /* !_LENOVO_WMI_CAPDATA_H_ */
diff --git a/drivers/platform/x86/lenovo/wmi-capdata01.c b/drivers/platform/x86/lenovo/wmi-capdata01.c
deleted file mode 100644
index fc7e3454e71d..000000000000
--- a/drivers/platform/x86/lenovo/wmi-capdata01.c
+++ /dev/null
@@ -1,302 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * Lenovo Capability Data 01 WMI Data Block driver.
- *
- * Lenovo Capability Data 01 provides information on tunable attributes used by
- * the "Other Mode" WMI interface. The data includes if the attribute is
- * supported by the hardware, the default_value, max_value, min_value, and step
- * increment. Each attribute has multiple pages, one for each of the thermal
- * modes managed by the Gamezone interface.
- *
- * Copyright (C) 2025 Derek J. Clark <derekjohn.clark@gmail.com>
- */
-
-#include <linux/acpi.h>
-#include <linux/cleanup.h>
-#include <linux/component.h>
-#include <linux/container_of.h>
-#include <linux/device.h>
-#include <linux/export.h>
-#include <linux/gfp_types.h>
-#include <linux/module.h>
-#include <linux/mutex.h>
-#include <linux/mutex_types.h>
-#include <linux/notifier.h>
-#include <linux/overflow.h>
-#include <linux/types.h>
-#include <linux/wmi.h>
-
-#include "wmi-capdata01.h"
-
-#define LENOVO_CAPABILITY_DATA_01_GUID "7A8F5407-CB67-4D6E-B547-39B3BE018154"
-
-#define ACPI_AC_CLASS "ac_adapter"
-#define ACPI_AC_NOTIFY_STATUS 0x80
-
-struct lwmi_cd01_priv {
- struct notifier_block acpi_nb; /* ACPI events */
- struct wmi_device *wdev;
- struct cd01_list *list;
-};
-
-struct cd01_list {
- struct mutex list_mutex; /* list R/W mutex */
- u8 count;
- struct capdata01 data[];
-};
-
-/**
- * lwmi_cd01_component_bind() - Bind component to master device.
- * @cd01_dev: Pointer to the lenovo-wmi-capdata01 driver parent device.
- * @om_dev: Pointer to the lenovo-wmi-other driver parent device.
- * @data: capdata01_list object pointer used to return the capability data.
- *
- * On lenovo-wmi-other's master bind, provide a pointer to the local capdata01
- * list. This is used to call lwmi_cd01_get_data to look up attribute data
- * from the lenovo-wmi-other driver.
- *
- * Return: 0
- */
-static int lwmi_cd01_component_bind(struct device *cd01_dev,
- struct device *om_dev, void *data)
-{
- struct lwmi_cd01_priv *priv = dev_get_drvdata(cd01_dev);
- struct cd01_list **cd01_list = data;
-
- *cd01_list = priv->list;
-
- return 0;
-}
-
-static const struct component_ops lwmi_cd01_component_ops = {
- .bind = lwmi_cd01_component_bind,
-};
-
-/**
- * lwmi_cd01_get_data - Get the data of the specified attribute
- * @list: The lenovo-wmi-capdata01 pointer to its cd01_list struct.
- * @attribute_id: The capdata attribute ID to be found.
- * @output: Pointer to a capdata01 struct to return the data.
- *
- * Retrieves the capability data 01 struct pointer for the given
- * attribute for its specified thermal mode.
- *
- * Return: 0 on success, or -EINVAL.
- */
-int lwmi_cd01_get_data(struct cd01_list *list, u32 attribute_id, struct capdata01 *output)
-{
- u8 idx;
-
- guard(mutex)(&list->list_mutex);
- for (idx = 0; idx < list->count; idx++) {
- if (list->data[idx].id != attribute_id)
- continue;
- memcpy(output, &list->data[idx], sizeof(list->data[idx]));
- return 0;
- }
-
- return -EINVAL;
-}
-EXPORT_SYMBOL_NS_GPL(lwmi_cd01_get_data, "LENOVO_WMI_CD01");
-
-/**
- * lwmi_cd01_cache() - Cache all WMI data block information
- * @priv: lenovo-wmi-capdata01 driver data.
- *
- * Loop through each WMI data block and cache the data.
- *
- * Return: 0 on success, or an error.
- */
-static int lwmi_cd01_cache(struct lwmi_cd01_priv *priv)
-{
- int idx;
-
- guard(mutex)(&priv->list->list_mutex);
- for (idx = 0; idx < priv->list->count; idx++) {
- union acpi_object *ret_obj __free(kfree) = NULL;
-
- ret_obj = wmidev_block_query(priv->wdev, idx);
- if (!ret_obj)
- return -ENODEV;
-
- if (ret_obj->type != ACPI_TYPE_BUFFER ||
- ret_obj->buffer.length < sizeof(priv->list->data[idx]))
- continue;
-
- memcpy(&priv->list->data[idx], ret_obj->buffer.pointer,
- ret_obj->buffer.length);
- }
-
- return 0;
-}
-
-/**
- * lwmi_cd01_alloc() - Allocate a cd01_list struct in drvdata
- * @priv: lenovo-wmi-capdata01 driver data.
- *
- * Allocate a cd01_list struct large enough to contain data from all WMI data
- * blocks provided by the interface.
- *
- * Return: 0 on success, or an error.
- */
-static int lwmi_cd01_alloc(struct lwmi_cd01_priv *priv)
-{
- struct cd01_list *list;
- size_t list_size;
- int count, ret;
-
- count = wmidev_instance_count(priv->wdev);
- list_size = struct_size(list, data, count);
-
- list = devm_kzalloc(&priv->wdev->dev, list_size, GFP_KERNEL);
- if (!list)
- return -ENOMEM;
-
- ret = devm_mutex_init(&priv->wdev->dev, &list->list_mutex);
- if (ret)
- return ret;
-
- list->count = count;
- priv->list = list;
-
- return 0;
-}
-
-/**
- * lwmi_cd01_setup() - Cache all WMI data block information
- * @priv: lenovo-wmi-capdata01 driver data.
- *
- * Allocate a cd01_list struct large enough to contain data from all WMI data
- * blocks provided by the interface. Then loop through each data block and
- * cache the data.
- *
- * Return: 0 on success, or an error code.
- */
-static int lwmi_cd01_setup(struct lwmi_cd01_priv *priv)
-{
- int ret;
-
- ret = lwmi_cd01_alloc(priv);
- if (ret)
- return ret;
-
- return lwmi_cd01_cache(priv);
-}
-
-/**
- * lwmi_cd01_notifier_call() - Call method for lenovo-wmi-capdata01 driver notifier.
- * block call chain.
- * @nb: The notifier_block registered to lenovo-wmi-events driver.
- * @action: Unused.
- * @data: The ACPI event.
- *
- * For LWMI_EVENT_THERMAL_MODE, set current_mode and notify platform_profile
- * of a change.
- *
- * Return: notifier_block status.
- */
-static int lwmi_cd01_notifier_call(struct notifier_block *nb, unsigned long action,
- void *data)
-{
- struct acpi_bus_event *event = data;
- struct lwmi_cd01_priv *priv;
- int ret;
-
- if (strcmp(event->device_class, ACPI_AC_CLASS) != 0)
- return NOTIFY_DONE;
-
- priv = container_of(nb, struct lwmi_cd01_priv, acpi_nb);
-
- switch (event->type) {
- case ACPI_AC_NOTIFY_STATUS:
- ret = lwmi_cd01_cache(priv);
- if (ret)
- return NOTIFY_BAD;
-
- return NOTIFY_OK;
- default:
- return NOTIFY_DONE;
- }
-}
-
-/**
- * lwmi_cd01_unregister() - Unregister the cd01 ACPI notifier_block.
- * @data: The ACPI event notifier_block to unregister.
- */
-static void lwmi_cd01_unregister(void *data)
-{
- struct notifier_block *acpi_nb = data;
-
- unregister_acpi_notifier(acpi_nb);
-}
-
-static int lwmi_cd01_probe(struct wmi_device *wdev, const void *context)
-
-{
- struct lwmi_cd01_priv *priv;
- int ret;
-
- priv = devm_kzalloc(&wdev->dev, sizeof(*priv), GFP_KERNEL);
- if (!priv)
- return -ENOMEM;
-
- priv->wdev = wdev;
- dev_set_drvdata(&wdev->dev, priv);
-
- ret = lwmi_cd01_setup(priv);
- if (ret)
- return ret;
-
- priv->acpi_nb.notifier_call = lwmi_cd01_notifier_call;
-
- ret = register_acpi_notifier(&priv->acpi_nb);
- if (ret)
- return ret;
-
- ret = devm_add_action_or_reset(&wdev->dev, lwmi_cd01_unregister, &priv->acpi_nb);
- if (ret)
- return ret;
-
- return component_add(&wdev->dev, &lwmi_cd01_component_ops);
-}
-
-static void lwmi_cd01_remove(struct wmi_device *wdev)
-{
- component_del(&wdev->dev, &lwmi_cd01_component_ops);
-}
-
-static const struct wmi_device_id lwmi_cd01_id_table[] = {
- { LENOVO_CAPABILITY_DATA_01_GUID, NULL },
- {}
-};
-
-static struct wmi_driver lwmi_cd01_driver = {
- .driver = {
- .name = "lenovo_wmi_cd01",
- .probe_type = PROBE_PREFER_ASYNCHRONOUS,
- },
- .id_table = lwmi_cd01_id_table,
- .probe = lwmi_cd01_probe,
- .remove = lwmi_cd01_remove,
- .no_singleton = true,
-};
-
-/**
- * lwmi_cd01_match() - Match rule for the master driver.
- * @dev: Pointer to the capability data 01 parent device.
- * @data: Unused void pointer for passing match criteria.
- *
- * Return: int.
- */
-int lwmi_cd01_match(struct device *dev, void *data)
-{
- return dev->driver == &lwmi_cd01_driver.driver;
-}
-EXPORT_SYMBOL_NS_GPL(lwmi_cd01_match, "LENOVO_WMI_CD01");
-
-module_wmi_driver(lwmi_cd01_driver);
-
-MODULE_DEVICE_TABLE(wmi, lwmi_cd01_id_table);
-MODULE_AUTHOR("Derek J. Clark <derekjohn.clark@gmail.com>");
-MODULE_DESCRIPTION("Lenovo Capability Data 01 WMI Driver");
-MODULE_LICENSE("GPL");
diff --git a/drivers/platform/x86/lenovo/wmi-capdata01.h b/drivers/platform/x86/lenovo/wmi-capdata01.h
deleted file mode 100644
index bd06c5751f68..000000000000
--- a/drivers/platform/x86/lenovo/wmi-capdata01.h
+++ /dev/null
@@ -1,25 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-
-/* Copyright (C) 2025 Derek J. Clark <derekjohn.clark@gmail.com> */
-
-#ifndef _LENOVO_WMI_CAPDATA01_H_
-#define _LENOVO_WMI_CAPDATA01_H_
-
-#include <linux/types.h>
-
-struct device;
-struct cd01_list;
-
-struct capdata01 {
- u32 id;
- u32 supported;
- u32 default_value;
- u32 step;
- u32 min_value;
- u32 max_value;
-};
-
-int lwmi_cd01_get_data(struct cd01_list *list, u32 attribute_id, struct capdata01 *output);
-int lwmi_cd01_match(struct device *dev, void *data);
-
-#endif /* !_LENOVO_WMI_CAPDATA01_H_ */
diff --git a/drivers/platform/x86/lenovo/wmi-other.c b/drivers/platform/x86/lenovo/wmi-other.c
index dc7c60df7ad3..eaab92506876 100644
--- a/drivers/platform/x86/lenovo/wmi-other.c
+++ b/drivers/platform/x86/lenovo/wmi-other.c
@@ -34,7 +34,7 @@
#include <linux/types.h>
#include <linux/wmi.h>
-#include "wmi-capdata01.h"
+#include "wmi-capdata.h"
#include "wmi-events.h"
#include "wmi-helpers.h"
#include "wmi-other.h"
@@ -73,7 +73,10 @@ enum attribute_property {
struct lwmi_om_priv {
struct component_master_ops *ops;
- struct cd01_list *cd01_list; /* only valid after capdata01 bind */
+
+ /* only valid after capdata bind */
+ struct cd_list *cd01_list;
+
struct device *fw_attr_dev;
struct kset *fw_attr_kset;
struct notifier_block nb;
@@ -574,7 +577,7 @@ static void lwmi_om_fw_attr_remove(struct lwmi_om_priv *priv)
static int lwmi_om_master_bind(struct device *dev)
{
struct lwmi_om_priv *priv = dev_get_drvdata(dev);
- struct cd01_list *tmp_list;
+ struct cd_list *tmp_list;
int ret;
ret = component_bind_all(dev, &tmp_list);
@@ -655,7 +658,7 @@ static struct wmi_driver lwmi_other_driver = {
module_wmi_driver(lwmi_other_driver);
-MODULE_IMPORT_NS("LENOVO_WMI_CD01");
+MODULE_IMPORT_NS("LENOVO_WMI_CAPDATA");
MODULE_IMPORT_NS("LENOVO_WMI_HELPERS");
MODULE_DEVICE_TABLE(wmi, lwmi_other_id_table);
MODULE_AUTHOR("Derek J. Clark <derekjohn.clark@gmail.com>");
diff --git a/drivers/power/supply/ltc2941-battery-gauge.c b/drivers/power/supply/ltc2941-battery-gauge.c
index a1ddc4b060ce..bf0b268b10bf 100644
--- a/drivers/power/supply/ltc2941-battery-gauge.c
+++ b/drivers/power/supply/ltc2941-battery-gauge.c
@@ -464,9 +464,11 @@ static int ltc294x_i2c_probe(struct i2c_client *client)
/* r_sense can be negative, when sense+ is connected to the battery
* instead of the sense-. This results in reversed measurements. */
ret = of_property_read_u32(np, "lltc,resistor-sense", &r_sense);
- if (ret < 0)
+ if (ret < 0) {
+ of_node_put(np);
return dev_err_probe(&client->dev, ret,
"Could not find lltc,resistor-sense in devicetree\n");
+ }
info->r_sense = r_sense;
ret = of_property_read_u32(np, "lltc,prescaler-exponent",
@@ -477,6 +479,8 @@ static int ltc294x_i2c_probe(struct i2c_client *client)
prescaler_exp = LTC2941_MAX_PRESCALER_EXP;
}
+ of_node_put(np);
+
if (info->id == LTC2943_ID) {
if (prescaler_exp > LTC2943_MAX_PRESCALER_EXP)
prescaler_exp = LTC2943_MAX_PRESCALER_EXP;
diff --git a/drivers/ptp/ptp_sysfs.c b/drivers/ptp/ptp_sysfs.c
index 200eaf500696..0d4b081daf63 100644
--- a/drivers/ptp/ptp_sysfs.c
+++ b/drivers/ptp/ptp_sysfs.c
@@ -80,13 +80,17 @@ static ssize_t extts_fifo_show(struct device *dev,
size_t qcnt;
int cnt = 0;
+ spin_lock_irqsave(&ptp->tsevqs_lock, flags);
cnt = list_count_nodes(&ptp->tsevqs);
- if (cnt <= 0)
+ if (cnt <= 0) {
+ spin_unlock_irqrestore(&ptp->tsevqs_lock, flags);
goto out;
+ }
/* The sysfs fifo will always draw from the fist queue */
queue = list_first_entry(&ptp->tsevqs, struct timestamp_event_queue,
qlist);
+ spin_unlock_irqrestore(&ptp->tsevqs_lock, flags);
memset(&event, 0, sizeof(event));
spin_lock_irqsave(&queue->lock, flags);
diff --git a/drivers/s390/char/vmcp.c b/drivers/s390/char/vmcp.c
index 69899bb86b3e..1d07d7b6acb8 100644
--- a/drivers/s390/char/vmcp.c
+++ b/drivers/s390/char/vmcp.c
@@ -105,13 +105,11 @@ static int vmcp_open(struct inode *inode, struct file *file)
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
- session = kmalloc(sizeof(*session), GFP_KERNEL);
+ session = kzalloc(sizeof(*session), GFP_KERNEL);
if (!session)
return -ENOMEM;
session->bufsize = PAGE_SIZE;
- session->response = NULL;
- session->resp_size = 0;
mutex_init(&session->mutex);
file->private_data = session;
return nonseekable_open(inode, file);
diff --git a/drivers/scsi/aic7xxx/aic79xx_core.c b/drivers/scsi/aic7xxx/aic79xx_core.c
index 6b87ea004e53..ff7efba29ada 100644
--- a/drivers/scsi/aic7xxx/aic79xx_core.c
+++ b/drivers/scsi/aic7xxx/aic79xx_core.c
@@ -8942,8 +8942,10 @@ ahd_handle_scsi_status(struct ahd_softc *ahd, struct scb *scb)
}
#endif
- if (ahd_perform_autosense(scb) == 0)
+ if (ahd_perform_autosense(scb) == 0) {
+ ahd_done(ahd, scb);
break;
+ }
ahd_compile_devinfo(&devinfo, SCB_GET_OUR_ID(scb),
SCB_GET_TARGET(ahd, scb),
diff --git a/drivers/scsi/aic7xxx/aic79xx_osm.h b/drivers/scsi/aic7xxx/aic79xx_osm.h
index 793fe19993a9..3bcfcfe18f08 100644
--- a/drivers/scsi/aic7xxx/aic79xx_osm.h
+++ b/drivers/scsi/aic7xxx/aic79xx_osm.h
@@ -499,6 +499,8 @@ int ahd_linux_show_info(struct seq_file *,struct Scsi_Host *);
static inline
void ahd_cmd_set_transaction_status(struct scsi_cmnd *cmd, uint32_t status)
{
+ if (!cmd)
+ return;
cmd->result &= ~(CAM_STATUS_MASK << 16);
cmd->result |= status << 16;
}
@@ -512,6 +514,8 @@ void ahd_set_transaction_status(struct scb *scb, uint32_t status)
static inline
void ahd_cmd_set_scsi_status(struct scsi_cmnd *cmd, uint32_t status)
{
+ if (!cmd)
+ return;
cmd->result &= ~0xFFFF;
cmd->result |= status;
}
@@ -525,6 +529,8 @@ void ahd_set_scsi_status(struct scb *scb, uint32_t status)
static inline
uint32_t ahd_cmd_get_transaction_status(struct scsi_cmnd *cmd)
{
+ if (!cmd)
+ return CAM_REQ_INPROG;
return ((cmd->result >> 16) & CAM_STATUS_MASK);
}
@@ -537,6 +543,8 @@ uint32_t ahd_get_transaction_status(struct scb *scb)
static inline
uint32_t ahd_cmd_get_scsi_status(struct scsi_cmnd *cmd)
{
+ if (!cmd)
+ return SAM_STAT_GOOD;
return (cmd->result & 0xFFFF);
}
@@ -564,13 +572,16 @@ u_long ahd_get_transfer_length(struct scb *scb)
static inline
int ahd_get_transfer_dir(struct scb *scb)
{
+ if (!scb->io_ctx)
+ return DMA_NONE;
return (scb->io_ctx->sc_data_direction);
}
static inline
void ahd_set_residual(struct scb *scb, u_long resid)
{
- scsi_set_resid(scb->io_ctx, resid);
+ if (scb->io_ctx)
+ scsi_set_resid(scb->io_ctx, resid);
}
static inline
@@ -582,6 +593,8 @@ void ahd_set_sense_residual(struct scb *scb, u_long resid)
static inline
u_long ahd_get_residual(struct scb *scb)
{
+ if (!scb->io_ctx)
+ return 0;
return scsi_get_resid(scb->io_ctx);
}
@@ -595,11 +608,12 @@ static inline
int ahd_perform_autosense(struct scb *scb)
{
/*
- * We always perform autosense in Linux.
- * On other platforms this is set on a
- * per-transaction basis.
+ * We always perform autosense in Linux, except for the device
+ * reset SCB, which has no command to return sense data to.
*/
- return (1);
+ if (!scb->io_ctx)
+ return 0;
+ return 1;
}
static inline uint32_t
@@ -629,6 +643,8 @@ void ahd_platform_freeze_devq(struct ahd_softc *ahd, struct scb *scb);
static inline void
ahd_freeze_scb(struct scb *scb)
{
+ if (!scb->io_ctx)
+ return;
if ((scb->io_ctx->result & (CAM_DEV_QFRZN << 16)) == 0) {
scb->io_ctx->result |= CAM_DEV_QFRZN << 16;
scb->platform_data->dev->qfrozen++;
diff --git a/drivers/scsi/aic7xxx/aic7xxx_core.c b/drivers/scsi/aic7xxx/aic7xxx_core.c
index a396f048a031..d747d0b72a13 100644
--- a/drivers/scsi/aic7xxx/aic7xxx_core.c
+++ b/drivers/scsi/aic7xxx/aic7xxx_core.c
@@ -5266,9 +5266,13 @@ ahc_init(struct ahc_softc *ahc)
/*
* Assume we have a board at this stage and it has been reset.
+ * PCI controllers keep the host ID ahc_pci_config() chose.
*/
- if ((ahc->flags & AHC_USEDEFAULTS) != 0)
- ahc->our_id = ahc->our_id_b = 7;
+ if ((ahc->flags & AHC_USEDEFAULTS) != 0) {
+ if ((ahc->chip & AHC_PCI) == 0)
+ ahc->our_id = 7;
+ ahc->our_id_b = 7;
+ }
/*
* Default to allowing initiator operations.
diff --git a/drivers/scsi/aic7xxx/aic7xxx_pci.c b/drivers/scsi/aic7xxx/aic7xxx_pci.c
index 2d4c85426dc3..5a04c7d55e4b 100644
--- a/drivers/scsi/aic7xxx/aic7xxx_pci.c
+++ b/drivers/scsi/aic7xxx/aic7xxx_pci.c
@@ -701,6 +701,21 @@ ahc_find_pci_device(ahc_dev_softc_t pci)
return (NULL);
}
+/*
+ * The SGI O2 firmware runs its onboard controllers, in slots 1 and 2 of the
+ * root bus, without the Adaptec BIOS and at host ID 0. It leaves no settings
+ * in scratch RAM, which may still hold power-on contents with bad parity.
+ */
+static bool
+ahc_pci_sgi_o2_onboard(struct ahc_softc *ahc)
+{
+ int slot = ahc_get_pci_slot(ahc->dev_softc);
+
+ return IS_ENABLED(CONFIG_SGI_IP32) &&
+ ahc_get_pci_bus(ahc->dev_softc) == 0 &&
+ (slot == 1 || slot == 2);
+}
+
int
ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry)
{
@@ -782,6 +797,9 @@ ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry)
ahc->bus_intr = ahc_pci_intr;
ahc->bus_chip_init = ahc_pci_chip_init;
+ if (ahc_pci_sgi_o2_onboard(ahc))
+ ahc->flags |= AHC_NO_BIOS_INIT;
+
/* Remember how the card was setup in case there is no SEEPROM */
if ((ahc_inb(ahc, HCNTRL) & POWRDN) == 0) {
ahc_pause(ahc);
@@ -898,7 +916,7 @@ ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry)
* Assume only one connector and always turn
* on termination.
*/
- our_id = 0x07;
+ our_id = ahc_pci_sgi_o2_onboard(ahc) ? 0 : 0x07;
sxfrctl1 = STPWEN;
}
ahc_outb(ahc, SCSICONF, our_id|ENSPCHK|RESET_SCSI);
diff --git a/drivers/scsi/scsi_bsg.c b/drivers/scsi/scsi_bsg.c
index a9a9ec086a7e..e2e4b8bb559f 100644
--- a/drivers/scsi/scsi_bsg.c
+++ b/drivers/scsi/scsi_bsg.c
@@ -15,7 +15,7 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr,
struct scsi_cmnd *scmd;
struct request *rq;
struct bio *bio;
- int ret;
+ int ret, unmap_ret;
if (hdr->protocol != BSG_PROTOCOL_SCSI ||
hdr->subprotocol != BSG_SUB_PROTOCOL_SCSI_CMD)
@@ -89,7 +89,9 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr,
else
hdr->dout_resid = scmd->resid_len;
- blk_rq_unmap_user(bio);
+ unmap_ret = blk_rq_unmap_user(bio);
+ if (!ret)
+ ret = unmap_ret;
out_put_request:
blk_mq_free_request(rq);
diff --git a/drivers/scsi/scsi_ioctl.c b/drivers/scsi/scsi_ioctl.c
index 0ddc95bafc71..0b4bc15148de 100644
--- a/drivers/scsi/scsi_ioctl.c
+++ b/drivers/scsi/scsi_ioctl.c
@@ -9,6 +9,7 @@
#include <linux/blkdev.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
+#include <linux/jiffies.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/mm.h>
@@ -192,10 +193,14 @@ static int sg_set_timeout(struct scsi_device *sdev, int __user *p)
{
int timeout, err = get_user(timeout, p);
- if (!err)
- sdev->sg_timeout = clock_t_to_jiffies(timeout);
+ if (err)
+ return err;
+ if (timeout < 0)
+ return -EIO;
- return err;
+ sdev->sg_timeout = min(clock_t_to_jiffies(timeout),
+ min(UINT_MAX, MAX_JIFFY_OFFSET));
+ return 0;
}
static int sg_get_reserved_size(struct scsi_device *sdev, int __user *p)
diff --git a/drivers/soc/qcom/pmic_glink_altmode.c b/drivers/soc/qcom/pmic_glink_altmode.c
index 7f11acd33323..22e4c1af745c 100644
--- a/drivers/soc/qcom/pmic_glink_altmode.c
+++ b/drivers/soc/qcom/pmic_glink_altmode.c
@@ -303,7 +303,7 @@ static void pmic_glink_altmode_sc8180xp_notify(struct pmic_glink_altmode *altmod
alt_port->mode = mode;
alt_port->hpd_state = hpd_state;
alt_port->hpd_irq = hpd_irq;
- schedule_work(&alt_port->work);
+ queue_work(system_freezable_wq, &alt_port->work);
}
#define SC8280XP_DPAM_MASK 0x3f
@@ -346,7 +346,7 @@ static void pmic_glink_altmode_sc8280xp_notify(struct pmic_glink_altmode *altmod
alt_port->mode = mode;
alt_port->hpd_state = hpd_state;
alt_port->hpd_irq = hpd_irq;
- schedule_work(&alt_port->work);
+ queue_work(system_freezable_wq, &alt_port->work);
}
static void pmic_glink_altmode_callback(const void *data, size_t len, void *priv)
diff --git a/drivers/spi/atmel-quadspi.c b/drivers/spi/atmel-quadspi.c
index d7a3d85d00c2..d3c14e2ec8c8 100644
--- a/drivers/spi/atmel-quadspi.c
+++ b/drivers/spi/atmel-quadspi.c
@@ -256,8 +256,11 @@ static const struct atmel_qspi_pcal pcal[ATMEL_QSPI_PCAL_ARRAY_SIZE] = {
{200000000, 7},
};
+struct atmel_qspi;
+
struct atmel_qspi_caps {
u32 max_speed_hz;
+ int (*init)(struct atmel_qspi *aq);
bool has_qspick;
bool has_gclk;
bool has_ricr;
@@ -1156,7 +1159,7 @@ static int atmel_qspi_sama7g5_setup(struct spi_device *spi)
/* The controller can communicate with a single peripheral device (target). */
aq->target_max_speed_hz = spi->max_speed_hz;
- return atmel_qspi_sama7g5_init(aq);
+ return aq->caps->init(aq);
}
static int atmel_qspi_setup(struct spi_device *spi)
@@ -1540,12 +1543,14 @@ static int __maybe_unused atmel_qspi_suspend(struct device *dev)
if (aq->caps->has_gclk) {
ret = atmel_qspi_sama7g5_suspend(aq);
- clk_disable_unprepare(aq->pclk);
- return ret;
+ if (ret) {
+ pm_runtime_put_autosuspend(dev);
+ return ret;
+ }
+ } else {
+ atmel_qspi_write(QSPI_CR_QSPIDIS, aq, QSPI_CR);
}
- atmel_qspi_write(QSPI_CR_QSPIDIS, aq, QSPI_CR);
-
pm_runtime_mark_last_busy(dev);
pm_runtime_force_suspend(dev);
@@ -1571,20 +1576,20 @@ static int __maybe_unused atmel_qspi_resume(struct device *dev)
return ret;
}
- if (aq->caps->has_gclk)
- return atmel_qspi_sama7g5_init(aq);
-
ret = pm_runtime_force_resume(dev);
if (ret < 0)
return ret;
- atmel_qspi_init(aq);
-
- atmel_qspi_write(aq->scr, aq, QSPI_SCR);
+ if (aq->caps->has_gclk) {
+ ret = aq->caps->init(aq);
+ } else {
+ atmel_qspi_init(aq);
+ atmel_qspi_write(aq->scr, aq, QSPI_SCR);
+ }
pm_runtime_put_autosuspend(dev);
- return 0;
+ return ret;
}
static int __maybe_unused atmel_qspi_runtime_suspend(struct device *dev)
@@ -1630,6 +1635,7 @@ static const struct atmel_qspi_caps atmel_sam9x60_qspi_caps = {
static const struct atmel_qspi_caps atmel_sam9x7_ospi_caps = {
.max_speed_hz = SAM9X7_QSPI_MAX_SPEED_HZ,
+ .init = atmel_qspi_sama7g5_init,
.has_gclk = true,
.octal = true,
.has_dma = true,
@@ -1640,6 +1646,7 @@ static const struct atmel_qspi_caps atmel_sam9x7_ospi_caps = {
static const struct atmel_qspi_caps atmel_sama7d65_ospi_caps = {
.max_speed_hz = SAMA7G5_QSPI0_MAX_SPEED_HZ,
+ .init = atmel_qspi_sama7g5_init,
.has_gclk = true,
.octal = true,
.has_dma = true,
@@ -1650,6 +1657,7 @@ static const struct atmel_qspi_caps atmel_sama7d65_ospi_caps = {
static const struct atmel_qspi_caps atmel_sama7d65_qspi_caps = {
.max_speed_hz = SAMA7G5_QSPI1_SDR_MAX_SPEED_HZ,
+ .init = atmel_qspi_sama7g5_init,
.has_gclk = true,
.has_dma = true,
.has_2xgclk = true,
@@ -1658,6 +1666,7 @@ static const struct atmel_qspi_caps atmel_sama7d65_qspi_caps = {
static const struct atmel_qspi_caps atmel_sama7g5_ospi_caps = {
.max_speed_hz = SAMA7G5_QSPI0_MAX_SPEED_HZ,
+ .init = atmel_qspi_sama7g5_init,
.has_gclk = true,
.octal = true,
.has_dma = true,
@@ -1667,6 +1676,7 @@ static const struct atmel_qspi_caps atmel_sama7g5_ospi_caps = {
static const struct atmel_qspi_caps atmel_sama7g5_qspi_caps = {
.max_speed_hz = SAMA7G5_QSPI1_SDR_MAX_SPEED_HZ,
+ .init = atmel_qspi_sama7g5_init,
.has_gclk = true,
.has_dma = true,
.has_dllon = true,
diff --git a/drivers/spi/spi-cadence-xspi.c b/drivers/spi/spi-cadence-xspi.c
index c3676b21e816..ca6af39e977a 100644
--- a/drivers/spi/spi-cadence-xspi.c
+++ b/drivers/spi/spi-cadence-xspi.c
@@ -2,7 +2,6 @@
// Cadence XSPI flash controller driver
// Copyright (C) 2020-21 Cadence
-#include <linux/acpi.h>
#include <linux/completion.h>
#include <linux/delay.h>
#include <linux/err.h>
@@ -21,6 +20,7 @@
#include <linux/limits.h>
#include <linux/log2.h>
#include <linux/bitrev.h>
+#include <linux/util_macros.h>
#define CDNS_XSPI_MAGIC_NUM_VALUE 0x6522
#define CDNS_XSPI_MAX_BANKS 8
@@ -316,39 +316,10 @@ struct cdns_xspi_driver_data {
u32 rfile_phy_dll_slave_ctrl;
};
-static struct cdns_xspi_driver_data marvell_driver_data = {
- .mrvl_hw_overlay = true,
- .dll_phy_ctrl = MARVELL_REGS_DLL_PHY_CTRL,
- .ctb_rfile_phy_ctrl = MARVELL_CTB_RFILE_PHY_CTRL,
- .rfile_phy_tsel = MARVELL_RFILE_PHY_TSEL,
- .rfile_phy_dq_timing = MARVELL_RFILE_PHY_DQ_TIMING,
- .rfile_phy_dqs_timing = MARVELL_RFILE_PHY_DQS_TIMING,
- .rfile_phy_gate_lpbk_ctrl = MARVELL_RFILE_PHY_GATE_LPBK_CTRL,
- .rfile_phy_dll_master_ctrl = MARVELL_RFILE_PHY_DLL_MASTER_CTRL,
- .rfile_phy_dll_slave_ctrl = MARVELL_RFILE_PHY_DLL_SLAVE_CTRL,
-};
-
static struct cdns_xspi_driver_data cdns_driver_data = {
.mrvl_hw_overlay = false,
};
-static const int cdns_mrvl_xspi_clk_div_list[] = {
- 4, //0x0 = Divide by 4. SPI clock is 200 MHz.
- 6, //0x1 = Divide by 6. SPI clock is 133.33 MHz.
- 8, //0x2 = Divide by 8. SPI clock is 100 MHz.
- 10, //0x3 = Divide by 10. SPI clock is 80 MHz.
- 12, //0x4 = Divide by 12. SPI clock is 66.666 MHz.
- 16, //0x5 = Divide by 16. SPI clock is 50 MHz.
- 18, //0x6 = Divide by 18. SPI clock is 44.44 MHz.
- 20, //0x7 = Divide by 20. SPI clock is 40 MHz.
- 24, //0x8 = Divide by 24. SPI clock is 33.33 MHz.
- 32, //0x9 = Divide by 32. SPI clock is 25 MHz.
- 40, //0xA = Divide by 40. SPI clock is 20 MHz.
- 50, //0xB = Divide by 50. SPI clock is 16 MHz.
- 64, //0xC = Divide by 64. SPI clock is 12.5 MHz.
- 128 //0xD = Divide by 128. SPI clock is 6.25 MHz.
-};
-
struct cdns_xspi_dev {
struct platform_device *pdev;
struct device *dev;
@@ -361,6 +332,7 @@ struct cdns_xspi_dev {
int irq;
int cur_cs;
unsigned int sdmasize;
+ unsigned int sdma_xfer_len;
struct completion cmd_complete;
struct completion auto_cmd_complete;
@@ -373,100 +345,13 @@ struct cdns_xspi_dev {
u8 hw_num_banks;
const struct cdns_xspi_driver_data *driver_data;
- void (*sdma_handler)(struct cdns_xspi_dev *cdns_xspi);
+ int (*sdma_handler)(struct cdns_xspi_dev *cdns_xspi);
void (*set_interrupts_handler)(struct cdns_xspi_dev *cdns_xspi, bool enabled);
bool xfer_in_progress;
int current_xfer_qword;
};
-static void cdns_xspi_reset_dll(struct cdns_xspi_dev *cdns_xspi)
-{
- u32 dll_cntrl = readl(cdns_xspi->iobase +
- CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL);
-
- /* Reset DLL */
- dll_cntrl |= CDNS_XSPI_DLL_RST_N;
- writel(dll_cntrl, cdns_xspi->iobase +
- CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL);
-}
-
-static bool cdns_xspi_is_dll_locked(struct cdns_xspi_dev *cdns_xspi)
-{
- u32 dll_lock;
-
- return !readl_relaxed_poll_timeout(cdns_xspi->iobase +
- CDNS_XSPI_INTR_STATUS_REG,
- dll_lock, ((dll_lock & CDNS_XSPI_DLL_LOCK) == 1), 10, 10000);
-}
-
-/* Static configuration of PHY */
-static bool cdns_xspi_configure_phy(struct cdns_xspi_dev *cdns_xspi)
-{
- writel(cdns_xspi->driver_data->dll_phy_ctrl,
- cdns_xspi->iobase + CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL);
- writel(cdns_xspi->driver_data->ctb_rfile_phy_ctrl,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_CTB_RFILE_PHY_CTRL);
- writel(cdns_xspi->driver_data->rfile_phy_tsel,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_CTB_RFILE_PHY_TSEL);
- writel(cdns_xspi->driver_data->rfile_phy_dq_timing,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DQ_TIMING);
- writel(cdns_xspi->driver_data->rfile_phy_dqs_timing,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DQS_TIMING);
- writel(cdns_xspi->driver_data->rfile_phy_gate_lpbk_ctrl,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_GATE_LPBK_CTRL);
- writel(cdns_xspi->driver_data->rfile_phy_dll_master_ctrl,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DLL_MASTER_CTRL);
- writel(cdns_xspi->driver_data->rfile_phy_dll_slave_ctrl,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DLL_SLAVE_CTRL);
-
- cdns_xspi_reset_dll(cdns_xspi);
-
- return cdns_xspi_is_dll_locked(cdns_xspi);
-}
-
-static bool cdns_mrvl_xspi_setup_clock(struct cdns_xspi_dev *cdns_xspi,
- int requested_clk)
-{
- int i = 0;
- int clk_val;
- u32 clk_reg;
- bool update_clk = false;
-
- while (i < (ARRAY_SIZE(cdns_mrvl_xspi_clk_div_list) - 1)) {
- clk_val = MRVL_XSPI_CLOCK_DIVIDED(
- cdns_mrvl_xspi_clk_div_list[i]);
- if (clk_val <= requested_clk)
- break;
- i++;
- }
-
- dev_dbg(cdns_xspi->dev, "Found clk div: %d, clk val: %d\n",
- cdns_mrvl_xspi_clk_div_list[i],
- MRVL_XSPI_CLOCK_DIVIDED(
- cdns_mrvl_xspi_clk_div_list[i]));
-
- clk_reg = readl(cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG);
-
- if (FIELD_GET(MRVL_XSPI_CLK_DIV, clk_reg) != i) {
- clk_reg &= ~MRVL_XSPI_CLK_ENABLE;
- writel(clk_reg,
- cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG);
- clk_reg = FIELD_PREP(MRVL_XSPI_CLK_DIV, i);
- clk_reg &= ~MRVL_XSPI_CLK_DIV;
- clk_reg |= FIELD_PREP(MRVL_XSPI_CLK_DIV, i);
- clk_reg |= MRVL_XSPI_CLK_ENABLE;
- clk_reg |= MRVL_XSPI_IRQ_ENABLE;
- update_clk = true;
- }
-
- if (update_clk)
- writel(clk_reg,
- cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG);
-
- return update_clk;
-}
-
static int cdns_xspi_wait_for_controller_idle(struct cdns_xspi_dev *cdns_xspi)
{
u32 ctrl_stat;
@@ -539,23 +424,6 @@ static void cdns_xspi_set_interrupts(struct cdns_xspi_dev *cdns_xspi,
writel(intr_enable, cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG);
}
-static void marvell_xspi_set_interrupts(struct cdns_xspi_dev *cdns_xspi,
- bool enabled)
-{
- u32 intr_enable;
- u32 irq_status;
-
- irq_status = readl(cdns_xspi->iobase + CDNS_XSPI_INTR_STATUS_REG);
- writel(irq_status, cdns_xspi->iobase + CDNS_XSPI_INTR_STATUS_REG);
-
- intr_enable = readl(cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG);
- if (enabled)
- intr_enable |= CDNS_XSPI_INTR_MASK;
- else
- intr_enable &= ~CDNS_XSPI_INTR_MASK;
- writel(intr_enable, cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG);
-}
-
static int cdns_xspi_controller_init(struct cdns_xspi_dev *cdns_xspi)
{
u32 ctrl_ver;
@@ -578,7 +446,7 @@ static int cdns_xspi_controller_init(struct cdns_xspi_dev *cdns_xspi)
return 0;
}
-static void cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
+static int cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
{
u32 sdma_size, sdma_trd_info;
u8 sdma_dir;
@@ -587,6 +455,13 @@ static void cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
sdma_trd_info = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_TRD_INFO_REG);
sdma_dir = FIELD_GET(CDNS_XSPI_SDMA_DIR, sdma_trd_info);
+ if (sdma_size > cdns_xspi->sdma_xfer_len) {
+ dev_err(cdns_xspi->dev,
+ "SDMA size %u exceeds the requested length %u\n",
+ sdma_size, cdns_xspi->sdma_xfer_len);
+ return -EINVAL;
+ }
+
switch (sdma_dir) {
case CDNS_XSPI_SDMA_DIR_READ:
ioread8_rep(cdns_xspi->sdmabase,
@@ -598,78 +473,8 @@ static void cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
cdns_xspi->out_buffer, sdma_size);
break;
}
-}
-
-static void m_ioreadq(void __iomem *addr, void *buf, int len)
-{
- if (IS_ALIGNED((long)buf, 8) && len >= 8) {
- u64 full_ops = len / 8;
- u64 *buffer = buf;
-
- len -= full_ops * 8;
- buf += full_ops * 8;
-
- do {
- u64 b = readq(addr);
- *buffer++ = b;
- } while (--full_ops);
- }
-
-
- while (len) {
- u64 tmp_buf;
-
- tmp_buf = readq(addr);
- memcpy(buf, &tmp_buf, min(len, 8));
- len = len > 8 ? len - 8 : 0;
- buf += 8;
- }
-}
-
-static void m_iowriteq(void __iomem *addr, const void *buf, int len)
-{
- if (IS_ALIGNED((long)buf, 8) && len >= 8) {
- u64 full_ops = len / 8;
- const u64 *buffer = buf;
-
- len -= full_ops * 8;
- buf += full_ops * 8;
-
- do {
- writeq(*buffer++, addr);
- } while (--full_ops);
- }
- while (len) {
- u64 tmp_buf;
-
- memcpy(&tmp_buf, buf, min(len, 8));
- writeq(tmp_buf, addr);
- len = len > 8 ? len - 8 : 0;
- buf += 8;
- }
-}
-
-static void marvell_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
-{
- u32 sdma_size, sdma_trd_info;
- u8 sdma_dir;
-
- sdma_size = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_SIZE_REG);
- sdma_trd_info = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_TRD_INFO_REG);
- sdma_dir = FIELD_GET(CDNS_XSPI_SDMA_DIR, sdma_trd_info);
-
- switch (sdma_dir) {
- case CDNS_XSPI_SDMA_DIR_READ:
- m_ioreadq(cdns_xspi->sdmabase,
- cdns_xspi->in_buffer, sdma_size);
- break;
-
- case CDNS_XSPI_SDMA_DIR_WRITE:
- m_iowriteq(cdns_xspi->sdmabase,
- cdns_xspi->out_buffer, sdma_size);
- break;
- }
+ return 0;
}
static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi,
@@ -715,6 +520,7 @@ static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi,
cdns_xspi->in_buffer = op->data.buf.in;
cdns_xspi->out_buffer = op->data.buf.out;
+ cdns_xspi->sdma_xfer_len = op->data.nbytes;
cdns_xspi_trigger_command(cdns_xspi, cmd_regs);
@@ -723,7 +529,11 @@ static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi,
cdns_xspi->set_interrupts_handler(cdns_xspi, false);
return -EIO;
}
- cdns_xspi->sdma_handler(cdns_xspi);
+ ret = cdns_xspi->sdma_handler(cdns_xspi);
+ if (ret) {
+ cdns_xspi->set_interrupts_handler(cdns_xspi, false);
+ return ret;
+ }
}
wait_for_completion(&cdns_xspi->cmd_complete);
@@ -761,33 +571,15 @@ static int cdns_xspi_mem_op_execute(struct spi_mem *mem,
return ret;
}
-static int marvell_xspi_mem_op_execute(struct spi_mem *mem,
- const struct spi_mem_op *op)
-{
- struct cdns_xspi_dev *cdns_xspi =
- spi_controller_get_devdata(mem->spi->controller);
- int ret = 0;
-
- cdns_mrvl_xspi_setup_clock(cdns_xspi, mem->spi->max_speed_hz);
-
- ret = cdns_xspi_mem_op(cdns_xspi, mem, op);
-
- return ret;
-}
-
-#ifdef CONFIG_ACPI
static bool cdns_xspi_supports_op(struct spi_mem *mem,
const struct spi_mem_op *op)
{
struct spi_device *spi = mem->spi;
- const union acpi_object *obj;
- struct acpi_device *adev;
+ struct device *dev = &spi->dev;
+ u32 value;
- adev = ACPI_COMPANION(&spi->dev);
-
- if (!acpi_dev_get_property(adev, "spi-tx-bus-width", ACPI_TYPE_INTEGER,
- &obj)) {
- switch (obj->integer.value) {
+ if (!device_property_read_u32(dev, "spi-tx-bus-width", &value)) {
+ switch (value) {
case 1:
break;
case 2:
@@ -800,16 +592,13 @@ static bool cdns_xspi_supports_op(struct spi_mem *mem,
spi->mode |= SPI_TX_OCTAL;
break;
default:
- dev_warn(&spi->dev,
- "spi-tx-bus-width %lld not supported\n",
- obj->integer.value);
+ dev_warn(dev, "spi-tx-bus-width %u not supported\n", value);
break;
}
}
- if (!acpi_dev_get_property(adev, "spi-rx-bus-width", ACPI_TYPE_INTEGER,
- &obj)) {
- switch (obj->integer.value) {
+ if (!device_property_read_u32(dev, "spi-rx-bus-width", &value)) {
+ switch (value) {
case 1:
break;
case 2:
@@ -822,9 +611,7 @@ static bool cdns_xspi_supports_op(struct spi_mem *mem,
spi->mode |= SPI_RX_OCTAL;
break;
default:
- dev_warn(&spi->dev,
- "spi-rx-bus-width %lld not supported\n",
- obj->integer.value);
+ dev_warn(dev, "spi-rx-bus-width %u not supported\n", value);
break;
}
}
@@ -834,7 +621,6 @@ static bool cdns_xspi_supports_op(struct spi_mem *mem,
return true;
}
-#endif
static int cdns_xspi_adjust_mem_op_size(struct spi_mem *mem, struct spi_mem_op *op)
{
@@ -847,21 +633,11 @@ static int cdns_xspi_adjust_mem_op_size(struct spi_mem *mem, struct spi_mem_op *
}
static const struct spi_controller_mem_ops cadence_xspi_mem_ops = {
-#ifdef CONFIG_ACPI
- .supports_op = cdns_xspi_supports_op,
-#endif
+ .supports_op = PTR_IF(IS_ENABLED(CONFIG_ACPI), cdns_xspi_supports_op),
.exec_op = cdns_xspi_mem_op_execute,
.adjust_op_size = cdns_xspi_adjust_mem_op_size,
};
-static const struct spi_controller_mem_ops marvell_xspi_mem_ops = {
-#ifdef CONFIG_ACPI
- .supports_op = cdns_xspi_supports_op,
-#endif
- .exec_op = marvell_xspi_mem_op_execute,
- .adjust_op_size = cdns_xspi_adjust_mem_op_size,
-};
-
static irqreturn_t cdns_xspi_irq_handler(int this_irq, void *dev)
{
struct cdns_xspi_dev *cdns_xspi = dev;
@@ -943,6 +719,239 @@ static void cdns_xspi_print_phy_config(struct cdns_xspi_dev *cdns_xspi)
readl(cdns_xspi->auxbase + CDNS_XSPI_CCP_PHY_DLL_SLAVE_CTRL));
}
+static struct cdns_xspi_driver_data marvell_driver_data = {
+ .mrvl_hw_overlay = true,
+ .dll_phy_ctrl = MARVELL_REGS_DLL_PHY_CTRL,
+ .ctb_rfile_phy_ctrl = MARVELL_CTB_RFILE_PHY_CTRL,
+ .rfile_phy_tsel = MARVELL_RFILE_PHY_TSEL,
+ .rfile_phy_dq_timing = MARVELL_RFILE_PHY_DQ_TIMING,
+ .rfile_phy_dqs_timing = MARVELL_RFILE_PHY_DQS_TIMING,
+ .rfile_phy_gate_lpbk_ctrl = MARVELL_RFILE_PHY_GATE_LPBK_CTRL,
+ .rfile_phy_dll_master_ctrl = MARVELL_RFILE_PHY_DLL_MASTER_CTRL,
+ .rfile_phy_dll_slave_ctrl = MARVELL_RFILE_PHY_DLL_SLAVE_CTRL,
+};
+
+static const int cdns_mrvl_xspi_clk_div_list[] = {
+ 4, //0x0 = Divide by 4. SPI clock is 200 MHz.
+ 6, //0x1 = Divide by 6. SPI clock is 133.33 MHz.
+ 8, //0x2 = Divide by 8. SPI clock is 100 MHz.
+ 10, //0x3 = Divide by 10. SPI clock is 80 MHz.
+ 12, //0x4 = Divide by 12. SPI clock is 66.666 MHz.
+ 16, //0x5 = Divide by 16. SPI clock is 50 MHz.
+ 18, //0x6 = Divide by 18. SPI clock is 44.44 MHz.
+ 20, //0x7 = Divide by 20. SPI clock is 40 MHz.
+ 24, //0x8 = Divide by 24. SPI clock is 33.33 MHz.
+ 32, //0x9 = Divide by 32. SPI clock is 25 MHz.
+ 40, //0xA = Divide by 40. SPI clock is 20 MHz.
+ 50, //0xB = Divide by 50. SPI clock is 16 MHz.
+ 64, //0xC = Divide by 64. SPI clock is 12.5 MHz.
+ 128 //0xD = Divide by 128. SPI clock is 6.25 MHz.
+};
+
+static void cdns_xspi_reset_dll(struct cdns_xspi_dev *cdns_xspi)
+{
+ u32 dll_cntrl = readl(cdns_xspi->iobase +
+ CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL);
+
+ /* Reset DLL */
+ dll_cntrl |= CDNS_XSPI_DLL_RST_N;
+ writel(dll_cntrl, cdns_xspi->iobase +
+ CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL);
+}
+
+static bool cdns_xspi_is_dll_locked(struct cdns_xspi_dev *cdns_xspi)
+{
+ u32 dll_lock;
+
+ return !readl_relaxed_poll_timeout(cdns_xspi->iobase +
+ CDNS_XSPI_INTR_STATUS_REG,
+ dll_lock, ((dll_lock & CDNS_XSPI_DLL_LOCK) == 1), 10, 10000);
+}
+
+/* Static configuration of PHY */
+static bool cdns_xspi_configure_phy(struct cdns_xspi_dev *cdns_xspi)
+{
+ writel(cdns_xspi->driver_data->dll_phy_ctrl,
+ cdns_xspi->iobase + CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL);
+ writel(cdns_xspi->driver_data->ctb_rfile_phy_ctrl,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_CTB_RFILE_PHY_CTRL);
+ writel(cdns_xspi->driver_data->rfile_phy_tsel,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_CTB_RFILE_PHY_TSEL);
+ writel(cdns_xspi->driver_data->rfile_phy_dq_timing,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DQ_TIMING);
+ writel(cdns_xspi->driver_data->rfile_phy_dqs_timing,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DQS_TIMING);
+ writel(cdns_xspi->driver_data->rfile_phy_gate_lpbk_ctrl,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_GATE_LPBK_CTRL);
+ writel(cdns_xspi->driver_data->rfile_phy_dll_master_ctrl,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DLL_MASTER_CTRL);
+ writel(cdns_xspi->driver_data->rfile_phy_dll_slave_ctrl,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DLL_SLAVE_CTRL);
+
+ cdns_xspi_reset_dll(cdns_xspi);
+
+ return cdns_xspi_is_dll_locked(cdns_xspi);
+}
+
+static bool cdns_mrvl_xspi_setup_clock(struct cdns_xspi_dev *cdns_xspi,
+ int requested_clk)
+{
+ int i = 0;
+ int clk_val;
+ u32 clk_reg;
+ bool update_clk = false;
+
+ while (i < (ARRAY_SIZE(cdns_mrvl_xspi_clk_div_list) - 1)) {
+ clk_val = MRVL_XSPI_CLOCK_DIVIDED(
+ cdns_mrvl_xspi_clk_div_list[i]);
+ if (clk_val <= requested_clk)
+ break;
+ i++;
+ }
+
+ dev_dbg(cdns_xspi->dev, "Found clk div: %d, clk val: %d\n",
+ cdns_mrvl_xspi_clk_div_list[i],
+ MRVL_XSPI_CLOCK_DIVIDED(
+ cdns_mrvl_xspi_clk_div_list[i]));
+
+ clk_reg = readl(cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG);
+
+ if (FIELD_GET(MRVL_XSPI_CLK_DIV, clk_reg) != i) {
+ clk_reg &= ~MRVL_XSPI_CLK_ENABLE;
+ writel(clk_reg,
+ cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG);
+ clk_reg = FIELD_PREP(MRVL_XSPI_CLK_DIV, i);
+ FIELD_MODIFY(MRVL_XSPI_CLK_DIV, &clk_reg, i);
+ clk_reg |= MRVL_XSPI_CLK_ENABLE;
+ clk_reg |= MRVL_XSPI_IRQ_ENABLE;
+ update_clk = true;
+ }
+
+ if (update_clk)
+ writel(clk_reg,
+ cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG);
+
+ return update_clk;
+}
+
+static void marvell_xspi_set_interrupts(struct cdns_xspi_dev *cdns_xspi,
+ bool enabled)
+{
+ u32 intr_enable;
+ u32 irq_status;
+
+ irq_status = readl(cdns_xspi->iobase + CDNS_XSPI_INTR_STATUS_REG);
+ writel(irq_status, cdns_xspi->iobase + CDNS_XSPI_INTR_STATUS_REG);
+
+ intr_enable = readl(cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG);
+ if (enabled)
+ intr_enable |= CDNS_XSPI_INTR_MASK;
+ else
+ intr_enable &= ~CDNS_XSPI_INTR_MASK;
+ writel(intr_enable, cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG);
+}
+
+static int marvell_xspi_mem_op_execute(struct spi_mem *mem,
+ const struct spi_mem_op *op)
+{
+ struct cdns_xspi_dev *cdns_xspi =
+ spi_controller_get_devdata(mem->spi->controller);
+ int ret = 0;
+
+ cdns_mrvl_xspi_setup_clock(cdns_xspi, mem->spi->max_speed_hz);
+
+ ret = cdns_xspi_mem_op(cdns_xspi, mem, op);
+
+ return ret;
+}
+
+static void m_ioreadq(void __iomem *addr, void *buf, int len)
+{
+ if (IS_ALIGNED((long)buf, 8) && len >= 8) {
+ u64 full_ops = len / 8;
+ u64 *buffer = buf;
+
+ len -= full_ops * 8;
+ buf += full_ops * 8;
+
+ do {
+ u64 b = readq(addr);
+ *buffer++ = b;
+ } while (--full_ops);
+ }
+
+
+ while (len) {
+ u64 tmp_buf;
+
+ tmp_buf = readq(addr);
+ memcpy(buf, &tmp_buf, min(len, 8));
+ len = len > 8 ? len - 8 : 0;
+ buf += 8;
+ }
+}
+
+static void m_iowriteq(void __iomem *addr, const void *buf, int len)
+{
+ if (IS_ALIGNED((long)buf, 8) && len >= 8) {
+ u64 full_ops = len / 8;
+ const u64 *buffer = buf;
+
+ len -= full_ops * 8;
+ buf += full_ops * 8;
+
+ do {
+ writeq(*buffer++, addr);
+ } while (--full_ops);
+ }
+
+ while (len) {
+ u64 tmp_buf;
+
+ memcpy(&tmp_buf, buf, min(len, 8));
+ writeq(tmp_buf, addr);
+ len = len > 8 ? len - 8 : 0;
+ buf += 8;
+ }
+}
+
+static int marvell_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
+{
+ u32 sdma_size, sdma_trd_info;
+ u8 sdma_dir;
+
+ sdma_size = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_SIZE_REG);
+ sdma_trd_info = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_TRD_INFO_REG);
+ sdma_dir = FIELD_GET(CDNS_XSPI_SDMA_DIR, sdma_trd_info);
+
+ if (sdma_size > cdns_xspi->sdma_xfer_len) {
+ dev_err(cdns_xspi->dev,
+ "SDMA size %u exceeds the requested length %u\n",
+ sdma_size, cdns_xspi->sdma_xfer_len);
+ return -EINVAL;
+ }
+
+ switch (sdma_dir) {
+ case CDNS_XSPI_SDMA_DIR_READ:
+ m_ioreadq(cdns_xspi->sdmabase,
+ cdns_xspi->in_buffer, sdma_size);
+ break;
+
+ case CDNS_XSPI_SDMA_DIR_WRITE:
+ m_iowriteq(cdns_xspi->sdmabase,
+ cdns_xspi->out_buffer, sdma_size);
+ break;
+ }
+
+ return 0;
+}
+
+static const struct spi_controller_mem_ops marvell_xspi_mem_ops = {
+ .supports_op = PTR_IF(IS_ENABLED(CONFIG_ACPI), cdns_xspi_supports_op),
+ .exec_op = marvell_xspi_mem_op_execute,
+ .adjust_op_size = cdns_xspi_adjust_mem_op_size,
+};
+
static int cdns_xspi_prepare_generic(int cs, const void *dout, int len, int glue, u32 *cmd_regs)
{
u8 *data = (u8 *)dout;
@@ -1099,9 +1108,11 @@ static int cdns_xspi_transfer_one_message_b0(struct spi_controller *controller,
cdns_xspi_prepare_transfer(cs, 1, current_transfer_len - 1,
cmd_regs);
cdns_xspi_trigger_command(cdns_xspi, cmd_regs);
+ cdns_xspi->sdma_xfer_len = current_transfer_len - 1;
if (!cdns_xspi_is_sdma_ready(cdns_xspi, true))
return -EIO;
- cdns_xspi->sdma_handler(cdns_xspi);
+ if (cdns_xspi->sdma_handler(cdns_xspi))
+ return -EIO;
if (!cdns_xspi_is_stig_ready(cdns_xspi, true))
return -EIO;
@@ -1170,15 +1181,14 @@ static int cdns_xspi_probe(struct platform_device *pdev)
return -ENODEV;
}
+ host->mem_ops = &cadence_xspi_mem_ops;
+ cdns_xspi->sdma_handler = &cdns_xspi_sdma_handle;
+ cdns_xspi->set_interrupts_handler = &cdns_xspi_set_interrupts;
if (cdns_xspi->driver_data->mrvl_hw_overlay) {
host->mem_ops = &marvell_xspi_mem_ops;
host->transfer_one_message = cdns_xspi_transfer_one_message_b0;
cdns_xspi->sdma_handler = &marvell_xspi_sdma_handle;
cdns_xspi->set_interrupts_handler = &marvell_xspi_set_interrupts;
- } else {
- host->mem_ops = &cadence_xspi_mem_ops;
- cdns_xspi->sdma_handler = &cdns_xspi_sdma_handle;
- cdns_xspi->set_interrupts_handler = &cdns_xspi_set_interrupts;
}
host->dev.of_node = pdev->dev.of_node;
host->bus_num = -1;
diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 7bb711b24c0d..91ba88f9934d 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -349,6 +349,19 @@ int core_enable_device_list_for_node(
new->attach_count++;
mutex_lock(&nacl->lun_entry_mutex);
+ /*
+ * Do not attach a new dev entry to an se_lun that is being torn down.
+ * lun_shutdown is set by core_tpg_remove_lun() before its
+ * core_clear_lun_from_tpg() sweep; checking it here under the same
+ * nacl->lun_entry_mutex the sweep takes guarantees that once the sweep
+ * has passed this ACL no dev entry can be left dangling on the
+ * se_lun->lun_deve_list that is about to be freed.
+ */
+ if (lun->lun_shutdown) {
+ mutex_unlock(&nacl->lun_entry_mutex);
+ ret = -ENODEV;
+ goto free_stats;
+ }
orig = target_nacl_find_deve(nacl, mapped_lun);
if (orig && orig->se_lun) {
struct se_lun *orig_lun = orig->se_lun;
@@ -375,12 +388,12 @@ int core_enable_device_list_for_node(
new->se_lun_acl = lun_acl;
hlist_del_rcu(&orig->link);
hlist_add_head_rcu(&new->link, &nacl->lun_entry_hlist);
- mutex_unlock(&nacl->lun_entry_mutex);
spin_lock(&lun->lun_deve_lock);
list_del(&orig->lun_link);
list_add_tail(&new->lun_link, &lun->lun_deve_list);
spin_unlock(&lun->lun_deve_lock);
+ mutex_unlock(&nacl->lun_entry_mutex);
kref_put(&orig->pr_kref, target_pr_kref_release);
wait_for_completion(&orig->pr_comp);
@@ -393,11 +406,10 @@ int core_enable_device_list_for_node(
new->se_lun = lun;
new->se_lun_acl = lun_acl;
hlist_add_head_rcu(&new->link, &nacl->lun_entry_hlist);
- mutex_unlock(&nacl->lun_entry_mutex);
-
spin_lock(&lun->lun_deve_lock);
list_add_tail(&new->lun_link, &lun->lun_deve_list);
spin_unlock(&lun->lun_deve_lock);
+ mutex_unlock(&nacl->lun_entry_mutex);
target_luns_data_has_changed(nacl, new, true);
return 0;
diff --git a/drivers/target/target_core_fabric_lib.c b/drivers/target/target_core_fabric_lib.c
index 3af9a26cdefc..a1ee106971cb 100644
--- a/drivers/target/target_core_fabric_lib.c
+++ b/drivers/target/target_core_fabric_lib.c
@@ -370,6 +370,16 @@ static bool iscsi_parse_pr_out_transport_id(
pr_err("Missing ISID in iSCSI Initiator port TransportID\n");
return false;
}
+ /*
+ * The registration code reads the ISID as a C string,
+ * and its first 8 bytes with get_unaligned_be64(), so
+ * require 8 bytes after the separator and a NUL inside
+ * the rest of the descriptor.
+ */
+ if (buf + tid_len - p < 8 || !memchr(p, '\0', buf + tid_len - p)) {
+ pr_err("Incomplete or unterminated ISID in iSCSI Initiator port TransportID\n");
+ return false;
+ }
*port_nexus_ptr = p;
/*
* Go ahead and do the lower case conversion of the received
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index c0e429e5ef31..d8f4538e48dc 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -266,6 +266,15 @@ struct se_node_acl *core_tpg_check_initiator_node_acl(
kref_get(&acl->acl_kref);
acl->dynamic_node_acl = 1;
+ /*
+ * Publish the dynamic NodeACL on tpg->acl_node_list before creating
+ * its demo-mode MappedLUN dev entries below, so that a concurrent LUN
+ * teardown sweep (core_clear_lun_from_tpg(), which walks acl_node_list)
+ * can observe and disable those dev entries. Otherwise a dev entry can
+ * be left on an se_lun->lun_deve_list that is about to be freed.
+ */
+ target_add_node_acl(acl);
+
/*
* Here we only create demo-mode MappedLUNs from the active
* TPG LUNs if the fabric is not explicitly asking for
@@ -275,7 +284,6 @@ struct se_node_acl *core_tpg_check_initiator_node_acl(
(tpg->se_tpg_tfo->tpg_check_demo_mode_login_only(tpg) != 1))
core_tpg_add_node_to_devs(acl, tpg, NULL);
- target_add_node_acl(acl);
return acl;
}
EXPORT_SYMBOL(core_tpg_check_initiator_node_acl);
diff --git a/drivers/target/target_core_user.c b/drivers/target/target_core_user.c
index 3fd963612775..72878700b1a6 100644
--- a/drivers/target/target_core_user.c
+++ b/drivers/target/target_core_user.c
@@ -2514,6 +2514,11 @@ static int tcmu_set_data_pages_per_blk(struct tcmu_dev *udev, substring_t *arg)
return ret;
}
+ if (val <= 0) {
+ pr_err("Invalid data_pages_per_blk %d.\n", val);
+ return -EINVAL;
+ }
+
if (val > TCMU_MBS_TO_PAGES(udev->data_area_mb)) {
pr_err("Invalid data_pages_per_blk %d: greater than max_data_area_mb %d -> %zd pages).\n",
val, udev->data_area_mb,
diff --git a/drivers/tty/serial/sc16is7xx.c b/drivers/tty/serial/sc16is7xx.c
index 42ad926f1fe6..870b27973a7b 100644
--- a/drivers/tty/serial/sc16is7xx.c
+++ b/drivers/tty/serial/sc16is7xx.c
@@ -538,10 +538,13 @@ EXPORT_SYMBOL_GPL(sc16is762_devtype);
static bool sc16is7xx_regmap_volatile(struct device *dev, unsigned int reg)
{
switch (reg) {
- case SC16IS7XX_RHR_REG:
- case SC16IS7XX_IIR_REG:
- case SC16IS7XX_LSR_REG:
- case SC16IS7XX_MSR_REG:
+ case SC16IS7XX_RHR_REG: /* Shared address space with THR & DLL */
+ case SC16IS7XX_IER_REG: /* Shared address space with DLH */
+ case SC16IS7XX_IIR_REG: /* Shared address space with FCR & EFR */
+ case SC16IS7XX_MCR_REG: /* Shared address space with XON1 */
+ case SC16IS7XX_LSR_REG: /* Shared address space with XON2 */
+ case SC16IS7XX_MSR_REG: /* Shared address space with TCR & XOFF1 */
+ case SC16IS7XX_SPR_REG: /* Shared address space with TLR & XOFF2 */
case SC16IS7XX_TXLVL_REG:
case SC16IS7XX_RXLVL_REG:
case SC16IS7XX_IOSTATE_REG:
@@ -784,6 +787,7 @@ static void sc16is7xx_update_mlines(struct sc16is7xx_one *one)
unsigned long flags;
unsigned int status, changed;
+ /* Lock required as MSR address is shared with TCR and XOFF1. */
lockdep_assert_held_once(&one->lock);
status = sc16is7xx_get_hwmctrl(port);
@@ -1646,7 +1650,7 @@ int sc16is7xx_probe(struct device *dev, const struct sc16is7xx_devtype *devtype,
s->p[i].port.irq = irq;
s->p[i].port.type = PORT_SC16IS7XX;
s->p[i].port.fifosize = SC16IS7XX_FIFO_SIZE;
- s->p[i].port.flags = UPF_FIXED_TYPE | UPF_LOW_LATENCY;
+ s->p[i].port.flags = UPF_FIXED_TYPE | UPF_LOW_LATENCY | UPF_SOFT_FLOW;
s->p[i].port.iobase = i;
/*
* Use all ones as membase to make sure uart_configure_port() in
diff --git a/drivers/ufs/core/ufs_bsg.c b/drivers/ufs/core/ufs_bsg.c
index 0e0b08a3821c..59a473261ae1 100644
--- a/drivers/ufs/core/ufs_bsg.c
+++ b/drivers/ufs/core/ufs_bsg.c
@@ -79,6 +79,9 @@ static int ufs_bsg_exec_advanced_rpmb_req(struct ufs_hba *hba, struct bsg_job *j
if (hba->ufs_version < ufshci_version(4, 0) || !hba->dev_info.b_advanced_rpmb_en)
return -EINVAL;
+ if (job->request_len < sizeof(*rpmb_request))
+ return -EINVAL;
+
if (rpmb_request->ehs_req.length != 2 || rpmb_request->ehs_req.ehs_type != 1)
return -EINVAL;
@@ -144,6 +147,11 @@ static int ufs_bsg_request(struct bsg_job *job)
memset(job->reply, 0, job->reply_len);
+ if (job->request_len < sizeof(*bsg_request)) {
+ ret = -EINVAL;
+ goto out_complete;
+ }
+
ufshcd_rpm_get_sync(hba);
msgcode = bsg_request->msgcode;
@@ -170,6 +178,10 @@ static int ufs_bsg_request(struct bsg_job *job)
break;
case UPIU_TRANSACTION_UIC_CMD:
memcpy(&uc, &bsg_request->upiu_req.uc, UIC_CMD_SIZE);
+ if (uc.argument2 & MASK_UIC_COMMAND_RESULT) {
+ ret = -EINVAL;
+ break;
+ }
ret = ufshcd_send_bsg_uic_cmd(hba, &uc);
if (ret)
dev_err(hba->dev, "send uic cmd: error code %d\n", ret);
@@ -193,6 +205,7 @@ static int ufs_bsg_request(struct bsg_job *job)
out:
ufshcd_rpm_put_sync(hba);
kfree(buff);
+out_complete:
bsg_reply->result = ret;
if (ret == 0)
job->reply_len = rpmb ? sizeof(struct ufs_rpmb_reply) :
diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index 97f86b6905b5..d595072f7c0a 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -5694,6 +5694,37 @@ static void ufshcd_clear_polled(struct ufs_hba *hba,
}
}
+/*
+ * Return a bitmask of the transfer requests cleared from
+ * @hba->outstanding_reqs.
+ */
+static unsigned long __ufshcd_clear_outstanding_reqs(struct ufs_hba *hba,
+ unsigned long mask,
+ bool from_irq)
+ __must_hold(&hba->outstanding_lock)
+{
+ unsigned long completed_reqs = mask & hba->outstanding_reqs;
+
+ if (from_irq) {
+ /* Do not complete polled requests from interrupt context. */
+ ufshcd_clear_polled(hba, &completed_reqs);
+ }
+ hba->outstanding_reqs &= ~completed_reqs;
+
+ return completed_reqs;
+}
+
+/*
+ * Return a bitmask of the transfer requests cleared from
+ * @hba->outstanding_reqs.
+ */
+static unsigned long ufshcd_clear_outstanding_reqs(struct ufs_hba *hba,
+ unsigned long mask)
+{
+ guard(spinlock_irqsave)(&hba->outstanding_lock);
+ return __ufshcd_clear_outstanding_reqs(hba, mask, false);
+}
+
/*
* Return: > 0 if one or more commands have been completed or 0 if no
* requests have been completed.
@@ -5704,6 +5735,7 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num)
unsigned long completed_reqs, flags;
u32 tr_doorbell;
struct ufs_hw_queue *hwq;
+ bool from_irq;
if (hba->mcq_enabled) {
hwq = &hba->uhq[queue_num];
@@ -5713,12 +5745,9 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num)
spin_lock_irqsave(&hba->outstanding_lock, flags);
tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL);
- completed_reqs = ~tr_doorbell & hba->outstanding_reqs;
- if (queue_num == UFSHCD_POLL_FROM_INTERRUPT_CONTEXT) {
- /* Do not complete polled requests from interrupt context. */
- ufshcd_clear_polled(hba, &completed_reqs);
- }
- hba->outstanding_reqs &= ~completed_reqs;
+ from_irq = queue_num == UFSHCD_POLL_FROM_INTERRUPT_CONTEXT;
+ completed_reqs = __ufshcd_clear_outstanding_reqs(hba, ~tr_doorbell,
+ from_irq);
spin_unlock_irqrestore(&hba->outstanding_lock, flags);
if (completed_reqs)
@@ -7739,7 +7768,6 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)
struct ufshcd_lrb *lrbp = &hba->lrb[tag];
unsigned long flags;
int err = FAILED;
- bool outstanding;
u32 reg;
ufshcd_hold(hba);
@@ -7820,11 +7848,7 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)
* Clear the corresponding bit from outstanding_reqs since the command
* has been aborted successfully.
*/
- spin_lock_irqsave(&hba->outstanding_lock, flags);
- outstanding = __test_and_clear_bit(tag, &hba->outstanding_reqs);
- spin_unlock_irqrestore(&hba->outstanding_lock, flags);
-
- if (outstanding)
+ if (ufshcd_clear_outstanding_reqs(hba, BIT(tag)))
ufshcd_release_scsi_cmd(hba, lrbp);
err = SUCCESS;
diff --git a/drivers/vdpa/solidrun/snet_ctrl.c b/drivers/vdpa/solidrun/snet_ctrl.c
index 3cef2571d15d..1c3b5f2e4499 100644
--- a/drivers/vdpa/solidrun/snet_ctrl.c
+++ b/drivers/vdpa/solidrun/snet_ctrl.c
@@ -291,7 +291,6 @@ static int snet_send_ctrl_msg(struct snet *snet, u16 opcode, u16 vq_idx)
/* Check for errors */
val = snet_read_ctrl(regs);
- ret = SNET_VAL_TO_ERR(val);
/* Clear the chunk ready bit */
val &= ~SNET_CTRL_CHUNK_RDY_MASK;
@@ -301,6 +300,8 @@ static int snet_send_ctrl_msg(struct snet *snet, u16 opcode, u16 vq_idx)
if (ret)
SNET_WARN(pdev, "Timeout waiting for DPU to complete a control command, err %d\n",
ret);
+ else
+ ret = SNET_VAL_TO_ERR(val);
exit:
mutex_unlock(&snet->ctrl_lock);
diff --git a/drivers/vdpa/vdpa_user/iova_domain.c b/drivers/vdpa/vdpa_user/iova_domain.c
index e8b837df328f..b8b2eb44410d 100644
--- a/drivers/vdpa/vdpa_user/iova_domain.c
+++ b/drivers/vdpa/vdpa_user/iova_domain.c
@@ -76,6 +76,10 @@ int vduse_domain_set_map(struct vduse_iova_domain *domain,
for (map = vhost_iotlb_itree_first(iotlb, start, last); map;
map = vhost_iotlb_itree_next(map, start, last)) {
map_file = (struct vdpa_map_file *)map->opaque;
+ if (!map_file || map_file->file == domain->file) {
+ ret = -EINVAL;
+ goto err;
+ }
ret = vduse_iotlb_add_range(domain, map->start, map->last,
map->addr, map->perm,
map_file->file,
@@ -226,6 +230,7 @@ vduse_domain_get_coherent_page(struct vduse_iova_domain *domain, u64 iova)
{
u64 start = iova & PAGE_MASK;
u64 last = start + PAGE_SIZE - 1;
+ struct vdpa_map_file *map_file;
struct vhost_iotlb_map *map;
struct page *page = NULL;
@@ -234,6 +239,10 @@ vduse_domain_get_coherent_page(struct vduse_iova_domain *domain, u64 iova)
if (!map)
goto out;
+ map_file = (struct vdpa_map_file *)map->opaque;
+ if (!map_file || map_file->file != domain->file)
+ goto out;
+
page = pfn_to_page((map->addr + iova - map->start) >> PAGE_SHIFT);
get_page(page);
out:
@@ -579,7 +588,7 @@ static int vduse_domain_mmap(struct file *file, struct vm_area_struct *vma)
{
struct vduse_iova_domain *domain = file->private_data;
- vm_flags_set(vma, VM_DONTDUMP | VM_DONTEXPAND);
+ vm_flags_set(vma, VM_IO | VM_DONTDUMP | VM_DONTEXPAND);
vma->vm_private_data = domain;
vma->vm_ops = &vduse_domain_mmap_ops;
diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_rdwr.c
index 25380b7dfe18..8338708b99c1 100644
--- a/drivers/vfio/pci/vfio_pci_rdwr.c
+++ b/drivers/vfio/pci/vfio_pci_rdwr.c
@@ -449,9 +449,6 @@ int vfio_pci_ioeventfd(struct vfio_pci_core_device *vdev, loff_t offset,
pos >= vdev->msix_offset + vdev->msix_size))
return -EINVAL;
- if (count == 8)
- return -EINVAL;
-
ret = vfio_pci_core_setup_barmap(vdev, bar);
if (ret)
return ret;
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index deb4c8bfd3d5..fac400b25dad 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -906,7 +906,12 @@ vhost_scsi_copy_iov_to_sgl(struct vhost_scsi_cmd *cmd, struct iov_iter *iter,
}
for_each_sgtable_sg(sg_table, sg, i) {
- page = alloc_page(GFP_KERNEL);
+ /*
+ * Completion copies the full bounce buffer, including bytes the
+ * target did not write. This is already the slow fallback for
+ * misaligned I/O, so zeroing is acceptable here.
+ */
+ page = alloc_page(GFP_KERNEL | __GFP_ZERO);
if (!page) {
ret = -ENOMEM;
goto err;
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index 219af4f0d06a..e4ddaf0e5abc 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -1149,7 +1149,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
unsigned long list_size = PAGE_SIZE / sizeof(struct page *);
unsigned int gup_flags = FOLL_LONGTERM;
unsigned long npages, cur_base, map_pfn, last_pfn = 0;
- unsigned long lock_limit, sz2pin, nchunks, i;
+ unsigned long lock_limit, sz2pin, i;
unsigned long page_offset;
u64 start = iova;
long pinned;
@@ -1185,7 +1185,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
cur_base = uaddr & PAGE_MASK;
iova &= PAGE_MASK;
- nchunks = 0;
while (npages) {
sz2pin = min_t(unsigned long, npages, list_size);
@@ -1200,7 +1199,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
}
goto out;
}
- nchunks++;
if (!last_pfn)
map_pfn = page_to_pfn(page_list[0]);
@@ -1231,7 +1229,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
map_pfn = this_pfn;
iova += csize;
- nchunks = 0;
}
last_pfn = this_pfn;
@@ -1246,7 +1243,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
PFN_PHYS(map_pfn), perm, NULL);
out:
if (ret) {
- if (nchunks) {
+ if (last_pfn) {
unsigned long pfn;
/*
@@ -1258,7 +1255,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
* the corresponding unpinning will be handled by
* vdpa_unmap().
*/
- WARN_ON(!last_pfn);
for (pfn = map_pfn; pfn <= last_pfn; pfn++)
unpin_user_page(pfn_to_page(pfn));
}
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index ebdbf1cf9143..78cee55cedf8 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -3145,8 +3145,7 @@ static int vhost_add_used_n_in_order(struct vhost_virtqueue *vq,
}
start += nheads[i];
new += nheads[i];
- if (start >= vq->num)
- start -= vq->num;
+ start &= vq->num - 1;
}
if (unlikely(vq->log_used)) {
diff --git a/drivers/vhost/vringh.c b/drivers/vhost/vringh.c
index 925858cc6096..339141337957 100644
--- a/drivers/vhost/vringh.c
+++ b/drivers/vhost/vringh.c
@@ -197,8 +197,9 @@ static int move_to_indirect(const struct vringh *vrh,
}
len = vringh32_to_cpu(vrh, desc->len);
- if (unlikely(len % sizeof(struct vring_desc))) {
- vringh_bad("Strange indirect len %u", desc->len);
+ if (unlikely(len < sizeof(struct vring_desc) ||
+ len % sizeof(struct vring_desc))) {
+ vringh_bad("Invalid indirect len %u", desc->len);
return -EINVAL;
}
@@ -333,6 +334,17 @@ __vringh_iov(struct vringh *vrh, u16 i,
if (unlikely(err))
goto fail;
+ if (up_next == -1)
+ count++;
+ else
+ indirect_count++;
+
+ if (count > vrh->vring.num || indirect_count > desc_max) {
+ vringh_bad("Descriptor loop in %p", descs);
+ err = -ELOOP;
+ goto fail;
+ }
+
if (unlikely(desc.flags &
cpu_to_vringh16(vrh, VRING_DESC_F_INDIRECT))) {
u64 a = vringh64_to_cpu(vrh, desc.addr);
@@ -358,17 +370,6 @@ __vringh_iov(struct vringh *vrh, u16 i,
continue;
}
- if (up_next == -1)
- count++;
- else
- indirect_count++;
-
- if (count > vrh->vring.num || indirect_count > desc_max) {
- vringh_bad("Descriptor loop in %p", descs);
- err = -ELOOP;
- goto fail;
- }
-
if (desc.flags & cpu_to_vringh16(vrh, VRING_DESC_F_WRITE))
iov = wiov;
else {
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index 65297f245b27..f00182e9efec 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -331,7 +331,7 @@ static struct sk_buff *
vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,
unsigned int out, unsigned int in)
{
- struct virtio_vsock_hdr *hdr;
+ struct virtio_vsock_hdr hdr;
struct iov_iter iov_iter;
struct sk_buff *skb;
size_t payload_len;
@@ -349,34 +349,32 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,
len > VIRTIO_VSOCK_MAX_PKT_BUF_SIZE + VIRTIO_VSOCK_SKB_HEADROOM)
return NULL;
- /* len contains both payload and hdr */
- skb = virtio_vsock_alloc_skb(len, GFP_KERNEL);
- if (!skb)
- return NULL;
-
iov_iter_init(&iov_iter, ITER_SOURCE, vq->iov, out, len);
- hdr = virtio_vsock_hdr(skb);
- nbytes = copy_from_iter(hdr, sizeof(*hdr), &iov_iter);
- if (nbytes != sizeof(*hdr)) {
+ nbytes = copy_from_iter(&hdr, sizeof(hdr), &iov_iter);
+ if (nbytes != sizeof(hdr)) {
vq_err(vq, "Expected %zu bytes for pkt->hdr, got %zu bytes\n",
- sizeof(*hdr), nbytes);
- kfree_skb(skb);
+ sizeof(hdr), nbytes);
return NULL;
}
- payload_len = le32_to_cpu(hdr->len);
+ payload_len = le32_to_cpu(hdr.len);
+
+ /* The pkt is too big or the length in the header is invalid */
+ if (payload_len > len - sizeof(hdr))
+ return NULL;
+
+ /* Allocate only for the payload declared in the header. */
+ skb = virtio_vsock_alloc_skb(payload_len + sizeof(hdr), GFP_KERNEL);
+ if (!skb)
+ return NULL;
+
+ memcpy(virtio_vsock_hdr(skb), &hdr, sizeof(hdr));
/* No payload */
if (!payload_len)
return skb;
- /* The pkt is too big or the length in the header is invalid */
- if (payload_len + sizeof(*hdr) > len) {
- kfree_skb(skb);
- return NULL;
- }
-
virtio_vsock_skb_put(skb, payload_len);
if (skb_copy_datagram_from_iter(skb, 0, &iov_iter, payload_len)) {
diff --git a/drivers/video/fbdev/core/fb_info.c b/drivers/video/fbdev/core/fb_info.c
index 52f9bd2c5417..05988a936beb 100644
--- a/drivers/video/fbdev/core/fb_info.c
+++ b/drivers/video/fbdev/core/fb_info.c
@@ -43,6 +43,7 @@ struct fb_info *framebuffer_alloc(size_t size, struct device *dev)
info->device = dev;
info->fbcon_rotate_hint = -1;
info->blank = FB_BLANK_UNBLANK;
+ INIT_LIST_HEAD(&info->modelist);
#if IS_ENABLED(CONFIG_FB_BACKLIGHT)
mutex_init(&info->bl_curve_mutex);
diff --git a/drivers/video/fbdev/core/fbcon_rotate.c b/drivers/video/fbdev/core/fbcon_rotate.c
index a3f507825eed..3105682075c8 100644
--- a/drivers/video/fbdev/core/fbcon_rotate.c
+++ b/drivers/video/fbdev/core/fbcon_rotate.c
@@ -27,7 +27,8 @@ static int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc)
u8 *dst;
if (vc->vc_font.data == par->fontdata &&
- par->p->con_rotate == par->cur_rotate)
+ par->p->con_rotate == par->cur_rotate &&
+ par->fontbuffer)
goto finished;
src = par->fontdata = vc->vc_font.data;
diff --git a/drivers/video/fbdev/nvidia/nvidia.c b/drivers/video/fbdev/nvidia/nvidia.c
index c9dcc9e89948..aebb81f9ac58 100644
--- a/drivers/video/fbdev/nvidia/nvidia.c
+++ b/drivers/video/fbdev/nvidia/nvidia.c
@@ -1289,7 +1289,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
NVTRACE_ENTER();
assert(pd != NULL);
- if (pci_enable_device(pd)) {
+ if (pcim_enable_device(pd)) {
printk(KERN_ERR PFX "cannot enable PCI device\n");
return -ENODEV;
}
@@ -1302,7 +1302,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
nvidiafb_fix.mmio_start = pci_resource_start(pd, 0);
nvidiafb_fix.mmio_len = pci_resource_len(pd, 0);
- REGS = ioremap(nvidiafb_fix.mmio_start, nvidiafb_fix.mmio_len);
+ REGS = devm_ioremap(&pd->dev, nvidiafb_fix.mmio_start, nvidiafb_fix.mmio_len);
if (!REGS) {
printk(KERN_ERR PFX "cannot ioremap MMIO base\n");
return -ENODEV;
@@ -1330,7 +1330,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
if (info->pixmap.addr == NULL)
goto err_out_kfree;
- if (pci_request_regions(pd, "nvidiafb")) {
+ if (pcim_request_all_regions(pd, "nvidiafb")) {
printk(KERN_ERR PFX "cannot request PCI regions\n");
goto err_out_enable;
}
@@ -1355,7 +1355,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
sprintf(nvidiafb_fix.id, "NV%x", (pd->device & 0x0ff0) >> 4);
if (NVCommonSetup(info))
- goto err_out_free_base0;
+ goto err_out_enable;
par->FbAddress = nvidiafb_fix.smem_start;
par->FbMapSize = par->RamAmountKBytes * 1024;
@@ -1375,8 +1375,8 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
par->ScratchBufferStart = par->FbUsableSize - par->ScratchBufferSize;
par->CursorStart = par->FbUsableSize + (32 * 1024);
- info->screen_base = ioremap_wc(nvidiafb_fix.smem_start,
- par->FbMapSize);
+ info->screen_base = devm_ioremap_wc(&pd->dev, nvidiafb_fix.smem_start,
+ par->FbMapSize);
info->screen_size = par->FbUsableSize;
nvidiafb_fix.smem_len = par->RamAmountKBytes * 1024;
@@ -1420,19 +1420,16 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
return 0;
err_out_iounmap_fb:
- iounmap(info->screen_base);
+ arch_phys_wc_del(par->wc_cookie);
fb_destroy_modelist(&info->modelist);
err_out_free_base1:
fb_destroy_modedb(info->monspecs.modedb);
nvidia_delete_i2c_busses(par);
-err_out_free_base0:
- pci_release_regions(pd);
err_out_enable:
kfree(info->pixmap.addr);
err_out_kfree:
framebuffer_release(info);
err_out:
- iounmap(REGS);
return -ENODEV;
}
@@ -1447,11 +1444,8 @@ static void nvidiafb_remove(struct pci_dev *pd)
unregister_framebuffer(info);
arch_phys_wc_del(par->wc_cookie);
- iounmap(info->screen_base);
fb_destroy_modedb(info->monspecs.modedb);
nvidia_delete_i2c_busses(par);
- iounmap(par->REGS);
- pci_release_regions(pd);
kfree(info->pixmap.addr);
framebuffer_release(info);
NVTRACE_LEAVE();
diff --git a/drivers/video/fbdev/sh_mobile_lcdcfb.c b/drivers/video/fbdev/sh_mobile_lcdcfb.c
index c6da5a62f8c5..07e1dbcec3af 100644
--- a/drivers/video/fbdev/sh_mobile_lcdcfb.c
+++ b/drivers/video/fbdev/sh_mobile_lcdcfb.c
@@ -1341,18 +1341,23 @@ overlay_rop3_store(struct device *dev, struct device_attribute *attr,
return count;
}
-static DEVICE_ATTR_RW(overlay_alpha);
-static DEVICE_ATTR_RW(overlay_mode);
-static DEVICE_ATTR_RW(overlay_position);
-static DEVICE_ATTR_RW(overlay_rop3);
+static struct device_attribute dev_attr_ovl_alpha =
+ __ATTR(ovl_alpha, 0644, overlay_alpha_show, overlay_alpha_store);
+static struct device_attribute dev_attr_ovl_mode =
+ __ATTR(ovl_mode, 0644, overlay_mode_show, overlay_mode_store);
+static struct device_attribute dev_attr_ovl_position =
+ __ATTR(ovl_position, 0644, overlay_position_show, overlay_position_store);
+static struct device_attribute dev_attr_ovl_rop3 =
+ __ATTR(ovl_rop3, 0644, overlay_rop3_show, overlay_rop3_store);
static struct attribute *overlay_sysfs_attrs[] = {
- &dev_attr_overlay_alpha.attr,
- &dev_attr_overlay_mode.attr,
- &dev_attr_overlay_position.attr,
- &dev_attr_overlay_rop3.attr,
+ &dev_attr_ovl_alpha.attr,
+ &dev_attr_ovl_mode.attr,
+ &dev_attr_ovl_position.attr,
+ &dev_attr_ovl_rop3.attr,
NULL,
};
+
ATTRIBUTE_GROUPS(overlay_sysfs);
static const struct fb_fix_screeninfo sh_mobile_lcdc_overlay_fix = {
@@ -1511,9 +1516,10 @@ sh_mobile_lcdc_overlay_fb_unregister(struct sh_mobile_lcdc_overlay *ovl)
{
struct fb_info *info = ovl->info;
- if (info == NULL || info->dev == NULL)
+ if (!info || !dev_of_fbinfo(info))
return;
+ device_remove_groups(dev_of_fbinfo(info), overlay_sysfs_groups);
unregister_framebuffer(ovl->info);
}
@@ -1531,6 +1537,15 @@ sh_mobile_lcdc_overlay_fb_register(struct sh_mobile_lcdc_overlay *ovl)
if (ret < 0)
return ret;
+ /* The framebuffer device is optional, see fb_device_create(). */
+ if (dev_of_fbinfo(info)) {
+ ret = device_add_groups(dev_of_fbinfo(info), overlay_sysfs_groups);
+ if (ret < 0) {
+ unregister_framebuffer(info);
+ return ret;
+ }
+ }
+
dev_info(lcdc->dev, "registered %s/overlay %u as %dx%d %dbpp.\n",
dev_name(lcdc->dev), ovl->index, info->var.xres,
info->var.yres, info->var.bits_per_pixel);
@@ -2638,7 +2653,6 @@ static int sh_mobile_lcdc_probe(struct platform_device *pdev)
static struct platform_driver sh_mobile_lcdc_driver = {
.driver = {
.name = "sh_mobile_lcdc_fb",
- .dev_groups = overlay_sysfs_groups,
.pm = &sh_mobile_lcdc_dev_pm_ops,
},
.probe = sh_mobile_lcdc_probe,
diff --git a/drivers/watchdog/hpwdt.c b/drivers/watchdog/hpwdt.c
index ae30e394d176..32e9ac24e0d5 100644
--- a/drivers/watchdog/hpwdt.c
+++ b/drivers/watchdog/hpwdt.c
@@ -22,6 +22,7 @@
#include <linux/watchdog.h>
#ifdef CONFIG_HPWDT_NMI_DECODING
#include <asm/nmi.h>
+#include <asm/uv/uv.h>
#endif
#include <linux/crash_dump.h>
@@ -158,24 +159,31 @@ static int hpwdt_set_pretimeout(struct watchdog_device *wdd, unsigned int req)
return 0;
}
-static int hpwdt_my_nmi(void)
-{
- return ioread8(hpwdt_nmistat) & 0x6;
-}
+#define NMISTAT_EASR BIT(0)
+#define NMISTAT_EWDOG BIT(1)
+#define NMISTAT_RTRAP BIT(2)
+#define NMISTAT_DBELL BIT(3)
+#define NMISTAT_EMSWDG BIT(4)
+#define NMISTAT_GPI BIT(5)
+#define NMISTAT_NMIOUT BIT(7)
/*
* NMI Handler
*/
static int hpwdt_pretimeout(unsigned int ulReason, struct pt_regs *regs)
{
- unsigned int mynmi = hpwdt_my_nmi();
- static char panic_msg[] =
+ u8 nmistat = ioread8(hpwdt_nmistat);
+ bool mynmi = (nmistat & (NMISTAT_EWDOG | NMISTAT_RTRAP)) != 0;
+ static char panic_msg_default[] =
"00: An NMI occurred. Depending on your system the reason "
"for the NMI is logged in any one of the following resources:\n"
"1. Integrated Management Log (IML)\n"
"2. OA Syslog\n"
"3. OA Forward Progress Log\n"
"4. iLO Event Log";
+ static char panic_msg_uv[] =
+ "00: A watchdog NMI occurred.";
+ char *panic_msg = is_uv_system() ? panic_msg_uv : panic_msg_default;
if (ulReason == NMI_UNKNOWN && !mynmi)
return NMI_DONE;
@@ -189,8 +197,8 @@ static int hpwdt_pretimeout(unsigned int ulReason, struct pt_regs *regs)
hpwdt_ping_ticks(SECS_TO_TICKS(val));
}
- hex_byte_pack(panic_msg, mynmi);
- nmi_panic(regs, panic_msg);
+ hex_byte_pack(panic_msg, nmistat);
+ nmi_panic(regs, "%s", panic_msg);
return NMI_HANDLED;
}
diff --git a/drivers/xen/gntdev-dmabuf.c b/drivers/xen/gntdev-dmabuf.c
index 550980dd3b0b..5d38506a14f4 100644
--- a/drivers/xen/gntdev-dmabuf.c
+++ b/drivers/xen/gntdev-dmabuf.c
@@ -96,8 +96,10 @@ dmabuf_exp_wait_obj_new(struct gntdev_dmabuf_priv *priv,
struct gntdev_dmabuf_wait_obj *obj;
obj = kzalloc(sizeof(*obj), GFP_KERNEL);
- if (!obj)
+ if (!obj) {
+ kref_put(&gntdev_dmabuf->u.exp.refcount, dmabuf_exp_release);
return ERR_PTR(-ENOMEM);
+ }
init_completion(&obj->completion);
obj->gntdev_dmabuf = gntdev_dmabuf;
diff --git a/fs/9p/v9fs.c b/fs/9p/v9fs.c
index bde3ffb0e319..20dc6a03826a 100644
--- a/fs/9p/v9fs.c
+++ b/fs/9p/v9fs.c
@@ -16,6 +16,7 @@
#include <linux/parser.h>
#include <linux/slab.h>
#include <linux/seq_file.h>
+#include <linux/sysfs.h>
#include <net/9p/9p.h>
#include <net/9p/client.h>
#include <net/9p/transport.h>
@@ -554,21 +555,14 @@ static ssize_t caches_show(struct kobject *kobj,
struct kobj_attribute *attr,
char *buf)
{
- ssize_t n = 0, count = 0, limit = PAGE_SIZE;
+ ssize_t count = 0;
struct v9fs_session_info *v9ses;
spin_lock(&v9fs_sessionlist_lock);
list_for_each_entry(v9ses, &v9fs_sessionlist, slist) {
- if (v9ses->cachetag) {
- n = snprintf(buf + count, limit, "%s\n", v9ses->cachetag);
- if (n < 0) {
- count = n;
- break;
- }
-
- count += n;
- limit -= n;
- }
+ if (v9ses->cachetag)
+ count += sysfs_emit_at(buf, count, "%s\n",
+ v9ses->cachetag);
}
spin_unlock(&v9fs_sessionlist_lock);
diff --git a/fs/9p/vfs_inode.c b/fs/9p/vfs_inode.c
index b2368334f6ee..34fda75c7760 100644
--- a/fs/9p/vfs_inode.c
+++ b/fs/9p/vfs_inode.c
@@ -493,6 +493,12 @@ static int v9fs_at_to_dotl_flags(int flags)
* nothing and races with concurrent metadata fetches that may already
* have observed the post-unlink value (nlink == 0).
*
+ * The same applies to a regular file whose cached nlink has already been
+ * driven to zero by the server: in the cached modes the attributes come
+ * from GETATTR and are applied verbatim, so decrementing a link count the
+ * server does not maintain only trips the drop_nlink() WARN_ON and
+ * underflows i_nlink.
+ *
* @inode: inode whose nlink is being dropped
*/
static void v9fs_dec_count(struct inode *inode)
@@ -501,6 +507,8 @@ static void v9fs_dec_count(struct inode *inode)
if (!(v9ses->cache & (CACHE_META | CACHE_LOOSE)))
return;
+ if (inode->i_nlink == 0)
+ return;
if (!S_ISDIR(inode->i_mode) || inode->i_nlink > 2)
drop_nlink(inode);
}
diff --git a/fs/backing-file.c b/fs/backing-file.c
index bf526fa2e29d..ed8e793564c3 100644
--- a/fs/backing-file.c
+++ b/fs/backing-file.c
@@ -106,6 +106,7 @@ struct backing_aio {
refcount_t ref;
struct kiocb *orig_iocb;
/* used for aio completion */
+ bool is_write;
void (*end_write)(struct kiocb *iocb, ssize_t);
struct work_struct work;
long res;
@@ -146,7 +147,8 @@ static void backing_aio_rw_complete(struct kiocb *iocb, long res)
struct backing_aio *aio = container_of(iocb, struct backing_aio, iocb);
struct kiocb *orig_iocb = aio->orig_iocb;
- if (iocb->ki_flags & IOCB_WRITE)
+ /* Pairs with kiocb_start_write() in vfs_iocb_iter_write() */
+ if (aio->is_write)
kiocb_end_write(iocb);
backing_aio_cleanup(aio, res);
@@ -287,6 +289,7 @@ ssize_t backing_file_write_iter(struct file *file, struct iov_iter *iter,
goto out;
aio->orig_iocb = iocb;
+ aio->is_write = true;
aio->end_write = ctx->end_write;
kiocb_clone(&aio->iocb, iocb, get_file(file));
aio->iocb.ki_flags = flags;
diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index d66681ce2b3d..204edb659304 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -1899,6 +1899,7 @@ void btrfs_sysfs_add_block_group_type(struct btrfs_block_group *cache)
if (space_info->block_group_kobjs[index]) {
spin_unlock(&space_info->lock);
kobject_put(&rkobj->kobj);
+ memalloc_nofs_restore(nofs_flag);
return;
} else {
space_info->block_group_kobjs[index] = &rkobj->kobj;
diff --git a/fs/buffer.c b/fs/buffer.c
index 261f521dbc58..b5a1ff2d559f 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -880,35 +880,6 @@ void invalidate_inode_buffers(struct inode *inode)
}
EXPORT_SYMBOL(invalidate_inode_buffers);
-/*
- * Remove any clean buffers from the inode's buffer list. This is called
- * when we're trying to free the inode itself. Those buffers can pin it.
- *
- * Returns true if all buffers were removed.
- */
-int remove_inode_buffers(struct inode *inode)
-{
- int ret = 1;
-
- if (inode_has_buffers(inode)) {
- struct address_space *mapping = &inode->i_data;
- struct list_head *list = &mapping->i_private_list;
- struct address_space *buffer_mapping = mapping->i_private_data;
-
- spin_lock(&buffer_mapping->i_private_lock);
- while (!list_empty(list)) {
- struct buffer_head *bh = BH_ENTRY(list->next);
- if (buffer_dirty(bh)) {
- ret = 0;
- break;
- }
- __remove_assoc_queue(bh);
- }
- spin_unlock(&buffer_mapping->i_private_lock);
- }
- return ret;
-}
-
/*
* Create the appropriate buffers when given a folio for data area and
* the size of each buffer.. Use the bh->b_this_page linked list to
diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
index 51aeae341f56..6f6c8d921436 100644
--- a/fs/ceph/addr.c
+++ b/fs/ceph/addr.c
@@ -2214,6 +2214,12 @@ void ceph_fill_inline_data(struct inode *inode, struct page *locked_page,
doutc(cl, "%p %llx.%llx len %zu locked_page %p\n", inode,
ceph_vinop(inode), len, locked_page);
+ if (len > PAGE_SIZE) {
+ pr_warn_ratelimited_client(cl, "oversized inline data %zu\n",
+ len);
+ len = PAGE_SIZE;
+ }
+
if (len > 0) {
void *kaddr = kmap_atomic(page);
memcpy(kaddr, data, len);
diff --git a/fs/ceph/crypto.c b/fs/ceph/crypto.c
index f23ba5c14878..eaf12c9c86ee 100644
--- a/fs/ceph/crypto.c
+++ b/fs/ceph/crypto.c
@@ -370,6 +370,7 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname,
struct fscrypt_str iname;
char *name = fname->name;
int name_len = fname->name_len;
+ int dst_len;
int ret;
if (WARN_ON_ONCE(tname && is_vmalloc_addr(tname->name)))
@@ -445,6 +446,9 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname,
iname.len = fname->ctext_len;
}
+ /* oname->len is overwritten by the decryption below */
+ dst_len = oname->len;
+
_oname.name = unlikely(is_vmalloc_addr(oname->name)) ?
tname->name : oname->name;
_oname.len = oname->len;
@@ -462,6 +466,10 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname,
name_len = snprintf(tmp_buf, sizeof(tmp_buf), "_%.*s_%ld",
oname->len, oname->name, dir->i_ino);
+ if (name_len > dst_len) {
+ ret = -ENAMETOOLONG;
+ goto out;
+ }
memcpy(oname->name, tmp_buf, name_len);
oname->len = name_len;
}
diff --git a/fs/dcache.c b/fs/dcache.c
index 3ad74a3abf9a..84952f6bbf40 100644
--- a/fs/dcache.c
+++ b/fs/dcache.c
@@ -1995,6 +1995,10 @@ void d_instantiate_new(struct dentry *entry, struct inode *inode)
spin_lock(&inode->i_lock);
__d_instantiate(entry, inode);
WARN_ON(!(inode->i_state & I_NEW));
+ /*
+ * Pairs with smp_rmb in wait_on_inode().
+ */
+ smp_wmb();
inode->i_state &= ~I_NEW & ~I_CREATING;
/*
* Pairs with the barrier in prepare_to_wait_event() to make sure
diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
index 0af259e22026..ab4b222aae89 100644
--- a/fs/erofs/zdata.c
+++ b/fs/erofs/zdata.c
@@ -1414,15 +1414,6 @@ static void z_erofs_decompressqueue_kthread_work(struct kthread_work *work)
}
#endif
-/* Use (kthread_)work in atomic contexts to minimize scheduling overhead */
-static inline bool z_erofs_in_atomic(void)
-{
- if (IS_ENABLED(CONFIG_PREEMPTION) && rcu_preempt_depth())
- return true;
- if (!IS_ENABLED(CONFIG_PREEMPT_COUNT))
- return true;
- return !preemptible();
-}
static void z_erofs_decompress_kickoff(struct z_erofs_decompressqueue *io,
int bios)
@@ -1439,7 +1430,7 @@ static void z_erofs_decompress_kickoff(struct z_erofs_decompressqueue *io,
if (atomic_add_return(bios, &io->pending_bios))
return;
- if (z_erofs_in_atomic()) {
+ if (bio_in_atomic()) {
/* See `sync_decompress` in sysfs-fs-erofs for more details */
if (sbi->sync_decompress == EROFS_SYNC_DECOMPRESS_AUTO)
sbi->sync_decompress = EROFS_SYNC_DECOMPRESS_FORCE_ON;
diff --git a/fs/exfat/file.c b/fs/exfat/file.c
index 536c8078f0c1..64569d8ba8d7 100644
--- a/fs/exfat/file.c
+++ b/fs/exfat/file.c
@@ -633,23 +633,18 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter)
struct file *file = iocb->ki_filp;
struct inode *inode = file_inode(file);
struct exfat_inode_info *ei = EXFAT_I(inode);
- loff_t pos = iocb->ki_pos;
- loff_t valid_size;
+ loff_t pos, valid_size;
if (unlikely(exfat_forced_shutdown(inode->i_sb)))
return -EIO;
inode_lock(inode);
- if (pos > i_size_read(inode))
- truncate_pagecache(inode, i_size_read(inode));
-
- valid_size = ei->valid_size;
-
ret = generic_write_checks(iocb, iter);
if (ret <= 0)
goto unlock;
+ pos = iocb->ki_pos;
if (iocb->ki_flags & IOCB_DIRECT) {
unsigned long align = pos | iov_iter_alignment(iter);
@@ -660,6 +655,10 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter)
}
}
+ if (pos > i_size_read(inode))
+ truncate_pagecache(inode, i_size_read(inode));
+
+ valid_size = ei->valid_size;
if (pos > valid_size) {
ret = exfat_extend_valid_size(inode, pos);
if (ret < 0 && ret != -ENOSPC) {
@@ -669,6 +668,8 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter)
}
if (ret < 0)
goto unlock;
+
+ pos = valid_size;
}
ret = __generic_file_write_iter(iocb, iter);
@@ -677,9 +678,6 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter)
inode_unlock(inode);
- if (pos > valid_size)
- pos = valid_size;
-
if (iocb->ki_pos > pos) {
ssize_t err = generic_write_sync(iocb, iocb->ki_pos - pos);
if (err < 0)
diff --git a/fs/ext2/file.c b/fs/ext2/file.c
index c59c19e5ff72..81f9cf3b884c 100644
--- a/fs/ext2/file.c
+++ b/fs/ext2/file.c
@@ -256,7 +256,8 @@ static ssize_t ext2_dio_write_iter(struct kiocb *iocb, struct iov_iter *from)
pos = iocb->ki_pos;
status = generic_perform_write(iocb, from);
if (unlikely(status < 0)) {
- ret = status;
+ if (!ret)
+ ret = status;
goto out_unlock;
}
diff --git a/fs/ext2/inode.c b/fs/ext2/inode.c
index 50d431949fd4..12f9b1b36e05 100644
--- a/fs/ext2/inode.c
+++ b/fs/ext2/inode.c
@@ -824,6 +824,12 @@ static int ext2_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
(first_block << blkbits) < i_size_read(inode))
create = 0;
+ ret = ext2_get_blocks(inode, first_block, max_blocks,
+ &bno, &new, &boundary, create);
+ if (ret < 0)
+ return ret;
+
+ iomap->flags = 0;
/*
* Writes that span EOF might trigger an IO size update on completion,
* so consider them to be dirty for the purposes of O_DSYNC even if
@@ -831,13 +837,6 @@ static int ext2_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
*/
if ((flags & IOMAP_WRITE) && offset + length > i_size_read(inode))
iomap->flags |= IOMAP_F_DIRTY;
-
- ret = ext2_get_blocks(inode, first_block, max_blocks,
- &bno, &new, &boundary, create);
- if (ret < 0)
- return ret;
-
- iomap->flags = 0;
iomap->offset = (u64)first_block << blkbits;
if (flags & IOMAP_DAX)
iomap->dax_dev = sbi->s_daxdev;
diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c
index 66d3714dda32..14db6b571bc2 100644
--- a/fs/f2fs/checkpoint.c
+++ b/fs/f2fs/checkpoint.c
@@ -93,7 +93,7 @@ static struct folio *__get_meta_folio(struct f2fs_sb_info *sbi, pgoff_t index,
f2fs_update_iostat(sbi, NULL, FS_META_READ_IO, F2FS_BLKSIZE);
folio_lock(folio);
- if (unlikely(!is_meta_folio(folio))) {
+ if (unlikely(!is_meta_folio(sbi, folio))) {
f2fs_folio_put(folio, true);
goto repeat;
}
@@ -411,7 +411,7 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type,
long nr_to_write, enum iostat_type io_type)
{
struct address_space *mapping = META_MAPPING(sbi);
- pgoff_t index = 0, prev = ULONG_MAX;
+ pgoff_t index = 0, next = ULONG_MAX;
struct folio_batch fbatch;
long nwritten = 0;
int nr_folios;
@@ -430,16 +430,15 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type,
for (i = 0; i < nr_folios; i++) {
struct folio *folio = fbatch.folios[i];
- if (nr_to_write != LONG_MAX && i != 0 &&
- folio->index != prev +
- folio_nr_pages(fbatch.folios[i-1])) {
+ if (nr_to_write != LONG_MAX && next != ULONG_MAX &&
+ folio->index != next) {
folio_batch_release(&fbatch);
goto stop;
}
folio_lock(folio);
- if (unlikely(!is_meta_folio(folio))) {
+ if (unlikely(!is_meta_folio(sbi, folio))) {
continue_unlock:
folio_unlock(folio);
continue;
@@ -460,7 +459,7 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type,
break;
}
nwritten += folio_nr_pages(folio);
- prev = folio->index;
+ next = folio->index + folio_nr_pages(folio);
if (unlikely(nwritten >= nr_to_write))
break;
}
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index eba1e025b9d6..868af1675510 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -153,7 +153,8 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task)
dec_page_count(F2FS_F_SB(folio), __read_io_type(folio));
if (bio->bi_status == BLK_STS_OK &&
- F2FS_F_SB(folio)->node_inode && is_node_folio(folio) &&
+ F2FS_F_SB(folio)->node_inode &&
+ is_node_folio(F2FS_F_SB(folio), folio) &&
f2fs_sanity_check_node_footer(F2FS_F_SB(folio),
folio, folio->index, NODE_TYPE_REGULAR, true))
bio->bi_status = BLK_STS_IOERR;
@@ -287,7 +288,7 @@ static void f2fs_read_end_io(struct bio *bio)
{
struct f2fs_sb_info *sbi = F2FS_F_SB(bio_first_folio_all(bio));
struct bio_post_read_ctx *ctx;
- bool intask = in_task() && !irqs_disabled();
+ bool intask = !bio_in_atomic();
iostat_update_and_unbind_ctx(bio);
ctx = bio->bi_private;
@@ -359,7 +360,7 @@ static void f2fs_write_end_io(struct bio *bio)
STOP_CP_REASON_WRITE_FAIL);
}
- if (is_node_folio(folio)) {
+ if (is_node_folio(sbi, folio)) {
f2fs_sanity_check_node_footer(sbi, folio,
folio->index, NODE_TYPE_REGULAR, true);
f2fs_bug_on(sbi, folio->index != nid_of_node(folio));
@@ -474,6 +475,10 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages)
struct bio *bio;
bdev = f2fs_target_device(sbi, fio->new_blkaddr, §or);
+ /* avoid reversed IO if bio is split due to exceeding max_segments */
+ if (npages > 1 && !is_read_io(fio->op) &&
+ f2fs_is_sequential_zone_area(sbi, fio->new_blkaddr))
+ npages = min_t(int, npages, bdev_max_segments(bdev));
bio = bio_alloc_bioset(bdev, npages,
fio->op | fio->op_flags | f2fs_io_flags(fio),
GFP_NOIO, &f2fs_bioset);
@@ -743,7 +748,19 @@ static bool page_is_mergeable(struct f2fs_sb_info *sbi, struct bio *bio,
return false;
if (last_blkaddr + 1 != cur_blkaddr)
return false;
- return bio->bi_bdev == f2fs_target_device(sbi, cur_blkaddr, NULL);
+ if (bio->bi_bdev != f2fs_target_device(sbi, cur_blkaddr, NULL))
+ return false;
+ /* avoid reversed IO if bio is split due to exceeding max_segments/sectors */
+ if (f2fs_is_sequential_zone_area(sbi, cur_blkaddr)) {
+ struct request_queue *q = bdev_get_queue(bio->bi_bdev);
+
+ if (bio->bi_vcnt >= queue_max_segments(q))
+ return false;
+ if (bio_sectors(bio) + (F2FS_BLKSIZE >> SECTOR_SHIFT) >
+ queue_max_sectors(q))
+ return false;
+ }
+ return true;
}
static bool io_type_is_mergeable(struct f2fs_bio_info *io,
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index ec1b9831e5af..82c3c4ebcb3e 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -2222,14 +2222,14 @@ static inline struct address_space *NODE_MAPPING(struct f2fs_sb_info *sbi)
return sbi->node_inode->i_mapping;
}
-static inline bool is_meta_folio(struct folio *folio)
+static inline bool is_meta_folio(struct f2fs_sb_info *sbi, struct folio *folio)
{
- return folio->mapping == META_MAPPING(F2FS_F_SB(folio));
+ return folio->mapping == META_MAPPING(sbi);
}
-static inline bool is_node_folio(struct folio *folio)
+static inline bool is_node_folio(struct f2fs_sb_info *sbi, struct folio *folio)
{
- return folio->mapping == NODE_MAPPING(F2FS_F_SB(folio));
+ return folio->mapping == NODE_MAPPING(sbi);
}
static inline bool is_sbi_flag_set(struct f2fs_sb_info *sbi, unsigned int type)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 8211ee75fb8f..d6798d795e3d 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -2431,7 +2431,7 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate)
fi->original_i_size = isize;
if (truncate) {
set_inode_flag(inode, FI_ATOMIC_REPLACE);
- truncate_inode_pages_final(inode->i_mapping);
+ truncate_inode_pages(inode->i_mapping, 0);
f2fs_i_size_write(inode, 0);
isize = 0;
}
diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
index e85d7f239e7d..e3ae70e297c8 100644
--- a/fs/f2fs/gc.c
+++ b/fs/f2fs/gc.c
@@ -1418,7 +1418,7 @@ static int move_data_block(struct inode *inode, block_t bidx,
F2FS_BLKSIZE);
folio_lock(mfolio);
- if (unlikely(!is_meta_folio(mfolio) ||
+ if (unlikely(!is_meta_folio(fio.sbi, mfolio) ||
!folio_test_uptodate(mfolio))) {
err = -EIO;
f2fs_folio_put(mfolio, true);
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index 60969e5f1201..ad59d52f17ae 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -327,7 +327,7 @@ static unsigned int __gang_lookup_nat_set(struct f2fs_nm_info *nm_i,
bool f2fs_in_warm_node_list(struct f2fs_sb_info *sbi, struct folio *folio)
{
- return is_node_folio(folio) && IS_DNODE(folio) && is_cold_node(folio);
+ return is_node_folio(sbi, folio) && IS_DNODE(folio) && is_cold_node(folio);
}
void f2fs_init_fsync_node_info(struct f2fs_sb_info *sbi)
@@ -811,8 +811,20 @@ int f2fs_get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode)
if (!dn->inode_folio) {
nfolio[0] = f2fs_get_inode_folio(sbi, nids[0]);
- if (IS_ERR(nfolio[0]))
- return PTR_ERR(nfolio[0]);
+ if (IS_ERR(nfolio[0])) {
+ err = PTR_ERR(nfolio[0]);
+ /*
+ * cp_error drops dirty node caches, so the inode block
+ * of an inode that was never written reads as NEW_ADDR
+ * and returns -ENOENT. It is not a hole; the inode is
+ * gone. Callers treat -ENOENT as a hole and keep
+ * looking block by block (e.g. fiemap up to maxbytes).
+ */
+ if (err == -ENOENT && (f2fs_cp_error(sbi) ||
+ is_sbi_flag_set(sbi, SBI_IS_SHUTDOWN)))
+ err = -EIO;
+ return err;
+ }
} else {
nfolio[0] = dn->inode_folio;
}
@@ -1266,7 +1278,7 @@ int f2fs_truncate_inode_blocks(struct inode *inode, pgoff_t from)
goto fail;
if (offset[1] == 0 && get_nid(folio, offset[0], true)) {
folio_lock(folio);
- BUG_ON(!is_node_folio(folio));
+ BUG_ON(!is_node_folio(sbi, folio));
set_nid(folio, offset[0], 0, true);
folio_unlock(folio);
}
@@ -1579,7 +1591,7 @@ static struct folio *__get_node_folio(struct f2fs_sb_info *sbi, pgoff_t nid,
folio_lock(folio);
- if (unlikely(!is_node_folio(folio))) {
+ if (unlikely(!is_node_folio(sbi, folio))) {
f2fs_folio_put(folio, true);
goto repeat;
}
@@ -1698,7 +1710,7 @@ static struct folio *last_fsync_dnode(struct f2fs_sb_info *sbi, nid_t ino)
folio_lock(folio);
- if (unlikely(!is_node_folio(folio))) {
+ if (unlikely(!is_node_folio(sbi, folio))) {
continue_unlock:
folio_unlock(folio);
continue;
@@ -1930,7 +1942,7 @@ int f2fs_fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode,
folio_lock(folio);
- if (unlikely(!is_node_folio(folio))) {
+ if (unlikely(!is_node_folio(sbi, folio))) {
continue_unlock:
folio_unlock(folio);
continue;
@@ -1988,7 +2000,7 @@ int f2fs_fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode,
f2fs_debug(sbi, "Retry to write fsync mark: ino=%u, idx=%lx",
ino, last_folio->index);
folio_lock(last_folio);
- if (unlikely(!is_node_folio(last_folio))) {
+ if (unlikely(!is_node_folio(sbi, last_folio))) {
f2fs_folio_put(last_folio, true);
ret = -EAGAIN;
goto out;
@@ -2066,7 +2078,7 @@ void f2fs_flush_inline_data(struct f2fs_sb_info *sbi)
folio_lock(folio);
- if (unlikely(!is_node_folio(folio)))
+ if (unlikely(!is_node_folio(sbi, folio)))
goto unlock;
if (!folio_test_dirty(folio))
goto unlock;
@@ -2138,7 +2150,7 @@ int f2fs_sync_node_pages(struct f2fs_sb_info *sbi,
else if (!folio_trylock(folio))
continue;
- if (unlikely(!is_node_folio(folio))) {
+ if (unlikely(!is_node_folio(sbi, folio))) {
continue_unlock:
folio_unlock(folio);
continue;
@@ -2839,6 +2851,45 @@ int f2fs_recover_inline_xattr(struct inode *inode, struct folio *folio)
f2fs_folio_wait_writeback(ifolio, NODE, true, true);
memcpy(dst_addr, src_addr, inline_size);
update_inode:
+ /*
+ * The xattr node of the checkpoint was removed before fsync, e.g. the
+ * xattrs became small enough to fit inline. Release it as well, or
+ * the inode keeps pointing to the stale xattrs.
+ */
+ if (!ri->i_xattr_nid && F2FS_I(inode)->i_xattr_nid) {
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ struct node_info ni;
+ int err;
+
+ err = f2fs_get_node_info(sbi, F2FS_I(inode)->i_xattr_nid,
+ &ni, false);
+ if (err) {
+ f2fs_folio_put(ifolio, true);
+ return err;
+ }
+
+ if (ni.blk_addr != NEW_ADDR &&
+ !f2fs_is_valid_blkaddr(sbi, ni.blk_addr,
+ DATA_GENERIC_ENHANCE)) {
+ f2fs_err_ratelimited(sbi,
+ "nat entry is corrupted, run fsck to fix it, ino:%u, nid:%u, blkaddr:%u",
+ ni.ino, ni.nid, ni.blk_addr);
+ set_sbi_flag(sbi, SBI_NEED_FSCK);
+ f2fs_handle_error(sbi, ERROR_INCONSISTENT_NAT);
+ f2fs_folio_put(ifolio, true);
+ return -EFSCORRUPTED;
+ }
+
+ f2fs_invalidate_blocks(sbi, ni.blk_addr, 1);
+ dec_valid_node_count(sbi, inode, false);
+ set_node_addr(sbi, &ni, NULL_ADDR, false);
+ /* drop the cached xattr node, which may be dirty */
+ truncate_inode_pages_range(NODE_MAPPING(sbi),
+ F2FS_BLK_TO_BYTES((loff_t)ni.nid),
+ F2FS_BLK_END_BYTES((loff_t)ni.nid));
+ f2fs_i_xnid_write(inode, 0);
+ }
+
f2fs_update_inode(inode, ifolio);
f2fs_folio_put(ifolio, true);
return 0;
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 46f14f891843..1097e688d165 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -193,7 +193,7 @@ void f2fs_abort_atomic_write(struct inode *inode, bool clean)
return;
if (clean)
- truncate_inode_pages_final(inode->i_mapping);
+ truncate_inode_pages(inode->i_mapping, 0);
release_atomic_write_cnt(inode);
clear_inode_flag(inode, FI_ATOMIC_COMMITTED);
@@ -3067,6 +3067,7 @@ static int change_curseg(struct f2fs_sb_info *sbi, int type)
if (IS_ERR(sum_folio)) {
/* GC won't be able to use stale summary pages by cp_error */
memset(curseg->sum_blk, 0, sbi->sum_entry_size);
+ reset_curseg_fields(curseg);
return PTR_ERR(sum_folio);
}
sum_node = SUM_BLK_PAGE_ADDR(sbi, sum_folio, new_segno);
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 69d36e1a301b..93e6575706ad 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -4128,6 +4128,7 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi)
block_t avail_node_count, valid_node_count;
unsigned int nat_blocks, nat_bits_bytes, nat_bits_blocks;
unsigned int sit_blk_cnt;
+ unsigned long long ckpt_size, sit_end, nat_end;
int i, j;
total = le32_to_cpu(raw_super->segment_count);
@@ -4259,6 +4260,29 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi)
return 1;
}
+ /*
+ * Nothing above ties the bitmap lengths to sbi->ckpt, which is
+ * (1 + cp_payload) blocks. Bound both ends the way __bitmap_ptr()
+ * lays them out.
+ */
+ ckpt_size = (unsigned long long)(cp_payload + 1) * F2FS_BLKSIZE;
+ if (__is_set_ckpt_flags(ckpt, CP_LARGE_NAT_BITMAP_FLAG)) {
+ nat_end = CP_MIN_CHKSUM_OFFSET + sizeof(__le32) + nat_bitmap_size;
+ sit_end = nat_end + sit_bitmap_size;
+ } else if (cp_payload > 0) {
+ nat_end = CP_MIN_CHKSUM_OFFSET + nat_bitmap_size;
+ sit_end = F2FS_BLKSIZE + sit_bitmap_size;
+ } else {
+ sit_end = CP_MIN_CHKSUM_OFFSET + sit_bitmap_size;
+ nat_end = sit_end + nat_bitmap_size;
+ }
+
+ if (sit_end > ckpt_size || nat_end > ckpt_size) {
+ f2fs_err(sbi, "Bitmaps do not fit the checkpoint pack: sit ends at %llu, nat at %llu, pack is %llu bytes",
+ sit_end, nat_end, ckpt_size);
+ return 1;
+ }
+
if (__is_set_ckpt_flags(ckpt, CP_LARGE_NAT_BITMAP_FLAG) &&
le32_to_cpu(ckpt->checksum_offset) != CP_MIN_CHKSUM_OFFSET) {
f2fs_warn(sbi, "using deprecated layout of large_nat_bitmap, "
diff --git a/fs/fat/inode.c b/fs/fat/inode.c
index 9cfe20a3daaf..1a951dfb33dc 100644
--- a/fs/fat/inode.c
+++ b/fs/fat/inode.c
@@ -215,7 +215,9 @@ static void fat_write_failed(struct address_space *mapping, loff_t to)
if (to > inode->i_size) {
truncate_pagecache(inode, inode->i_size);
+ down_write(&MSDOS_I(inode)->truncate_lock);
fat_truncate_blocks(inode, inode->i_size);
+ up_write(&MSDOS_I(inode)->truncate_lock);
}
}
diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index 3025fc7ddfe8..e479e310f92c 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -476,10 +476,11 @@ static bool inode_do_switch_wbs(struct inode *inode,
switched = true;
skip_switch:
/*
- * Paired with load_acquire in unlocked_inode_to_wb_begin() and
+ * Paired with an acquire fence in unlocked_inode_to_wb_begin() and
* ensures that the new wb is visible if they see !I_WB_SWITCH.
*/
- smp_store_release(&inode->i_state, inode->i_state & ~I_WB_SWITCH);
+ smp_wmb();
+ inode->i_state &= ~I_WB_SWITCH;
xa_unlock_irq(&mapping->i_pages);
spin_unlock(&inode->i_lock);
@@ -757,6 +758,13 @@ bool cleanup_offline_cgwb(struct bdi_writeback *wb)
new_wb = &wb->bdi->wb; /* wb_get() is noop for bdi's wb */
nr = 0;
+ /*
+ * The scan moves walked inodes to the newest end of b_dirty_time,
+ * past ones it didn't reach. If one of those was dirtied after a
+ * sync's cutoff, the sync's queue_io() stops there and misses the
+ * moved ones. Exclude syncs as process_inode_switch_wbs() does.
+ */
+ down_read(&wb->bdi->wb_switch_rwsem);
/*
* Paired with synchronize_rcu() in cgroup_writeback_umount().
* Holding rcu_read_lock across the SB_ACTIVE check, the inode grab
@@ -779,6 +787,7 @@ bool cleanup_offline_cgwb(struct bdi_writeback *wb)
restart = isw_prepare_wbs_switch(new_wb, isw, &wb->b_dirty_time,
&nr);
spin_unlock(&wb->list_lock);
+ up_read(&wb->bdi->wb_switch_rwsem);
/* no attached inodes? bail out */
if (nr == 0) {
diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c
index e475f7f9e83d..7075984a89aa 100644
--- a/fs/fuse/virtio_fs.c
+++ b/fs/fuse/virtio_fs.c
@@ -726,6 +726,10 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req)
unsigned int num_out;
unsigned int i;
+ /* fuse_request_end() returns this error; there are no args to copy. */
+ if (req->out.h.error)
+ goto out;
+
remaining = req->out.h.len - sizeof(req->out.h);
num_in = args->in_numargs - args->in_pages;
num_out = args->out_numargs - args->out_pages;
@@ -751,10 +755,57 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req)
if (args->out_argvar)
args->out_args[args->out_numargs - 1].size = remaining;
+out:
kfree(req->argbuf);
req->argbuf = NULL;
}
+/* Verify that the server properly follows the FUSE protocol */
+static bool virtio_fs_verify_response(struct fuse_req *req, unsigned int len)
+{
+ struct fuse_args *args = req->args;
+ struct fuse_out_header *oh = &req->out.h;
+ unsigned int expected;
+
+ if (len < sizeof(*oh)) {
+ pr_warn("virtio-fs: response too short (%u)\n", len);
+ return false;
+ }
+ if (oh->len != len) {
+ pr_warn("virtio-fs: oh.len mismatch (%u != %u)\n", oh->len, len);
+ return false;
+ }
+ if (oh->unique != req->in.h.unique) {
+ pr_warn("virtio-fs: oh.unique mismatch (%llu != %llu)\n",
+ oh->unique, req->in.h.unique);
+ return false;
+ }
+ if (oh->error <= -ERESTARTSYS || oh->error > 0) {
+ pr_warn("virtio-fs: invalid error value (%d)\n", oh->error);
+ return false;
+ }
+
+ if (oh->error) {
+ if (len != sizeof(*oh)) {
+ pr_warn("virtio-fs: error response too long (%u)\n", len);
+ return false;
+ }
+ return true;
+ }
+
+ expected = sizeof(*oh) +
+ fuse_len_args(args->out_numargs, args->out_args);
+ if (len > expected ||
+ (len < expected &&
+ (!args->out_argvar ||
+ expected - len > args->out_args[args->out_numargs - 1].size))) {
+ pr_warn("virtio-fs: invalid response length (%u, expected %u)\n",
+ len, expected);
+ return false;
+ }
+ return true;
+}
+
/* Work function for request completion */
static void virtio_fs_request_complete(struct fuse_req *req,
struct virtio_fs_vq *fsvq)
@@ -764,10 +815,6 @@ static void virtio_fs_request_complete(struct fuse_req *req,
unsigned int len, i, thislen;
struct folio *folio;
- /*
- * TODO verify that server properly follows FUSE protocol
- * (oh.uniq, oh.len)
- */
args = req->args;
copy_args_from_argbuf(args, req);
@@ -821,6 +868,10 @@ static void virtio_fs_requests_done_work(struct work_struct *work)
virtqueue_disable_cb(vq);
while ((req = virtqueue_get_buf(vq, &len)) != NULL) {
+ if (!virtio_fs_verify_response(req, len)) {
+ req->out.h.error = -EIO;
+ req->out.h.len = sizeof(struct fuse_out_header);
+ }
spin_lock(&fpq->lock);
list_move_tail(&req->list, &reqs);
spin_unlock(&fpq->lock);
diff --git a/fs/gfs2/glops.c b/fs/gfs2/glops.c
index 242e47926b39..35ad44d13e8b 100644
--- a/fs/gfs2/glops.c
+++ b/fs/gfs2/glops.c
@@ -392,6 +392,7 @@ static int gfs2_dinode_in(struct gfs2_inode *ip, const void *buf)
umode_t mode = be32_to_cpu(str->di_mode);
struct inode *inode = &ip->i_inode;
bool is_new = inode->i_state & I_NEW;
+ u64 size;
if (unlikely(ip->i_no_addr != be64_to_cpu(str->di_num.no_addr))) {
gfs2_consist_inode(ip);
@@ -417,7 +418,12 @@ static int gfs2_dinode_in(struct gfs2_inode *ip, const void *buf)
i_uid_write(inode, be32_to_cpu(str->di_uid));
i_gid_write(inode, be32_to_cpu(str->di_gid));
set_nlink(inode, be32_to_cpu(str->di_nlink));
- i_size_write(inode, be64_to_cpu(str->di_size));
+ size = be64_to_cpu(str->di_size);
+ if (unlikely(size > inode->i_sb->s_maxbytes)) {
+ gfs2_consist_inode(ip);
+ return -EIO;
+ }
+ i_size_write(inode, size);
gfs2_set_inode_blocks(inode, be64_to_cpu(str->di_blocks));
atime.tv_sec = be64_to_cpu(str->di_atime);
atime.tv_nsec = be32_to_cpu(str->di_atime_nsec);
@@ -456,7 +462,7 @@ static int gfs2_dinode_in(struct gfs2_inode *ip, const void *buf)
ip->i_depth = (u8)depth;
ip->i_entries = be32_to_cpu(str->di_entries);
- if (gfs2_is_stuffed(ip) && inode->i_size > gfs2_max_stuffed_size(ip)) {
+ if (gfs2_is_stuffed(ip) && size > gfs2_max_stuffed_size(ip)) {
gfs2_consist_inode(ip);
return -EIO;
}
diff --git a/fs/inode.c b/fs/inode.c
index 24762fd1f2f7..b60111fad771 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -16,7 +16,6 @@
#include <linux/fsnotify.h>
#include <linux/mount.h>
#include <linux/posix_acl.h>
-#include <linux/buffer_head.h> /* for inode_has_buffers */
#include <linux/ratelimit.h>
#include <linux/list_lru.h>
#include <linux/iversion.h>
@@ -366,7 +365,6 @@ struct inode *alloc_inode(struct super_block *sb)
void __destroy_inode(struct inode *inode)
{
- BUG_ON(inode_has_buffers(inode));
inode_detach_wb(inode);
security_inode_free(inode);
fsnotify_inode_delete(inode);
@@ -583,12 +581,11 @@ static void inode_pin_lru_isolating(struct inode *inode)
static void inode_unpin_lru_isolating(struct inode *inode)
{
- spin_lock(&inode->i_lock);
+ lockdep_assert_held(&inode->i_lock);
WARN_ON(!(inode->i_state & I_LRU_ISOLATING));
inode->i_state &= ~I_LRU_ISOLATING;
/* Called with inode->i_lock which ensures memory ordering. */
inode_wake_up_bit(inode, __I_LRU_ISOLATING);
- spin_unlock(&inode->i_lock);
}
static void inode_wait_for_lru_isolating(struct inode *inode)
@@ -927,6 +924,12 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
if (!spin_trylock(&inode->i_lock))
return LRU_SKIP;
+ /* Another walker is dropping its page cache, leave it to that one */
+ if (inode->i_state & I_LRU_ISOLATING) {
+ spin_unlock(&inode->i_lock);
+ return LRU_SKIP;
+ }
+
/*
* Inodes can get referenced, redirtied, or repopulated while
* they're already on the LRU, and this can make them
@@ -954,20 +957,25 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
* page cache in order to free up struct inodes: lowmem might
* be under pressure before the cache inside the highmem zone.
*/
- if (inode_has_buffers(inode) || !mapping_empty(&inode->i_data)) {
+ if (!mapping_empty(&inode->i_data)) {
+ unsigned long reap;
+
inode_pin_lru_isolating(inode);
spin_unlock(&inode->i_lock);
spin_unlock(&lru->lock);
- if (remove_inode_buffers(inode)) {
- unsigned long reap;
- reap = invalidate_mapping_pages(&inode->i_data, 0, -1);
- if (current_is_kswapd())
- __count_vm_events(KSWAPD_INODESTEAL, reap);
- else
- __count_vm_events(PGINODESTEAL, reap);
- mm_account_reclaimed_pages(reap);
- }
+ reap = invalidate_mapping_pages(&inode->i_data, 0, -1);
+ if (current_is_kswapd())
+ __count_vm_events(KSWAPD_INODESTEAL, reap);
+ else
+ __count_vm_events(PGINODESTEAL, reap);
+ mm_account_reclaimed_pages(reap);
+
+ spin_lock(&inode->i_lock);
+ /* The walk restarts here: don't retry leftovers right away */
+ if (!mapping_empty(&inode->i_data))
+ inode_state_set(inode, I_REFERENCED);
inode_unpin_lru_isolating(inode);
+ spin_unlock(&inode->i_lock);
return LRU_RETRY;
}
@@ -1177,6 +1185,10 @@ void unlock_new_inode(struct inode *inode)
lockdep_annotate_inode_mutex_key(inode);
spin_lock(&inode->i_lock);
WARN_ON(!(inode->i_state & I_NEW));
+ /*
+ * Pairs with smp_rmb in wait_on_inode().
+ */
+ smp_wmb();
inode->i_state &= ~I_NEW & ~I_CREATING;
/*
* Pairs with the barrier in prepare_to_wait_event() to make sure
@@ -1194,6 +1206,10 @@ void discard_new_inode(struct inode *inode)
lockdep_annotate_inode_mutex_key(inode);
spin_lock(&inode->i_lock);
WARN_ON(!(inode->i_state & I_NEW));
+ /*
+ * Pairs with smp_rmb in wait_on_inode().
+ */
+ smp_wmb();
inode->i_state &= ~I_NEW;
/*
* Pairs with the barrier in prepare_to_wait_event() to make sure
@@ -1875,6 +1891,7 @@ static void iput_final(struct inode *inode)
int drop;
WARN_ON(inode->i_state & I_NEW);
+ VFS_BUG_ON_INODE(atomic_read(&inode->i_count) != 0, inode);
if (op->drop_inode)
drop = op->drop_inode(inode);
@@ -1889,6 +1906,12 @@ static void iput_final(struct inode *inode)
return;
}
+ /*
+ * Re-check ->i_count in case the ->drop_inode() hooks played games.
+ * Note we only execute this if the verdict was to drop the inode.
+ */
+ VFS_BUG_ON_INODE(atomic_read(&inode->i_count) != 0, inode);
+
state = inode->i_state;
if (!drop) {
WRITE_ONCE(inode->i_state, state | I_WILL_FREE);
diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c
index c9ece378687b..cbd729ef50ab 100644
--- a/fs/iomap/buffered-io.c
+++ b/fs/iomap/buffered-io.c
@@ -766,6 +766,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter,
if (iomap_block_needs_zeroing(iter, block_start)) {
if (WARN_ON_ONCE(iter->flags & IOMAP_UNSHARE))
return -EIO;
+ /* uptodate once copied, see __iomap_write_end() */
folio_zero_segments(folio, poff, from, to, poff + plen);
} else {
int status;
@@ -781,8 +782,8 @@ static int __iomap_write_begin(const struct iomap_iter *iter,
folio, block_start, plen);
if (status)
return status;
+ iomap_set_range_uptodate(folio, poff, plen);
}
- iomap_set_range_uptodate(folio, poff, plen);
} while ((block_start += plen) < block_end);
return 0;
diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c
index bb276321c952..430114567425 100644
--- a/fs/iomap/direct-io.c
+++ b/fs/iomap/direct-io.c
@@ -24,13 +24,6 @@
#define IOMAP_DIO_WRITE (1U << 30)
#define IOMAP_DIO_DIRTY (1U << 31)
-/*
- * Used for sub block zeroing in iomap_dio_zero()
- */
-#define IOMAP_ZERO_PAGE_SIZE (SZ_64K)
-#define IOMAP_ZERO_PAGE_ORDER (get_order(IOMAP_ZERO_PAGE_SIZE))
-static struct page *zero_page;
-
struct iomap_dio {
struct kiocb *iocb;
const struct iomap_dio_ops *dops;
@@ -291,30 +284,39 @@ u32 iomap_finish_ioend_direct(struct iomap_ioend *ioend)
return vec_count;
}
-static int iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio,
+static void iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio,
loff_t pos, unsigned len)
{
struct inode *inode = file_inode(dio->iocb->ki_filp);
struct bio *bio;
+ struct folio *zero_folio = largest_zero_folio();
+ int nr_vecs = max(1, i_blocksize(inode) / folio_size(zero_folio));
if (!len)
- return 0;
+ return;
+
/*
- * Max block size supported is 64k
+ * This limit shall never be reached as most filesystems have a
+ * maximum blocksize of 64k.
*/
- if (WARN_ON_ONCE(len > IOMAP_ZERO_PAGE_SIZE))
- return -EINVAL;
+ if (WARN_ON_ONCE(nr_vecs > BIO_MAX_VECS))
+ return;
- bio = iomap_dio_alloc_bio(iter, dio, 1, REQ_OP_WRITE | REQ_SYNC | REQ_IDLE);
+ bio = iomap_dio_alloc_bio(iter, dio, nr_vecs,
+ REQ_OP_WRITE | REQ_SYNC | REQ_IDLE);
fscrypt_set_bio_crypt_ctx(bio, inode, pos >> inode->i_blkbits,
GFP_KERNEL);
bio->bi_iter.bi_sector = iomap_sector(&iter->iomap, pos);
bio->bi_private = dio;
bio->bi_end_io = iomap_dio_bio_end_io;
- __bio_add_page(bio, zero_page, len, 0);
+ while (len > 0) {
+ unsigned int io_len = min(len, folio_size(zero_folio));
+
+ bio_add_folio_nofail(bio, zero_folio, io_len, 0);
+ len -= io_len;
+ }
iomap_dio_submit_bio(iter, dio, bio, pos);
- return 0;
}
/*
@@ -420,10 +422,7 @@ static int iomap_dio_bio_iter(struct iomap_iter *iter, struct iomap_dio *dio)
if (need_zeroout) {
/* zero out from the start of the block to the write offset */
pad = pos & (fs_block_size - 1);
-
- ret = iomap_dio_zero(iter, dio, pos - pad, pad);
- if (ret)
- goto out;
+ iomap_dio_zero(iter, dio, pos - pad, pad);
}
nr_pages = bio_iov_vecs_to_alloc(dio->submit.iter, BIO_MAX_VECS);
@@ -505,8 +504,7 @@ static int iomap_dio_bio_iter(struct iomap_iter *iter, struct iomap_dio *dio)
/* zero out from the end of the write to the end of the block */
pad = pos & (fs_block_size - 1);
if (pad)
- ret = iomap_dio_zero(iter, dio, pos,
- fs_block_size - pad);
+ iomap_dio_zero(iter, dio, pos, fs_block_size - pad);
}
out:
/* Undo iter limitation to current extent */
@@ -841,15 +839,3 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,
return iomap_dio_complete(dio);
}
EXPORT_SYMBOL_GPL(iomap_dio_rw);
-
-static int __init iomap_dio_init(void)
-{
- zero_page = alloc_pages(GFP_KERNEL | __GFP_ZERO,
- IOMAP_ZERO_PAGE_ORDER);
-
- if (!zero_page)
- return -ENOMEM;
-
- return 0;
-}
-fs_initcall(iomap_dio_init);
diff --git a/fs/nfs/callback_xdr.c b/fs/nfs/callback_xdr.c
index 1aa6a6e14b05..34fa00f1a1c2 100644
--- a/fs/nfs/callback_xdr.c
+++ b/fs/nfs/callback_xdr.c
@@ -275,6 +275,13 @@ __be32 decode_devicenotify_args(struct svc_rqst *rqstp,
if (n == 0)
goto out;
+ /* sanity check the count against the remaining stream */
+ if (n > xdr_stream_remaining(xdr) /
+ ((4 * sizeof(uint32_t)) + NFS4_DEVICEID4_SIZE)) {
+ status = htonl(NFS4ERR_BADXDR);
+ goto out;
+ }
+
args->devs = kmalloc_array(n, sizeof(*args->devs), GFP_KERNEL);
if (!args->devs) {
status = htonl(NFS4ERR_DELAY);
diff --git a/fs/nfs/localio.c b/fs/nfs/localio.c
index 3c5891d3ab89..3138439d1910 100644
--- a/fs/nfs/localio.c
+++ b/fs/nfs/localio.c
@@ -682,6 +682,8 @@ static void nfs_local_call_read(struct work_struct *work)
n_iters = atomic_read(&iocb->n_iters);
for (int i = 0; i < n_iters ; i++) {
+ size_t expected;
+
if (iocb->iter_is_dio_aligned[i]) {
iocb->kiocb.ki_flags |= IOCB_DIRECT;
/* Only use AIO completion if DIO-aligned segment is last */
@@ -692,12 +694,14 @@ static void nfs_local_call_read(struct work_struct *work)
} else
iocb->kiocb.ki_flags &= ~IOCB_DIRECT;
+ /* read_iter() advances the iterator: measure it beforehand */
+ expected = iov_iter_count(&iocb->iters[i]);
save_cred = override_creds(filp->f_cred);
status = filp->f_op->read_iter(&iocb->kiocb, &iocb->iters[i]);
revert_creds(save_cred);
if (status != -EIOCBQUEUED) {
- if (unlikely(status >= 0 && status < iocb->iters[i].count))
+ if (unlikely(status >= 0 && (size_t)status < expected))
force_done = true; /* Partial read */
if (nfs_local_pgio_done(iocb, status, force_done)) {
nfs_local_read_iocb_done(iocb);
@@ -878,6 +882,8 @@ static void nfs_local_call_write(struct work_struct *work)
file_start_write(filp);
n_iters = atomic_read(&iocb->n_iters);
for (int i = 0; i < n_iters ; i++) {
+ size_t expected;
+
if (iocb->iter_is_dio_aligned[i]) {
iocb->kiocb.ki_flags |= IOCB_DIRECT;
/* Only use AIO completion if DIO-aligned segment is last */
@@ -888,12 +894,14 @@ static void nfs_local_call_write(struct work_struct *work)
} else
iocb->kiocb.ki_flags &= ~IOCB_DIRECT;
+ /* write_iter() advances the iterator: measure it beforehand */
+ expected = iov_iter_count(&iocb->iters[i]);
save_cred = override_creds(filp->f_cred);
status = filp->f_op->write_iter(&iocb->kiocb, &iocb->iters[i]);
revert_creds(save_cred);
if (status != -EIOCBQUEUED) {
- if (unlikely(status >= 0 && status < iocb->iters[i].count))
+ if (unlikely(status >= 0 && (size_t)status < expected))
force_done = true; /* Partial write */
if (nfs_local_pgio_done(iocb, status, force_done)) {
nfs_local_write_iocb_done(iocb);
diff --git a/fs/nfsd/blocklayout.c b/fs/nfsd/blocklayout.c
index e9512c34c9dc..ac1312fd1c45 100644
--- a/fs/nfsd/blocklayout.c
+++ b/fs/nfsd/blocklayout.c
@@ -2,7 +2,7 @@
/*
* Copyright (c) 2014-2016 Christoph Hellwig.
*/
-#include <linux/exportfs.h>
+#include <linux/exportfs_block.h>
#include <linux/iomap.h>
#include <linux/slab.h>
#include <linux/pr.h>
@@ -17,68 +17,44 @@
#define NFSDDBG_FACILITY NFSDDBG_PNFS
+/*
+ * Get an extent from the file system that starts at offset or below
+ * and may be shorter than the requested length.
+ */
static __be32
-nfsd4_block_proc_layoutget(struct svc_rqst *rqstp, struct inode *inode,
- const struct svc_fh *fhp, struct nfsd4_layoutget *args)
+nfsd4_block_map_extent(struct inode *inode, const struct svc_fh *fhp,
+ u64 offset, u64 length, u32 iomode, u64 minlength,
+ struct pnfs_block_extent *bex)
{
- struct nfsd4_layout_seg *seg = &args->lg_seg;
struct super_block *sb = inode->i_sb;
- u64 length;
- u32 block_size = i_blocksize(inode);
- struct pnfs_block_extent *bex;
struct iomap iomap;
u32 device_generation = 0;
int error;
- if (locks_in_grace(SVC_NET(rqstp)))
- return nfserr_grace;
-
- if (seg->offset & (block_size - 1)) {
- dprintk("pnfsd: I/O misaligned\n");
- goto out_layoutunavailable;
- }
-
- /*
- * Some clients barf on non-zero block numbers for NONE or INVALID
- * layouts, so make sure to zero the whole structure.
- */
- error = -ENOMEM;
- bex = kzalloc(sizeof(*bex), GFP_KERNEL);
- if (!bex)
- goto out_error;
- args->lg_content = bex;
-
- error = sb->s_export_op->map_blocks(inode, seg->offset, seg->length,
- &iomap, seg->iomode != IOMODE_READ,
- &device_generation);
+ error = sb->s_export_op->block_ops->map_blocks(inode, offset, length,
+ &iomap, iomode != IOMODE_READ, &device_generation);
if (error) {
if (error == -ENXIO)
- goto out_layoutunavailable;
- goto out_error;
- }
-
- length = iomap.offset + iomap.length - seg->offset;
- if (length < args->lg_minlength) {
- dprintk("pnfsd: extent smaller than minlength\n");
- goto out_layoutunavailable;
+ return nfserr_layoutunavailable;
+ return nfserrno(error);
}
switch (iomap.type) {
case IOMAP_MAPPED:
- if (seg->iomode == IOMODE_READ)
+ if (iomode == IOMODE_READ)
bex->es = PNFS_BLOCK_READ_DATA;
else
bex->es = PNFS_BLOCK_READWRITE_DATA;
bex->soff = iomap.addr;
break;
case IOMAP_UNWRITTEN:
- if (seg->iomode & IOMODE_RW) {
+ if (iomode & IOMODE_RW) {
/*
* Crack monkey special case from section 2.3.1.
*/
- if (args->lg_minlength == 0) {
+ if (minlength == 0) {
dprintk("pnfsd: no soup for you!\n");
- goto out_layoutunavailable;
+ return nfserr_layoutunavailable;
}
bex->es = PNFS_BLOCK_INVALID_DATA;
@@ -87,7 +63,7 @@ nfsd4_block_proc_layoutget(struct svc_rqst *rqstp, struct inode *inode,
}
fallthrough;
case IOMAP_HOLE:
- if (seg->iomode == IOMODE_READ) {
+ if (iomode == IOMODE_READ) {
bex->es = PNFS_BLOCK_NONE_DATA;
break;
}
@@ -95,27 +71,68 @@ nfsd4_block_proc_layoutget(struct svc_rqst *rqstp, struct inode *inode,
case IOMAP_DELALLOC:
default:
WARN(1, "pnfsd: filesystem returned %d extent\n", iomap.type);
- goto out_layoutunavailable;
+ return nfserr_layoutunavailable;
}
error = nfsd4_set_deviceid(&bex->vol_id, fhp, device_generation);
if (error)
- goto out_error;
+ return nfserrno(error);
+
bex->foff = iomap.offset;
bex->len = iomap.length;
+ return nfs_ok;
+}
+
+static __be32
+nfsd4_block_proc_layoutget(struct svc_rqst *rqstp, struct inode *inode,
+ const struct svc_fh *fhp, struct nfsd4_layoutget *args)
+{
+ struct nfsd4_layout_seg *seg = &args->lg_seg;
+ struct pnfs_block_extent *bex;
+ u64 length;
+ u32 block_size = i_blocksize(inode);
+ __be32 nfserr;
+
+ if (locks_in_grace(SVC_NET(rqstp)))
+ return nfserr_grace;
- seg->offset = iomap.offset;
- seg->length = iomap.length;
+ nfserr = nfserr_layoutunavailable;
+ if (seg->offset & (block_size - 1)) {
+ dprintk("pnfsd: I/O misaligned\n");
+ goto out_error;
+ }
+
+ /*
+ * Some clients barf on non-zero block numbers for NONE or INVALID
+ * layouts, so make sure to zero the whole structure.
+ */
+ nfserr = nfserrno(-ENOMEM);
+ bex = kzalloc(sizeof(*bex), GFP_KERNEL);
+ if (!bex)
+ goto out_error;
+ args->lg_content = bex;
+
+ nfserr = nfsd4_block_map_extent(inode, fhp, seg->offset, seg->length,
+ seg->iomode, args->lg_minlength, bex);
+ if (nfserr != nfs_ok)
+ goto out_error;
+
+ nfserr = nfserr_layoutunavailable;
+ length = bex->foff + bex->len - seg->offset;
+ if (length < args->lg_minlength) {
+ dprintk("pnfsd: extent smaller than minlength\n");
+ goto out_error;
+ }
+
+ seg->offset = bex->foff;
+ seg->length = bex->len;
dprintk("GET: 0x%llx:0x%llx %d\n", bex->foff, bex->len, bex->es);
- return 0;
+ return nfs_ok;
out_error:
seg->length = 0;
- return nfserrno(error);
-out_layoutunavailable:
- seg->length = 0;
- return nfserr_layoutunavailable;
+ return nfserr;
}
static __be32
@@ -137,8 +154,8 @@ nfsd4_block_commit_blocks(struct inode *inode, struct nfsd4_layoutcommit *lcp,
iattr.ia_size = lcp->lc_newsize;
}
- error = inode->i_sb->s_export_op->commit_blocks(inode, iomaps,
- nr_iomaps, &iattr);
+ error = inode->i_sb->s_export_op->block_ops->commit_blocks(inode,
+ iomaps, nr_iomaps, &iattr);
kfree(iomaps);
return nfserrno(error);
}
@@ -161,8 +178,8 @@ nfsd4_block_get_device_info_simple(struct super_block *sb,
b->type = PNFS_BLOCK_VOLUME_SIMPLE;
b->simple.sig_len = PNFS_BLOCK_UUID_LEN;
- return sb->s_export_op->get_uuid(sb, b->simple.sig, &b->simple.sig_len,
- &b->simple.offset);
+ return sb->s_export_op->block_ops->get_uuid(sb, b->simple.sig,
+ &b->simple.sig_len, &b->simple.offset);
}
static __be32
diff --git a/fs/nfsd/export.c b/fs/nfsd/export.c
index 2a1499f2ad19..e33843a937dd 100644
--- a/fs/nfsd/export.c
+++ b/fs/nfsd/export.c
@@ -722,7 +722,8 @@ static int svc_export_parse(struct cache_detail *cd, char *mesg, int mlen)
goto out4;
err = 0;
- nfsd4_setup_layout_type(&exp);
+ if (exp.ex_flags & NFSEXP_PNFS)
+ nfsd4_setup_layout_type(&exp);
}
expp = svc_export_lookup(&exp);
diff --git a/fs/nfsd/nfs2acl.c b/fs/nfsd/nfs2acl.c
index 2998640f259d..87269be93d0f 100644
--- a/fs/nfsd/nfs2acl.c
+++ b/fs/nfsd/nfs2acl.c
@@ -15,6 +15,62 @@
#define NFSDDBG_FACILITY NFSDDBG_PROC
+/*
+ * These maps are identical to the NFSv3 maps (nfs3proc.c). This enables
+ * the behavior of the two versions to diverge if needed.
+ */
+static const struct nfsd_access_map nfsd2_regaccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_map nfsd2_diraccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_LOOKUP, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE },
+ { NFS3_ACCESS_DELETE, NFSD_MAY_REMOVE },
+ { 0, 0 }
+};
+
+/*
+ * Some clients - Solaris 2.6 at least, make an access call to the NFS
+ * server to check for access for things like /dev/null (which really,
+ * NFSD doesn't care about). So NFSD provides simple access checking
+ * for those objects, looking mainly at mode bits, ignoring read-only
+ * filesystem checks.
+ */
+static const struct nfsd_access_map nfsd2_otheraccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_maps nfsd2_access_maps = {
+ .regular = nfsd2_regaccess,
+ .directory = nfsd2_diraccess,
+ .other = nfsd2_otheraccess,
+};
+
+static __be32 nfsacld_map_status(__be32 status)
+{
+ switch (status) {
+ case nfserr_nofilehandle:
+ case nfserr_badhandle:
+ status = nfserr_stale;
+ break;
+ case nfserr_wrongsec:
+ status = nfserr_acces;
+ break;
+ }
+ return status;
+}
+
/*
* NULL call.
*/
@@ -79,6 +135,7 @@ static __be32 nfsacld_proc_getacl(struct svc_rqst *rqstp)
/* resp->acl_{access,default} are released in nfssvc_release_getacl. */
out:
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
fail:
@@ -137,6 +194,7 @@ static __be32 nfsacld_proc_setacl(struct svc_rqst *rqstp)
out:
/* argp->acl_{access,default} are released in nfsaclsvc_release_setacl. */
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
out_drop_lock:
@@ -163,6 +221,7 @@ static __be32 nfsacld_proc_getattr(struct svc_rqst *rqstp)
goto out;
resp->status = fh_getattr(&resp->fh, &resp->stat);
out:
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
}
@@ -180,11 +239,14 @@ static __be32 nfsacld_proc_access(struct svc_rqst *rqstp)
fh_copy(&resp->fh, &argp->fh);
resp->access = argp->access;
- resp->status = nfsd_access(rqstp, &resp->fh, &resp->access, NULL);
+
+ resp->status = nfsd_access(rqstp, &resp->fh, &nfsd2_access_maps,
+ &resp->access, NULL);
if (resp->status != nfs_ok)
goto out;
resp->status = fh_getattr(&resp->fh, &resp->stat);
out:
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
}
diff --git a/fs/nfsd/nfs3proc.c b/fs/nfsd/nfs3proc.c
index fbf573c31719..87c5e766133c 100644
--- a/fs/nfsd/nfs3proc.c
+++ b/fs/nfsd/nfs3proc.c
@@ -48,6 +48,58 @@ static bool nfsd3_time_in_range(const struct iattr *iap)
return true;
}
+static const struct nfsd_access_map nfsd3_regaccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_map nfsd3_diraccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_LOOKUP, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE },
+ { NFS3_ACCESS_DELETE, NFSD_MAY_REMOVE },
+ { 0, 0 }
+};
+
+/*
+ * Some clients - Solaris 2.6 at least, make an access call to the NFS
+ * server to check for access for things like /dev/null (which really,
+ * NFSD doesn't care about). So NFSD provides simple access checking
+ * for those objects, looking mainly at mode bits, ignoring read-only
+ * filesystem checks.
+ */
+static const struct nfsd_access_map nfsd3_otheraccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_maps nfsd3_access_maps = {
+ .regular = nfsd3_regaccess,
+ .directory = nfsd3_diraccess,
+ .other = nfsd3_otheraccess,
+};
+
+static int nfsd3_iocb_flags(enum nfs3_stable_how how)
+{
+ switch (how) {
+ case NFS_FILE_SYNC:
+ /* persist data and timestamps */
+ return IOCB_DSYNC | IOCB_SYNC;
+ case NFS_DATA_SYNC:
+ /* persist data only */
+ return IOCB_DSYNC;
+ default:
+ return 0;
+ }
+}
+
static __be32 nfsd3_map_status(__be32 status)
{
switch (status) {
@@ -171,7 +223,8 @@ nfsd3_proc_access(struct svc_rqst *rqstp)
fh_copy(&resp->fh, &argp->fh);
resp->access = argp->access;
- resp->status = nfsd_access(rqstp, &resp->fh, &resp->access, NULL);
+ resp->status = nfsd_access(rqstp, &resp->fh, &nfsd3_access_maps,
+ &resp->access, NULL);
resp->status = nfsd3_map_status(resp->status);
return rpc_success;
}
@@ -260,7 +313,8 @@ nfsd3_proc_write(struct svc_rqst *rqstp)
resp->committed = argp->stable;
resp->status = nfsd_write(rqstp, &resp->fh, argp->offset,
&argp->payload, &cnt,
- resp->committed, resp->verf);
+ nfsd3_iocb_flags(resp->committed),
+ resp->verf);
resp->count = cnt;
resp->status = nfsd3_map_status(resp->status);
return rpc_success;
diff --git a/fs/nfsd/nfs3xdr.c b/fs/nfsd/nfs3xdr.c
index ef4971d71ac4..067ae1c8aaa5 100644
--- a/fs/nfsd/nfs3xdr.c
+++ b/fs/nfsd/nfs3xdr.c
@@ -556,6 +556,8 @@ nfs3svc_decode_writeargs(struct svc_rqst *rqstp, struct xdr_stream *xdr)
return false;
if (xdr_stream_decode_u32(xdr, &args->stable) < 0)
return false;
+ if (args->stable > NFS_FILE_SYNC)
+ return false;
/* opaque data */
if (xdr_stream_decode_u32(xdr, &args->len) < 0)
diff --git a/fs/nfsd/nfs4layouts.c b/fs/nfsd/nfs4layouts.c
index 2121ef26e7a9..8ec980c69bec 100644
--- a/fs/nfsd/nfs4layouts.c
+++ b/fs/nfsd/nfs4layouts.c
@@ -2,7 +2,7 @@
/*
* Copyright (c) 2014 Christoph Hellwig.
*/
-#include <linux/blkdev.h>
+#include <linux/exportfs_block.h>
#include <linux/kmod.h>
#include <linux/file.h>
#include <linux/jhash.h>
@@ -125,30 +125,19 @@ nfsd4_set_deviceid(struct nfsd4_deviceid *id, const struct svc_fh *fhp,
void nfsd4_setup_layout_type(struct svc_export *exp)
{
-#if defined(CONFIG_NFSD_BLOCKLAYOUT) || defined(CONFIG_NFSD_SCSILAYOUT)
+ struct nfsd_net *nn = net_generic(exp->cd->net, nfsd_net_id);
struct super_block *sb = exp->ex_path.mnt->mnt_sb;
-#endif
-
- if (!(exp->ex_flags & NFSEXP_PNFS))
- return;
+ expfs_block_layouts_t block_supported = exportfs_layouts_supported(sb);
-#ifdef CONFIG_NFSD_FLEXFILELAYOUT
- exp->ex_layout_types |= 1 << LAYOUT_FLEX_FILES;
-#endif
-#ifdef CONFIG_NFSD_BLOCKLAYOUT
- if (sb->s_export_op->get_uuid &&
- sb->s_export_op->map_blocks &&
- sb->s_export_op->commit_blocks)
+ if (IS_ENABLED(CONFIG_NFSD_FLEXFILELAYOUT) &&
+ nfsd_vers(nn, 3, NFSD_TEST))
+ exp->ex_layout_types |= 1 << LAYOUT_FLEX_FILES;
+ if (IS_ENABLED(CONFIG_NFSD_BLOCKLAYOUT) &&
+ (block_supported & EXPFS_BLOCK_IN_BAND_ID))
exp->ex_layout_types |= 1 << LAYOUT_BLOCK_VOLUME;
-#endif
-#ifdef CONFIG_NFSD_SCSILAYOUT
- if (sb->s_export_op->map_blocks &&
- sb->s_export_op->commit_blocks &&
- sb->s_bdev &&
- sb->s_bdev->bd_disk->fops->pr_ops &&
- sb->s_bdev->bd_disk->fops->get_unique_id)
+ if (IS_ENABLED(CONFIG_NFSD_SCSILAYOUT) &&
+ (block_supported & EXPFS_BLOCK_OUT_OF_BAND_ID))
exp->ex_layout_types |= 1 << LAYOUT_SCSI;
-#endif
}
void nfsd4_close_layout(struct nfs4_layout_stateid *ls)
diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 1bf4a84d6a19..6c33eebd1a14 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -68,6 +68,57 @@ MODULE_PARM_DESC(nfsd4_ssc_umount_timeout,
#define NFSDDBG_FACILITY NFSDDBG_PROC
+static const struct nfsd_access_map nfsd4_regaccess[] = {
+ { NFS4_ACCESS_READ, NFSD_MAY_READ },
+ { NFS4_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS4_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS4_ACCESS_EXTEND, NFSD_MAY_WRITE },
+ { NFS4_ACCESS_XAREAD, NFSD_MAY_READ },
+ { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE },
+ { NFS4_ACCESS_XALIST, NFSD_MAY_READ },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_map nfsd4_diraccess[] = {
+ { NFS4_ACCESS_READ, NFSD_MAY_READ },
+ { NFS4_ACCESS_LOOKUP, NFSD_MAY_EXEC },
+ { NFS4_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS4_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE },
+ { NFS4_ACCESS_DELETE, NFSD_MAY_REMOVE },
+ { NFS4_ACCESS_XAREAD, NFSD_MAY_READ },
+ { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE },
+ { NFS4_ACCESS_XALIST, NFSD_MAY_READ },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_map nfsd4_otheraccess[] = {
+ { NFS4_ACCESS_READ, NFSD_MAY_READ },
+ { NFS4_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS4_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { NFS4_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_maps nfsd4_access_maps = {
+ .regular = nfsd4_regaccess,
+ .directory = nfsd4_diraccess,
+ .other = nfsd4_otheraccess,
+};
+
+static int nfsd4_iocb_flags(enum stable_how4 how)
+{
+ switch (how) {
+ case FILE_SYNC4:
+ /* persist data and timestamps */
+ return IOCB_DSYNC | IOCB_SYNC;
+ case DATA_SYNC4:
+ /* persist data only */
+ return IOCB_DSYNC;
+ default:
+ return 0;
+ }
+}
+
static u32 nfsd_attrmask[] = {
NFSD_WRITEABLE_ATTRS_WORD0,
NFSD_WRITEABLE_ATTRS_WORD1,
@@ -750,10 +801,9 @@ nfsd4_access(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
if (access->ac_req_access & ~access_full)
return nfserr_inval;
-
access->ac_resp_access = access->ac_req_access;
- return nfsd_access(rqstp, &cstate->current_fh, &access->ac_resp_access,
- &access->ac_supported);
+ return nfsd_access(rqstp, &cstate->current_fh, &nfsd4_access_maps,
+ &access->ac_resp_access, &access->ac_supported);
}
static __be32
@@ -1302,7 +1352,7 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
write->wr_how_written = write->wr_stable_how;
status = nfsd_vfs_write(rqstp, &cstate->current_fh, nf,
write->wr_offset, &write->wr_payload,
- &cnt, write->wr_how_written,
+ &cnt, nfsd4_iocb_flags(write->wr_how_written),
(__be32 *)write->wr_verifier.data);
nfsd_file_put(nf);
@@ -1826,7 +1876,7 @@ static void nfsd4_init_copy_res(struct nfsd4_copy *copy, bool sync)
{
copy->cp_res.wr_stable_how =
test_bit(NFSD4_COPY_F_COMMITTED, ©->cp_flags) ?
- NFS_FILE_SYNC : NFS_UNSTABLE;
+ FILE_SYNC4 : UNSTABLE4;
nfsd4_copy_set_sync(copy, sync);
}
diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
index d04a7d3aabbc..83aba7ad3908 100644
--- a/fs/nfsd/nfs4recover.c
+++ b/fs/nfsd/nfs4recover.c
@@ -711,6 +711,7 @@ struct cld_upcall {
struct completion cu_done;
/* daemon has read the whole upcall; protected by cn_lock */
bool cu_inflight;
+ struct rpc_pipe_msg cu_pipe_msg;
union {
struct cld_msg_hdr cu_hdr;
struct cld_msg cu_msg;
@@ -722,22 +723,22 @@ static int
__cld_pipe_upcall(struct rpc_pipe *pipe, void *cmsg, struct nfsd_net *nn)
{
int ret;
- struct rpc_pipe_msg msg;
struct cld_upcall *cup = container_of(cmsg, struct cld_upcall, cu_u);
+ struct rpc_pipe_msg *msg = &cup->cu_pipe_msg;
- memset(&msg, 0, sizeof(msg));
- msg.data = cmsg;
- msg.len = nn->client_tracking_ops->msglen;
+ memset(msg, 0, sizeof(*msg));
+ msg->data = cmsg;
+ msg->len = nn->client_tracking_ops->msglen;
- ret = rpc_queue_upcall(pipe, &msg);
+ ret = rpc_queue_upcall(pipe, msg);
if (ret < 0) {
goto out;
}
wait_for_completion(&cup->cu_done);
- if (msg.errno < 0)
- ret = msg.errno;
+ if (msg->errno < 0)
+ ret = msg->errno;
out:
return ret;
}
@@ -900,8 +901,11 @@ cld_pipe_downcall(struct file *filp, const char __user *src, size_t mlen)
if (status == -EINPROGRESS)
return __cld_pipe_inprogress_downcall(cmsg, nn);
- if (copy_from_user(&cup->cu_u.cu_msg_v2, src, mlen) != 0)
+ if (copy_from_user(&cup->cu_u.cu_msg_v2, src, mlen) != 0) {
+ cup->cu_pipe_msg.errno = -EFAULT;
+ complete(&cup->cu_done);
return -EFAULT;
+ }
complete(&cup->cu_done);
return mlen;
@@ -910,9 +914,8 @@ cld_pipe_downcall(struct file *filp, const char __user *src, size_t mlen)
static void
cld_pipe_destroy_msg(struct rpc_pipe_msg *msg)
{
- struct cld_msg *cmsg = msg->data;
- struct cld_upcall *cup = container_of(cmsg, struct cld_upcall,
- cu_u.cu_msg);
+ struct cld_upcall *cup = container_of(msg, struct cld_upcall,
+ cu_pipe_msg);
struct cld_net *cn = cup->cu_net;
/*
@@ -929,10 +932,34 @@ cld_pipe_destroy_msg(struct rpc_pipe_msg *msg)
complete(&cup->cu_done);
}
+/*
+ * An upcall the daemon has consumed is off every pipe list, so the
+ * purge in rpc_pipe_release() and rpc_close_pipes() cannot reach it.
+ * Release its waiter here instead; no downcall can arrive now.
+ */
+static void
+cld_release_pipe(struct inode *inode)
+{
+ struct nfsd_net *nn = net_generic(inode->i_sb->s_fs_info, nfsd_net_id);
+ struct cld_net *cn = nn->cld_net;
+ struct cld_upcall *cup;
+
+ spin_lock(&cn->cn_lock);
+ list_for_each_entry(cup, &cn->cn_list, cu_list) {
+ if (!cup->cu_inflight)
+ continue;
+ cup->cu_inflight = false;
+ cup->cu_pipe_msg.errno = -EPIPE;
+ complete(&cup->cu_done);
+ }
+ spin_unlock(&cn->cn_lock);
+}
+
static const struct rpc_pipe_ops cld_upcall_ops = {
.upcall = rpc_pipe_generic_upcall,
.downcall = cld_pipe_downcall,
.destroy_msg = cld_pipe_destroy_msg,
+ .release_pipe = cld_release_pipe,
};
static int
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 42035bfebda7..44ca5746911c 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1423,6 +1423,7 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp)
dp->dl_stid.sc_type = SC_TYPE_DELEG;
list_add(&dp->dl_perfile, &fp->fi_delegations);
list_add(&dp->dl_perclnt, &clp->cl_delegations);
+ clp->cl_deleg_count++;
return 0;
}
@@ -1454,6 +1455,7 @@ unhash_delegation_locked(struct nfs4_delegation *dp, unsigned short statusmask)
++dp->dl_time;
spin_lock(&fp->fi_lock);
list_del_init(&dp->dl_perclnt);
+ dp->dl_stid.sc_client->cl_deleg_count--;
list_del_init(&dp->dl_recall_lru);
list_del_init(&dp->dl_perfile);
spin_unlock(&fp->fi_lock);
@@ -7304,6 +7306,7 @@ deleg_reaper(struct nfsd_net *nn)
{
struct list_head *pos, *next;
struct nfs4_client *clp;
+ unsigned int count;
spin_lock(&nn->client_lock);
list_for_each_safe(pos, next, &nn->client_lru) {
@@ -7313,21 +7316,33 @@ deleg_reaper(struct nfsd_net *nn)
continue;
if (clp->cl_state != NFSD4_ACTIVE)
continue;
- if (list_empty(&clp->cl_delegations))
- continue;
if (atomic_read(&clp->cl_delegs_in_recall))
continue;
if (ktime_get_boottime_seconds() - clp->cl_ra_time < 5)
continue;
if (clp->cl_cb_state != NFSD4_CB_UP)
continue;
+ /*
+ * This read races with hash_delegation_locked() and
+ * unhash_delegation_locked() on other CPUs. A stale
+ * count only skews the keep value; the next
+ * laundromat pass sees a more current one.
+ */
+ count = data_race(READ_ONCE(clp->cl_deleg_count));
+ if (!count)
+ continue;
if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &clp->cl_ra->ra_cb.cb_flags))
continue;
/* release in nfsd4_cb_recall_any_release */
kref_get(&clp->cl_nfsdfs.cl_ref);
clp->cl_ra_time = ktime_get_boottime_seconds();
- clp->cl_ra->ra_keep = 0;
+ /*
+ * Ask for a single delegation. Recalling one before it
+ * is needed costs the client an OPEN when it next
+ * touches the file.
+ */
+ clp->cl_ra->ra_keep = count - 1;
clp->cl_ra->ra_bmval[0] = BIT(RCA4_TYPE_MASK_RDATA_DLG) |
BIT(RCA4_TYPE_MASK_WDATA_DLG);
trace_nfsd_cb_recall_any(clp->cl_ra);
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 2fe4767dca5b..7b243d9d7173 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -1458,7 +1458,7 @@ nfsd4_decode_write(struct nfsd4_compoundargs *argp, union nfsd4_op_u *u)
return nfserr_bad_xdr;
if (xdr_stream_decode_u32(argp->xdr, &write->wr_stable_how) < 0)
return nfserr_bad_xdr;
- if (write->wr_stable_how > NFS_FILE_SYNC)
+ if (write->wr_stable_how > FILE_SYNC4)
return nfserr_bad_xdr;
if (xdr_stream_decode_u32(argp->xdr, &write->wr_buflen) < 0)
return nfserr_bad_xdr;
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 967c5bf30d05..15fb9ffc12a7 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -2026,8 +2026,13 @@ static int nfsd_nl_validate_listeners(struct genl_info *info)
int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
{
struct net *net = genl_info_net(info);
+ const struct nlattr *bad_attr = NULL;
struct svc_xprt *xprt, *tmp;
+ const char *bad_xprt = NULL;
+ unsigned int rpcb_failures;
const struct nlattr *attr;
+ bool skipped_rpcb = false;
+ bool bad_rpcb = false;
struct svc_serv *serv;
LIST_HEAD(permsocks);
struct nfsd_net *nn;
@@ -2117,13 +2122,16 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
if (delete)
svc_xprt_destroy_all(serv, net, false);
+ rpcb_failures = svc_rpcb_failure_count(serv);
+
/* walk list of addrs again, open any that still don't exist */
nlmsg_for_each_attr_type(attr, NFSD_A_SERVER_SOCK_ADDR, info->nlhdr,
GENL_HDRLEN, rem) {
struct nlattr *tb[NFSD_A_SOCK_MAX + 1];
const char *xcl_name;
struct sockaddr *sa;
- int ret;
+ bool hit_rpcb;
+ int flags, ret;
/* validated up front in nfsd_nl_validate_listeners() */
if (nla_parse_nested(tb, NFSD_A_SOCK_MAX, attr,
@@ -2142,11 +2150,53 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
continue;
}
- ret = svc_xprt_create_from_sa(serv, xcl_name, net, sa, 0,
+ flags = skipped_rpcb ? SVC_SOCK_ANONYMOUS : 0;
+ ret = svc_xprt_create_from_sa(serv, xcl_name, net, sa, flags,
current_cred());
+
+ hit_rpcb = false;
+ if (!skipped_rpcb &&
+ svc_rpcb_failure_count(serv) != rpcb_failures) {
+ skipped_rpcb = true;
+ hit_rpcb = true;
+ if (ret < 0)
+ ret = svc_xprt_create_from_sa(serv, xcl_name,
+ net, sa,
+ SVC_SOCK_ANONYMOUS,
+ current_cred());
+ }
+
/* always save the latest error */
- if (ret < 0)
+ if (ret < 0) {
+ bad_attr = attr;
+ bad_xprt = xcl_name;
+ bad_rpcb = hit_rpcb;
err = ret;
+ }
+ }
+
+ /*
+ * The ack carries the errno of the last entry that failed. Point at
+ * that entry as well, since several entries can share a transport
+ * name and the errno alone cannot tell them apart.
+ */
+ if (err) {
+ NL_SET_BAD_ATTR(info->extack, bad_attr);
+ if (bad_rpcb)
+ NL_SET_ERR_MSG_FMT(info->extack,
+ "cannot create %s listener; rpcbind did not answer",
+ bad_xprt);
+ else if (skipped_rpcb)
+ NL_SET_ERR_MSG_FMT(info->extack,
+ "cannot create %s listener; rpcbind did not answer earlier",
+ bad_xprt);
+ else
+ NL_SET_ERR_MSG_FMT(info->extack,
+ "cannot create %s listener",
+ bad_xprt);
+ } else if (skipped_rpcb) {
+ NL_SET_ERR_MSG(info->extack,
+ "rpcbind did not answer, some listeners are not registered");
}
if (!serv->sv_nrthreads && list_empty(&nn->nfsd_serv->sv_permsocks))
diff --git a/fs/nfsd/nfsproc.c b/fs/nfsd/nfsproc.c
index c561d7014b49..aab68a65ad56 100644
--- a/fs/nfsd/nfsproc.c
+++ b/fs/nfsd/nfsproc.c
@@ -280,7 +280,7 @@ nfsd_proc_write(struct svc_rqst *rqstp)
fh_copy(&resp->fh, &argp->fh);
resp->status = nfsd_write(rqstp, &resp->fh, argp->offset,
- &argp->payload, &cnt, NFS_DATA_SYNC, NULL);
+ &argp->payload, &cnt, IOCB_DSYNC, NULL);
if (resp->status == nfs_ok)
resp->status = fh_getattr(&resp->fh, &resp->stat);
else if (resp->status == nfserr_jukebox)
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index 926e4e8063b3..fd61bac52721 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -525,6 +525,8 @@ struct nfs4_client {
unsigned int cl_state;
atomic_t cl_delegs_in_recall;
+ /* Length of cl_delegations, updated under nn->deleg_lock */
+ unsigned int cl_deleg_count;
struct nfsd4_cb_recall_any *cl_ra;
time64_t cl_ra_time;
diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c
index 53b55ee4e00e..7094902ca93a 100644
--- a/fs/nfsd/vfs.c
+++ b/fs/nfsd/vfs.c
@@ -753,64 +753,21 @@ __be32 nfsd4_vfs_fallocate(struct svc_rqst *rqstp, struct svc_fh *fhp,
}
#endif /* defined(CONFIG_NFSD_V4) */
-/*
- * Check server access rights to a file system object
+/**
+ * nfsd_access - Check caller's access rights to a file system object
+ * @rqstp: RPC transaction context
+ * @fhp: target NFS filehandle
+ * @maps: tables mapping on-the-wire access bits to NFSD_MAY flags
+ * @access: requested access bits on entry, permitted bits on return
+ * @supported: optional output of the access bits the server supports
+ *
+ * Return: nfs_ok on success, otherwise an nfserr status code
*/
-struct accessmap {
- u32 access;
- int how;
-};
-static struct accessmap nfs3_regaccess[] = {
- { NFS3_ACCESS_READ, NFSD_MAY_READ },
- { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
- { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
- { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE },
-
-#ifdef CONFIG_NFSD_V4
- { NFS4_ACCESS_XAREAD, NFSD_MAY_READ },
- { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE },
- { NFS4_ACCESS_XALIST, NFSD_MAY_READ },
-#endif
-
- { 0, 0 }
-};
-
-static struct accessmap nfs3_diraccess[] = {
- { NFS3_ACCESS_READ, NFSD_MAY_READ },
- { NFS3_ACCESS_LOOKUP, NFSD_MAY_EXEC },
- { NFS3_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC},
- { NFS3_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE },
- { NFS3_ACCESS_DELETE, NFSD_MAY_REMOVE },
-
-#ifdef CONFIG_NFSD_V4
- { NFS4_ACCESS_XAREAD, NFSD_MAY_READ },
- { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE },
- { NFS4_ACCESS_XALIST, NFSD_MAY_READ },
-#endif
-
- { 0, 0 }
-};
-
-static struct accessmap nfs3_anyaccess[] = {
- /* Some clients - Solaris 2.6 at least, make an access call
- * to the server to check for access for things like /dev/null
- * (which really, the server doesn't care about). So
- * We provide simple access checking for them, looking
- * mainly at mode bits, and we make sure to ignore read-only
- * filesystem checks
- */
- { NFS3_ACCESS_READ, NFSD_MAY_READ },
- { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
- { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
- { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
-
- { 0, 0 }
-};
-
-__be32
-nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, u32 *access, u32 *supported)
+__be32 nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp,
+ const struct nfsd_access_maps *maps,
+ u32 *access, u32 *supported)
{
- struct accessmap *map;
+ const struct nfsd_access_map *map;
struct svc_export *export;
struct dentry *dentry;
u32 query, result = 0, sresult = 0;
@@ -824,12 +781,11 @@ nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, u32 *access, u32 *suppor
dentry = fhp->fh_dentry;
if (d_is_reg(dentry))
- map = nfs3_regaccess;
+ map = maps->regular;
else if (d_is_dir(dentry))
- map = nfs3_diraccess;
+ map = maps->directory;
else
- map = nfs3_anyaccess;
-
+ map = maps->other;
query = *access;
for (; map->access; map++) {
@@ -839,7 +795,7 @@ nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, u32 *access, u32 *suppor
sresult |= map->access;
err2 = nfsd_permission(&rqstp->rq_cred, export,
- dentry, map->how);
+ dentry, map->may);
switch (err2) {
case nfs_ok:
result |= map->access;
@@ -1185,7 +1141,7 @@ static int wait_for_concurrent_writes(struct file *file)
* @offset: Byte offset of start
* @payload: xdr_buf containing the write payload
* @cnt: IN: number of bytes to write, OUT: number of bytes actually written
- * @stable: An NFS stable_how value
+ * @iocb_flags: VFS IOCB_* flags expressing the requested write stability
* @verf: NFS WRITE verifier
*
* Upon return, caller must invoke fh_put on @fhp.
@@ -1197,7 +1153,7 @@ __be32
nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
struct nfsd_file *nf, loff_t offset,
const struct xdr_buf *payload, unsigned long *cnt,
- int stable, __be32 *verf)
+ int iocb_flags, __be32 *verf)
{
struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
struct file *file = nf->nf_file;
@@ -1234,21 +1190,11 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
exp = fhp->fh_export;
if (!EX_ISSYNC(exp))
- stable = NFS_UNSTABLE;
+ iocb_flags = 0;
init_sync_kiocb(&kiocb, file);
kiocb.ki_pos = offset;
- if (likely(!fhp->fh_use_wgather)) {
- switch (stable) {
- case NFS_FILE_SYNC:
- /* persist data and timestamps */
- kiocb.ki_flags |= IOCB_DSYNC | IOCB_SYNC;
- break;
- case NFS_DATA_SYNC:
- /* persist data only */
- kiocb.ki_flags |= IOCB_DSYNC;
- break;
- }
- }
+ if (likely(!fhp->fh_use_wgather))
+ kiocb.ki_flags |= iocb_flags;
nvecs = xdr_buf_to_bvec(rqstp->rq_bvec, rqstp->rq_maxpages, payload);
if (nvecs < 0) {
@@ -1284,7 +1230,7 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
goto out_nfserr;
}
- if (stable && fhp->fh_use_wgather) {
+ if (iocb_flags && fhp->fh_use_wgather) {
host_err = wait_for_concurrent_writes(file);
if (host_err < 0)
commit_reset_write_verifier(nn, rqstp, host_err);
@@ -1375,7 +1321,7 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
* @offset: Byte offset of start
* @payload: xdr_buf containing the write payload
* @cnt: IN: number of bytes to write, OUT: number of bytes actually written
- * @stable: An NFS stable_how value
+ * @iocb_flags: VFS IOCB_* flags expressing the requested write stability
* @verf: NFS WRITE verifier
*
* Upon return, caller must invoke fh_put on @fhp.
@@ -1385,8 +1331,8 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
*/
__be32
nfsd_write(struct svc_rqst *rqstp, struct svc_fh *fhp, loff_t offset,
- const struct xdr_buf *payload, unsigned long *cnt, int stable,
- __be32 *verf)
+ const struct xdr_buf *payload, unsigned long *cnt,
+ int iocb_flags, __be32 *verf)
{
struct nfsd_file *nf;
__be32 err;
@@ -1398,7 +1344,7 @@ nfsd_write(struct svc_rqst *rqstp, struct svc_fh *fhp, loff_t offset,
goto out;
err = nfsd_vfs_write(rqstp, fhp, nf, offset, payload, cnt,
- stable, verf);
+ iocb_flags, verf);
nfsd_file_put(nf);
out:
trace_nfsd_write_done(rqstp, fhp, offset, *cnt);
diff --git a/fs/nfsd/vfs.h b/fs/nfsd/vfs.h
index 1dd3ae3ceb3a..7f6b90b6272d 100644
--- a/fs/nfsd/vfs.h
+++ b/fs/nfsd/vfs.h
@@ -38,6 +38,17 @@
#define NFSD_MAY_CREATE (NFSD_MAY_EXEC|NFSD_MAY_WRITE)
#define NFSD_MAY_REMOVE (NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC)
+struct nfsd_access_map {
+ u32 access;
+ int may;
+};
+
+struct nfsd_access_maps {
+ const struct nfsd_access_map *regular;
+ const struct nfsd_access_map *directory;
+ const struct nfsd_access_map *other;
+};
+
struct nfsd_file;
/*
@@ -96,7 +107,9 @@ __be32 nfsd_create_locked(struct svc_rqst *, struct svc_fh *,
__be32 nfsd_create(struct svc_rqst *, struct svc_fh *,
char *name, int len, struct nfsd_attrs *attrs,
int type, dev_t rdev, struct svc_fh *res);
-__be32 nfsd_access(struct svc_rqst *, struct svc_fh *, u32 *, u32 *);
+__be32 nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp,
+ const struct nfsd_access_maps *maps,
+ u32 *access, u32 *supported);
__be32 nfsd_create_setattr(struct svc_rqst *rqstp, struct svc_fh *fhp,
struct svc_fh *resfhp, struct nfsd_attrs *iap);
__be32 nfsd_commit(struct svc_rqst *rqst, struct svc_fh *fhp,
@@ -131,11 +144,13 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
u32 *eof);
__be32 nfsd_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
loff_t offset, const struct xdr_buf *payload,
- unsigned long *cnt, int stable, __be32 *verf);
+ unsigned long *cnt, int iocb_flags,
+ __be32 *verf);
__be32 nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
struct nfsd_file *nf, loff_t offset,
const struct xdr_buf *payload,
- unsigned long *cnt, int stable, __be32 *verf);
+ unsigned long *cnt, int iocb_flags,
+ __be32 *verf);
__be32 nfsd_readlink(struct svc_rqst *, struct svc_fh *,
char *, int *);
__be32 nfsd_symlink(struct svc_rqst *, struct svc_fh *,
diff --git a/fs/nfsd/xdr3.h b/fs/nfsd/xdr3.h
index 522067b7fd75..7b5e3e3bac00 100644
--- a/fs/nfsd/xdr3.h
+++ b/fs/nfsd/xdr3.h
@@ -38,7 +38,7 @@ struct nfsd3_writeargs {
svc_fh fh;
__u64 offset;
__u32 count;
- int stable;
+ __u32 stable;
__u32 len;
struct xdr_buf payload;
};
diff --git a/fs/notify/inotify/inotify_fsnotify.c b/fs/notify/inotify/inotify_fsnotify.c
index 7c326ec2e8a8..6e3af6452dcc 100644
--- a/fs/notify/inotify/inotify_fsnotify.c
+++ b/fs/notify/inotify/inotify_fsnotify.c
@@ -36,7 +36,7 @@ static bool event_compare(struct fsnotify_event *old_fsn,
old = INOTIFY_E(old_fsn);
new = INOTIFY_E(new_fsn);
- if (old->mask & FS_IN_IGNORED)
+ if (old->mask & (FS_IN_IGNORED | FS_MOVE))
return false;
if ((old->mask == new->mask) &&
(old->wd == new->wd) &&
diff --git a/fs/ntfs3/xattr.c b/fs/ntfs3/xattr.c
index 0dbb410d51df..03006ff6a296 100644
--- a/fs/ntfs3/xattr.c
+++ b/fs/ntfs3/xattr.c
@@ -146,29 +146,32 @@ static int ntfs_read_ea(struct ntfs_inode *ni, struct EA_FULL **ea,
for (off = 0; off < size; off += ea_size) {
const struct EA_FULL *ef = Add2Ptr(ea_p, off);
u32 bytes = size - off;
- size_t need;
-
- /* Check if we can use field ea->size. */
- if (bytes < sizeof(ef->size))
- goto out1;
+ u32 packed_size;
/* Check if we can use fields ef->name_len and ef->elength. */
if (bytes < offsetof(struct EA_FULL, name))
goto out1;
- /* Size needed to hold this record's name and value. */
- need = struct_size(ef, name,
- 1 + ef->name_len + le16_to_cpu(ef->elength));
+ /*
+ * The name and the value have to fit in the entry whether or
+ * not ef->size is set, because ntfs_get_ea() copies elength
+ * bytes out of this buffer and checks them only against the
+ * caller supplied buffer size.
+ */
+ packed_size =
+ ALIGN(struct_size(ef, name,
+ 1 + ef->name_len +
+ le16_to_cpu(ef->elength)),
+ 4);
if (ef->size) {
ea_size = le32_to_cpu(ef->size);
- /* ef->size must fit the list and cover the record. */
- if (ea_size > bytes || ea_size < need)
+ if (ea_size > bytes || packed_size > ea_size)
goto out1;
continue;
}
- ea_size = ALIGN(need, 4);
+ ea_size = packed_size;
if (ea_size > bytes)
goto out1;
}
diff --git a/fs/ocfs2/alloc.c b/fs/ocfs2/alloc.c
index fb7d8dc2129b..c12154014b89 100644
--- a/fs/ocfs2/alloc.c
+++ b/fs/ocfs2/alloc.c
@@ -1796,6 +1796,7 @@ static int __ocfs2_find_path(struct ocfs2_caching_info *ci,
int i, ret = 0;
u32 range;
u64 blkno;
+ u32 prev_depth = OCFS2_MAX_PATH_DEPTH;
struct buffer_head *bh = NULL;
struct ocfs2_extent_block *eb;
struct ocfs2_extent_list *el;
@@ -1803,22 +1804,25 @@ static int __ocfs2_find_path(struct ocfs2_caching_info *ci,
el = root_el;
while (el->l_tree_depth) {
- if (unlikely(le16_to_cpu(el->l_tree_depth) >= OCFS2_MAX_PATH_DEPTH)) {
+ if (unlikely(le16_to_cpu(el->l_tree_depth) >= prev_depth)) {
ocfs2_error(ocfs2_metadata_cache_get_super(ci),
- "Owner %llu has invalid tree depth %u in extent list\n",
+ "Owner %llu has invalid tree depth %u in extent list (max %u)\n",
(unsigned long long)ocfs2_metadata_cache_owner(ci),
- le16_to_cpu(el->l_tree_depth));
+ le16_to_cpu(el->l_tree_depth), prev_depth - 1);
ret = -EROFS;
goto out;
}
- if (le16_to_cpu(el->l_next_free_rec) == 0) {
+ prev_depth = le16_to_cpu(el->l_tree_depth);
+
+ if (!el->l_next_free_rec || !el->l_count) {
ocfs2_error(ocfs2_metadata_cache_get_super(ci),
- "Owner %llu has empty extent list at depth %u\n",
+ "Owner %llu has empty extent list at depth %u\n"
+ "(next free=%u count=%u)\n",
(unsigned long long)ocfs2_metadata_cache_owner(ci),
- le16_to_cpu(el->l_tree_depth));
+ le16_to_cpu(el->l_tree_depth),
+ le16_to_cpu(el->l_next_free_rec), le16_to_cpu(el->l_count));
ret = -EROFS;
goto out;
-
}
for(i = 0; i < le16_to_cpu(el->l_next_free_rec) - 1; i++) {
@@ -6891,7 +6895,7 @@ static void ocfs2_zero_cluster_folios(struct inode *inode, loff_t start,
ocfs2_map_and_dirty_folio(inode, handle, from, to, folio, 1,
&phys);
- start = folio_next_index(folio) << PAGE_SHIFT;
+ start = folio_next_pos(folio);
}
out:
if (folios)
diff --git a/fs/ocfs2/cluster/tcp.c b/fs/ocfs2/cluster/tcp.c
index 32048c65b66b..07b225277941 100644
--- a/fs/ocfs2/cluster/tcp.c
+++ b/fs/ocfs2/cluster/tcp.c
@@ -391,6 +391,9 @@ static void sc_kref_release(struct kref *kref)
sclog(sc, "releasing\n");
+ /* Unlink before freeing anything the debugfs readers dereference. */
+ o2net_debug_del_sc(sc);
+
if (sc->sc_sock) {
sock_release(sc->sc_sock);
sc->sc_sock = NULL;
@@ -400,8 +403,6 @@ static void sc_kref_release(struct kref *kref)
o2nm_node_put(sc->sc_node);
sc->sc_node = NULL;
- o2net_debug_del_sc(sc);
-
if (sc->sc_page)
__free_page(sc->sc_page);
kfree(sc);
diff --git a/fs/ocfs2/dcache.h b/fs/ocfs2/dcache.h
index 7f246c5692d8..046ab9a80769 100644
--- a/fs/ocfs2/dcache.h
+++ b/fs/ocfs2/dcache.h
@@ -17,9 +17,8 @@ struct ocfs2_dentry_lock {
u64 dl_parent_blkno;
/*
- * The ocfs2_dentry_lock keeps an inode reference until
- * dl_lockres has been destroyed. This is usually done in
- * ->d_iput() anyway, so there should be minimal impact.
+ * The inode reference is dropped before dl_lockres is unlocked.
+ * Callbacks must not use it to find the mount context during teardown.
*/
struct inode *dl_inode;
struct ocfs2_lock_res dl_lockres;
diff --git a/fs/ocfs2/dlm/dlmmaster.c b/fs/ocfs2/dlm/dlmmaster.c
index 4ab8c48582b1..ee28a01e7be3 100644
--- a/fs/ocfs2/dlm/dlmmaster.c
+++ b/fs/ocfs2/dlm/dlmmaster.c
@@ -2526,6 +2526,52 @@ static int dlm_is_lockres_migratable(struct dlm_ctxt *dlm,
*/
+/*
+ * Drop the sender's references on a migration MLE after the attempt has
+ * failed. The finishing assert from the target may already have unlinked
+ * the MLE and consumed its base reference; decide under the locks the
+ * assert handler holds, and only drop the base reference while the MLE is
+ * still hashed. Returns true when the assert got there first: the target
+ * has become the master, so the caller must complete the migration.
+ */
+static bool dlm_put_migration_mle(struct dlm_ctxt *dlm,
+ struct dlm_master_list_entry *mle)
+{
+ bool asserted;
+
+ spin_lock(&dlm->spinlock);
+ spin_lock(&dlm->master_lock);
+ __dlm_mle_detach_hb_events(dlm, mle);
+ asserted = hlist_unhashed(&mle->master_hash_node);
+ if (!asserted) {
+ __dlm_unlink_mle(dlm, mle);
+ __dlm_put_mle(mle);
+ }
+ spin_unlock(&dlm->master_lock);
+ spin_unlock(&dlm->spinlock);
+
+ dlm_put_mle_inuse(mle);
+ return asserted;
+}
+
+/*
+ * The finishing assert got there first: it switched the owner to the
+ * target and cleared DLM_LOCK_RES_MIGRATING. Drop the other nodes' locks
+ * as a completed migration does, unless recovery has taken the lockres
+ * over since, and let the usage be recalculated.
+ */
+static void dlm_migrate_lockres_asserted(struct dlm_ctxt *dlm,
+ struct dlm_lock_resource *res,
+ u8 target)
+{
+ spin_lock(&res->spinlock);
+ if (res->owner == target)
+ dlm_remove_nonlocal_locks(dlm, res);
+ spin_unlock(&res->spinlock);
+
+ dlm_lockres_calc_usage(dlm, res);
+}
+
static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
struct dlm_lock_resource *res, u8 target)
{
@@ -2610,9 +2656,10 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
if (ret < 0) {
if (mle_added) {
- dlm_mle_detach_hb_events(dlm, mle);
- dlm_put_mle(mle);
- dlm_put_mle_inuse(mle);
+ if (dlm_put_migration_mle(dlm, mle)) {
+ dlm_migrate_lockres_asserted(dlm, res, target);
+ ret = 0;
+ }
} else if (mle) {
kmem_cache_free(dlm_mle_cache, mle);
mle = NULL;
@@ -2641,9 +2688,11 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
mlog(0, "migration to node %u failed with %d\n",
target, ret);
/* migration failed, detach and clean up mle */
- dlm_mle_detach_hb_events(dlm, mle);
- dlm_put_mle(mle);
- dlm_put_mle_inuse(mle);
+ if (dlm_put_migration_mle(dlm, mle)) {
+ dlm_migrate_lockres_asserted(dlm, res, target);
+ ret = 0;
+ goto leave;
+ }
spin_lock(&res->spinlock);
res->state &= ~DLM_LOCK_RES_MIGRATING;
wake = 1;
@@ -2687,9 +2736,12 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
res->lockname.name, target);
ret = -EINVAL;
/* migration failed, detach and clean up mle */
- dlm_mle_detach_hb_events(dlm, mle);
- dlm_put_mle(mle);
- dlm_put_mle_inuse(mle);
+ if (dlm_put_migration_mle(dlm, mle)) {
+ dlm_migrate_lockres_asserted(dlm, res,
+ target);
+ ret = 0;
+ goto leave;
+ }
spin_lock(&res->spinlock);
res->state &= ~DLM_LOCK_RES_MIGRATING;
wake = 1;
diff --git a/fs/ocfs2/dlmglue.c b/fs/ocfs2/dlmglue.c
index 90d23dd7d4c8..fdd0ed900e7e 100644
--- a/fs/ocfs2/dlmglue.c
+++ b/fs/ocfs2/dlmglue.c
@@ -57,7 +57,6 @@ struct ocfs2_mask_waiter {
#endif
};
-static struct ocfs2_super *ocfs2_get_dentry_osb(struct ocfs2_lock_res *lockres);
static struct ocfs2_super *ocfs2_get_inode_osb(struct ocfs2_lock_res *lockres);
static struct ocfs2_super *ocfs2_get_file_osb(struct ocfs2_lock_res *lockres);
static struct ocfs2_super *ocfs2_get_qinfo_osb(struct ocfs2_lock_res *lockres);
@@ -257,7 +256,6 @@ static const struct ocfs2_lock_res_ops ocfs2_orphan_scan_lops = {
};
static const struct ocfs2_lock_res_ops ocfs2_dentry_lops = {
- .get_osb = ocfs2_get_dentry_osb,
.post_unlock = ocfs2_dentry_post_unlock,
.downconvert_worker = ocfs2_dentry_convert_worker,
.flags = 0,
@@ -308,7 +306,7 @@ static inline struct ocfs2_dentry_lock *ocfs2_lock_res_dl(struct ocfs2_lock_res
{
BUG_ON(lockres->l_type != OCFS2_LOCK_TYPE_DENTRY);
- return (struct ocfs2_dentry_lock *)lockres->l_priv;
+ return container_of(lockres, struct ocfs2_dentry_lock, dl_lockres);
}
static inline struct ocfs2_mem_dqinfo *ocfs2_lock_res_qinfo(struct ocfs2_lock_res *lockres)
@@ -609,13 +607,6 @@ static __u64 ocfs2_get_dentry_lock_ino(struct ocfs2_lock_res *lockres)
return be64_to_cpu(inode_blkno_be);
}
-static struct ocfs2_super *ocfs2_get_dentry_osb(struct ocfs2_lock_res *lockres)
-{
- struct ocfs2_dentry_lock *dl = lockres->l_priv;
-
- return OCFS2_SB(dl->dl_inode->i_sb);
-}
-
void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl,
u64 parent, struct inode *inode)
{
@@ -623,6 +614,7 @@ void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl,
u64 inode_blkno = OCFS2_I(inode)->ip_blkno;
__be64 inode_blkno_be = cpu_to_be64(inode_blkno);
struct ocfs2_lock_res *lockres = &dl->dl_lockres;
+ struct ocfs2_super *osb = OCFS2_SB(inode->i_sb);
ocfs2_lock_res_init_once(lockres);
@@ -651,9 +643,9 @@ void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl,
memcpy(&lockres->l_name[OCFS2_DENTRY_LOCK_INO_START], &inode_blkno_be,
sizeof(__be64));
- ocfs2_lock_res_init_common(OCFS2_SB(inode->i_sb), lockres,
+ ocfs2_lock_res_init_common(osb, lockres,
OCFS2_LOCK_TYPE_DENTRY, &ocfs2_dentry_lops,
- dl);
+ osb);
}
static void ocfs2_super_lock_res_init(struct ocfs2_lock_res *res,
@@ -3335,6 +3327,14 @@ static void ocfs2_dlm_shutdown_debug(struct ocfs2_super *osb)
{
struct ocfs2_dlm_debug *dlm_debug = osb->osb_dlm_debug;
+ /*
+ * Drain the readers before the cluster stack goes away: the
+ * locking_state reader resolves ocfs2_dlm_lvb() through the
+ * stack. Every caller removes the parent directory later.
+ */
+ debugfs_lookup_and_remove("locking_state", osb->osb_debug_root);
+ debugfs_lookup_and_remove("locking_filter", osb->osb_debug_root);
+
if (dlm_debug)
ocfs2_put_dlm_debug(dlm_debug);
}
@@ -3349,8 +3349,6 @@ int ocfs2_dlm_init(struct ocfs2_super *osb)
goto local;
}
- ocfs2_dlm_init_debug(osb);
-
/* launch downconvert thread */
osb->dc_task = kthread_run(ocfs2_downconvert_thread, osb, "ocfs2dc-%s",
osb->uuid_str);
@@ -3390,9 +3388,10 @@ int ocfs2_dlm_init(struct ocfs2_super *osb)
ocfs2_orphan_scan_lock_res_init(&osb->osb_orphan_scan.os_lockres, osb);
osb->cconn = conn;
+ if (conn)
+ ocfs2_dlm_init_debug(osb);
bail:
if (status < 0) {
- ocfs2_dlm_shutdown_debug(osb);
if (osb->dc_task)
kthread_stop(osb->dc_task);
}
@@ -3422,10 +3421,11 @@ void ocfs2_dlm_shutdown(struct ocfs2_super *osb,
ocfs2_lock_res_free(&osb->osb_orphan_scan.os_lockres);
if (osb->cconn) {
+ /* The disconnect below may release the selected stack. */
+ ocfs2_dlm_shutdown_debug(osb);
+
ocfs2_cluster_disconnect(osb->cconn, hangup_pending);
osb->cconn = NULL;
-
- ocfs2_dlm_shutdown_debug(osb);
}
}
diff --git a/fs/ocfs2/filecheck.c b/fs/ocfs2/filecheck.c
index 47659ef73ca2..e676839078f4 100644
--- a/fs/ocfs2/filecheck.c
+++ b/fs/ocfs2/filecheck.c
@@ -189,6 +189,8 @@ int ocfs2_filecheck_create_sysfs(struct ocfs2_super *osb)
/* Mount failure still calls ocfs2_filecheck_remove_sysfs(). */
entry->fs_fcheck = NULL;
kobject_put(&entry->fs_kobj);
+ /* Keep ocfs2_super alive until the kobject is released. */
+ wait_for_completion(&entry->fs_kobj_unregister);
kfree(fcheck);
return ret;
}
diff --git a/fs/ocfs2/journal.c b/fs/ocfs2/journal.c
index 2b8b57e914f0..602b6c8dc39f 100644
--- a/fs/ocfs2/journal.c
+++ b/fs/ocfs2/journal.c
@@ -230,13 +230,22 @@ void ocfs2_recovery_disable_quota(struct ocfs2_super *osb)
ocfs2_recovery_disable(osb, OCFS2_REC_QUOTA_WANT_DISABLE);
}
+/* Stop producers and drain completion work without freeing recovery state. */
+void ocfs2_recovery_stop(struct ocfs2_super *osb)
+{
+ ocfs2_recovery_disable(osb, OCFS2_REC_WANT_DISABLE);
+}
+
void ocfs2_recovery_exit(struct ocfs2_super *osb)
{
struct ocfs2_recovery_map *rm;
- /* disable any new recovery threads and wait for any currently
- * running ones to exit. Do this before setting the vol_state. */
- ocfs2_recovery_disable(osb, OCFS2_REC_WANT_DISABLE);
+ /*
+ * Disable producers and drain work before freeing recovery state.
+ * Do this before setting vol_state to VOLUME_DISMOUNTED: completion
+ * work may still be waiting in __ocfs2_wait_on_mount().
+ */
+ ocfs2_recovery_stop(osb);
/*
* Now that recovery is shut down, and the osb is about to be
diff --git a/fs/ocfs2/journal.h b/fs/ocfs2/journal.h
index 19fc920d26b1..4b52bf7b0938 100644
--- a/fs/ocfs2/journal.h
+++ b/fs/ocfs2/journal.h
@@ -147,6 +147,7 @@ void ocfs2_complete_recovery(struct work_struct *work);
void ocfs2_wait_for_recovery(struct ocfs2_super *osb);
int ocfs2_recovery_init(struct ocfs2_super *osb);
+void ocfs2_recovery_stop(struct ocfs2_super *osb);
void ocfs2_recovery_exit(struct ocfs2_super *osb);
void ocfs2_recovery_disable_quota(struct ocfs2_super *osb);
diff --git a/fs/ocfs2/super.c b/fs/ocfs2/super.c
index fe4ab1659e0d..ed5b303f56bf 100644
--- a/fs/ocfs2/super.c
+++ b/fs/ocfs2/super.c
@@ -1829,6 +1829,9 @@ static void ocfs2_dismount_volume(struct super_block *sb, int mnt_err)
/* Wait for worker to be done with the work structure in osb */
cancel_work_sync(&osb->dquot_drop_work);
+ /* Stop recovery producers before releasing their local allocators. */
+ ocfs2_recovery_stop(osb);
+
ocfs2_shutdown_local_alloc(osb);
ocfs2_truncate_log_shutdown(osb);
diff --git a/fs/ocfs2/xattr.c b/fs/ocfs2/xattr.c
index 734801de8e89..921d050c3758 100644
--- a/fs/ocfs2/xattr.c
+++ b/fs/ocfs2/xattr.c
@@ -256,6 +256,7 @@ static int ocfs2_xattr_tree_list_index_block(struct inode *inode,
size_t buffer_size);
static int ocfs2_xattr_create_index_block(struct inode *inode,
+ struct ocfs2_xattr_info *xi,
struct ocfs2_xattr_search *xs,
struct ocfs2_xattr_set_ctxt *ctxt);
@@ -3013,7 +3014,7 @@ static int ocfs2_xattr_block_set(struct inode *inode,
else if ((ret != -ENOSPC) || ctxt->set_abort)
goto end;
else {
- ret = ocfs2_xattr_create_index_block(inode, xs, ctxt);
+ ret = ocfs2_xattr_create_index_block(inode, xi, xs, ctxt);
if (ret)
goto end;
}
@@ -4295,6 +4296,21 @@ static void ocfs2_cp_xattr_block_to_bucket(struct inode *inode,
for (i = 0; i < blks; i++)
memset(bucket_block(bucket, i), 0, blocksize);
+ /*
+ * An empty block has no names or values to carry over, and the
+ * xh_entries[count - 1] below would be xh_entries[-1]. Leave the
+ * bucket empty, but carry over the hash ocfs2_xa_remove_entry() left
+ * in the block's xh_entries[0] so that the bucket still holds the key
+ * it is looked up by.
+ */
+ if (!count) {
+ xh->xh_num_buckets = cpu_to_le16(1);
+ xh->xh_free_start = cpu_to_le16(OCFS2_XATTR_BUCKET_SIZE);
+ xh->xh_entries[0].xe_name_hash =
+ xb_xh->xh_entries[0].xe_name_hash;
+ return;
+ }
+
/*
* Since the xe_name_offset is based on ocfs2_xattr_header,
* there is a offset change corresponding to the change of
@@ -4341,13 +4357,10 @@ static void ocfs2_cp_xattr_block_to_bucket(struct inode *inode,
* real place of the xattr.
*/
static void ocfs2_xattr_update_xattr_search(struct inode *inode,
- struct ocfs2_xattr_search *xs,
- struct buffer_head *old_bh)
+ struct ocfs2_xattr_info *xi,
+ struct ocfs2_xattr_search *xs)
{
- char *buf = old_bh->b_data;
- struct ocfs2_xattr_block *old_xb = (struct ocfs2_xattr_block *)buf;
- struct ocfs2_xattr_header *old_xh = &old_xb->xb_attrs.xb_header;
- int i;
+ int i, count;
xs->header = bucket_xh(xs->bucket);
xs->base = bucket_block(xs->bucket, 0);
@@ -4356,11 +4369,47 @@ static void ocfs2_xattr_update_xattr_search(struct inode *inode,
if (xs->not_found)
return;
- i = xs->here - old_xh->xh_entries;
- xs->here = &xs->header->xh_entries[i];
+ /*
+ * A block holds its entries in insertion order while a bucket holds
+ * them by name hash, so ocfs2_cp_xattr_block_to_bucket() sorted the
+ * copies and the index this entry had in the block no longer says
+ * where it is. It copied every entry and every name unchanged, and
+ * (type, name) pairs are unique within a block, so find it again by
+ * name.
+ *
+ * Search for the name being set, not for the name xs->here carries.
+ * A failed set may have removed that entry, and ocfs2_xa_remove_entry()
+ * then memmoves its successor into the slot. Comparing every entry
+ * rather than narrowing on the hash also finds an entry stored under
+ * the legacy signed hash, which ocfs2_find_xe_in_bucket() would skip.
+ */
+ count = le16_to_cpu(xs->header->xh_count);
+ for (i = 0; i < count; i++) {
+ struct ocfs2_xattr_entry *xe = &xs->header->xh_entries[i];
+
+ if (ocfs2_xattr_get_type(xe) != xi->xi_name_index)
+ continue;
+ if (xe->xe_name_len != xi->xi_name_len)
+ continue;
+ if (!memcmp(xs->base + le16_to_cpu(xe->xe_name_offset),
+ xi->xi_name, xi->xi_name_len)) {
+ xs->here = xe;
+ return;
+ }
+ }
+
+ /*
+ * The entry did not survive the failed set, so this has become an
+ * insert. Point xs->here back inside the bucket: it still refers to
+ * the block being converted, which is about to be rewritten as a tree
+ * root.
+ */
+ xs->not_found = -ENODATA;
+ xs->here = xs->header->xh_entries;
}
static int ocfs2_xattr_create_index_block(struct inode *inode,
+ struct ocfs2_xattr_info *xi,
struct ocfs2_xattr_search *xs,
struct ocfs2_xattr_set_ctxt *ctxt)
{
@@ -4427,7 +4476,7 @@ static int ocfs2_xattr_create_index_block(struct inode *inode,
ocfs2_cp_xattr_block_to_bucket(inode, xb_bh, xs->bucket);
ocfs2_xattr_bucket_journal_dirty(handle, xs->bucket);
- ocfs2_xattr_update_xattr_search(inode, xs, xb_bh);
+ ocfs2_xattr_update_xattr_search(inode, xi, xs);
/* Change from ocfs2_xattr_header to ocfs2_xattr_tree_root */
memset(&xb->xb_attrs, 0, inode->i_sb->s_blocksize -
@@ -4780,7 +4829,15 @@ static int ocfs2_divide_xattr_bucket(struct inode *inode,
start = ocfs2_xattr_find_divide_pos(xh);
if (start == count) {
- xe = &xh->xh_entries[start-1];
+ /*
+ * start == count means every entry shares one hash, so the
+ * first of them carries it too. An empty bucket has no
+ * entries[start - 1], but its entries[0] still carries the
+ * bucket key: either the hash ocfs2_xa_remove_entry()
+ * deliberately leaves behind, or the sentinel a previous
+ * divide wrote.
+ */
+ xe = &xh->xh_entries[0];
/*
* initialized a new empty bucket here.
@@ -5610,11 +5667,20 @@ static int ocfs2_check_xattr_bucket_collision(struct ocfs2_xattr_bucket *bucket,
u32 name_hash)
{
struct ocfs2_xattr_header *xh = bucket_xh(bucket);
+ u16 count = le16_to_cpu(xh->xh_count);
+
+ /*
+ * An empty bucket keeps the hash of the entry that was removed in
+ * xh_entries[0] so that it can still be found, but it holds no
+ * entries to collide with.
+ */
+ if (!count)
+ return 0;
if (name_hash != le32_to_cpu(xh->xh_entries[0].xe_name_hash))
return 0;
- if (xh->xh_entries[le16_to_cpu(xh->xh_count) - 1].xe_name_hash ==
+ if (xh->xh_entries[count - 1].xe_name_hash ==
xh->xh_entries[0].xe_name_hash) {
mlog(ML_ERROR, "Too much hash collision in xattr bucket %llu, "
"hash = %u\n",
@@ -5635,13 +5701,18 @@ static int ocfs2_xattr_set_entry_bucket(struct inode *inode,
struct ocfs2_xattr_search *xs,
struct ocfs2_xattr_set_ctxt *ctxt)
{
- int ret;
+ int ret, found = 0;
+ u16 xe_index;
+ u32 name_hash;
struct ocfs2_xa_loc loc;
trace_ocfs2_xattr_set_entry_bucket(xi->xi_name);
ocfs2_init_xattr_bucket_xa_loc(&loc, xs->bucket,
xs->not_found ? NULL : xs->here);
+ /* Defragmenting below may move this entry, so remember its hash. */
+ name_hash = loc.xl_entry ? le32_to_cpu(loc.xl_entry->xe_name_hash) : 0;
+
ret = ocfs2_xa_set(&loc, xi, ctxt);
if (!ret) {
xs->here = loc.xl_entry;
@@ -5660,6 +5731,29 @@ static int ocfs2_xattr_set_entry_bucket(struct inode *inode,
goto out;
}
+ /*
+ * ocfs2_defrag_xattr_bucket() sorts the entries by name hash and
+ * lib/sort.c is not stable, so entries sharing a hash may have traded
+ * slots. Look the entry up again before writing through it.
+ */
+ if (loc.xl_entry) {
+ ret = ocfs2_find_xe_in_bucket(inode, xs->bucket,
+ xi->xi_name_index, xi->xi_name,
+ name_hash, &xe_index, &found);
+ if (ret) {
+ mlog_errno(ret);
+ goto out;
+ }
+ if (!found) {
+ mlog(ML_ERROR, "xattr entry not found in bucket %llu after defrag\n",
+ (unsigned long long)bucket_blkno(xs->bucket));
+ ret = -EFSCORRUPTED;
+ goto out;
+ }
+ loc.xl_entry = &bucket_xh(xs->bucket)->xh_entries[xe_index];
+ xs->here = loc.xl_entry;
+ }
+
ret = ocfs2_xa_set(&loc, xi, ctxt);
if (!ret) {
xs->here = loc.xl_entry;
diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
index 83f80fdb1567..d961533c9c9b 100644
--- a/fs/overlayfs/export.c
+++ b/fs/overlayfs/export.c
@@ -718,9 +718,9 @@ static struct dentry *ovl_lower_fh_to_d(struct super_block *sb,
}
}
- /* Then lookup indexed upper/whiteout by origin fh */
+ /* Then lookup indexed upper/whiteout by origin */
if (ovl_indexdir(sb)) {
- index = ovl_get_index_fh(ofs, fh);
+ index = ovl_get_index_origin(ofs, origin.dentry);
err = PTR_ERR(index);
if (IS_ERR(index)) {
index = NULL;
diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 78803e40c961..79ad351298fe 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -657,6 +657,14 @@ int ovl_fileattr_set(struct mnt_idmap *idmap,
unsigned int flags;
int err;
+ /*
+ * Lookup refuses a directory that does not agree with the casefold
+ * state of the mount. Only FS_IOC_SETFLAGS can ask for a change.
+ */
+ if (S_ISDIR(inode->i_mode) && fa->flags_valid &&
+ !!(fa->flags & FS_CASEFOLD_FL) != OVL_FS(inode->i_sb)->casefold)
+ return -EOPNOTSUPP;
+
err = ovl_copy_up(dentry);
if (!err) {
ovl_path_real(dentry, &upperpath);
diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index c20be56c9b9d..3d03e0312561 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -775,14 +775,21 @@ int ovl_get_index_name(struct ovl_fs *ofs, struct dentry *origin,
return err;
}
-/* Lookup index by file handle for NFS export */
-struct dentry *ovl_get_index_fh(struct ovl_fs *ofs, struct ovl_fh *fh)
+/*
+ * Lookup index by decoded origin for NFS export.
+ *
+ * The name of an index is the hex of the file handle that copy up encoded.
+ * The bytes of the handle to decode cannot be used as the name, because a
+ * lower fs may decode many handles to the same inode. For example, it may
+ * ignore the fid type, extra fid bytes or a zero generation.
+ */
+struct dentry *ovl_get_index_origin(struct ovl_fs *ofs, struct dentry *origin)
{
struct dentry *index;
struct qstr name;
int err;
- err = ovl_get_index_name_fh(fh, &name);
+ err = ovl_get_index_name(ofs, origin, &name);
if (err)
return ERR_PTR(err);
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 22555205c84c..6268db46483e 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -736,7 +736,7 @@ int ovl_verify_index(struct ovl_fs *ofs, struct dentry *index);
int ovl_get_index_name_fh(const struct ovl_fh *fh, struct qstr *name);
int ovl_get_index_name(struct ovl_fs *ofs, struct dentry *origin,
struct qstr *name);
-struct dentry *ovl_get_index_fh(struct ovl_fs *ofs, struct ovl_fh *fh);
+struct dentry *ovl_get_index_origin(struct ovl_fs *ofs, struct dentry *origin);
struct dentry *ovl_lookup_index(struct ovl_fs *ofs, struct dentry *upper,
struct dentry *origin, bool verify);
int ovl_path_next(int idx, struct dentry *dentry, struct path *path,
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 8d4ef5b2f178..52d6f32a5a7c 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -1370,7 +1370,6 @@ static struct dentry *ovl_get_root(struct super_block *sb,
ovl_dentry_set_flag(OVL_E_CONNECTED, root);
ovl_set_upperdata(d_inode(root));
ovl_inode_init(d_inode(root), &oip, ino, fsid);
- WARN_ON(!!IS_CASEFOLDED(d_inode(root)) != ofs->casefold);
ovl_dentry_init_flags(root, upperdentry, oe, DCACHE_OP_WEAK_REVALIDATE);
/* root keeps a reference of upperdentry */
dget(upperdentry);
@@ -1576,6 +1575,17 @@ int ovl_fill_super(struct super_block *sb, struct fs_context *fc)
if (!root_dentry)
goto out_free_oe;
+ /*
+ * The casefold flag of an empty layer root may have changed since the
+ * mount options were parsed. The root inode owns oe by now.
+ */
+ if (!!IS_CASEFOLDED(d_inode(root_dentry)) != ofs->casefold) {
+ pr_err("casefold flag of the root directory has changed\n");
+ dput(root_dentry);
+ err = -EINVAL;
+ goto out_err;
+ }
+
sb->s_root = root_dentry;
ovl_revert_creds(old_cred);
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 0f08913cfa52..823e8ee1b095 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -3179,7 +3179,7 @@ static long do_pagemap_cmd(struct file *file, unsigned int cmd,
return do_pagemap_scan(mm, arg);
default:
- return -EINVAL;
+ return -ENOIOCTLCMD;
}
}
diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
index 8d751026dd4a..2d5a4a26c931 100644
--- a/fs/smb/client/cifsglob.h
+++ b/fs/smb/client/cifsglob.h
@@ -16,6 +16,7 @@
#include <linux/scatterlist.h>
#include <linux/mm.h>
#include <linux/mempool.h>
+#include <linux/refcount.h>
#include <linux/workqueue.h>
#include <linux/utsname.h>
#include <linux/sched/mm.h>
@@ -27,6 +28,7 @@
#include <uapi/linux/cifs/cifs_mount.h>
#include "../common/cifsglob.h"
#include "../common/smb2pdu.h"
+#include "../common/fscc.h"
#include "smb2pdu.h"
#include <linux/filelock.h>
@@ -678,6 +680,7 @@ struct TCP_Server_Info {
spinlock_t srv_lock; /* protect anything here that is not protected */
__u64 conn_id; /* connection identifier (useful for debugging) */
int srv_count; /* reference counter */
+ refcount_t obj_refs; /* struct lifetime: srv_count users + cifsd */
int rfc1001_sessinit; /* whether to estasblish netbios session */
bool with_rfc1001; /* if netbios session is used */
/* 15 character server name + 0x20 16th byte indicating type = srv */
diff --git a/fs/smb/client/cifspdu.h b/fs/smb/client/cifspdu.h
index c86a329e5822..c5c890cf3605 100644
--- a/fs/smb/client/cifspdu.h
+++ b/fs/smb/client/cifspdu.h
@@ -12,6 +12,7 @@
#include <net/sock.h>
#include <linux/unaligned.h>
#include "../common/smbfsctl.h"
+#include "../common/smb2pdu.h"
#define CIFS_PROT 0
#define POSIX_PROT (CIFS_PROT+1)
@@ -180,98 +181,7 @@
#define SMBFLG2_ERR_STATUS cpu_to_le16(0x4000)
#define SMBFLG2_UNICODE cpu_to_le16(0x8000)
-/*
- * These are the file access permission bits defined in CIFS for the
- * NTCreateAndX as well as the level 0x107
- * TRANS2_QUERY_PATH_INFORMATION API. The level 0x107, SMB_QUERY_FILE_ALL_INFO
- * responds with the AccessFlags.
- * The AccessFlags specifies the access permissions a caller has to the
- * file and can have any suitable combination of the following values:
- */
-
-#define FILE_READ_DATA 0x00000001 /* Data can be read from the file */
- /* or directory child entries can */
- /* be listed together with the */
- /* associated child attributes */
- /* (so the FILE_READ_ATTRIBUTES on */
- /* the child entry is not needed) */
-#define FILE_WRITE_DATA 0x00000002 /* Data can be written to the file */
- /* or new file can be created in */
- /* the directory */
-#define FILE_APPEND_DATA 0x00000004 /* Data can be appended to the file */
- /* (for non-local files over SMB it */
- /* is same as FILE_WRITE_DATA) */
- /* or new subdirectory can be */
- /* created in the directory */
-#define FILE_READ_EA 0x00000008 /* Extended attributes associated */
- /* with the file can be read */
-#define FILE_WRITE_EA 0x00000010 /* Extended attributes associated */
- /* with the file can be written */
-#define FILE_EXECUTE 0x00000020 /*Data can be read into memory from */
- /* the file using system paging I/O */
- /* for executing the file / script */
- /* or right to traverse directory */
- /* (but by default all users have */
- /* directory bypass traverse */
- /* privilege and do not need this */
- /* permission on directories at all)*/
-#define FILE_DELETE_CHILD 0x00000040 /* Child entry can be deleted from */
- /* the directory (so the DELETE on */
- /* the child entry is not needed) */
-#define FILE_READ_ATTRIBUTES 0x00000080 /* Attributes associated with the */
- /* file or directory can be read */
-#define FILE_WRITE_ATTRIBUTES 0x00000100 /* Attributes associated with the */
- /* file or directory can be written */
-#define DELETE 0x00010000 /* The file or dir can be deleted */
-#define READ_CONTROL 0x00020000 /* The discretionary access control */
- /* list and ownership associated */
- /* with the file or dir can be read */
-#define WRITE_DAC 0x00040000 /* The discretionary access control */
- /* list associated with the file or */
- /* directory can be written */
-#define WRITE_OWNER 0x00080000 /* Ownership information associated */
- /* with the file/dir can be written */
-#define SYNCHRONIZE 0x00100000 /* The file handle can waited on to */
- /* synchronize with the completion */
- /* of an input/output request */
-#define SYSTEM_SECURITY 0x01000000 /* The system access control list */
- /* associated with the file or */
- /* directory can be read or written */
- /* (cannot be in DACL, can in SACL) */
-#define MAXIMUM_ALLOWED 0x02000000 /* Maximal subset of GENERIC_ALL */
- /* permissions which can be granted */
- /* (cannot be in DACL nor SACL) */
-#define GENERIC_ALL 0x10000000 /* Same as: GENERIC_EXECUTE | */
- /* GENERIC_WRITE | */
- /* GENERIC_READ | */
- /* FILE_DELETE_CHILD | */
- /* DELETE | */
- /* WRITE_DAC | */
- /* WRITE_OWNER */
- /* So GENERIC_ALL contains all bits */
- /* mentioned above except these two */
- /* SYSTEM_SECURITY MAXIMUM_ALLOWED */
-#define GENERIC_EXECUTE 0x20000000 /* Same as: FILE_EXECUTE | */
- /* FILE_READ_ATTRIBUTES | */
- /* READ_CONTROL | */
- /* SYNCHRONIZE */
-#define GENERIC_WRITE 0x40000000 /* Same as: FILE_WRITE_DATA | */
- /* FILE_APPEND_DATA | */
- /* FILE_WRITE_EA | */
- /* FILE_WRITE_ATTRIBUTES | */
- /* READ_CONTROL | */
- /* SYNCHRONIZE */
-#define GENERIC_READ 0x80000000 /* Same as: FILE_READ_DATA | */
- /* FILE_READ_EA | */
- /* FILE_READ_ATTRIBUTES | */
- /* READ_CONTROL | */
- /* SYNCHRONIZE */
-
-#define FILE_READ_RIGHTS (FILE_READ_DATA | FILE_READ_EA | FILE_READ_ATTRIBUTES)
-#define FILE_WRITE_RIGHTS (FILE_WRITE_DATA | FILE_APPEND_DATA \
- | FILE_WRITE_EA | FILE_WRITE_ATTRIBUTES)
-#define FILE_EXEC_RIGHTS (FILE_EXECUTE)
-
+/* Combinations of file access permission bits */
#define SET_FILE_READ_RIGHTS (FILE_READ_DATA | FILE_READ_EA | FILE_WRITE_EA \
| FILE_READ_ATTRIBUTES \
| FILE_WRITE_ATTRIBUTES \
@@ -283,15 +193,6 @@
| FILE_WRITE_ATTRIBUTES \
| DELETE | READ_CONTROL | WRITE_DAC \
| WRITE_OWNER | SYNCHRONIZE)
-#define SET_FILE_EXEC_RIGHTS (FILE_READ_EA | FILE_WRITE_EA | FILE_EXECUTE \
- | FILE_READ_ATTRIBUTES \
- | FILE_WRITE_ATTRIBUTES \
- | DELETE | READ_CONTROL | WRITE_DAC \
- | WRITE_OWNER | SYNCHRONIZE)
-
-#define SET_MINIMUM_RIGHTS (FILE_READ_EA | FILE_READ_ATTRIBUTES \
- | READ_CONTROL | SYNCHRONIZE)
-
/*
* Invalid readdir handle
@@ -417,38 +318,6 @@
#define GETU16(var) (*((__u16 *)var)) /* BB check for endian issues */
#define GETU32(var) (*((__u32 *)var)) /* BB check for endian issues */
-struct smb_hdr {
- __be32 smb_buf_length; /* BB length is only two (rarely three) bytes,
- with one or two byte "type" preceding it that will be
- zero - we could mask the type byte off */
- __u8 Protocol[4];
- __u8 Command;
- union {
- struct {
- __u8 ErrorClass;
- __u8 Reserved;
- __le16 Error;
- } __attribute__((packed)) DosError;
- __le32 CifsError;
- } __attribute__((packed)) Status;
- __u8 Flags;
- __le16 Flags2; /* note: le */
- __le16 PidHigh;
- union {
- struct {
- __le32 SequenceNumber; /* le */
- __u32 Reserved; /* zero */
- } __attribute__((packed)) Sequence;
- __u8 SecuritySignature[8]; /* le */
- } __attribute__((packed)) Signature;
- __u8 pad[2];
- __u16 Tid;
- __le16 Pid;
- __u16 Uid;
- __le16 Mid;
- __u8 WordCount;
-} __attribute__((packed));
-
/* given a pointer to an smb_hdr, retrieve a void pointer to the ByteCount */
static inline void *
BCC(struct smb_hdr *smb)
@@ -2167,28 +2036,6 @@ typedef struct {
#define CIFS_POSIX_EXTENSIONS 0x00000010 /* support for new QFSInfo */
-typedef struct {
- /* For undefined recommended transfer size return -1 in that field */
- __le32 OptimalTransferSize; /* bsize on some os, iosize on other os */
- __le32 BlockSize;
- /* The next three fields are in terms of the block size.
- (above). If block size is unknown, 4096 would be a
- reasonable block size for a server to report.
- Note that returning the blocks/blocksavail removes need
- to make a second call (to QFSInfo level 0x103 to get this info.
- UserBlockAvail is typically less than or equal to BlocksAvail,
- if no distinction is made return the same value in each */
- __le64 TotalBlocks;
- __le64 BlocksAvail; /* bfree */
- __le64 UserBlocksAvail; /* bavail */
- /* For undefined Node fields or FSID return -1 */
- __le64 TotalFileNodes;
- __le64 FreeFileNodes;
- __le64 FileSysIdentifier; /* fsid */
- /* NB Namelen comes from FILE_SYSTEM_ATTRIBUTE_INFO call */
- /* NB flags can come from FILE_SYSTEM_DEVICE_INFO call */
-} __attribute__((packed)) FILE_SYSTEM_POSIX_INFO;
-
/* DeviceType Flags */
#define FILE_DEVICE_CD_ROM 0x00000002
#define FILE_DEVICE_CD_ROM_FILE_SYSTEM 0x00000003
@@ -2223,41 +2070,9 @@ typedef struct {
#define FILE_PORTABLE_DEVICE 0x00004000
#define FILE_DEVICE_ALLOW_APPCONTAINER_TRAVERSAL 0x00020000
-typedef struct {
- __le32 DeviceType;
- __le32 DeviceCharacteristics;
-} __attribute__((packed)) FILE_SYSTEM_DEVICE_INFO; /* device info level 0x104 */
-
/* minimum includes first three fields, and empty FS Name */
#define MIN_FS_ATTR_INFO_SIZE 12
-
-/* List of FileSystemAttributes - see 2.5.1 of MS-FSCC */
-#define FILE_SUPPORTS_SPARSE_VDL 0x10000000 /* faster nonsparse extend */
-#define FILE_SUPPORTS_BLOCK_REFCOUNTING 0x08000000 /* allow ioctl dup extents */
-#define FILE_SUPPORT_INTEGRITY_STREAMS 0x04000000
-#define FILE_SUPPORTS_USN_JOURNAL 0x02000000
-#define FILE_SUPPORTS_OPEN_BY_FILE_ID 0x01000000
-#define FILE_SUPPORTS_EXTENDED_ATTRIBUTES 0x00800000
-#define FILE_SUPPORTS_HARD_LINKS 0x00400000
-#define FILE_SUPPORTS_TRANSACTIONS 0x00200000
-#define FILE_SEQUENTIAL_WRITE_ONCE 0x00100000
-#define FILE_READ_ONLY_VOLUME 0x00080000
-#define FILE_NAMED_STREAMS 0x00040000
-#define FILE_SUPPORTS_ENCRYPTION 0x00020000
-#define FILE_SUPPORTS_OBJECT_IDS 0x00010000
-#define FILE_VOLUME_IS_COMPRESSED 0x00008000
-#define FILE_SUPPORTS_POSIX_UNLINK_RENAME 0x00000400
-#define FILE_RETURNS_CLEANUP_RESULT_INFO 0x00000200
-#define FILE_SUPPORTS_REMOTE_STORAGE 0x00000100
-#define FILE_SUPPORTS_REPARSE_POINTS 0x00000080
-#define FILE_SUPPORTS_SPARSE_FILES 0x00000040
-#define FILE_VOLUME_QUOTAS 0x00000020
-#define FILE_FILE_COMPRESSION 0x00000010
-#define FILE_PERSISTENT_ACLS 0x00000008
-#define FILE_UNICODE_ON_DISK 0x00000004
-#define FILE_CASE_PRESERVED_NAMES 0x00000002
-#define FILE_CASE_SENSITIVE_SEARCH 0x00000001
typedef struct {
__le32 Attributes;
__le32 MaxPathNameComponentLength;
@@ -2490,70 +2305,6 @@ typedef struct {
};
} __attribute__((packed)) FILE_UNIX_INFO; /* level 0x202 */
-typedef struct {
- __le32 NextEntryOffset;
- __u32 FileIndex;
- __le64 CreationTime;
- __le64 LastAccessTime;
- __le64 LastWriteTime;
- __le64 ChangeTime;
- __le64 EndOfFile;
- __le64 AllocationSize;
- __le32 ExtFileAttributes;
- __le32 FileNameLength;
- char FileName[];
-} __attribute__((packed)) FILE_DIRECTORY_INFO; /* level 0x101 FF resp data */
-
-typedef struct {
- __le32 NextEntryOffset;
- __u32 FileIndex;
- __le64 CreationTime;
- __le64 LastAccessTime;
- __le64 LastWriteTime;
- __le64 ChangeTime;
- __le64 EndOfFile;
- __le64 AllocationSize;
- __le32 ExtFileAttributes;
- __le32 FileNameLength;
- __le32 EaSize; /* length of the xattrs */
- char FileName[];
-} __attribute__((packed)) FILE_FULL_DIRECTORY_INFO; /* level 0x102 rsp data */
-
-typedef struct {
- __le32 NextEntryOffset;
- __u32 FileIndex;
- __le64 CreationTime;
- __le64 LastAccessTime;
- __le64 LastWriteTime;
- __le64 ChangeTime;
- __le64 EndOfFile;
- __le64 AllocationSize;
- __le32 ExtFileAttributes;
- __le32 FileNameLength;
- __le32 EaSize; /* EA size */
- __le32 Reserved;
- __le64 UniqueId; /* inode num - le since Samba puts ino in low 32 bit*/
- char FileName[];
-} __attribute__((packed)) SEARCH_ID_FULL_DIR_INFO; /* level 0x105 FF rsp data */
-
-typedef struct {
- __le32 NextEntryOffset;
- __u32 FileIndex;
- __le64 CreationTime;
- __le64 LastAccessTime;
- __le64 LastWriteTime;
- __le64 ChangeTime;
- __le64 EndOfFile;
- __le64 AllocationSize;
- __le32 ExtFileAttributes;
- __le32 FileNameLength;
- __le32 EaSize; /* length of the xattrs */
- __u8 ShortNameLength;
- __u8 Reserved;
- __u8 ShortName[24];
- char FileName[];
-} __attribute__((packed)) FILE_BOTH_DIRECTORY_INFO; /* level 0x104 FFrsp data */
-
typedef struct {
__u32 ResumeKey;
__le16 CreationDate; /* SMB Date */
diff --git a/fs/smb/client/cifsproto.h b/fs/smb/client/cifsproto.h
index 325408073888..d47ad5d86b4b 100644
--- a/fs/smb/client/cifsproto.h
+++ b/fs/smb/client/cifsproto.h
@@ -161,6 +161,7 @@ cifs_mark_tcp_ses_conns_for_reconnect(struct TCP_Server_Info *server,
bool mark_smb_session);
extern int cifs_reconnect(struct TCP_Server_Info *server,
bool mark_smb_session);
+void smb2_add_credits_from_hdr(char *buffer, struct TCP_Server_Info *server);
extern int checkSMB(char *buf, unsigned int len, struct TCP_Server_Info *srvr);
extern bool is_valid_oplock_break(char *, struct TCP_Server_Info *);
extern bool backup_cred(struct cifs_sb_info *);
diff --git a/fs/smb/client/cifssmb.c b/fs/smb/client/cifssmb.c
index 6e4b845a4078..320274e7f705 100644
--- a/fs/smb/client/cifssmb.c
+++ b/fs/smb/client/cifssmb.c
@@ -2338,6 +2338,7 @@ CIFSSMBPosixLock(const unsigned int xid, struct cifs_tcon *tcon,
cifs_dbg(FYI, "Send error in Posix Lock = %d\n", rc);
} else if (pLockData) {
/* lock structure can be returned on get */
+ unsigned int rsp_len;
__u16 data_offset;
__u16 data_count;
rc = validate_t2((struct smb_t2_rsp *)pSMBr);
@@ -2348,10 +2349,17 @@ CIFSSMBPosixLock(const unsigned int xid, struct cifs_tcon *tcon,
}
data_offset = le16_to_cpu(pSMBr->t2.DataOffset);
data_count = le16_to_cpu(pSMBr->t2.DataCount);
+ rsp_len = smbCalcSize(&pSMBr->hdr);
if (data_count < sizeof(struct cifs_posix_lock)) {
rc = -EIO;
goto plk_err_exit;
}
+ if (data_offset > rsp_len ||
+ data_count > rsp_len - data_offset) {
+ rc = smb_EIO2(smb_eio_trace_lock_data_area,
+ data_offset, data_count);
+ goto plk_err_exit;
+ }
parm_data = (struct cifs_posix_lock *)
((char *)&pSMBr->hdr.Protocol + data_offset);
if (parm_data->lock_type == cpu_to_le16(CIFS_UNLCK))
@@ -4317,8 +4325,9 @@ CIFSFindFirst(const unsigned int xid, struct cifs_tcon *tcon,
/* level 257 SMB_ */
TRANSACTION2_FFIRST_REQ *pSMB = NULL;
TRANSACTION2_FFIRST_RSP *pSMBr = NULL;
- T2_FFIRST_RSP_PARMS *parms;
+ unsigned int data_off, data_count;
struct nls_table *nls_codepage;
+ T2_FFIRST_RSP_PARMS *parms;
unsigned int lnoff;
__u16 params, byte_count;
int bytes_returned = 0;
@@ -4437,11 +4446,19 @@ CIFSFindFirst(const unsigned int xid, struct cifs_tcon *tcon,
return rc;
}
+ data_off = le16_to_cpu(pSMBr->t2.DataOffset);
+ data_count = le16_to_cpu(pSMBr->t2.DataCount);
+ if (data_off + data_count > (unsigned int)bytes_returned) {
+ cifs_dbg(VFS, "%s: data area offset %u count %u past response length %d\n",
+ __func__, data_off, data_count, bytes_returned);
+ cifs_buf_release(pSMB);
+ return -EINVAL;
+ }
+
psrch_inf->unicode = !!(pSMBr->hdr.Flags2 & SMBFLG2_UNICODE);
psrch_inf->ntwrk_buf_start = (char *)pSMBr;
psrch_inf->smallBuf = false;
- psrch_inf->srch_entries_start = (char *)&pSMBr->hdr.Protocol +
- le16_to_cpu(pSMBr->t2.DataOffset);
+ psrch_inf->srch_entries_start = (char *)&pSMBr->hdr.Protocol + data_off;
parms = (T2_FFIRST_RSP_PARMS *)((char *)&pSMBr->hdr.Protocol +
le16_to_cpu(pSMBr->t2.ParameterOffset));
@@ -4474,8 +4491,9 @@ int CIFSFindNext(const unsigned int xid, struct cifs_tcon *tcon,
{
TRANSACTION2_FNEXT_REQ *pSMB = NULL;
TRANSACTION2_FNEXT_RSP *pSMBr = NULL;
- T2_FNEXT_RSP_PARMS *parms;
+ unsigned int data_off, data_count;
unsigned int name_len;
+ T2_FNEXT_RSP_PARMS *parms;
unsigned int lnoff;
__u16 params, byte_count;
char *response_data;
@@ -4558,13 +4576,22 @@ int CIFSFindNext(const unsigned int xid, struct cifs_tcon *tcon,
cifs_buf_release(pSMB);
return rc;
}
+
+ data_off = le16_to_cpu(pSMBr->t2.DataOffset);
+ data_count = le16_to_cpu(pSMBr->t2.DataCount);
+ if (data_off + data_count > (unsigned int)bytes_returned) {
+ cifs_dbg(VFS, "%s: data area offset %u count %u past response length %d\n",
+ __func__, data_off, data_count, bytes_returned);
+ cifs_buf_release(pSMB);
+ return -EINVAL;
+ }
+
/* BB fixme add lock for file (srch_info) struct here */
psrch_inf->unicode = !!(pSMBr->hdr.Flags2 & SMBFLG2_UNICODE);
response_data = (char *)&pSMBr->hdr.Protocol +
le16_to_cpu(pSMBr->t2.ParameterOffset);
parms = (T2_FNEXT_RSP_PARMS *)response_data;
- response_data = (char *)&pSMBr->hdr.Protocol +
- le16_to_cpu(pSMBr->t2.DataOffset);
+ response_data = (char *)&pSMBr->hdr.Protocol + data_off;
if (psrch_inf->smallBuf)
cifs_small_buf_release(psrch_inf->ntwrk_buf_start);
@@ -6082,9 +6109,11 @@ CIFSSMBQAllEAs(const unsigned int xid, struct cifs_tcon *tcon,
int rc = 0;
int bytes_returned;
int list_len;
+ u32 ea_list_len;
struct fealist *ea_response_data;
struct fea *temp_fea;
char *temp_ptr;
+ char *byte_area;
char *end_of_smb;
__u16 params, byte_count, data_offset;
unsigned int ea_name_len = ea_name ? strlen(ea_name) : 0;
@@ -6161,9 +6190,19 @@ CIFSSMBQAllEAs(const unsigned int xid, struct cifs_tcon *tcon,
ea_response_data = (struct fealist *)
(((char *) &pSMBr->hdr.Protocol) + data_offset);
- list_len = le32_to_cpu(ea_response_data->list_len);
- cifs_dbg(FYI, "ea length %d\n", list_len);
- if (list_len <= 8) {
+ byte_area = pByteArea(&pSMBr->hdr);
+ end_of_smb = byte_area + get_bcc(&pSMBr->hdr);
+ if ((char *)ea_response_data < byte_area ||
+ (char *)ea_response_data >
+ end_of_smb - sizeof(ea_response_data->list_len)) {
+ rc = smb_EIO2(smb_eio_trace_qalleas_overlong,
+ data_offset, get_bcc(&pSMBr->hdr));
+ goto QAllEAsOut;
+ }
+
+ ea_list_len = le32_to_cpu(ea_response_data->list_len);
+ cifs_dbg(FYI, "%s: ea length: %u\n", __func__, ea_list_len);
+ if (ea_list_len <= 8) {
cifs_dbg(FYI, "empty EA list returned from server\n");
/* didn't find the named attribute */
if (ea_name)
@@ -6172,12 +6211,14 @@ CIFSSMBQAllEAs(const unsigned int xid, struct cifs_tcon *tcon,
}
/* make sure list_len doesn't go past end of SMB */
- end_of_smb = (char *)pByteArea(&pSMBr->hdr) + get_bcc(&pSMBr->hdr);
- if ((char *)ea_response_data + list_len > end_of_smb) {
+ if (ea_list_len > end_of_smb - (char *)ea_response_data) {
cifs_dbg(FYI, "EA list appears to go beyond SMB\n");
- rc = -EIO;
+ rc = smb_EIO2(smb_eio_trace_qalleas_overlong,
+ ea_list_len,
+ end_of_smb - (char *)ea_response_data);
goto QAllEAsOut;
}
+ list_len = ea_list_len;
/* account for ea list len */
list_len -= 4;
diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index 2cbbbf0c93b8..b4e139471e86 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -1059,6 +1059,19 @@ cifs_enable_signing(struct TCP_Server_Info *server, bool mnt_sign_required)
return 0;
}
+static void cifs_tcp_server_put(struct TCP_Server_Info *server)
+{
+ if (!refcount_dec_and_test(&server->obj_refs))
+ return;
+
+ cifs_crypto_secmech_release(server);
+ kfree_sensitive(server->session_key.response);
+ put_net(cifs_net_ns(server));
+ kfree(server->leaf_fullpath);
+ kfree(server->hostname);
+ kfree_sensitive(server);
+}
+
static noinline_for_stack void
clean_demultiplex_info(struct TCP_Server_Info *server)
{
@@ -1144,14 +1157,11 @@ clean_demultiplex_info(struct TCP_Server_Info *server)
*/
}
- put_net(cifs_net_ns(server));
- kfree(server->leaf_fullpath);
- kfree(server->hostname);
- kfree(server);
-
length = atomic_dec_return(&tcpSesAllocCount);
if (length > 0)
mempool_resize(cifs_req_poolp, length + cifs_min_rcv);
+
+ cifs_tcp_server_put(server);
}
static int
@@ -1225,8 +1235,7 @@ cifs_handle_standard(struct TCP_Server_Info *server, struct mid_q_entry *mid)
return 0;
}
-static void
-smb2_add_credits_from_hdr(char *buffer, struct TCP_Server_Info *server)
+void smb2_add_credits_from_hdr(char *buffer, struct TCP_Server_Info *server)
{
struct smb2_hdr *shdr = (struct smb2_hdr *)buffer;
int scredits, in_flight;
@@ -1730,7 +1739,7 @@ cifs_put_tcp_session(struct TCP_Server_Info *server, int from_reconnect)
else
cancel_delayed_work_sync(&server->reconnect);
- /* For secondary channels, we pick up ref-count on the primary server */
+ /* For secondary channels, drop the reference on the primary server */
if (SERVER_IS_CHAN(server))
cifs_put_tcp_session(server->primary_server, from_reconnect);
@@ -1738,15 +1747,11 @@ cifs_put_tcp_session(struct TCP_Server_Info *server, int from_reconnect)
server->tcpStatus = CifsExiting;
spin_unlock(&server->srv_lock);
- cifs_crypto_secmech_release(server);
-
- kfree_sensitive(server->session_key.response);
- server->session_key.response = NULL;
- server->session_key.len = 0;
-
task = xchg(&server->tsk, NULL);
if (task)
send_sig(SIGKILL, task, 1);
+
+ cifs_tcp_server_put(server);
}
struct TCP_Server_Info *
@@ -1768,6 +1773,7 @@ cifs_get_tcp_session(struct smb3_fs_context *ctx,
rc = -ENOMEM;
goto out_err;
}
+ refcount_set(&tcp_ses->obj_refs, 1);
tcp_ses->hostname = kstrdup(ctx->server_hostname, GFP_KERNEL);
if (!tcp_ses->hostname) {
@@ -1906,6 +1912,8 @@ cifs_get_tcp_session(struct smb3_fs_context *ctx,
module_put(THIS_MODULE);
goto out_err_crypto_release;
}
+ /* Keep the server alive until the demultiplex thread has exited */
+ refcount_inc(&tcp_ses->obj_refs);
/* thread created, put it on the list */
spin_lock(&cifs_tcp_ses_lock);
list_add(&tcp_ses->tcp_ses_list, &cifs_tcp_ses_list);
diff --git a/fs/smb/client/inode.c b/fs/smb/client/inode.c
index 7c638b8d5490..b8f9dd47a56e 100644
--- a/fs/smb/client/inode.c
+++ b/fs/smb/client/inode.c
@@ -87,7 +87,9 @@ static void cifs_set_ops(struct inode *inode)
else
inode->i_data.a_ops = &cifs_addr_ops;
mapping_set_large_folios(inode->i_mapping);
- if (cifs_sb_master_tcon(cifs_sb)->ses->server->sign)
+ if (cifs_sb_master_tcon(cifs_sb)->ses->server->sign ||
+ (cifs_sb_master_tcon(cifs_sb)->ses->server->compression.enabled &&
+ (cifs_sb_master_tcon(cifs_sb)->share_flags & SMB2_SHAREFLAG_COMPRESS_DATA)))
mapping_set_stable_writes(inode->i_mapping);
break;
case S_IFDIR:
@@ -1348,7 +1350,7 @@ static int cifs_get_fattr(struct cifs_open_info_data *data,
/* for easier reading */
FILE_ALL_INFO *fi;
FILE_DIRECTORY_INFO *fdi;
- SEARCH_ID_FULL_DIR_INFO *si;
+ FILE_ID_FULL_DIR_INFO *si;
rc = cifs_backup_query_path_info(xid, tcon, sb,
full_path,
@@ -1359,7 +1361,7 @@ static int cifs_get_fattr(struct cifs_open_info_data *data,
move_cifs_info_to_smb2(&data->fi, fi);
fdi = (FILE_DIRECTORY_INFO *)fi;
- si = (SEARCH_ID_FULL_DIR_INFO *)fi;
+ si = (FILE_ID_FULL_DIR_INFO *)fi;
cifs_dir_info_to_fattr(fattr, fdi, cifs_sb);
fattr->cf_uniqueid = le64_to_cpu(si->UniqueId);
diff --git a/fs/smb/client/readdir.c b/fs/smb/client/readdir.c
index b19c36f6a78b..00c3b7a82f4c 100644
--- a/fs/smb/client/readdir.c
+++ b/fs/smb/client/readdir.c
@@ -455,17 +455,20 @@ initiate_cifs_search(const unsigned int xid, struct file *file,
}
/* return length of unicode string in bytes */
-static int cifs_unicode_bytelen(const char *str)
+static size_t cifs_unicode_bytelen(const char *str, size_t maxlen)
{
- int len;
const __le16 *ustr = (const __le16 *)str;
+ size_t len;
- for (len = 0; len <= PATH_MAX; len++) {
+ maxlen = umin(PATH_MAX + 1, maxlen / sizeof(__le16));
+ for (len = 0; len < maxlen; len++) {
if (ustr[len] == 0)
- return len << 1;
+ return len * sizeof(__le16);
}
- cifs_dbg(FYI, "Unicode string longer than PATH_MAX found\n");
- return len << 1;
+ len *= sizeof(__le16);
+ cifs_dbg(FYI, "%s: Unicode string not terminated within %zu bytes\n",
+ __func__, len);
+ return len;
}
static char *nxt_dir_entry(char *old_entry, char *end_of_smb, int level)
@@ -531,13 +534,13 @@ static void cifs_fill_dirent_posix(struct cifs_dirent *de,
}
static void cifs_fill_dirent_unix(struct cifs_dirent *de,
- const FILE_UNIX_INFO *info, bool is_unicode)
+ const FILE_UNIX_INFO *info, bool is_unicode, size_t maxlen)
{
de->name = &info->FileName[0];
if (is_unicode)
- de->namelen = cifs_unicode_bytelen(de->name);
+ de->namelen = cifs_unicode_bytelen(de->name, maxlen);
else
- de->namelen = strnlen(de->name, PATH_MAX);
+ de->namelen = strnlen(de->name, min_t(size_t, maxlen, PATH_MAX));
de->resume_key = info->ResumeKey;
de->ino = le64_to_cpu(info->basic.UniqueId);
}
@@ -559,7 +562,7 @@ static void cifs_fill_dirent_full(struct cifs_dirent *de,
}
static void cifs_fill_dirent_search(struct cifs_dirent *de,
- const SEARCH_ID_FULL_DIR_INFO *info)
+ const FILE_ID_FULL_DIR_INFO *info)
{
de->name = &info->FileName[0];
de->namelen = le32_to_cpu(info->FileNameLength);
@@ -585,30 +588,49 @@ static void cifs_fill_dirent_std(struct cifs_dirent *de,
}
static int cifs_fill_dirent(struct cifs_dirent *de, const void *info,
- u16 level, bool is_unicode)
+ const char *end, u16 level, bool is_unicode)
{
+ size_t len = end > (const char *)info ? end - (const char *)info : 0;
+ size_t nul_len = 0;
+
memset(de, 0, sizeof(*de));
switch (level) {
case SMB_FIND_FILE_POSIX_INFO:
+ if (len < sizeof(struct smb2_posix_info))
+ goto too_short;
cifs_fill_dirent_posix(de, info);
break;
case SMB_FIND_FILE_UNIX:
- cifs_fill_dirent_unix(de, info, is_unicode);
+ if (len < offsetof(FILE_UNIX_INFO, FileName))
+ goto too_short;
+ cifs_fill_dirent_unix(de, info, is_unicode,
+ len - offsetof(FILE_UNIX_INFO, FileName));
+ nul_len = is_unicode ? sizeof(__le16) : sizeof(__u8);
break;
case SMB_FIND_FILE_DIRECTORY_INFO:
+ if (len < offsetof(FILE_DIRECTORY_INFO, FileName))
+ goto too_short;
cifs_fill_dirent_dir(de, info);
break;
case SMB_FIND_FILE_FULL_DIRECTORY_INFO:
+ if (len < offsetof(FILE_FULL_DIRECTORY_INFO, FileName))
+ goto too_short;
cifs_fill_dirent_full(de, info);
break;
case SMB_FIND_FILE_ID_FULL_DIR_INFO:
+ if (len < offsetof(FILE_ID_FULL_DIR_INFO, FileName))
+ goto too_short;
cifs_fill_dirent_search(de, info);
break;
case SMB_FIND_FILE_BOTH_DIRECTORY_INFO:
+ if (len < offsetof(FILE_BOTH_DIRECTORY_INFO, FileName))
+ goto too_short;
cifs_fill_dirent_both(de, info);
break;
case SMB_FIND_FILE_INFO_STANDARD:
+ if (len < offsetof(FIND_FILE_STANDARD_INFO, FileName))
+ goto too_short;
cifs_fill_dirent_std(de, info);
break;
default:
@@ -616,7 +638,20 @@ static int cifs_fill_dirent(struct cifs_dirent *de, const void *info,
return -EINVAL;
}
+ if (de->name &&
+ (end < de->name ||
+ (size_t)(end - de->name) < de->namelen + nul_len)) {
+ cifs_dbg(VFS, "%s: search entry name extends past end of SMB\n",
+ __func__);
+ return -EINVAL;
+ }
+
return 0;
+
+too_short:
+ cifs_dbg(VFS, "%s: search entry extends past end of SMB\n",
+ __func__);
+ return -EINVAL;
}
#define UNICODE_DOT cpu_to_le16(0x2e)
@@ -671,10 +706,14 @@ static int is_dir_changed(struct file *file)
static int cifs_save_resume_key(const char *current_entry,
struct cifsFileInfo *file_info)
{
+ struct TCP_Server_Info *server = tlink_tcon(file_info->tlink)->ses->server;
+ char *buf = file_info->srch_inf.ntwrk_buf_start;
+ const char *end = buf + server->ops->calc_smb_size(buf);
struct cifs_dirent de;
int rc;
- rc = cifs_fill_dirent(&de, current_entry, file_info->srch_inf.info_level,
+ rc = cifs_fill_dirent(&de, current_entry, end,
+ file_info->srch_inf.info_level,
file_info->srch_inf.unicode);
if (!rc) {
file_info->srch_inf.presume_name = de.name;
@@ -973,7 +1012,8 @@ static int cifs_filldir(char *find_entry, struct file *file,
struct qstr name;
int rc = 0;
- rc = cifs_fill_dirent(&de, find_entry, file_info->srch_inf.info_level,
+ rc = cifs_fill_dirent(&de, find_entry, end_of_smb,
+ file_info->srch_inf.info_level,
file_info->srch_inf.unicode);
if (rc)
return rc;
diff --git a/fs/smb/client/smb2misc.c b/fs/smb/client/smb2misc.c
index 34d2401c6c66..46c98d024f06 100644
--- a/fs/smb/client/smb2misc.c
+++ b/fs/smb/client/smb2misc.c
@@ -194,6 +194,21 @@ smb2_check_message(char *buf, unsigned int len, struct TCP_Server_Info *server)
return 1;
}
+ /*
+ * A SESSION_SETUP response that asks for more processing carries a
+ * security blob and is parsed as a normal reply. Do not let the
+ * short-error path below accept a frame that cannot hold the fixed
+ * response fields.
+ */
+ if (command == SMB2_SESSION_SETUP_HE &&
+ shdr->Status == STATUS_MORE_PROCESSING_REQUIRED &&
+ len < sizeof(struct smb2_sess_setup_rsp)) {
+ cifs_server_dbg(VFS,
+ "%s: session setup response too short: %u < %zu\n",
+ __func__, len, sizeof(struct smb2_sess_setup_rsp));
+ return 1;
+ }
+
if (len < pdu_size) {
if ((len >= hdr_size)
&& (shdr->Status != 0)) {
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index 7cf6419df037..f7f8e8b981f7 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -635,8 +635,8 @@ parse_server_interfaces(struct network_interface_info_ioctl_rsp *buf,
struct network_interface_info_ioctl_rsp *p;
struct sockaddr_in *addr4;
struct sockaddr_in6 *addr6;
- struct iface_info_ipv4 *p4;
- struct iface_info_ipv6 *p6;
+ struct smb_sockaddr_in *p4;
+ struct smb_sockaddr_in6 *p6;
struct cifs_server_iface *info = NULL, *iface = NULL, *niface = NULL;
struct cifs_server_iface tmp_iface;
__be16 port;
@@ -698,7 +698,7 @@ parse_server_interfaces(struct network_interface_info_ioctl_rsp *buf,
*/
case INTERNETWORK:
addr4 = (struct sockaddr_in *)&tmp_iface.sockaddr;
- p4 = (struct iface_info_ipv4 *)p->Buffer;
+ p4 = (struct smb_sockaddr_in *)p->Buffer;
addr4->sin_family = AF_INET;
memcpy(&addr4->sin_addr, &p4->IPv4Address, 4);
@@ -710,7 +710,7 @@ parse_server_interfaces(struct network_interface_info_ioctl_rsp *buf,
break;
case INTERNETWORKV6:
addr6 = (struct sockaddr_in6 *)&tmp_iface.sockaddr;
- p6 = (struct iface_info_ipv6 *)p->Buffer;
+ p6 = (struct smb_sockaddr_in6 *)p->Buffer;
addr6->sin6_family = AF_INET6;
memcpy(&addr6->sin6_addr, &p6->IPv6Address, 16);
@@ -5256,9 +5256,10 @@ static void smb2_decrypt_offload(struct work_struct *work)
dw->server->lstrp = jiffies;
mid = smb2_find_dequeue_mid(dw->server, dw->buf);
- if (mid == NULL)
+ if (mid == NULL) {
cifs_dbg(FYI, "mid not found\n");
- else {
+ smb2_add_credits_from_hdr(dw->buf, dw->server);
+ } else {
mid->decrypted = true;
rc = handle_read_data(dw->server, mid, dw->buf,
dw->server->vals->read_rsp_size,
@@ -5381,6 +5382,7 @@ receive_encrypted_read(struct TCP_Server_Info *server, struct mid_q_entry **mid,
*mid = smb2_find_mid(server, buf);
if (*mid == NULL) {
cifs_dbg(FYI, "mid not found\n");
+ smb2_add_credits_from_hdr(buf, server);
rc = -1;
} else {
cifs_dbg(FYI, "mid found\n");
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index 6d610f136f43..3302923f9889 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -1757,15 +1757,21 @@ SMB2_sess_auth_rawntlmssp_negotiate(struct SMB2_sess_data *sess_data)
if (rc)
goto out;
- if (offsetof(struct smb2_sess_setup_rsp, Buffer) !=
- le16_to_cpu(rsp->SecurityBufferOffset)) {
- cifs_dbg(VFS, "Invalid security buffer offset %d\n",
- le16_to_cpu(rsp->SecurityBufferOffset));
+ u16 boff = le16_to_cpu(rsp->SecurityBufferOffset);
+ u16 blen = le16_to_cpu(rsp->SecurityBufferLength);
+
+ if (offsetof(struct smb2_sess_setup_rsp, Buffer) != boff) {
+ cifs_dbg(VFS, "Invalid security buffer offset %d\n", boff);
rc = -EIO;
goto out;
}
- rc = decode_ntlmssp_challenge(rsp->Buffer,
- le16_to_cpu(rsp->SecurityBufferLength), ses);
+ if ((u32)boff + blen > sess_data->iov[0].iov_len) {
+ cifs_dbg(VFS, "%s: invalid security buffer length: %u\n",
+ __func__, blen);
+ rc = -EINVAL;
+ goto out;
+ }
+ rc = decode_ntlmssp_challenge(rsp->Buffer, blen, ses);
if (rc)
goto out;
@@ -2456,21 +2462,21 @@ add_lease_context(struct TCP_Server_Info *server,
return 0;
}
-static struct create_durable_v2 *
+static struct create_durable_req_v2 *
create_durable_v2_buf(struct cifs_open_parms *oparms)
{
struct cifs_fid *pfid = oparms->fid;
- struct create_durable_v2 *buf;
+ struct create_durable_req_v2 *buf;
- buf = kzalloc(sizeof(struct create_durable_v2), GFP_KERNEL);
+ buf = kzalloc(sizeof(struct create_durable_req_v2), GFP_KERNEL);
if (!buf)
return NULL;
buf->ccontext.DataOffset = cpu_to_le16(offsetof
- (struct create_durable_v2, dcontext));
- buf->ccontext.DataLength = cpu_to_le32(sizeof(struct durable_context_v2));
+ (struct create_durable_req_v2, dcontext));
+ buf->ccontext.DataLength = cpu_to_le32(sizeof(struct durable_context_v2_req));
buf->ccontext.NameOffset = cpu_to_le16(offsetof
- (struct create_durable_v2, Name));
+ (struct create_durable_req_v2, Name));
buf->ccontext.NameLength = cpu_to_le16(4);
/*
@@ -2540,7 +2546,7 @@ add_durable_v2_context(struct kvec *iov, unsigned int *num_iovec,
iov[num].iov_base = create_durable_v2_buf(oparms);
if (iov[num].iov_base == NULL)
return -ENOMEM;
- iov[num].iov_len = sizeof(struct create_durable_v2);
+ iov[num].iov_len = sizeof(struct create_durable_req_v2);
*num_iovec = num + 1;
return 0;
}
@@ -4839,6 +4845,7 @@ SMB2_read(const unsigned int xid, struct cifs_io_parms *io_parms,
struct kvec iov[1];
struct kvec rsp_iov;
unsigned int total_len;
+ u16 data_offset;
int flags = CIFS_LOG_ERROR;
struct cifs_ses *ses = io_parms->tcon->ses;
@@ -4897,8 +4904,28 @@ SMB2_read(const unsigned int xid, struct cifs_io_parms *io_parms,
*nbytes = 0;
}
+ data_offset = rsp->DataOffset;
+ if (data_offset < io_parms->server->vals->read_rsp_size) {
+ /*
+ * win2k8 sometimes sends an offset of 0 when the read is beyond
+ * the EOF. Treat it as if the data starts just after the header,
+ * as handle_read_data() does.
+ */
+ cifs_dbg(FYI, "data offset %u inside read response header\n",
+ data_offset);
+ data_offset = io_parms->server->vals->read_rsp_size;
+ }
+
+ /* The data area must lie within the response actually received */
+ if (*nbytes && (size_t)data_offset + *nbytes > rsp_iov.iov_len) {
+ cifs_dbg(FYI, "bad data offset %u length %u for response of %zu\n",
+ data_offset, *nbytes, rsp_iov.iov_len);
+ rc = -EIO;
+ *nbytes = 0;
+ }
+
if (*buf) {
- memcpy(*buf, (char *)rsp + rsp->DataOffset, *nbytes);
+ memcpy(*buf, (char *)rsp + data_offset, *nbytes);
free_rsp_buf(resp_buftype, rsp_iov.iov_base);
} else if (resp_buftype != CIFS_NO_BUFFER) {
*buf = rsp_iov.iov_base;
@@ -5555,7 +5582,7 @@ smb2_parse_query_directory(struct cifs_tcon *tcon,
info_buf_size = sizeof(FILE_DIRECTORY_INFO);
break;
case SMB_FIND_FILE_ID_FULL_DIR_INFO:
- info_buf_size = sizeof(SEARCH_ID_FULL_DIR_INFO);
+ info_buf_size = sizeof(FILE_ID_FULL_DIR_INFO);
break;
case SMB_FIND_FILE_POSIX_INFO:
/* note that posix payload are variable size */
diff --git a/fs/smb/client/smb2pdu.h b/fs/smb/client/smb2pdu.h
index cdf7c588cc00..2a2f52be7d6f 100644
--- a/fs/smb/client/smb2pdu.h
+++ b/fs/smb/client/smb2pdu.h
@@ -133,46 +133,6 @@ struct share_redirect_error_context_rsp {
#define SMB2_LEASE_HANDLE_CACHING_HE 0x02
#define SMB2_LEASE_WRITE_CACHING_HE 0x04
-
-/* See MS-SMB2 2.2.13.2.11 */
-/* Flags */
-#define SMB2_DHANDLE_FLAG_PERSISTENT 0x00000002
-struct durable_context_v2 {
- __le32 Timeout;
- __le32 Flags;
- __u64 Reserved;
- __u8 CreateGuid[16];
-} __packed;
-
-struct create_durable_v2 {
- struct create_context_hdr ccontext;
- __u8 Name[8];
- struct durable_context_v2 dcontext;
-} __packed;
-
-/* See MS-SMB2 2.2.13.2.12 */
-struct durable_reconnect_context_v2 {
- struct {
- __u64 PersistentFileId;
- __u64 VolatileFileId;
- } Fid;
- __u8 CreateGuid[16];
- __le32 Flags; /* see above DHANDLE_FLAG_PERSISTENT */
-} __packed;
-
-/* See MS-SMB2 2.2.14.2.12 */
-struct durable_reconnect_context_v2_rsp {
- __le32 Timeout;
- __le32 Flags; /* see above DHANDLE_FLAG_PERSISTENT */
-} __packed;
-
-struct create_durable_handle_reconnect_v2 {
- struct create_context_hdr ccontext;
- __u8 Name[8];
- struct durable_reconnect_context_v2 dcontext;
- __u8 Pad[4];
-} __packed;
-
/* See MS-SMB2 2.2.13.2.5 */
struct crt_twarp_ctxt {
struct create_context_hdr ccontext;
@@ -239,35 +199,6 @@ struct network_resiliency_req {
} __packed;
/* There is no buffer for the response ie no struct network_resiliency_rsp */
-#define RSS_CAPABLE cpu_to_le32(0x00000001)
-#define RDMA_CAPABLE cpu_to_le32(0x00000002)
-
-#define INTERNETWORK cpu_to_le16(0x0002)
-#define INTERNETWORKV6 cpu_to_le16(0x0017)
-
-struct network_interface_info_ioctl_rsp {
- __le32 Next; /* next interface. zero if this is last one */
- __le32 IfIndex;
- __le32 Capability; /* RSS or RDMA Capable */
- __le32 Reserved;
- __le64 LinkSpeed;
- __le16 Family;
- __u8 Buffer[126];
-} __packed;
-
-struct iface_info_ipv4 {
- __be16 Port;
- __be32 IPv4Address;
- __be64 Reserved;
-} __packed;
-
-struct iface_info_ipv6 {
- __be16 Port;
- __be32 FlowInfo;
- __u8 IPv6Address[16];
- __be32 ScopeId;
-} __packed;
-
#define NO_FILE_ID 0xFFFFFFFFFFFFFFFFULL /* general ioctls to srv not to file */
struct compress_ioctl {
@@ -299,19 +230,6 @@ struct smb2_file_reparse_point_info {
__le32 Tag;
} __packed;
-struct smb2_file_network_open_info {
- struct_group_attr(network_open_info, __packed,
- __le64 CreationTime;
- __le64 LastAccessTime;
- __le64 LastWriteTime;
- __le64 ChangeTime;
- __le64 AllocationSize;
- __le64 EndOfFile;
- __le32 Attributes;
- );
- __le32 Reserved;
-} __packed; /* level 34 Query also similar returned in close rsp and open rsp */
-
/* See MS-FSCC 2.4.21 */
struct smb2_file_id_information {
__le64 VolumeSerialNumber;
diff --git a/fs/smb/client/trace.h b/fs/smb/client/trace.h
index a56fbacf28d1..670612d1a449 100644
--- a/fs/smb/client/trace.h
+++ b/fs/smb/client/trace.h
@@ -42,6 +42,7 @@
EM(smb_eio_trace_ioctl_no_rsp, "ioctl_no_rsp") \
EM(smb_eio_trace_ioctl_out_off, "ioctl_out_off") \
EM(smb_eio_trace_lock_bcc_too_small, "lock_bcc_too_small") \
+ EM(smb_eio_trace_lock_data_area, "lock_data_area") \
EM(smb_eio_trace_lock_data_too_small, "lock_data_too_small") \
EM(smb_eio_trace_malformed_ksid_key, "malformed_ksid_key") \
EM(smb_eio_trace_malformed_sid_key, "malformed_sid_key") \
diff --git a/fs/smb/common/fscc.h b/fs/smb/common/fscc.h
new file mode 100644
index 000000000000..6fa7c65d304b
--- /dev/null
+++ b/fs/smb/common/fscc.h
@@ -0,0 +1,157 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+/*
+ *
+ * Copyright (c) International Business Machines Corp., 2009, 2013
+ * Etersoft, 2012
+ * 2018 Samsung Electronics Co., Ltd.
+ * Author(s): Steve French (sfrench@us.ibm.com)
+ * Pavel Shilovsky (pshilovsky@samba.org) 2012
+ * Namjae Jeon (linkinjeon@kernel.org)
+ *
+ */
+#ifndef _COMMON_SMB_FSCC_H
+#define _COMMON_SMB_FSCC_H
+
+/* See MS-FSCC 2.4.8 */
+typedef struct {
+ __le32 NextEntryOffset;
+ __u32 FileIndex;
+ __le64 CreationTime;
+ __le64 LastAccessTime;
+ __le64 LastWriteTime;
+ __le64 ChangeTime;
+ __le64 EndOfFile;
+ __le64 AllocationSize;
+ __le32 ExtFileAttributes;
+ __le32 FileNameLength;
+ __le32 EaSize; /* length of the xattrs */
+ __u8 ShortNameLength;
+ __u8 Reserved;
+ __u8 ShortName[24];
+ char FileName[];
+} __packed FILE_BOTH_DIRECTORY_INFO; /* level 0x104 FFrsp data */
+
+/* See MS-FSCC 2.4.10 */
+typedef struct {
+ __le32 NextEntryOffset;
+ __u32 FileIndex;
+ __le64 CreationTime;
+ __le64 LastAccessTime;
+ __le64 LastWriteTime;
+ __le64 ChangeTime;
+ __le64 EndOfFile;
+ __le64 AllocationSize;
+ __le32 ExtFileAttributes;
+ __le32 FileNameLength;
+ char FileName[];
+} __packed FILE_DIRECTORY_INFO; /* level 0x101 FF resp data */
+
+/* See MS-FSCC 2.4.14 */
+typedef struct {
+ __le32 NextEntryOffset;
+ __u32 FileIndex;
+ __le64 CreationTime;
+ __le64 LastAccessTime;
+ __le64 LastWriteTime;
+ __le64 ChangeTime;
+ __le64 EndOfFile;
+ __le64 AllocationSize;
+ __le32 ExtFileAttributes;
+ __le32 FileNameLength;
+ __le32 EaSize; /* length of the xattrs */
+ char FileName[];
+} __packed FILE_FULL_DIRECTORY_INFO; /* level 0x102 rsp data */
+
+/* See MS-FSCC 2.4.24 */
+typedef struct {
+ __le32 NextEntryOffset;
+ __u32 FileIndex;
+ __le64 CreationTime;
+ __le64 LastAccessTime;
+ __le64 LastWriteTime;
+ __le64 ChangeTime;
+ __le64 EndOfFile;
+ __le64 AllocationSize;
+ __le32 ExtFileAttributes;
+ __le32 FileNameLength;
+ __le32 EaSize; /* EA size */
+ __le32 Reserved;
+ __le64 UniqueId; /* inode num - le since Samba puts ino in low 32 bit*/
+ char FileName[];
+} __packed FILE_ID_FULL_DIR_INFO; /* level 0x105 FF rsp data */
+
+/* See MS-FSCC 2.4.34 */
+struct smb2_file_network_open_info {
+ struct_group_attr(network_open_info, __packed,
+ __le64 CreationTime;
+ __le64 LastAccessTime;
+ __le64 LastWriteTime;
+ __le64 ChangeTime;
+ __le64 AllocationSize;
+ __le64 EndOfFile;
+ __le32 Attributes;
+ );
+ __le32 Reserved;
+} __packed; /* level 34 Query also similar returned in close rsp and open rsp */
+
+/* List of FileSystemAttributes - see MS-FSCC 2.5.1 */
+#define FILE_SUPPORTS_SPARSE_VDL 0x10000000 /* faster nonsparse extend */
+#define FILE_SUPPORTS_BLOCK_REFCOUNTING 0x08000000 /* allow ioctl dup extents */
+#define FILE_SUPPORT_INTEGRITY_STREAMS 0x04000000
+#define FILE_SUPPORTS_USN_JOURNAL 0x02000000
+#define FILE_SUPPORTS_OPEN_BY_FILE_ID 0x01000000
+#define FILE_SUPPORTS_EXTENDED_ATTRIBUTES 0x00800000
+#define FILE_SUPPORTS_HARD_LINKS 0x00400000
+#define FILE_SUPPORTS_TRANSACTIONS 0x00200000
+#define FILE_SEQUENTIAL_WRITE_ONCE 0x00100000
+#define FILE_READ_ONLY_VOLUME 0x00080000
+#define FILE_NAMED_STREAMS 0x00040000
+#define FILE_SUPPORTS_ENCRYPTION 0x00020000
+#define FILE_SUPPORTS_OBJECT_IDS 0x00010000
+#define FILE_VOLUME_IS_COMPRESSED 0x00008000
+#define FILE_SUPPORTS_POSIX_UNLINK_RENAME 0x00000400
+#define FILE_RETURNS_CLEANUP_RESULT_INFO 0x00000200
+#define FILE_SUPPORTS_REMOTE_STORAGE 0x00000100
+#define FILE_SUPPORTS_REPARSE_POINTS 0x00000080
+#define FILE_SUPPORTS_SPARSE_FILES 0x00000040
+#define FILE_VOLUME_QUOTAS 0x00000020
+#define FILE_FILE_COMPRESSION 0x00000010
+#define FILE_PERSISTENT_ACLS 0x00000008
+#define FILE_UNICODE_ON_DISK 0x00000004
+#define FILE_CASE_PRESERVED_NAMES 0x00000002
+#define FILE_CASE_SENSITIVE_SEARCH 0x00000001
+
+/* See MS-FSCC 2.5.10 */
+typedef struct {
+ __le32 DeviceType;
+ __le32 DeviceCharacteristics;
+} __packed FILE_SYSTEM_DEVICE_INFO; /* device info level 0x104 */
+
+/*
+ * See POSIX Extensions to MS-FSCC 2.3.2.1
+ * Link: https://gitlab.com/samba-team/smb3-posix-spec/-/blob/master/fscc_posix_extensions.md
+ */
+typedef struct {
+ /* For undefined recommended transfer size return -1 in that field */
+ __le32 OptimalTransferSize; /* bsize on some os, iosize on other os */
+ __le32 BlockSize;
+ /* The next three fields are in terms of the block size.
+ * (above). If block size is unknown, 4096 would be a
+ * reasonable block size for a server to report.
+ * Note that returning the blocks/blocksavail removes need
+ * to make a second call (to QFSInfo level 0x103 to get this info.
+ * UserBlockAvail is typically less than or equal to BlocksAvail,
+ * if no distinction is made return the same value in each
+ */
+ __le64 TotalBlocks;
+ __le64 BlocksAvail; /* bfree */
+ __le64 UserBlocksAvail; /* bavail */
+ /* For undefined Node fields or FSID return -1 */
+ __le64 TotalFileNodes;
+ __le64 FreeFileNodes;
+ __le64 FileSysIdentifier; /* fsid */
+ /* NB Namelen comes from FILE_SYSTEM_ATTRIBUTE_INFO call */
+ /* NB flags can come from FILE_SYSTEM_DEVICE_INFO call */
+} __packed FILE_SYSTEM_POSIX_INFO;
+
+#endif /* _COMMON_SMB_FSCC_H */
diff --git a/fs/smb/common/smb2pdu.h b/fs/smb/common/smb2pdu.h
index 528ba77247e8..4b8ed28a72c6 100644
--- a/fs/smb/common/smb2pdu.h
+++ b/fs/smb/common/smb2pdu.h
@@ -1149,12 +1149,6 @@ struct smb2_server_client_notification {
#define FILE_OVERWRITE_IF_LE cpu_to_le32(0x00000005)
#define FILE_CREATE_MASK_LE cpu_to_le32(0x00000007)
-#define FILE_READ_RIGHTS (FILE_READ_DATA | FILE_READ_EA \
- | FILE_READ_ATTRIBUTES)
-#define FILE_WRITE_RIGHTS (FILE_WRITE_DATA | FILE_APPEND_DATA \
- | FILE_WRITE_EA | FILE_WRITE_ATTRIBUTES)
-#define FILE_EXEC_RIGHTS (FILE_EXECUTE)
-
/* CreateOptions Flags */
#define FILE_DIRECTORY_FILE_LE cpu_to_le32(0x00000001)
/* same as #define CREATE_NOT_FILE_LE cpu_to_le32(0x00000001) */
@@ -1290,6 +1284,57 @@ struct create_mxac_req {
__le64 Timestamp;
} __packed;
+/*
+ * Flags
+ * See MS-SMB2 2.2.13.2.11
+ * MS-SMB2 2.2.13.2.12
+ * MS-SMB2 2.2.14.2.12
+ */
+#define SMB2_DHANDLE_FLAG_PERSISTENT 0x00000002
+
+/* See MS-SMB2 2.2.13.2.11 */
+struct durable_context_v2_req {
+ __le32 Timeout;
+ __le32 Flags; /* see SMB2_DHANDLE_FLAG_PERSISTENT */
+ __u64 Reserved;
+ __u8 CreateGuid[16];
+} __packed;
+
+struct create_durable_req_v2 {
+ struct create_context_hdr ccontext;
+ __u8 Name[8];
+ struct durable_context_v2_req dcontext;
+} __packed;
+
+/* See MS-SMB2 2.2.13.2.12 */
+struct durable_reconnect_context_v2 {
+ struct {
+ __u64 PersistentFileId;
+ __u64 VolatileFileId;
+ } Fid;
+ __u8 CreateGuid[16];
+ __le32 Flags; /* see SMB2_DHANDLE_FLAG_PERSISTENT */
+} __packed;
+
+struct create_durable_handle_reconnect_v2 {
+ struct create_context_hdr ccontext;
+ __u8 Name[8];
+ struct durable_reconnect_context_v2 dcontext;
+ __u8 Pad[4];
+} __packed;
+
+/* See MS-SMB2 2.2.14.2.12 */
+struct durable_context_v2_rsp {
+ __le32 Timeout;
+ __le32 Flags; /* see SMB2_DHANDLE_FLAG_PERSISTENT */
+} __packed;
+
+struct create_durable_rsp_v2 {
+ struct create_context_hdr ccontext;
+ __u8 Name[8];
+ struct durable_context_v2_rsp dcontext;
+} __packed;
+
/* See MS-SMB2 2.2.14.2.5 */
struct create_mxac_rsp {
struct create_context_hdr ccontext;
@@ -1433,6 +1478,41 @@ struct smb2_ioctl_rsp {
__u8 Buffer[];
} __packed;
+/* See MS-SMB2 2.2.32.5.1.1 */
+struct smb_sockaddr_in {
+ __be16 Port;
+ __be32 IPv4Address;
+ __u8 Reserved[8];
+} __packed;
+
+/* See MS-SMB2 2.2.32.5.1.2 */
+struct smb_sockaddr_in6 {
+ __be16 Port;
+ __be32 FlowInfo;
+ __u8 IPv6Address[16];
+ __be32 ScopeId;
+} __packed;
+
+/* See MS-SMB2 2.2.32.5 and MS-SMB2 2.2.32.5.1 */
+#define RSS_CAPABLE cpu_to_le32(0x00000001)
+#define RDMA_CAPABLE cpu_to_le32(0x00000002)
+#define INTERNETWORK cpu_to_le16(0x0002)
+#define INTERNETWORKV6 cpu_to_le16(0x0017)
+struct network_interface_info_ioctl_rsp {
+ __le32 Next; /* next interface. zero if this is last one */
+ __le32 IfIndex;
+ __le32 Capability; /* RSS or RDMA Capable */
+ __le32 Reserved;
+ __le64 LinkSpeed;
+ union {
+ char SockAddr_Storage[128];
+ struct {
+ __le16 Family;
+ __u8 Buffer[126];
+ };
+ };
+} __packed;
+
/* this goes in the ioctl buffer when doing FSCTL_SET_ZERO_DATA */
struct file_zero_data_information {
__le64 FileOffset;
@@ -1922,6 +2002,144 @@ struct smb2_lease_ack {
__le64 LeaseDuration;
} __packed;
+/*
+ * See MS-CIFS 2.2.3.1
+ * MS-SMB 2.2.3.1
+ */
+struct smb_hdr {
+ __be32 smb_buf_length; /* BB length is only two (rarely three) bytes,
+ with one or two byte "type" preceding it that will be
+ zero - we could mask the type byte off */
+ __u8 Protocol[4];
+ __u8 Command;
+ union {
+ struct {
+ __u8 ErrorClass;
+ __u8 Reserved;
+ __le16 Error;
+ } __packed DosError;
+ __le32 CifsError;
+ } __packed Status;
+ __u8 Flags;
+ __le16 Flags2; /* note: le */
+ __le16 PidHigh;
+ union {
+ struct {
+ __le32 SequenceNumber; /* le */
+ __u32 Reserved; /* zero */
+ } __packed Sequence;
+ __u8 SecuritySignature[8]; /* le */
+ } __packed Signature;
+ __u8 pad[2];
+ __u16 Tid;
+ __le16 Pid;
+ __u16 Uid;
+ __le16 Mid;
+ __u8 WordCount;
+} __packed;
+
#define OP_BREAK_STRUCT_SIZE_20 24
#define OP_BREAK_STRUCT_SIZE_21 36
+
+/*
+ * See MS-SMB2 2.2.13.1.1
+ * MS-SMB 2.2.1.4.1
+ * These are the file access permission bits defined in CIFS for the
+ * NTCreateAndX as well as the level 0x107
+ * TRANS2_QUERY_PATH_INFORMATION API. The level 0x107, SMB_QUERY_FILE_ALL_INFO
+ * responds with the AccessFlags.
+ * The AccessFlags specifies the access permissions a caller has to the
+ * file and can have any suitable combination of the following values:
+ */
+#define FILE_READ_DATA 0x00000001 /* Data can be read from the file */
+ /* or directory child entries can */
+ /* be listed together with the */
+ /* associated child attributes */
+ /* (so the FILE_READ_ATTRIBUTES on */
+ /* the child entry is not needed) */
+#define FILE_WRITE_DATA 0x00000002 /* Data can be written to the file */
+ /* or new file can be created in */
+ /* the directory */
+#define FILE_APPEND_DATA 0x00000004 /* Data can be appended to the file */
+ /* (for non-local files over SMB it */
+ /* is same as FILE_WRITE_DATA) */
+ /* or new subdirectory can be */
+ /* created in the directory */
+#define FILE_READ_EA 0x00000008 /* Extended attributes associated */
+ /* with the file can be read */
+#define FILE_WRITE_EA 0x00000010 /* Extended attributes associated */
+ /* with the file can be written */
+#define FILE_EXECUTE 0x00000020 /*Data can be read into memory from */
+ /* the file using system paging I/O */
+ /* for executing the file / script */
+ /* or right to traverse directory */
+ /* (but by default all users have */
+ /* directory bypass traverse */
+ /* privilege and do not need this */
+ /* permission on directories at all)*/
+#define FILE_DELETE_CHILD 0x00000040 /* Child entry can be deleted from */
+ /* the directory (so the DELETE on */
+ /* the child entry is not needed) */
+#define FILE_READ_ATTRIBUTES 0x00000080 /* Attributes associated with the */
+ /* file or directory can be read */
+#define FILE_WRITE_ATTRIBUTES 0x00000100 /* Attributes associated with the */
+ /* file or directory can be written */
+#define DELETE 0x00010000 /* The file or dir can be deleted */
+#define READ_CONTROL 0x00020000 /* The discretionary access control */
+ /* list and ownership associated */
+ /* with the file or dir can be read */
+#define WRITE_DAC 0x00040000 /* The discretionary access control */
+ /* list associated with the file or */
+ /* directory can be written */
+#define WRITE_OWNER 0x00080000 /* Ownership information associated */
+ /* with the file/dir can be written */
+#define SYNCHRONIZE 0x00100000 /* The file handle can waited on to */
+ /* synchronize with the completion */
+ /* of an input/output request */
+#define SYSTEM_SECURITY 0x01000000 /* The system access control list */
+ /* associated with the file or */
+ /* directory can be read or written */
+ /* (cannot be in DACL, can in SACL) */
+#define MAXIMUM_ALLOWED 0x02000000 /* Maximal subset of GENERIC_ALL */
+ /* permissions which can be granted */
+ /* (cannot be in DACL nor SACL) */
+#define GENERIC_ALL 0x10000000 /* Same as: GENERIC_EXECUTE | */
+ /* GENERIC_WRITE | */
+ /* GENERIC_READ | */
+ /* FILE_DELETE_CHILD | */
+ /* DELETE | */
+ /* WRITE_DAC | */
+ /* WRITE_OWNER */
+ /* So GENERIC_ALL contains all bits */
+ /* mentioned above except these two */
+ /* SYSTEM_SECURITY MAXIMUM_ALLOWED */
+#define GENERIC_EXECUTE 0x20000000 /* Same as: FILE_EXECUTE | */
+ /* FILE_READ_ATTRIBUTES | */
+ /* READ_CONTROL | */
+ /* SYNCHRONIZE */
+#define GENERIC_WRITE 0x40000000 /* Same as: FILE_WRITE_DATA | */
+ /* FILE_APPEND_DATA | */
+ /* FILE_WRITE_EA | */
+ /* FILE_WRITE_ATTRIBUTES | */
+ /* READ_CONTROL | */
+ /* SYNCHRONIZE */
+#define GENERIC_READ 0x80000000 /* Same as: FILE_READ_DATA | */
+ /* FILE_READ_EA | */
+ /* FILE_READ_ATTRIBUTES | */
+ /* READ_CONTROL | */
+ /* SYNCHRONIZE */
+
+/* Combinations of file access permission bits */
+#define FILE_READ_RIGHTS (FILE_READ_DATA | FILE_READ_EA | FILE_READ_ATTRIBUTES)
+#define FILE_WRITE_RIGHTS (FILE_WRITE_DATA | FILE_APPEND_DATA \
+ | FILE_WRITE_EA | FILE_WRITE_ATTRIBUTES)
+#define FILE_EXEC_RIGHTS (FILE_EXECUTE)
+#define SET_FILE_EXEC_RIGHTS (FILE_READ_EA | FILE_WRITE_EA | FILE_EXECUTE \
+ | FILE_READ_ATTRIBUTES \
+ | FILE_WRITE_ATTRIBUTES \
+ | DELETE | READ_CONTROL | WRITE_DAC \
+ | WRITE_OWNER | SYNCHRONIZE)
+#define SET_MINIMUM_RIGHTS (FILE_READ_EA | FILE_READ_ATTRIBUTES \
+ | READ_CONTROL | SYNCHRONIZE)
+
#endif /* _COMMON_SMB2PDU_H */
diff --git a/fs/smb/server/mgmt/tree_connect.c b/fs/smb/server/mgmt/tree_connect.c
index c7a89eddcda7..6d8f3ebcbc53 100644
--- a/fs/smb/server/mgmt/tree_connect.c
+++ b/fs/smb/server/mgmt/tree_connect.c
@@ -82,6 +82,8 @@ ksmbd_tree_conn_connect(struct ksmbd_work *work, const char *share_name)
down_write(&sess->tree_conns_lock);
ret = xa_err(xa_store(&sess->tree_conns, tree_conn->id, tree_conn,
KSMBD_DEFAULT_GFP));
+ if (!ret)
+ atomic_inc(&tree_conn->refcount);
up_write(&sess->tree_conns_lock);
if (ret) {
status.ret = -ENOMEM;
@@ -122,6 +124,12 @@ int ksmbd_tree_conn_disconnect(struct ksmbd_session *sess,
struct ksmbd_tree_connect *tree_conn)
{
down_write(&sess->tree_conns_lock);
+ if (tree_conn->t_state == TREE_DISCONNECTED ||
+ xa_load(&sess->tree_conns, tree_conn->id) != tree_conn) {
+ up_write(&sess->tree_conns_lock);
+ return -ENOENT;
+ }
+ tree_conn->t_state = TREE_DISCONNECTED;
xa_erase(&sess->tree_conns, tree_conn->id);
up_write(&sess->tree_conns_lock);
diff --git a/fs/smb/server/oplock.c b/fs/smb/server/oplock.c
index 90cd8c962f78..c6a6c1181be6 100644
--- a/fs/smb/server/oplock.c
+++ b/fs/smb/server/oplock.c
@@ -1858,9 +1858,9 @@ void create_durable_rsp_buf(char *cc)
*/
void create_durable_v2_rsp_buf(char *cc, struct ksmbd_file *fp)
{
- struct create_durable_v2_rsp *buf;
+ struct create_durable_rsp_v2 *buf;
- buf = (struct create_durable_v2_rsp *)cc;
+ buf = (struct create_durable_rsp_v2 *)cc;
memset(buf, 0, sizeof(struct create_durable_rsp));
buf->ccontext.DataOffset = cpu_to_le16(offsetof
(struct create_durable_rsp, Data));
@@ -1874,9 +1874,9 @@ void create_durable_v2_rsp_buf(char *cc, struct ksmbd_file *fp)
buf->Name[2] = '2';
buf->Name[3] = 'Q';
- buf->Timeout = cpu_to_le32(fp->durable_timeout);
+ buf->dcontext.Timeout = cpu_to_le32(fp->durable_timeout);
if (fp->is_persistent)
- buf->Flags = cpu_to_le32(SMB2_DHANDLE_FLAG_PERSISTENT);
+ buf->dcontext.Flags = cpu_to_le32(SMB2_DHANDLE_FLAG_PERSISTENT);
}
/**
diff --git a/fs/smb/server/smb2ops.c b/fs/smb/server/smb2ops.c
index 4c50b7f57f76..5aa8e63f3190 100644
--- a/fs/smb/server/smb2ops.c
+++ b/fs/smb/server/smb2ops.c
@@ -59,7 +59,7 @@ static struct smb_version_values smb30_server_values = {
.cap_large_files = SMB2_LARGE_FILES,
.create_lease_size = sizeof(struct create_lease_v2),
.create_durable_size = sizeof(struct create_durable_rsp),
- .create_durable_v2_size = sizeof(struct create_durable_v2_rsp),
+ .create_durable_v2_size = sizeof(struct create_durable_rsp_v2),
.create_mxac_size = sizeof(struct create_mxac_rsp),
.create_disk_id_size = sizeof(struct create_disk_id_rsp),
.create_posix_size = sizeof(struct create_posix_rsp),
@@ -86,7 +86,7 @@ static struct smb_version_values smb302_server_values = {
.cap_large_files = SMB2_LARGE_FILES,
.create_lease_size = sizeof(struct create_lease_v2),
.create_durable_size = sizeof(struct create_durable_rsp),
- .create_durable_v2_size = sizeof(struct create_durable_v2_rsp),
+ .create_durable_v2_size = sizeof(struct create_durable_rsp_v2),
.create_mxac_size = sizeof(struct create_mxac_rsp),
.create_disk_id_size = sizeof(struct create_disk_id_rsp),
.create_posix_size = sizeof(struct create_posix_rsp),
@@ -113,7 +113,7 @@ static struct smb_version_values smb311_server_values = {
.cap_large_files = SMB2_LARGE_FILES,
.create_lease_size = sizeof(struct create_lease_v2),
.create_durable_size = sizeof(struct create_durable_rsp),
- .create_durable_v2_size = sizeof(struct create_durable_v2_rsp),
+ .create_durable_v2_size = sizeof(struct create_durable_rsp_v2),
.create_mxac_size = sizeof(struct create_mxac_rsp),
.create_disk_id_size = sizeof(struct create_disk_id_rsp),
.create_posix_size = sizeof(struct create_posix_rsp),
diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c
index a8a42fde4ca0..ec4c54c9c6dc 100644
--- a/fs/smb/server/smb2pdu.c
+++ b/fs/smb/server/smb2pdu.c
@@ -2023,6 +2023,7 @@ int smb2_tree_connect(struct ksmbd_work *work)
struct ksmbd_session *sess = work->sess;
char *treename = NULL, *name = NULL;
struct ksmbd_tree_conn_status status;
+ struct ksmbd_tree_connect *tree_conn = NULL;
struct ksmbd_share_config *share = NULL;
int rc = -EINVAL;
@@ -2049,9 +2050,10 @@ int smb2_tree_connect(struct ksmbd_work *work)
name, treename);
status = ksmbd_tree_conn_connect(work, name);
- if (status.ret == KSMBD_TREE_CONN_STATUS_OK)
+ if (status.ret == KSMBD_TREE_CONN_STATUS_OK) {
+ tree_conn = status.tree_conn;
rsp->hdr.Id.SyncId.TreeId = cpu_to_le32(status.tree_conn->id);
- else
+ } else
goto out_err1;
share = status.tree_conn->share_conf;
@@ -2083,8 +2085,15 @@ int smb2_tree_connect(struct ksmbd_work *work)
status.tree_conn->posix_extensions = true;
down_write(&sess->tree_conns_lock);
- status.tree_conn->t_state = TREE_CONNECTED;
+ if (status.tree_conn->t_state == TREE_DISCONNECTED) {
+ status.ret = KSMBD_TREE_CONN_STATUS_ERROR;
+ share = NULL;
+ } else {
+ status.tree_conn->t_state = TREE_CONNECTED;
+ }
up_write(&sess->tree_conns_lock);
+ if (status.ret != KSMBD_TREE_CONN_STATUS_OK)
+ goto out_err1;
rsp->StructureSize = cpu_to_le16(16);
out_err1:
/*
@@ -2099,9 +2108,6 @@ int smb2_tree_connect(struct ksmbd_work *work)
rc = ksmbd_iov_pin_rsp(work, rsp, sizeof(struct smb2_tree_connect_rsp));
if (rc) {
if (status.ret == KSMBD_TREE_CONN_STATUS_OK) {
- down_write(&sess->tree_conns_lock);
- status.tree_conn->t_state = TREE_DISCONNECTED;
- up_write(&sess->tree_conns_lock);
ksmbd_tree_conn_disconnect(sess, status.tree_conn);
status.tree_conn = NULL;
}
@@ -2142,6 +2148,9 @@ int smb2_tree_connect(struct ksmbd_work *work)
if (status.ret != KSMBD_TREE_CONN_STATUS_OK)
smb2_set_err_rsp(work);
+ if (tree_conn)
+ ksmbd_tree_connect_put(tree_conn);
+
return rc;
}
@@ -2245,17 +2254,6 @@ int smb2_tree_disconnect(struct ksmbd_work *work)
ksmbd_close_tree_conn_fds(work);
- down_write(&sess->tree_conns_lock);
- if (tcon->t_state == TREE_DISCONNECTED) {
- up_write(&sess->tree_conns_lock);
- rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED;
- err = -ENOENT;
- goto err_out;
- }
-
- tcon->t_state = TREE_DISCONNECTED;
- up_write(&sess->tree_conns_lock);
-
err = ksmbd_tree_conn_disconnect(sess, tcon);
if (err) {
rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED;
@@ -2818,7 +2816,7 @@ static int parse_durable_handle_context(struct ksmbd_work *work,
switch (dh_idx) {
case DURABLE_RECONN_V2:
{
- struct create_durable_reconn_v2_req *recon_v2;
+ struct create_durable_handle_reconnect_v2 *recon_v2;
if (dh_info->type == DURABLE_RECONN ||
dh_info->type == DURABLE_REQ_V2) {
@@ -2828,13 +2826,13 @@ static int parse_durable_handle_context(struct ksmbd_work *work,
if (le16_to_cpu(context->DataOffset) +
le32_to_cpu(context->DataLength) <
- sizeof(struct create_durable_reconn_v2_req)) {
+ sizeof(struct create_durable_handle_reconnect_v2)) {
err = -EINVAL;
goto out;
}
- recon_v2 = (struct create_durable_reconn_v2_req *)context;
- persistent_id = recon_v2->Fid.PersistentFileId;
+ recon_v2 = (struct create_durable_handle_reconnect_v2 *)context;
+ persistent_id = recon_v2->dcontext.Fid.PersistentFileId;
dh_info->fp = ksmbd_lookup_durable_fd(persistent_id);
if (!dh_info->fp) {
ksmbd_debug(SMB, "Failed to get durable handle state\n");
@@ -2842,7 +2840,7 @@ static int parse_durable_handle_context(struct ksmbd_work *work,
goto out;
}
- if (memcmp(dh_info->fp->create_guid, recon_v2->CreateGuid,
+ if (memcmp(dh_info->fp->create_guid, recon_v2->dcontext.CreateGuid,
SMB2_CREATE_GUID_SIZE)) {
err = -EBADF;
ksmbd_put_durable_fd(dh_info->fp);
@@ -2908,7 +2906,7 @@ static int parse_durable_handle_context(struct ksmbd_work *work,
durable_v2_blob =
(struct create_durable_req_v2 *)context;
ksmbd_debug(SMB, "Request for durable v2 open\n");
- dh_info->fp = ksmbd_lookup_fd_cguid(durable_v2_blob->CreateGuid);
+ dh_info->fp = ksmbd_lookup_fd_cguid(durable_v2_blob->dcontext.CreateGuid);
if (dh_info->fp) {
if (!memcmp(conn->ClientGUID, dh_info->fp->client_guid,
SMB2_CLIENT_GUID_SIZE)) {
@@ -2933,11 +2931,11 @@ static int parse_durable_handle_context(struct ksmbd_work *work,
if ((lc && (lc->req_state & SMB2_LEASE_HANDLE_CACHING_LE)) ||
req_op_level == SMB2_OPLOCK_LEVEL_BATCH) {
dh_info->CreateGuid =
- durable_v2_blob->CreateGuid;
+ durable_v2_blob->dcontext.CreateGuid;
dh_info->persistent =
- le32_to_cpu(durable_v2_blob->Flags);
+ le32_to_cpu(durable_v2_blob->dcontext.Flags);
dh_info->timeout =
- le32_to_cpu(durable_v2_blob->Timeout);
+ le32_to_cpu(durable_v2_blob->dcontext.Timeout);
dh_info->type = dh_idx;
}
break;
@@ -3178,6 +3176,14 @@ int smb2_open(struct ksmbd_work *work)
req->CreateOptions = ~(FILE_NO_COMPRESSION_LE);
}
}
+
+ /* MS-FSA 2.1.5.1: delete-on-close requires DELETE access */
+ if (req->CreateOptions & FILE_DELETE_ON_CLOSE_LE &&
+ !(smb_map_generic_desired_access(req->DesiredAccess) &
+ FILE_DELETE_LE)) {
+ rc = -EINVAL;
+ goto err_out2;
+ }
}
if (le32_to_cpu(req->CreateDisposition) >
@@ -3357,6 +3363,13 @@ int smb2_open(struct ksmbd_work *work)
maximal_access = daccess;
}
+ /* MAXIMUM_ALLOWED may not have granted the requested DELETE */
+ if (req->CreateOptions & FILE_DELETE_ON_CLOSE_LE &&
+ !(daccess & FILE_DELETE_LE)) {
+ rc = -EACCES;
+ goto err_out;
+ }
+
open_flags = smb2_create_open_flags(file_present, daccess,
req->CreateDisposition,
&may_flags,
@@ -3965,15 +3978,15 @@ static int readdir_info_level_struct_sz(int info_level)
{
switch (info_level) {
case FILE_FULL_DIRECTORY_INFORMATION:
- return sizeof(struct file_full_directory_info);
+ return sizeof(FILE_FULL_DIRECTORY_INFO);
case FILE_BOTH_DIRECTORY_INFORMATION:
- return sizeof(struct file_both_directory_info);
+ return sizeof(FILE_BOTH_DIRECTORY_INFO);
case FILE_DIRECTORY_INFORMATION:
- return sizeof(struct file_directory_info);
+ return sizeof(FILE_DIRECTORY_INFO);
case FILE_NAMES_INFORMATION:
return sizeof(struct file_names_info);
case FILEID_FULL_DIRECTORY_INFORMATION:
- return sizeof(struct file_id_full_dir_info);
+ return sizeof(FILE_ID_FULL_DIR_INFO);
case FILEID_BOTH_DIRECTORY_INFORMATION:
return sizeof(struct file_id_both_directory_info);
case SMB_FIND_FILE_POSIX_INFO:
@@ -3988,9 +4001,9 @@ static int dentry_name(struct ksmbd_dir_info *d_info, int info_level)
switch (info_level) {
case FILE_FULL_DIRECTORY_INFORMATION:
{
- struct file_full_directory_info *ffdinfo;
+ FILE_FULL_DIRECTORY_INFO *ffdinfo;
- ffdinfo = (struct file_full_directory_info *)d_info->rptr;
+ ffdinfo = (FILE_FULL_DIRECTORY_INFO *)d_info->rptr;
d_info->rptr += le32_to_cpu(ffdinfo->NextEntryOffset);
d_info->name = ffdinfo->FileName;
d_info->name_len = le32_to_cpu(ffdinfo->FileNameLength);
@@ -3998,9 +4011,9 @@ static int dentry_name(struct ksmbd_dir_info *d_info, int info_level)
}
case FILE_BOTH_DIRECTORY_INFORMATION:
{
- struct file_both_directory_info *fbdinfo;
+ FILE_BOTH_DIRECTORY_INFO *fbdinfo;
- fbdinfo = (struct file_both_directory_info *)d_info->rptr;
+ fbdinfo = (FILE_BOTH_DIRECTORY_INFO *)d_info->rptr;
d_info->rptr += le32_to_cpu(fbdinfo->NextEntryOffset);
d_info->name = fbdinfo->FileName;
d_info->name_len = le32_to_cpu(fbdinfo->FileNameLength);
@@ -4008,9 +4021,9 @@ static int dentry_name(struct ksmbd_dir_info *d_info, int info_level)
}
case FILE_DIRECTORY_INFORMATION:
{
- struct file_directory_info *fdinfo;
+ FILE_DIRECTORY_INFO *fdinfo;
- fdinfo = (struct file_directory_info *)d_info->rptr;
+ fdinfo = (FILE_DIRECTORY_INFO *)d_info->rptr;
d_info->rptr += le32_to_cpu(fdinfo->NextEntryOffset);
d_info->name = fdinfo->FileName;
d_info->name_len = le32_to_cpu(fdinfo->FileNameLength);
@@ -4028,9 +4041,9 @@ static int dentry_name(struct ksmbd_dir_info *d_info, int info_level)
}
case FILEID_FULL_DIRECTORY_INFORMATION:
{
- struct file_id_full_dir_info *dinfo;
+ FILE_ID_FULL_DIR_INFO *dinfo;
- dinfo = (struct file_id_full_dir_info *)d_info->rptr;
+ dinfo = (FILE_ID_FULL_DIR_INFO *)d_info->rptr;
d_info->rptr += le32_to_cpu(dinfo->NextEntryOffset);
d_info->name = dinfo->FileName;
d_info->name_len = le32_to_cpu(dinfo->FileNameLength);
@@ -4113,9 +4126,9 @@ static int smb2_populate_readdir_entry(struct ksmbd_conn *conn, int info_level,
switch (info_level) {
case FILE_FULL_DIRECTORY_INFORMATION:
{
- struct file_full_directory_info *ffdinfo;
+ FILE_FULL_DIRECTORY_INFO *ffdinfo;
- ffdinfo = (struct file_full_directory_info *)kstat;
+ ffdinfo = (FILE_FULL_DIRECTORY_INFO *)kstat;
ffdinfo->FileNameLength = cpu_to_le32(conv_len);
ffdinfo->EaSize =
smb2_get_reparse_tag_special_file(ksmbd_kstat->kstat->mode);
@@ -4129,9 +4142,9 @@ static int smb2_populate_readdir_entry(struct ksmbd_conn *conn, int info_level,
}
case FILE_BOTH_DIRECTORY_INFORMATION:
{
- struct file_both_directory_info *fbdinfo;
+ FILE_BOTH_DIRECTORY_INFO *fbdinfo;
- fbdinfo = (struct file_both_directory_info *)kstat;
+ fbdinfo = (FILE_BOTH_DIRECTORY_INFO *)kstat;
fbdinfo->FileNameLength = cpu_to_le32(conv_len);
fbdinfo->EaSize =
smb2_get_reparse_tag_special_file(ksmbd_kstat->kstat->mode);
@@ -4147,9 +4160,9 @@ static int smb2_populate_readdir_entry(struct ksmbd_conn *conn, int info_level,
}
case FILE_DIRECTORY_INFORMATION:
{
- struct file_directory_info *fdinfo;
+ FILE_DIRECTORY_INFO *fdinfo;
- fdinfo = (struct file_directory_info *)kstat;
+ fdinfo = (FILE_DIRECTORY_INFO *)kstat;
fdinfo->FileNameLength = cpu_to_le32(conv_len);
if (d_info->hide_dot_file && d_info->name[0] == '.')
fdinfo->ExtFileAttributes |= FILE_ATTRIBUTE_HIDDEN_LE;
@@ -4169,9 +4182,9 @@ static int smb2_populate_readdir_entry(struct ksmbd_conn *conn, int info_level,
}
case FILEID_FULL_DIRECTORY_INFORMATION:
{
- struct file_id_full_dir_info *dinfo;
+ FILE_ID_FULL_DIR_INFO *dinfo;
- dinfo = (struct file_id_full_dir_info *)kstat;
+ dinfo = (FILE_ID_FULL_DIR_INFO *)kstat;
dinfo->FileNameLength = cpu_to_le32(conv_len);
dinfo->EaSize =
smb2_get_reparse_tag_special_file(ksmbd_kstat->kstat->mode);
@@ -4375,9 +4388,9 @@ static int reserve_populate_dentry(struct ksmbd_dir_info *d_info,
switch (info_level) {
case FILE_FULL_DIRECTORY_INFORMATION:
{
- struct file_full_directory_info *ffdinfo;
+ FILE_FULL_DIRECTORY_INFO *ffdinfo;
- ffdinfo = (struct file_full_directory_info *)d_info->wptr;
+ ffdinfo = (FILE_FULL_DIRECTORY_INFO *)d_info->wptr;
memcpy(ffdinfo->FileName, d_info->name, d_info->name_len);
ffdinfo->FileName[d_info->name_len] = 0x00;
ffdinfo->FileNameLength = cpu_to_le32(d_info->name_len);
@@ -4386,9 +4399,9 @@ static int reserve_populate_dentry(struct ksmbd_dir_info *d_info,
}
case FILE_BOTH_DIRECTORY_INFORMATION:
{
- struct file_both_directory_info *fbdinfo;
+ FILE_BOTH_DIRECTORY_INFO *fbdinfo;
- fbdinfo = (struct file_both_directory_info *)d_info->wptr;
+ fbdinfo = (FILE_BOTH_DIRECTORY_INFO *)d_info->wptr;
memcpy(fbdinfo->FileName, d_info->name, d_info->name_len);
fbdinfo->FileName[d_info->name_len] = 0x00;
fbdinfo->FileNameLength = cpu_to_le32(d_info->name_len);
@@ -4397,9 +4410,9 @@ static int reserve_populate_dentry(struct ksmbd_dir_info *d_info,
}
case FILE_DIRECTORY_INFORMATION:
{
- struct file_directory_info *fdinfo;
+ FILE_DIRECTORY_INFO *fdinfo;
- fdinfo = (struct file_directory_info *)d_info->wptr;
+ fdinfo = (FILE_DIRECTORY_INFO *)d_info->wptr;
memcpy(fdinfo->FileName, d_info->name, d_info->name_len);
fdinfo->FileName[d_info->name_len] = 0x00;
fdinfo->FileNameLength = cpu_to_le32(d_info->name_len);
@@ -4419,9 +4432,9 @@ static int reserve_populate_dentry(struct ksmbd_dir_info *d_info,
}
case FILEID_FULL_DIRECTORY_INFORMATION:
{
- struct file_id_full_dir_info *dinfo;
+ FILE_ID_FULL_DIR_INFO *dinfo;
- dinfo = (struct file_id_full_dir_info *)d_info->wptr;
+ dinfo = (FILE_ID_FULL_DIR_INFO *)d_info->wptr;
memcpy(dinfo->FileName, d_info->name, d_info->name_len);
dinfo->FileName[d_info->name_len] = 0x00;
dinfo->FileNameLength = cpu_to_le32(d_info->name_len);
@@ -4686,7 +4699,7 @@ int smb2_query_dir(struct ksmbd_work *work)
goto err_out;
} else {
no_buf_len:
- ((struct file_directory_info *)
+ ((FILE_DIRECTORY_INFO *)
((char *)rsp->Buffer + d_info.last_entry_offset))
->NextEntryOffset = 0;
if (d_info.data_count >= d_info.last_entry_off_align)
@@ -5275,17 +5288,8 @@ static int get_file_stream_info(struct ksmbd_work *work,
streamlen *= 2;
kfree(stream_buf);
file_info->StreamNameLength = cpu_to_le32(streamlen);
- /*
- * stream_name_len is the byte length of the xattr's *name*,
- * not its value -- same class of bug ksmbd_stream_eof()
- * (smb2pdu.c) already fixes for EndOfFile/AllocationSize on
- * a stream handle; this enumeration path needs the same
- * real xattr value length, not the name length reused as a
- * size.
- */
- slen = ksmbd_vfs_casexattr_len(file_mnt_idmap(fp->filp),
- path->dentry, stream_name,
- strlen(stream_name) + 1, NULL);
+ slen = ksmbd_vfs_xattr_len(file_mnt_idmap(fp->filp),
+ path->dentry, stream_name);
ssize = slen < 0 ? 0 : (loff_t)slen;
file_info->StreamSize = cpu_to_le64(ssize);
file_info->StreamAllocationSize = cpu_to_le64(ssize);
@@ -5341,7 +5345,7 @@ static int get_file_internal_info(struct smb2_query_info_rsp *rsp,
static int get_file_network_open_info(struct smb2_query_info_rsp *rsp,
struct ksmbd_file *fp, void *rsp_org)
{
- struct smb2_file_ntwrk_info *file_info;
+ struct smb2_file_network_open_info *file_info;
struct kstat stat;
u64 time;
int ret;
@@ -5357,7 +5361,7 @@ static int get_file_network_open_info(struct smb2_query_info_rsp *rsp,
if (ret)
return ret;
- file_info = (struct smb2_file_ntwrk_info *)rsp->Buffer;
+ file_info = (struct smb2_file_network_open_info *)rsp->Buffer;
file_info->CreationTime = cpu_to_le64(fp->create_time);
time = ksmbd_UnixTimeToNT(stat.atime);
@@ -5376,7 +5380,7 @@ static int get_file_network_open_info(struct smb2_query_info_rsp *rsp,
}
file_info->Reserved = cpu_to_le32(0);
rsp->OutputBufferLength =
- cpu_to_le32(sizeof(struct smb2_file_ntwrk_info));
+ cpu_to_le32(sizeof(struct smb2_file_network_open_info));
return 0;
}
@@ -5714,9 +5718,9 @@ static int smb2_get_info_filesystem(struct ksmbd_work *work,
switch (fsinfoclass) {
case FS_DEVICE_INFORMATION:
{
- struct filesystem_device_info *info;
+ FILE_SYSTEM_DEVICE_INFO *info;
- info = (struct filesystem_device_info *)rsp->Buffer;
+ info = (FILE_SYSTEM_DEVICE_INFO *)rsp->Buffer;
info->DeviceType = cpu_to_le32(FILE_DEVICE_DISK);
info->DeviceCharacteristics =
@@ -5882,14 +5886,14 @@ static int smb2_get_info_filesystem(struct ksmbd_work *work,
}
case FS_POSIX_INFORMATION:
{
- struct filesystem_posix_info *info;
+ FILE_SYSTEM_POSIX_INFO *info;
if (!work->tcon->posix_extensions) {
pr_err("client doesn't negotiate with SMB3.1.1 POSIX Extensions\n");
path_put(&path);
return -EOPNOTSUPP;
} else {
- info = (struct filesystem_posix_info *)(rsp->Buffer);
+ info = (FILE_SYSTEM_POSIX_INFO *)(rsp->Buffer);
info->OptimalTransferSize = cpu_to_le32(stfs.f_bsize);
info->BlockSize = cpu_to_le32(stfs.f_bsize);
info->TotalBlocks = cpu_to_le64(stfs.f_blocks);
@@ -8263,9 +8267,9 @@ static int fsctl_query_iface_info_ioctl(struct ksmbd_conn *conn,
nii_rsp->Capability = 0;
if (netdev->real_num_tx_queues > 1)
- nii_rsp->Capability |= cpu_to_le32(RSS_CAPABLE);
+ nii_rsp->Capability |= RSS_CAPABLE;
if (ksmbd_rdma_capable_netdev(netdev))
- nii_rsp->Capability |= cpu_to_le32(RDMA_CAPABLE);
+ nii_rsp->Capability |= RDMA_CAPABLE;
nii_rsp->Next = cpu_to_le32(152);
nii_rsp->Reserved = 0;
@@ -8291,13 +8295,13 @@ static int fsctl_query_iface_info_ioctl(struct ksmbd_conn *conn,
if (!ipv4_set) {
struct in_device *idev;
- sockaddr_storage->Family = cpu_to_le16(INTERNETWORK);
+ sockaddr_storage->Family = INTERNETWORK;
sockaddr_storage->addr4.Port = 0;
idev = __in_dev_get_rtnl(netdev);
if (!idev)
continue;
- sockaddr_storage->addr4.IPv4address =
+ sockaddr_storage->addr4.IPv4Address =
idev_ipv4_address(idev);
nbytes += sizeof(struct network_interface_info_ioctl_rsp);
ipv4_set = true;
@@ -8305,9 +8309,9 @@ static int fsctl_query_iface_info_ioctl(struct ksmbd_conn *conn,
} else {
struct inet6_dev *idev6;
struct inet6_ifaddr *ifa;
- __u8 *ipv6_addr = sockaddr_storage->addr6.IPv6address;
+ __u8 *ipv6_addr = sockaddr_storage->addr6.IPv6Address;
- sockaddr_storage->Family = cpu_to_le16(INTERNETWORKV6);
+ sockaddr_storage->Family = INTERNETWORKV6;
sockaddr_storage->addr6.Port = 0;
sockaddr_storage->addr6.FlowInfo = 0;
diff --git a/fs/smb/server/smb2pdu.h b/fs/smb/server/smb2pdu.h
index 23abb70ea5d3..ea07c5552f41 100644
--- a/fs/smb/server/smb2pdu.h
+++ b/fs/smb/server/smb2pdu.h
@@ -66,15 +66,6 @@ struct preauth_integrity_info {
/* Apple Defined Contexts */
#define SMB2_CREATE_AAPL "AAPL"
-struct create_durable_req_v2 {
- struct create_context_hdr ccontext;
- __u8 Name[8];
- __le32 Timeout;
- __le32 Flags;
- __u8 Reserved[8];
- __u8 CreateGuid[16];
-} __packed;
-
#define DURABLE_HANDLE_MAX_TIMEOUT 300000
struct create_durable_reconn_req {
@@ -89,17 +80,6 @@ struct create_durable_reconn_req {
} Data;
} __packed;
-struct create_durable_reconn_v2_req {
- struct create_context_hdr ccontext;
- __u8 Name[8];
- struct {
- __u64 PersistentFileId;
- __u64 VolatileFileId;
- } Fid;
- __u8 CreateGuid[16];
- __le32 Flags;
-} __packed;
-
struct create_alloc_size_req {
struct create_context_hdr ccontext;
__u8 Name[8];
@@ -115,16 +95,6 @@ struct create_durable_rsp {
} Data;
} __packed;
-/* See MS-SMB2 2.2.13.2.11 */
-/* Flags */
-#define SMB2_DHANDLE_FLAG_PERSISTENT 0x00000002
-struct create_durable_v2_rsp {
- struct create_context_hdr ccontext;
- __u8 Name[8];
- __le32 Timeout;
- __le32 Flags;
-} __packed;
-
/* equivalent of the contents of SMB3.1.1 POSIX open context response */
struct create_posix_rsp {
struct create_context_hdr ccontext;
@@ -138,22 +108,6 @@ struct create_posix_rsp {
#define SMB2_0_IOCTL_IS_FSCTL 0x00000001
-struct smb_sockaddr_in {
- __be16 Port;
- __be32 IPv4address;
- __u8 Reserved[8];
-} __packed;
-
-struct smb_sockaddr_in6 {
- __be16 Port;
- __be32 FlowInfo;
- __u8 IPv6address[16];
- __be32 ScopeId;
-} __packed;
-
-#define INTERNETWORK 0x0002
-#define INTERNETWORKV6 0x0017
-
struct sockaddr_storage_rsp {
__le16 Family;
union {
@@ -162,18 +116,6 @@ struct sockaddr_storage_rsp {
};
} __packed;
-#define RSS_CAPABLE 0x00000001
-#define RDMA_CAPABLE 0x00000002
-
-struct network_interface_info_ioctl_rsp {
- __le32 Next; /* next interface. zero if this is last one */
- __le32 IfIndex;
- __le32 Capability; /* RSS or RDMA Capable */
- __le32 Reserved;
- __le64 LinkSpeed;
- char SockAddr_Storage[128];
-} __packed;
-
struct file_object_buf_type1_ioctl_rsp {
__u8 ObjectId[16];
__u8 BirthVolumeId[16];
@@ -282,17 +224,6 @@ struct smb2_file_stream_info {
char StreamName[];
} __packed;
-struct smb2_file_ntwrk_info {
- __le64 CreationTime;
- __le64 LastAccessTime;
- __le64 LastWriteTime;
- __le64 ChangeTime;
- __le64 AllocationSize;
- __le64 EndOfFile;
- __le32 Attributes;
- __le32 Reserved;
-} __packed;
-
struct smb2_file_standard_info {
__le64 AllocationSize;
__le64 EndOfFile;
diff --git a/fs/smb/server/smb_common.h b/fs/smb/server/smb_common.h
index 95062e922de9..d6f72195482f 100644
--- a/fs/smb/server/smb_common.h
+++ b/fs/smb/server/smb_common.h
@@ -12,6 +12,7 @@
#include "nterr.h"
#include "../common/cifsglob.h"
#include "../common/smb2pdu.h"
+#include "../common/fscc.h"
#include "smb2pdu.h"
/* ksmbd's Specific ERRNO */
@@ -50,69 +51,7 @@
#define ATTR_NO_BUFFERING 0x20000000
#define ATTR_WRITE_THROUGH 0x80000000
-/* List of FileSystemAttributes - see 2.5.1 of MS-FSCC */
-#define FILE_SUPPORTS_SPARSE_VDL 0x10000000 /* faster nonsparse extend */
-#define FILE_SUPPORTS_BLOCK_REFCOUNTING 0x08000000 /* allow ioctl dup extents */
-#define FILE_SUPPORT_INTEGRITY_STREAMS 0x04000000
-#define FILE_SUPPORTS_USN_JOURNAL 0x02000000
-#define FILE_SUPPORTS_OPEN_BY_FILE_ID 0x01000000
-#define FILE_SUPPORTS_EXTENDED_ATTRIBUTES 0x00800000
-#define FILE_SUPPORTS_HARD_LINKS 0x00400000
-#define FILE_SUPPORTS_TRANSACTIONS 0x00200000
-#define FILE_SEQUENTIAL_WRITE_ONCE 0x00100000
-#define FILE_READ_ONLY_VOLUME 0x00080000
-#define FILE_NAMED_STREAMS 0x00040000
-#define FILE_SUPPORTS_ENCRYPTION 0x00020000
-#define FILE_SUPPORTS_OBJECT_IDS 0x00010000
-#define FILE_VOLUME_IS_COMPRESSED 0x00008000
-#define FILE_SUPPORTS_POSIX_UNLINK_RENAME 0x00000400
-#define FILE_RETURNS_CLEANUP_RESULT_INFO 0x00000200
-#define FILE_SUPPORTS_REMOTE_STORAGE 0x00000100
-#define FILE_SUPPORTS_REPARSE_POINTS 0x00000080
-#define FILE_SUPPORTS_SPARSE_FILES 0x00000040
-#define FILE_VOLUME_QUOTAS 0x00000020
-#define FILE_FILE_COMPRESSION 0x00000010
-#define FILE_PERSISTENT_ACLS 0x00000008
-#define FILE_UNICODE_ON_DISK 0x00000004
-#define FILE_CASE_PRESERVED_NAMES 0x00000002
-#define FILE_CASE_SENSITIVE_SEARCH 0x00000001
-
-#define FILE_READ_DATA 0x00000001 /* Data can be read from the file */
-#define FILE_WRITE_DATA 0x00000002 /* Data can be written to the file */
-#define FILE_APPEND_DATA 0x00000004 /* Data can be appended to the file */
-#define FILE_READ_EA 0x00000008 /* Extended attributes associated */
-/* with the file can be read */
-#define FILE_WRITE_EA 0x00000010 /* Extended attributes associated */
-/* with the file can be written */
-#define FILE_EXECUTE 0x00000020 /*Data can be read into memory from */
-/* the file using system paging I/O */
-#define FILE_DELETE_CHILD 0x00000040
-#define FILE_READ_ATTRIBUTES 0x00000080 /* Attributes associated with the */
-/* file can be read */
-#define FILE_WRITE_ATTRIBUTES 0x00000100 /* Attributes associated with the */
-/* file can be written */
-#define DELETE 0x00010000 /* The file can be deleted */
-#define READ_CONTROL 0x00020000 /* The access control list and */
-/* ownership associated with the */
-/* file can be read */
-#define WRITE_DAC 0x00040000 /* The access control list and */
-/* ownership associated with the */
-/* file can be written. */
-#define WRITE_OWNER 0x00080000 /* Ownership information associated */
-/* with the file can be written */
-#define SYNCHRONIZE 0x00100000 /* The file handle can waited on to */
-/* synchronize with the completion */
-/* of an input/output request */
-#define GENERIC_ALL 0x10000000
-#define GENERIC_EXECUTE 0x20000000
-#define GENERIC_WRITE 0x40000000
-#define GENERIC_READ 0x80000000
-/* In summary - Relevant file */
-/* access flags from CIFS are */
-/* file_read_data, file_write_data */
-/* file_execute, file_read_attributes*/
-/* write_dac, and delete. */
-
+/* Combinations of file access permission bits */
#define SET_FILE_READ_RIGHTS (FILE_READ_DATA | FILE_READ_EA \
| FILE_READ_ATTRIBUTES \
| DELETE | READ_CONTROL | WRITE_DAC \
@@ -123,14 +62,6 @@
| FILE_WRITE_ATTRIBUTES \
| DELETE | READ_CONTROL | WRITE_DAC \
| WRITE_OWNER | SYNCHRONIZE)
-#define SET_FILE_EXEC_RIGHTS (FILE_READ_EA | FILE_WRITE_EA | FILE_EXECUTE \
- | FILE_READ_ATTRIBUTES \
- | FILE_WRITE_ATTRIBUTES \
- | DELETE | READ_CONTROL | WRITE_DAC \
- | WRITE_OWNER | SYNCHRONIZE)
-
-#define SET_MINIMUM_RIGHTS (FILE_READ_EA | FILE_READ_ATTRIBUTES \
- | READ_CONTROL | SYNCHRONIZE)
/* generic flags for file open */
#define GENERIC_READ_FLAGS (READ_CONTROL | FILE_READ_DATA | \
@@ -162,36 +93,6 @@
#define SMBFLG2_ERR_STATUS cpu_to_le16(0x4000)
#define SMBFLG2_UNICODE cpu_to_le16(0x8000)
-struct smb_hdr {
- __be32 smb_buf_length;
- __u8 Protocol[4];
- __u8 Command;
- union {
- struct {
- __u8 ErrorClass;
- __u8 Reserved;
- __le16 Error;
- } __packed DosError;
- __le32 CifsError;
- } __packed Status;
- __u8 Flags;
- __le16 Flags2; /* note: le */
- __le16 PidHigh;
- union {
- struct {
- __le32 SequenceNumber; /* le */
- __u32 Reserved; /* zero */
- } __packed Sequence;
- __u8 SecuritySignature[8]; /* le */
- } __packed Signature;
- __u8 pad[2];
- __le16 Tid;
- __le16 Pid;
- __le16 Uid;
- __le16 Mid;
- __u8 WordCount;
-} __packed;
-
struct smb_negotiate_req {
struct smb_hdr hdr; /* wct = 0 */
__le16 ByteCount;
@@ -211,11 +112,6 @@ struct filesystem_attribute_info {
__le16 FileSystemName[]; /* do not have to save this - get subset? */
} __packed;
-struct filesystem_device_info {
- __le32 DeviceType;
- __le32 DeviceCharacteristics;
-} __packed; /* device info level 0x104 */
-
struct filesystem_vol_info {
__le64 VolumeCreationTime;
__le32 SerialNumber;
@@ -247,20 +143,6 @@ struct object_id_info {
struct fs_extended_info extended_info;
} __packed;
-struct file_directory_info {
- __le32 NextEntryOffset;
- __u32 FileIndex;
- __le64 CreationTime;
- __le64 LastAccessTime;
- __le64 LastWriteTime;
- __le64 ChangeTime;
- __le64 EndOfFile;
- __le64 AllocationSize;
- __le32 ExtFileAttributes;
- __le32 FileNameLength;
- char FileName[];
-} __packed; /* level 0x101 FF resp data */
-
struct file_names_info {
__le32 NextEntryOffset;
__u32 FileIndex;
@@ -268,39 +150,6 @@ struct file_names_info {
char FileName[];
} __packed; /* level 0xc FF resp data */
-struct file_full_directory_info {
- __le32 NextEntryOffset;
- __u32 FileIndex;
- __le64 CreationTime;
- __le64 LastAccessTime;
- __le64 LastWriteTime;
- __le64 ChangeTime;
- __le64 EndOfFile;
- __le64 AllocationSize;
- __le32 ExtFileAttributes;
- __le32 FileNameLength;
- __le32 EaSize;
- char FileName[];
-} __packed; /* level 0x102 FF resp */
-
-struct file_both_directory_info {
- __le32 NextEntryOffset;
- __u32 FileIndex;
- __le64 CreationTime;
- __le64 LastAccessTime;
- __le64 LastWriteTime;
- __le64 ChangeTime;
- __le64 EndOfFile;
- __le64 AllocationSize;
- __le32 ExtFileAttributes;
- __le32 FileNameLength;
- __le32 EaSize; /* length of the xattrs */
- __u8 ShortNameLength;
- __u8 Reserved;
- __u8 ShortName[24];
- char FileName[];
-} __packed; /* level 0x104 FFrsp data */
-
struct file_id_both_directory_info {
__le32 NextEntryOffset;
__u32 FileIndex;
@@ -321,46 +170,6 @@ struct file_id_both_directory_info {
char FileName[];
} __packed;
-struct file_id_full_dir_info {
- __le32 NextEntryOffset;
- __u32 FileIndex;
- __le64 CreationTime;
- __le64 LastAccessTime;
- __le64 LastWriteTime;
- __le64 ChangeTime;
- __le64 EndOfFile;
- __le64 AllocationSize;
- __le32 ExtFileAttributes;
- __le32 FileNameLength;
- __le32 EaSize; /* EA size */
- __le32 Reserved;
- __le64 UniqueId; /* inode num - le since Samba puts ino in low 32 bit*/
- char FileName[];
-} __packed; /* level 0x105 FF rsp data */
-
-struct filesystem_posix_info {
- /* For undefined recommended transfer size return -1 in that field */
- __le32 OptimalTransferSize; /* bsize on some os, iosize on other os */
- __le32 BlockSize;
- /* The next three fields are in terms of the block size.
- * (above). If block size is unknown, 4096 would be a
- * reasonable block size for a server to report.
- * Note that returning the blocks/blocksavail removes need
- * to make a second call (to QFSInfo level 0x103 to get this info.
- * UserBlockAvail is typically less than or equal to BlocksAvail,
- * if no distinction is made return the same value in each
- */
- __le64 TotalBlocks;
- __le64 BlocksAvail; /* bfree */
- __le64 UserBlocksAvail; /* bavail */
- /* For undefined Node fields or FSID return -1 */
- __le64 TotalFileNodes;
- __le64 FreeFileNodes;
- __le64 FileSysIdentifier; /* fsid */
- /* NB Namelen comes from FILE_SYSTEM_ATTRIBUTE_INFO call */
- /* NB flags can come from FILE_SYSTEM_DEVICE_INFO call */
-} __packed;
-
struct smb_version_ops {
u16 (*get_cmd_val)(struct ksmbd_work *swork);
int (*init_rsp_hdr)(struct ksmbd_work *swork);
diff --git a/fs/smb/server/smbacl.c b/fs/smb/server/smbacl.c
index 33a607e709f1..63f7bccb4d9a 100644
--- a/fs/smb/server/smbacl.c
+++ b/fs/smb/server/smbacl.c
@@ -1606,7 +1606,10 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
else
continue;
- if (id == uid) {
+ /* A group entry grants only to the group's members */
+ if (id == uid &&
+ (pa_entry->e_tag == ACL_USER ||
+ in_group_p(make_kgid(&init_user_ns, id)))) {
mode_to_access_flags(pa_entry->e_perm,
0777,
&access_bits);
diff --git a/fs/smb/server/vfs.c b/fs/smb/server/vfs.c
index 130dd0a4001f..a3db77b3c272 100644
--- a/fs/smb/server/vfs.c
+++ b/fs/smb/server/vfs.c
@@ -950,8 +950,8 @@ ssize_t ksmbd_vfs_listxattr(struct dentry *dentry, char **list)
return size;
}
-static ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap,
- struct dentry *dentry, char *xattr_name)
+ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap,
+ struct dentry *dentry, char *xattr_name)
{
return vfs_getxattr(idmap, dentry, xattr_name, NULL, 0);
}
@@ -1798,7 +1798,7 @@ int ksmbd_vfs_get_dos_attrib_xattr(struct mnt_idmap *idmap,
*/
void *ksmbd_vfs_init_kstat(char **p, struct ksmbd_kstat *ksmbd_kstat)
{
- struct file_directory_info *info = (struct file_directory_info *)(*p);
+ FILE_DIRECTORY_INFO *info = (FILE_DIRECTORY_INFO *)(*p);
struct kstat *kstat = ksmbd_kstat->kstat;
u64 time;
diff --git a/fs/smb/server/vfs.h b/fs/smb/server/vfs.h
index e9a5798d7bae..ca5048ebb130 100644
--- a/fs/smb/server/vfs.h
+++ b/fs/smb/server/vfs.h
@@ -106,6 +106,8 @@ ssize_t ksmbd_vfs_getxattr(struct mnt_idmap *idmap,
struct dentry *dentry,
char *xattr_name,
char **xattr_buf);
+ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap,
+ struct dentry *dentry, char *xattr_name);
ssize_t ksmbd_vfs_casexattr_len(struct mnt_idmap *idmap,
struct dentry *dentry, char *attr_name,
int attr_name_len, char *actual_name);
diff --git a/fs/udf/balloc.c b/fs/udf/balloc.c
index 30cec5600149..75124188e745 100644
--- a/fs/udf/balloc.c
+++ b/fs/udf/balloc.c
@@ -457,6 +457,7 @@ static void udf_table_free_blocks(struct super_block *sb,
int adsize;
+ eloc.partitionReferenceNum = bloc->partitionReferenceNum;
eloc.logicalBlockNum = start;
elen = EXT_RECORDED_ALLOCATED |
(count << sb->s_blocksize_bits);
@@ -532,7 +533,7 @@ static int udf_table_prealloc_blocks(struct super_block *sb,
eloc.logicalBlockNum, elen, first_block);
}
- if (first_block == eloc.logicalBlockNum) {
+ if (ret > 0 && first_block == eloc.logicalBlockNum) {
epos.offset -= adsize;
alloc_count = (elen >> sb->s_blocksize_bits);
@@ -572,7 +573,7 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb,
uint32_t elen, goal_elen = 0;
struct kernel_lb_addr eloc, goal_eloc;
struct extent_position epos, goal_epos;
- int8_t etype;
+ int8_t etype, goal_etype = 0;
struct udf_inode_info *iinfo = UDF_I(table);
int ret = 0;
@@ -623,7 +624,8 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb,
goal_epos.block = epos.block;
goal_epos.offset = epos.offset - adsize;
goal_eloc = eloc;
- goal_elen = (etype << 30) | elen;
+ goal_elen = elen;
+ goal_etype = etype;
}
}
@@ -647,7 +649,8 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb,
goal_elen -= sb->s_blocksize;
if (goal_elen)
- udf_write_aext(table, &goal_epos, &goal_eloc, goal_elen, 1);
+ udf_write_aext(table, &goal_epos, &goal_eloc,
+ (goal_etype << 30) | goal_elen, 1);
else
udf_delete_aext(table, goal_epos, &freed);
brelse(goal_epos.bh);
diff --git a/fs/udf/inode.c b/fs/udf/inode.c
index d00b78523a96..a9538a0f9734 100644
--- a/fs/udf/inode.c
+++ b/fs/udf/inode.c
@@ -1346,6 +1346,26 @@ int udf_setsize(struct inode *inode, loff_t newsize)
return err;
}
+/*
+ * Verify validity of struct deviceSpec on disk. udf_get_extendedattr() has
+ * already verified the generic header and made sure attribute fits in the
+ * inode so we just have to make sure attribute space is large enough for
+ * deviceSpec struct and required impUse information.
+ */
+static bool udf_device_spec_valid(struct deviceSpec *dsea)
+{
+ u32 attr_length, imp_use_length;
+
+ attr_length = le32_to_cpu(dsea->attrLength);
+ imp_use_length = le32_to_cpu(dsea->impUseLength);
+ if (attr_length < sizeof(struct deviceSpec) ||
+ imp_use_length < sizeof(struct regid) ||
+ imp_use_length > attr_length - sizeof(struct deviceSpec))
+ return false;
+
+ return true;
+}
+
/*
* Maximum length of linked list formed by ICB hierarchy. The chosen number is
* arbitrary - just that we hopefully don't limit any real use of rewritten
@@ -1664,13 +1684,12 @@ static int udf_read_inode(struct inode *inode, bool hidden_inode)
if (S_ISCHR(inode->i_mode) || S_ISBLK(inode->i_mode)) {
struct deviceSpec *dsea =
(struct deviceSpec *)udf_get_extendedattr(inode, 12, 1);
- if (dsea) {
- init_special_inode(inode, inode->i_mode,
+
+ if (!dsea || !udf_device_spec_valid(dsea))
+ goto out;
+ init_special_inode(inode, inode->i_mode,
MKDEV(le32_to_cpu(dsea->majorDeviceIdent),
le32_to_cpu(dsea->minorDeviceIdent)));
- /* Developer ID ??? */
- } else
- goto out;
}
ret = 0;
out:
@@ -1818,6 +1837,10 @@ static int udf_update_inode(struct inode *inode, int do_sync)
udf_add_extendedattr(inode,
sizeof(struct deviceSpec) +
sizeof(struct regid), 12, 0x3);
+ if (!dsea) {
+ err = -ENOSPC;
+ goto out_unlock;
+ }
dsea->attrType = cpu_to_le32(12);
dsea->attrSubtype = 1;
dsea->attrLength = cpu_to_le32(
@@ -1825,6 +1848,10 @@ static int udf_update_inode(struct inode *inode, int do_sync)
sizeof(struct regid));
dsea->impUseLength = cpu_to_le32(sizeof(struct regid));
}
+ if (!udf_device_spec_valid(dsea)) {
+ err = -EFSCORRUPTED;
+ goto out_unlock;
+ }
eid = (struct regid *)dsea->impUse;
memset(eid, 0, sizeof(*eid));
strcpy(eid->ident, UDF_ID_DEVELOPER);
@@ -1966,6 +1993,11 @@ static int udf_update_inode(struct inode *inode, int do_sync)
brelse(bh);
return err;
+
+out_unlock:
+ unlock_buffer(bh);
+ brelse(bh);
+ return err;
}
struct inode *__udf_iget(struct super_block *sb, struct kernel_lb_addr *ino,
diff --git a/fs/udf/super.c b/fs/udf/super.c
index 556540f2797a..8d3d758714da 100644
--- a/fs/udf/super.c
+++ b/fs/udf/super.c
@@ -1105,6 +1105,73 @@ static int check_partition_desc(struct super_block *sb,
return 0;
}
+/*
+ * Check the Unallocated Space Table once when it is loaded: the allocator
+ * hands out blocks from the start of each extent and trusts that every
+ * extent covers at least one whole block inside the partition.
+ */
+static int udf_check_unalloc_table(struct super_block *sb,
+ struct inode *table, u32 partition_len)
+{
+ struct extent_position epos = {
+ .block = UDF_I(table)->i_location,
+ .offset = sizeof(struct unallocSpaceEntry),
+ };
+ struct kernel_lb_addr eloc;
+ uint32_t elen, blocks;
+ struct kernel_lb_addr seen_block = {};
+ uint32_t seen_offset = 0;
+ u64 steps = 0, period = 1;
+ int8_t etype;
+ int ret;
+
+ /* The walk below assumes the layout of an Unallocated Space Entry */
+ if (!UDF_I(table)->i_use) {
+ udf_err(sb, "unallocated space table is not an unallocated space entry\n");
+ return -EFSCORRUPTED;
+ }
+
+ while ((ret = udf_next_aext(table, &epos, &eloc, &elen, &etype, 1)) > 0) {
+ blocks = elen >> sb->s_blocksize_bits;
+ if (!blocks || (elen & (sb->s_blocksize - 1)) ||
+ eloc.logicalBlockNum >= partition_len ||
+ blocks > partition_len - eloc.logicalBlockNum) {
+ udf_err(sb, "invalid unallocated space table extent (block %u, length %u)\n",
+ eloc.logicalBlockNum, elen);
+ ret = -EFSCORRUPTED;
+ break;
+ }
+ /*
+ * A chain of allocation extents can lead back to itself, and
+ * then the walk returns the same extents forever. Remember a
+ * position at doubling intervals; a walk that comes back to it
+ * is in a loop.
+ */
+ if (epos.block.logicalBlockNum == seen_block.logicalBlockNum &&
+ epos.block.partitionReferenceNum ==
+ seen_block.partitionReferenceNum &&
+ epos.offset == seen_offset) {
+ udf_err(sb, "unallocated space table loops back on itself\n");
+ ret = -EFSCORRUPTED;
+ break;
+ }
+ if (++steps == period) {
+ seen_block = epos.block;
+ seen_offset = epos.offset;
+ steps = 0;
+ period <<= 1;
+ }
+ if (fatal_signal_pending(current)) {
+ udf_err(sb, "interrupted while checking the unallocated space table\n");
+ ret = -EINTR;
+ break;
+ }
+ cond_resched();
+ }
+ brelse(epos.bh);
+ return ret;
+}
+
static int udf_fill_partdesc_info(struct super_block *sb,
struct partitionDesc *p, int p_index)
{
@@ -1165,6 +1232,16 @@ static int udf_fill_partdesc_info(struct super_block *sb,
p_index);
return PTR_ERR(inode);
}
+ err = udf_check_unalloc_table(sb, inode, map->s_partition_len);
+ if (err) {
+ iput(inode);
+ if (err == -EINTR)
+ return err;
+ if (!sb_rdonly(sb))
+ return -EACCES;
+ UDF_SET_FLAG(sb, UDF_FLAG_RW_INCOMPAT);
+ return 0;
+ }
map->s_uspace.s_table = inode;
map->s_partition_flags |= UDF_PART_FLAG_UNALLOC_TABLE;
udf_debug("unallocSpaceTable (part %d) @ %lu\n",
@@ -2228,8 +2305,9 @@ static int udf_fill_super(struct super_block *sb, struct fs_context *fc)
/*
* EACCES is special - we want to propagate to
* upper layers that we cannot handle RW mount.
+ * EINTR means that a fatal signal is pending.
*/
- if (ret == -EACCES)
+ if (ret == -EACCES || ret == -EINTR)
break;
} else
break;
diff --git a/fs/udf/truncate.c b/fs/udf/truncate.c
index da5a61306390..85bbd6fa2a30 100644
--- a/fs/udf/truncate.c
+++ b/fs/udf/truncate.c
@@ -124,8 +124,8 @@ void udf_discard_prealloc(struct inode *inode)
{
struct extent_position epos = {};
struct extent_position prev_epos = {};
- struct kernel_lb_addr eloc;
- uint32_t elen;
+ struct kernel_lb_addr eloc, tmpeloc;
+ uint32_t elen, tmpelen;
uint64_t lbcount = 0;
int8_t etype = -1;
struct udf_inode_info *iinfo = UDF_I(inode);
@@ -141,7 +141,8 @@ void udf_discard_prealloc(struct inode *inode)
/* Find the last extent in the file */
while (1) {
- ret = udf_next_aext(inode, &epos, &eloc, &elen, &tmpetype, 0);
+ ret = udf_next_aext(inode, &epos, &tmpeloc, &tmpelen,
+ &tmpetype, 0);
if (ret < 0)
goto out;
if (ret == 0)
diff --git a/fs/verity/enable.c b/fs/verity/enable.c
index 95ec42b84797..05fbcae57e5d 100644
--- a/fs/verity/enable.c
+++ b/fs/verity/enable.c
@@ -287,6 +287,14 @@ static int enable_verity(struct file *filp,
* just after the filesystem has successfully enabled verity.
*/
fsverity_set_info(inode, vi);
+
+ /*
+ * Now that verity is enabled, report the signature (if any) to
+ * LSMs. We tolerate that a process racing with this ioctl can
+ * briefly see verity enabled with no signature reported yet.
+ */
+ fsverity_report_valid_sig(inode, desc->signature,
+ le32_to_cpu(desc->sig_size));
}
out:
kfree(params.hashstate);
diff --git a/fs/verity/fsverity_private.h b/fs/verity/fsverity_private.h
index dd20b138d452..0a2ddca3c838 100644
--- a/fs/verity/fsverity_private.h
+++ b/fs/verity/fsverity_private.h
@@ -142,6 +142,8 @@ void __init fsverity_init_info_cache(void);
extern int fsverity_require_signatures;
int fsverity_verify_signature(const struct fsverity_info *vi,
const u8 *signature, size_t sig_size);
+void fsverity_report_valid_sig(const struct inode *inode, const u8 *signature,
+ size_t sig_size);
void __init fsverity_init_signature(void);
#else /* !CONFIG_FS_VERITY_BUILTIN_SIGNATURES */
@@ -152,6 +154,12 @@ fsverity_verify_signature(const struct fsverity_info *vi,
return 0;
}
+static inline void fsverity_report_valid_sig(const struct inode *inode,
+ const u8 *signature,
+ size_t sig_size)
+{
+}
+
static inline void fsverity_init_signature(void)
{
}
diff --git a/fs/verity/measure.c b/fs/verity/measure.c
index 433d963edf3b..5ec36d845008 100644
--- a/fs/verity/measure.c
+++ b/fs/verity/measure.c
@@ -31,6 +31,16 @@ int fsverity_ioctl_measure(struct file *filp, void __user *_uarg)
const struct fsverity_hash_alg *hash_alg;
struct fsverity_digest arg;
+ /*
+ * This ioctl returns a hash of the file contents, so it should require
+ * read permission. Disallow it on ioctl-only (i.e., nonstandard access
+ * mode 3) fds, which can in some cases be acquired without passing all
+ * LSM read-permission checks. Also disallow write-only fds, which can
+ * refer only to non-verity files which would fail with ENODATA anyway.
+ */
+ if (!(filp->f_mode & FMODE_READ))
+ return -EBADF;
+
vi = fsverity_get_info(inode);
if (!vi)
return -ENODATA; /* not a verity file */
diff --git a/fs/verity/open.c b/fs/verity/open.c
index 02e2c14ab3ff..dbc2a25e9158 100644
--- a/fs/verity/open.c
+++ b/fs/verity/open.c
@@ -369,6 +369,15 @@ static int ensure_verity_info(struct inode *inode)
goto out_free_desc;
}
+ /*
+ * Now that the signature (if any) has been verified, report it to LSMs.
+ * Do this before setting the fsverity_info, so that anyone who
+ * concurrently retrieves the fsverity_info for the same inode is
+ * guaranteed to see the LSM's signature status populated.
+ */
+ fsverity_report_valid_sig(inode, desc->signature,
+ le32_to_cpu(desc->sig_size));
+
fsverity_set_info(inode, vi);
err = 0;
out_free_desc:
diff --git a/fs/verity/read_metadata.c b/fs/verity/read_metadata.c
index cba5d6af4e04..cfe9227a41c4 100644
--- a/fs/verity/read_metadata.c
+++ b/fs/verity/read_metadata.c
@@ -157,13 +157,19 @@ int fsverity_ioctl_read_metadata(struct file *filp, const void __user *uarg)
int length;
void __user *buf;
+ /*
+ * This ioctl returns file contents block hashes, so it should require
+ * read permission. Disallow it on ioctl-only (i.e., nonstandard access
+ * mode 3) fds, which can in some cases be acquired without passing all
+ * LSM read-permission checks. Also disallow write-only fds, which can
+ * refer only to non-verity files which would fail with ENODATA anyway.
+ */
+ if (!(filp->f_mode & FMODE_READ))
+ return -EBADF;
+
vi = fsverity_get_info(inode);
if (!vi)
return -ENODATA; /* not a verity file */
- /*
- * Note that we don't have to explicitly check that the file is open for
- * reading, since verity files can only be opened for reading.
- */
if (copy_from_user(&arg, uarg, sizeof(arg)))
return -EFAULT;
diff --git a/fs/verity/signature.c b/fs/verity/signature.c
index 0302a4e506ec..212e873c8499 100644
--- a/fs/verity/signature.c
+++ b/fs/verity/signature.c
@@ -42,13 +42,16 @@ static struct key *fsverity_keyring;
* @sig_size: size of signature in bytes, or 0 if no signature
*
* If the file includes a signature of its fs-verity file digest, verify it
- * against the certificates in the fs-verity keyring. Note that signatures
- * are verified regardless of the state of the 'fsverity_require_signatures'
- * variable and the LSM subsystem relies on this behavior to help enforce
- * file integrity policies. Please discuss changes with the LSM list
- * (thank you!).
+ * against the certificates in the fs-verity keyring.
*
- * Return: 0 on success (signature valid or not required); -errno on failure
+ * Note that signatures are verified regardless of the state of the
+ * 'fsverity_require_signatures' variable, and the LSM subsystem relies on this
+ * behavior to help enforce file integrity policies. (See
+ * fsverity_report_valid_sig(), which assumes that the signature, if present,
+ * was verified here.) Please discuss changes with the LSM list (thank you!).
+ *
+ * Return: 0 if the signature is present and valid, or absent and not required;
+ * -errno on failure
*/
int fsverity_verify_signature(const struct fsverity_info *vi,
const u8 *signature, size_t sig_size)
@@ -111,18 +114,40 @@ int fsverity_verify_signature(const struct fsverity_info *vi,
return err;
}
- err = security_inode_setintegrity(inode,
- LSM_INT_FSVERITY_BUILTINSIG_VALID,
- signature,
- sig_size);
+ return 0;
+}
- if (err) {
+/*
+ * If the signature exists, this tells the LSM that it was verified.
+ *
+ * fsverity_verify_signature() must have already been called with the same
+ * signature, *and* the inode must have verity enabled with the signature
+ * matching it. If called from FS_IOC_ENABLE_VERITY, the latter means that we
+ * must be past the point of no return, i.e. verity won't fail to be enabled at
+ * a later point. This is critical, since LSMs must be told about the signature
+ * only when the verity status is permanent in the in-core inode.
+ */
+void fsverity_report_valid_sig(const struct inode *inode, const u8 *signature,
+ size_t sig_size)
+{
+ int err;
+
+ if (WARN_ON_ONCE(!IS_VERITY(inode)))
+ return;
+
+ if (sig_size == 0)
+ return;
+
+ err = security_inode_setintegrity(
+ inode, LSM_INT_FSVERITY_BUILTINSIG_VALID, signature, sig_size);
+ if (err)
fsverity_err(inode, "Error %d exposing file signature to LSMs",
err);
- return err;
- }
-
- return 0;
+ /*
+ * Besides the logging above, ignore errors here. The LSM hook just
+ * records the signature and doesn't perform access control. And in the
+ * case of FS_IOC_ENABLE_VERITY verity was already enabled.
+ */
}
void __init fsverity_init_signature(void)
diff --git a/fs/xfs/libxfs/xfs_attr_leaf.c b/fs/xfs/libxfs/xfs_attr_leaf.c
index 914f5dfb6463..a6fe8a845079 100644
--- a/fs/xfs/libxfs/xfs_attr_leaf.c
+++ b/fs/xfs/libxfs/xfs_attr_leaf.c
@@ -2948,7 +2948,14 @@ xfs_attr3_leaf_setflag(
#endif
entry = &xfs_attr3_leaf_entryp(leaf)[args->index];
- ASSERT((entry->flags & XFS_ATTR_INCOMPLETE) == 0);
+ if (entry->flags & XFS_ATTR_INCOMPLETE) {
+ if (!(args->op_flags & XFS_DA_OP_RECOVERY)) {
+ ASSERT(args->op_flags & XFS_DA_OP_RECOVERY);
+ return -EFSCORRUPTED;
+ }
+ return 0;
+ }
+
entry->flags |= XFS_ATTR_INCOMPLETE;
xfs_trans_log_buf(args->trans, bp,
XFS_DA_LOGRANGE(leaf, entry, sizeof(*entry)));
diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index bdffade9848f..8fc6ee96e1f5 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -4130,6 +4130,8 @@ xfs_bmapi_finish(
* Returns a negative error code on failure, including -ENOSPC when it could not
* allocate any blocks and -ENOSR when it did allocate blocks to convert a
* delalloc range, but those blocks were before the passed in range.
+ * With XFS_BMAPI_CONVERT_ONLY, it stops at a hole or a delalloc extent instead
+ * of allocating blocks, and returns -ENOSR if it stops before mapping anything.
*/
int
xfs_bmapi_write(
@@ -4189,6 +4191,8 @@ xfs_bmapi_write(
*/
ASSERT((flags & (XFS_BMAPI_PREALLOC | XFS_BMAPI_ZERO)) !=
(XFS_BMAPI_PREALLOC | XFS_BMAPI_ZERO));
+ ASSERT(!(flags & XFS_BMAPI_CONVERT_ONLY) ||
+ (flags & XFS_BMAPI_CONVERT));
if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) ||
XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) {
@@ -4237,6 +4241,9 @@ xfs_bmapi_write(
* that we found, if any.
*/
if (need_alloc || wasdelay) {
+ if (flags & XFS_BMAPI_CONVERT_ONLY)
+ break;
+
bma.eof = eof;
bma.conv = !!(flags & XFS_BMAPI_CONVERT);
bma.wasdel = wasdelay;
@@ -4332,10 +4339,13 @@ xfs_bmapi_write(
*
* To avoid a successful return with *nmap set to 0, return the magic
* -ENOSR error code for this particular case so that the caller can
- * handle it.
+ * handle it. The same goes for XFS_BMAPI_CONVERT_ONLY stopping at a
+ * hole or a delalloc extent at the start of the range, in which case
+ * nothing has been changed.
*/
if (!n) {
- ASSERT(bma.nallocs >= *nmap);
+ ASSERT((flags & XFS_BMAPI_CONVERT_ONLY) ? bma.nallocs == 0 :
+ bma.nallocs >= *nmap);
return -ENOSR;
}
*nmap = n;
diff --git a/fs/xfs/libxfs/xfs_bmap.h b/fs/xfs/libxfs/xfs_bmap.h
index d5f2729305fa..76d25e5d5d9c 100644
--- a/fs/xfs/libxfs/xfs_bmap.h
+++ b/fs/xfs/libxfs/xfs_bmap.h
@@ -90,6 +90,13 @@ struct xfs_bmalloca {
/* Try to align allocations to the extent size hint */
#define XFS_BMAPI_EXTSZALIGN (1u << 11)
+/*
+ * Use with XFS_BMAPI_CONVERT to convert unwritten extents without allocating
+ * blocks. Stop at a hole or a delalloc extent: return the mappings found so
+ * far, or -ENOSR if there are none.
+ */
+#define XFS_BMAPI_CONVERT_ONLY (1u << 12)
+
#define XFS_BMAPI_FLAGS \
{ XFS_BMAPI_ENTIRE, "ENTIRE" }, \
{ XFS_BMAPI_METADATA, "METADATA" }, \
@@ -102,7 +109,8 @@ struct xfs_bmalloca {
{ XFS_BMAPI_COWFORK, "COWFORK" }, \
{ XFS_BMAPI_NODISCARD, "NODISCARD" }, \
{ XFS_BMAPI_NORMAP, "NORMAP" },\
- { XFS_BMAPI_EXTSZALIGN, "EXTSZALIGN" }
+ { XFS_BMAPI_EXTSZALIGN, "EXTSZALIGN" }, \
+ { XFS_BMAPI_CONVERT_ONLY, "CONVERT_ONLY" }
static inline int xfs_bmapi_aflag(int w)
diff --git a/fs/xfs/libxfs/xfs_btree.c b/fs/xfs/libxfs/xfs_btree.c
index dbe9df8c3300..b0824d381e7c 100644
--- a/fs/xfs/libxfs/xfs_btree.c
+++ b/fs/xfs/libxfs/xfs_btree.c
@@ -2567,7 +2567,7 @@ xfs_btree_lshift(
if (XFS_IS_CORRUPT(tcur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
- goto error0;
+ goto error1;
}
error = xfs_btree_decrement(tcur, level, &i);
@@ -2737,7 +2737,7 @@ xfs_btree_rshift(
if (XFS_IS_CORRUPT(tcur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
- goto error0;
+ goto error1;
}
error = xfs_btree_increment(tcur, level, &i);
diff --git a/fs/xfs/scrub/attr.c b/fs/xfs/scrub/attr.c
index 7315d0cb1e2e..68d9666ab201 100644
--- a/fs/xfs/scrub/attr.c
+++ b/fs/xfs/scrub/attr.c
@@ -293,6 +293,78 @@ xchk_xattr_set_map(
return ret;
}
+static unsigned int
+xchk_xattr_local_entry(
+ struct xchk_da_btree *ds,
+ int level,
+ char *buf_end,
+ struct xfs_attr_leafblock *leaf,
+ int idx)
+{
+ struct xfs_attr_leaf_name_local *lentry =
+ xfs_attr3_leaf_name_local(leaf, idx);
+ char *name_end;
+ unsigned int namesize;
+
+ name_end = (char *)lentry + offsetof(typeof(*lentry), nameval[0]);
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ namesize = xfs_attr_leaf_entsize_local(lentry->namelen,
+ be16_to_cpu(lentry->valuelen));
+ name_end = (char *)lentry + namesize;
+ if (lentry->namelen == 0) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ return namesize;
+}
+
+static unsigned int
+xchk_xattr_remote_entry(
+ struct xchk_da_btree *ds,
+ int level,
+ char *buf_end,
+ struct xfs_attr_leafblock *leaf,
+ int idx,
+ struct xfs_attr_leaf_entry *ent)
+{
+ struct xfs_attr_leaf_name_remote *rentry =
+ xfs_attr3_leaf_name_remote(leaf, idx);
+ char *name_end;
+ unsigned int namesize;
+
+ name_end = (char *)rentry + offsetof(typeof(*rentry), name[0]);
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ namesize = xfs_attr_leaf_entsize_remote(rentry->namelen);
+ name_end = (char *)rentry + namesize;
+ if (rentry->namelen == 0) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+ if (rentry->valueblk == 0 && !(ent->flags & XFS_ATTR_INCOMPLETE)) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ return namesize;
+}
+
/*
* Check this leaf entry's relations to everything else.
* Returns the number of bytes used for the name/value data.
@@ -311,9 +383,6 @@ xchk_xattr_entry(
{
struct xfs_mount *mp = ds->state->mp;
struct xchk_xattr_buf *ab = ds->sc->buf;
- char *name_end;
- struct xfs_attr_leaf_name_local *lentry;
- struct xfs_attr_leaf_name_remote *rentry;
unsigned int nameidx;
unsigned int namesize;
@@ -333,25 +402,14 @@ xchk_xattr_entry(
}
/* Check the name information. */
- if (ent->flags & XFS_ATTR_LOCAL) {
- lentry = xfs_attr3_leaf_name_local(leaf, idx);
- namesize = xfs_attr_leaf_entsize_local(lentry->namelen,
- be16_to_cpu(lentry->valuelen));
- name_end = (char *)lentry + namesize;
- if (lentry->namelen == 0)
- xchk_da_set_corrupt(ds, level);
- } else {
- rentry = xfs_attr3_leaf_name_remote(leaf, idx);
- namesize = xfs_attr_leaf_entsize_remote(rentry->namelen);
- name_end = (char *)rentry + namesize;
- if (rentry->namelen == 0)
- xchk_da_set_corrupt(ds, level);
- if (rentry->valueblk == 0 &&
- !(ent->flags & XFS_ATTR_INCOMPLETE))
- xchk_da_set_corrupt(ds, level);
- }
- if (name_end > buf_end)
- xchk_da_set_corrupt(ds, level);
+ if (ent->flags & XFS_ATTR_LOCAL)
+ namesize = xchk_xattr_local_entry(ds, level, buf_end, leaf,
+ idx);
+ else
+ namesize = xchk_xattr_remote_entry(ds, level, buf_end, leaf,
+ idx, ent);
+ if (!namesize)
+ return;
if (!xchk_xattr_set_map(ds->sc, ab->usedmap, nameidx, namesize))
xchk_da_set_corrupt(ds, level);
diff --git a/fs/xfs/scrub/attr_repair.c b/fs/xfs/scrub/attr_repair.c
index ab5bd7a4d4ab..2c573897c0c6 100644
--- a/fs/xfs/scrub/attr_repair.c
+++ b/fs/xfs/scrub/attr_repair.c
@@ -285,6 +285,7 @@ xrep_xattr_salvage_local_attr(
struct xfs_attr_leaf_name_local *lentry)
{
struct xchk_xattr_buf *ab = rx->sc->buf;
+ char *name_end;
unsigned char *value;
unsigned int valuelen;
unsigned int namesize;
@@ -293,10 +294,14 @@ xrep_xattr_salvage_local_attr(
* Decode the leaf local entry format. If something seems wrong, we
* junk the attribute.
*/
+ name_end = (char *)lentry + offsetof(typeof(*lentry), nameval[0]);
+ if (name_end > buf_end)
+ return 0;
value = &lentry->nameval[lentry->namelen];
valuelen = be16_to_cpu(lentry->valuelen);
namesize = xfs_attr_leaf_entsize_local(lentry->namelen, valuelen);
- if ((char *)lentry + namesize > buf_end)
+ name_end = (char *)lentry + namesize;
+ if (name_end > buf_end)
return 0;
if (!xrep_xattr_want_salvage(rx, ent->flags, lentry->nameval,
lentry->namelen, value, valuelen))
@@ -333,8 +338,8 @@ xrep_xattr_salvage_remote_attr(
.attr_filter = ent->flags & XFS_ATTR_NSP_ONDISK_MASK,
.namelen = rentry->namelen,
.name = rentry->name,
- .valuelen = be32_to_cpu(rentry->valuelen),
};
+ char *name_end;
unsigned int namesize;
int error;
@@ -343,9 +348,14 @@ xrep_xattr_salvage_remote_attr(
* junk the attribute. Note that we should never find a zero-length
* remote attribute value.
*/
+ name_end = (char *)rentry + offsetof(typeof(*rentry), name[0]);
+ if (name_end > buf_end)
+ return 0;
namesize = xfs_attr_leaf_entsize_remote(rentry->namelen);
- if ((char *)rentry + namesize > buf_end)
+ name_end = (char *)rentry + namesize;
+ if (name_end > buf_end)
return 0;
+ args.valuelen = be32_to_cpu(rentry->valuelen);
if (args.valuelen == 0 ||
!xrep_xattr_want_salvage(rx, ent->flags, rentry->name,
rentry->namelen, NULL, args.valuelen))
diff --git a/fs/xfs/scrub/dir.c b/fs/xfs/scrub/dir.c
index 7b0a9d2aea3e..495fdf3b519a 100644
--- a/fs/xfs/scrub/dir.c
+++ b/fs/xfs/scrub/dir.c
@@ -313,6 +313,74 @@ xchk_dir_actor(
return error;
}
+/*
+ * Deal with an unused directory entry. Returns a positive number of bytes to
+ * advance in the directory block, zero to stop processing, or -1 if corruption
+ * was found.
+ */
+static int
+xchk_dir_check_unused(
+ struct xchk_da_btree *ds,
+ struct xfs_buf *bp,
+ unsigned int iter_off,
+ unsigned int end,
+ xfs_dablk_t rec_bno)
+{
+ struct xfs_dir2_data_unused *dup = bp->b_addr + iter_off;
+ unsigned int advance;
+
+ if (iter_off + sizeof(*dup) > end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ advance = xfs_dir2_data_unusedsize(be16_to_cpu(dup->length));
+ if (!advance || iter_off + advance > end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ return advance;
+}
+
+/*
+ * Check a directory entry. Returns a positive number of bytes to advance
+ * in the directory block, zero to stop processing, or -1 if corruption was
+ * found.
+ */
+static int
+xchk_dir_check_entry(
+ struct xchk_da_btree *ds,
+ struct xfs_buf *bp,
+ unsigned int iter_off,
+ unsigned int end,
+ struct xfs_dir2_data_entry *dent,
+ xfs_dablk_t rec_bno)
+{
+ struct xfs_dir2_data_entry *dep = bp->b_addr + iter_off;
+ struct xfs_mount *mp = ds->state->mp;
+ unsigned int advance;
+
+ /* must have namelen */
+ advance = offsetof(typeof(*dep), name);
+ if (iter_off + advance >= end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ /* Don't walk off the end of the block. */
+ advance = xfs_dir2_data_entsize(mp, dep->namelen);
+ if (!advance || iter_off + advance > end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ if (dep == dent)
+ return 0;
+
+ return advance;
+}
+
/* Scrub a directory btree record. */
STATIC int
xchk_dir_rec(
@@ -338,6 +406,7 @@ xchk_dir_rec(
xfs_dahash_t hash;
struct xfs_dir3_icleaf_hdr hdr;
unsigned int tag;
+ bool foundit = false;
int error;
ASSERT(blk->magic == XFS_DIR2_LEAF1_MAGIC ||
@@ -382,28 +451,42 @@ xchk_dir_rec(
dent = bp->b_addr + off;
/* Make sure we got a real directory entry. */
- iter_off = geo->data_entry_offset;
end = xfs_dir3_data_end_offset(geo, bp->b_addr);
if (!end) {
xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
goto out_relse;
}
- for (;;) {
- struct xfs_dir2_data_entry *dep = bp->b_addr + iter_off;
+ iter_off = geo->data_entry_offset;
+ while (iter_off < end) {
struct xfs_dir2_data_unused *dup = bp->b_addr + iter_off;
+ int advance;
- if (iter_off >= end) {
+ /* must have freetag */
+ advance = offsetof(typeof(*dup), length);
+ if (iter_off + advance >= end) {
xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
goto out_relse;
}
- if (be16_to_cpu(dup->freetag) == XFS_DIR2_DATA_FREE_TAG) {
- iter_off += be16_to_cpu(dup->length);
- continue;
- }
- if (dep == dent)
+ if (dup->freetag == cpu_to_be16(XFS_DIR2_DATA_FREE_TAG))
+ advance = xchk_dir_check_unused(ds, bp, iter_off, end,
+ rec_bno);
+ else
+ advance = xchk_dir_check_entry(ds, bp, iter_off, end,
+ dent, rec_bno);
+ if (advance < 0)
+ goto out_relse;
+ if (!advance) {
+ foundit = true;
break;
- iter_off += xfs_dir2_data_entsize(mp, dep->namelen);
+ }
+ iter_off += advance;
+ }
+
+ /* Hash tree must point to the exact dirent */
+ if (!foundit) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ goto out_relse;
}
/* Retrieve the entry, sanity check it, and compare hashes. */
@@ -417,6 +500,13 @@ xchk_dir_rec(
goto out_relse;
}
+ /* Name must not overflow end of block */
+ if ((char *)dent->name + dent->namelen >
+ (char *)bp->b_addr + BBTOB(bp->b_length)) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ goto out_relse;
+ }
+
/* Does the directory hash match? */
dname.name = dent->name;
dname.len = dent->namelen;
diff --git a/fs/xfs/scrub/dir_repair.c b/fs/xfs/scrub/dir_repair.c
index 9aafe5c31cbe..c7fd39fd83a4 100644
--- a/fs/xfs/scrub/dir_repair.c
+++ b/fs/xfs/scrub/dir_repair.c
@@ -462,6 +462,63 @@ xrep_dir_salvage_data_entry(
return xrep_dir_salvage_entry(rd, dep->name, dep->namelen, ino);
}
+/*
+ * Deal with an unused directory entry. Returns a positive number of bytes to
+ * advance in the directory block, or zero to stop processing.
+ */
+static int
+xrep_dir_recover_unused(
+ struct xfs_buf *bp,
+ unsigned int offset,
+ unsigned int end)
+{
+ struct xfs_dir2_data_unused *dup = bp->b_addr + offset;
+ unsigned int advance;
+
+ if (offset + sizeof(*dup) > end)
+ return 0;
+
+ advance = xfs_dir2_data_unusedsize(be16_to_cpu(dup->length));
+ if (!advance || offset + advance > end)
+ return 0;
+
+ return advance;
+}
+
+/*
+ * Salvage a directory entry. Returns a positive number of bytes to advance
+ * in the directory block, zero to stop processing, or a negative errno.
+ */
+static int
+xrep_dir_recover_entry(
+ struct xrep_dir *rd,
+ struct xfs_buf *bp,
+ unsigned int offset,
+ unsigned int end)
+{
+ struct xfs_dir2_data_entry *dep = bp->b_addr + offset;
+ unsigned int advance;
+ int error;
+
+ /* must have namelen */
+ advance = offsetof(typeof(*dep), name);
+ if (offset + advance >= end)
+ return 0;
+
+ /* Don't walk off the end of the block. */
+ advance = xfs_dir2_data_entsize(rd->sc->mp, dep->namelen);
+ if (!advance || offset + advance > end)
+ return 0;
+
+ /* Ok, let's save this entry. */
+ error = xrep_dir_salvage_data_entry(rd, dep);
+ if (error < 0)
+ return error;
+
+ return advance;
+}
+
+
/* Try to recover block/data format directory entries. */
STATIC int
xrep_dir_recover_data(
@@ -483,33 +540,25 @@ xrep_dir_recover_data(
while (offset < end) {
struct xfs_dir2_data_unused *dup = bp->b_addr + offset;
- struct xfs_dir2_data_entry *dep = bp->b_addr + offset;
- unsigned int advance;
+ int advance;
if (xchk_should_terminate(rd->sc, &error))
return error;
- /* Skip unused entries. */
- if (be16_to_cpu(dup->freetag) == XFS_DIR2_DATA_FREE_TAG) {
- if (!dup->length)
- break;
- offset += be16_to_cpu(dup->length);
- continue;
- }
+ /* must have freetag */
+ advance = offsetof(typeof(*dup), length);
+ if (offset + advance >= end)
+ break;
- /* Don't walk off the end of the block. */
- advance = xfs_dir2_data_entsize(rd->sc->mp, dep->namelen);
+ if (dup->freetag == cpu_to_be16(XFS_DIR2_DATA_FREE_TAG))
+ advance = xrep_dir_recover_unused(bp, offset, end);
+ else
+ advance = xrep_dir_recover_entry(rd, bp, offset, end);
+ if (advance < 0)
+ return advance;
if (!advance)
break;
offset += advance;
- if (offset > end)
- break;
-
- /* Ok, let's save this entry. */
- error = xrep_dir_salvage_data_entry(rd, dep);
- if (error)
- return error;
-
}
return 0;
@@ -532,6 +581,19 @@ xrep_dir_recover_sf(
hdr = ifp->if_data;
end = (unsigned char *)ifp->if_data + ifp->if_bytes;
+ /* sf header must be big enough for count/i8count */
+ if (!hdr || ifp->if_bytes < 2)
+ return 0;
+
+ /* and large enough for the parent inumber */
+ if (hdr->i8count) {
+ if (ifp->if_bytes < offsetof(struct xfs_dir2_sf_hdr, parent[8]))
+ return 0;
+ } else {
+ if (ifp->if_bytes < offsetof(struct xfs_dir2_sf_hdr, parent[4]))
+ return 0;
+ }
+
ino = xfs_dir2_sf_get_parent_ino(hdr);
trace_xrep_dir_salvaged_parent(rd->sc->ip, ino);
diff --git a/fs/xfs/scrub/quota_repair.c b/fs/xfs/scrub/quota_repair.c
index baf77a158eac..44806ce8c0bf 100644
--- a/fs/xfs/scrub/quota_repair.c
+++ b/fs/xfs/scrub/quota_repair.c
@@ -299,7 +299,6 @@ xrep_quota_block(
xfs_dqid_t id)
{
struct xfs_dqblk *dqblk;
- struct xfs_disk_dquot *ddq;
struct xfs_quotainfo *qi = sc->mp->m_quotainfo;
struct xfs_def_quota *defq = xfs_get_defquota(qi, dqtype);
struct xfs_buf *bp = NULL;
@@ -321,14 +320,21 @@ xrep_quota_block(
break;
case 0:
dqblk = bp->b_addr;
- ddq = &dqblk[0].dd_diskdq;
+ error = 0;
/*
* If there's nothing that would impede a dqiterate, we're
* done.
*/
- if ((ddq->d_type & XFS_DQTYPE_REC_MASK) == dqtype &&
- id == be32_to_cpu(ddq->d_id)) {
+ for (i = 0; i < qi->qi_dqperchunk; i++, dqblk++) {
+ struct xfs_disk_dquot *ddq = &dqblk->dd_diskdq;
+
+ if ((ddq->d_type & XFS_DQTYPE_REC_MASK) != dqtype ||
+ id + i != be32_to_cpu(ddq->d_id))
+ error++;
+ }
+
+ if (!error) {
xfs_trans_brelse(sc->tp, bp);
return 0;
}
@@ -341,7 +347,9 @@ xrep_quota_block(
dqblk = bp->b_addr;
bp->b_ops = &xfs_dquot_buf_ops;
for (i = 0; i < qi->qi_dqperchunk; i++, dqblk++) {
- ddq = &dqblk->dd_diskdq;
+ struct xfs_disk_dquot *ddq = &dqblk->dd_diskdq;
+ bool was_bigtime =
+ !!(ddq->d_type & XFS_DQTYPE_BIGTIME);
trace_xrep_disk_dquot(sc->mp, dqtype, id + i);
@@ -350,8 +358,21 @@ xrep_quota_block(
ddq->d_type = dqtype;
ddq->d_id = cpu_to_be32(id + i);
- if (xfs_has_bigtime(sc->mp) && ddq->d_id)
+ if (xfs_has_bigtime(sc->mp) && ddq->d_id) {
+ /*
+ * If something was broken with this dquot and it was a
+ * non-bigtime dquot, we'll grant everyone a fresh
+ * grace period by zeroing the timer field rather than
+ * try to interpret what might be garbage.
+ */
+ if (!was_bigtime) {
+ ddq->d_btimer = 0;
+ ddq->d_itimer = 0;
+ ddq->d_rtbtimer = 0;
+ }
+
ddq->d_type |= XFS_DQTYPE_BIGTIME;
+ }
xrep_quota_fix_timer(sc->mp, ddq, ddq->d_blk_softlimit,
ddq->d_bcount, &ddq->d_btimer,
diff --git a/fs/xfs/scrub/xfarray.c b/fs/xfs/scrub/xfarray.c
index ccf7cfb63bbc..5e1b56defabd 100644
--- a/fs/xfs/scrub/xfarray.c
+++ b/fs/xfs/scrub/xfarray.c
@@ -794,6 +794,46 @@ xfarray_sort_scan_done(
si->folio = NULL;
}
+static int
+xfarray_sort_load_folio(
+ struct xfarray_sortinfo *si,
+ xfarray_idx_t idx,
+ loff_t idx_pos)
+{
+ struct folio *folio;
+ loff_t next_pos;
+
+ folio = xfile_get_folio(si->array->xfile, idx_pos, si->array->obj_size,
+ XFILE_ALLOC);
+ if (IS_ERR(folio))
+ return PTR_ERR(folio);
+ si->folio = folio;
+
+ /* No folio? Get the caller to read into the scratchpad. */
+ if (!si->folio)
+ return 0;
+
+ si->first_folio_idx = xfarray_idx(si->array,
+ folio_pos(si->folio) + si->array->obj_size - 1);
+
+ next_pos = folio_next_pos(si->folio);
+ si->last_folio_idx = xfarray_idx(si->array, next_pos - 1);
+ if (xfarray_pos(si->array, si->last_folio_idx + 1) > next_pos)
+ si->last_folio_idx--;
+
+ /*
+ * If this folio still doesn't cover the desired element, it must cross
+ * a folio boundary. Get the caller to read into the scratchpad.
+ */
+ if (idx < si->first_folio_idx || idx > si->last_folio_idx) {
+ xfarray_sort_scan_done(si);
+ return 0;
+ }
+
+ trace_xfarray_sort_scan(si, idx);
+ return 0;
+}
+
/*
* Cache the folio backing the start of the given array element. If the array
* element is contained entirely within the folio, return a pointer to the
@@ -819,33 +859,18 @@ xfarray_sort_scan(
(idx < si->first_folio_idx || idx > si->last_folio_idx))
xfarray_sort_scan_done(si);
- /* Grab the first folio that backs this array element. */
+ /* Grab the folio that backs this array element. */
if (!si->folio) {
- struct folio *folio;
- loff_t next_pos;
-
- folio = xfile_get_folio(si->array->xfile, idx_pos,
- si->array->obj_size, XFILE_ALLOC);
- if (IS_ERR(folio))
- return PTR_ERR(folio);
- si->folio = folio;
-
- si->first_folio_idx = xfarray_idx(si->array,
- folio_pos(si->folio) + si->array->obj_size - 1);
-
- next_pos = folio_pos(si->folio) + folio_size(si->folio);
- si->last_folio_idx = xfarray_idx(si->array, next_pos - 1);
- if (xfarray_pos(si->array, si->last_folio_idx + 1) > next_pos)
- si->last_folio_idx--;
-
- trace_xfarray_sort_scan(si, idx);
+ error = xfarray_sort_load_folio(si, idx, idx_pos);
+ if (error)
+ return error;
}
/*
- * If this folio still doesn't cover the desired element, it must cross
- * a folio boundary. Read into the scratchpad and we're done.
+ * If we don't have a folio mapping the entire array element, read into
+ * the scratchpad and we're done.
*/
- if (idx < si->first_folio_idx || idx > si->last_folio_idx) {
+ if (!si->folio) {
void *temp = xfarray_scratch(si->array);
error = xfile_load(si->array->xfile, temp, si->array->obj_size,
diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
index 9845a5fcf857..4c9afca35fef 100644
--- a/fs/xfs/xfs_aops.c
+++ b/fs/xfs/xfs_aops.c
@@ -271,7 +271,7 @@ xfs_discard_folio(
* folio itself and not the start offset that is passed in.
*/
xfs_bmap_punch_delalloc_range(ip, XFS_DATA_FORK, pos,
- folio_pos(folio) + folio_size(folio), NULL);
+ folio_next_pos(folio), NULL);
}
/*
diff --git a/fs/xfs/xfs_discard.c b/fs/xfs/xfs_discard.c
index 6917de832191..98933c172521 100644
--- a/fs/xfs/xfs_discard.c
+++ b/fs/xfs/xfs_discard.c
@@ -166,6 +166,8 @@ struct xfs_trim_cur {
xfs_agblock_t end;
xfs_extlen_t minlen;
bool by_bno;
+ /* restart point (with count) for the by-length walk */
+ xfs_agblock_t restart_bno;
};
static int
@@ -209,14 +211,15 @@ xfs_trim_gather_extents(
error = xfs_alloc_lookup_le(cur, tcur->start, 0, &i);
if (!error && !i)
error = xfs_alloc_lookup_ge(cur, tcur->start, 0, &i);
- } else if (tcur->start == 0) {
- /* first time through a by-len starts with max length */
- cur = xfs_cntbt_init_cursor(mp, tp, agbp, pag);
- error = xfs_alloc_lookup_ge(cur, 0, tcur->count, &i);
} else {
- /* nth time through a by-len starts where we left off */
+ /*
+ * A by-len walk starts at the last record of the longest
+ * length (restart_bno is NULLAGBLOCK until the first batch
+ * ends) and then where the previous batch stopped.
+ */
cur = xfs_cntbt_init_cursor(mp, tp, agbp, pag);
- error = xfs_alloc_lookup_le(cur, tcur->start, tcur->count, &i);
+ error = xfs_alloc_lookup_le(cur, tcur->restart_bno,
+ tcur->count, &i);
}
if (error)
goto out_del_cursor;
@@ -246,9 +249,15 @@ xfs_trim_gather_extents(
if (--batch <= 0) {
/*
* Update the cursor to point at this extent so we
- * restart the next batch from this extent.
+ * restart the next batch from this extent. The
+ * by-length walk does not visit extents in block
+ * order, so it must keep the start of the range and
+ * restart from restart_bno instead.
*/
- tcur->start = fbno;
+ if (tcur->by_bno)
+ tcur->start = fbno;
+ else
+ tcur->restart_bno = fbno;
tcur->count = flen;
break;
}
@@ -351,6 +360,7 @@ xfs_trim_perag_extents(
.start = start,
.end = end,
.minlen = minlen,
+ .restart_bno = NULLAGBLOCK,
};
int error = 0;
diff --git a/fs/xfs/xfs_export.c b/fs/xfs/xfs_export.c
index 201489d3de08..f64e035cacb5 100644
--- a/fs/xfs/xfs_export.c
+++ b/fs/xfs/xfs_export.c
@@ -244,8 +244,6 @@ const struct export_operations xfs_export_operations = {
.get_parent = xfs_fs_get_parent,
.commit_metadata = xfs_fs_nfs_commit_metadata,
#ifdef CONFIG_EXPORTFS_BLOCK_OPS
- .get_uuid = xfs_fs_get_uuid,
- .map_blocks = xfs_fs_map_blocks,
- .commit_blocks = xfs_fs_commit_blocks,
+ .block_ops = &xfs_export_block_ops,
#endif
};
diff --git a/fs/xfs/xfs_ioctl32.c b/fs/xfs/xfs_ioctl32.c
index b64785dc4354..c2e01fcb6015 100644
--- a/fs/xfs/xfs_ioctl32.c
+++ b/fs/xfs/xfs_ioctl32.c
@@ -123,8 +123,8 @@ xfs_ioctl32_bstat_copyin(
xfs_ioctl32_bstime_copyin(&bstat->bs_atime, &bstat32->bs_atime) ||
xfs_ioctl32_bstime_copyin(&bstat->bs_mtime, &bstat32->bs_mtime) ||
xfs_ioctl32_bstime_copyin(&bstat->bs_ctime, &bstat32->bs_ctime) ||
- get_user(bstat->bs_blocks, &bstat32->bs_size) ||
- get_user(bstat->bs_xflags, &bstat32->bs_size) ||
+ get_user(bstat->bs_blocks, &bstat32->bs_blocks) ||
+ get_user(bstat->bs_xflags, &bstat32->bs_xflags) ||
get_user(bstat->bs_extsize, &bstat32->bs_extsize) ||
get_user(bstat->bs_extents, &bstat32->bs_extents) ||
get_user(bstat->bs_gen, &bstat32->bs_gen) ||
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index a1c88df45dd0..c7dea4242e8b 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -282,6 +282,14 @@ xfs_iomap_write_direct(
ASSERT(count_fsb > 0);
+ /*
+ * xfs_bmapi_write() maps at most XFS_MAX_BMBT_EXTLEN blocks in one
+ * mapping, so don't reserve blocks for more. A pNFS layout range
+ * comes from a client, and its block count may not even fit the
+ * 32-bit reservation. The caller maps the rest with a later call.
+ */
+ count_fsb = min_t(xfs_filblks_t, count_fsb, XFS_MAX_BMBT_EXTLEN);
+
resaligned = xfs_aligned_fsb_count(offset_fsb, count_fsb,
xfs_get_extsz_hint(ip));
if (unlikely(XFS_IS_REALTIME_INODE(ip))) {
diff --git a/fs/xfs/xfs_pnfs.c b/fs/xfs/xfs_pnfs.c
index afe7497012d4..298fdc9e9f1c 100644
--- a/fs/xfs/xfs_pnfs.c
+++ b/fs/xfs/xfs_pnfs.c
@@ -13,6 +13,7 @@
#include "xfs_bmap.h"
#include "xfs_iomap.h"
#include "xfs_pnfs.h"
+#include <linux/exportfs_block.h>
/*
* Ensure that we do not have any outstanding pNFS layouts that can be used by
@@ -45,11 +46,22 @@ xfs_break_leased_layouts(
return error;
}
+static expfs_block_layouts_t
+xfs_fs_layouts_supported(
+ struct super_block *sb)
+{
+ expfs_block_layouts_t supported = EXPFS_BLOCK_IN_BAND_ID;
+
+ if (exportfs_bdev_supports_out_of_band_id(sb->s_bdev))
+ supported |= EXPFS_BLOCK_OUT_OF_BAND_ID;
+ return supported;
+}
+
/*
* Get a unique ID including its location so that the client can identify
* the exported device.
*/
-int
+static int
xfs_fs_get_uuid(
struct super_block *sb,
u8 *buf,
@@ -98,13 +110,21 @@ xfs_fs_map_update_inode(
ip->i_diflags |= XFS_DIFLAG_PREALLOC;
xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
- return xfs_trans_commit(tp);
+ error = xfs_trans_commit(tp);
+ if (error)
+ return error;
+
+ /*
+ * Force the log so that the blocks handed out to the client are
+ * guaranteed to be present even after a server crash.
+ */
+ return xfs_log_force_inode(ip);
}
/*
* Get a layout for the pNFS client.
*/
-int
+static int
xfs_fs_map_blocks(
struct inode *inode,
loff_t offset,
@@ -123,6 +143,7 @@ xfs_fs_map_blocks(
uint lock_flags;
int error = 0;
u64 seq;
+ bool update_inode = false;
if (xfs_is_shutdown(mp))
return -EIO;
@@ -156,10 +177,9 @@ xfs_fs_map_blocks(
if (!write)
limit = max(limit, round_up(i_size_read(inode),
inode->i_sb->s_blocksize));
- if (offset > limit)
+ if (offset < 0 || offset >= limit)
goto out_unlock;
- if (offset > limit - length)
- length = limit - offset;
+ length = min_t(u64, length, limit - offset);
error = filemap_write_and_wait(inode->i_mapping);
if (error)
@@ -191,20 +211,27 @@ xfs_fs_map_blocks(
end_fsb - offset_fsb, 0, &imap, &seq);
if (error)
goto out_unlock;
+ update_inode = true;
+ } else {
+ xfs_iunlock(ip, lock_flags);
/*
- * Ensure the next transaction is committed synchronously so
- * that the blocks allocated and handed out to the client are
- * guaranteed to be present even after a server crash.
+ * Blocks past EOF that were allocated for buffered writes, such
+ * as speculative preallocation, can be freed by
+ * xfs_free_eofblocks() without breaking the layout unless the
+ * inode is marked as preallocated. Mark it before handing them
+ * out for writing, as for blocks allocated above.
*/
+ if (write && !(ip->i_diflags & XFS_DIFLAG_PREALLOC) &&
+ imap.br_startoff + imap.br_blockcount >
+ XFS_B_TO_FSB(mp, XFS_ISIZE(ip)))
+ update_inode = true;
+ }
+
+ if (update_inode) {
error = xfs_fs_map_update_inode(ip);
- if (!error)
- error = xfs_log_force_inode(ip);
if (error)
goto out_unlock;
-
- } else {
- xfs_iunlock(ip, lock_flags);
}
xfs_iunlock(ip, XFS_IOLOCK_EXCL);
@@ -252,7 +279,7 @@ xfs_pnfs_validate_isize(
* to manually flush the cache here similar to what the fsync code path does
* for datasyncs on files that have no dirty metadata.
*/
-int
+static int
xfs_fs_commit_blocks(
struct inode *inode,
struct iomap *maps,
@@ -283,9 +310,7 @@ xfs_fs_commit_blocks(
if (start > size)
continue;
- end = start + maps[i].length;
- if (end > size)
- end = size;
+ end = start + min_t(u64, maps[i].length, size - start);
length = end - start;
if (!length)
@@ -332,3 +357,10 @@ xfs_fs_commit_blocks(
xfs_iunlock(ip, XFS_IOLOCK_EXCL);
return error;
}
+
+const struct exportfs_block_ops xfs_export_block_ops = {
+ .layouts_supported = xfs_fs_layouts_supported,
+ .get_uuid = xfs_fs_get_uuid,
+ .map_blocks = xfs_fs_map_blocks,
+ .commit_blocks = xfs_fs_commit_blocks,
+};
diff --git a/fs/xfs/xfs_pnfs.h b/fs/xfs/xfs_pnfs.h
index 940c6c2ad88c..bf43b2009e4c 100644
--- a/fs/xfs/xfs_pnfs.h
+++ b/fs/xfs/xfs_pnfs.h
@@ -2,13 +2,9 @@
#ifndef _XFS_PNFS_H
#define _XFS_PNFS_H 1
-#ifdef CONFIG_EXPORTFS_BLOCK_OPS
-int xfs_fs_get_uuid(struct super_block *sb, u8 *buf, u32 *len, u64 *offset);
-int xfs_fs_map_blocks(struct inode *inode, loff_t offset, u64 length,
- struct iomap *iomap, bool write, u32 *device_generation);
-int xfs_fs_commit_blocks(struct inode *inode, struct iomap *maps, int nr_maps,
- struct iattr *iattr);
+#include <linux/exportfs_block.h>
+#ifdef CONFIG_EXPORTFS_BLOCK_OPS
int xfs_break_leased_layouts(struct inode *inode, uint *iolock,
bool *did_unlock);
#else
@@ -18,4 +14,7 @@ xfs_break_leased_layouts(struct inode *inode, uint *iolock, bool *did_unlock)
return 0;
}
#endif /* CONFIG_EXPORTFS_BLOCK_OPS */
+
+extern const struct exportfs_block_ops xfs_export_block_ops;
+
#endif /* _XFS_PNFS_H */
diff --git a/include/acpi/cppc_acpi.h b/include/acpi/cppc_acpi.h
index 13fa81504844..382a74430e74 100644
--- a/include/acpi/cppc_acpi.h
+++ b/include/acpi/cppc_acpi.h
@@ -171,6 +171,7 @@ extern int cppc_set_epp(int cpu, u64 epp_val);
extern int cppc_get_auto_act_window(int cpu, u64 *auto_act_window);
extern int cppc_set_auto_act_window(int cpu, u64 auto_act_window);
extern int cppc_get_auto_sel(int cpu, bool *enable);
+bool cppc_auto_sel_is_immutable(int cpu);
extern int cppc_set_auto_sel(int cpu, bool enable);
extern int amd_get_highest_perf(unsigned int cpu, u32 *highest_perf);
extern int amd_get_boost_ratio_numerator(unsigned int cpu, u64 *numerator);
@@ -256,6 +257,11 @@ static inline int cppc_get_auto_sel(int cpu, bool *enable)
{
return -EOPNOTSUPP;
}
+static inline bool cppc_auto_sel_is_immutable(int cpu)
+{
+ return false;
+}
+
static inline int cppc_set_auto_sel(int cpu, bool enable)
{
return -EOPNOTSUPP;
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index 4d679d2a206b..16dd46d37655 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -395,8 +395,7 @@ struct mmu_gather {
void tlb_flush_mmu(struct mmu_gather *tlb);
static inline void __tlb_adjust_range(struct mmu_gather *tlb,
- unsigned long address,
- unsigned int range_size)
+ unsigned long address, unsigned long range_size)
{
tlb->start = min(tlb->start, address);
tlb->end = max(tlb->end, address + range_size);
diff --git a/include/dt-bindings/clock/nuvoton,ma35d1-clk.h b/include/dt-bindings/clock/nuvoton,ma35d1-clk.h
index ba2d70f776a6..764d837d6162 100644
--- a/include/dt-bindings/clock/nuvoton,ma35d1-clk.h
+++ b/include/dt-bindings/clock/nuvoton,ma35d1-clk.h
@@ -248,6 +248,7 @@
#define ADC_GATE 233
#define EADC_DIV 234
#define EADC_GATE 235
-#define CLK_MAX_IDX 236
+#define PCLK3_DIV4096 236
+#define PCLK4_DIV4096 237
#endif /* __DT_BINDINGS_CLOCK_NUVOTON_MA35D1_CLK_H */
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 7310841f4512..080096df87c7 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -186,4 +186,19 @@ static inline void timer_set_offset(struct arch_timer_context *ctxt, u64 offset)
WRITE_ONCE(*ctxt->offset.vm_offset, offset);
}
+/*
+ * CVAL to program when the hardware won't apply the timer's offset, so that
+ * its compare against the raw counter matches the guest's at 'now'. An
+ * expired timer stays expired; one past the wrap never fires.
+ */
+static inline u64 timer_apply_offset(u64 cval, u64 offset, u64 now)
+{
+ u64 hw = cval + offset;
+
+ if (now - offset >= cval)
+ return 0;
+
+ return hw > now ? hw : U64_MAX;
+}
+
#endif
diff --git a/include/linux/assoc_array_priv.h b/include/linux/assoc_array_priv.h
index dca733ef6750..37a9f76e1505 100644
--- a/include/linux/assoc_array_priv.h
+++ b/include/linux/assoc_array_priv.h
@@ -80,7 +80,7 @@ struct assoc_array_edit {
struct assoc_array_ptr **leaf_p;
struct assoc_array_ptr *dead_leaf;
struct assoc_array_ptr *new_meta[3];
- struct assoc_array_ptr *excised_meta[1];
+ struct assoc_array_ptr *excised_meta[2];
struct assoc_array_ptr *excised_subtree;
struct assoc_array_ptr **set_backpointers[ASSOC_ARRAY_FAN_OUT];
struct assoc_array_ptr *set_backpointers_to;
diff --git a/include/linux/backing-dev.h b/include/linux/backing-dev.h
index 3e64f14739dd..065cba5dc111 100644
--- a/include/linux/backing-dev.h
+++ b/include/linux/backing-dev.h
@@ -277,10 +277,11 @@ unlocked_inode_to_wb_begin(struct inode *inode, struct wb_lock_cookie *cookie)
rcu_read_lock();
/*
- * Paired with store_release in inode_switch_wbs_work_fn() and
+ * Paired with a release fence in inode_do_switch_wbs() and
* ensures that we see the new wb if we see cleared I_WB_SWITCH.
*/
- cookie->locked = smp_load_acquire(&inode->i_state) & I_WB_SWITCH;
+ cookie->locked = inode->i_state & I_WB_SWITCH;
+ smp_rmb();
if (unlikely(cookie->locked))
xa_lock_irqsave(&inode->i_mapping->i_pages, cookie->flags);
diff --git a/include/linux/bio.h b/include/linux/bio.h
index fb747b053b98..a5763f11fb13 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -376,6 +376,21 @@ static inline struct bio *bio_alloc(struct block_device *bdev,
void submit_bio(struct bio *bio);
+/**
+ * bio_in_atomic - check if the current context is unsafe for bio completion
+ *
+ * Return: %true in atomic contexts (e.g. hard/soft IRQ, preempt-disabled);
+ * %false when a bio can be safely completed in the current context.
+ */
+static inline bool bio_in_atomic(void)
+{
+ if (IS_ENABLED(CONFIG_PREEMPTION) && rcu_preempt_depth())
+ return true;
+ if (!IS_ENABLED(CONFIG_PREEMPT_COUNT))
+ return true;
+ return !preemptible();
+}
+
extern void bio_endio(struct bio *);
/**
diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h
index b16b88bfbc3e..631bf971efc0 100644
--- a/include/linux/buffer_head.h
+++ b/include/linux/buffer_head.h
@@ -517,7 +517,6 @@ void buffer_init(void);
bool try_to_free_buffers(struct folio *folio);
int inode_has_buffers(struct inode *inode);
void invalidate_inode_buffers(struct inode *inode);
-int remove_inode_buffers(struct inode *inode);
int sync_mapping_buffers(struct address_space *mapping);
void invalidate_bh_lrus(void);
void invalidate_bh_lrus_cpu(void);
@@ -528,9 +527,7 @@ extern int buffer_heads_over_limit;
static inline void buffer_init(void) {}
static inline bool try_to_free_buffers(struct folio *folio) { return true; }
-static inline int inode_has_buffers(struct inode *inode) { return 0; }
static inline void invalidate_inode_buffers(struct inode *inode) {}
-static inline int remove_inode_buffers(struct inode *inode) { return 1; }
static inline int sync_mapping_buffers(struct address_space *mapping) { return 0; }
static inline void invalidate_bh_lrus(void) {}
static inline void invalidate_bh_lrus_cpu(void) {}
diff --git a/include/linux/can/can-ml.h b/include/linux/can/can-ml.h
index 8afa92d15a66..f10444c6115f 100644
--- a/include/linux/can/can-ml.h
+++ b/include/linux/can/can-ml.h
@@ -45,6 +45,7 @@
#include <linux/can.h>
#include <linux/list.h>
#include <linux/netdevice.h>
+#include <linux/spinlock.h>
#define CAN_SFF_RCV_ARRAY_SZ (1 << CAN_SFF_ID_BITS)
#define CAN_EFF_RCV_HASH_BITS 10
@@ -53,6 +54,7 @@
enum { RX_ERR, RX_ALL, RX_FIL, RX_INV, RX_MAX };
struct can_dev_rcv_lists {
+ spinlock_t lock; /* protects the receive lists and entries */
struct hlist_head rx[RX_MAX];
struct hlist_head rx_sff[CAN_SFF_RCV_ARRAY_SZ];
struct hlist_head rx_eff[CAN_EFF_RCV_ARRAY_SZ];
@@ -74,6 +76,7 @@ static inline struct can_ml_priv *can_get_ml_priv(struct net_device *dev)
static inline void can_set_ml_priv(struct net_device *dev,
struct can_ml_priv *ml_priv)
{
+ spin_lock_init(&ml_priv->dev_rcv_lists.lock);
netdev_set_ml_priv(dev, ml_priv, ML_PRIV_CAN);
}
diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h
index 7b2807009155..e53524027b73 100644
--- a/include/linux/cgroup.h
+++ b/include/linux/cgroup.h
@@ -42,6 +42,14 @@ struct kernel_clone_args;
#ifdef CONFIG_CGROUPS
+/*
+ * To avoid confusing the compiler (and generating warnings) with code
+ * that attempts to access what would be a 0-element array (i.e. sized
+ * to a potentially empty array when CGROUP_SUBSYS_COUNT == 0), this
+ * constant expression can be added.
+ */
+#define CGROUP_HAS_SUBSYS_CONFIG (CGROUP_SUBSYS_COUNT > 0)
+
enum css_task_iter_flags {
CSS_TASK_ITER_PROCS = (1U << 0), /* walk only threadgroup leaders */
CSS_TASK_ITER_THREADED = (1U << 1), /* walk all threaded css_sets in the domain */
@@ -76,6 +84,7 @@ enum cgroup_lifetime_events {
extern struct file_system_type cgroup_fs_type;
extern struct cgroup_root cgrp_dfl_root;
extern struct css_set init_css_set;
+extern struct mutex cgroup_mutex;
extern spinlock_t css_set_lock;
extern struct blocking_notifier_head cgroup_lifetime_notifier;
@@ -103,6 +112,8 @@ extern struct blocking_notifier_head cgroup_lifetime_notifier;
#define cgroup_subsys_on_dfl(ss) \
static_branch_likely(&ss ## _on_dfl_key)
+bool cgroup_on_dfl(const struct cgroup *cgrp);
+
bool css_has_online_children(struct cgroup_subsys_state *css);
struct cgroup_subsys_state *css_from_id(int id, struct cgroup_subsys *ss);
struct cgroup_subsys_state *cgroup_e_css(struct cgroup *cgroup,
@@ -273,6 +284,32 @@ void css_task_iter_end(struct css_task_iter *it);
for ((pos) = css_next_descendant_post(NULL, (css)); (pos); \
(pos) = css_next_descendant_post((pos), (css)))
+/* iterate over child cgrps, lock should be held throughout iteration */
+#define cgroup_for_each_live_child(child, cgrp) \
+ list_for_each_entry((child), &(cgrp)->self.children, self.sibling) \
+ if (({ lockdep_assert_held(&cgroup_mutex); \
+ cgroup_is_dead(child); })) \
+ ; \
+ else
+
+/* walk live descendants in pre order */
+#define cgroup_for_each_live_descendant_pre(dsct, d_css, cgrp) \
+ css_for_each_descendant_pre((d_css), cgroup_css((cgrp), NULL)) \
+ if (({ lockdep_assert_held(&cgroup_mutex); \
+ (dsct) = (d_css)->cgroup; \
+ cgroup_is_dead(dsct); })) \
+ ; \
+ else
+
+/* walk live descendants in postorder */
+#define cgroup_for_each_live_descendant_post(dsct, d_css, cgrp) \
+ css_for_each_descendant_post((d_css), cgroup_css((cgrp), NULL)) \
+ if (({ lockdep_assert_held(&cgroup_mutex); \
+ (dsct) = (d_css)->cgroup; \
+ cgroup_is_dead(dsct); })) \
+ ; \
+ else
+
/**
* cgroup_taskset_for_each - iterate cgroup_taskset
* @task: the loop cursor
@@ -335,6 +372,27 @@ static inline u64 cgroup_id(const struct cgroup *cgrp)
return cgrp->kn->id;
}
+/**
+ * cgroup_css - obtain a cgroup's css for the specified subsystem
+ * @cgrp: the cgroup of interest
+ * @ss: the subsystem of interest (%NULL returns @cgrp->self)
+ *
+ * Return @cgrp's css (cgroup_subsys_state) associated with @ss. This
+ * function must be called either under cgroup_mutex or rcu_read_lock() and
+ * the caller is responsible for pinning the returned css if it wants to
+ * keep accessing it outside the said locks. This function may return
+ * %NULL if @cgrp doesn't have @subsys_id enabled.
+ */
+static inline struct cgroup_subsys_state *cgroup_css(struct cgroup *cgrp,
+ struct cgroup_subsys *ss)
+{
+ if (CGROUP_HAS_SUBSYS_CONFIG && ss)
+ return rcu_dereference_check(cgrp->subsys[ss->id],
+ lockdep_is_held(&cgroup_mutex));
+ else
+ return &cgrp->self;
+}
+
/**
* css_is_dying - test whether the specified css is dying
* @css: target css
@@ -371,6 +429,11 @@ static inline bool css_is_self(struct cgroup_subsys_state *css)
return false;
}
+static inline bool cgroup_is_dead(const struct cgroup *cgrp)
+{
+ return !(cgrp->self.flags & CSS_ONLINE);
+}
+
static inline void cgroup_get(struct cgroup *cgrp)
{
css_get(&cgrp->self);
@@ -386,8 +449,6 @@ static inline void cgroup_put(struct cgroup *cgrp)
css_put(&cgrp->self);
}
-extern struct mutex cgroup_mutex;
-
static inline void cgroup_lock(void)
{
mutex_lock(&cgroup_mutex);
diff --git a/include/linux/damon.h b/include/linux/damon.h
index 656a314ad215..125fbe1c73a9 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -804,6 +804,12 @@ struct damon_ctx {
struct list_head adaptive_targets;
struct list_head schemes;
+/* private: */
+ /*
+ * number of regions after last merge. Read kdamond_split_regions()
+ * for more details.
+ */
+ unsigned long last_nr_regions;
};
static inline struct damon_region *damon_next_region(struct damon_region *r)
diff --git a/include/linux/exportfs.h b/include/linux/exportfs.h
index f0cf2714ec52..dfeadad435a6 100644
--- a/include/linux/exportfs.h
+++ b/include/linux/exportfs.h
@@ -6,9 +6,8 @@
#include <linux/path.h>
struct dentry;
-struct iattr;
+struct exportfs_block_ops;
struct inode;
-struct iomap;
struct super_block;
struct vfsmount;
@@ -201,7 +200,7 @@ struct handle_to_path_ctx {
* @commit_metadata: commit metadata changes to stable storage
*
* See Documentation/filesystems/nfs/exporting.rst for details on how to use
- * this interface correctly.
+ * this interface correctly and the definition of the flags.
*
* encode_fh:
* @encode_fh should store in the file handle fragment @fh (using at most
@@ -252,6 +251,13 @@ struct handle_to_path_ctx {
* commit_metadata:
* @commit_metadata should commit metadata changes to stable storage.
*
+ * @flags:
+ * Allows the filesystem to communicate to nfsd that it may want to do things
+ * differently when dealing with it.
+ *
+ * @block_ops:
+ * Operations for layout grants to block on the underlying device.
+ *
* Locking rules:
* get_parent is called with child->d_inode->i_rwsem down
* get_name is not (which is possibly inconsistent)
@@ -269,12 +275,6 @@ struct export_operations {
struct dentry * (*get_parent)(struct dentry *child);
int (*commit_metadata)(struct inode *inode);
- int (*get_uuid)(struct super_block *sb, u8 *buf, u32 *len, u64 *offset);
- int (*map_blocks)(struct inode *inode, loff_t offset,
- u64 len, struct iomap *iomap,
- bool write, u32 *device_generation);
- int (*commit_blocks)(struct inode *inode, struct iomap *iomaps,
- int nr_iomaps, struct iattr *iattr);
int (*permission)(struct handle_to_path_ctx *ctx, unsigned int oflags);
struct file * (*open)(const struct path *path, unsigned int oflags);
#define EXPORT_OP_NOWCC (0x1) /* don't collect v3 wcc data */
@@ -287,6 +287,10 @@ struct export_operations {
#define EXPORT_OP_FLUSH_ON_CLOSE (0x20) /* fs flushes file data on close */
#define EXPORT_OP_NOLOCKS (0x40) /* no file locking support */
unsigned long flags;
+
+#ifdef CONFIG_EXPORTFS_BLOCK_OPS
+ const struct exportfs_block_ops *block_ops;
+#endif
};
/**
diff --git a/include/linux/exportfs_block.h b/include/linux/exportfs_block.h
new file mode 100644
index 000000000000..92f990d190bb
--- /dev/null
+++ b/include/linux/exportfs_block.h
@@ -0,0 +1,89 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2014-2026 Christoph Hellwig.
+ *
+ * Support for exportfs-based layout grants for direct block device access.
+ */
+#ifndef LINUX_EXPORTFS_BLOCK_H
+#define LINUX_EXPORTFS_BLOCK_H 1
+
+#include <linux/blkdev.h>
+#include <linux/exportfs.h>
+#include <linux/fs.h>
+
+struct iattr;
+struct inode;
+struct iomap;
+struct super_block;
+
+/*
+ * There are the two types of block-style layout support:
+ * - In-band implies a device identified by a unique cookie inside the actual
+ * device address space checked by the ->get_uuid method as used by the pNFS
+ * block layout. This is a bit dangerous and deprecated.
+ * - Out of band implies identification by out of band unique identifiers
+ * specified by the storage protocol, which is much safer and used by the
+ * pNFS SCSI/NVMe layouts.
+ */
+typedef unsigned int __bitwise expfs_block_layouts_t;
+#define EXPFS_BLOCK_FLAG(__bit) \
+ ((__force expfs_block_layouts_t)(1u << __bit))
+#define EXPFS_BLOCK_IN_BAND_ID EXPFS_BLOCK_FLAG(0)
+#define EXPFS_BLOCK_OUT_OF_BAND_ID EXPFS_BLOCK_FLAG(1)
+
+struct exportfs_block_ops {
+ /*
+ * Returns the EXPFS_BLOCK_* bitmap of supported layout types.
+ */
+ expfs_block_layouts_t (*layouts_supported)(struct super_block *sb);
+
+ /*
+ * Get the in-band device unique signature exposed to clients.
+ */
+ int (*get_uuid)(struct super_block *sb, u8 *buf, u32 *len, u64 *offset);
+
+ /*
+ * Map blocks for direct block access.
+ * If @write is %true, also allocate the blocks for the range if needed.
+ */
+ int (*map_blocks)(struct inode *inode, loff_t offset, u64 len,
+ struct iomap *iomap, bool write,
+ u32 *device_generation);
+
+ /*
+ * Commit blocks previously handed out by ->map_blocks and written to by
+ * the client.
+ */
+ int (*commit_blocks)(struct inode *inode, struct iomap *iomaps,
+ int nr_iomaps, struct iattr *iattr);
+};
+
+static inline bool
+exportfs_bdev_supports_out_of_band_id(struct block_device *bdev)
+{
+ return bdev->bd_disk->fops->pr_ops &&
+ bdev->bd_disk->fops->get_unique_id;
+}
+
+#ifdef CONFIG_EXPORTFS_BLOCK_OPS
+static inline expfs_block_layouts_t
+exportfs_layouts_supported(struct super_block *sb)
+{
+ const struct exportfs_block_ops *bops = sb->s_export_op->block_ops;
+
+ if (!bops ||
+ !bops->layouts_supported ||
+ WARN_ON_ONCE(!bops->map_blocks) ||
+ WARN_ON_ONCE(!bops->commit_blocks))
+ return 0;
+ return bops->layouts_supported(sb);
+}
+#else
+static inline expfs_block_layouts_t
+exportfs_layouts_supported(struct super_block *sb)
+{
+ return 0;
+}
+#endif /* CONFIG_EXPORTFS_BLOCK_OPS */
+
+#endif /* LINUX_EXPORTFS_BLOCK_H */
diff --git a/include/linux/fb.h b/include/linux/fb.h
index f622f5f520cf..e50fc8f0801e 100644
--- a/include/linux/fb.h
+++ b/include/linux/fb.h
@@ -631,6 +631,15 @@ static inline void unlock_fb_info(struct fb_info *info)
mutex_unlock(&info->lock);
}
+static inline struct device *dev_of_fbinfo(const struct fb_info *info)
+{
+#ifdef CONFIG_FB_DEVICE
+ return info->dev;
+#else
+ return NULL;
+#endif
+}
+
static inline void __fb_pad_aligned_buffer(u8 *dst, u32 d_pitch,
u8 *src, u32 s_pitch, u32 height)
{
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 349b7fc57db3..09cbcfbf295f 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -759,7 +759,7 @@ enum inode_state_bits {
/* reserved wait address bit 3 */
};
-enum inode_state_flags_t {
+enum inode_state_flags_enum {
I_NEW = (1U << __I_NEW),
I_SYNC = (1U << __I_SYNC),
I_LRU_ISOLATING = (1U << __I_LRU_ISOLATING),
@@ -843,7 +843,7 @@ struct inode {
#endif
/* Misc */
- enum inode_state_flags_t i_state;
+ enum inode_state_flags_enum i_state;
/* 32-bit hole */
struct rw_semaphore i_rwsem;
@@ -904,6 +904,80 @@ struct inode {
void *i_private; /* fs or device private pointer */
} __randomize_layout;
+/*
+ * i_state handling
+ *
+ * We hide all of it behind helpers so that we can validate consumers.
+ */
+static inline enum inode_state_flags_enum inode_state_read_once(struct inode *inode)
+{
+ return READ_ONCE(inode->i_state);
+}
+
+static inline enum inode_state_flags_enum inode_state_read(struct inode *inode)
+{
+ lockdep_assert_held(&inode->i_lock);
+ return inode->i_state;
+}
+
+static inline void inode_state_set_raw(struct inode *inode,
+ enum inode_state_flags_enum flags)
+{
+ WRITE_ONCE(inode->i_state, inode->i_state | flags);
+}
+
+static inline void inode_state_set(struct inode *inode,
+ enum inode_state_flags_enum flags)
+{
+ lockdep_assert_held(&inode->i_lock);
+ inode_state_set_raw(inode, flags);
+}
+
+static inline void inode_state_clear_raw(struct inode *inode,
+ enum inode_state_flags_enum flags)
+{
+ WRITE_ONCE(inode->i_state, inode->i_state & ~flags);
+}
+
+static inline void inode_state_clear(struct inode *inode,
+ enum inode_state_flags_enum flags)
+{
+ lockdep_assert_held(&inode->i_lock);
+ inode_state_clear_raw(inode, flags);
+}
+
+static inline void inode_state_assign_raw(struct inode *inode,
+ enum inode_state_flags_enum flags)
+{
+ WRITE_ONCE(inode->i_state, flags);
+}
+
+static inline void inode_state_assign(struct inode *inode,
+ enum inode_state_flags_enum flags)
+{
+ lockdep_assert_held(&inode->i_lock);
+ inode_state_assign_raw(inode, flags);
+}
+
+static inline void inode_state_replace_raw(struct inode *inode,
+ enum inode_state_flags_enum clearflags,
+ enum inode_state_flags_enum setflags)
+{
+ enum inode_state_flags_enum flags;
+ flags = inode->i_state;
+ flags &= ~clearflags;
+ flags |= setflags;
+ inode_state_assign_raw(inode, flags);
+}
+
+static inline void inode_state_replace(struct inode *inode,
+ enum inode_state_flags_enum clearflags,
+ enum inode_state_flags_enum setflags)
+{
+ lockdep_assert_held(&inode->i_lock);
+ inode_state_replace_raw(inode, clearflags, setflags);
+}
+
static inline void inode_set_cached_link(struct inode *inode, char *link, int linklen)
{
VFS_WARN_ON_INODE(strlen(link) != linklen, inode);
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index ca6d6eb81711..f7a98fc53baf 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -122,10 +122,9 @@ void hugepage_put_subpool(struct hugepage_subpool *spool);
void hugetlb_dup_vma_private(struct vm_area_struct *vma);
void clear_vma_resv_huge_pages(struct vm_area_struct *vma);
-int move_hugetlb_page_tables(struct vm_area_struct *vma,
- struct vm_area_struct *new_vma,
- unsigned long old_addr, unsigned long new_addr,
- unsigned long len);
+unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma,
+ struct vm_area_struct *new_vma, unsigned long old_addr,
+ unsigned long new_addr, unsigned long len);
int copy_hugetlb_page_range(struct mm_struct *, struct mm_struct *,
struct vm_area_struct *, struct vm_area_struct *);
void unmap_hugepage_range(struct vm_area_struct *,
@@ -157,7 +156,8 @@ long hugetlb_unreserve_pages(struct inode *inode, long start, long end,
bool folio_isolate_hugetlb(struct folio *folio, struct list_head *list);
int get_hwpoison_hugetlb_folio(struct folio *folio, bool *hugetlb, bool unpoison);
void folio_putback_hugetlb(struct folio *folio);
-void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio, int reason);
+void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio,
+ enum migrate_reason reason);
void hugetlb_fix_reserve_counts(struct inode *inode);
extern struct mutex *hugetlb_fault_mutex_table;
u32 hugetlb_fault_mutex_hash(struct address_space *mapping, pgoff_t idx);
@@ -340,11 +340,9 @@ static inline int copy_hugetlb_page_range(struct mm_struct *dst,
return 0;
}
-static inline int move_hugetlb_page_tables(struct vm_area_struct *vma,
- struct vm_area_struct *new_vma,
- unsigned long old_addr,
- unsigned long new_addr,
- unsigned long len)
+static inline unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma,
+ struct vm_area_struct *new_vma, unsigned long old_addr,
+ unsigned long new_addr, unsigned long len)
{
BUG();
return 0;
@@ -428,7 +426,7 @@ static inline void folio_putback_hugetlb(struct folio *folio)
}
static inline void move_hugetlb_state(struct folio *old_folio,
- struct folio *new_folio, int reason)
+ struct folio *new_folio, enum migrate_reason reason)
{
}
@@ -963,7 +961,7 @@ static inline gfp_t htlb_modify_alloc_mask(struct hstate *h, gfp_t gfp_mask)
return modified_mask;
}
-static inline bool htlb_allow_alloc_fallback(int reason)
+static inline bool htlb_allow_alloc_fallback(enum migrate_reason reason)
{
bool allowed_fallback = false;
@@ -1255,7 +1253,7 @@ static inline gfp_t htlb_modify_alloc_mask(struct hstate *h, gfp_t gfp_mask)
return 0;
}
-static inline bool htlb_allow_alloc_fallback(int reason)
+static inline bool htlb_allow_alloc_fallback(enum migrate_reason reason)
{
return false;
}
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 706ddda4a60b..fc13c85ea903 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -662,7 +662,8 @@ static inline int mem_cgroup_charge(struct folio *folio, struct mm_struct *mm,
return __mem_cgroup_charge(folio, mm, gfp);
}
-int mem_cgroup_charge_hugetlb(struct folio* folio, gfp_t gfp);
+int mem_cgroup_charge_hugetlb(struct folio *folio, struct mm_struct *mm,
+ gfp_t gfp);
int mem_cgroup_swapin_charge_folio(struct folio *folio, struct mm_struct *mm,
gfp_t gfp, swp_entry_t entry);
@@ -953,6 +954,9 @@ unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx);
unsigned long lruvec_page_state_local(struct lruvec *lruvec,
enum node_stat_item idx);
+void mod_memcg_lruvec_state(struct lruvec *lruvec,
+ enum node_stat_item idx, int val);
+
void mem_cgroup_flush_stats(struct mem_cgroup *memcg);
void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg);
@@ -1172,9 +1176,10 @@ static inline int mem_cgroup_charge(struct folio *folio,
return 0;
}
-static inline int mem_cgroup_charge_hugetlb(struct folio* folio, gfp_t gfp)
+static inline int mem_cgroup_charge_hugetlb(struct folio *folio,
+ struct mm_struct *mm, gfp_t gfp)
{
- return 0;
+ return 0;
}
static inline int mem_cgroup_swapin_charge_folio(struct folio *folio,
@@ -1422,6 +1427,11 @@ static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec,
return node_page_state(lruvec_pgdat(lruvec), idx);
}
+static inline void mod_memcg_lruvec_state(struct lruvec *lruvec,
+ enum node_stat_item idx, int val)
+{
+}
+
static inline void mem_cgroup_flush_stats(struct mem_cgroup *memcg)
{
}
@@ -1869,6 +1879,16 @@ static inline void mem_cgroup_node_filter_allowed(struct mem_cgroup *memcg,
}
#endif /* CONFIG_MEMCG */
+#if defined(CONFIG_MEMCG) && defined(CONFIG_LRU_GEN)
+void mem_cgroup_calculate_protection_path(struct mem_cgroup *root,
+ struct mem_cgroup *memcg);
+#else
+static inline void mem_cgroup_calculate_protection_path(struct mem_cgroup *root,
+ struct mem_cgroup *memcg)
+{
+}
+#endif
+
#if defined(CONFIG_MEMCG) && defined(CONFIG_ZSWAP)
bool obj_cgroup_may_zswap(struct obj_cgroup *objcg);
void obj_cgroup_charge_zswap(struct obj_cgroup *objcg, size_t size);
diff --git a/include/linux/memfd.h b/include/linux/memfd.h
index 6f606d9573c3..97b8a94cd0b3 100644
--- a/include/linux/memfd.h
+++ b/include/linux/memfd.h
@@ -2,8 +2,17 @@
#ifndef __LINUX_MEMFD_H
#define __LINUX_MEMFD_H
+#include <linux/fcntl.h>
#include <linux/file.h>
+static inline int memfd_check_seals_mode(unsigned int seals, umode_t old_mode,
+ umode_t new_mode)
+{
+ if ((seals & F_SEAL_EXEC) && ((old_mode ^ new_mode) & 0111))
+ return -EPERM;
+ return 0;
+}
+
#ifdef CONFIG_MEMFD_CREATE
extern long memfd_fcntl(struct file *file, unsigned int cmd, unsigned int arg);
struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx);
diff --git a/include/linux/migrate.h b/include/linux/migrate.h
index 8061efd89041..4ce64a3e77f4 100644
--- a/include/linux/migrate.h
+++ b/include/linux/migrate.h
@@ -57,8 +57,9 @@ void putback_movable_pages(struct list_head *l);
int migrate_folio(struct address_space *mapping, struct folio *dst,
struct folio *src, enum migrate_mode mode);
int migrate_pages(struct list_head *l, new_folio_t new, free_folio_t free,
- unsigned long private, enum migrate_mode mode, int reason,
- unsigned int *ret_succeeded);
+ unsigned long private, enum migrate_mode mode,
+ enum migrate_reason reason,
+ unsigned int *ret_succeeded);
struct folio *alloc_migration_target(struct folio *src, unsigned long private);
bool isolate_movable_ops_page(struct page *page, isolate_mode_t mode);
bool isolate_folio_to_list(struct folio *folio, struct list_head *list);
@@ -77,7 +78,8 @@ int set_movable_ops(const struct movable_operations *ops, enum pagetype type);
static inline void putback_movable_pages(struct list_head *l) {}
static inline int migrate_pages(struct list_head *l, new_folio_t new,
free_folio_t free, unsigned long private,
- enum migrate_mode mode, int reason, unsigned int *ret_succeeded)
+ enum migrate_mode mode, enum migrate_reason reason,
+ unsigned int *ret_succeeded)
{ return -ENOSYS; }
static inline struct folio *alloc_migration_target(struct folio *src,
unsigned long private)
diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h
index d1094c2d5fb6..c46f44340da7 100644
--- a/include/linux/mmu_notifier.h
+++ b/include/linux/mmu_notifier.h
@@ -444,7 +444,8 @@ mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range)
/*
* This version of mmu_notifier_invalidate_range_start() avoids blocking, but it
* can return an error if a notifier can't proceed without blocking, in which
- * case you're not allowed to modify PTEs in the specified range.
+ * case you're not allowed to modify PTEs in the specified range or call
+ * mmu_notifier_invalidate_range_end().
*
* This is mainly intended for OOM handling.
*/
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index e947af6a3684..7c5ab7ae7b53 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -390,6 +390,12 @@ enum why_no_delegation4 { /* new to v4.1 */
WND4_IS_DIR = 8,
};
+enum stable_how4 {
+ UNSTABLE4 = 0,
+ DATA_SYNC4 = 1,
+ FILE_SYNC4 = 2,
+};
+
enum lock_type4 {
NFS4_UNLOCK_LT = 0,
NFS4_READ_LT = 1,
diff --git a/include/linux/page_owner.h b/include/linux/page_owner.h
index 3328357f6dba..8188ddc5c412 100644
--- a/include/linux/page_owner.h
+++ b/include/linux/page_owner.h
@@ -3,6 +3,7 @@
#define __LINUX_PAGE_OWNER_H
#include <linux/jump_label.h>
+#include <linux/migrate_mode.h>
#ifdef CONFIG_PAGE_OWNER
extern struct static_key_false page_owner_inited;
@@ -14,7 +15,7 @@ extern void __set_page_owner(struct page *page,
extern void __split_page_owner(struct page *page, int old_order,
int new_order);
extern void __folio_copy_owner(struct folio *newfolio, struct folio *old);
-extern void __folio_set_owner_migrate_reason(struct folio *folio, int reason);
+void __folio_set_owner_migrate_reason(struct folio *folio, enum migrate_reason reason);
extern void __dump_page_owner(const struct page *page);
extern void pagetypeinfo_showmixedcount_print(struct seq_file *m,
pg_data_t *pgdat, struct zone *zone);
@@ -43,7 +44,7 @@ static inline void folio_copy_owner(struct folio *newfolio, struct folio *old)
if (static_branch_unlikely(&page_owner_inited))
__folio_copy_owner(newfolio, old);
}
-static inline void folio_set_owner_migrate_reason(struct folio *folio, int reason)
+static inline void folio_set_owner_migrate_reason(struct folio *folio, enum migrate_reason reason)
{
if (static_branch_unlikely(&page_owner_inited))
__folio_set_owner_migrate_reason(folio, reason);
@@ -68,7 +69,7 @@ static inline void split_page_owner(struct page *page, int old_order,
static inline void folio_copy_owner(struct folio *newfolio, struct folio *folio)
{
}
-static inline void folio_set_owner_migrate_reason(struct folio *folio, int reason)
+static inline void folio_set_owner_migrate_reason(struct folio *folio, enum migrate_reason reason)
{
}
static inline void dump_page_owner(const struct page *page)
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 09b581c1d878..e16576e3763a 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -941,6 +941,17 @@ static inline pgoff_t folio_next_index(const struct folio *folio)
return folio->index + folio_nr_pages(folio);
}
+/**
+ * folio_next_pos - Get the file position of the next folio.
+ * @folio: The current folio.
+ *
+ * Return: The position of the folio which follows this folio in the file.
+ */
+static inline loff_t folio_next_pos(const struct folio *folio)
+{
+ return (loff_t)folio_next_index(folio) << PAGE_SHIFT;
+}
+
/**
* folio_file_page - The page for a particular index.
* @folio: The folio which contains this index.
diff --git a/include/linux/panic.h b/include/linux/panic.h
index 6f972a66c13e..2ccea58bea81 100644
--- a/include/linux/panic.h
+++ b/include/linux/panic.h
@@ -13,7 +13,8 @@ __printf(1, 2)
void panic(const char *fmt, ...) __noreturn __cold;
__printf(1, 0)
void vpanic(const char *fmt, va_list args) __noreturn __cold;
-void nmi_panic(struct pt_regs *regs, const char *msg);
+__printf(2, 3)
+void nmi_panic(struct pt_regs *regs, const char *fmt, ...);
void check_panic_on_warn(const char *origin);
extern void oops_enter(void);
extern void oops_exit(void);
diff --git a/include/linux/rculist_nulls.h b/include/linux/rculist_nulls.h
index 89186c499dd4..c26cb83ca071 100644
--- a/include/linux/rculist_nulls.h
+++ b/include/linux/rculist_nulls.h
@@ -52,6 +52,13 @@ static inline void hlist_nulls_del_init_rcu(struct hlist_nulls_node *n)
#define hlist_nulls_next_rcu(node) \
(*((struct hlist_nulls_node __rcu __force **)&(node)->next))
+/**
+ * hlist_nulls_pprev_rcu - returns the dereferenced pprev of @node.
+ * @node: element of the list.
+ */
+#define hlist_nulls_pprev_rcu(node) \
+ (*((struct hlist_nulls_node __rcu __force **)(node)->pprev))
+
/**
* hlist_nulls_del_rcu - deletes entry from hash list without re-initialization
* @n: the element to delete from the hash list.
@@ -152,6 +159,58 @@ static inline void hlist_nulls_add_fake(struct hlist_nulls_node *n)
n->next = (struct hlist_nulls_node *)NULLS_MARKER(NULL);
}
+/**
+ * hlist_nulls_replace_rcu - replace an old entry by a new one
+ * @old: the element to be replaced
+ * @new: the new element to insert
+ *
+ * Description:
+ * Replace the old entry with the new one in a RCU-protected hlist_nulls, while
+ * permitting racing traversals.
+ *
+ * The caller must take whatever precautions are necessary (such as holding
+ * appropriate locks) to avoid racing with another list-mutation primitive, such
+ * as hlist_nulls_add_head_rcu() or hlist_nulls_del_rcu(), running on this same
+ * list. However, it is perfectly legal to run concurrently with the _rcu
+ * list-traversal primitives, such as hlist_nulls_for_each_entry_rcu().
+ */
+static inline void hlist_nulls_replace_rcu(struct hlist_nulls_node *old,
+ struct hlist_nulls_node *new)
+{
+ struct hlist_nulls_node *next = old->next;
+
+ WRITE_ONCE(new->next, next);
+ WRITE_ONCE(new->pprev, old->pprev);
+ rcu_assign_pointer(hlist_nulls_pprev_rcu(new), new);
+ if (!is_a_nulls(next))
+ WRITE_ONCE(next->pprev, &new->next);
+}
+
+/**
+ * hlist_nulls_replace_init_rcu - replace an old entry by a new one and
+ * initialize the old
+ * @old: the element to be replaced
+ * @new: the new element to insert
+ *
+ * Description:
+ * Replace the old entry with the new one in a RCU-protected hlist_nulls, while
+ * permitting racing traversals, and reinitialize the old entry.
+ *
+ * Note: @old must be hashed.
+ *
+ * The caller must take whatever precautions are necessary (such as holding
+ * appropriate locks) to avoid racing with another list-mutation primitive, such
+ * as hlist_nulls_add_head_rcu() or hlist_nulls_del_rcu(), running on this same
+ * list. However, it is perfectly legal to run concurrently with the _rcu
+ * list-traversal primitives, such as hlist_nulls_for_each_entry_rcu().
+ */
+static inline void hlist_nulls_replace_init_rcu(struct hlist_nulls_node *old,
+ struct hlist_nulls_node *new)
+{
+ hlist_nulls_replace_rcu(old, new);
+ WRITE_ONCE(old->pprev, NULL);
+}
+
/**
* hlist_nulls_for_each_entry_rcu - iterate over rcu list of given type
* @tpos: the type * to use as a loop cursor.
diff --git a/include/linux/srcu.h b/include/linux/srcu.h
index ada65b58bc4c..d4fa064600f6 100644
--- a/include/linux/srcu.h
+++ b/include/linux/srcu.h
@@ -18,6 +18,7 @@
#include <linux/mutex.h>
#include <linux/rcupdate.h>
+#include <linux/rculist.h>
#include <linux/workqueue.h>
#include <linux/rcu_segcblist.h>
@@ -68,6 +69,41 @@ void call_srcu(struct srcu_struct *ssp, struct rcu_head *head,
void cleanup_srcu_struct(struct srcu_struct *ssp);
void synchronize_srcu(struct srcu_struct *ssp);
+/**
+ * list_splice_init_srcu - splice an SRCU-protected list into an existing list
+ * @list: the SRCU-protected list to splice
+ * @head: the place in the existing list to splice the first list into
+ * @ssp: the srcu_struct that readers of @list use
+ *
+ * Like list_splice_init_rcu(), but wait for the SRCU readers of @list
+ * instead of RCU ones, so that none of them can follow a next pointer
+ * onto @head.
+ *
+ * Note that this function blocks, and that the caller must prevent any
+ * other updates to @list.
+ */
+static inline void list_splice_init_srcu(struct list_head *list,
+ struct list_head *head,
+ struct srcu_struct *ssp)
+{
+ struct list_head *first = list->next;
+ struct list_head *last = list->prev;
+ struct list_head *next = head->next;
+
+ if (list_empty(list))
+ return;
+
+ INIT_LIST_HEAD_RCU(list);
+ synchronize_srcu(ssp);
+ ASSERT_EXCLUSIVE_ACCESS(*first);
+ ASSERT_EXCLUSIVE_ACCESS(*last);
+
+ last->next = next;
+ rcu_assign_pointer(list_next_rcu(head), first);
+ first->prev = head;
+ next->prev = last;
+}
+
#define SRCU_GET_STATE_COMPLETED 0x1
/**
diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index 3c2b8c355ab3..30344c0d6a9d 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -199,7 +199,8 @@ struct rpc_xprt *rpc_task_get_xprt(struct rpc_clnt *clnt,
int rpcb_create_local(struct net *);
void rpcb_put_local(struct net *);
-int rpcb_register(struct net *, u32, u32, int, unsigned short);
+int rpcb_register(struct net *net, u32 prog, u32 vers, int prot,
+ unsigned short port);
int rpcb_v4_register(struct net *net, const u32 program,
const u32 version,
const struct sockaddr *address,
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index 5506d20857c3..1668804db44d 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -75,6 +75,7 @@ struct svc_serv {
unsigned int sv_max_payload; /* datagram payload size */
unsigned int sv_max_mesg; /* max_payload + 1 page for overheads */
unsigned int sv_xdrsize; /* XDR buffer size */
+ atomic_t sv_rpcb_failures; /* unanswered rpcbind calls */
struct list_head sv_permsocks; /* all permanent sockets */
struct list_head sv_tempsocks; /* all temporary sockets */
int sv_tmpcnt; /* count of temporary "valid" sockets */
@@ -435,6 +436,7 @@ int sunrpc_set_pool_mode(const char *val);
int sunrpc_get_pool_mode(char *val, size_t size);
void svc_rpcb_cleanup(struct svc_serv *serv, struct net *net);
int svc_bind(struct svc_serv *serv, struct net *net);
+unsigned int svc_rpcb_failure_count(struct svc_serv *serv);
struct svc_serv *svc_create(struct svc_program *, unsigned int,
int (*threadfn)(void *data));
bool svc_rqst_replace_page(struct svc_rqst *rqstp,
@@ -450,8 +452,9 @@ int svc_set_num_threads(struct svc_serv *, struct svc_pool *, int);
int svc_pool_stats_open(struct svc_info *si, struct file *file);
void svc_process(struct svc_rqst *rqstp);
void svc_process_bc(struct rpc_rqst *req, struct svc_rqst *rqstp);
-int svc_register(const struct svc_serv *, struct net *, const int,
- const unsigned short, const unsigned short);
+int svc_register(struct svc_serv *serv, struct net *net,
+ const int family, const unsigned short proto,
+ const unsigned short port);
void svc_wake_up(struct svc_serv *);
void svc_reserve(struct svc_rqst *rqstp, int space);
diff --git a/include/linux/writeback.h b/include/linux/writeback.h
index 22dd4adc5667..e1e1231a6830 100644
--- a/include/linux/writeback.h
+++ b/include/linux/writeback.h
@@ -194,6 +194,10 @@ static inline void wait_on_inode(struct inode *inode)
{
wait_var_event(inode_state_wait_address(inode, __I_NEW),
!(READ_ONCE(inode->i_state) & I_NEW));
+ /*
+ * Pairs with routines clearing I_NEW.
+ */
+ smp_rmb();
}
#ifdef CONFIG_CGROUP_WRITEBACK
diff --git a/include/net/espintcp.h b/include/net/espintcp.h
index c70efd704b6d..7cad3c6822df 100644
--- a/include/net/espintcp.h
+++ b/include/net/espintcp.h
@@ -34,7 +34,8 @@ static inline struct espintcp_ctx *espintcp_getctx(const struct sock *sk)
{
const struct inet_connection_sock *icsk = inet_csk(sk);
- /* RCU is only needed for diag */
- return (__force void *)icsk->icsk_ulp_data;
+ /* pairs with smp_wmb() in espintcp_init_sk() */
+ smp_rmb();
+ return (__force void *)READ_ONCE(icsk->icsk_ulp_data);
}
#endif
diff --git a/include/net/netns/can.h b/include/net/netns/can.h
index 48b79f7e6236..54f23681e0d4 100644
--- a/include/net/netns/can.h
+++ b/include/net/netns/can.h
@@ -29,7 +29,7 @@ struct netns_can {
/* receive filters subscribed for 'all' CAN devices */
struct can_dev_rcv_lists *rx_alldev_list;
- spinlock_t rcvlists_lock;
+ spinlock_t rcvlists_lock; /* protects the rcv_entries(_max) stats */
struct timer_list stattimer; /* timer for statistics update */
struct can_pkg_stats *pkg_stats;
struct can_rcv_lists_stats *rcv_lists_stats;
diff --git a/include/net/netrom.h b/include/net/netrom.h
index 2b4685d52bd8..3f6fd2f89434 100644
--- a/include/net/netrom.h
+++ b/include/net/netrom.h
@@ -87,6 +87,19 @@ struct nr_sock {
#define nr_sk(sk) ((struct nr_sock *)(sk))
+/*
+ * Window negotiation, for a CONNREQ received or a CONNACK: use the smaller
+ * of our window and the peer's. A window of 0 would never let a frame
+ * out: take it as 1.
+ */
+static inline void nr_negotiate_window(struct nr_sock *nr, unsigned char window)
+{
+ if (window == 0)
+ window = 1;
+ if (window < nr->window)
+ nr->window = window;
+}
+
struct nr_neigh {
struct hlist_node neigh_node;
ax25_address callsign;
@@ -189,7 +202,7 @@ void nr_loopback_clear(void);
int nr_loopback_queue(struct sk_buff *);
/* nr_out.c */
-void nr_output(struct sock *, struct sk_buff *);
+int nr_output(struct sock *sk, struct sk_buff *skb);
void nr_send_nak_frame(struct sock *);
void nr_kick(struct sock *);
void nr_transmit_buffer(struct sock *, struct sk_buff *);
diff --git a/include/net/rose.h b/include/net/rose.h
index 2fa0331e36f0..30b0809dd0a6 100644
--- a/include/net/rose.h
+++ b/include/net/rose.h
@@ -231,6 +231,8 @@ int rose_rcv_frame(struct sock *sk, struct sk_buff *skb);
int rose_backlog_rcv(struct sock *sk, struct sk_buff *skb);
/* rose_link.c */
+void rose_neigh_init_timers(struct rose_neigh *neigh);
+bool rose_neigh_clear_ax25(struct rose_neigh *neigh, ax25_cb *ax25);
void rose_start_ftimer(struct rose_neigh *);
void rose_stop_ftimer(struct rose_neigh *);
void rose_stop_t0timer(struct rose_neigh *);
@@ -266,6 +268,7 @@ void rose_link_device_down(struct net_device *);
struct net_device *rose_dev_first(void);
struct net_device *rose_dev_get(rose_address *);
struct rose_route *rose_route_free_lci(unsigned int, struct rose_neigh *);
+unsigned int rose_assign_lci(struct sock *sk);
struct rose_neigh *rose_get_neigh(rose_address *, unsigned char *,
unsigned char *, int);
int rose_rt_ioctl(unsigned int, void __user *);
diff --git a/include/net/sctp/sm.h b/include/net/sctp/sm.h
index 3bfd261a53cc..76605d1ee839 100644
--- a/include/net/sctp/sm.h
+++ b/include/net/sctp/sm.h
@@ -252,6 +252,7 @@ struct sctp_chunk *sctp_make_fwdtsn(const struct sctp_association *asoc,
struct sctp_fwdtsn_skip *skiplist);
struct sctp_chunk *sctp_make_auth(const struct sctp_association *asoc,
__u16 key_id);
+void sctp_control_set_owner_w(struct sctp_chunk *chunk);
struct sctp_chunk *sctp_make_strreset_req(const struct sctp_association *asoc,
__u16 stream_num, __be16 *stream_list,
bool out, bool in);
diff --git a/include/sound/hwdep.h b/include/sound/hwdep.h
index b0da633184cd..7357727f0660 100644
--- a/include/sound/hwdep.h
+++ b/include/sound/hwdep.h
@@ -57,6 +57,7 @@ struct snd_hwdep {
struct mutex open_mutex;
int used; /* reference counter */
+ struct mutex dsp_mutex; /* serializes DSP load requests */
unsigned int dsp_loaded; /* bit fields of loaded dsp indices */
unsigned int exclusive:1; /* exclusive access mode */
};
diff --git a/include/trace/events/migrate.h b/include/trace/events/migrate.h
index cd01dd7b3640..8869f1fe3b98 100644
--- a/include/trace/events/migrate.h
+++ b/include/trace/events/migrate.h
@@ -51,7 +51,7 @@ TRACE_EVENT(mm_migrate_pages,
TP_PROTO(unsigned long succeeded, unsigned long failed,
unsigned long thp_succeeded, unsigned long thp_failed,
unsigned long thp_split, unsigned long large_folio_split,
- enum migrate_mode mode, int reason),
+ enum migrate_mode mode, enum migrate_reason reason),
TP_ARGS(succeeded, failed, thp_succeeded, thp_failed,
thp_split, large_folio_split, mode, reason),
@@ -64,7 +64,7 @@ TRACE_EVENT(mm_migrate_pages,
__field( unsigned long, thp_split)
__field( unsigned long, large_folio_split)
__field( enum migrate_mode, mode)
- __field( int, reason)
+ __field( enum migrate_reason, reason)
),
TP_fast_assign(
@@ -91,13 +91,13 @@ TRACE_EVENT(mm_migrate_pages,
TRACE_EVENT(mm_migrate_pages_start,
- TP_PROTO(enum migrate_mode mode, int reason),
+ TP_PROTO(enum migrate_mode mode, enum migrate_reason reason),
TP_ARGS(mode, reason),
TP_STRUCT__entry(
__field(enum migrate_mode, mode)
- __field(int, reason)
+ __field(enum migrate_reason, reason)
),
TP_fast_assign(
diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h
index bdd20666d1eb..65e564965e98 100644
--- a/include/uapi/linux/iommufd.h
+++ b/include/uapi/linux/iommufd.h
@@ -775,6 +775,12 @@ enum iommufd_hwpt_get_dirty_bitmap_flags {
* Walk the IOMMU pagetables for a given IOVA range to return a bitmap
* with the dirty IOVAs. In doing so it will also by default clear any
* dirty bit metadata set in the IOPTE.
+ *
+ * The IOVA range given must fully enclose any IOPTE. If the range starts or
+ * ends inside a dirty IOPTE then the ioctl will fail. This avoids partially
+ * reporting a dirty range.
+ *
+ * On failure the dirty state becomes unknown and the userspace must restart.
*/
struct iommu_hwpt_get_dirty_bitmap {
__u32 size;
diff --git a/io_uring/cancel.c b/io_uring/cancel.c
index 50ec818aa19f..471d6f2e5f18 100644
--- a/io_uring/cancel.c
+++ b/io_uring/cancel.c
@@ -192,7 +192,8 @@ static int __io_async_cancel(struct io_cancel_data *cd,
break;
if (!all)
return ret;
- nr++;
+ /* poll, futex and waitid return how many they cancelled */
+ nr += max(ret, 1);
} while (1);
/* slow path, try all io-wq's */
@@ -641,7 +642,16 @@ __cold void io_uring_cancel_generic(bool cancel_all, struct io_sq_data *sqd)
}
if (loop) {
+ /*
+ * A request that runs in io-wq and doesn't end makes
+ * each round come here. Don't hold sqd->lock for that
+ * long either.
+ */
+ if (sqd)
+ mutex_unlock(&sqd->lock);
cond_resched();
+ if (sqd)
+ mutex_lock(&sqd->lock);
continue;
}
@@ -659,8 +669,20 @@ __cold void io_uring_cancel_generic(bool cancel_all, struct io_sq_data *sqd)
* avoids a race where a completion comes in before we did
* prepare_to_wait().
*/
- if (inflight == tctx_inflight(tctx, !cancel_all))
+ if (inflight == tctx_inflight(tctx, !cancel_all)) {
+ /*
+ * A zero-copy notification can't be canceled, and a
+ * peer that doesn't read keeps it pending. Don't hold
+ * sqd->lock while waiting for it, or whoever wants the
+ * lock sleeps as long. No ring is used across this,
+ * the next round walks ->ctx_list anew.
+ */
+ if (sqd)
+ mutex_unlock(&sqd->lock);
schedule();
+ if (sqd)
+ mutex_lock(&sqd->lock);
+ }
end_wait:
finish_wait(&tctx->wait, &wait);
} while (1);
diff --git a/io_uring/poll.c b/io_uring/poll.c
index 48381f0cbe32..2594c1f04e70 100644
--- a/io_uring/poll.c
+++ b/io_uring/poll.c
@@ -128,12 +128,24 @@ static void io_poll_req_insert(struct io_kiocb *req)
hlist_add_head(&req->hash_node, &table->hbs[index].list);
}
+#define IO_POLL_UNMASK (EPOLLERR|EPOLLHUP|EPOLLNVAL)
+
+static __poll_t io_poll_unmask(__poll_t events)
+{
+ /*
+ * A peer that stopped sending is an event for a reader, but a write
+ * only poll must stay armed until the file is writable again.
+ */
+ if (events & (EPOLLIN | EPOLLRDNORM | EPOLLRDBAND))
+ return IO_POLL_UNMASK | EPOLLRDHUP;
+ return IO_POLL_UNMASK;
+}
+
static void io_init_poll_iocb(struct io_poll *poll, __poll_t events)
{
poll->head = NULL;
-#define IO_POLL_UNMASK (EPOLLERR|EPOLLHUP|EPOLLNVAL|EPOLLRDHUP)
/* mask in events that we always want/need */
- poll->events = events | IO_POLL_UNMASK;
+ poll->events = events | io_poll_unmask(events);
INIT_LIST_HEAD(&poll->wait.entry);
init_waitqueue_func_entry(&poll->wait, io_poll_wake);
}
@@ -724,7 +736,7 @@ int io_arm_apoll(struct io_kiocb *req, unsigned issue_flags, __poll_t mask)
int io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags)
{
const struct io_issue_def *def = &io_issue_defs[req->opcode];
- __poll_t mask = POLLPRI | POLLERR;
+ __poll_t mask = EPOLLERR;
if (!def->pollin && !def->pollout)
return IO_APOLL_ABORTED;
@@ -732,7 +744,8 @@ int io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags)
return IO_APOLL_ABORTED;
if (def->pollin) {
- mask |= EPOLLIN | EPOLLRDNORM;
+ /* urgent data is an event for a reader only */
+ mask |= EPOLLIN | EPOLLRDNORM | EPOLLPRI;
/* If reading from MSG_ERRQUEUE using recvmsg, ignore POLLIN */
if (req->flags & REQ_F_CLEAR_POLLIN)
@@ -824,12 +837,47 @@ static int io_poll_disarm(struct io_kiocb *req)
return 0;
}
+/*
+ * Cancel every request matching @cd in one pass. Cancelled requests stay
+ * hashed until their task work runs, so finding them one at a time would
+ * walk past all the earlier ones on each lookup.
+ */
+static int io_poll_cancel_all(struct io_ring_ctx *ctx, struct io_cancel_data *cd)
+{
+ unsigned int nr_buckets = 1U << ctx->cancel_table.hash_bits;
+ unsigned int i = 0;
+ struct hlist_node *tmp;
+ struct io_kiocb *req;
+ int nr = 0;
+
+ if (!(cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP |
+ IORING_ASYNC_CANCEL_ANY))) {
+ /* keyed by user_data, so only one bucket can match */
+ i = hash_long(cd->data, ctx->cancel_table.hash_bits);
+ nr_buckets = i + 1;
+ }
+
+ for (; i < nr_buckets; i++) {
+ struct io_hash_bucket *hb = &ctx->cancel_table.hbs[i];
+
+ hlist_for_each_entry_safe(req, tmp, &hb->list, hash_node) {
+ if (!io_cancel_req_match(req, cd))
+ continue;
+ io_poll_cancel_req(req);
+ nr++;
+ }
+ }
+ return nr ?: -ENOENT;
+}
+
static int __io_poll_cancel(struct io_ring_ctx *ctx, struct io_cancel_data *cd)
{
struct io_kiocb *req;
- if (cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP |
- IORING_ASYNC_CANCEL_ANY))
+ if (cd->flags & (IORING_ASYNC_CANCEL_ALL | IORING_ASYNC_CANCEL_ANY))
+ return io_poll_cancel_all(ctx, cd);
+
+ if (cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP))
req = io_poll_file_find(ctx, cd);
else
req = io_poll_find(ctx, false, cd);
@@ -968,7 +1016,7 @@ int io_poll_remove(struct io_kiocb *req, unsigned int issue_flags)
poll->events &= ~0xffff;
poll->events |= poll_update->events & 0xffff;
- poll->events |= IO_POLL_UNMASK;
+ poll->events |= io_poll_unmask(poll->events);
}
if (poll_update->update_user_data)
preq->cqe.user_data = poll_update->new_user_data;
diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
index 41e5755bd9aa..7170b3f001d4 100644
--- a/io_uring/rsrc.c
+++ b/io_uring/rsrc.c
@@ -429,11 +429,12 @@ static int io_files_update_with_index_alloc(struct io_kiocb *req,
{
struct io_rsrc_update *up = io_kiocb_to_cmd(req, struct io_rsrc_update);
__s32 __user *fds = u64_to_user_ptr(up->arg);
+ struct io_ring_ctx *ctx = req->ctx;
unsigned int done;
struct file *file;
int ret, fd;
- if (!req->ctx->file_table.data.nr)
+ if (!ctx->file_table.data.nr)
return -ENXIO;
for (done = 0; done < up->nr_args; done++) {
@@ -447,15 +448,25 @@ static int io_files_update_with_index_alloc(struct io_kiocb *req,
ret = -EBADF;
break;
}
- ret = io_fixed_fd_install(req, issue_flags, file,
- IORING_FILE_INDEX_ALLOC);
- if (ret < 0)
+ /*
+ * Hold the lock across the copy_to_user(), if it's dropped the
+ * slot can be closed and reused before the error path runs.
+ */
+ io_ring_submit_lock(ctx, issue_flags);
+ ret = __io_fixed_fd_install(ctx, file,
+ IORING_FILE_INDEX_ALLOC);
+ if (ret < 0) {
+ io_ring_submit_unlock(ctx, issue_flags);
+ fput(file);
break;
+ }
if (copy_to_user(&fds[done], &ret, sizeof(ret))) {
- __io_close_fixed(req->ctx, issue_flags, ret);
+ io_fixed_fd_remove(ctx, ret);
+ io_ring_submit_unlock(ctx, issue_flags);
ret = -EFAULT;
break;
}
+ io_ring_submit_unlock(ctx, issue_flags);
}
if (done)
diff --git a/io_uring/sqpoll.c b/io_uring/sqpoll.c
index 406029c14fbc..66405f4630e7 100644
--- a/io_uring/sqpoll.c
+++ b/io_uring/sqpoll.c
@@ -28,6 +28,7 @@
enum {
IO_SQ_THREAD_SHOULD_STOP = 0,
IO_SQ_THREAD_SHOULD_PARK,
+ IO_SQ_THREAD_EXITING,
};
void io_sq_thread_unpark(struct io_sq_data *sqd)
@@ -357,6 +358,18 @@ static int io_sq_thread(void *data)
timeout = jiffies + sqd->sq_thread_idle;
}
+ /*
+ * The SQPOLL thread never returns to userspace, so do the
+ * return-to-userspace work, like memory.high reclaim, here.
+ * It can sleep for a long time, so drop sqd->lock.
+ */
+ if (test_thread_flag(TIF_NOTIFY_RESUME)) {
+ mutex_unlock(&sqd->lock);
+ resume_user_mode_work(NULL);
+ mutex_lock(&sqd->lock);
+ sqd->sq_cpu = raw_smp_processor_id();
+ }
+
cap_entries = !list_is_singular(&sqd->ctx_list);
list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) {
int ret = __io_sq_thread(ctx, sqd, cap_entries, &ist);
@@ -438,6 +451,13 @@ static int io_sq_thread(void *data)
if (retry_list)
io_sq_tw(&retry_list, UINT_MAX);
+ /*
+ * Cancelation drops sqd->lock while it waits, and a ring could attach
+ * to a thread that is on its way out. A fatal signal doesn't set the
+ * stop bit, so mark the thread as exiting here for
+ * io_sq_offload_create() to see.
+ */
+ set_bit(IO_SQ_THREAD_EXITING, &sqd->state);
io_uring_cancel_generic(true, sqd);
rcu_assign_pointer(sqd->thread, NULL);
put_task_struct(current);
@@ -506,7 +526,11 @@ __cold int io_sq_offload_create(struct io_ring_ctx *ctx,
list_add(&ctx->sqd_list, &sqd->ctx_list);
io_sqd_update_thread_idle(sqd);
/* don't attach to a dying SQPOLL thread, would be racy */
- ret = (attached && !sqd->thread) ? -ENXIO : 0;
+ ret = 0;
+ if (attached && (!sqd->thread ||
+ test_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state) ||
+ test_bit(IO_SQ_THREAD_EXITING, &sqd->state)))
+ ret = -ENXIO;
io_sq_thread_unpark(sqd);
if (ret < 0)
diff --git a/ipc/namespace.c b/ipc/namespace.c
index 7aab1e6e0f86..b69da8c68f53 100644
--- a/ipc/namespace.c
+++ b/ipc/namespace.c
@@ -254,3 +254,10 @@ const struct proc_ns_operations ipcns_operations = {
.install = ipcns_install,
.owner = ipcns_owner,
};
+
+static int __init ipc_ns_tree_init(void)
+{
+ ns_tree_add(&init_ipc_ns);
+ return 0;
+}
+pure_initcall(ipc_ns_tree_init);
diff --git a/ipc/shm.c b/ipc/shm.c
index 2f5cee0c78c7..59fade2253d8 100644
--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -45,7 +45,6 @@
#include <linux/mount.h>
#include <linux/ipc_namespace.h>
#include <linux/rhashtable.h>
-#include <linux/nstree.h>
#include <linux/uaccess.h>
@@ -149,7 +148,6 @@ void shm_exit_ns(struct ipc_namespace *ns)
static int __init ipc_ns_init(void)
{
shm_init_ns(&init_ipc_ns);
- ns_tree_add(&init_ipc_ns);
return 0;
}
diff --git a/kernel/cgroup/cgroup-internal.h b/kernel/cgroup/cgroup-internal.h
index 22051b4f1ccb..e4a872e2e08c 100644
--- a/kernel/cgroup/cgroup-internal.h
+++ b/kernel/cgroup/cgroup-internal.h
@@ -184,11 +184,6 @@ extern bool cgrp_dfl_visible;
for ((ssid) = 0; (ssid) < CGROUP_SUBSYS_COUNT && \
(((ss) = cgroup_subsys[ssid]) || true); (ssid)++)
-static inline bool cgroup_is_dead(const struct cgroup *cgrp)
-{
- return !(cgrp->self.flags & CSS_ONLINE);
-}
-
static inline bool notify_on_release(const struct cgroup *cgrp)
{
return test_bit(CGRP_NOTIFY_ON_RELEASE, &cgrp->flags);
@@ -222,7 +217,6 @@ static inline void get_css_set(struct css_set *cset)
}
bool cgroup_ssid_enabled(int ssid);
-bool cgroup_on_dfl(const struct cgroup *cgrp);
struct cgroup_root *cgroup_root_from_kf(struct kernfs_root *kf_root);
struct cgroup *task_cgroup_from_root(struct task_struct *task,
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index aa629b7d76e5..ac7b2103df68 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -70,14 +70,6 @@
/* let's not notify more than 100 times per second */
#define CGROUP_FILE_NOTIFY_MIN_INTV DIV_ROUND_UP(HZ, 100)
-/*
- * To avoid confusing the compiler (and generating warnings) with code
- * that attempts to access what would be a 0-element array (i.e. sized
- * to a potentially empty array when CGROUP_SUBSYS_COUNT == 0), this
- * constant expression can be added.
- */
-#define CGROUP_HAS_SUBSYS_CONFIG (CGROUP_SUBSYS_COUNT > 0)
-
/*
* cgroup_mutex is the master lock. Any modification to cgroup or its
* hierarchy must be performed while holding it.
@@ -513,27 +505,6 @@ static u16 cgroup_ss_mask(struct cgroup *cgrp)
return cgrp->root->subsys_mask;
}
-/**
- * cgroup_css - obtain a cgroup's css for the specified subsystem
- * @cgrp: the cgroup of interest
- * @ss: the subsystem of interest (%NULL returns @cgrp->self)
- *
- * Return @cgrp's css (cgroup_subsys_state) associated with @ss. This
- * function must be called either under cgroup_mutex or rcu_read_lock() and
- * the caller is responsible for pinning the returned css if it wants to
- * keep accessing it outside the said locks. This function may return
- * %NULL if @cgrp doesn't have @subsys_id enabled.
- */
-static struct cgroup_subsys_state *cgroup_css(struct cgroup *cgrp,
- struct cgroup_subsys *ss)
-{
- if (CGROUP_HAS_SUBSYS_CONFIG && ss)
- return rcu_dereference_check(cgrp->subsys[ss->id],
- lockdep_is_held(&cgroup_mutex));
- else
- return &cgrp->self;
-}
-
/**
* cgroup_e_css_by_mask - obtain a cgroup's effective css for the specified ss
* @cgrp: the cgroup of interest
@@ -745,32 +716,6 @@ EXPORT_SYMBOL_GPL(of_css);
} \
} while (false)
-/* iterate over child cgrps, lock should be held throughout iteration */
-#define cgroup_for_each_live_child(child, cgrp) \
- list_for_each_entry((child), &(cgrp)->self.children, self.sibling) \
- if (({ lockdep_assert_held(&cgroup_mutex); \
- cgroup_is_dead(child); })) \
- ; \
- else
-
-/* walk live descendants in pre order */
-#define cgroup_for_each_live_descendant_pre(dsct, d_css, cgrp) \
- css_for_each_descendant_pre((d_css), cgroup_css((cgrp), NULL)) \
- if (({ lockdep_assert_held(&cgroup_mutex); \
- (dsct) = (d_css)->cgroup; \
- cgroup_is_dead(dsct); })) \
- ; \
- else
-
-/* walk live descendants in postorder */
-#define cgroup_for_each_live_descendant_post(dsct, d_css, cgrp) \
- css_for_each_descendant_post((d_css), cgroup_css((cgrp), NULL)) \
- if (({ lockdep_assert_held(&cgroup_mutex); \
- (dsct) = (d_css)->cgroup; \
- cgroup_is_dead(dsct); })) \
- ; \
- else
-
/*
* The default css_set - used by init and its children prior to any
* hierarchies being mounted. It contains a pointer to the root state
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 469f89cb92f1..ec2d21af68f6 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -7324,6 +7324,15 @@ static void perf_sigtrap(struct perf_event *event)
if (event->ctx->task == TASK_TOMBSTONE)
return;
+ /*
+ * The event was removed after this signal was queued, e.g. by exec()
+ * (remove_on_exec), which may also have closed its fd (close-on-exec).
+ * The new program has default signal handlers, so a SIGTRAP would
+ * kill it.
+ */
+ if (READ_ONCE(event->state) <= PERF_EVENT_STATE_EXIT)
+ return;
+
/*
* We'd expect this to only occur if the irq_work is delayed and either
* ctx->task or current has changed in the meantime. This can be the
diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c
index 9691f6bc2fba..e503573e0319 100644
--- a/kernel/kexec_handover.c
+++ b/kernel/kexec_handover.c
@@ -10,6 +10,7 @@
#include <linux/cleanup.h>
#include <linux/cma.h>
+#include <linux/crash_dump.h>
#include <linux/count_zeros.h>
#include <linux/kexec.h>
#include <linux/kexec_handover.h>
@@ -1325,6 +1326,12 @@ void __init kho_memory_init(void)
{
struct folio *folio;
+ if (is_kdump_kernel()) {
+ kho_enable = false;
+ pr_info("disabled in the kdump kernel\n");
+ return;
+ }
+
if (kho_in.scratch_phys) {
kho_scratch = phys_to_virt(kho_in.scratch_phys);
kho_release_scratch();
diff --git a/kernel/panic.c b/kernel/panic.c
index 7f8725f2154b..e1d0c16fc526 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -360,13 +360,19 @@ EXPORT_SYMBOL(panic_on_other_cpu);
* nmi_panic_self_stop() which can provide architecture dependent code such
* as saving register state for crash dump.
*/
-void nmi_panic(struct pt_regs *regs, const char *msg)
+void nmi_panic(struct pt_regs *regs, const char *fmt, ...)
{
+ va_list args;
+
+ va_start(args, fmt);
+
if (panic_try_start())
- panic("%s", msg);
+ vpanic(fmt, args);
if (panic_on_other_cpu())
nmi_panic_self_stop(regs);
+
+ va_end(args);
}
EXPORT_SYMBOL(nmi_panic);
diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
index 78f15ddc0aa5..3117d1ef869c 100644
--- a/kernel/sched/ext.c
+++ b/kernel/sched/ext.c
@@ -1438,7 +1438,11 @@ static void ops_dequeue(struct rq *rq, struct task_struct *p, u64 deq_flags)
struct scx_sched *sch = scx_root;
unsigned long opss;
- /* dequeue is always temporary, don't reset runnable_at */
+ /*
+ * Don't reset runnable_at. If @p is waiting, the wait continues across
+ * temporary dequeues, e.g. for a migration or an attribute change.
+ * scx_disable_task() ends it if @p leaves its scheduler.
+ */
clr_task_runnable(p, false);
/* acquire ensures that we see the preceding updates on QUEUED */
@@ -2959,6 +2963,14 @@ static void scx_disable_task(struct task_struct *p)
if (SCX_HAS_OP(sch, disable))
SCX_CALL_OP_TASK(sch, SCX_KF_REST, disable, rq, p);
scx_set_task_state(p, SCX_TASK_READY);
+
+ /*
+ * If @p is runnable but not running, its wait started at runnable_at,
+ * which ops_dequeue() didn't reset. Leaving @sch ends the wait. Reset
+ * runnable_at the next time @p is enqueued on SCX, so that the watchdog
+ * doesn't count the time @p spends elsewhere.
+ */
+ p->scx.flags |= SCX_TASK_RESET_RUNNABLE_AT;
}
static void scx_exit_task(struct task_struct *p)
diff --git a/kernel/time/posix-stubs.c b/kernel/time/posix-stubs.c
index 9b6fcb8d85e7..5a27335a917e 100644
--- a/kernel/time/posix-stubs.c
+++ b/kernel/time/posix-stubs.c
@@ -83,7 +83,7 @@ SYSCALL_DEFINE2(clock_getres, const clockid_t, which_clock, struct __kernel_time
case CLOCK_REALTIME:
case CLOCK_MONOTONIC:
case CLOCK_BOOTTIME:
- if (put_timespec64(&rtn_tp, tp))
+ if (tp && put_timespec64(&rtn_tp, tp))
return -EFAULT;
return 0;
default:
@@ -166,7 +166,7 @@ SYSCALL_DEFINE2(clock_getres_time32, clockid_t, which_clock,
case CLOCK_REALTIME:
case CLOCK_MONOTONIC:
case CLOCK_BOOTTIME:
- if (put_old_timespec32(&rtn_tp, tp))
+ if (tp && put_old_timespec32(&rtn_tp, tp))
return -EFAULT;
return 0;
default:
diff --git a/kernel/time/posix-timers.c b/kernel/time/posix-timers.c
index 3082129d836c..03bf012fae53 100644
--- a/kernel/time/posix-timers.c
+++ b/kernel/time/posix-timers.c
@@ -306,6 +306,8 @@ static bool common_hrtimer_rearm(struct k_itimer *timr)
static bool __posixtimer_deliver_signal(struct kernel_siginfo *info, struct k_itimer *timr)
{
+ bool queued;
+
guard(spinlock)(&timr->it_lock);
/*
@@ -319,12 +321,18 @@ static bool __posixtimer_deliver_signal(struct kernel_siginfo *info, struct k_it
if (!timr->it_interval || WARN_ON_ONCE(timr->it_status != POSIX_TIMER_REQUEUE_PENDING))
return true;
- timr->kclock->timer_rearm(timr);
- timr->it_status = POSIX_TIMER_ARMED;
+ /* timer_rearm() updates timr::it_overrun */
+ queued = timr->kclock->timer_rearm(timr);
+
timr->it_overrun_last = timr->it_overrun;
timr->it_overrun = -1LL;
++timr->it_signal_seq;
info->si_overrun = timer_overrun_to_int(timr);
+
+ if (queued)
+ timr->it_status = POSIX_TIMER_ARMED;
+ else
+ posix_timer_queue_signal(timr);
return true;
}
@@ -694,16 +702,8 @@ void common_timer_get(struct k_itimer *timr, struct itimerspec64 *cur_setting)
if (iv) {
cur_setting->it_interval = ktime_to_timespec64(iv);
} else if (timr->it_status == POSIX_TIMER_DISARMED) {
- /*
- * SIGEV_NONE oneshot timers are never queued and therefore
- * timr->it_status is always DISARMED. The check below
- * vs. remaining time will handle this case.
- *
- * For all other timers there is nothing to update here, so
- * return.
- */
- if (!sig_none)
- return;
+ /* Nothing to update for a disarmed timer */
+ return;
}
now = kc->clock_get_ktime(timr->it_clock);
@@ -713,7 +713,7 @@ void common_timer_get(struct k_itimer *timr, struct itimerspec64 *cur_setting)
* is a SIGEV_NONE timer move the expiry time forward by intervals,
* so expiry is > now.
*/
- if (iv && timr->it_status != POSIX_TIMER_ARMED)
+ if (iv && (sig_none || timr->it_status != POSIX_TIMER_ARMED))
timr->it_overrun += kc->timer_forward(timr, now);
remaining = kc->timer_remaining(timr, now);
@@ -912,9 +912,20 @@ int common_timer_set(struct k_itimer *timr, int flags,
expires = timens_ktime_to_host(timr->it_clock, expires);
sigev_none = timr->it_sigev_notify == SIGEV_NONE;
- kc->timer_arm(timr, expires, flags & TIMER_ABSTIME, sigev_none);
- if (!sigev_none)
+ if (kc->timer_arm(timr, expires, flags & TIMER_ABSTIME, sigev_none)) {
+ /*
+ * SIGEV_NONE timers are not queued, but mark them armed
+ * so that common_timer_get() can tell them apart from
+ * disarmed timers. An expired SIGEV_NONE timer stays
+ * armed, which is harmless: common_timer_get() reports
+ * zero remaining time for it and it never reaches the
+ * signal delivery code.
+ */
timr->it_status = POSIX_TIMER_ARMED;
+ } else {
+ /* Timer was already expired, queue the signal */
+ posix_timer_queue_signal(timr);
+ }
return 0;
}
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 4f04c621c922..ea0e7354930a 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -377,9 +377,21 @@ static bool can_stop_full_tick(int cpu, struct tick_sched *ts)
return true;
}
+static void tick_nohz_full_update_tick(struct tick_sched *ts);
+
static void nohz_full_kick_func(struct irq_work *work)
{
- /* Empty, the tick restart happens on tick_nohz_irq_exit() */
+ struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched);
+
+ if (WARN_ON_ONCE(!tick_nohz_full_cpu(smp_processor_id())))
+ return;
+
+ /* Task-context flushes may not be followed by an IRQ exit. */
+ if (!in_hardirq()) {
+ if (WARN_ON_ONCE(tick_sched_flag_test(ts, TS_FLAG_INIDLE)))
+ return;
+ tick_nohz_full_update_tick(ts);
+ }
}
static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) =
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 32f59de2ba4c..bf25112b0156 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -14,6 +14,7 @@
#include <linux/sched.h>
#include <linux/sched/loadavg.h>
#include <linux/sched/clock.h>
+#include <linux/security.h>
#include <linux/syscore_ops.h>
#include <linux/clocksource.h>
#include <linux/jiffies.h>
@@ -2937,11 +2938,15 @@ static int aux_clock_set(const clockid_t id, const struct timespec64 *tnew)
struct tk_data *aux_tkd = aux_get_tk_data(id);
struct timekeeper *aux_tks;
ktime_t tnow, nsecs;
+ int ret;
if (!timespec64_valid_settod(tnew))
return -EINVAL;
if (!aux_tkd)
return -ENODEV;
+ ret = security_settime64(tnew, NULL);
+ if (ret)
+ return ret;
aux_tks = &aux_tkd->shadow_timekeeper;
diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c
index 882e10e776fe..93afc706e402 100644
--- a/kernel/trace/trace_events_hist.c
+++ b/kernel/trace/trace_events_hist.c
@@ -1757,6 +1757,9 @@ static bool expr_field_str(struct hist_field *field, struct seq_buf *s)
seq_buf_printf(s, ".%s", flags_str);
}
+ if (field->buckets)
+ seq_buf_printf(s, "=%lu", field->buckets);
+
return !seq_buf_has_overflowed(s);
}
diff --git a/lib/assoc_array.c b/lib/assoc_array.c
index 01619e88a52c..d3ee08fa2ab5 100644
--- a/lib/assoc_array.c
+++ b/lib/assoc_array.c
@@ -470,33 +470,32 @@ static bool assoc_array_insert_in_empty_tree(struct assoc_array_edit *edit)
}
/*
- * Handle insertion into a terminal node.
+ * The pointer that leads to a node or shortcut with the given back pointer
+ * and parent slot: the root, a slot in a node, or a shortcut's next node.
*/
-static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
- const struct assoc_array_ops *ops,
- const void *index_key,
- struct assoc_array_walk_result *result)
+static struct assoc_array_ptr **assoc_array_parent_link(struct assoc_array *array,
+ struct assoc_array_ptr *back_pointer,
+ int parent_slot)
{
- struct assoc_array_shortcut *shortcut, *new_s0;
- struct assoc_array_node *node, *new_n0, *new_n1, *side;
- struct assoc_array_ptr *ptr;
- unsigned long dissimilarity, base_seg, blank;
- size_t keylen;
- bool have_meta;
- int level, diff;
- int slot, next_slot, free_slot, i, j;
-
- node = result->terminal_node.node;
- level = result->terminal_node.level;
- edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = result->terminal_node.slot;
+ if (!back_pointer)
+ return &array->root;
+ if (assoc_array_ptr_is_node(back_pointer))
+ return &assoc_array_ptr_to_node(back_pointer)->slots[parent_slot];
+ return &assoc_array_ptr_to_shortcut(back_pointer)->next_node;
+}
- pr_devel("-->%s()\n", __func__);
+/*
+ * Insert into a terminal node without changing its shape, if that can be
+ * done: in place of a leaf with the same index key, or in a free slot.
+ */
+static bool assoc_array_insert_in_node(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const void *index_key,
+ struct assoc_array_node *node)
+{
+ struct assoc_array_ptr *ptr;
+ int free_slot, i;
- /* We arrived at a node which doesn't have an onward node or shortcut
- * pointer that we have to follow. This means that (a) the leaf we
- * want must go here (either by insertion or replacement) or (b) we
- * need to split this node and insert in one of the fragments.
- */
free_slot = -1;
/* Firstly, we have to check the leaves in this node to see if there's
@@ -514,7 +513,6 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
pr_devel("replace in slot %d\n", i);
edit->leaf_p = &node->slots[i];
edit->dead_leaf = node->slots[i];
- pr_devel("<--%s() = ok [replace]\n", __func__);
return true;
}
}
@@ -526,28 +524,28 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
pr_devel("insert in free slot %d\n", free_slot);
edit->leaf_p = &node->slots[free_slot];
edit->adjust_count_on = node;
- pr_devel("<--%s() = ok [insert]\n", __func__);
return true;
}
- /* The node has no spare slots - so we're either going to have to split
- * it or insert another node before it.
- *
- * Whatever, we're going to need at least two new nodes - so allocate
- * those now. We may also need a new shortcut, but we deal with that
- * when we need it.
- */
- new_n0 = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL);
- if (!new_n0)
- return false;
- edit->new_meta[0] = assoc_array_node_to_ptr(new_n0);
- new_n1 = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL);
- if (!new_n1)
- return false;
- edit->new_meta[1] = assoc_array_node_to_ptr(new_n1);
+ return false;
+}
+
+/*
+ * Fill in the edit script's segment cache for a full node: the segment at
+ * this level of each leaf's index key, or 0xff for a metadata pointer.
+ * The new key's segment, in the last entry, is left to the caller. Returns
+ * true if there are any metadata pointers.
+ */
+static bool assoc_array_fill_segment_cache(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const struct assoc_array_node *node,
+ int level)
+{
+ struct assoc_array_ptr *ptr;
+ unsigned long base_seg;
+ bool have_meta;
+ int i;
- /* We need to find out how similar the leaves are. */
- pr_devel("no spare slots\n");
have_meta = false;
for (i = 0; i < ASSOC_ARRAY_FAN_OUT; i++) {
ptr = node->slots[i];
@@ -561,65 +559,42 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK;
edit->segment_cache[i] = base_seg & ASSOC_ARRAY_FAN_MASK;
}
+ return have_meta;
+}
- if (have_meta) {
- pr_devel("have meta\n");
- goto split_node;
- }
-
- /* The node contains only leaves */
- dissimilarity = 0;
- base_seg = edit->segment_cache[0];
- for (i = 1; i < ASSOC_ARRAY_FAN_OUT; i++)
- dissimilarity |= edit->segment_cache[i] ^ base_seg;
-
- pr_devel("only leaves; dissimilarity=%lx\n", dissimilarity);
+/*
+ * Whether the leaves of a full node and the new key all share one segment,
+ * so that splitting the node cannot make room.
+ */
+static bool assoc_array_all_one_segment(const struct assoc_array_edit *edit)
+{
+ int i;
- if ((dissimilarity & ASSOC_ARRAY_FAN_MASK) == 0) {
- /* The old leaves all cluster in the same slot. We will need
- * to insert a shortcut if the new node wants to cluster with them.
- */
- if ((edit->segment_cache[ASSOC_ARRAY_FAN_OUT] ^ base_seg) == 0)
- goto all_leaves_cluster_together;
+ for (i = 1; i < ASSOC_ARRAY_FAN_OUT + 1; i++)
+ if (edit->segment_cache[i] != edit->segment_cache[0])
+ return false;
+ return true;
+}
- /* Otherwise all the old leaves cluster in the same slot, but
- * the new leaf wants to go into a different slot - so we
- * create a new node (n0) to hold the new leaf and a pointer to
- * a new node (n1) holding all the old leaves.
- *
- * This can be done by falling through to the node splitting
- * path.
- */
- pr_devel("present leaves cluster but not new leaf\n");
- }
+/*
+ * Split a full node into new_n0, which takes its place, and new_n1, below
+ * new_n0 in the slot of a segment that at least two of the leaves share,
+ * the new one included. The caller has filled in the segment cache and
+ * set new_n0's back pointer and parent slot.
+ */
+static void assoc_array_split_node(struct assoc_array_edit *edit,
+ struct assoc_array_node *node,
+ struct assoc_array_node *new_n0,
+ struct assoc_array_node *new_n1)
+{
+ struct assoc_array_shortcut *shortcut;
+ struct assoc_array_node *side;
+ struct assoc_array_ptr *ptr;
+ int slot, next_slot, free_slot, i, j;
-split_node:
pr_devel("split node\n");
- /* We need to split the current node. The node must contain anything
- * from a single leaf (in the one leaf case, this leaf will cluster
- * with the new leaf) and the rest meta-pointers, to all leaves, some
- * of which may cluster.
- *
- * It won't contain the case in which all the current leaves plus the
- * new leaves want to cluster in the same slot.
- *
- * We need to expel at least two leaves out of a set consisting of the
- * leaves in the node and the new leaf. The current meta pointers can
- * just be copied as they shouldn't cluster with any of the leaves.
- *
- * We need a new node (n0) to replace the current one and a new node to
- * take the expelled nodes (n1).
- */
- edit->set[0].to = assoc_array_node_to_ptr(new_n0);
- new_n0->back_pointer = node->back_pointer;
- new_n0->parent_slot = node->parent_slot;
new_n1->back_pointer = assoc_array_node_to_ptr(new_n0);
- new_n1->parent_slot = -1; /* Need to calculate this */
-
-do_split_node:
- pr_devel("do_split_node\n");
-
new_n0->nr_leaves_on_branch = node->nr_leaves_on_branch;
new_n1->nr_leaves_on_branch = 0;
@@ -699,26 +674,29 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
}
}
}
+}
- ptr = node->back_pointer;
- if (!ptr)
- edit->set[0].ptr = &edit->array->root;
- else if (assoc_array_ptr_is_node(ptr))
- edit->set[0].ptr = &assoc_array_ptr_to_node(ptr)->slots[node->parent_slot];
- else
- edit->set[0].ptr = &assoc_array_ptr_to_shortcut(ptr)->next_node;
- edit->excised_meta[0] = assoc_array_node_to_ptr(node);
- pr_devel("<--%s() = ok [split node]\n", __func__);
- return true;
+/*
+ * All the leaves, new and old, want to cluster together in this node in the
+ * same slot, so we have to replace this node with a shortcut to skip over
+ * the identical parts of the key, leading to new_n0. If the node is the
+ * target of a shortcut, the new one replaces that shortcut as well. The
+ * caller then splits the leaves between new_n0 and new_n1 at the shortcut's
+ * far end, for which this fills in the segment cache.
+ */
+static struct assoc_array_shortcut *
+assoc_array_insert_shortcut(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const void *index_key,
+ struct assoc_array_node *node, int level,
+ struct assoc_array_node *new_n0)
+{
+ struct assoc_array_shortcut *new_s0;
+ unsigned long base_seg, blank;
+ size_t keylen;
+ int diff, i;
-all_leaves_cluster_together:
- /* All the leaves, new and old, want to cluster together in this node
- * in the same slot, so we have to replace this node with a shortcut to
- * skip over the identical parts of the key and then place a pair of
- * nodes, one inside the other, at the end of the shortcut and
- * distribute the keys between them.
- *
- * Firstly we need to work out where the leaves start diverging as a
+ /* Firstly we need to work out where the leaves start diverging as a
* bit position into their keys so that we know how big the shortcut
* needs to be.
*
@@ -744,17 +722,28 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
new_s0 = kzalloc(struct_size(new_s0, index_key, keylen), GFP_KERNEL);
if (!new_s0)
- return false;
+ return NULL;
edit->new_meta[2] = assoc_array_shortcut_to_ptr(new_s0);
- edit->set[0].to = assoc_array_shortcut_to_ptr(new_s0);
new_s0->back_pointer = node->back_pointer;
new_s0->parent_slot = node->parent_slot;
+
+ /* A shortcut has to lead to a node, so a node at the end of a shortcut
+ * is replaced together with that shortcut. The new one holds the index
+ * key from the root, so it covers what the old one did.
+ */
+ if (node->back_pointer &&
+ assoc_array_ptr_is_shortcut(node->back_pointer)) {
+ struct assoc_array_shortcut *above =
+ assoc_array_ptr_to_shortcut(node->back_pointer);
+
+ new_s0->back_pointer = above->back_pointer;
+ new_s0->parent_slot = above->parent_slot;
+ edit->excised_meta[1] = node->back_pointer;
+ }
new_s0->next_node = assoc_array_node_to_ptr(new_n0);
new_n0->back_pointer = assoc_array_shortcut_to_ptr(new_s0);
new_n0->parent_slot = 0;
- new_n1->back_pointer = assoc_array_node_to_ptr(new_n0);
- new_n1->parent_slot = -1; /* Need to calculate this */
new_s0->skip_to_level = level = diff & ~ASSOC_ARRAY_LEVEL_STEP_MASK;
pr_devel("skip_to_level = %d [diff %d]\n", level, diff);
@@ -773,18 +762,104 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
/* This now reduces to a node splitting exercise for which we'll need
* to regenerate the disparity table.
*/
- for (i = 0; i < ASSOC_ARRAY_FAN_OUT; i++) {
- ptr = node->slots[i];
- base_seg = ops->get_object_key_chunk(assoc_array_ptr_to_leaf(ptr),
- level);
- base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK;
- edit->segment_cache[i] = base_seg & ASSOC_ARRAY_FAN_MASK;
- }
-
+ assoc_array_fill_segment_cache(edit, ops, node, level);
base_seg = ops->get_key_chunk(index_key, level);
base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK;
edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = base_seg & ASSOC_ARRAY_FAN_MASK;
- goto do_split_node;
+ return new_s0;
+}
+
+/*
+ * Handle insertion into a terminal node.
+ */
+static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const void *index_key,
+ struct assoc_array_walk_result *result)
+{
+ struct assoc_array_shortcut *new_s0;
+ struct assoc_array_node *node, *new_n0, *new_n1;
+ bool have_meta;
+ int level;
+
+ node = result->terminal_node.node;
+ level = result->terminal_node.level;
+
+ pr_devel("-->%s()\n", __func__);
+
+ /* We arrived at a node which doesn't have an onward node or shortcut
+ * pointer that we have to follow. This means that (a) the leaf we
+ * want must go here (either by insertion or replacement) or (b) we
+ * need to split this node and insert in one of the fragments.
+ */
+ if (assoc_array_insert_in_node(edit, ops, index_key, node)) {
+ pr_devel("<--%s() = ok [in node]\n", __func__);
+ return true;
+ }
+
+ /* The node has no spare slots - so we're either going to have to split
+ * it or insert another node before it.
+ *
+ * Whatever, we're going to need at least two new nodes - so allocate
+ * those now. We may also need a new shortcut, but we deal with that
+ * when we need it.
+ */
+ new_n0 = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL);
+ if (!new_n0)
+ return false;
+ edit->new_meta[0] = assoc_array_node_to_ptr(new_n0);
+ new_n1 = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL);
+ if (!new_n1)
+ return false;
+ edit->new_meta[1] = assoc_array_node_to_ptr(new_n1);
+
+ /* We need to find out how similar the leaves are. */
+ pr_devel("no spare slots\n");
+ have_meta = assoc_array_fill_segment_cache(edit, ops, node, level);
+ edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = result->terminal_node.slot;
+
+ if (!have_meta && assoc_array_all_one_segment(edit)) {
+ /* The node only contains leaves and all the leaves cluster in
+ * the same slot, so we need to insert a shortcut to skip the
+ * common portion of the key and split the node at the
+ * shortcut's far end. The shortcut goes wherever its back
+ * pointer says, which may be further up than the node.
+ */
+ new_s0 = assoc_array_insert_shortcut(edit, ops, index_key,
+ node, level, new_n0);
+ if (!new_s0)
+ return false;
+ edit->set[0].ptr = assoc_array_parent_link(edit->array,
+ new_s0->back_pointer,
+ new_s0->parent_slot);
+ edit->set[0].to = assoc_array_shortcut_to_ptr(new_s0);
+ } else {
+ /* We need to split the current node. The node must contain
+ * anything from a single leaf (in the one leaf case, this leaf
+ * will cluster with the new leaf) and the rest meta-pointers,
+ * to all leaves, some of which may cluster.
+ *
+ * We need to expel at least two leaves out of a set consisting
+ * of the leaves in the node and the new leaf. The current meta
+ * pointers can just be copied as they shouldn't cluster with
+ * any of the leaves. If all the old leaves cluster in one slot
+ * and the new leaf does not, they all go and the new leaf stays.
+ *
+ * We need a new node (n0) to replace the current one and a new
+ * node to take the expelled nodes (n1).
+ */
+ new_n0->back_pointer = node->back_pointer;
+ new_n0->parent_slot = node->parent_slot;
+ edit->set[0].ptr = assoc_array_parent_link(edit->array,
+ new_n0->back_pointer,
+ new_n0->parent_slot);
+ edit->set[0].to = assoc_array_node_to_ptr(new_n0);
+ }
+
+ assoc_array_split_node(edit, node, new_n0, new_n1);
+ edit->excised_meta[0] = assoc_array_node_to_ptr(node);
+ pr_devel("<--%s() = ok [split node]\n", __func__);
+ return true;
}
/*
@@ -843,7 +918,8 @@ static bool assoc_array_insert_mid_shortcut(struct assoc_array_edit *edit,
* zero length - otherwise we just connect the new node directly to the
* parent.
*/
- level += ASSOC_ARRAY_LEVEL_STEP;
+ if (shortcut->back_pointer)
+ level += ASSOC_ARRAY_LEVEL_STEP;
if (diff > level) {
pr_devel("pre-shortcut %d...%d\n", level, diff);
keylen = round_up(diff, ASSOC_ARRAY_KEY_CHUNK_SIZE);
@@ -866,9 +942,11 @@ static bool assoc_array_insert_mid_shortcut(struct assoc_array_edit *edit,
memcpy(new_s0->index_key, shortcut->index_key,
flex_array_size(new_s0, index_key, keylen));
- blank = ULONG_MAX << (diff & ASSOC_ARRAY_KEY_CHUNK_MASK);
- pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, diff, blank);
- new_s0->index_key[keylen - 1] &= ~blank;
+ if (diff & ASSOC_ARRAY_KEY_CHUNK_MASK) {
+ blank = ULONG_MAX << (diff & ASSOC_ARRAY_KEY_CHUNK_MASK);
+ pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, diff, blank);
+ new_s0->index_key[keylen - 1] &= ~blank;
+ }
} else {
pr_devel("no pre-shortcut\n");
edit->set[0].to = assoc_array_node_to_ptr(new_n0);
@@ -1484,17 +1562,26 @@ int assoc_array_gc(struct assoc_array *array,
cursor = array->root;
descend:
- /* If this point is a shortcut, then we need to duplicate it and
- * advance the target cursor.
+ /* The node at this position is duplicated, and so is the shortcut in
+ * front of it if there is one. The node is allocated first: the copy
+ * of the shortcut starts out pointing at the old node, and must not be
+ * in the new tree if the node cannot be allocated, or the cleanup
+ * would follow it into the old tree.
*/
+ new_n = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL);
+ if (!new_n)
+ goto enomem;
+
if (assoc_array_ptr_is_shortcut(cursor)) {
shortcut = assoc_array_ptr_to_shortcut(cursor);
keylen = round_up(shortcut->skip_to_level, ASSOC_ARRAY_KEY_CHUNK_SIZE);
keylen >>= ASSOC_ARRAY_KEY_CHUNK_SHIFT;
new_s = kmalloc(struct_size(new_s, index_key, keylen),
GFP_KERNEL);
- if (!new_s)
+ if (!new_s) {
+ kfree(new_n);
goto enomem;
+ }
pr_devel("dup shortcut %p -> %p\n", shortcut, new_s);
memcpy(new_s, shortcut, struct_size(new_s, index_key, keylen));
new_s->back_pointer = new_parent;
@@ -1504,11 +1591,7 @@ int assoc_array_gc(struct assoc_array *array,
cursor = shortcut->next_node;
}
- /* Duplicate the node at this position */
node = assoc_array_ptr_to_node(cursor);
- new_n = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL);
- if (!new_n)
- goto enomem;
pr_devel("dup node %p -> %p\n", node, new_n);
new_n->back_pointer = new_parent;
new_n->parent_slot = node->parent_slot;
diff --git a/lib/cmdline.c b/lib/cmdline.c
index 90ed997d9570..f5dc59b3548e 100644
--- a/lib/cmdline.c
+++ b/lib/cmdline.c
@@ -11,6 +11,7 @@
#include <linux/export.h>
#include <linux/kernel.h>
+#include <linux/overflow.h>
#include <linux/string.h>
#include <linux/ctype.h>
@@ -26,7 +27,9 @@ static int get_range(char **str, int *pint, int n)
(*str)++;
upper_range = simple_strtol((*str), NULL, 0);
- inc_counter = upper_range - *pint;
+ /* Keep the sign of the result when the difference doesn't fit */
+ if (check_sub_overflow(upper_range, *pint, &inc_counter))
+ inc_counter = upper_range < *pint ? -1 : INT_MAX;
for (x = *pint; n && x < upper_range; x++, n--)
*pint++ = x;
return inc_counter;
@@ -122,6 +125,14 @@ char *get_options(const char *str, int nints, int *ints)
range_nums = get_range((char **)&str, pint, n);
if (range_nums < 0)
break;
+ /* The range didn't fit, so the array is full */
+ if (!validate && range_nums > n) {
+ i = nints;
+ break;
+ }
+ /* Leave room for the upper number of the range */
+ if (range_nums >= INT_MAX - i)
+ break;
/*
* Decrement the result by one to leave out the
* last number in the range. The next iteration
diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig
index fa882108ba62..2f2edeaf9de3 100644
--- a/lib/crypto/Kconfig
+++ b/lib/crypto/Kconfig
@@ -142,7 +142,7 @@ config CRYPTO_LIB_SHA1_ARCH
depends on CRYPTO_LIB_SHA1 && !UML
default y if ARM
default y if ARM64 && KERNEL_MODE_NEON
- default y if MIPS && CPU_CAVIUM_OCTEON
+ default y if MIPS && CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN
default y if PPC
default y if S390
default y if SPARC64
@@ -160,7 +160,7 @@ config CRYPTO_LIB_SHA256_ARCH
depends on CRYPTO_LIB_SHA256 && !UML
default y if ARM && !CPU_V7M
default y if ARM64
- default y if MIPS && CPU_CAVIUM_OCTEON
+ default y if MIPS && CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN
default y if PPC && SPE
default y if RISCV && 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \
RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS
@@ -180,7 +180,7 @@ config CRYPTO_LIB_SHA512_ARCH
depends on CRYPTO_LIB_SHA512 && !UML
default y if ARM && !CPU_V7M
default y if ARM64
- default y if MIPS && CPU_CAVIUM_OCTEON
+ default y if MIPS && CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN
default y if RISCV && 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \
RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS
default y if S390
diff --git a/lib/raid6/avx2.c b/lib/raid6/avx2.c
index 059024234dce..2d03ce97f655 100644
--- a/lib/raid6/avx2.c
+++ b/lib/raid6/avx2.c
@@ -76,6 +76,7 @@ static void raid6_avx21_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -124,6 +125,7 @@ static void raid6_avx21_xor_syndrome(int disks, int start, int stop,
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -186,6 +188,7 @@ static void raid6_avx22_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -254,6 +257,7 @@ static void raid6_avx22_xor_syndrome(int disks, int start, int stop,
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -347,6 +351,7 @@ static void raid6_avx24_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -457,6 +462,7 @@ static void raid6_avx24_xor_syndrome(int disks, int start, int stop,
asm volatile("vmovntdq %%ymm14,%0" : "=m" (q[d+96]));
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
diff --git a/lib/raid6/avx512.c b/lib/raid6/avx512.c
index 009bd0adeebf..d989ca64847a 100644
--- a/lib/raid6/avx512.c
+++ b/lib/raid6/avx512.c
@@ -93,6 +93,7 @@ static void raid6_avx5121_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -152,6 +153,7 @@ static void raid6_avx5121_xor_syndrome(int disks, int start, int stop,
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -225,6 +227,7 @@ static void raid6_avx5122_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -309,6 +312,7 @@ static void raid6_avx5122_xor_syndrome(int disks, int start, int stop,
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -415,6 +419,7 @@ static void raid6_avx5124_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -548,6 +553,7 @@ static void raid6_avx5124_xor_syndrome(int disks, int start, int stop,
"m" (q[d+128]), "m" (q[d+192]));
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
const struct raid6_calls raid6_avx512x4 = {
diff --git a/lib/raid6/recov_avx2.c b/lib/raid6/recov_avx2.c
index 97d598d2535c..0ae0a621538a 100644
--- a/lib/raid6/recov_avx2.c
+++ b/lib/raid6/recov_avx2.c
@@ -180,6 +180,7 @@ static void raid6_2data_recov_avx2(int disks, size_t bytes, int faila,
#endif
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -297,6 +298,7 @@ static void raid6_datap_recov_avx2(int disks, size_t bytes, int faila,
#endif
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
diff --git a/lib/raid6/recov_avx512.c b/lib/raid6/recov_avx512.c
index 7986120ca444..32a9720be269 100644
--- a/lib/raid6/recov_avx512.c
+++ b/lib/raid6/recov_avx512.c
@@ -219,6 +219,7 @@ static void raid6_2data_recov_avx512(int disks, size_t bytes, int faila,
#endif
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -361,6 +362,7 @@ static void raid6_datap_recov_avx512(int disks, size_t bytes, int faila,
#endif
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
diff --git a/lib/tests/cmdline_kunit.c b/lib/tests/cmdline_kunit.c
index c1602f797637..74c4b5ea3225 100644
--- a/lib/tests/cmdline_kunit.c
+++ b/lib/tests/cmdline_kunit.c
@@ -139,11 +139,112 @@ static void cmdline_test_range(struct kunit *test)
} while (++i < ARRAY_SIZE(cmdline_test_range_strings));
}
+static const struct {
+ const char *in;
+ int parsed[4];
+ int validated;
+} cmdline_test_range_overflow_cases[] = {
+ { "1-100", { 3, 1, 2, 3, }, 100, },
+ { "1,5-100,7", { 3, 1, 5, 6, }, 98, },
+ { "1-2147483646", { 3, 1, 2, 3, }, 2147483646, },
+ { "0-2147483647", { 3, 0, 1, 2, }, 0, },
+ { "-5-2147483647", { 3, -5, -4, -3, }, 0, },
+ { "2147483647--5", { 0, 2147483647, }, 0, },
+ { "1,0-2147483647", { 3, 1, 0, 1, }, 1, },
+};
+
+static void cmdline_test_range_overflow(struct kunit *test)
+{
+ unsigned int i, j;
+
+ for (i = 0; i < ARRAY_SIZE(cmdline_test_range_overflow_cases); i++) {
+ const char *in = cmdline_test_range_overflow_cases[i].in;
+ const int *e = cmdline_test_range_overflow_cases[i].parsed;
+ /* Two guard elements past the array handed to get_options() */
+ int r[ARRAY_SIZE(cmdline_test_range_overflow_cases[0].parsed) + 2];
+ int n;
+
+ memset(r, 0, sizeof(r));
+ get_options(in, ARRAY_SIZE(r) - 2, r);
+ for (j = 0; j < ARRAY_SIZE(r) - 2; j++)
+ KUNIT_EXPECT_EQ_MSG(test, r[j], e[j], "Pattern: %s at %u", in, j);
+ for (; j < ARRAY_SIZE(r); j++)
+ KUNIT_EXPECT_EQ_MSG(test, r[j], 0, "Pattern: %s out of bound at %u", in, j);
+
+ get_options(in, 0, &n);
+ KUNIT_EXPECT_EQ_MSG(test, n, cmdline_test_range_overflow_cases[i].validated,
+ "Pattern: %s (validated)", in);
+ }
+}
+
+static void cmdline_test_next_arg_quoted_value(struct kunit *test)
+{
+ char in[] = "foo=\"bar baz\" qux=1";
+ char *next, *param, *val;
+
+ next = next_arg(in, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "foo");
+ KUNIT_ASSERT_NOT_NULL(test, val);
+ KUNIT_EXPECT_STREQ(test, val, "bar baz");
+ KUNIT_EXPECT_STREQ(test, next, "qux=1");
+
+ next = next_arg(next, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "qux");
+ KUNIT_ASSERT_NOT_NULL(test, val);
+ KUNIT_EXPECT_STREQ(test, val, "1");
+ KUNIT_EXPECT_STREQ(test, next, "");
+}
+
+static void cmdline_test_next_arg_bare_quote_regression(struct kunit *test)
+{
+ char in[] = "foo=bar \"";
+ char *next, *param, *val;
+
+ next = next_arg(in, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "foo");
+ KUNIT_ASSERT_NOT_NULL(test, val);
+ KUNIT_EXPECT_STREQ(test, val, "bar");
+ KUNIT_EXPECT_STREQ(test, next, "\"");
+
+ /* This hits the i == 0 quoted-token case fixed by 9847f21225c4. */
+ next = next_arg(next, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "");
+ KUNIT_EXPECT_PTR_EQ(test, val, NULL);
+ KUNIT_EXPECT_STREQ(test, next, "");
+}
+
+static void cmdline_test_next_arg_mixed_tokens(struct kunit *test)
+{
+ char in[] = "bbb= jjj kkk=\"a=b\"";
+ char *next, *param, *val;
+
+ next = next_arg(in, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "bbb");
+ KUNIT_ASSERT_NOT_NULL(test, val);
+ KUNIT_EXPECT_STREQ(test, val, "");
+ KUNIT_EXPECT_STREQ(test, next, "jjj kkk=\"a=b\"");
+
+ next = next_arg(next, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "jjj");
+ KUNIT_EXPECT_NULL(test, val);
+ KUNIT_EXPECT_STREQ(test, next, "kkk=\"a=b\"");
+
+ next = next_arg(next, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "kkk");
+ KUNIT_ASSERT_NOT_NULL(test, val);
+ KUNIT_EXPECT_STREQ(test, val, "a=b");
+ KUNIT_EXPECT_STREQ(test, next, "");
+}
+
static struct kunit_case cmdline_test_cases[] = {
KUNIT_CASE(cmdline_test_noint),
KUNIT_CASE(cmdline_test_lead_int),
KUNIT_CASE(cmdline_test_tail_int),
KUNIT_CASE(cmdline_test_range),
+ KUNIT_CASE(cmdline_test_range_overflow),
+ KUNIT_CASE(cmdline_test_next_arg_quoted_value),
+ KUNIT_CASE(cmdline_test_next_arg_bare_quote_regression),
+ KUNIT_CASE(cmdline_test_next_arg_mixed_tokens),
{}
};
diff --git a/mm/cma.c b/mm/cma.c
index 90d3ab262dae..f5c9da62aeb5 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -377,6 +377,8 @@ static int __init cma_fixed_reserve(phys_addr_t base, phys_addr_t size)
return -EBUSY;
}
+ accept_memory(base, size);
+
return 0;
}
@@ -674,6 +676,8 @@ int __init cma_declare_contiguous_multi(phys_addr_t total_size,
break;
}
+ accept_memory(mlp->base, size);
+
pr_debug("created region %d: %016llx - %016llx\n",
nr, (u64)mlp->base, (u64)mlp->base + size);
cmrp = &cma->ranges[nr++];
diff --git a/mm/damon/core.c b/mm/damon/core.c
index 56f2343ffefa..ec44433200c6 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -348,6 +348,8 @@ struct damos_quota_goal *damos_new_quota_goal(
return NULL;
goal->metric = metric;
goal->target_value = target_value;
+ if (metric == DAMOS_QUOTA_SOME_MEM_PSI_US)
+ goal->last_psi_total = U64_MAX;
INIT_LIST_HEAD(&goal->list);
return goal;
}
@@ -798,6 +800,9 @@ static void damos_commit_quota_goal_union(
struct damos_quota_goal *dst, struct damos_quota_goal *src)
{
switch (dst->metric) {
+ case DAMOS_QUOTA_SOME_MEM_PSI_US:
+ dst->last_psi_total = U64_MAX;
+ break;
case DAMOS_QUOTA_NODE_MEM_USED_BP:
case DAMOS_QUOTA_NODE_MEM_FREE_BP:
dst->nid = src->nid;
@@ -816,7 +821,6 @@ static int damos_commit_quota_goal(
dst->target_value = src->target_value;
if (dst->metric == DAMOS_QUOTA_USER_INPUT)
dst->current_value = src->current_value;
- /* keep last_psi_total as is, since it will be updated in next cycle */
damos_commit_quota_goal_union(dst, src);
return 0;
}
@@ -1700,7 +1704,7 @@ static void kdamond_tune_intervals(struct damon_ctx *c)
damon_set_attrs(c, &new_attrs);
}
-static void damon_split_region_at(struct damon_target *t,
+static int damon_split_region_at(struct damon_target *t,
struct damon_region *r, unsigned long sz_r);
static bool __damos_valid_target(struct damon_region *r, struct damos *s)
@@ -1808,13 +1812,14 @@ static void damos_update_stat(struct damos *s,
static bool damos_filter_match(struct damon_ctx *ctx, struct damon_target *t,
struct damon_region *r, struct damos_filter *filter,
- unsigned long min_sz_region)
+ unsigned long min_sz_region, int *err)
{
bool matched = false;
struct damon_target *ti;
int target_idx = 0;
unsigned long start, end;
+ *err = 0;
switch (filter->type) {
case DAMOS_FILTER_TYPE_TARGET:
damon_for_each_target(ti, ctx) {
@@ -1840,12 +1845,13 @@ static bool damos_filter_match(struct damon_ctx *ctx, struct damon_target *t,
}
/* start before the range and overlap */
if (r->ar.start < start) {
- damon_split_region_at(t, r, start - r->ar.start);
+ *err = damon_split_region_at(t, r,
+ start - r->ar.start);
matched = false;
break;
}
/* start inside the range */
- damon_split_region_at(t, r, end - r->ar.start);
+ *err = damon_split_region_at(t, r, end - r->ar.start);
matched = true;
break;
default:
@@ -1859,10 +1865,16 @@ static bool damos_filter_out(struct damon_ctx *ctx, struct damon_target *t,
struct damon_region *r, struct damos *s)
{
struct damos_filter *filter;
+ bool matching;
+ int err;
s->core_filters_allowed = false;
damos_for_each_filter(filter, s) {
- if (damos_filter_match(ctx, t, r, filter, ctx->min_sz_region)) {
+ matching = damos_filter_match(ctx, t, r, filter,
+ ctx->min_sz_region, &err);
+ if (err)
+ return true;
+ if (matching) {
if (filter->allow)
s->core_filters_allowed = true;
return !filter->allow;
@@ -2000,7 +2012,8 @@ static void damos_apply_scheme(struct damon_ctx *c, struct damon_target *t,
c->min_sz_region);
if (!sz)
goto update_stat;
- damon_split_region_at(t, r, sz);
+ if (damon_split_region_at(t, r, sz))
+ goto update_stat;
}
if (damos_filter_out(c, t, r, s))
return;
@@ -2178,7 +2191,12 @@ static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal)
break;
case DAMOS_QUOTA_SOME_MEM_PSI_US:
now_psi_total = damos_get_some_mem_psi_total();
- goal->current_value = now_psi_total - goal->last_psi_total;
+ /* uninitialized last_psi_total; make no effect this round */
+ if (goal->last_psi_total == U64_MAX)
+ goal->current_value = goal->target_value;
+ else
+ goal->current_value = now_psi_total -
+ goal->last_psi_total;
goal->last_psi_total = now_psi_total;
break;
case DAMOS_QUOTA_NODE_MEM_USED_BP:
@@ -2259,6 +2277,26 @@ static void damos_trace_esz(struct damon_ctx *c, struct damos *s,
trace_damos_esz(cidx, sidx, quota->esz);
}
+static void damos_fix_stale_charge_pointers(struct damon_ctx *c,
+ struct damos *s)
+{
+ struct damos_quota *quota;
+ struct damon_target *t;
+
+ quota = &s->quota;
+ if (!quota->charge_target_from)
+ return;
+ damon_for_each_target(t, c) {
+ if (t == quota->charge_target_from) {
+ if (damon_nr_regions(t) > 0)
+ return;
+ break;
+ }
+ }
+ quota->charge_target_from = NULL;
+ quota->charge_addr_from = 0;
+}
+
static void damos_adjust_quota(struct damon_ctx *c, struct damos *s)
{
struct damos_quota *quota = &s->quota;
@@ -2295,6 +2333,8 @@ static void damos_adjust_quota(struct damon_ctx *c, struct damos *s)
damos_trace_esz(c, s, quota);
}
+ damos_fix_stale_charge_pointers(c, s);
+
if (!c->ops.get_scheme_score)
return;
@@ -2462,15 +2502,17 @@ static void kdamond_merge_regions(struct damon_ctx *c, unsigned int threshold,
*
* r the region to be split
* sz_r size of the first sub-region that will be made
+ *
+ * Return: 0 on success, negative error code otherwise.
*/
-static void damon_split_region_at(struct damon_target *t,
+static int damon_split_region_at(struct damon_target *t,
struct damon_region *r, unsigned long sz_r)
{
struct damon_region *new;
new = damon_new_region(r->ar.start + sz_r, r->ar.end);
if (!new)
- return;
+ return -ENOMEM;
r->ar.end = new->ar.start;
@@ -2480,6 +2522,7 @@ static void damon_split_region_at(struct damon_target *t,
new->nr_accesses = r->nr_accesses;
damon_insert_region(new, r, damon_next_region(r), t);
+ return 0;
}
/* Split every region in the given target into 'nr_subs' regions */
@@ -2525,7 +2568,6 @@ static void kdamond_split_regions(struct damon_ctx *ctx)
{
struct damon_target *t;
unsigned int nr_regions = 0;
- static unsigned int last_nr_regions;
int nr_subregions = 2;
damon_for_each_target(t, ctx)
@@ -2535,14 +2577,14 @@ static void kdamond_split_regions(struct damon_ctx *ctx)
return;
/* Maybe the middle of the region has different access frequency */
- if (last_nr_regions == nr_regions &&
+ if (ctx->last_nr_regions == nr_regions &&
nr_regions < ctx->attrs.max_nr_regions / 3)
nr_subregions = 3;
damon_for_each_target(t, ctx)
damon_split_regions_of(t, nr_subregions, ctx->min_sz_region);
- last_nr_regions = nr_regions;
+ ctx->last_nr_regions = nr_regions;
}
/*
diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c
index e95cddca0a10..f90d079dc962 100644
--- a/mm/damon/sysfs-schemes.c
+++ b/mm/damon/sysfs-schemes.c
@@ -1072,7 +1072,10 @@ static ssize_t target_metric_store(struct kobject *kobj,
metric_name = &damos_sysfs_qgoal_metric_names[i];
if (sysfs_streq(buf, metric_name->name)) {
+ if (!mutex_trylock(&damon_sysfs_lock))
+ return -EBUSY;
goal->metric = metric_name->metric;
+ mutex_unlock(&damon_sysfs_lock);
return count;
}
}
@@ -2493,12 +2496,11 @@ static int damon_sysfs_add_scheme_filters(struct damos *scheme,
} else if (filter->type == DAMOS_FILTER_TYPE_TARGET) {
filter->target_idx = sysfs_filter->target_idx;
} else if (filter->type == DAMOS_FILTER_TYPE_HUGEPAGE_SIZE) {
- if (sysfs_filter->sz_range.min >
- sysfs_filter->sz_range.max) {
+ filter->sz_range = sysfs_filter->sz_range;
+ if (filter->sz_range.min > filter->sz_range.max) {
damos_destroy_filter(filter);
return -EINVAL;
}
- filter->sz_range = sysfs_filter->sz_range;
}
damos_add_filter(scheme, filter);
diff --git a/mm/damon/tests/core-kunit.h b/mm/damon/tests/core-kunit.h
index 0ce648131943..814197da4f50 100644
--- a/mm/damon/tests/core-kunit.h
+++ b/mm/damon/tests/core-kunit.h
@@ -548,6 +548,7 @@ static void damos_test_filter_out(struct kunit *test)
struct damon_target *t;
struct damon_region *r, *r2;
struct damos_filter *f;
+ int err;
f = damos_new_filter(DAMOS_FILTER_TYPE_ADDR, true, false);
if (!f)
@@ -570,28 +571,32 @@ static void damos_test_filter_out(struct kunit *test)
/* region in the range */
KUNIT_EXPECT_TRUE(test,
- damos_filter_match(NULL, t, r, f, DAMON_MIN_REGION));
+ damos_filter_match(NULL, t, r, f, DAMON_MIN_REGION,
+ &err));
KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1);
/* region before the range */
r->ar.start = DAMON_MIN_REGION * 1;
r->ar.end = DAMON_MIN_REGION * 2;
KUNIT_EXPECT_FALSE(test,
- damos_filter_match(NULL, t, r, f, DAMON_MIN_REGION));
+ damos_filter_match(NULL, t, r, f, DAMON_MIN_REGION,
+ &err));
KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1);
/* region after the range */
r->ar.start = DAMON_MIN_REGION * 6;
r->ar.end = DAMON_MIN_REGION * 8;
KUNIT_EXPECT_FALSE(test,
- damos_filter_match(NULL, t, r, f, DAMON_MIN_REGION));
+ damos_filter_match(NULL, t, r, f, DAMON_MIN_REGION,
+ &err));
KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1);
/* region started before the range */
r->ar.start = DAMON_MIN_REGION * 1;
r->ar.end = DAMON_MIN_REGION * 4;
KUNIT_EXPECT_FALSE(test,
- damos_filter_match(NULL, t, r, f, DAMON_MIN_REGION));
+ damos_filter_match(NULL, t, r, f, DAMON_MIN_REGION,
+ &err));
/* filter should have split the region */
KUNIT_EXPECT_EQ(test, r->ar.start, DAMON_MIN_REGION * 1);
KUNIT_EXPECT_EQ(test, r->ar.end, DAMON_MIN_REGION * 2);
@@ -607,7 +612,8 @@ static void damos_test_filter_out(struct kunit *test)
r->ar.start = DAMON_MIN_REGION * 2;
r->ar.end = DAMON_MIN_REGION * 8;
KUNIT_EXPECT_TRUE(test,
- damos_filter_match(NULL, t, r, f, DAMON_MIN_REGION));
+ damos_filter_match(NULL, t, r, f, DAMON_MIN_REGION,
+ &err));
/* filter should have split the region */
KUNIT_EXPECT_EQ(test, r->ar.start, DAMON_MIN_REGION * 2);
KUNIT_EXPECT_EQ(test, r->ar.end, DAMON_MIN_REGION * 6);
diff --git a/mm/filemap.c b/mm/filemap.c
index 1a0358810609..cd7af384cff2 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -1383,10 +1383,22 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr,
* return value based on that state without races.
*
* Also note that WQ_FLAG_WOKEN is sufficient for a non-exclusive
- * waiter, but an exclusive one requires WQ_FLAG_DONE.
+ * waiter, but an exclusive one must also acquire the bit.
*/
- if (behavior == EXCLUSIVE)
- return wait->flags & WQ_FLAG_DONE ? 0 : -EINTR;
+ if (behavior == EXCLUSIVE) {
+ if (wait->flags & WQ_FLAG_DONE)
+ return 0;
+ /*
+ * We might have just been woken up and the waker will not
+ * wake up anybody else. If we ignore that now, the remaining
+ * waiters may not get woken up: take the bit now if it is
+ * still free.
+ */
+ if ((wait->flags & WQ_FLAG_WOKEN) &&
+ !test_and_set_bit(bit_nr, folio_flags(folio, 0)))
+ return 0;
+ return -EINTR;
+ }
return wait->flags & WQ_FLAG_WOKEN ? 0 : -EINTR;
}
@@ -3210,11 +3222,11 @@ loff_t mapping_seek_hole_data(struct address_space *mapping, loff_t start,
* @folio - the folio to lock.
* @fpin - the pointer to the file we may pin (or is already pinned).
*
- * This works similar to lock_folio_or_retry in that it can drop the
- * mmap_lock. It differs in that it actually returns the folio locked
- * if it returns 1 and 0 if it couldn't lock the folio. If we did have
- * to drop the mmap_lock then fpin will point to the pinned file and
- * needs to be fput()'ed at a later point.
+ * Returns 1 with the folio locked, or 0 if locking failed.
+ *
+ * On both entry and return, a non-NULL *fpin means the fault lock has
+ * already been dropped and the caller must eventually fput() the file.
+ * Otherwise the fault lock is still held, including on failure.
*/
static int lock_folio_maybe_drop_mmap(struct vm_fault *vmf, struct folio *folio,
struct file **fpin)
@@ -3231,21 +3243,9 @@ static int lock_folio_maybe_drop_mmap(struct vm_fault *vmf, struct folio *folio,
return 0;
*fpin = maybe_unlock_mmap_for_io(vmf, *fpin);
- if (vmf->flags & FAULT_FLAG_KILLABLE) {
- if (__folio_lock_killable(folio)) {
- /*
- * We didn't have the right flags to drop the
- * fault lock, but all fault_handlers only check
- * for fatal signals if we return VM_FAULT_RETRY,
- * so we need to drop the fault lock here and
- * return 0 if we don't have a fpin.
- */
- if (*fpin == NULL)
- release_fault_lock(vmf);
- return 0;
- }
- } else
- __folio_lock(folio);
+ if (vmf->flags & FAULT_FLAG_KILLABLE)
+ return !__folio_lock_killable(folio);
+ __folio_lock(folio);
return 1;
}
@@ -3447,7 +3447,7 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf)
* vma->vm_mm->mmap_lock must be held on entry.
*
* If our return value has VM_FAULT_RETRY set, it's because the mmap_lock
- * may be dropped before doing I/O or by lock_folio_maybe_drop_mmap().
+ * may be dropped before doing I/O or when aborting an interrupted wait.
*
* If our return value does not have VM_FAULT_RETRY set, the mmap_lock
* has not been released.
@@ -3603,9 +3603,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf)
out_retry:
/*
- * We dropped the mmap_lock, we need to return to the fault handler to
- * re-find the vma and come back and find our hopefully still populated
- * page.
+ * With fpin set, the fault lock was already dropped. Otherwise it is
+ * still held. Release invalidate_lock before dropping either protection.
*/
if (!IS_ERR(folio))
folio_put(folio);
@@ -3613,6 +3612,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf)
filemap_invalidate_unlock_shared(mapping);
if (fpin)
fput(fpin);
+ else if (!(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
+ release_fault_lock(vmf);
return ret | VM_FAULT_RETRY;
}
EXPORT_SYMBOL(filemap_fault);
diff --git a/mm/gup.c b/mm/gup.c
index 2264a36bc773..c345bca01e3a 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -658,9 +658,6 @@ static struct page *follow_huge_pud(struct vm_area_struct *vma,
assert_spin_locked(pud_lockptr(mm, pudp));
- if (!pud_present(pud))
- return NULL;
-
if ((flags & FOLL_WRITE) &&
!can_follow_write_pud(pud, pfn_to_page(pfn), vma, flags))
return NULL;
@@ -896,15 +893,17 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
}
static struct page *follow_pmd_mask(struct vm_area_struct *vma,
- unsigned long address, pud_t *pudp,
- unsigned int flags,
- unsigned long *page_mask)
+ unsigned long address, pud_t *pudp, pud_t pud,
+ unsigned int flags, unsigned long *page_mask)
{
pmd_t *pmd, pmdval;
spinlock_t *ptl;
struct page *page;
struct mm_struct *mm = vma->vm_mm;
+ if (unlikely(pud_bad(pud)))
+ return no_page_table(vma, flags, address);
+
pmd = pmd_offset(pudp, address);
pmdval = pmdp_get_lockless(pmd);
if (pmd_none(pmdval))
@@ -953,18 +952,23 @@ static struct page *follow_pud_mask(struct vm_area_struct *vma,
pud = pudp_get(pudp);
if (!pud_present(pud))
return no_page_table(vma, flags, address);
- if (pud_leaf(pud)) {
- ptl = pud_lock(mm, pudp);
- page = follow_huge_pud(vma, address, pudp, flags, page_mask);
+ if (!pud_leaf(pud))
+ return follow_pmd_mask(vma, address, pudp, pud, flags, page_mask);
+
+ ptl = pud_lock(mm, pudp);
+ pud = *pudp;
+ if (unlikely(!pud_present(pud))) {
spin_unlock(ptl);
- if (page)
- return page;
return no_page_table(vma, flags, address);
}
- if (unlikely(pud_bad(pud)))
- return no_page_table(vma, flags, address);
+ if (unlikely(!pud_leaf(pud))) {
+ spin_unlock(ptl);
+ return follow_pmd_mask(vma, address, pudp, pud, flags, page_mask);
+ }
- return follow_pmd_mask(vma, address, pudp, flags, page_mask);
+ page = follow_huge_pud(vma, address, pudp, flags, page_mask);
+ spin_unlock(ptl);
+ return page;
}
static struct page *follow_p4d_mask(struct vm_area_struct *vma,
@@ -3203,6 +3207,9 @@ static int gup_fast_fallback(unsigned long start, unsigned long nr_pages,
FOLL_PCI_P2PDMA | FOLL_HONOR_NUMA_FAULT)))
return -EINVAL;
+ if (unlikely(!nr_pages))
+ return 0;
+
if (gup_flags & FOLL_PIN)
mm_set_has_pinned_flag(current->mm);
@@ -3210,8 +3217,8 @@ static int gup_fast_fallback(unsigned long start, unsigned long nr_pages,
might_lock_read(¤t->mm->mmap_lock);
start = untagged_addr(start) & PAGE_MASK;
- len = nr_pages << PAGE_SHIFT;
- if (check_add_overflow(start, len, &end))
+ if (check_shl_overflow(nr_pages, PAGE_SHIFT, &len) ||
+ check_add_overflow(start, len, &end))
return -EOVERFLOW;
if (end > TASK_SIZE_MAX)
return -EFAULT;
@@ -3456,7 +3463,7 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
{
unsigned int flags, nr_folios, nr_found;
unsigned int i, pgshift = PAGE_SHIFT;
- pgoff_t start_idx, end_idx;
+ pgoff_t start_idx, end_idx, retry_idx;
struct folio *folio = NULL;
struct folio_batch fbatch;
struct hstate *h;
@@ -3497,6 +3504,7 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
* reason, we try to allocate them and add them to the
* page cache.
*/
+ retry_idx = start_idx;
nr_found = filemap_get_folios_contig(memfd->f_mapping,
&start_idx,
end_idx,
@@ -3509,12 +3517,38 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
for (i = 0; i < nr_found; i++) {
folio = fbatch.folios[i];
+ /*
+ * The folio might have been removed from the
+ * page cache (e.g., truncated or collapsed)
+ * after the lookup. Recheck under the folio
+ * lock.
+ */
+ folio_lock(folio);
+ if (unlikely(folio->mapping != memfd->f_mapping)) {
+ folio_unlock(folio);
+ start_idx = retry_idx;
+ break;
+ }
+
if (try_grab_folio(folio, 1, FOLL_PIN)) {
+ folio_unlock(folio);
folio_batch_release(&fbatch);
ret = -EINVAL;
goto err;
}
+ /*
+ * fallocate() can leave folios in the
+ * page cache that were never cleared.
+ */
+ if (!folio_test_uptodate(folio)) {
+ folio_zero_range(folio, 0, folio_size(folio));
+ flush_dcache_folio(folio);
+ folio_mark_uptodate(folio);
+ }
+ retry_idx = folio_next_index(folio);
+ folio_unlock(folio);
+
if (nr_folios == 0)
*offset = offset_in_folio(folio, start);
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index df4a3e6215d4..24f5887eb48e 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2822,6 +2822,8 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma,
spinlock_t *ptl;
pud_t orig_pud;
+ tlb_change_page_size(tlb, HPAGE_PUD_SIZE);
+
ptl = __pud_trans_huge_lock(pud, vma);
if (!ptl)
return 0;
@@ -3208,16 +3210,66 @@ void split_huge_pmd_address(struct vm_area_struct *vma, unsigned long address,
__split_huge_pmd(vma, pmd, address, freeze);
}
-static inline void split_huge_pmd_if_needed(struct vm_area_struct *vma, unsigned long address)
+static bool pud_split_required(const struct vm_area_struct *vma,
+ unsigned long address)
{
+ if (!IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD))
+ return false;
+ if (IS_ALIGNED(address, HPAGE_PUD_SIZE))
+ return false;
/*
- * If the new address isn't hpage aligned and it could previously
- * contain an hugepage: check if we need to split an huge pmd.
+ * Only selected VMAs support mapping PUDs for now. Lift the
+ * restriction once this changes.
*/
- if (!IS_ALIGNED(address, HPAGE_PMD_SIZE) &&
- range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PMD_SIZE),
- ALIGN(address, HPAGE_PMD_SIZE)))
- split_huge_pmd_address(vma, address, false);
+ if (!vma_is_dax(vma) && !(vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)))
+ return false;
+ return range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PUD_SIZE),
+ ALIGN(address, HPAGE_PUD_SIZE));
+}
+
+static bool pmd_split_required(const struct vm_area_struct *vma,
+ unsigned long address)
+{
+ if (IS_ALIGNED(address, HPAGE_PMD_SIZE))
+ return false;
+
+ return range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PMD_SIZE),
+ ALIGN(address, HPAGE_PMD_SIZE));
+}
+
+static void split_huge_leaf(struct vm_area_struct *vma,
+ unsigned long address)
+{
+ const bool split_pud = pud_split_required(vma, address);
+ const bool split_pmd = pmd_split_required(vma, address);
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+
+ /*
+ * mmap, VMA and rmap write locks protect the walk; the split helpers
+ * take the page table locks.
+ */
+ if (!split_pud && !split_pmd)
+ return;
+
+ pgd = pgd_offset(vma->vm_mm, address);
+ if (!pgd_present(*pgd))
+ return;
+
+ p4d = p4d_offset(pgd, address);
+ if (!p4d_present(*p4d))
+ return;
+
+ pud = pud_offset(p4d, address);
+ if (pud_leaf(*pud))
+ __split_huge_pud(vma, pud, address);
+ if (!split_pmd || !pud_present(*pud))
+ return;
+
+ pmd = pmd_offset(pud, address);
+ __split_huge_pmd(vma, pmd, address, false);
}
void vma_adjust_trans_huge(struct vm_area_struct *vma,
@@ -3226,14 +3278,14 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,
struct vm_area_struct *next)
{
/* Check if we need to split start first. */
- split_huge_pmd_if_needed(vma, start);
+ split_huge_leaf(vma, start);
/* Check if we need to split end next. */
- split_huge_pmd_if_needed(vma, end);
+ split_huge_leaf(vma, end);
/* If we're incrementing next->vm_start, we might need to split it. */
if (next)
- split_huge_pmd_if_needed(next, end);
+ split_huge_leaf(next, end);
}
static void unmap_folio(struct folio *folio)
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 42aace6e3ca1..a8da4cb7ad10 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -25,6 +25,7 @@
#include <linux/mmdebug.h>
#include <linux/sched/signal.h>
#include <linux/rmap.h>
+#include <linux/rcupdate.h>
#include <linux/string_choices.h>
#include <linux/string_helpers.h>
#include <linux/swap.h>
@@ -126,6 +127,10 @@ static int __huge_pmd_unshare(struct mmu_gather *tlb,
bool check_locks);
static void hugetlb_unshare_pmds(struct vm_area_struct *vma,
unsigned long start, unsigned long end, bool take_locks);
+static void hugetlb_unmap_range(struct mmu_gather *tlb,
+ struct vm_area_struct *vma, unsigned long start,
+ unsigned long end, struct folio *folio,
+ zap_flags_t zap_flags);
static struct resv_map *vma_resv_map(struct vm_area_struct *vma);
static void hugetlb_free_folio(struct folio *folio)
@@ -3120,7 +3125,7 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
}
}
- ret = mem_cgroup_charge_hugetlb(folio, gfp);
+ ret = mem_cgroup_charge_hugetlb(folio, vma->vm_mm, gfp);
/*
* Unconditionally increment NR_HUGETLB here. If it turns out that
* mem_cgroup_charge_hugetlb failed, then immediately free the page and
@@ -3129,8 +3134,12 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
lruvec_stat_mod_folio(folio, NR_HUGETLB, pages_per_huge_page(h));
if (ret == -ENOMEM) {
- free_huge_folio(folio);
- return ERR_PTR(-ENOMEM);
+ folio_put(folio);
+ /*
+ * Return -ENOSPC, since retrying the fault is futile:
+ * the OOM killer is not triggered for HugeTLB.
+ */
+ return ERR_PTR(-ENOSPC);
}
return folio;
@@ -5625,10 +5634,11 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
raw_write_seqcount_begin(&src->write_protect_seq);
} else {
/*
- * For shared mappings the vma lock must be held before
- * calling hugetlb_walk() in the src vma. Otherwise, the
- * returned ptep could go away if part of a shared pmd and
- * another thread calls huge_pmd_unshare.
+ * Shared mappings only need copying when we could have
+ * information stored in page tables (e.g., userfaultfd PTE
+ * bits or markers), which is incompatible with PMD sharing.
+ * Still take the hugetlb VMA lock for now to satisfy
+ * hugetlb_walk()'s locking requirements.
*/
hugetlb_vma_lock_read(src_vma);
}
@@ -5636,6 +5646,13 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
last_addr_mask = hugetlb_mask_last_page(h);
for (addr = src_vma->vm_start; addr < src_vma->vm_end; addr += sz) {
spinlock_t *src_ptl, *dst_ptl;
+
+ if (WARN_ON_ONCE(want_pmd_share(src_vma, addr) ||
+ want_pmd_share(dst_vma, addr))) {
+ ret = -EINVAL;
+ break;
+ }
+
src_pte = hugetlb_walk(src_vma, addr, sz);
if (!src_pte) {
addr |= last_addr_mask;
@@ -5647,14 +5664,6 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
break;
}
-#ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING
- /* If the pagetables are shared, there is nothing to do */
- if (ptdesc_pmd_is_shared(virt_to_ptdesc(dst_pte))) {
- addr |= last_addr_mask;
- continue;
- }
-#endif
-
dst_ptl = huge_pte_lock(h, dst, dst_pte);
src_ptl = huge_pte_lockptr(h, src, src_pte);
spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING);
@@ -5828,10 +5837,9 @@ static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr,
spin_unlock(dst_ptl);
}
-int move_hugetlb_page_tables(struct vm_area_struct *vma,
- struct vm_area_struct *new_vma,
- unsigned long old_addr, unsigned long new_addr,
- unsigned long len)
+unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma,
+ struct vm_area_struct *new_vma, unsigned long old_addr,
+ unsigned long new_addr, unsigned long len)
{
struct hstate *h = hstate_vma(vma);
struct address_space *mapping = vma->vm_file->f_mapping;
@@ -5861,6 +5869,8 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma,
for (; old_addr < old_end; old_addr += sz, new_addr += sz) {
const unsigned long offset_to_last_entry =
(old_addr | last_addr_mask) - old_addr;
+ const unsigned long offset_to_last_dst_entry =
+ (new_addr | last_addr_mask) - new_addr;
src_pte = hugetlb_walk(vma, old_addr, sz);
if (!src_pte) {
@@ -5877,6 +5887,24 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma,
continue;
}
+ /*
+ * Avoid entering the destination PMD sharing path while holding
+ * i_mmap_rwsem for write. Leave destination page table
+ * population to the fault path, which can share the PMD lazily.
+ */
+ if (sz == PMD_SIZE && want_pmd_share(new_vma, new_addr)) {
+ const unsigned long unmap_size =
+ min(old_end - old_addr,
+ offset_to_last_dst_entry + sz);
+
+ hugetlb_unmap_range(&tlb, vma, old_addr,
+ old_addr + unmap_size, NULL,
+ ZAP_FLAG_DROP_MARKER);
+ old_addr += unmap_size - sz;
+ new_addr += unmap_size - sz;
+ continue;
+ }
+
dst_pte = huge_pte_alloc(mm, new_vma, new_addr, sz);
if (!dst_pte)
break;
@@ -5896,9 +5924,10 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma,
return len + old_addr - old_end;
}
-void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
- unsigned long start, unsigned long end,
- struct folio *folio, zap_flags_t zap_flags)
+static void hugetlb_unmap_range(struct mmu_gather *tlb,
+ struct vm_area_struct *vma, unsigned long start,
+ unsigned long end, struct folio *folio,
+ zap_flags_t zap_flags)
{
struct mm_struct *mm = vma->vm_mm;
const bool folio_provided = !!folio;
@@ -5916,13 +5945,6 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
BUG_ON(start & ~huge_page_mask(h));
BUG_ON(end & ~huge_page_mask(h));
- /*
- * This is a hugetlb vma, all the pte entries should point
- * to huge page.
- */
- tlb_change_page_size(tlb, sz);
- tlb_start_vma(tlb, vma);
-
last_addr_mask = hugetlb_mask_last_page(h);
address = start;
for (; address < end; address += sz) {
@@ -6051,6 +6073,15 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
if (folio_provided)
break;
}
+}
+
+void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
+ unsigned long start, unsigned long end,
+ struct folio *folio, zap_flags_t zap_flags)
+{
+ tlb_change_page_size(tlb, huge_page_size(hstate_vma(vma)));
+ tlb_start_vma(tlb, vma);
+ hugetlb_unmap_range(tlb, vma, start, end, folio, zap_flags);
tlb_end_vma(tlb, vma);
huge_pmd_unshare_flush(tlb, vma);
@@ -6129,6 +6160,8 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,
struct address_space *mapping;
pgoff_t pgoff;
+ VM_WARN_ON_ONCE(!folio_test_anon(folio));
+
/*
* vm_pgoff is in PAGE_SIZE units, hence the different calculation
* from page cache lookup which is in HPAGE_SIZE units.
@@ -6164,10 +6197,17 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,
* could insert a zeroed page instead of the data existing
* from the time of fork. This would look like data corruption
*/
- if (!is_vma_resv_set(iter_vma, HPAGE_RESV_OWNER))
- unmap_hugepage_range(iter_vma, address,
- address + huge_page_size(h),
- folio, 0);
+ if (is_vma_resv_set(iter_vma, HPAGE_RESV_OWNER))
+ continue;
+
+ /*
+ * The address in other VMAs can differ. Even though our old
+ * folio is anon, we always operate on the underlying hugetlb
+ * file.
+ */
+ address = vma_address(iter_vma, pgoff, 1);
+ unmap_hugepage_range(iter_vma, address,
+ address + huge_page_size(h), folio, 0);
}
i_mmap_unlock_write(mapping);
}
@@ -7931,11 +7971,36 @@ void folio_putback_hugetlb(struct folio *folio)
folio_put(folio);
}
-void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio, int reason)
+static void move_hugetlb_lruvec_stat(struct folio *old_folio,
+ struct folio *new_folio)
+{
+ struct mem_cgroup *memcg;
+ long nr_pages = folio_nr_pages(old_folio);
+ int old_nid = folio_nid(old_folio);
+ int new_nid = folio_nid(new_folio);
+
+ if (old_nid == new_nid)
+ return;
+
+ guard(rcu)();
+
+ memcg = folio_memcg(new_folio);
+ if (!memcg)
+ return;
+
+ mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(old_nid)),
+ NR_HUGETLB, -nr_pages);
+ mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(new_nid)),
+ NR_HUGETLB, nr_pages);
+}
+
+void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio,
+ enum migrate_reason reason)
{
struct hstate *h = folio_hstate(old_folio);
hugetlb_cgroup_migrate(old_folio, new_folio);
+ move_hugetlb_lruvec_stat(old_folio, new_folio);
folio_set_owner_migrate_reason(new_folio, reason);
/*
diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c
index 795e7dbbca7d..efe4bcc85da2 100644
--- a/mm/hugetlb_vmemmap.c
+++ b/mm/hugetlb_vmemmap.c
@@ -45,6 +45,8 @@ struct vmemmap_remap_walk {
#define VMEMMAP_SPLIT_NO_TLB_FLUSH BIT(0)
/* Skip the TLB flush when we remap the PTE */
#define VMEMMAP_REMAP_NO_TLB_FLUSH BIT(1)
+/* synchronize_rcu() to avoid writes from page_ref_add_unless() */
+#define VMEMMAP_SYNCHRONIZE_RCU BIT(2)
unsigned long flags;
};
@@ -405,6 +407,9 @@ static int __hugetlb_vmemmap_restore_folio(const struct hstate *h,
if (!folio_test_hugetlb_vmemmap_optimized(folio))
return 0;
+ if (flags & VMEMMAP_SYNCHRONIZE_RCU)
+ synchronize_rcu();
+
vmemmap_start = (unsigned long)&folio->page;
vmemmap_end = vmemmap_start + hugetlb_vmemmap_size(h);
@@ -437,7 +442,7 @@ static int __hugetlb_vmemmap_restore_folio(const struct hstate *h,
*/
int hugetlb_vmemmap_restore_folio(const struct hstate *h, struct folio *folio)
{
- return __hugetlb_vmemmap_restore_folio(h, folio, 0);
+ return __hugetlb_vmemmap_restore_folio(h, folio, VMEMMAP_SYNCHRONIZE_RCU);
}
/**
@@ -460,11 +465,14 @@ long hugetlb_vmemmap_restore_folios(const struct hstate *h,
struct folio *folio, *t_folio;
long restored = 0;
long ret = 0;
- unsigned long flags = VMEMMAP_REMAP_NO_TLB_FLUSH;
+ unsigned long flags = VMEMMAP_REMAP_NO_TLB_FLUSH | VMEMMAP_SYNCHRONIZE_RCU;
list_for_each_entry_safe(folio, t_folio, folio_list, lru) {
if (folio_test_hugetlb_vmemmap_optimized(folio)) {
ret = __hugetlb_vmemmap_restore_folio(h, folio, flags);
+ /* only need to synchronize_rcu() once for each batch */
+ flags &= ~VMEMMAP_SYNCHRONIZE_RCU;
+
if (ret)
break;
restored++;
@@ -513,6 +521,8 @@ static int __hugetlb_vmemmap_optimize_folio(const struct hstate *h,
static_branch_inc(&hugetlb_optimize_vmemmap_key);
+ if (flags & VMEMMAP_SYNCHRONIZE_RCU)
+ synchronize_rcu();
/*
* Very Subtle
* If VMEMMAP_REMAP_NO_TLB_FLUSH is set, TLB flushing is not performed
@@ -572,7 +582,7 @@ void hugetlb_vmemmap_optimize_folio(const struct hstate *h, struct folio *folio)
{
LIST_HEAD(vmemmap_pages);
- __hugetlb_vmemmap_optimize_folio(h, folio, &vmemmap_pages, 0);
+ __hugetlb_vmemmap_optimize_folio(h, folio, &vmemmap_pages, VMEMMAP_SYNCHRONIZE_RCU);
free_vmemmap_page_list(&vmemmap_pages);
}
@@ -600,7 +610,7 @@ static void __hugetlb_vmemmap_optimize_folios(struct hstate *h,
struct folio *folio;
int nr_to_optimize;
LIST_HEAD(vmemmap_pages);
- unsigned long flags = VMEMMAP_REMAP_NO_TLB_FLUSH;
+ unsigned long flags = VMEMMAP_REMAP_NO_TLB_FLUSH | VMEMMAP_SYNCHRONIZE_RCU;
nr_to_optimize = 0;
list_for_each_entry(folio, folio_list, lru) {
@@ -654,6 +664,8 @@ static void __hugetlb_vmemmap_optimize_folios(struct hstate *h,
int ret;
ret = __hugetlb_vmemmap_optimize_folio(h, folio, &vmemmap_pages, flags);
+ /* only need to synchronize_rcu() once for each batch */
+ flags &= ~VMEMMAP_SYNCHRONIZE_RCU;
/*
* Pages to be freed may have been accumulated. If we
diff --git a/mm/internal.h b/mm/internal.h
index ef42eee87c7b..e0adafc45c8c 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -594,6 +594,8 @@ struct alloc_context {
*/
enum zone_type highest_zoneidx;
bool spread_dirty_pages;
+ /* Set by __alloc_pages_slowpath() before reclaim and compaction */
+ unsigned int reclaim_compact_order;
};
/*
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index cc3a09a0c259..6bff1096de7d 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1555,7 +1555,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr,
result = SCAN_FAIL;
start_pte = pte_offset_map_lock(mm, pmd, haddr, &ptl);
- if (!start_pte) /* mmap_lock + page lock should prevent this */
+ if (!start_pte)
goto drop_folio;
/* step 1: check all mapped PTEs are to the right huge page */
@@ -1586,6 +1586,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr,
}
pte_unmap_unlock(start_pte, ptl);
+ ptl = NULL;
mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, mm,
haddr, haddr + HPAGE_PMD_SIZE);
mmu_notifier_invalidate_range_start(&range);
@@ -1603,7 +1604,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr,
pml = pmd_lock(mm, pmd);
start_pte = pte_offset_map_rw_nolock(mm, pmd, haddr, &pgt_pmd, &ptl);
- if (!start_pte) /* mmap_lock + page lock should prevent this */
+ if (!start_pte)
goto abort;
if (!pml)
spin_lock(ptl);
@@ -1645,9 +1646,10 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr,
/*
* Must clear entry, or a racing truncate may re-remove it.
* TLB flush can be left until pmdp_collapse_flush() does it.
- * PTE dirty? Shmem page is already dirty; file is read-only.
*/
- clear_ptes(mm, addr, pte, nr_batch_ptes);
+ ptent = get_and_clear_ptes(mm, addr, pte, nr_batch_ptes);
+ if (pte_dirty(ptent))
+ folio_mark_dirty(folio);
folio_remove_rmap_ptes(folio, page, nr_batch_ptes, vma);
nr_mapped_ptes += nr_batch_ptes;
}
@@ -1664,12 +1666,11 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr,
/* step 4: remove empty page table */
if (!pml) {
pml = pmd_lock(mm, pmd);
- if (ptl != pml) {
+ if (ptl != pml)
spin_lock_nested(ptl, SINGLE_DEPTH_NESTING);
- if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) {
- flush_tlb_mm(mm);
- goto unlock;
- }
+ if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) {
+ flush_tlb_mm(mm);
+ goto unlock;
}
}
pgt_pmd = pmdp_collapse_flush(vma, haddr, pmd);
@@ -2068,6 +2069,11 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr,
index += folio_nr_pages(folio);
continue;
out_unlock:
+ /*
+ * The folio may have been unmapped with TTU_BATCH_FLUSH.
+ * Flush before releasing the lock and our last reference.
+ */
+ try_to_unmap_flush();
folio_unlock(folio);
folio_put(folio);
goto xa_unlocked;
@@ -2078,9 +2084,8 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr,
xa_unlocked:
/*
- * If collapse is successful, flush must be done now before copying.
- * If collapse is unsuccessful, does flush actually need to be done?
- * Do it anyway, to clear the state.
+ * Flush before copying the folios, or releasing them in rollback.
+ * This is a no-op if out_unlock already flushed the batch.
*/
try_to_unmap_flush();
diff --git a/mm/ksm.c b/mm/ksm.c
index 80274ace8155..bcaca6e99b1c 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -469,6 +469,7 @@ static void advisor_stop_scan(void)
#ifdef CONFIG_NUMA
/* Zeroed when merging across nodes is not allowed */
static unsigned int ksm_merge_across_nodes = 1;
+/* Number of allocated stable and unstable tree roots. */
static int ksm_nr_node_ids = 1;
#else
#define ksm_merge_across_nodes 1U
@@ -1414,6 +1415,10 @@ static int replace_page(struct vm_area_struct *vma, struct page *page,
dec_mm_counter(mm, MM_ANONPAGES);
}
+ /* Preserve soft-dirty when replacing the page. */
+ if (pte_soft_dirty(orig_pte))
+ newpte = pte_mksoft_dirty(newpte);
+
flush_cache_page(vma, addr, pte_pfn(ptep_get(ptep)));
/*
* No need to notify as we are replacing a read only page with another
@@ -3568,14 +3573,13 @@ static ssize_t merge_across_nodes_store(struct kobject *kobj,
else {
root_stable_tree = buf;
root_unstable_tree = buf + nr_node_ids;
+ ksm_nr_node_ids = nr_node_ids;
/* Stable tree is empty but not the unstable */
root_unstable_tree[0] = one_unstable_tree[0];
}
}
- if (!err) {
+ if (!err)
ksm_merge_across_nodes = knob;
- ksm_nr_node_ids = knob ? 1 : nr_node_ids;
- }
}
mutex_unlock(&ksm_thread_mutex);
diff --git a/mm/madvise.c b/mm/madvise.c
index 729d8f2f07fd..bd0a5093a2df 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -755,6 +755,15 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
if (folio_test_swapcache(folio) || folio_test_dirty(folio)) {
if (!folio_trylock(folio))
continue;
+ /*
+ * KSM folios can gain new mappings at any time, so we must
+ * not clear their dirty flag.
+ */
+ if (folio_test_ksm(folio)) {
+ folio_unlock(folio);
+ continue;
+ }
+
/*
* If we have a large folio at this point, we know it is
* fully mapped so if its mapcount is the same as its
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 2aa741316b49..3ce613dab616 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -604,7 +604,7 @@ static void __mem_cgroup_flush_stats(struct mem_cgroup *memcg, bool force)
return;
if (mem_cgroup_is_root(memcg))
- WRITE_ONCE(flush_last_time, jiffies_64);
+ WRITE_ONCE(flush_last_time, get_jiffies_64());
css_rstat_flush(&memcg->css);
}
@@ -632,7 +632,7 @@ void mem_cgroup_flush_stats(struct mem_cgroup *memcg)
void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg)
{
/* Only flush if the periodic flusher is one full cycle late */
- if (time_after64(jiffies_64, READ_ONCE(flush_last_time) + 2*FLUSH_TIME))
+ if (time_after64(get_jiffies_64(), READ_ONCE(flush_last_time) + 2 * FLUSH_TIME))
mem_cgroup_flush_stats(memcg);
}
@@ -678,21 +678,12 @@ static int memcg_state_val_in_pages(int idx, int val)
return max(val * unit / PAGE_SIZE, 1UL);
}
-/**
- * mod_memcg_state - update cgroup memory statistics
- * @memcg: the memory cgroup
- * @idx: the stat item - can be enum memcg_stat_item or enum node_stat_item
- * @val: delta to add to the counter, can be negative
- */
-void mod_memcg_state(struct mem_cgroup *memcg, enum memcg_stat_item idx,
- int val)
+static void __mod_memcg_state(struct mem_cgroup *memcg,
+ enum memcg_stat_item idx, int val)
{
int i = memcg_stats_index(idx);
int cpu;
- if (mem_cgroup_disabled())
- return;
-
if (WARN_ONCE(BAD_STAT_IDX(i), "%s: missing stat item %d\n", __func__, idx))
return;
@@ -706,6 +697,21 @@ void mod_memcg_state(struct mem_cgroup *memcg, enum memcg_stat_item idx,
put_cpu();
}
+/**
+ * mod_memcg_state - update cgroup memory statistics
+ * @memcg: the memory cgroup
+ * @idx: the stat item - can be enum memcg_stat_item or enum node_stat_item
+ * @val: delta to add to the counter, can be negative
+ */
+void mod_memcg_state(struct mem_cgroup *memcg, enum memcg_stat_item idx,
+ int val)
+{
+ if (mem_cgroup_disabled())
+ return;
+
+ __mod_memcg_state(memcg, idx, val);
+}
+
#ifdef CONFIG_MEMCG_V1
/* idx can be of type enum memcg_stat_item or node_stat_item. */
unsigned long memcg_page_state_local(struct mem_cgroup *memcg, int idx)
@@ -725,21 +731,16 @@ unsigned long memcg_page_state_local(struct mem_cgroup *memcg, int idx)
}
#endif
-static void mod_memcg_lruvec_state(struct lruvec *lruvec,
- enum node_stat_item idx,
- int val)
+static void __mod_memcg_lruvec_state(struct mem_cgroup_per_node *pn,
+ enum node_stat_item idx, int val)
{
- struct mem_cgroup_per_node *pn;
- struct mem_cgroup *memcg;
+ struct mem_cgroup *memcg = pn->memcg;
int i = memcg_stats_index(idx);
int cpu;
if (WARN_ONCE(BAD_STAT_IDX(i), "%s: missing stat item %d\n", __func__, idx))
return;
- pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec);
- memcg = pn->memcg;
-
cpu = get_cpu();
/* Update memcg */
@@ -755,6 +756,16 @@ static void mod_memcg_lruvec_state(struct lruvec *lruvec,
put_cpu();
}
+void mod_memcg_lruvec_state(struct lruvec *lruvec,
+ enum node_stat_item idx, int val)
+{
+ struct mem_cgroup_per_node *pn;
+
+ pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec);
+
+ __mod_memcg_lruvec_state(pn, idx, val);
+}
+
/**
* __mod_lruvec_state - update lruvec memory statistics
* @lruvec: the lruvec
@@ -2062,7 +2073,10 @@ static void high_work_func(struct work_struct *work)
struct mem_cgroup *memcg;
memcg = container_of(work, struct mem_cgroup, high_work);
+ if (!css_tryget_online(&memcg->css))
+ return;
reclaim_high(memcg, MEMCG_CHARGE_BATCH, GFP_KERNEL);
+ css_put(&memcg->css);
}
/*
@@ -3512,7 +3526,7 @@ void mem_cgroup_flush_foreign(struct bdi_writeback *wb)
{
struct mem_cgroup *memcg = mem_cgroup_from_css(wb->memcg_css);
unsigned long intv = msecs_to_jiffies(dirty_expire_interval * 10);
- u64 now = jiffies_64;
+ u64 now = get_jiffies_64();
int i;
for (i = 0; i < MEMCG_CGWB_FRN_CNT; i++) {
@@ -3906,6 +3920,8 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css)
page_counter_set_min(&memcg->memory, 0);
page_counter_set_low(&memcg->memory, 0);
+ page_counter_set_high(&memcg->memory, PAGE_COUNTER_MAX);
+ page_counter_set_high(&memcg->swap, PAGE_COUNTER_MAX);
zswap_memcg_offline_cleanup(memcg);
@@ -4732,6 +4748,51 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root,
page_counter_calculate_protection(&root->memory, &memcg->memory, recursive_protection);
}
+#ifdef CONFIG_LRU_GEN
+/**
+ * mem_cgroup_calculate_protection_path - compute protection along a path
+ * @root: the top ancestor of the sub-tree being checked (NULL for root_mem_cgroup)
+ * @memcg: the target memory cgroup
+ *
+ * Walk the ancestor path from @root down to @memcg and compute the effective
+ * protection at each level. This is safe for isolated queries because it
+ * ensures parents are computed before children.
+ */
+void mem_cgroup_calculate_protection_path(struct mem_cgroup *root,
+ struct mem_cgroup *memcg)
+{
+ bool recursive_protection =
+ cgrp_dfl_root.flags & CGRP_ROOT_MEMORY_RECURSIVE_PROT;
+ struct cgroup *cg;
+ int root_level, i;
+
+ if (mem_cgroup_disabled())
+ return;
+
+ if (!root)
+ root = root_mem_cgroup;
+
+ if (memcg == root)
+ return;
+
+ root_level = root->css.cgroup->level;
+ cg = memcg->css.cgroup;
+
+ rcu_read_lock();
+ for (i = root_level + 1; i <= cg->level; i++) {
+ struct mem_cgroup *cur;
+
+ cur = mem_cgroup_from_css(cgroup_css(cg->ancestors[i],
+ &memory_cgrp_subsys));
+ if (cur)
+ page_counter_calculate_protection(&root->memory,
+ &cur->memory,
+ recursive_protection);
+ }
+ rcu_read_unlock();
+}
+#endif /* CONFIG_LRU_GEN */
+
static int charge_memcg(struct folio *folio, struct mem_cgroup *memcg,
gfp_t gfp)
{
@@ -4763,6 +4824,7 @@ int __mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, gfp_t gfp)
/**
* mem_cgroup_charge_hugetlb - charge the memcg for a hugetlb folio
* @folio: folio being charged
+ * @mm: mm context of the allocation target
* @gfp: reclaim mode
*
* This function is called when allocating a huge page folio, after the page has
@@ -4772,9 +4834,10 @@ int __mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, gfp_t gfp)
* Returns ENOMEM if the memcg is already full.
* Returns 0 if either the charge was successful, or if we skip the charging.
*/
-int mem_cgroup_charge_hugetlb(struct folio *folio, gfp_t gfp)
+int mem_cgroup_charge_hugetlb(struct folio *folio, struct mm_struct *mm,
+ gfp_t gfp)
{
- struct mem_cgroup *memcg = get_mem_cgroup_from_current();
+ struct mem_cgroup *memcg = get_mem_cgroup_from_mm(mm);
int ret = 0;
/*
diff --git a/mm/memfd.c b/mm/memfd.c
index f7ba1e9dea49..d09157cd486a 100644
--- a/mm/memfd.c
+++ b/mm/memfd.c
@@ -82,22 +82,31 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx)
struct hstate *h = hstate_file(memfd);
int err = -ENOMEM;
long nr_resv;
+ u32 hash;
gfp_mask = htlb_alloc_mask(h);
gfp_mask &= ~(__GFP_HIGHMEM | __GFP_MOVABLE);
idx >>= huge_page_order(h);
+ /*
+ * Serialize hugepage allocation and instantiation to prevent
+ * races with concurrent allocations, as required by all other
+ * callers of hugetlb_add_to_page_cache().
+ */
+ hash = hugetlb_fault_mutex_hash(memfd->f_mapping, idx);
+ mutex_lock(&hugetlb_fault_mutex_table[hash]);
+
nr_resv = hugetlb_reserve_pages(inode, idx, idx + 1, NULL, 0);
- if (nr_resv < 0)
- return ERR_PTR(nr_resv);
+ if (nr_resv < 0) {
+ err = nr_resv;
+ goto out_unlock;
+ }
folio = alloc_hugetlb_folio_reserve(h,
numa_node_id(),
NULL,
gfp_mask);
if (folio) {
- u32 hash;
-
/*
* Zero the folio to prevent information leaks to userspace.
* Use folio_zero_user() which is optimized for huge/gigantic
@@ -112,20 +121,9 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx)
*/
__folio_mark_uptodate(folio);
- /*
- * Serialize hugepage allocation and instantiation to prevent
- * races with concurrent allocations, as required by all other
- * callers of hugetlb_add_to_page_cache().
- */
- hash = hugetlb_fault_mutex_hash(memfd->f_mapping, idx);
- mutex_lock(&hugetlb_fault_mutex_table[hash]);
-
err = hugetlb_add_to_page_cache(folio,
memfd->f_mapping,
idx);
-
- mutex_unlock(&hugetlb_fault_mutex_table[hash]);
-
if (err) {
folio_put(folio);
goto err_unresv;
@@ -133,11 +131,14 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx)
hugetlb_set_folio_subpool(folio, subpool_inode(inode));
folio_unlock(folio);
+ mutex_unlock(&hugetlb_fault_mutex_table[hash]);
return folio;
}
err_unresv:
if (nr_resv > 0)
hugetlb_unreserve_pages(inode, idx, idx + 1, 0);
+out_unlock:
+ mutex_unlock(&hugetlb_fault_mutex_table[hash]);
return ERR_PTR(err);
}
#endif
@@ -472,6 +473,8 @@ static struct file *alloc_file(const char *name, unsigned int flags)
if (IS_ERR(file))
return file;
file->f_mode |= FMODE_LSEEK | FMODE_PREAD | FMODE_PWRITE;
+ /* a regular file like any other: POSIX.1-2008/SUSv4 Section XSI 2.9.7 */
+ file->f_mode |= FMODE_ATOMIC_POS;
file->f_flags |= O_LARGEFILE;
if (flags & MFD_NOEXEC_SEAL) {
diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c
index 864811fff409..fb3d63be1b89 100644
--- a/mm/memory-tiers.c
+++ b/mm/memory-tiers.c
@@ -561,21 +561,14 @@ static struct memory_tier *set_node_memory_tier(int node)
return memtier;
}
-static void destroy_memory_tier(struct memory_tier *memtier)
+static struct memory_tier *clear_node_memory_tier(int node)
{
- list_del(&memtier->list);
- device_unregister(&memtier->dev);
-}
-
-static bool clear_node_memory_tier(int node)
-{
- bool cleared = false;
pg_data_t *pgdat;
struct memory_tier *memtier;
pgdat = NODE_DATA(node);
if (!pgdat)
- return false;
+ return NULL;
/*
* Make sure that anybody looking at NODE_DATA who finds
@@ -593,14 +586,10 @@ static bool clear_node_memory_tier(int node)
synchronize_rcu();
memtype = node_memory_types[node].memtype;
node_clear(node, memtype->nodes);
- if (nodes_empty(memtype->nodes)) {
+ if (nodes_empty(memtype->nodes))
list_del_init(&memtype->tier_sibling);
- if (list_empty(&memtier->memory_types))
- destroy_memory_tier(memtier);
- }
- cleared = true;
}
- return cleared;
+ return memtier;
}
static void release_memtype(struct kref *kref)
@@ -873,13 +862,21 @@ static int __meminit memtier_hotplug_callback(struct notifier_block *self,
{
struct memory_tier *memtier;
struct node_notify *nn = _arg;
+ bool empty = false;
switch (action) {
case NODE_REMOVED_LAST_MEMORY:
mutex_lock(&memory_tier_lock);
- if (clear_node_memory_tier(nn->nid))
+ memtier = clear_node_memory_tier(nn->nid);
+ if (memtier) {
+ empty = list_empty(&memtier->memory_types);
+ if (empty)
+ list_del(&memtier->list);
establish_demotion_targets();
+ }
mutex_unlock(&memory_tier_lock);
+ if (empty)
+ device_unregister(&memtier->dev);
break;
case NODE_ADDED_FIRST_MEMORY:
mutex_lock(&memory_tier_lock);
diff --git a/mm/memory.c b/mm/memory.c
index 01d14ca0e74d..e37ae367c8c1 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -1877,6 +1877,15 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,
}
} while (pte += nr, addr += PAGE_SIZE * nr, addr != end);
+ add_mm_rss_vec(mm, rss);
+ arch_leave_lazy_mmu_mode();
+
+ /* Do the actual TLB flush before dropping ptl */
+ if (force_flush) {
+ tlb_flush_mmu_tlbonly(tlb);
+ tlb_flush_rmaps(tlb, vma);
+ }
+
/*
* Fast path: try to hold the pmd lock and unmap the PTE page.
*
@@ -1884,18 +1893,12 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,
* to hold the pmd lock failed, then we need to recheck all pte entries
* to ensure they are still none, thereby preventing the pte entries
* from being repopulated by another thread.
+ *
+ * Clear the PMD after the rmaps for the table have been removed, in case
+ * an incorrect mapcount is observed after unmapping the folio.
*/
if (can_reclaim_pt && direct_reclaim && addr == end)
direct_reclaim = try_get_and_clear_pmd(mm, pmd, &pmdval);
-
- add_mm_rss_vec(mm, rss);
- arch_leave_lazy_mmu_mode();
-
- /* Do the actual TLB flush before dropping ptl */
- if (force_flush) {
- tlb_flush_mmu_tlbonly(tlb);
- tlb_flush_rmaps(tlb, vma);
- }
pte_unmap_unlock(start_pte, ptl);
/*
@@ -1942,19 +1945,26 @@ static inline unsigned long zap_pmd_range(struct mmu_gather *tlb,
addr = next;
continue;
}
+ /* We already synchronized with concurrent zapping. */
+ if (pmd_none(*pmd)) {
+ addr = next;
+ continue;
+ }
/* fall through */
- } else if (details && details->single_folio &&
- folio_test_pmd_mappable(details->single_folio) &&
- next - addr == HPAGE_PMD_SIZE && pmd_none(*pmd)) {
- spinlock_t *ptl = pmd_lock(tlb->mm, pmd);
- /*
- * Take and drop THP pmd lock so that we cannot return
- * prematurely, while zap_huge_pmd() has cleared *pmd,
- * but not yet decremented compound_mapcount().
- */
- spin_unlock(ptl);
- }
- if (pmd_none(*pmd)) {
+ } else if (pmd_none(*pmd)) {
+ if (details && details->single_folio) {
+ spinlock_t *ptl = pmd_lock(tlb->mm, pmd);
+
+ /*
+ * zap_huge_pmd() may have cleared the PMD but not
+ * yet updated the folio mapcount, and
+ * try_get_and_clear_pmd() clears the PMD only after
+ * the rmaps for the table have been removed, but
+ * nothing orders the pmd_none() read against later
+ * mapcount reads without this synchronization.
+ */
+ spin_unlock(ptl);
+ }
addr = next;
continue;
}
@@ -3387,6 +3397,13 @@ static inline int __wp_page_copy_user(struct page *dst, struct page *src,
return 0;
}
+ /*
+ * __copy_from_user_inatomic() only works as expected for local faults.
+ * Reject remote faults.
+ */
+ if (unlikely(mm != current->mm))
+ return -EFAULT;
+
/*
* If the source page was a PFN mapping, we don't have
* a "struct page" for it. We do a best-effort copy by
@@ -3712,6 +3729,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
folio_put(old_folio);
delayacct_wpcopy_end();
+ if (err == -EFAULT)
+ return VM_FAULT_SIGBUS;
return err == -EHWPOISON ? VM_FAULT_HWPOISON : 0;
}
kmsan_copy_page_meta(&new_folio->page, vmf->page);
@@ -4268,8 +4287,8 @@ EXPORT_SYMBOL_GPL(unmap_mapping_pages);
void unmap_mapping_range(struct address_space *mapping,
loff_t const holebegin, loff_t const holelen, int even_cows)
{
- pgoff_t hba = (pgoff_t)(holebegin) >> PAGE_SHIFT;
- pgoff_t hlen = ((pgoff_t)(holelen) + PAGE_SIZE - 1) >> PAGE_SHIFT;
+ pgoff_t hba = (u64)holebegin >> PAGE_SHIFT;
+ pgoff_t hlen = ((u64)holelen + PAGE_SIZE - 1) >> PAGE_SHIFT;
/* Check for overflow. */
if (sizeof(holelen) > sizeof(hlen)) {
@@ -6532,8 +6551,12 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
lru_gen_exit_fault();
- /* If the mapping is droppable, then errors due to OOM aren't fatal. */
- if (is_droppable)
+ /*
+ * If the mapping is droppable, then errors due to OOM aren't fatal: the
+ * task returns to user mode, handles its signals and tries again. A
+ * fault from kernel mode has no such return, so it must see the error.
+ */
+ if (is_droppable && (flags & FAULT_FLAG_USER))
ret &= ~VM_FAULT_OOM;
if (flags & FAULT_FLAG_USER) {
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 919a505cabae..5a9743c2c00f 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -646,7 +646,8 @@ static inline bool queue_folio_required(struct folio *folio,
return node_isset(nid, *qp->nmask) == !(flags & MPOL_MF_INVERT);
}
-static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
+static void queue_folios_pmd(pmd_t *pmd, unsigned long addr,
+ struct mm_walk *walk)
{
struct folio *folio;
struct queue_pages *qp = walk->private;
@@ -655,11 +656,9 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
qp->nr_failed++;
return;
}
- folio = pmd_folio(*pmd);
- if (is_huge_zero_folio(folio)) {
- walk->action = ACTION_CONTINUE;
+ folio = vm_normal_folio_pmd(walk->vma, addr, *pmd);
+ if (!folio || folio_is_zone_device(folio))
return;
- }
if (!queue_folio_required(folio, qp))
return;
if (!(qp->flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) ||
@@ -692,7 +691,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr,
ptl = pmd_trans_huge_lock(pmd, vma);
if (ptl) {
- queue_folios_pmd(pmd, walk);
+ queue_folios_pmd(pmd, addr, walk);
spin_unlock(ptl);
goto out;
}
@@ -3269,11 +3268,11 @@ void __init numa_policy_init(void)
policy_cache = kmem_cache_create("numa_policy",
sizeof(struct mempolicy),
- 0, SLAB_PANIC, NULL);
+ 0, SLAB_PANIC | SLAB_ACCOUNT, NULL);
sn_cache = kmem_cache_create("shared_policy_node",
sizeof(struct sp_node),
- 0, SLAB_PANIC, NULL);
+ 0, SLAB_PANIC | SLAB_ACCOUNT, NULL);
for_each_node(nid) {
preferred_node_policy[nid] = (struct mempolicy) {
diff --git a/mm/migrate.c b/mm/migrate.c
index 08f138231802..98d3c9c789f0 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -1465,7 +1465,7 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
static int unmap_and_move_huge_page(new_folio_t get_new_folio,
free_folio_t put_new_folio, unsigned long private,
struct folio *src, int force, enum migrate_mode mode,
- int reason, struct list_head *ret)
+ enum migrate_reason reason, struct list_head *ret)
{
struct folio *dst;
int rc = -EAGAIN;
@@ -1622,7 +1622,7 @@ struct migrate_pages_stats {
*/
static int migrate_hugetlbs(struct list_head *from, new_folio_t get_new_folio,
free_folio_t put_new_folio, unsigned long private,
- enum migrate_mode mode, int reason,
+ enum migrate_mode mode, enum migrate_reason reason,
struct migrate_pages_stats *stats,
struct list_head *ret_folios)
{
@@ -1712,7 +1712,7 @@ static int migrate_hugetlbs(struct list_head *from, new_folio_t get_new_folio,
static void migrate_folios_move(struct list_head *src_folios,
struct list_head *dst_folios,
free_folio_t put_new_folio, unsigned long private,
- enum migrate_mode mode, int reason,
+ enum migrate_mode mode, enum migrate_reason reason,
struct list_head *ret_folios,
struct migrate_pages_stats *stats,
int *retry, int *thp_retry, int *nr_failed,
@@ -1795,7 +1795,7 @@ static void migrate_folios_undo(struct list_head *src_folios,
*/
static int migrate_pages_batch(struct list_head *from,
new_folio_t get_new_folio, free_folio_t put_new_folio,
- unsigned long private, enum migrate_mode mode, int reason,
+ unsigned long private, enum migrate_mode mode, enum migrate_reason reason,
struct list_head *ret_folios, struct list_head *split_folios,
struct migrate_pages_stats *stats, int nr_pass)
{
@@ -2007,7 +2007,7 @@ static int migrate_pages_batch(struct list_head *from,
static int migrate_pages_sync(struct list_head *from, new_folio_t get_new_folio,
free_folio_t put_new_folio, unsigned long private,
- enum migrate_mode mode, int reason,
+ enum migrate_mode mode, enum migrate_reason reason,
struct list_head *ret_folios, struct list_head *split_folios,
struct migrate_pages_stats *stats)
{
@@ -2084,7 +2084,7 @@ static int migrate_pages_sync(struct list_head *from, new_folio_t get_new_folio,
*/
int migrate_pages(struct list_head *from, new_folio_t get_new_folio,
free_folio_t put_new_folio, unsigned long private,
- enum migrate_mode mode, int reason, unsigned int *ret_succeeded)
+ enum migrate_mode mode, enum migrate_reason reason, unsigned int *ret_succeeded)
{
int rc, rc_gather;
int nr_pages;
@@ -2448,7 +2448,7 @@ static void do_pages_stat_array(struct mm_struct *mm, unsigned long nr_pages,
mmap_read_lock(mm);
for (i = 0; i < nr_pages; i++) {
- unsigned long addr = (unsigned long)(*pages);
+ unsigned long addr = (unsigned long)untagged_addr_remote(mm, *pages);
struct vm_area_struct *vma;
struct folio_walk fw;
struct folio *folio;
@@ -2737,6 +2737,7 @@ int migrate_misplaced_folio(struct folio *folio, int node)
pg_data_t *pgdat = NODE_DATA(node);
int nr_remaining;
unsigned int nr_succeeded;
+ int src_node = folio_nid(folio);
LIST_HEAD(migratepages);
struct mem_cgroup *memcg = get_mem_cgroup_from_folio(folio);
struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat);
@@ -2751,7 +2752,7 @@ int migrate_misplaced_folio(struct folio *folio, int node)
count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded);
count_memcg_events(memcg, NUMA_PAGE_MIGRATE, nr_succeeded);
if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING)
- && !node_is_toptier(folio_nid(folio))
+ && !node_is_toptier(src_node)
&& node_is_toptier(node))
mod_lruvec_state(lruvec, PGPROMOTE_SUCCESS, nr_succeeded);
}
diff --git a/mm/mm_init.c b/mm/mm_init.c
index a27ba1c1a719..4e00624e3c7b 100644
--- a/mm/mm_init.c
+++ b/mm/mm_init.c
@@ -2524,8 +2524,6 @@ void __init memblock_free_pages(struct page *page, unsigned long pfn,
return;
}
- /* pages were reserved and not allocated */
- clear_page_tag_ref(page);
__free_pages_core(page, order, MEMINIT_EARLY);
}
diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c
index 8e0125dc0522..c1b566243da9 100644
--- a/mm/mmu_notifier.c
+++ b/mm/mmu_notifier.c
@@ -522,16 +522,25 @@ int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range)
{
struct mmu_notifier_subscriptions *subscriptions =
range->mm->notifier_subscriptions;
- int ret;
+ const bool has_itree = subscriptions->has_itree;
+ int ret = 0;
- if (subscriptions->has_itree) {
+ if (has_itree) {
ret = mn_itree_invalidate(subscriptions, range);
if (ret)
return ret;
}
if (!hlist_empty(&subscriptions->list))
- return mn_hlist_invalidate_range_start(subscriptions, range);
- return 0;
+ ret = mn_hlist_invalidate_range_start(subscriptions, range);
+
+ /*
+ * On error, the non-blocking caller is not allowed to call
+ * invalidate_range_end(), as in mn_itree_invalidate().
+ */
+ if (ret && has_itree && !mmu_notifier_range_blockable(range))
+ mn_itree_inv_end(subscriptions);
+
+ return ret;
}
static void
@@ -892,7 +901,6 @@ static int __mmu_interval_notifier_insert(
struct mmu_notifier_subscriptions *subscriptions, unsigned long start,
unsigned long length, const struct mmu_interval_notifier_ops *ops)
{
- interval_sub->mm = mm;
interval_sub->ops = ops;
RB_CLEAR_NODE(&interval_sub->interval_tree.rb);
interval_sub->interval_tree.start = start;
@@ -909,8 +917,12 @@ static int __mmu_interval_notifier_insert(
if (WARN_ON(atomic_read(&mm->mm_users) <= 0))
return -EINVAL;
- /* pairs with mmdrop in mmu_interval_notifier_remove() */
+ /*
+ * Pairs with mmdrop in mmu_interval_notifier_remove(). Drivers take a
+ * non-NULL ->mm for "registered", so set it only with the reference.
+ */
mmgrab(mm);
+ interval_sub->mm = mm;
/*
* If some invalidate_range_start/end region is going on in parallel
diff --git a/mm/mremap.c b/mm/mremap.c
index 8ed7c50e499b..9f7f45281b48 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -87,7 +87,7 @@ static pud_t *get_old_pud(struct mm_struct *mm, unsigned long addr)
return NULL;
pud = pud_offset(p4d, addr);
- if (pud_none_or_clear_bad(pud))
+ if (!pud_trans_huge(*pud) && pud_none_or_clear_bad(pud))
return NULL;
return pud;
@@ -448,14 +448,18 @@ static bool move_normal_pud(struct pagetable_move_control *pmc,
if (new_ptl != old_ptl)
spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
- /* Clear the pud */
pud = *old_pud;
+ if (unlikely(pud_none(pud)))
+ goto out_unlock;
+
+ /* Clear the pud */
pud_clear(old_pud);
VM_BUG_ON(!pud_none(*new_pud));
pud_populate(mm, new_pud, pud_pgtable(pud));
flush_tlb_range(vma, pmc->old_addr, pmc->old_addr + PUD_SIZE);
+out_unlock:
if (new_ptl != old_ptl)
spin_unlock(new_ptl);
spin_unlock(old_ptl);
@@ -490,14 +494,15 @@ static bool move_huge_pud(struct pagetable_move_control *pmc,
* We don't have to worry about the ordering of src and dst
* ptlocks because exclusive mmap_lock prevents deadlock.
*/
- old_ptl = pud_lock(mm, old_pud);
+ old_ptl = __pud_trans_huge_lock(old_pud, vma);
+ if (unlikely(!old_ptl))
+ return false;
new_ptl = pud_lockptr(mm, new_pud);
if (new_ptl != old_ptl)
spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
/* Clear the pud */
- pud = *old_pud;
- pud_clear(old_pud);
+ pud = pudp_huge_get_and_clear(mm, pmc->old_addr, old_pud);
VM_BUG_ON(!pud_none(*new_pud));
@@ -829,11 +834,10 @@ unsigned long move_page_tables(struct pagetable_move_control *pmc)
if (!new_pud)
break;
if (pud_trans_huge(*old_pud)) {
- if (extent == HPAGE_PUD_SIZE) {
- move_pgt_entry(pmc, HPAGE_PUD, old_pud, new_pud);
- /* We ignore and continue on error? */
+ if (extent == HPAGE_PUD_SIZE &&
+ move_pgt_entry(pmc, HPAGE_PUD, old_pud, new_pud))
continue;
- }
+ split_huge_pud(pmc->old, old_pud, pmc->old_addr);
} else if (IS_ENABLED(CONFIG_HAVE_MOVE_PUD) && extent == PUD_SIZE) {
if (move_pgt_entry(pmc, NORMAL_PUD, old_pud, new_pud))
continue;
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 757bc4d3b5b5..17b12b4c4b66 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -902,6 +902,7 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc,
{
struct wb_domain *dom = dtc_dom(dtc);
struct bdi_writeback *wb = dtc->wb;
+ u64 threshold = thresh;
u64 wb_thresh;
u64 wb_max_thresh;
unsigned long numerator, denominator;
@@ -913,13 +914,14 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc,
fprop_fraction_percpu(&dom->completions, dtc->wb_completions,
&numerator, &denominator);
- wb_thresh = (thresh * (100 * BDI_RATIO_SCALE - bdi_min_ratio)) / (100 * BDI_RATIO_SCALE);
+ wb_thresh = div_u64(threshold * (100 * BDI_RATIO_SCALE - bdi_min_ratio),
+ 100 * BDI_RATIO_SCALE);
wb_thresh *= numerator;
wb_thresh = div64_ul(wb_thresh, denominator);
wb_min_max_ratio(wb, &wb_min_ratio, &wb_max_ratio);
- wb_thresh += (thresh * wb_min_ratio) / (100 * BDI_RATIO_SCALE);
+ wb_thresh += div_u64(threshold * wb_min_ratio, 100 * BDI_RATIO_SCALE);
/*
* It's very possible that wb_thresh is close to 0 not because the
@@ -935,7 +937,7 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc,
wb_thresh = max(wb_thresh, (thresh - dtc->dirty) / 8);
}
- wb_max_thresh = thresh * wb_max_ratio / (100 * BDI_RATIO_SCALE);
+ wb_max_thresh = div_u64(threshold * wb_max_ratio, 100 * BDI_RATIO_SCALE);
if (wb_thresh > wb_max_thresh)
wb_thresh = wb_max_thresh;
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 8bdc35efad26..2685e570bf7d 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1662,6 +1662,9 @@ void __meminit __free_pages_core(struct page *page, unsigned int order,
atomic_long_add(nr_pages, &page_zone(page)->managed_pages);
}
+ /* pages were never allocated */
+ clear_page_tag_ref(page);
+
if (page_contains_unaccepted(page, order)) {
if (order == MAX_PAGE_ORDER && __free_unaccepted(page))
return;
@@ -4130,6 +4133,12 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
return page;
}
+static inline bool order_defrag_promoted(unsigned int order,
+ const struct alloc_context *ac)
+{
+ return ac->reclaim_compact_order != order;
+}
+
/*
* Maximum number of compaction retries with a progress before OOM
* killer is consider as the only way to move forward.
@@ -4152,22 +4161,7 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
.order = order,
.page = NULL,
};
- int compact_order = order;
-
- /*
- * If fallbacks are not permitted (defrag_mode), we either
- * need to reclaim space in a block of matching type, or clear
- * out an entire block to allow __rmqueue_claim() to convert.
- *
- * Reclaim by itself is primarily freeing space in movable
- * blocks, since that's where the LRU pages live. So this
- * works for movable requests, but not for others.
- *
- * For those, promote the order to help make blocks, instead
- * of spinning in reclaim alone unproductively.
- */
- if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
- compact_order = max(order, pageblock_order);
+ int compact_order = ac->reclaim_compact_order;
if (!compact_order)
return NULL;
@@ -4258,7 +4252,7 @@ should_compact_retry(struct alloc_context *ac, int order, int alloc_flags,
int retries = *compaction_retries;
enum compact_priority priority = *compact_priority;
- if (!order)
+ if (!ac->reclaim_compact_order)
return false;
if (fatal_signal_pending(current))
@@ -4267,9 +4261,11 @@ should_compact_retry(struct alloc_context *ac, int order, int alloc_flags,
/*
* Compaction was skipped due to a lack of free order-0
* migration targets. Continue if reclaim can help.
+ * Promoted requests fall back instead.
*/
if (compact_result == COMPACT_SKIPPED) {
- ret = compaction_zonelist_suitable(ac, order, alloc_flags);
+ if (!order_defrag_promoted(order, ac))
+ ret = compaction_zonelist_suitable(ac, order, alloc_flags);
goto out;
}
@@ -4467,11 +4463,7 @@ __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order,
struct page *page = NULL;
unsigned long pflags;
bool drained = false;
- int reclaim_order = order;
-
- /* Match the slowpath compaction promotion in __alloc_pages_direct_compact */
- if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
- reclaim_order = max(order, pageblock_order);
+ int reclaim_order = ac->reclaim_compact_order;
psi_memstall_enter(&pflags);
*did_some_progress = __perform_reclaim(gfp_mask, reclaim_order, ac);
@@ -4639,9 +4631,12 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
/*
* Costly allocations might have made a progress but this doesn't mean
* their order will become available due to high fragmentation so
- * always increment the no progress counter for them
+ * always increment the no progress counter for them. The same goes for
+ * promoted requests: their reclaim reports progress when it backs off
+ * for compaction without freeing anything.
*/
- if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER)
+ if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER &&
+ !order_defrag_promoted(order, ac))
*no_progress_loops = 0;
else
(*no_progress_loops)++;
@@ -4838,6 +4833,11 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
(costly_order ||
(order > 0 && ac->migratetype != MIGRATE_MOVABLE))
&& !gfp_pfmemalloc_allowed(gfp_mask)) {
+ /* Promote as the retry loop below does */
+ ac->reclaim_compact_order = order;
+ if ((alloc_flags & ALLOC_NOFRAGMENT) &&
+ ac->migratetype != MIGRATE_MOVABLE)
+ ac->reclaim_compact_order = max(order, pageblock_order);
page = __alloc_pages_direct_compact(gfp_mask, order,
alloc_flags, ac,
INIT_COMPACT_PRIORITY,
@@ -4936,6 +4936,22 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
if (current->flags & PF_MEMALLOC)
goto nopage;
+ /*
+ * If fallbacks are not permitted (defrag_mode), we either
+ * need to reclaim space in a block of matching type, or clear
+ * out an entire block to allow __rmqueue_claim() to convert.
+ *
+ * Reclaim by itself is primarily freeing space in movable
+ * blocks, since that's where the LRU pages live. So this
+ * works for movable requests, but not for others.
+ *
+ * For those, promote the order to help make blocks, instead
+ * of spinning in reclaim alone unproductively.
+ */
+ ac->reclaim_compact_order = order;
+ if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
+ ac->reclaim_compact_order = max(order, pageblock_order);
+
/* Try direct reclaim and then allocating */
page = __alloc_pages_direct_reclaim(gfp_mask, order, alloc_flags, ac,
&did_some_progress);
@@ -4979,6 +4995,11 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
/* Reclaim/compaction failed to prevent the fallback */
if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT)) {
alloc_flags &= ~ALLOC_NOFRAGMENT;
+ /* Retries went into making blocks, start the fallback afresh */
+ if (order_defrag_promoted(order, ac)) {
+ no_progress_loops = 0;
+ compaction_retries = 0;
+ }
goto retry;
}
diff --git a/mm/page_counter.c b/mm/page_counter.c
index 661e0f2a5127..ea0d1646cff8 100644
--- a/mm/page_counter.c
+++ b/mm/page_counter.c
@@ -8,6 +8,7 @@
#include <linux/page_counter.h>
#include <linux/atomic.h>
#include <linux/kernel.h>
+#include <linux/math64.h>
#include <linux/string.h>
#include <linux/sched.h>
#include <linux/bug.h>
@@ -356,7 +357,8 @@ static unsigned long effective_protection(unsigned long usage,
* otherwise get a smaller chunk than what they claimed.
*/
if (siblings_protected > parent_effective)
- return protected * parent_effective / siblings_protected;
+ return mul_u64_u64_div_u64(protected, parent_effective,
+ siblings_protected);
/*
* Ok, utilized protection of all children is within what the
@@ -397,13 +399,20 @@ static unsigned long effective_protection(unsigned long usage,
if (parent_effective > siblings_protected &&
parent_usage > siblings_protected &&
usage > protected) {
- unsigned long unclaimed;
+ unsigned long parent_unclaimed, parent_unprotected, unprotected;
- unclaimed = parent_effective - siblings_protected;
- unclaimed *= usage - protected;
- unclaimed /= parent_usage - siblings_protected;
+ parent_unclaimed = parent_effective - siblings_protected;
+ parent_unprotected = parent_usage - siblings_protected;
- ep += unclaimed;
+ /*
+ * The usages aren't read atomically, so a child can transiently
+ * appear to use more than its parent, making the ratio exceed 1
+ * and the quotient overflow 64 bits (#DE on x86). Cap it.
+ */
+ unprotected = min(usage - protected, parent_unprotected);
+
+ ep += mul_u64_u64_div_u64(parent_unclaimed, unprotected,
+ parent_unprotected);
}
return ep;
diff --git a/mm/page_owner.c b/mm/page_owner.c
index bc26764142ba..4862fc52c5ec 100644
--- a/mm/page_owner.c
+++ b/mm/page_owner.c
@@ -336,7 +336,7 @@ noinline void __set_page_owner(struct page *page, unsigned short order,
inc_stack_record_count(handle, gfp_mask, 1 << order);
}
-void __folio_set_owner_migrate_reason(struct folio *folio, int reason)
+void __folio_set_owner_migrate_reason(struct folio *folio, enum migrate_reason reason)
{
struct page_ext *page_ext = page_ext_get(&folio->page);
struct page_owner *page_owner;
@@ -936,7 +936,7 @@ static const struct file_operations page_owner_stack_operations = {
.open = page_owner_stack_open,
.read = seq_read,
.llseek = seq_lseek,
- .release = seq_release_private,
+ .release = seq_release,
};
static int page_owner_threshold_get(void *data, u64 *val)
diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
index 41cf3192a8f6..dfbbbf5e3f53 100644
--- a/mm/page_vma_mapped.c
+++ b/mm/page_vma_mapped.c
@@ -260,18 +260,18 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
if (!thp_migration_supported() ||
!(pvmw->flags & PVMW_MIGRATION))
- return not_found(pvmw);
+ goto next_pmd;
entry = pmd_to_swp_entry(pmde);
if (!is_migration_entry(entry) ||
!check_pmd(swp_offset_pfn(entry), pvmw))
- return not_found(pvmw);
+ goto next_pmd;
return true;
}
if (likely(pmd_trans_huge(pmde))) {
if (pvmw->flags & PVMW_MIGRATION)
- return not_found(pvmw);
+ goto next_pmd;
if (!check_pmd(pmd_pfn(pmde), pvmw))
- return not_found(pvmw);
+ goto next_pmd;
return true;
}
/* THP pmd was split under us: handle on pte level */
@@ -291,8 +291,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
spin_unlock(ptl);
}
- step_forward(pvmw, PMD_SIZE);
- continue;
+ goto next_pmd;
}
if (!map_pte(pvmw, &pmde, &ptl)) {
if (!pvmw->pte)
@@ -331,6 +330,17 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
pvmw->ptl = ptl;
}
goto this_pte;
+next_pmd:
+ /*
+ * A large folio can be PTE-mapped across PMDs, so a
+ * nonmatching PMD does not end the walk.
+ */
+ if (pvmw->ptl) {
+ spin_unlock(pvmw->ptl);
+ pvmw->ptl = NULL;
+ }
+ pvmw->pmd = NULL;
+ step_forward(pvmw, PMD_SIZE);
} while (pvmw->address < end);
return false;
diff --git a/mm/shmem.c b/mm/shmem.c
index 68a3a0734e3d..5ce1c0ddf8cd 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -76,6 +76,7 @@ static struct vfsmount *shm_mnt __ro_after_init;
#include <linux/magic.h>
#include <linux/syscalls.h>
#include <linux/fcntl.h>
+#include <linux/memfd.h>
#include <uapi/linux/memfd.h>
#include <linux/rmap.h>
#include <linux/uuid.h>
@@ -947,6 +948,7 @@ static long shmem_free_swap(struct address_space *mapping,
pgoff_t index, pgoff_t end, void *radswap)
{
XA_STATE(xas, &mapping->i_pages, index);
+ const swp_entry_t swp = radix_to_swp_entry(radswap);
unsigned int nr_pages = 0;
pgoff_t base;
void *entry;
@@ -963,8 +965,9 @@ static long shmem_free_swap(struct address_space *mapping,
}
xas_unlock_irq(&xas);
- if (nr_pages)
- free_swap_and_cache_nr(radix_to_swp_entry(radswap), nr_pages);
+ /* A swapin-error marker holds no swap slot, so just drop it. */
+ if (nr_pages && !non_swap_entry(swp))
+ free_swap_and_cache_nr(swp, nr_pages);
return nr_pages;
}
@@ -1057,12 +1060,10 @@ void shmem_unlock_mapping(struct address_space *mapping)
static struct folio *shmem_get_partial_folio(struct inode *inode, pgoff_t index)
{
+ const gfp_t gfp = mapping_gfp_mask(inode->i_mapping);
struct folio *folio;
- /*
- * At first avoid shmem_get_folio(,,,SGP_READ): that fails
- * beyond i_size, and reports fallocated folios as holes.
- */
+retry:
folio = filemap_get_entry(inode->i_mapping, index);
if (!folio)
return folio;
@@ -1073,13 +1074,14 @@ static struct folio *shmem_get_partial_folio(struct inode *inode, pgoff_t index)
/* The folio has been swapped out */
folio_unlock(folio);
folio_put(folio);
+ goto retry;
}
/*
- * But read a folio back from swap if any of it is within i_size
- * (although in some cases this is just a waste of time).
+ * Truncation can look up a large swap entry at or beyond i_size.
+ * Swap it in directly to bypass the size check in shmem_get_folio().
*/
- folio = NULL;
- shmem_get_folio(inode, index, 0, &folio, SGP_READ);
+ if (shmem_swapin_folio(inode, index, &folio, SGP_READ, gfp, NULL, NULL) == -EEXIST)
+ goto retry;
return folio;
}
@@ -1198,14 +1200,16 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend,
order = shmem_confirm_swap(mapping, indices[i],
radix_to_swp_entry(folio));
/*
- * If found a large swap entry cross the end or start
- * border, skip it as the truncate_inode_partial_folio
- * above should have at least zerod its content once.
+ * Skip boundary entries when restarting
+ * the scan to avoid endless loops after
+ * swapin failure.
*/
if (order > 0) {
base = round_down(base, 1 << order);
- if (base < start || base + (1 << order) > end)
+ if (base < start || base + (1 << order) > end) {
+ start = max(start, base + (1 << order));
continue;
+ }
}
/* Swap was replaced by page or extended, retry */
index = base;
@@ -1227,22 +1231,34 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend,
VM_BUG_ON_FOLIO(folio_test_writeback(folio),
folio);
- if (!folio_test_large(folio)) {
+ /*
+ * A shared mapping can have dirty PTEs while
+ * the folio is still clean. Mark the folio
+ * dirty so a failed split cannot discard data
+ * outside the range.
+ */
+ if (!unfalloc && folio_test_large(folio))
+ folio_mark_dirty(folio);
+
+ if (unfalloc || !folio_test_large(folio)) {
truncate_inode_folio(mapping, folio);
} else if (truncate_inode_partial_folio(folio, lstart, lend)) {
/*
- * If we split a page, reset the loop so
- * that we pick up the new sub pages.
- * Otherwise the THP was entirely
- * dropped or the target range was
- * zeroed, so just continue the loop as
- * is.
+ * The folio was split. Retry the scan
+ * to find the split folios.
*/
- if (!folio_test_large(folio)) {
- folio_unlock(folio);
- index = start;
- break;
- }
+ folio_unlock(folio);
+ index = start;
+ break;
+ } else if (folio->mapping) {
+ /*
+ * Splitting failed, but the part in the
+ * range is already zeroed. The folio
+ * was not dropped, so skip it when
+ * restarting the scan to avoid retrying
+ * it forever.
+ */
+ start = max(start, folio_next_index(folio));
}
}
folio_unlock(folio);
@@ -1308,10 +1324,11 @@ static int shmem_setattr(struct mnt_idmap *idmap,
if (error)
return error;
- if ((info->seals & F_SEAL_EXEC) && (attr->ia_valid & ATTR_MODE)) {
- if ((inode->i_mode ^ attr->ia_mode) & 0111) {
- return -EPERM;
- }
+ if (attr->ia_valid & ATTR_MODE) {
+ error = memfd_check_seals_mode(info->seals, inode->i_mode,
+ attr->ia_mode);
+ if (error)
+ return error;
}
if (S_ISREG(inode->i_mode) && (attr->ia_valid & ATTR_SIZE)) {
@@ -1338,7 +1355,8 @@ static int shmem_setattr(struct mnt_idmap *idmap,
if (oldsize > holebegin)
unmap_mapping_range(inode->i_mapping,
holebegin, 0, 1);
- if (info->alloced)
+ /* Swapin-error markers are not counted in alloced. */
+ if (info->alloced || !mapping_empty(inode->i_mapping))
shmem_truncate_range(inode,
newsize, (loff_t)-1);
/* unmap again to remove racily COWed private pages */
@@ -1505,6 +1523,10 @@ static int shmem_unuse_inode(struct inode *inode, unsigned int type)
if (ret < 0)
break;
+ if (signal_pending(current)) {
+ ret = -ERESTARTSYS;
+ break;
+ }
start = indices[folio_batch_count(&fbatch) - 1];
} while (true);
@@ -5277,12 +5299,35 @@ static const struct file_operations shmem_file_operations = {
#endif
};
+#ifdef CONFIG_TMPFS_POSIX_ACL
+static int shmem_set_acl(struct mnt_idmap *idmap, struct dentry *dentry,
+ struct posix_acl *acl, int type)
+{
+ struct inode *inode = d_inode(dentry);
+ umode_t mode = inode->i_mode;
+ int error;
+
+ if (type == ACL_TYPE_ACCESS) {
+ error = posix_acl_equiv_mode(acl, &mode);
+ if (error < 0)
+ return error;
+ error = memfd_check_seals_mode(SHMEM_I(inode)->seals,
+ inode->i_mode, mode);
+ if (error)
+ return error;
+ }
+ return simple_set_acl(idmap, dentry, acl, type);
+}
+#else
+#define shmem_set_acl simple_set_acl
+#endif
+
static const struct inode_operations shmem_inode_operations = {
.getattr = shmem_getattr,
.setattr = shmem_setattr,
#ifdef CONFIG_TMPFS_XATTR
.listxattr = shmem_listxattr,
- .set_acl = simple_set_acl,
+ .set_acl = shmem_set_acl,
.fileattr_get = shmem_fileattr_get,
.fileattr_set = shmem_fileattr_set,
#endif
diff --git a/mm/shmem_quota.c b/mm/shmem_quota.c
index d1e32ac01407..868b3e05f1aa 100644
--- a/mm/shmem_quota.c
+++ b/mm/shmem_quota.c
@@ -234,7 +234,7 @@ static int shmem_acquire_dquot(struct dquot *dquot)
return ret;
}
-static bool shmem_is_empty_dquot(struct dquot *dquot)
+static bool shmem_is_default_dquot(struct dquot *dquot)
{
struct shmem_sb_info *sbinfo = dquot->dq_sb->s_fs_info;
qsize_t bhardlimit;
@@ -248,19 +248,21 @@ static bool shmem_is_empty_dquot(struct dquot *dquot)
ihardlimit = sbinfo->qlimits.grpquota_ihardlimit;
}
- if (test_bit(DQ_FAKE_B, &dquot->dq_flags) ||
- (dquot->dq_dqb.dqb_curspace == 0 &&
- dquot->dq_dqb.dqb_curinodes == 0 &&
- dquot->dq_dqb.dqb_bhardlimit == bhardlimit &&
- dquot->dq_dqb.dqb_ihardlimit == ihardlimit))
- return true;
-
- return false;
+ /*
+ * Defaults are zero usage and soft limits, and the mount's hard limits.
+ * DQ_FAKE_B only means that *all* limits are zero, which need not match
+ * the mount defaults.
+ */
+ return dquot->dq_dqb.dqb_curspace == 0 &&
+ dquot->dq_dqb.dqb_curinodes == 0 &&
+ dquot->dq_dqb.dqb_bsoftlimit == 0 &&
+ dquot->dq_dqb.dqb_isoftlimit == 0 &&
+ dquot->dq_dqb.dqb_bhardlimit == bhardlimit &&
+ dquot->dq_dqb.dqb_ihardlimit == ihardlimit;
}
/*
- * Store limits from dquot in the tree unless it's fake. If it is fake
- * remove the id from the tree since there is no useful information in
- * there.
+ * Remove the ID from the tree if it has no usage and its limits match
+ * the mount defaults. Otherwise, save its limits for the next acquire.
*/
static int shmem_release_dquot(struct dquot *dquot)
{
@@ -295,7 +297,7 @@ static int shmem_release_dquot(struct dquot *dquot)
return -ENOENT;
found:
- if (shmem_is_empty_dquot(dquot)) {
+ if (shmem_is_default_dquot(dquot)) {
/* Remove entry from the tree */
rb_erase(&entry->node, info->dqi_priv);
kfree(entry);
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 3ffa258de79e..3a786446dacb 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -2349,6 +2349,8 @@ static inline int unuse_pmd_range(struct vm_area_struct *vma, pud_t *pud,
pmd = pmd_offset(pud, addr);
do {
cond_resched();
+ if (signal_pending(current))
+ return -ERESTARTSYS;
next = pmd_addr_end(addr, end);
ret = unuse_pte_range(vma, pmd, addr, next, type);
if (ret)
@@ -2562,7 +2564,7 @@ static int try_to_unuse(unsigned int type)
if (swap_usage_in_pages(si)) {
if (!signal_pending(current))
goto retry;
- return -EINTR;
+ return -ERESTARTSYS;
}
success:
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 579445bddf56..e4a0cb3ad838 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -48,8 +48,7 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
vma = vma_lookup(mm, addr);
if (!vma)
vma = ERR_PTR(-ENOENT);
- else if (!(vma->vm_flags & VM_SHARED) &&
- unlikely(anon_vma_prepare(vma)))
+ else if (is_cow_mapping(vma->vm_flags) && unlikely(anon_vma_prepare(vma)))
vma = ERR_PTR(-ENOMEM);
return vma;
@@ -77,7 +76,7 @@ static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,
* We know we're going to need to use anon_vma, so check
* that early.
*/
- if (!(vma->vm_flags & VM_SHARED) && unlikely(!vma->anon_vma))
+ if (is_cow_mapping(vma->vm_flags) && unlikely(!vma->anon_vma))
vma_end_read(vma);
else
return vma;
diff --git a/mm/vma.c b/mm/vma.c
index d1627a8ef32b..0d4f34495664 100644
--- a/mm/vma.c
+++ b/mm/vma.c
@@ -485,6 +485,15 @@ void unmap_region(struct ma_state *mas, struct vm_area_struct *vma,
tlb_finish_mmu(&tlb);
}
+static int check_split(struct vm_area_struct *vma, unsigned long addr)
+{
+ if (!vma->vm_ops)
+ return 0;
+ if (!vma->vm_ops->may_split)
+ return 0;
+ return vma->vm_ops->may_split(vma, addr);
+}
+
/*
* __split_vma() bypasses sysctl_max_map_count checking. We use this where it
* has already been checked or doesn't make sense to fail.
@@ -501,11 +510,9 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
WARN_ON(vma->vm_start >= addr);
WARN_ON(vma->vm_end <= addr);
- if (vma->vm_ops && vma->vm_ops->may_split) {
- err = vma->vm_ops->may_split(vma, addr);
- if (err)
- return err;
- }
+ err = check_split(vma, addr);
+ if (err)
+ return err;
new = vm_area_dup(vma);
if (!new)
@@ -875,6 +882,10 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(
if (vmg->__remove_middle && !can_merge_remove_vma(middle))
return NULL;
+ /* If middle isn't removed then it's resized, i.e. splits. */
+ if (!vmg->__remove_middle && check_split(middle, left_side ? end : start))
+ return NULL;
+
/*
* If we merge both VMAs, then next is also deleted. This implies
* merge_will_delete_vma also.
@@ -1046,6 +1057,7 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
struct vm_area_struct *next = vmg->next;
unsigned long end = vmg->end;
bool can_merge_left, can_merge_right;
+ bool move_iter_back = false;
mmap_assert_write_locked(vmg->mm);
VM_WARN_ON_VMG(vmg->middle, vmg);
@@ -1083,23 +1095,30 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
vmg->end = end;
/* In expand-only case we are already positioned at prev. */
- if (!vmg->just_expand) {
- /* Equivalent to going to the previous range. */
- vma_prev(vmg->vmi);
- }
+ if (!vmg->just_expand)
+ move_iter_back = true;
}
+ if (!can_merge_left && !can_merge_right)
+ return NULL;
+
+ if (move_iter_back)
+ vma_prev(vmg->vmi);
+
/*
* Now try to expand adjacent VMA(s). This takes care of removing the
* following VMA if we have VMAs on both sides.
*/
- if (vmg->target && !vma_expand(vmg)) {
- khugepaged_enter_vma(vmg->target, vmg->vm_flags);
- vmg->state = VMA_MERGE_SUCCESS;
- return vmg->target;
+ if (vma_expand(vmg)) {
+ /* Error occurred, restore iterator. */
+ if (move_iter_back)
+ vma_iter_next_range(vmg->vmi);
+ return NULL;
}
- return NULL;
+ khugepaged_enter_vma(vmg->target, vmg->vm_flags);
+ vmg->state = VMA_MERGE_SUCCESS;
+ return vmg->target;
}
/*
@@ -1126,6 +1145,11 @@ static struct vm_area_struct *vma_merge_copied_range(struct vma_merge_struct *vm
vmg->copied_from = vmg->middle;
vmg->middle = NULL;
+
+ /* If not registered for remap, uffd context will be cleared. */
+ if (vma_has_uffd_without_event_remap(vmg->copied_from))
+ return NULL;
+
return vma_merge_new_range(vmg);
}
@@ -1190,7 +1214,7 @@ int vma_expand(struct vma_merge_struct *vmg)
if (!ret && vmg->copied_from)
ret = dup_anon_vma(target, vmg->copied_from, &anon_dup);
if (ret)
- return ret;
+ goto nomem;
if (remove_next) {
vma_start_write(next);
@@ -2443,14 +2467,17 @@ static int __mmap_setup(struct mmap_state *map, struct list_head *uf)
/* Private writable mapping: check memory availability. */
if (accountable_mapping(map->file, map->vm_flags)) {
- map->charged = map->pglen;
- map->charged -= vms->nr_accounted;
- if (map->charged) {
- error = security_vm_enough_memory_mm(map->mm, map->charged);
+ const unsigned long uncharged = map->pglen - vms->nr_accounted;
+
+ /* Charge only what's not already charged by mappings we replace. */
+ if (uncharged) {
+ error = security_vm_enough_memory_mm(map->mm, uncharged);
if (error)
return error;
}
+ /* Our new mapping owns the entire charge now. */
+ map->charged = map->pglen;
vms->nr_accounted = 0;
map->vm_flags |= VM_ACCOUNT;
}
diff --git a/mm/vmscan.c b/mm/vmscan.c
index e72d50bab730..8556318ff721 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3750,7 +3750,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end,
walk->mm_stats[MM_NONLEAF_FOUND]++;
- if (!walk_pte_range(&val, addr, next, args))
+ if (!walk_pte_range(pmd + i, addr, next, args))
continue;
walk->mm_stats[MM_NONLEAF_ADDED]++;
@@ -3786,7 +3786,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long start, unsigned long end,
if (!pud_present(val) || WARN_ON_ONCE(pud_leaf(val)))
continue;
- walk_pmd_range(&val, addr, next, args);
+ walk_pmd_range(pud + i, addr, next, args);
if (need_resched() || walk->batched >= MAX_LRU_BATCH) {
end = (addr | ~PUD_MASK) + 1;
@@ -4844,10 +4844,6 @@ static bool should_abort_scan(struct lruvec *lruvec, struct scan_control *sc)
int i;
enum zone_watermarks mark;
- /* don't abort memcg reclaim to ensure fairness */
- if (!root_reclaim(sc))
- return false;
-
if (sc->nr_reclaimed >= max(sc->nr_to_reclaim, compact_gap(sc->order)))
return true;
@@ -4931,7 +4927,13 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc)
struct mem_cgroup *memcg = lruvec_memcg(lruvec);
struct pglist_data *pgdat = lruvec_pgdat(lruvec);
- /* lru_gen_age_node() called mem_cgroup_calculate_protection() */
+ /*
+ * For kswapd, mem_cgroup_calculate_protection() has already
+ * been called during the top-down cgroup traversal.
+ */
+ if (!current_is_kswapd())
+ mem_cgroup_calculate_protection_path(NULL, memcg);
+
if (mem_cgroup_below_min(NULL, memcg))
return MEMCG_LRU_YOUNG;
diff --git a/net/9p/trans_usbg.c b/net/9p/trans_usbg.c
index 468f7e8f0277..938e3130ab81 100644
--- a/net/9p/trans_usbg.c
+++ b/net/9p/trans_usbg.c
@@ -148,7 +148,8 @@ static void usb9pfs_tx_complete(struct usb_ep *ep, struct usb_request *req)
{
struct f_usb9pfs *usb9pfs = ep->driver_data;
struct usb_composite_dev *cdev = usb9pfs->function.config->cdev;
- struct p9_req_t *p9_tx_req = req->context;
+ struct p9_client *client;
+ struct p9_req_t *p9_tx_req;
unsigned long flags;
/* reset zero packages */
@@ -164,18 +165,25 @@ static void usb9pfs_tx_complete(struct usb_ep *ep, struct usb_request *req)
ep->name, req->status, req->actual, req->length);
spin_lock_irqsave(&usb9pfs->lock, flags);
- WRITE_ONCE(p9_tx_req->status, REQ_STATUS_SENT);
+ client = usb9pfs->client;
+ p9_tx_req = req->context;
+ req->context = NULL;
- p9_req_put(usb9pfs->client, p9_tx_req);
+ if (!client || !p9_tx_req)
+ goto unlock_complete;
- req->context = NULL;
+ WRITE_ONCE(p9_tx_req->status, REQ_STATUS_SENT);
+
+ p9_req_put(client, p9_tx_req);
+unlock_complete:
spin_unlock_irqrestore(&usb9pfs->lock, flags);
complete(&usb9pfs->send);
}
-static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
+static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs,
+ struct p9_client *client, void *buf)
{
struct p9_req_t *p9_rx_req;
struct p9_fcall rc;
@@ -201,7 +209,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
"mux %p pkt: size: %d bytes tag: %d\n",
usb9pfs, rc.size, rc.tag);
- p9_rx_req = p9_tag_lookup(usb9pfs->client, rc.tag);
+ p9_rx_req = p9_tag_lookup(client, rc.tag);
if (!p9_rx_req || p9_rx_req->status != REQ_STATUS_SENT) {
p9_debug(P9_DEBUG_ERROR, "Unexpected packet tag %d\n", rc.tag);
return NULL;
@@ -211,7 +219,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
p9_debug(P9_DEBUG_ERROR,
"requested packet size too big: %d for tag %d with capacity %zd\n",
rc.size, rc.tag, p9_rx_req->rc.capacity);
- p9_req_put(usb9pfs->client, p9_rx_req);
+ p9_req_put(client, p9_rx_req);
return NULL;
}
@@ -219,7 +227,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
p9_debug(P9_DEBUG_ERROR,
"No recv fcall for tag %d (req %p), disconnecting!\n",
rc.tag, p9_rx_req);
- p9_req_put(usb9pfs->client, p9_rx_req);
+ p9_req_put(client, p9_rx_req);
return NULL;
}
@@ -230,8 +238,10 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req)
{
struct f_usb9pfs *usb9pfs = ep->driver_data;
struct usb_composite_dev *cdev = usb9pfs->function.config->cdev;
+ struct p9_client *client;
struct p9_req_t *p9_rx_req;
unsigned int req_size = req->actual;
+ unsigned long flags;
int status = REQ_STATUS_RCVD;
if (req->status) {
@@ -240,9 +250,16 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req)
return;
}
- p9_rx_req = usb9pfs_rx_header(usb9pfs, req->buf);
- if (!p9_rx_req)
+ spin_lock_irqsave(&usb9pfs->lock, flags);
+ client = usb9pfs->client;
+ if (!client) {
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
return;
+ }
+
+ p9_rx_req = usb9pfs_rx_header(usb9pfs, client, req->buf);
+ if (!p9_rx_req)
+ goto out_unlock;
if (req_size > p9_rx_req->rc.capacity) {
dev_err(&cdev->gadget->dev,
@@ -256,8 +273,11 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req)
p9_rx_req->rc.size = req_size;
- p9_client_cb(usb9pfs->client, p9_rx_req, status);
- p9_req_put(usb9pfs->client, p9_rx_req);
+ p9_client_cb(client, p9_rx_req, status);
+ p9_req_put(client, p9_rx_req);
+
+out_unlock:
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
complete(&usb9pfs->received);
}
@@ -277,18 +297,18 @@ static void disable_usb9pfs(struct f_usb9pfs *usb9pfs)
struct usb_composite_dev *cdev =
usb9pfs->function.config->cdev;
+ disable_ep(cdev, usb9pfs->in_ep);
+ disable_ep(cdev, usb9pfs->out_ep);
+
if (usb9pfs->in_req) {
usb_ep_free_request(usb9pfs->in_ep, usb9pfs->in_req);
usb9pfs->in_req = NULL;
}
if (usb9pfs->out_req) {
- usb_ep_free_request(usb9pfs->out_ep, usb9pfs->out_req);
+ free_ep_req(usb9pfs->out_ep, usb9pfs->out_req);
usb9pfs->out_req = NULL;
}
-
- disable_ep(cdev, usb9pfs->in_ep);
- disable_ep(cdev, usb9pfs->out_ep);
dev_dbg(&cdev->gadget->dev, "%s disabled\n",
usb9pfs->function.name);
}
@@ -313,10 +333,6 @@ static int alloc_requests(struct usb_composite_dev *cdev,
usb9pfs->in_req->complete = usb9pfs_tx_complete;
usb9pfs->out_req->complete = usb9pfs_rx_complete;
- /* length will be set in complete routine */
- usb9pfs->in_req->context = usb9pfs;
- usb9pfs->out_req->context = usb9pfs;
-
return 0;
fail_in:
@@ -389,7 +405,7 @@ static int p9_usbg_create(struct p9_client *client, const char *devname, char *a
guard(mutex)(&usb9pfs_lock);
list_for_each_entry(dev, &usbg_instance_list, usb9pfs_instance) {
- if (!strncmp(devname, dev->tag, strlen(devname))) {
+ if (!strcmp(devname, dev->tag)) {
if (!dev->inuse) {
dev->inuse = true;
found = 1;
@@ -414,7 +430,9 @@ static int p9_usbg_create(struct p9_client *client, const char *devname, char *a
client->status = Disconnected;
else
client->status = Connected;
+ spin_lock_irq(&usb9pfs->lock);
usb9pfs->client = client;
+ spin_unlock_irq(&usb9pfs->lock);
client->trans_mod->maxsize = usb9pfs->buflen;
@@ -425,18 +443,26 @@ static int p9_usbg_create(struct p9_client *client, const char *devname, char *a
static void usb9pfs_clear_tx(struct f_usb9pfs *usb9pfs)
{
+ struct p9_client *client;
struct p9_req_t *req;
+ unsigned long flags;
- guard(spinlock_irqsave)(&usb9pfs->lock);
+ spin_lock_irqsave(&usb9pfs->lock, flags);
+ client = usb9pfs->client;
+ usb9pfs->client = NULL;
+ req = usb9pfs->in_req ? usb9pfs->in_req->context : NULL;
+ if (usb9pfs->in_req)
+ usb9pfs->in_req->context = NULL;
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
- req = usb9pfs->in_req->context;
- if (!req)
+ if (!req || !client)
return;
if (!req->t_err)
req->t_err = -ECONNRESET;
- p9_client_cb(usb9pfs->client, req, REQ_STATUS_ERROR);
+ p9_client_cb(client, req, REQ_STATUS_ERROR);
+ usb9pfs->in_req->context = NULL;
}
static void p9_usbg_close(struct p9_client *client)
@@ -464,6 +490,9 @@ static void p9_usbg_close(struct p9_client *client)
mutex_lock(&usb9pfs_lock);
dev->inuse = false;
mutex_unlock(&usb9pfs_lock);
+
+ disable_usb9pfs(usb9pfs);
+ reinit_completion(&usb9pfs->send);
}
static int p9_usbg_request(struct p9_client *client, struct p9_req_t *p9_req)
@@ -745,8 +774,18 @@ static int usb9pfs_set_alt(struct usb_function *f,
static void usb9pfs_disable(struct usb_function *f)
{
struct f_usb9pfs *usb9pfs = func_to_usb9pfs(f);
+ unsigned long flags;
+ spin_lock_irqsave(&usb9pfs->lock, flags);
+ if (usb9pfs->client)
+ usb9pfs->client->status = Disconnected;
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
usb9pfs_clear_tx(usb9pfs);
+
+ if (usb9pfs->in_ep->enabled)
+ disable_usb9pfs(usb9pfs);
+
+ reinit_completion(&usb9pfs->send);
}
static struct usb_function *usb9pfs_alloc(struct usb_function_instance *fi)
diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c
index 0b8086f58ad5..d1a680f69911 100644
--- a/net/9p/trans_virtio.c
+++ b/net/9p/trans_virtio.c
@@ -629,6 +629,11 @@ static int p9_virtio_probe(struct virtio_device *vdev)
err = -EINVAL;
goto out_free_vq;
}
+ if (tag_len > NAME_MAX) {
+ dev_err(&vdev->dev, "mount tag too long (%u bytes)\n", tag_len);
+ err = -EINVAL;
+ goto out_free_vq;
+ }
tag = kzalloc(tag_len + 1, GFP_KERNEL);
if (!tag) {
err = -ENOMEM;
diff --git a/net/9p/trans_xen.c b/net/9p/trans_xen.c
index 068d57515dd5..67f464acf617 100644
--- a/net/9p/trans_xen.c
+++ b/net/9p/trans_xen.c
@@ -198,9 +198,19 @@ static void p9_xen_response(struct work_struct *work)
masked_prod, &masked_cons,
XEN_9PFS_RING_SIZE(ring));
+ if (h.size < sizeof(h) ||
+ h.size > xen_9pfs_queued(prod, cons,
+ XEN_9PFS_RING_SIZE(ring))) {
+ dev_warn_ratelimited(&priv->dev->dev,
+ "bad response size %u from backend\n", h.size);
+ break;
+ }
+
req = p9_tag_lookup(priv->client, h.tag);
if (!req || req->status != REQ_STATUS_SENT) {
dev_warn(&priv->dev->dev, "Wrong req tag=%x\n", h.tag);
+ if (req)
+ p9_req_put(priv->client, req);
cons += h.size;
virt_mb();
ring->intf->in_cons = cons;
diff --git a/net/ax25/af_ax25.c b/net/ax25/af_ax25.c
index 5ce59d1ba822..d0ac09cc46e7 100644
--- a/net/ax25/af_ax25.c
+++ b/net/ax25/af_ax25.c
@@ -78,6 +78,7 @@ static void ax25_kill_by_device(struct net_device *dev)
ax25_dev *ax25_dev;
ax25_cb *s;
struct sock *sk;
+ bool orphan;
if ((ax25_dev = ax25_dev_ax25dev(dev)) == NULL)
return;
@@ -109,6 +110,16 @@ static void ax25_kill_by_device(struct net_device *dev)
sock_hold(sk);
spin_unlock_bh(&ax25_list_lock);
lock_sock(sk);
+ /*
+ * A socket closed by its owner, or a connection left
+ * unaccepted by a closed listener, is freed by its
+ * heartbeat once it is down. ax25_disconnect() below
+ * stops that heartbeat for good, so free the socket
+ * here, or it is never freed and pins the module.
+ */
+ orphan = sock_flag(sk, SOCK_DESTROY) ||
+ (sk->sk_state == TCP_LISTEN &&
+ sock_flag(sk, SOCK_DEAD));
ax25_disconnect(s, ENETUNREACH);
WRITE_ONCE(s->ax25_dev, NULL);
if (sk->sk_socket) {
@@ -116,7 +127,10 @@ static void ax25_kill_by_device(struct net_device *dev)
&s->dev_tracker);
ax25_dev_put(ax25_dev);
}
- ax25_cb_del(s);
+ if (orphan)
+ ax25_destroy_socket(s);
+ else
+ ax25_cb_del(s);
release_sock(sk);
spin_lock_bh(&ax25_list_lock);
sock_put(sk);
@@ -694,6 +708,12 @@ static int ax25_setsockopt(struct socket *sock, int level, int optname,
case AX25_EXTSEQ:
ax25->modulus = opt ? AX25_EMODULUS : AX25_MODULUS;
+ /*
+ * A window set for modulo 128 may not fit modulo 8:
+ * ax25_kick() would then find it full and send nothing.
+ */
+ if (ax25->modulus == AX25_MODULUS && ax25->window > 7)
+ ax25->window = 7;
break;
case AX25_PIDINCL:
@@ -1233,7 +1253,7 @@ static int __must_check ax25_connect(struct socket *sock,
struct sock *sk = sock->sk;
ax25_cb *ax25 = sk_to_ax25(sk), *ax25t;
struct full_sockaddr_ax25 *fsa = (struct full_sockaddr_ax25 *)uaddr;
- ax25_digi *digi = NULL;
+ ax25_digi *digi = NULL, *old_digi;
int ct = 0, err = 0;
/*
@@ -1282,11 +1302,28 @@ static int __must_check ax25_connect(struct socket *sock,
goto out_release;
}
+ /*
+ * A connection that failed or ended through ax25_disconnect() left
+ * the socket SOCK_DEAD and shut down for sending. The socket is still
+ * its owner's: start afresh, or the state changes of the new attempt
+ * would wake no one and every send() would fail with -EPIPE.
+ */
+ sock_reset_flag(sk, SOCK_DEAD);
+ sk->sk_shutdown = 0;
+
sk->sk_state = TCP_CLOSE;
sock->state = SS_UNCONNECTED;
- kfree(ax25->digipeat);
+ /*
+ * A bound socket is on ax25_list, where the receive path and
+ * /proc/net/ax25 read its digipeater path under ax25_list_lock:
+ * take the old path off under that lock before freeing it.
+ */
+ spin_lock_bh(&ax25_list_lock);
+ old_digi = ax25->digipeat;
ax25->digipeat = NULL;
+ spin_unlock_bh(&ax25_list_lock);
+ kfree(old_digi);
/*
* Handle digi-peaters to be used.
@@ -1337,6 +1374,15 @@ static int __must_check ax25_connect(struct socket *sock,
goto out_release;
}
+ /*
+ * The socket is on ax25_list with the destination of its previous
+ * attempt, if any: forget it, or the lookup below finds the socket
+ * itself when the same station is called again.
+ */
+ spin_lock_bh(&ax25_list_lock);
+ ax25->dest_addr = null_ax25_address;
+ spin_unlock_bh(&ax25_list_lock);
+
if (sk->sk_type == SOCK_SEQPACKET &&
(ax25t=ax25_find_cb(&ax25->source_addr, &fsa->fsa_ax25.sax25_call, digi,
ax25->ax25_dev->dev))) {
@@ -1346,8 +1392,10 @@ static int __must_check ax25_connect(struct socket *sock,
goto out_release;
}
+ spin_lock_bh(&ax25_list_lock);
ax25->dest_addr = fsa->fsa_ax25.sax25_call;
ax25->digipeat = digi;
+ spin_unlock_bh(&ax25_list_lock);
/* First the easy one */
if (sk->sk_type != SOCK_SEQPACKET) {
@@ -1484,16 +1532,29 @@ static int ax25_accept(struct socket *sock, struct socket *newsock,
goto out;
newsk = skb->sk;
+
+ /*
+ * ax25_kill_by_device() detaches a connection from a device going
+ * down under the lock of its socket, clears its ax25_dev and drops
+ * the device references if the socket has been grafted by then.
+ * Graft the socket and take those references under the same lock,
+ * and take none if the device has already gone: the connection is
+ * closed then, and the socket is returned as such.
+ */
+ lock_sock_nested(newsk, SINGLE_DEPTH_NESTING);
sock_graft(newsk, newsock);
+ ax25 = sk_to_ax25(newsk);
+ ax25_dev = ax25->ax25_dev;
+ if (ax25_dev) {
+ netdev_hold(ax25_dev->dev, &ax25->dev_tracker, GFP_ATOMIC);
+ ax25_dev_hold(ax25_dev);
+ }
+ release_sock(newsk);
/* Now attach up the new socket */
kfree_skb(skb);
sk_acceptq_removed(sk);
newsock->state = SS_CONNECTED;
- ax25 = sk_to_ax25(newsk);
- ax25_dev = ax25->ax25_dev;
- netdev_hold(ax25_dev->dev, &ax25->dev_tracker, GFP_ATOMIC);
- ax25_dev_hold(ax25_dev);
out:
release_sock(sk);
@@ -1555,7 +1616,7 @@ static int ax25_sendmsg(struct socket *sock, struct msghdr *msg, size_t len)
struct sock *sk = sock->sk;
struct sockaddr_ax25 sax;
struct sk_buff *skb;
- ax25_digi dtmp, *dp;
+ ax25_digi dtmp, *dp = NULL;
ax25_cb *ax25;
size_t size;
int lv, err, addr_len = msg->msg_namelen;
@@ -1630,6 +1691,7 @@ static int ax25_sendmsg(struct socket *sock, struct msghdr *msg, size_t len)
}
dtmp.lastrepeat = 0;
+ dp = &dtmp;
}
sax = *usax;
@@ -1638,10 +1700,6 @@ static int ax25_sendmsg(struct socket *sock, struct msghdr *msg, size_t len)
err = -EISCONN;
goto out;
}
- if (usax->sax25_ndigis == 0)
- dp = NULL;
- else
- dp = &dtmp;
} else {
/*
* FIXME: 1003.1g - if the socket is like this because
@@ -1779,25 +1837,36 @@ static int ax25_recvmsg(struct socket *sock, struct msghdr *msg, size_t size,
if (msg->msg_name) {
ax25_digi digi;
ax25_address src;
- const unsigned char *mac = skb_mac_header(skb);
+ const unsigned char *mac;
DECLARE_SOCKADDR(struct sockaddr_ax25 *, sax, msg->msg_name);
memset(sax, 0, sizeof(struct full_sockaddr_ax25));
- ax25_addr_parse(mac + 1, skb->data - mac - 1, &src, NULL,
- &digi, NULL, NULL);
sax->sax25_family = AF_AX25;
- /* We set this correctly, even though we may not let the
- application know the digi calls further down (because it
- did NOT ask to know them). This could get political... **/
- sax->sax25_ndigis = digi.ndigi;
- sax->sax25_call = src;
-
- if (sax->sax25_ndigis != 0) {
- int ct;
- struct full_sockaddr_ax25 *fsa = (struct full_sockaddr_ax25 *)sax;
-
- for (ct = 0; ct < digi.ndigi; ct++)
- fsa->fsa_digipeater[ct] = digi.calls[ct];
+
+ /*
+ * A frame put together from fragments by ax25_rx_fragment()
+ * has no address header left, and no mac header: give no
+ * address then. ax25_addr_parse() fills neither src nor digi
+ * when it fails.
+ */
+ mac = skb_mac_header_was_set(skb) ? skb_mac_header(skb) : NULL;
+ if (mac && ax25_addr_parse(mac + 1, skb->data - mac - 1, &src,
+ NULL, &digi, NULL, NULL)) {
+ /*
+ * We set this correctly, even though we may not let the
+ * application know the digi calls further down (because
+ * it did NOT ask to know them). This could get political...
+ */
+ sax->sax25_ndigis = digi.ndigi;
+ sax->sax25_call = src;
+
+ if (sax->sax25_ndigis != 0) {
+ int ct;
+ struct full_sockaddr_ax25 *fsa = (struct full_sockaddr_ax25 *)sax;
+
+ for (ct = 0; ct < digi.ndigi; ct++)
+ fsa->fsa_digipeater[ct] = digi.calls[ct];
+ }
}
msg->msg_namelen = sizeof(struct full_sockaddr_ax25);
}
diff --git a/net/ax25/ax25_ds_timer.c b/net/ax25/ax25_ds_timer.c
index 0c9e7775aa54..a0d6e9c5ec9f 100644
--- a/net/ax25/ax25_ds_timer.c
+++ b/net/ax25/ax25_ds_timer.c
@@ -211,7 +211,7 @@ void ax25_ds_t1_timeout(ax25_cb *ax25)
case AX25_STATE_2:
if (ax25->n2count == ax25->n2) {
ax25_send_control(ax25, AX25_DISC, AX25_POLLON, AX25_COMMAND);
- if (!sock_flag(ax25->sk, SOCK_DESTROY))
+ if (!ax25->sk || !sock_flag(ax25->sk, SOCK_DESTROY))
ax25_disconnect(ax25, ETIMEDOUT);
return;
} else {
diff --git a/net/ax25/ax25_in.c b/net/ax25/ax25_in.c
index 1bcb5e321467..fd8669e5b731 100644
--- a/net/ax25/ax25_in.c
+++ b/net/ax25/ax25_in.c
@@ -201,7 +201,7 @@ static int ax25_rcv(struct sk_buff *skb, struct net_device *dev,
int type = 0, mine = 0, dama;
struct sock *make, *sk;
ax25_digi dp, reverse_dp;
- ax25_cb *ax25;
+ ax25_cb *ax25, *existing;
ax25_dev *ax25_dev;
/*
@@ -401,11 +401,12 @@ static int ax25_rcv(struct sk_buff *skb, struct net_device *dev,
}
}
- skb_set_owner_r(skb, make);
- skb_queue_head(&sk->sk_receive_queue, skb);
-
- make->sk_state = TCP_ESTABLISHED;
-
+ /*
+ * Keep a place in the backlog, but put the skb on the
+ * listener's queue only once the connection is set up
+ * below: accept() takes it from there, and close() on the
+ * new socket frees the connection.
+ */
sk_acceptq_added(sk);
bh_unlock_sock(sk);
} else {
@@ -450,6 +451,24 @@ static int ax25_rcv(struct sk_buff *skb, struct net_device *dev,
ax25->window = ax25_dev->values[AX25_VALUES_WINDOW];
}
+ if (!sk) {
+ /*
+ * ax25_send_frame() may have created a connection for this
+ * link since the lookup above: add ours only if there is
+ * none, otherwise give the SABM to the existing one. Frames
+ * queued on ours before it reaches state 3 are sent by the
+ * ax25_kick() below.
+ */
+ existing = ax25_cb_add_unique(ax25);
+ if (existing) {
+ ax25_cb_put(ax25);
+ if (ax25_process_rx_frame(existing, skb, type, dama) == 0)
+ kfree_skb(skb);
+ ax25_cb_put(existing);
+ return 0;
+ }
+ }
+
ax25_send_control(ax25, AX25_UA, AX25_POLLON, AX25_RESPONSE);
#ifdef CONFIG_AX25_DAMA_SLAVE
@@ -459,15 +478,36 @@ static int ax25_rcv(struct sk_buff *skb, struct net_device *dev,
ax25->state = AX25_STATE_3;
- ax25_cb_add(ax25);
+ if (sk)
+ ax25_cb_add(ax25);
ax25_start_heartbeat(ax25);
ax25_start_t3timer(ax25);
ax25_start_idletimer(ax25);
+ if (!sk)
+ ax25_kick(ax25);
+
if (sk) {
- if (!sock_flag(sk, SOCK_DEAD))
+ bh_lock_sock(sk);
+ skb_set_owner_r(skb, make);
+ if (sk->sk_state == TCP_LISTEN && !sock_flag(sk, SOCK_DEAD)) {
+ skb_queue_head(&sk->sk_receive_queue, skb);
sk->sk_data_ready(sk);
+ } else {
+ /*
+ * The listener was closed meanwhile: leave the
+ * connection to its heartbeat, as
+ * ax25_destroy_socket() does with the connections
+ * that were still waiting on its queue.
+ */
+ sk_acceptq_removed(sk);
+ kfree_skb(skb);
+ sock_orphan(make);
+ make->sk_state = TCP_LISTEN;
+ ax25->state = AX25_STATE_0;
+ }
+ bh_unlock_sock(sk);
sock_put(sk);
} else {
free:
diff --git a/net/ax25/ax25_ip.c b/net/ax25/ax25_ip.c
index 215d4ccf12b9..b24cb88d75de 100644
--- a/net/ax25/ax25_ip.c
+++ b/net/ax25/ax25_ip.c
@@ -176,9 +176,14 @@ netdev_tx_t ax25_ip_xmit(struct sk_buff *skb)
ax25_dev->values[AX25_VALUES_PACLEN],
&src_c,
&dst_c, digipeat, dev);
- if (ax25) {
+ /*
+ * ax25_send_frame() leaves the skb to its caller
+ * when it gives up, as ROSE and NET/ROM expect.
+ */
+ if (ax25)
ax25_cb_put(ax25);
- }
+ else
+ kfree_skb(ourskb);
goto put;
}
}
diff --git a/net/ax25/ax25_out.c b/net/ax25/ax25_out.c
index 7c4ebef5ae6c..50b014f7b0df 100644
--- a/net/ax25/ax25_out.c
+++ b/net/ax25/ax25_out.c
@@ -137,7 +137,9 @@ EXPORT_SYMBOL(ax25_send_frame);
*/
void ax25_output(ax25_cb *ax25, int paclen, struct sk_buff *skb)
{
+ struct sk_buff_head frags;
struct sk_buff *skbn;
+ unsigned long flags;
unsigned char *p;
int frontlen, len, fragno, ka9qfrag, first = 1;
@@ -161,11 +163,20 @@ void ax25_output(ax25_cb *ax25, int paclen, struct sk_buff *skb)
frontlen = skb_headroom(skb); /* Address space + CTRL */
+ /*
+ * Build the fragments on a list of our own: a message that
+ * cannot be fragmented whole is not sent at all, and its
+ * fragments go on the write queue one after the other.
+ */
+ __skb_queue_head_init(&frags);
+
while (skb->len > 0) {
spin_lock_bh(&ax25_frag_lock);
if ((skbn = alloc_skb(paclen + 2 + frontlen, GFP_ATOMIC)) == NULL) {
spin_unlock_bh(&ax25_frag_lock);
printk(KERN_CRIT "AX.25: ax25_output - out of memory\n");
+ __skb_queue_purge(&frags);
+ kfree_skb(skb);
return;
}
@@ -200,10 +211,14 @@ void ax25_output(ax25_cb *ax25, int paclen, struct sk_buff *skb)
}
skb_pull(skb, len);
- skb_queue_tail(&ax25->write_queue, skbn); /* Throw it on the queue */
+ __skb_queue_tail(&frags, skbn);
}
kfree_skb(skb);
+
+ spin_lock_irqsave(&ax25->write_queue.lock, flags);
+ skb_queue_splice_tail_init(&frags, &ax25->write_queue);
+ spin_unlock_irqrestore(&ax25->write_queue.lock, flags);
} else {
skb_queue_tail(&ax25->write_queue, skb); /* Throw it on the queue */
}
diff --git a/net/ax25/ax25_std_timer.c b/net/ax25/ax25_std_timer.c
index b17da41210cb..05ff1032748f 100644
--- a/net/ax25/ax25_std_timer.c
+++ b/net/ax25/ax25_std_timer.c
@@ -143,7 +143,7 @@ void ax25_std_t1timer_expiry(ax25_cb *ax25)
case AX25_STATE_2:
if (ax25->n2count == ax25->n2) {
ax25_send_control(ax25, AX25_DISC, AX25_POLLON, AX25_COMMAND);
- if (!sock_flag(ax25->sk, SOCK_DESTROY))
+ if (!ax25->sk || !sock_flag(ax25->sk, SOCK_DESTROY))
ax25_disconnect(ax25, ETIMEDOUT);
return;
} else {
diff --git a/net/ax25/sysctl_net_ax25.c b/net/ax25/sysctl_net_ax25.c
index 68753aa30334..41a83e3a7e55 100644
--- a/net/ax25/sysctl_net_ax25.c
+++ b/net/ax25/sysctl_net_ax25.c
@@ -20,7 +20,7 @@ static int min_t2[] = {1}, max_t2[] = {20000};
static int min_t3[1], max_t3[] = {3600000};
static int min_idle[1], max_idle[] = {65535000};
static int min_n2[] = {1}, max_n2[] = {31};
-static int min_paclen[] = {1}, max_paclen[] = {512};
+static int min_paclen[] = {16}, max_paclen[] = {512};
static int min_proto[1], max_proto[] = { AX25_PROTO_MAX };
#ifdef CONFIG_AX25_DAMA_SLAVE
static int min_ds_timeout[1], max_ds_timeout[] = {65535000};
diff --git a/net/batman-adv/bat_iv_ogm.c b/net/batman-adv/bat_iv_ogm.c
index 6e79f69c2fed..38320f6a1dd1 100644
--- a/net/batman-adv/bat_iv_ogm.c
+++ b/net/batman-adv/bat_iv_ogm.c
@@ -1787,6 +1787,11 @@ static void batadv_iv_send_outstanding_bat_ogm_packet(struct work_struct *work)
goto out;
}
+ /* Mark aggregate as full before forcing emit. */
+ spin_lock_bh(&bat_priv->forw_bat_list_lock);
+ forw_packet->num_packets = BATADV_MAX_AGGREGATION_PACKETS;
+ spin_unlock_bh(&bat_priv->forw_bat_list_lock);
+
batadv_iv_ogm_emit(forw_packet);
/* we have to have at least one packet in the queue to determine the
diff --git a/net/batman-adv/distributed-arp-table.c b/net/batman-adv/distributed-arp-table.c
index 72717e2952ab..126a546fa033 100644
--- a/net/batman-adv/distributed-arp-table.c
+++ b/net/batman-adv/distributed-arp-table.c
@@ -582,6 +582,7 @@ static void batadv_choose_next_candidate(struct batadv_priv *bat_priv,
* NOT_FOUND
*/
cands[select].type = BATADV_DAT_CANDIDATE_NOT_FOUND;
+ cands[select].orig_node = NULL;
/* iterate over the originator list and find the node with the closest
* dat_address which has not been selected yet
diff --git a/net/batman-adv/main.c b/net/batman-adv/main.c
index f1b41078835b..6ffa65ef3d16 100644
--- a/net/batman-adv/main.c
+++ b/net/batman-adv/main.c
@@ -249,6 +249,11 @@ void batadv_mesh_free(struct net_device *mesh_iface)
atomic_set(&bat_priv->mesh_state, BATADV_MESH_DEACTIVATING);
+ /* the originator purge worker accesses (among others) the TT tables
+ * and must therefore be stopped before any of them are freed
+ */
+ disable_delayed_work_sync(&bat_priv->orig_work);
+
batadv_purge_outstanding_packets(bat_priv, NULL);
batadv_tp_stop_all(bat_priv);
@@ -427,8 +432,10 @@ int batadv_batman_skb_recv(struct sk_buff *skb, struct net_device *dev,
* shut down otherwise the packet may trigger de-reference errors
* further down in the receive path.
*/
- if (!kref_get_unless_zero(&hard_iface->refcount))
- goto err_out;
+ if (!kref_get_unless_zero(&hard_iface->refcount)) {
+ kfree_skb(skb);
+ return NET_RX_DROP;
+ }
skb = skb_share_check(skb, GFP_ATOMIC);
@@ -487,7 +494,6 @@ int batadv_batman_skb_recv(struct sk_buff *skb, struct net_device *dev,
kfree_skb(skb);
err_put:
batadv_hardif_put(hard_iface);
-err_out:
return NET_RX_DROP;
}
diff --git a/net/batman-adv/multicast_forw.c b/net/batman-adv/multicast_forw.c
index c99579dc1ca9..1638278178d9 100644
--- a/net/batman-adv/multicast_forw.c
+++ b/net/batman-adv/multicast_forw.c
@@ -950,8 +950,11 @@ static int batadv_mcast_forw_packet(struct batadv_priv *bat_priv,
dest = (u8 *)mcast_tracker + sizeof(*mcast_tracker);
- /* check if full tracker tvlv is within skb length */
+ /* check if full tracker tvlv, including padding, is within skb length */
tvlv_len = sizeof(*mcast_tracker) + ETH_ALEN * num_dests;
+ if (!(num_dests % 2))
+ tvlv_len += 2;
+
if (tvlv_len > skb_network_header_len(skb))
return -EINVAL;
diff --git a/net/batman-adv/tp_meter.c b/net/batman-adv/tp_meter.c
index 02af19aaaff2..aa297e551cd0 100644
--- a/net/batman-adv/tp_meter.c
+++ b/net/batman-adv/tp_meter.c
@@ -226,9 +226,9 @@ static void batadv_tp_batctl_notify(enum batadv_tp_meter_reason reason,
unsigned long start_time, u64 total_sent,
u32 cookie)
{
+ u64 total_bytes;
u32 test_time;
u8 result;
- u32 total_bytes;
if (!batadv_tp_is_error(reason)) {
result = BATADV_TP_REASON_COMPLETE;
diff --git a/net/batman-adv/translation-table.c b/net/batman-adv/translation-table.c
index 877475c68825..dfec5cce5fe5 100644
--- a/net/batman-adv/translation-table.c
+++ b/net/batman-adv/translation-table.c
@@ -415,6 +415,9 @@ batadv_tt_orig_list_entry_put(struct batadv_tt_orig_list_entry *orig_entry)
* @bat_priv: the bat priv with all the mesh interface information
* @tt_local_entry: the TT entry involved in the event
* @event_flags: flags to store in the event structure
+ *
+ * Only a single change per client is queued. It describes the latest state of
+ * the client and a new event therefore replaces an already queued one.
*/
static void batadv_tt_local_event(struct batadv_priv *bat_priv,
struct batadv_tt_local_entry *tt_local_entry,
@@ -423,14 +426,13 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv,
struct batadv_tt_change_node *tt_change_node, *entry, *safe;
struct batadv_tt_common_entry *common = &tt_local_entry->common;
u8 flags = common->flags | event_flags;
- bool del_op_requested, del_op_entry;
+ bool del_op_requested;
size_t changes;
tt_change_node = kmem_cache_alloc(batadv_tt_change_cache, GFP_ATOMIC);
if (!tt_change_node)
return;
- tt_change_node->change.flags = flags;
memset(tt_change_node->change.reserved, 0,
sizeof(tt_change_node->change.reserved));
ether_addr_copy(tt_change_node->change.addr, common->addr);
@@ -438,6 +440,15 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv,
del_op_requested = flags & BATADV_TT_CLIENT_DEL;
+ /* the ROAM flag is only meaningful for the receiver in combination
+ * with a DEL. An ADD with ROAM would be interpreted as a roaming
+ * advertisement
+ */
+ if (!del_op_requested)
+ flags &= ~BATADV_TT_CLIENT_ROAM;
+
+ tt_change_node->change.flags = flags;
+
/* check for ADD+DEL, DEL+ADD, ADD+ADD or DEL+DEL events */
spin_lock_bh(&bat_priv->tt.changes_list_lock);
changes = READ_ONCE(bat_priv->tt.local_changes);
@@ -449,25 +460,12 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv,
if (entry->change.vid != tt_change_node->change.vid)
continue;
- del_op_entry = entry->change.flags & BATADV_TT_CLIENT_DEL;
- if (del_op_requested != del_op_entry) {
- /* DEL+ADD in the same orig interval have no effect and
- * can be removed to avoid silly behaviour on the
- * receiver side. The other way around (ADD+DEL) can
- * happen in case of roaming of a client still in the
- * NEW state. Roaming of NEW clients is now possible due
- * to automatically recognition of "temporary" clients
- */
- list_del(&entry->list);
- kmem_cache_free(batadv_tt_change_cache, entry);
- changes--;
- } else {
- /* this is a second add or del in the same originator
- * interval. It could mean that flags have been changed
- * (e.g. double add): update them
- */
- entry->change.flags = flags;
- }
+ /* the other nodes may know the client with different
+ * flags. A DEL+ADD must therefore still announce the
+ * current flags and an ADD+DEL must still remove the
+ * client. ADD+ADD and DEL+DEL only update the flags
+ */
+ entry->change.flags = flags;
kmem_cache_free(batadv_tt_change_cache, tt_change_node);
goto update_changes;
@@ -604,6 +602,7 @@ bool batadv_tt_local_add(struct net_device *mesh_iface, const u8 *addr,
int hash_added, table_size, packet_size_max;
bool ret = false;
bool roamed_back = false;
+ bool added = false;
u8 remote_flags;
u32 match_mark;
@@ -630,7 +629,7 @@ bool batadv_tt_local_add(struct net_device *mesh_iface, const u8 *addr,
* flag can be reset like it was never enqueued
*/
tt_local->common.flags &= ~BATADV_TT_CLIENT_PENDING;
- goto add_event;
+ added = true;
}
if (tt_local->common.flags & BATADV_TT_CLIENT_ROAM) {
@@ -711,14 +710,17 @@ bool batadv_tt_local_add(struct net_device *mesh_iface, const u8 *addr,
goto out;
}
-add_event:
- batadv_tt_local_event(bat_priv, tt_local, BATADV_NO_FLAGS);
+ added = true;
check_roaming:
+ if (added)
+ batadv_tt_local_event(bat_priv, tt_local, BATADV_NO_FLAGS);
+
/* Check whether it is a roaming, but don't do anything if the roaming
* process has already been handled
*/
- if (tt_global && !(tt_global->common.flags & BATADV_TT_CLIENT_ROAM)) {
+ if (tt_global &&
+ (roamed_back || !(tt_global->common.flags & BATADV_TT_CLIENT_ROAM))) {
/* These node are probably going to update their tt table */
head = &tt_global->orig_list;
rcu_read_lock();
@@ -976,20 +978,22 @@ batadv_tt_prepare_tvlv_local_data(struct batadv_priv *bat_priv,
* batadv_tt_tvlv_container_update() - update the translation table tvlv
* container after local tt changes have been committed
* @bat_priv: the bat priv with all the mesh interface information
+ * @changes_list: list of committed changes which will be consumed
+ * @tt_diff_entries_num: number of entries in @changes_list
*/
-static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv)
+static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv,
+ struct list_head *changes_list,
+ int tt_diff_entries_num)
{
struct batadv_tt_change_node *entry, *safe;
struct batadv_tvlv_tt_data *tt_data;
struct batadv_tvlv_tt_change *tt_change;
int tt_diff_len, tt_change_len = 0;
- int tt_diff_entries_num = 0;
int tt_diff_entries_count = 0;
bool drop_changes = false;
size_t tt_extra_len = 0;
u16 tvlv_len;
- tt_diff_entries_num = READ_ONCE(bat_priv->tt.local_changes);
tt_diff_len = batadv_tt_len(tt_diff_entries_num);
/* if we have too many changes for one packet don't send any
@@ -1007,19 +1011,39 @@ static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv)
tvlv_len = batadv_tt_prepare_tvlv_local_data(bat_priv, &tt_data,
&tt_change, &tt_diff_len);
- if (!tvlv_len)
+ if (!tvlv_len) {
+ /* the changes are lost and the other nodes have to request the
+ * full table. Don't answer TT requests for the new TTVN with
+ * the stale changeset of the previous TTVN. Without changes, the
+ * TTVN was not incremented and the last changeset is still valid
+ */
+ if (drop_changes || tt_diff_entries_num > 0) {
+ list_for_each_entry_safe(entry, safe, changes_list,
+ list) {
+ list_del(&entry->list);
+ kmem_cache_free(batadv_tt_change_cache, entry);
+ }
+
+ spin_lock_bh(&bat_priv->tt.last_changeset_lock);
+ kfree(bat_priv->tt.last_changeset);
+ bat_priv->tt.last_changeset = NULL;
+ bat_priv->tt.last_changeset_len = 0;
+ spin_unlock_bh(&bat_priv->tt.last_changeset_lock);
+ }
+
+ /* announce the already committed TTVN in the next OGM interval
+ * instead of waiting for BATADV_TT_OGM_APPEND_MAX idle rounds
+ */
+ atomic_set(&bat_priv->tt.ogm_append_cnt, 0);
return;
+ }
tt_data->flags = BATADV_TT_OGM_DIFF;
if (!drop_changes && tt_diff_len == 0)
goto container_register;
- spin_lock_bh(&bat_priv->tt.changes_list_lock);
- WRITE_ONCE(bat_priv->tt.local_changes, 0);
-
- list_for_each_entry_safe(entry, safe, &bat_priv->tt.changes_list,
- list) {
+ list_for_each_entry_safe(entry, safe, changes_list, list) {
if (tt_diff_entries_count < tt_diff_entries_num) {
memcpy(tt_change + tt_diff_entries_count,
&entry->change,
@@ -1029,7 +1053,6 @@ static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv)
list_del(&entry->list);
kmem_cache_free(batadv_tt_change_cache, entry);
}
- spin_unlock_bh(&bat_priv->tt.changes_list_lock);
tt_extra_len = batadv_tt_len(tt_diff_entries_num -
tt_diff_entries_count);
@@ -2691,8 +2714,12 @@ static bool batadv_tt_local_valid(const void *entry_ptr,
if (tt_common_entry->flags & BATADV_TT_CLIENT_NEW)
return false;
+ /* the entry is (still) announced. A ROAM flag of an entry which is
+ * pending to be removed must not be transmitted. Otherwise, the
+ * receiver would handle it like a roaming advertisement
+ */
if (flags)
- *flags = tt_common_entry->flags;
+ *flags = tt_common_entry->flags & ~BATADV_TT_CLIENT_ROAM;
return true;
}
@@ -3252,20 +3279,29 @@ static void _batadv_tt_update_changes(struct batadv_priv *bat_priv,
{
int i;
int roams;
+ u8 flags;
for (i = 0; i < tt_num_changes; i++) {
- if ((tt_change + i)->flags & BATADV_TT_CLIENT_DEL) {
- roams = (tt_change + i)->flags & BATADV_TT_CLIENT_ROAM;
+ flags = (tt_change + i)->flags;
+
+ if (flags & BATADV_TT_CLIENT_DEL) {
+ roams = flags & BATADV_TT_CLIENT_ROAM;
batadv_tt_global_del(bat_priv, orig_node,
(tt_change + i)->addr,
ntohs((tt_change + i)->vid),
"tt removed by changes",
roams);
} else {
+ /* an announced client cannot have roamed away from
+ * orig_node. Only a ROAMING_ADV is allowed to add a
+ * global entry with the ROAM flag
+ */
+ flags &= ~BATADV_TT_CLIENT_ROAM;
+
if (!batadv_tt_global_add(bat_priv, orig_node,
(tt_change + i)->addr,
ntohs((tt_change + i)->vid),
- (tt_change + i)->flags, ttvn))
+ flags, ttvn))
/* In case of problem while storing a
* global_entry, we stop the updating
* procedure without committing the
@@ -3680,11 +3716,25 @@ static void batadv_tt_local_purge_pending_clients(struct batadv_priv *bat_priv)
*/
static void batadv_tt_local_commit_changes_nolock(struct batadv_priv *bat_priv)
{
+ LIST_HEAD(changes_list);
+ int changes_num;
+
lockdep_assert_held(&bat_priv->tt.commit_lock);
- if (READ_ONCE(bat_priv->tt.local_changes) == 0) {
+ /* atomically get the list of changes. The TTVN decision and the
+ * announced changeset must cover the same events. Events queued
+ * afterwards stay in the tt.changes_list for the next TT round
+ */
+ spin_lock_bh(&bat_priv->tt.changes_list_lock);
+ list_splice_init(&bat_priv->tt.changes_list, &changes_list);
+ changes_num = READ_ONCE(bat_priv->tt.local_changes);
+ WRITE_ONCE(bat_priv->tt.local_changes, 0);
+ spin_unlock_bh(&bat_priv->tt.changes_list_lock);
+
+ if (changes_num == 0) {
if (!batadv_atomic_dec_not_zero(&bat_priv->tt.ogm_append_cnt))
- batadv_tt_tvlv_container_update(bat_priv);
+ batadv_tt_tvlv_container_update(bat_priv, &changes_list,
+ changes_num);
return;
}
@@ -3701,7 +3751,7 @@ static void batadv_tt_local_commit_changes_nolock(struct batadv_priv *bat_priv)
/* reset the sending counter */
atomic_set(&bat_priv->tt.ogm_append_cnt, BATADV_TT_OGM_APPEND_MAX);
- batadv_tt_tvlv_container_update(bat_priv);
+ batadv_tt_tvlv_container_update(bat_priv, &changes_list, changes_num);
}
/**
diff --git a/net/batman-adv/types.h b/net/batman-adv/types.h
index 51d04f351fdd..1944398e0ed6 100644
--- a/net/batman-adv/types.h
+++ b/net/batman-adv/types.h
@@ -1970,7 +1970,13 @@ struct batadv_forw_packet {
/** @direct_link_flags: direct link flags for aggregated OGM packets */
DECLARE_BITMAP(direct_link_flags, BATADV_MAX_AGGREGATION_PACKETS);
- /** @num_packets: counter for aggregated OGMv1 packets */
+ /**
+ * @num_packets: counter for aggregated OGMv1 packets. Set to
+ * BATADV_MAX_AGGREGATION_PACKETS (independent of the real number of
+ * aggregated packets) by the send worker to prevent further
+ * aggregation while the packet is emitted. It must therefore not be
+ * used to iterate over the aggregated packets
+ */
u8 num_packets;
/** @delayed_work: work queue callback item for packet sending */
diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c
index 8da58c61e972..9cb1cdbff3fd 100644
--- a/net/bluetooth/hci_sync.c
+++ b/net/bluetooth/hci_sync.c
@@ -2708,7 +2708,7 @@ static int hci_pause_advertising_sync(struct hci_dev *hdev)
static int hci_resume_advertising_sync(struct hci_dev *hdev)
{
struct adv_info *adv, *tmp;
- int err;
+ int err = 0;
/* If advertising has not been paused there is nothing to do. */
if (!hdev->advertising_paused)
@@ -2741,13 +2741,19 @@ static int hci_resume_advertising_sync(struct hci_dev *hdev)
*/
if (hci_dev_test_and_clear_flag(hdev, HCI_LE_ADV_0))
err = hci_enable_ext_advertising_sync(hdev, 0x00);
- } else {
+ } else if (hdev->cur_adv_instance) {
/* Schedule for most recent instance to be restarted and begin
* the software rotation loop
*/
err = hci_schedule_adv_instance_sync(hdev,
hdev->cur_adv_instance,
true);
+ } else if (hci_dev_test_flag(hdev, HCI_ADVERTISING) &&
+ !hci_dev_test_flag(hdev, HCI_LE_ADV)) {
+ /* hci_schedule_adv_instance_sync() rejects instance 0x00 while
+ * HCI_ADVERTISING is set, so enable it directly.
+ */
+ err = hci_start_adv_sync(hdev, 0x00);
}
hdev->advertising_paused = false;
@@ -6140,6 +6146,7 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
u8 filter_policy = 0x00;
/* Default is to enable duplicates filter */
u8 filter_dup = LE_SCAN_FILTER_DUP_ENABLE;
+ bool paused;
int err;
bt_dev_dbg(hdev, "");
@@ -6163,6 +6170,12 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
if (err)
goto failed;
+ /* LE Set Random Address is disallowed while advertising is enabled. */
+ paused = !hdev->advertising_paused;
+ err = hci_pause_advertising_sync(hdev);
+ if (err)
+ goto failed;
+
/* All active scans will be done with either a resolvable private
* address (when privacy feature has been enabled) or non-resolvable
* private address.
@@ -6172,6 +6185,9 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
if (err < 0)
own_addr_type = ADDR_LE_DEV_PUBLIC;
+ if (paused)
+ hci_resume_advertising_sync(hdev);
+
if (hci_is_adv_monitoring(hdev) ||
(hci_test_quirk(hdev, HCI_QUIRK_STRICT_DUPLICATE_FILTER) &&
hdev->discovery.result_filtering)) {
@@ -6195,9 +6211,8 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
return err;
failed:
- /* Resume advertising if it was paused */
- if (ll_privacy_capable(hdev))
- hci_resume_advertising_sync(hdev);
+ /* No-op when advertising was not paused. */
+ hci_resume_advertising_sync(hdev);
/* Resume passive scanning */
hci_update_passive_scan_sync(hdev);
diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
index 0ba7b1ea435c..d648d5729954 100644
--- a/net/bluetooth/l2cap_core.c
+++ b/net/bluetooth/l2cap_core.c
@@ -161,6 +161,28 @@ static struct l2cap_chan *__l2cap_get_chan_by_ident(struct l2cap_conn *conn,
return NULL;
}
+/* Our connection request has been sent and is still outstanding */
+static bool l2cap_chan_connect_pending(struct l2cap_chan *chan)
+{
+ return chan->state == BT_CONNECT &&
+ test_bit(CONF_CONNECT_PEND, &chan->conf_state);
+}
+
+/* Other channels may hold the same ident for a response they still have
+ * to send, so only look at channels waiting for a connection response.
+ */
+static struct l2cap_chan *__l2cap_get_connect_chan_by_ident(struct l2cap_conn *conn,
+ u8 ident)
+{
+ struct l2cap_chan *c;
+
+ list_for_each_entry(c, &conn->chan_l, list) {
+ if (c->ident == ident && l2cap_chan_connect_pending(c))
+ return c;
+ }
+ return NULL;
+}
+
static struct l2cap_chan *__l2cap_global_chan_by_addr(__le16 psm, bdaddr_t *src,
u8 src_type)
{
@@ -4297,7 +4319,7 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn,
if (!chan)
return -EBADSLT;
} else {
- chan = __l2cap_get_chan_by_ident(conn, cmd->ident);
+ chan = __l2cap_get_connect_chan_by_ident(conn, cmd->ident);
if (!chan)
return -EBADSLT;
}
@@ -4310,6 +4332,13 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn,
l2cap_chan_lock(chan);
+ /* Only accept a response matching a connection request we sent and
+ * that is still outstanding. A channel waiting in BT_CONNECT for
+ * security or the feature mask has not sent one yet.
+ */
+ if (!l2cap_chan_connect_pending(chan) || chan->ident != cmd->ident)
+ goto done;
+
switch (result) {
case L2CAP_CR_SUCCESS:
if (__l2cap_get_chan_by_dcid(conn, dcid)) {
@@ -4339,6 +4368,7 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn,
break;
}
+done:
l2cap_chan_unlock(chan);
l2cap_chan_put(chan);
@@ -4471,7 +4501,13 @@ static inline int l2cap_config_req(struct l2cap_conn *conn,
goto unlock;
}
- if (!test_and_set_bit(CONF_REQ_SENT, &chan->conf_state)) {
+ /* A channel in BT_CONNECT2 has not passed security or userspace
+ * accept yet. Answer the peer's request, but hold our own until
+ * the channel reaches BT_CONFIG: the CONNECT2 -> CONFIG sites send
+ * it. Connected channels may also need to reconfigure.
+ */
+ if ((chan->state == BT_CONFIG || chan->state == BT_CONNECTED) &&
+ !test_and_set_bit(CONF_REQ_SENT, &chan->conf_state)) {
u8 buf[64];
l2cap_send_cmd(conn, l2cap_get_ident(conn), L2CAP_CONF_REQ,
l2cap_build_conf_req(chan, buf, sizeof(buf)), buf);
@@ -4519,6 +4555,10 @@ static inline int l2cap_config_rsp(struct l2cap_conn *conn,
if (!chan)
return 0;
+ /* Connected channels may also receive reconfiguration responses. */
+ if (chan->state != BT_CONFIG && chan->state != BT_CONNECTED)
+ goto done;
+
switch (result) {
case L2CAP_CONF_SUCCESS:
l2cap_conf_rfc_get(chan, rsp->data, len);
@@ -4586,6 +4626,9 @@ static inline int l2cap_config_rsp(struct l2cap_conn *conn,
if (test_bit(CONF_OUTPUT_DONE, &chan->conf_state)) {
set_default_fcs(chan);
+ if (chan->state != BT_CONFIG)
+ goto done;
+
if (chan->mode == L2CAP_MODE_ERTM ||
chan->mode == L2CAP_MODE_STREAMING)
err = l2cap_ertm_init(chan);
diff --git a/net/bluetooth/selftest.c b/net/bluetooth/selftest.c
index f49604d44b87..6abf70c14f59 100644
--- a/net/bluetooth/selftest.c
+++ b/net/bluetooth/selftest.c
@@ -191,6 +191,7 @@ static ssize_t test_ecdh_read(struct file *file, char __user *user_buf,
}
static const struct file_operations test_ecdh_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = test_ecdh_read,
.llseek = default_llseek,
diff --git a/net/bluetooth/smp.c b/net/bluetooth/smp.c
index 81bf67af87fc..e4cdbb7df16a 100644
--- a/net/bluetooth/smp.c
+++ b/net/bluetooth/smp.c
@@ -3735,6 +3735,7 @@ static ssize_t test_smp_read(struct file *file, char __user *user_buf,
}
static const struct file_operations test_smp_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = test_smp_read,
.llseek = default_llseek,
diff --git a/net/can/af_can.c b/net/can/af_can.c
index 52d72d56f2dd..c1e0569f777b 100644
--- a/net/can/af_can.c
+++ b/net/can/af_can.c
@@ -226,7 +226,7 @@ int can_send(struct sk_buff *skb, int loop)
goto inval_skb;
}
- if (unlikely(skb->dev->type != ARPHRD_CAN)) {
+ if (unlikely(!can_get_ml_priv(skb->dev))) {
err = -EPERM;
goto inval_skb;
}
@@ -452,7 +452,7 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
/* insert new receiver (dev,canid,mask) -> (func,data) */
- if (dev && (dev->type != ARPHRD_CAN || !can_get_ml_priv(dev)))
+ if (dev && !can_get_ml_priv(dev))
return -ENODEV;
if (dev && !net_eq(net, dev_net(dev)))
@@ -462,9 +462,10 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
if (!rcv)
return -ENOMEM;
- spin_lock_bh(&net->can.rcvlists_lock);
-
dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
+
+ spin_lock_bh(&dev_rcv_lists->lock);
+
rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists);
rcv->can_id = can_id;
@@ -478,6 +479,9 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
hlist_add_head_rcu(&rcv->list, rcv_list);
dev_rcv_lists->entries++;
+ spin_unlock_bh(&dev_rcv_lists->lock);
+
+ spin_lock_bh(&net->can.rcvlists_lock);
rcv_lists_stats->rcv_entries++;
rcv_lists_stats->rcv_entries_max = max(rcv_lists_stats->rcv_entries_max,
rcv_lists_stats->rcv_entries);
@@ -509,6 +513,14 @@ static void can_rx_delete_receiver(struct rcu_head *rp)
*
* Description:
* Removes subscription entry depending on given (subscription) values.
+ * The receive lists of a CAN device are protected by a per-device lock,
+ * so the subscription is found even when the device has been moved to
+ * another network namespace after can_rx_register(). The caller has to
+ * ensure that @dev stays valid, e.g. by holding a reference to @dev or
+ * RTNL. @net should be the namespace that has been used for
+ * can_rx_register(). It selects the 'any' CAN device lists for
+ * @dev == NULL and is otherwise only used for the receive list
+ * statistics, which become imprecise when another namespace is passed.
*/
void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
canid_t mask, void (*func)(struct sk_buff *, void *),
@@ -519,15 +531,13 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats;
struct can_dev_rcv_lists *dev_rcv_lists;
- if (dev && dev->type != ARPHRD_CAN)
+ if (dev && !can_get_ml_priv(dev))
return;
- if (dev && !net_eq(net, dev_net(dev)))
- return;
+ dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
- spin_lock_bh(&net->can.rcvlists_lock);
+ spin_lock_bh(&dev_rcv_lists->lock);
- dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists);
/* Search the receiver list for the item to delete. This should
@@ -547,26 +557,26 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
* a warning here.
*/
if (!rcv) {
+ spin_unlock_bh(&dev_rcv_lists->lock);
pr_warn("can: receive list entry not found for dev %s, id %03X, mask %03X\n",
DNAME(dev), can_id, mask);
- goto out;
+ return;
}
hlist_del_rcu(&rcv->list);
dev_rcv_lists->entries--;
+ spin_unlock_bh(&dev_rcv_lists->lock);
+
+ spin_lock_bh(&net->can.rcvlists_lock);
if (rcv_lists_stats->rcv_entries > 0)
rcv_lists_stats->rcv_entries--;
-
- out:
spin_unlock_bh(&net->can.rcvlists_lock);
/* schedule the receiver item for deletion */
- if (rcv) {
- if (rcv->sk)
- sock_hold(rcv->sk);
- call_rcu(&rcv->rcu, can_rx_delete_receiver);
- }
+ if (rcv->sk)
+ sock_hold(rcv->sk);
+ call_rcu(&rcv->rcu, can_rx_delete_receiver);
}
EXPORT_SYMBOL(can_rx_unregister);
@@ -685,7 +695,7 @@ static void can_receive(struct sk_buff *skb, struct net_device *dev)
static int can_rcv(struct sk_buff *skb, struct net_device *dev,
struct packet_type *pt, struct net_device *orig_dev)
{
- if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || !can_is_can_skb(skb))) {
+ if (unlikely(!can_get_ml_priv(dev) || !can_is_can_skb(skb))) {
pr_warn_once("PF_CAN: dropped non conform CAN skbuff: dev type %d, len %d\n",
dev->type, skb->len);
@@ -700,7 +710,7 @@ static int can_rcv(struct sk_buff *skb, struct net_device *dev,
static int canfd_rcv(struct sk_buff *skb, struct net_device *dev,
struct packet_type *pt, struct net_device *orig_dev)
{
- if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || !can_is_canfd_skb(skb))) {
+ if (unlikely(!can_get_ml_priv(dev) || !can_is_canfd_skb(skb))) {
pr_warn_once("PF_CAN: dropped non conform CAN FD skbuff: dev type %d, len %d\n",
dev->type, skb->len);
@@ -715,7 +725,7 @@ static int canfd_rcv(struct sk_buff *skb, struct net_device *dev,
static int canxl_rcv(struct sk_buff *skb, struct net_device *dev,
struct packet_type *pt, struct net_device *orig_dev)
{
- if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || !can_is_canxl_skb(skb))) {
+ if (unlikely(!can_get_ml_priv(dev) || !can_is_canxl_skb(skb))) {
pr_warn_once("PF_CAN: dropped non conform CAN XL skbuff: dev type %d, len %d\n",
dev->type, skb->len);
@@ -797,6 +807,7 @@ static int can_pernet_init(struct net *net)
kzalloc(sizeof(*net->can.rx_alldev_list), GFP_KERNEL);
if (!net->can.rx_alldev_list)
goto out;
+ spin_lock_init(&net->can.rx_alldev_list->lock);
net->can.pkg_stats = kzalloc(sizeof(*net->can.pkg_stats), GFP_KERNEL);
if (!net->can.pkg_stats)
goto out_free_rx_alldev_list;
diff --git a/net/can/bcm.c b/net/can/bcm.c
index 6d67af0a3b14..1c7c4ad3b759 100644
--- a/net/can/bcm.c
+++ b/net/can/bcm.c
@@ -54,6 +54,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/skb.h>
#include <linux/can/bcm.h>
@@ -956,7 +957,7 @@ static void bcm_remove_op(struct bcm_op *op)
static void bcm_rx_unreg(struct net_device *dev, struct bcm_op *op)
{
if (op->rx_reg_dev == dev) {
- can_rx_unregister(dev_net(dev), dev, op->can_id,
+ can_rx_unregister(sock_net(op->sk), dev, op->can_id,
REGMASK(op->can_id), bcm_rx_handler, op);
/* mark as removed subscription */
@@ -982,21 +983,10 @@ static int bcm_delete_rx_op(struct list_head *ops, struct bcm_msg_head *mh,
/* disable automatic timer on frame reception */
op->flags |= RX_NO_AUTOTIMER;
- /*
- * Don't care if we're bound or not (due to netdev
- * problems) can_rx_unregister() is always a save
- * thing to do here.
- */
if (op->ifindex) {
- /*
- * Only remove subscriptions that had not
- * been removed due to NETDEV_UNREGISTER
- * in bcm_notifier()
- *
- * op->rx_reg_dev is a tracked reference taken
- * when the subscription was registered, so it
- * stays valid here even if a concurrent
- * NETDEV_UNREGISTER already unlisted the dev.
+ /* Only remove subscriptions that had not been
+ * removed in bcm_notify(). op->rx_reg_dev keeps
+ * the device's receive lists accessible.
*/
if (op->rx_reg_dev)
bcm_rx_unreg(op->rx_reg_dev, op);
@@ -1706,7 +1696,7 @@ static int bcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
goto out_release;
}
- if (dev->type != ARPHRD_CAN) {
+ if (!can_get_ml_priv(dev)) {
dev_put(dev);
ret = -ENODEV;
goto out_release;
@@ -1854,7 +1844,7 @@ static int bcm_notifier(struct notifier_block *nb, unsigned long msg,
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN)
return NOTIFY_DONE;
@@ -1922,6 +1912,7 @@ static int bcm_release(struct socket *sock)
list_del(&bo->notifier);
spin_unlock(&bcm_notifier_lock);
+ /* bcm_notify() can not access this socket anymore */
lock_sock(sk);
#if IS_ENABLED(CONFIG_PROC_FS)
@@ -1935,29 +1926,20 @@ static int bcm_release(struct socket *sock)
bcm_remove_op(op);
}
- list_for_each_entry_safe(op, next, &bo->rx_ops, list) {
- /*
- * Don't care if we're bound or not (due to netdev problems)
- * can_rx_unregister() is always a save thing to do here.
- */
+ list_for_each_entry(op, &bo->rx_ops, list) {
if (op->ifindex) {
- /*
- * Only remove subscriptions that had not
- * been removed due to NETDEV_UNREGISTER
- * in bcm_notifier()
- *
- * op->rx_reg_dev is a tracked reference taken
- * when the subscription was registered, so it
- * stays valid here even if a concurrent
- * NETDEV_UNREGISTER already unlisted the device.
+ /* Only remove subscriptions that had not been removed
+ * in bcm_notify(). op->rx_reg_dev keeps the device's
+ * receive lists accessible.
*/
if (op->rx_reg_dev)
bcm_rx_unreg(op->rx_reg_dev, op);
- } else
+ } else {
can_rx_unregister(net, NULL, op->can_id,
REGMASK(op->can_id),
bcm_rx_handler, op);
-
+ }
+ cond_resched();
}
synchronize_rcu();
@@ -2011,7 +1993,7 @@ static int bcm_connect(struct socket *sock, struct sockaddr *uaddr, int len,
ret = -ENODEV;
goto fail;
}
- if (dev->type != ARPHRD_CAN) {
+ if (!can_get_ml_priv(dev)) {
dev_put(dev);
ret = -ENODEV;
goto fail;
diff --git a/net/can/gw.c b/net/can/gw.c
index 79fa58cb232e..de222fa93d6c 100644
--- a/net/can/gw.c
+++ b/net/can/gw.c
@@ -52,6 +52,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/skb.h>
#include <linux/can/gw.h>
@@ -603,7 +604,7 @@ static int cgw_notifier(struct notifier_block *nb,
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
struct net *net = dev_net(dev);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg == NETDEV_UNREGISTER) {
@@ -1154,7 +1155,7 @@ static int cgw_create_job(struct sk_buff *skb, struct nlmsghdr *nlh,
if (!gwj->src.dev)
goto out;
- if (gwj->src.dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(gwj->src.dev))
goto out;
gwj->dst.dev = __dev_get_by_index(net, gwj->ccgw.dst_idx);
@@ -1162,7 +1163,7 @@ static int cgw_create_job(struct sk_buff *skb, struct nlmsghdr *nlh,
if (!gwj->dst.dev)
goto out;
- if (gwj->dst.dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(gwj->dst.dev))
goto out;
/* is sending the skb back to the incoming interface intended? */
diff --git a/net/can/isotp.c b/net/can/isotp.c
index 6c88f7777965..3597e72a88ca 100644
--- a/net/can/isotp.c
+++ b/net/can/isotp.c
@@ -65,6 +65,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/skb.h>
#include <linux/can/isotp.h>
@@ -750,6 +751,14 @@ static void isotp_rcv(struct sk_buff *skb, void *data)
if (skb->len != so->ll.mtu)
return;
+ /* check for correct CAN CC/FD frame content */
+ if (so->ll.mtu == CAN_MTU) {
+ if (!can_is_can_skb(skb))
+ return;
+ } else if (!can_is_canfd_skb(skb)) {
+ return;
+ }
+
cf = (struct canfd_frame *)skb->data;
/* if enabled: check reception of my configured extended address */
@@ -1107,7 +1116,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
int off;
int err;
- if (!so->bound || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN)
+ if (!READ_ONCE(so->bound) || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN)
return -EADDRNOTAVAIL;
/* claim the socket under so->rx_lock: this serializes the claim
@@ -1152,7 +1161,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
/* so->bound is only checked once above - a wakeup may have
* unbound/rebound the socket meanwhile => recheck
*/
- if (!so->bound) {
+ if (!READ_ONCE(so->bound)) {
err = -EADDRNOTAVAIL;
goto err_out_drop;
}
@@ -1392,7 +1401,7 @@ static int isotp_recvmsg(struct socket *sock, struct msghdr *msg, size_t size,
if (flags & ~(MSG_DONTWAIT | MSG_TRUNC | MSG_PEEK | MSG_CMSG_COMPAT))
return -EINVAL;
- if (!so->bound)
+ if (!READ_ONCE(so->bound))
return -EADDRNOTAVAIL;
skb = skb_recv_datagram(sk, flags, &ret);
@@ -1466,11 +1475,11 @@ static int isotp_release(struct socket *sock)
list_del(&so->notifier);
spin_unlock(&isotp_notifier_lock);
- rtnl_lock();
lock_sock(sk);
/* remove current filters & unregister
- * tracked reference so->dev is taken at bind() time with rtnl_lock
+ * the tracked reference so->dev keeps the device's receive lists
+ * accessible, even after a namespace change
*/
if (so->bound && so->dev) {
if (isotp_register_rxid(so))
@@ -1485,11 +1494,9 @@ static int isotp_release(struct socket *sock)
}
so->ifindex = 0;
- so->bound = 0;
+ WRITE_ONCE(so->bound, 0);
so->dev = NULL;
- rtnl_unlock();
-
/* Always wait for a grace period before touching the timers below.
* A concurrent NETDEV_UNREGISTER may have already unregistered our
* filters and cleared so->bound in isotp_notify() without waiting
@@ -1519,7 +1526,6 @@ static int isotp_bind(struct socket *sock, struct sockaddr *uaddr, int len)
struct sock *sk = sock->sk;
struct isotp_sock *so = isotp_sk(sk);
struct net *net = sock_net(sk);
- int ifindex;
struct net_device *dev;
canid_t tx_id = addr->can_addr.tp.tx_id;
canid_t rx_id = addr->can_addr.tp.rx_id;
@@ -1557,7 +1563,7 @@ static int isotp_bind(struct socket *sock, struct sockaddr *uaddr, int len)
if (!addr->can_ifindex)
return -ENODEV;
- rtnl_lock();
+ /* lock_sock() serializes the binding with isotp_notify() */
lock_sock(sk);
if (so->bound) {
@@ -1583,61 +1589,83 @@ static int isotp_bind(struct socket *sock, struct sockaddr *uaddr, int len)
goto out;
}
- dev = dev_get_by_index(net, addr->can_ifindex);
- if (!dev) {
- err = -ENODEV;
- goto out;
- }
- if (dev->type != ARPHRD_CAN) {
+ /* Publish the device inside the RCU section of the lookup: When it
+ * is unlisted concurrently, either isotp_notify() sees so->dev after
+ * synchronize_net() or the lookup fails. Check the device before
+ * publishing it.
+ */
+ rcu_read_lock();
+ dev = dev_get_by_index_rcu(net, addr->can_ifindex);
+ if (!dev || !can_get_ml_priv(dev))
err = -ENODEV;
- goto out_put_dev;
- }
- if (READ_ONCE(dev->mtu) < so->ll.mtu) {
+ else if (READ_ONCE(dev->mtu) < so->ll.mtu)
err = -EINVAL;
- goto out_put_dev;
+ if (!err) {
+ dev_hold(dev);
+ WRITE_ONCE(so->dev, dev);
}
- if (!(dev->flags & IFF_UP))
- notify_enetdown = 1;
-
- ifindex = dev->ifindex;
+ rcu_read_unlock();
+ if (err)
+ goto out;
- if (isotp_register_rxid(so))
- can_rx_register(net, dev, rx_id, SINGLE_MASK(rx_id),
- isotp_rcv, sk, "isotp", sk);
+ /* switch to new settings */
+ so->ifindex = dev->ifindex;
+ so->rxid = rx_id;
+ so->txid = tx_id;
/* no consecutive frame echo skb in flight */
WRITE_ONCE(so->cfecho, 0);
- /* register for echo skb's */
- can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id),
- isotp_rcv_echo, sk, "isotpe", sk);
+ /* register for echo skb's first (no echo traffic at this point) */
+ err = can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id),
+ isotp_rcv_echo, sk, "isotpe", sk);
+ if (err)
+ goto out_put_dev;
- /* switch to new settings */
- so->ifindex = ifindex;
- so->rxid = rx_id;
- so->txid = tx_id;
- so->bound = 1;
+ if (isotp_register_rxid(so)) {
+ err = can_rx_register(net, dev, rx_id, SINGLE_MASK(rx_id),
+ isotp_rcv, sk, "isotp", sk);
+ if (err) {
+ /* unregister echo skb subscription */
+ can_rx_unregister(net, dev, tx_id, SINGLE_MASK(tx_id),
+ isotp_rcv_echo, sk);
+ goto out_put_dev;
+ }
+ }
+
+ WRITE_ONCE(so->bound, 1);
/* bind() ok -> hold a reference for so->dev so that isotp_release()
* can safely reach the device later, even if a concurrent
* NETDEV_UNREGISTER has already unlisted it by ifindex.
*/
- so->dev = dev;
netdev_hold(so->dev, &so->dev_tracker, GFP_KERNEL);
+ /* Either isotp_notify() sees so->dev on NETDEV_DOWN or we see the
+ * cleared IFF_UP, pairs with isotp_notify()
+ */
+ smp_mb();
+ if (!(READ_ONCE(dev->flags) & IFF_UP))
+ notify_enetdown = 1;
+
out_put_dev:
- /* remove potential reference from dev_get_by_index() */
+ /* revert so->ifindex and so->dev on failed bind() */
+ if (err) {
+ so->ifindex = 0;
+ WRITE_ONCE(so->dev, NULL);
+ }
+
+ /* remove reference from the device lookup */
dev_put(dev);
out:
- release_sock(sk);
- rtnl_unlock();
-
if (notify_enetdown) {
sk->sk_err = ENETDOWN;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
}
+ release_sock(sk);
+
return err;
}
@@ -1659,8 +1687,8 @@ static int isotp_getname(struct socket *sock, struct sockaddr *uaddr, int peer)
return ISOTP_MIN_NAMELEN;
}
-static int isotp_setsockopt_locked(struct socket *sock, int level, int optname,
- sockptr_t optval, unsigned int optlen)
+static int isotp_setsockopt_locked(struct socket *sock, int optname,
+ const void *val)
{
struct sock *sk = sock->sk;
struct isotp_sock *so = isotp_sk(sk);
@@ -1671,11 +1699,7 @@ static int isotp_setsockopt_locked(struct socket *sock, int level, int optname,
switch (optname) {
case CAN_ISOTP_OPTS:
- if (optlen != sizeof(struct can_isotp_options))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->opt, optval, optlen))
- return -EFAULT;
+ memcpy(&so->opt, val, sizeof(so->opt));
/* no separate rx_ext_address is given => use ext_address */
if (!(so->opt.flags & CAN_ISOTP_RX_EXT_ADDR))
@@ -1700,55 +1724,37 @@ static int isotp_setsockopt_locked(struct socket *sock, int level, int optname,
break;
case CAN_ISOTP_RECV_FC:
- if (optlen != sizeof(struct can_isotp_fc_options))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->rxfc, optval, optlen))
- return -EFAULT;
+ memcpy(&so->rxfc, val, sizeof(so->rxfc));
break;
case CAN_ISOTP_TX_STMIN:
- if (optlen != sizeof(u32))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->force_tx_stmin, optval, optlen))
- return -EFAULT;
+ so->force_tx_stmin = *(const u32 *)val;
break;
case CAN_ISOTP_RX_STMIN:
- if (optlen != sizeof(u32))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->force_rx_stmin, optval, optlen))
- return -EFAULT;
+ so->force_rx_stmin = *(const u32 *)val;
break;
- case CAN_ISOTP_LL_OPTS:
- if (optlen == sizeof(struct can_isotp_ll_options)) {
- struct can_isotp_ll_options ll;
-
- if (copy_from_sockptr(&ll, optval, optlen))
- return -EFAULT;
+ case CAN_ISOTP_LL_OPTS: {
+ const struct can_isotp_ll_options *ll = val;
- /* check for correct ISO 11898-1 DLC data length */
- if (ll.tx_dl != padlen(ll.tx_dl))
- return -EINVAL;
+ /* check for correct ISO 11898-1 DLC data length */
+ if (ll->tx_dl != padlen(ll->tx_dl))
+ return -EINVAL;
- if (ll.mtu != CAN_MTU && ll.mtu != CANFD_MTU)
- return -EINVAL;
+ if (ll->mtu != CAN_MTU && ll->mtu != CANFD_MTU)
+ return -EINVAL;
- if (ll.mtu == CAN_MTU &&
- (ll.tx_dl > CAN_MAX_DLEN || ll.tx_flags != 0))
- return -EINVAL;
+ if (ll->mtu == CAN_MTU &&
+ (ll->tx_dl > CAN_MAX_DLEN || ll->tx_flags != 0))
+ return -EINVAL;
- memcpy(&so->ll, &ll, sizeof(ll));
+ memcpy(&so->ll, ll, sizeof(*ll));
- /* set ll_dl for tx path to similar place as for rx */
- so->tx.ll_dl = ll.tx_dl;
- } else {
- return -EINVAL;
- }
+ /* set ll_dl for tx path to similar place as for rx */
+ so->tx.ll_dl = ll->tx_dl;
break;
+ }
default:
ret = -ENOPROTOOPT;
@@ -1762,13 +1768,49 @@ static int isotp_setsockopt(struct socket *sock, int level, int optname,
{
struct sock *sk = sock->sk;
+ union {
+ struct can_isotp_options opt;
+ struct can_isotp_fc_options rxfc;
+ struct can_isotp_ll_options ll;
+ u32 stmin;
+ } val;
+ unsigned int size;
int ret;
if (level != SOL_CAN_ISOTP)
return -EINVAL;
+ /* copy the option values from user space before lock_sock() */
+ switch (optname) {
+ case CAN_ISOTP_OPTS:
+ size = sizeof(val.opt);
+ break;
+
+ case CAN_ISOTP_RECV_FC:
+ size = sizeof(val.rxfc);
+ break;
+
+ case CAN_ISOTP_TX_STMIN:
+ case CAN_ISOTP_RX_STMIN:
+ size = sizeof(val.stmin);
+ break;
+
+ case CAN_ISOTP_LL_OPTS:
+ size = sizeof(val.ll);
+ break;
+
+ default:
+ return -ENOPROTOOPT;
+ }
+
+ if (optlen != size)
+ return -EINVAL;
+
+ if (copy_from_sockptr(&val, optval, size))
+ return -EFAULT;
+
lock_sock(sk);
- ret = isotp_setsockopt_locked(sock, level, optname, optval, optlen);
+ ret = isotp_setsockopt_locked(sock, optname, &val);
release_sock(sk);
return ret;
}
@@ -1833,12 +1875,21 @@ static void isotp_notify(struct isotp_sock *so, unsigned long msg,
if (!net_eq(dev_net(dev), sock_net(sk)))
return;
- if (so->dev != dev)
+ if (msg == NETDEV_DOWN)
+ smp_mb(); /* pairs with isotp_bind(), see raw_notify() */
+
+ /* lockless fast path, see raw_notify() */
+ if (READ_ONCE(so->dev) != dev)
return;
switch (msg) {
case NETDEV_UNREGISTER:
lock_sock(sk);
+ if (so->dev != dev) {
+ release_sock(sk);
+ break;
+ }
+
/* remove current filters & unregister */
if (so->bound) {
if (isotp_register_rxid(so))
@@ -1853,16 +1904,19 @@ static void isotp_notify(struct isotp_sock *so, unsigned long msg,
}
so->ifindex = 0;
- so->bound = 0;
- so->dev = NULL;
- release_sock(sk);
+ WRITE_ONCE(so->bound, 0);
+ WRITE_ONCE(so->dev, NULL);
sk->sk_err = ENODEV;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
+ release_sock(sk);
break;
case NETDEV_DOWN:
+ if (!READ_ONCE(so->bound))
+ break;
+
sk->sk_err = ENETDOWN;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
@@ -1875,7 +1929,7 @@ static int isotp_notifier(struct notifier_block *nb, unsigned long msg,
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN)
return NOTIFY_DONE;
diff --git a/net/can/proc.c b/net/can/proc.c
index de4d05ae3459..fb144d2adc35 100644
--- a/net/can/proc.c
+++ b/net/can/proc.c
@@ -76,16 +76,39 @@ static const char rx_list_name[][8] = {
* af_can statistics stuff
*/
+static void can_reset_pkg_stats(struct can_pkg_stats *pkg_stats)
+{
+ atomic_long_set(&pkg_stats->rx_frames, 0);
+ atomic_long_set(&pkg_stats->tx_frames, 0);
+ atomic_long_set(&pkg_stats->matches, 0);
+
+ pkg_stats->total_rx_rate = 0;
+ pkg_stats->total_tx_rate = 0;
+ pkg_stats->total_rx_match_ratio = 0;
+
+ pkg_stats->current_rx_rate = 0;
+ pkg_stats->current_tx_rate = 0;
+ pkg_stats->current_rx_match_ratio = 0;
+
+ pkg_stats->max_rx_rate = 0;
+ pkg_stats->max_tx_rate = 0;
+ pkg_stats->max_rx_match_ratio = 0;
+
+ atomic_long_set(&pkg_stats->rx_frames_delta, 0);
+ atomic_long_set(&pkg_stats->tx_frames_delta, 0);
+ atomic_long_set(&pkg_stats->matches_delta, 0);
+}
+
static void can_init_stats(struct net *net)
{
struct can_pkg_stats *pkg_stats = net->can.pkg_stats;
struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats;
/*
- * This memset function is called from a timer context (when
+ * This stats reset is called from a timer context (when
* can_stattimer is active which is the default) OR in a process
* context (reading the proc_fs when can_stattimer is disabled).
*/
- memset(pkg_stats, 0, sizeof(struct can_pkg_stats));
+ can_reset_pkg_stats(pkg_stats);
pkg_stats->jiffies_init = jiffies;
rcv_lists_stats->stats_reset++;
@@ -121,29 +144,23 @@ void can_stat_update(struct timer_list *t)
long rx_frames = atomic_long_read(&pkg_stats->rx_frames);
long tx_frames = atomic_long_read(&pkg_stats->tx_frames);
long matches = atomic_long_read(&pkg_stats->matches);
- long rx_frames_delta = atomic_long_read(&pkg_stats->rx_frames_delta);
- long tx_frames_delta = atomic_long_read(&pkg_stats->tx_frames_delta);
- long matches_delta = atomic_long_read(&pkg_stats->matches_delta);
+ /* fetch and clear values for 'current rate' calculation */
+ long rx_frames_delta = atomic_long_xchg(&pkg_stats->rx_frames_delta, 0);
+ long tx_frames_delta = atomic_long_xchg(&pkg_stats->tx_frames_delta, 0);
+ long matches_delta = atomic_long_xchg(&pkg_stats->matches_delta, 0);
- /* restart counting in timer context on user request */
- if (user_reset)
- can_init_stats(net);
-
- /* restart counting on jiffies overflow */
- if (j < pkg_stats->jiffies_init)
- can_init_stats(net);
-
- /* prevent overflow in calc_rate() */
- if (rx_frames > (LONG_MAX / HZ))
- can_init_stats(net);
-
- /* prevent overflow in calc_rate() */
- if (tx_frames > (LONG_MAX / HZ))
- can_init_stats(net);
-
- /* matches overflow - very improbable */
- if (matches > (LONG_MAX / 100))
+ /*
+ * Restart counting in timer context on user request, on jiffies
+ * overflow or to prevent overflows in calc_rate() and in the
+ * match ratio calculation. The local snapshots are stale after
+ * the reset, so skip the calculation for this interval.
+ */
+ if (user_reset || j < pkg_stats->jiffies_init ||
+ rx_frames > (LONG_MAX / HZ) || tx_frames > (LONG_MAX / HZ) ||
+ matches > (LONG_MAX / 100)) {
can_init_stats(net);
+ goto out;
+ }
/* calc total values */
if (rx_frames)
@@ -172,11 +189,7 @@ void can_stat_update(struct timer_list *t)
if (pkg_stats->max_rx_match_ratio < pkg_stats->current_rx_match_ratio)
pkg_stats->max_rx_match_ratio = pkg_stats->current_rx_match_ratio;
- /* clear values for 'current rate' calculation */
- atomic_long_set(&pkg_stats->tx_frames_delta, 0);
- atomic_long_set(&pkg_stats->rx_frames_delta, 0);
- atomic_long_set(&pkg_stats->matches_delta, 0);
-
+out:
/* restart timer (one second) */
mod_timer(&net->can.stattimer, round_jiffies(jiffies + HZ));
}
diff --git a/net/can/raw.c b/net/can/raw.c
index 9f0fa8baff85..1e357783292b 100644
--- a/net/can/raw.c
+++ b/net/can/raw.c
@@ -49,6 +49,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/dev.h> /* for can_is_canxl_dev_mtu() */
#include <linux/can/skb.h>
@@ -84,6 +85,7 @@ struct raw_sock {
struct sock sk;
struct net_device *dev;
netdevice_tracker dev_tracker;
+ struct net_device *bind_dev; /* hint for raw_notify(), no reference */
struct list_head notifier;
int ifindex;
unsigned int bound:1;
@@ -219,6 +221,7 @@ static int raw_enable_filters(struct net *net, struct net_device *dev,
int i;
for (i = 0; i < count; i++) {
+ cond_resched();
err = can_rx_register(net, dev, filter[i].can_id,
filter[i].can_mask,
raw_rcv, sk, "raw", sk);
@@ -253,9 +256,11 @@ static void raw_disable_filters(struct net *net, struct net_device *dev,
{
int i;
- for (i = 0; i < count; i++)
+ for (i = 0; i < count; i++) {
can_rx_unregister(net, dev, filter[i].can_id,
filter[i].can_mask, raw_rcv, sk);
+ cond_resched();
+ }
}
static inline void raw_disable_errfilter(struct net *net,
@@ -300,16 +305,31 @@ static void raw_notify(struct raw_sock *ro, unsigned long msg,
struct net_device *dev)
{
struct sock *sk = &ro->sk;
+ struct net_device *bind_dev;
if (!net_eq(dev_net(dev), sock_net(sk)))
return;
- if (ro->dev != dev)
+ if (msg == NETDEV_DOWN)
+ smp_mb(); /* IFF_UP is cleared before, pairs with raw_bind() */
+
+ /* Lockless fast path, ro->bind_dev covers a concurrent bind().
+ * Read it before ro->dev, pairs with smp_store_release() in
+ * raw_bind(). A stale ro->bind_dev only costs a lock_sock().
+ */
+ bind_dev = smp_load_acquire(&ro->bind_dev);
+ if (READ_ONCE(ro->dev) != dev &&
+ (msg != NETDEV_UNREGISTER || bind_dev != dev))
return;
switch (msg) {
case NETDEV_UNREGISTER:
lock_sock(sk);
+ if (ro->dev != dev) {
+ release_sock(sk);
+ break;
+ }
+
/* remove current filters & unregister */
if (ro->bound) {
raw_disable_allfilters(dev_net(dev), dev, sk);
@@ -321,13 +341,13 @@ static void raw_notify(struct raw_sock *ro, unsigned long msg,
ro->ifindex = 0;
ro->bound = 0;
- ro->dev = NULL;
+ WRITE_ONCE(ro->dev, NULL);
ro->count = 0;
- release_sock(sk);
sk->sk_err = ENODEV;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
+ release_sock(sk);
break;
case NETDEV_DOWN:
@@ -343,7 +363,7 @@ static int raw_notifier(struct notifier_block *nb, unsigned long msg,
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN)
return NOTIFY_DONE;
@@ -376,6 +396,7 @@ static int raw_init(struct sock *sk)
ro->bound = 0;
ro->ifindex = 0;
ro->dev = NULL;
+ ro->bind_dev = NULL;
/* set default filter to single entry dfilter */
ro->dfilter.can_id = 0;
@@ -426,13 +447,15 @@ static int raw_release(struct socket *sock)
list_del(&ro->notifier);
spin_unlock(&raw_notifier_lock);
- rtnl_lock();
lock_sock(sk);
- /* remove current filters & unregister */
+ /* remove current filters & unregister
+ * the tracked reference ro->dev keeps the device's receive lists
+ * accessible, even after a namespace change
+ */
if (ro->bound) {
if (ro->dev) {
- raw_disable_allfilters(dev_net(ro->dev), ro->dev, sk);
+ raw_disable_allfilters(net, ro->dev, sk);
netdev_put(ro->dev, &ro->dev_tracker);
} else {
raw_disable_allfilters(net, NULL, sk);
@@ -451,7 +474,6 @@ static int raw_release(struct socket *sock)
sock->sk = NULL;
release_sock(sk);
- rtnl_unlock();
sock_prot_inuse_add(net, sk->sk_prot, -1);
sock_put(sk);
@@ -474,26 +496,34 @@ static int raw_bind(struct socket *sock, struct sockaddr *uaddr, int len)
if (addr->can_family != AF_CAN)
return -EINVAL;
- rtnl_lock();
+ /* lock_sock() serializes the binding with raw_notify() */
lock_sock(sk);
if (ro->bound && addr->can_ifindex == ro->ifindex)
goto out;
if (addr->can_ifindex) {
- dev = dev_get_by_index(sock_net(sk), addr->can_ifindex);
+ /* Publish the device inside the RCU section of the lookup:
+ * When it is unlisted concurrently, either raw_notify() sees
+ * ro->bind_dev after synchronize_net() or the lookup fails.
+ */
+ rcu_read_lock();
+ dev = dev_get_by_index_rcu(sock_net(sk), addr->can_ifindex);
+ if (dev) {
+ dev_hold(dev);
+ /* pairs with smp_load_acquire() in raw_notify() */
+ smp_store_release(&ro->bind_dev, dev);
+ }
+ rcu_read_unlock();
if (!dev) {
err = -ENODEV;
goto out;
}
- if (dev->type != ARPHRD_CAN) {
+ if (!can_get_ml_priv(dev)) {
err = -ENODEV;
goto out_put_dev;
}
- if (!(dev->flags & IFF_UP))
- notify_enetdown = 1;
-
ifindex = dev->ifindex;
/* filters set by default/setsockopt */
@@ -512,7 +542,7 @@ static int raw_bind(struct socket *sock, struct sockaddr *uaddr, int len)
if (ro->bound) {
/* unregister old filters */
if (ro->dev) {
- raw_disable_allfilters(dev_net(ro->dev),
+ raw_disable_allfilters(sock_net(sk),
ro->dev, sk);
/* drop reference to old ro->dev */
netdev_put(ro->dev, &ro->dev_tracker);
@@ -523,24 +553,32 @@ static int raw_bind(struct socket *sock, struct sockaddr *uaddr, int len)
ro->ifindex = ifindex;
ro->bound = 1;
/* bind() ok -> hold a reference for new ro->dev */
- ro->dev = dev;
+ WRITE_ONCE(ro->dev, dev);
if (ro->dev)
netdev_hold(ro->dev, &ro->dev_tracker, GFP_KERNEL);
+
+ if (dev) {
+ /* Either raw_notify() sees ro->dev on NETDEV_DOWN or
+ * we see the cleared IFF_UP, pairs with raw_notify()
+ */
+ smp_mb();
+ if (!(READ_ONCE(dev->flags) & IFF_UP))
+ notify_enetdown = 1;
+ }
}
out_put_dev:
- /* remove potential reference from dev_get_by_index() */
+ /* remove reference from the device lookup */
dev_put(dev);
out:
- release_sock(sk);
- rtnl_unlock();
-
if (notify_enetdown) {
sk->sk_err = ENETDOWN;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
}
+ release_sock(sk);
+
return err;
}
@@ -562,58 +600,36 @@ static int raw_getname(struct socket *sock, struct sockaddr *uaddr,
}
static int raw_setsockopt_locked(struct socket *sock, int optname,
- sockptr_t optval, unsigned int optlen)
+ const void *val, struct can_filter *filter,
+ int count)
{
struct sock *sk = sock->sk;
struct raw_sock *ro = raw_sk(sk);
- struct can_filter *filter = NULL; /* dyn. alloc'ed filters */
- struct can_filter sfilter; /* single filter */
struct net_device *dev = NULL;
- can_err_mask_t err_mask = 0;
- int count = 0;
+ can_err_mask_t err_mask;
int flag;
int err = 0;
switch (optname) {
case CAN_RAW_FILTER:
- if (optlen % sizeof(struct can_filter) != 0)
- return -EINVAL;
-
- if (optlen > CAN_RAW_FILTER_MAX * sizeof(struct can_filter))
- return -EINVAL;
-
- count = optlen / sizeof(struct can_filter);
-
- if (count > 1) {
- /* filter does not fit into dfilter => alloc space */
- filter = memdup_sockptr(optval, optlen);
- if (IS_ERR(filter))
- return PTR_ERR(filter);
- } else if (count == 1) {
- if (copy_from_sockptr(&sfilter, optval, sizeof(sfilter)))
- return -EFAULT;
- }
-
+ /* @filter has been copied by raw_setsockopt(). It is linked
+ * to the socket on success and freed by the caller on error.
+ *
+ * The reg_state check is only a shortcut: On a concurrent
+ * unregistration or namespace change raw_notify() waits for
+ * lock_sock() and removes the filters set here.
+ */
dev = ro->dev;
- if (ro->bound && dev && dev->reg_state != NETREG_REGISTERED) {
- if (count > 1)
- kfree(filter);
+ if (ro->bound && dev &&
+ READ_ONCE(dev->reg_state) != NETREG_REGISTERED)
return -ENODEV;
- }
if (ro->bound) {
/* (try to) register the new filters */
- if (count == 1)
- err = raw_enable_filters(sock_net(sk), dev, sk,
- &sfilter, 1);
- else
- err = raw_enable_filters(sock_net(sk), dev, sk,
- filter, count);
- if (err) {
- if (count > 1)
- kfree(filter);
+ err = raw_enable_filters(sock_net(sk), dev, sk, filter,
+ count);
+ if (err)
return err;
- }
/* remove old filter registrations */
raw_disable_filters(sock_net(sk), dev, sk, ro->filter,
@@ -627,7 +643,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
/* link new filters to the socket */
if (count == 1) {
/* copy filter data for single filter */
- ro->dfilter = sfilter;
+ ro->dfilter = *filter;
filter = &ro->dfilter;
}
ro->filter = filter;
@@ -635,16 +651,12 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_ERR_FILTER:
- if (optlen != sizeof(err_mask))
- return -EINVAL;
-
- if (copy_from_sockptr(&err_mask, optval, optlen))
- return -EFAULT;
-
- err_mask &= CAN_ERR_MASK;
+ err_mask = *(const can_err_mask_t *)val & CAN_ERR_MASK;
+ /* lockless shortcut, see CAN_RAW_FILTER */
dev = ro->dev;
- if (ro->bound && dev && dev->reg_state != NETREG_REGISTERED)
+ if (ro->bound && dev &&
+ READ_ONCE(dev->reg_state) != NETREG_REGISTERED)
return -ENODEV;
/* remove current error mask */
@@ -666,31 +678,15 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_LOOPBACK:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
-
- ro->loopback = !!flag;
+ ro->loopback = !!*(const int *)val;
break;
case CAN_RAW_RECV_OWN_MSGS:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
-
- ro->recv_own_msgs = !!flag;
+ ro->recv_own_msgs = !!*(const int *)val;
break;
case CAN_RAW_FD_FRAMES:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
+ flag = *(const int *)val;
/* Enabling CAN XL includes CAN FD */
if (ro->xl_frames && !flag)
@@ -700,13 +696,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_XL_FRAMES:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
-
- ro->xl_frames = !!flag;
+ ro->xl_frames = !!*(const int *)val;
/* Enabling CAN XL includes CAN FD */
if (ro->xl_frames)
@@ -714,11 +704,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_XL_VCID_OPTS:
- if (optlen != sizeof(ro->raw_vcid_opts))
- return -EINVAL;
-
- if (copy_from_sockptr(&ro->raw_vcid_opts, optval, optlen))
- return -EFAULT;
+ memcpy(&ro->raw_vcid_opts, val, sizeof(ro->raw_vcid_opts));
/* prepare 32 bit values for handling in hot path */
ro->tx_vcid_shifted = ro->raw_vcid_opts.tx_vcid << CANXL_VCID_OFFSET;
@@ -727,13 +713,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_JOIN_FILTERS:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
-
- ro->join_filters = !!flag;
+ ro->join_filters = !!*(const int *)val;
break;
default:
@@ -746,18 +726,83 @@ static int raw_setsockopt(struct socket *sock, int level, int optname,
sockptr_t optval, unsigned int optlen)
{
struct sock *sk = sock->sk;
+ struct can_filter *filter = NULL; /* dyn. alloc'ed filters */
+ struct can_filter sfilter; /* single filter */
+ union {
+ can_err_mask_t err_mask;
+ int flag;
+ struct can_raw_vcid_options vcid_opts;
+ } val;
+ unsigned int size = 0;
+ int count = 0;
int err;
if (level != SOL_CAN_RAW)
return -EINVAL;
- rtnl_lock();
+ /* Copy the option values from user space before lock_sock(), so a
+ * stalled copy can not block raw_notify() waiting for lock_sock()
+ * under RTNL.
+ */
+ switch (optname) {
+ case CAN_RAW_FILTER:
+ if (optlen % sizeof(struct can_filter) != 0)
+ return -EINVAL;
+
+ if (optlen > CAN_RAW_FILTER_MAX * sizeof(struct can_filter))
+ return -EINVAL;
+
+ count = optlen / sizeof(struct can_filter);
+
+ if (count > 1) {
+ /* filter does not fit into dfilter => alloc space */
+ filter = memdup_sockptr(optval, optlen);
+ if (IS_ERR(filter))
+ return PTR_ERR(filter);
+ } else if (count == 1) {
+ if (copy_from_sockptr(&sfilter, optval, sizeof(sfilter)))
+ return -EFAULT;
+ filter = &sfilter;
+ }
+ break;
+
+ case CAN_RAW_ERR_FILTER:
+ size = sizeof(val.err_mask);
+ break;
+
+ case CAN_RAW_XL_VCID_OPTS:
+ size = sizeof(val.vcid_opts);
+ break;
+
+ case CAN_RAW_LOOPBACK:
+ case CAN_RAW_RECV_OWN_MSGS:
+ case CAN_RAW_FD_FRAMES:
+ case CAN_RAW_XL_FRAMES:
+ case CAN_RAW_JOIN_FILTERS:
+ size = sizeof(val.flag);
+ break;
+
+ default:
+ return -ENOPROTOOPT;
+ }
+
+ if (size) {
+ if (optlen != size)
+ return -EINVAL;
+
+ if (copy_from_sockptr(&val, optval, size))
+ return -EFAULT;
+ }
+
lock_sock(sk);
- err = raw_setsockopt_locked(sock, optname, optval, optlen);
+ err = raw_setsockopt_locked(sock, optname, &val, filter, count);
release_sock(sk);
- rtnl_unlock();
+
+ /* on error free the filters allocated for CAN_RAW_FILTER above */
+ if (err && count > 1)
+ kfree(filter);
return err;
}
@@ -780,29 +825,40 @@ static int raw_getsockopt(struct socket *sock, int level, int optname,
switch (optname) {
case CAN_RAW_FILTER: {
+ struct can_filter *filter = NULL;
+ int fsize = 0;
int err = 0;
lock_sock(sk);
if (ro->count > 0) {
- int fsize = ro->count * sizeof(struct can_filter);
+ fsize = ro->count * sizeof(struct can_filter);
/* user space buffer to small for filter list? */
if (len < fsize) {
/* return -ERANGE and needed space in optlen */
err = -ERANGE;
- if (put_user(fsize, optlen))
- err = -EFAULT;
} else {
if (len > fsize)
len = fsize;
- if (copy_to_user(optval, ro->filter, len))
- err = -EFAULT;
+ /* copy to user space after release_sock() */
+ filter = kmemdup(ro->filter, len, GFP_KERNEL);
+ if (!filter)
+ err = -ENOMEM;
}
} else {
len = 0;
}
release_sock(sk);
+ if (err == -ERANGE) {
+ if (put_user(fsize, optlen))
+ err = -EFAULT;
+ } else if (filter) {
+ if (copy_to_user(optval, filter, len))
+ err = -EFAULT;
+ kfree(filter);
+ }
+
if (!err)
err = put_user(len, optlen);
return err;
diff --git a/net/core/filter.c b/net/core/filter.c
index 692690316359..5c3c0939e2d2 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -2270,9 +2270,13 @@ static int bpf_out_neigh_v6(struct net *net, struct sk_buff *skb,
static int __bpf_redirect_neigh_v6(struct sk_buff *skb, struct net_device *dev,
struct bpf_nh_params *nh)
{
- const struct ipv6hdr *ip6h = ipv6_hdr(skb);
struct net *net = dev_net(dev);
int err, ret = NET_XMIT_DROP;
+ const struct ipv6hdr *ip6h;
+
+ if (unlikely(!pskb_may_pull(skb, sizeof(*ip6h))))
+ goto out_drop;
+ ip6h = ipv6_hdr(skb);
if (!nh) {
struct dst_entry *dst;
@@ -2375,9 +2379,13 @@ static int bpf_out_neigh_v4(struct net *net, struct sk_buff *skb,
static int __bpf_redirect_neigh_v4(struct sk_buff *skb, struct net_device *dev,
struct bpf_nh_params *nh)
{
- const struct iphdr *ip4h = ip_hdr(skb);
struct net *net = dev_net(dev);
int err, ret = NET_XMIT_DROP;
+ const struct iphdr *ip4h;
+
+ if (unlikely(!pskb_may_pull(skb, sizeof(*ip4h))))
+ goto out_drop;
+ ip4h = ip_hdr(skb);
if (!nh) {
struct flowi4 fl4 = {
@@ -2427,15 +2435,14 @@ static int __bpf_redirect_neigh_v4(struct sk_buff *skb, struct net_device *dev,
static int __bpf_redirect_neigh(struct sk_buff *skb, struct net_device *dev,
struct bpf_nh_params *nh)
{
- struct ethhdr *ethh = eth_hdr(skb);
-
- if (unlikely(skb->mac_header >= skb->network_header))
+ if (unlikely(skb_network_offset(skb) < ETH_HLEN ||
+ !pskb_may_pull(skb, ETH_HLEN)))
goto out;
bpf_push_mac_rcsum(skb);
- if (is_multicast_ether_addr(ethh->h_dest))
+ if (is_multicast_ether_addr(eth_hdr(skb)->h_dest))
goto out;
- skb_pull(skb, sizeof(*ethh));
+ skb_pull(skb, ETH_HLEN);
skb_unset_mac_header(skb);
skb_reset_network_header(skb);
@@ -7684,7 +7691,7 @@ BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags)
if (sk_unhashed(sk))
return -EOPNOTSUPP;
if (sk_is_refcounted(sk) &&
- unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
return -ENOENT;
skb_orphan(skb);
diff --git a/net/core/net_namespace.c b/net/core/net_namespace.c
index 4bece5eda9d1..ae1f9cc021b8 100644
--- a/net/core/net_namespace.c
+++ b/net/core/net_namespace.c
@@ -442,7 +442,9 @@ static __net_init int setup_net(struct net *net)
down_write(&net_rwsem);
list_add_tail_rcu(&net->list, &net_namespace_list);
up_write(&net_rwsem);
- ns_tree_add_raw(net);
+ /* Without CONFIG_NET_NS init_net isn't exposed as a namespace. */
+ if (IS_ENABLED(CONFIG_NET_NS))
+ ns_tree_add_raw(net);
out:
return error;
diff --git a/net/core/sock.c b/net/core/sock.c
index 03136176c7d0..486989dfaaee 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -3185,7 +3185,7 @@ void __release_sock(struct sock *sk)
spin_unlock_bh(&sk->sk_lock.slock);
- while (1) {
+ do {
next = skb->next;
prefetch(next);
DEBUG_NET_WARN_ON_ONCE(skb_dst_is_noref(skb));
@@ -3193,12 +3193,10 @@ void __release_sock(struct sock *sk)
sk_backlog_rcv(sk, skb);
skb = next;
- if (!skb)
- break;
if (!(++nb & 15))
cond_resched();
- }
+ } while (skb);
spin_lock_bh(&sk->sk_lock.slock);
}
diff --git a/net/ipv4/cipso_ipv4.c b/net/ipv4/cipso_ipv4.c
index e741f0a47afb..793c556c206d 100644
--- a/net/ipv4/cipso_ipv4.c
+++ b/net/ipv4/cipso_ipv4.c
@@ -2014,6 +2014,10 @@ static int cipso_v4_delopt(struct ip_options_rcu __rcu **opt_ptr)
cipso_ptr = &opt->opt.__data[cipso_off];
cipso_len = cipso_ptr[1];
+ optlen_new = cipso_v4_get_actual_opt_len(opt->opt.__data,
+ opt->opt.optlen);
+ optlen_new -= cipso_len;
+
if (opt->opt.srr > opt->opt.cipso)
opt->opt.srr -= cipso_len;
if (opt->opt.rr > opt->opt.cipso)
@@ -2027,11 +2031,12 @@ static int cipso_v4_delopt(struct ip_options_rcu __rcu **opt_ptr)
memmove(cipso_ptr, cipso_ptr + cipso_len,
opt->opt.optlen - cipso_off - cipso_len);
- optlen_new = cipso_v4_get_actual_opt_len(opt->opt.__data,
- opt->opt.optlen);
hdr_delta = opt->opt.optlen;
opt->opt.optlen = (optlen_new + 3) & ~3;
hdr_delta -= opt->opt.optlen;
+
+ memset(&opt->opt.__data[optlen_new], IPOPT_END,
+ opt->opt.optlen - optlen_new);
} else {
/* only the cipso option was present on the socket so we can
* remove the entire option struct */
diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c
index dd14944ad3ed..543a6f2d12df 100644
--- a/net/ipv4/inet_connection_sock.c
+++ b/net/ipv4/inet_connection_sock.c
@@ -1083,8 +1083,8 @@ static void reqsk_timer_handler(struct timer_list *t)
* hold another count to prevent use-after-free and
* call reqsk_put() just before return.
*/
- refcount_set(&nreq->rsk_refcnt, 2 + 1);
timer_setup(&nreq->rsk_timer, reqsk_timer_handler, TIMER_PINNED);
+ refcount_set_release(&nreq->rsk_refcnt, 2 + 1);
reqsk_queue_migrated(&inet_csk(nsk)->icsk_accept_queue, req);
req = nreq;
@@ -1461,7 +1461,7 @@ struct sock *inet_csk_complete_hashdance(struct sock *sk, struct sock *child,
goto child_put;
}
- refcount_set(&nreq->rsk_refcnt, 1);
+ refcount_set_release(&nreq->rsk_refcnt, 1);
if (inet_csk_reqsk_queue_add(sk, nreq, child)) {
__NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMIGRATEREQSUCCESS);
reqsk_migrate_reset(req);
@@ -1514,7 +1514,7 @@ void inet_csk_listen_stop(struct sock *sk)
if (nsk) {
nreq = inet_reqsk_clone(req, nsk);
if (nreq) {
- refcount_set(&nreq->rsk_refcnt, 1);
+ refcount_set_release(&nreq->rsk_refcnt, 1);
rcu_read_lock();
if (inet_csk_reqsk_queue_add(nsk, nreq, child)) {
diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c
index 49c70d752a89..c0accacdc3eb 100644
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -547,7 +547,7 @@ struct sock *__inet_lookup_established(const struct net *net,
if (sk->sk_hash != hash)
continue;
if (likely(inet_match(net, sk, acookie, ports, dif, sdif))) {
- if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
goto out;
if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) ||
!inet_match(net, sk, acookie, ports, dif,
diff --git a/net/ipv4/inet_timewait_sock.c b/net/ipv4/inet_timewait_sock.c
index c96d61d08854..8997cc9db752 100644
--- a/net/ipv4/inet_timewait_sock.c
+++ b/net/ipv4/inet_timewait_sock.c
@@ -161,7 +161,7 @@ void inet_twsk_hashdance_schedule(struct inet_timewait_sock *tw,
* Also note that after this point, we lost our implicit reference
* so we are not allowed to use tw anymore.
*/
- refcount_set(&tw->tw_refcnt, 3);
+ refcount_set_release(&tw->tw_refcnt, 3);
inet_twsk_schedule(tw, timeo);
@@ -339,7 +339,7 @@ void inet_twsk_purge(struct inet_hashinfo *hashinfo)
if (check_net(sock_net(sk)))
continue;
- if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
continue;
if (check_net(sock_net(sk))) {
diff --git a/net/ipv4/ip_gre.c b/net/ipv4/ip_gre.c
index b1ec4f92d346..7df1ada13c2a 100644
--- a/net/ipv4/ip_gre.c
+++ b/net/ipv4/ip_gre.c
@@ -1366,6 +1366,7 @@ static int erspan_tunnel_init(struct net_device *dev)
dev->features |= GRE_FEATURES;
dev->hw_features |= GRE_FEATURES;
dev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
+ dev->lltx = true;
netif_keep_dst(dev);
return ip_tunnel_init(dev);
diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c
index 66a7f25f236b..5133fd14029e 100644
--- a/net/ipv4/ipmr.c
+++ b/net/ipv4/ipmr.c
@@ -530,6 +530,12 @@ static netdev_tx_t reg_vif_xmit(struct sk_buff *skb, struct net_device *dev)
};
int err;
+ if (skb->protocol != htons(ETH_P_IP) || !pskb_inet_may_pull(skb)) {
+ DEV_STATS_INC(dev, tx_errors);
+ kfree_skb(skb);
+ return NETDEV_TX_OK;
+ }
+
rcu_read_lock();
err = ipmr_fib_lookup(net, &fl4, &mrt);
if (err < 0) {
@@ -1049,6 +1055,7 @@ static void ipmr_cache_resolve(struct net *net, struct mr_table *mrt,
static int ipmr_cache_report(const struct mr_table *mrt,
struct sk_buff *pkt, vifi_t vifi, int assert)
{
+ unsigned char pkt_cb[sizeof(pkt->cb)];
const int ihl = ip_hdrlen(pkt);
struct sock *mroute_sk;
struct igmphdr *igmp;
@@ -1104,17 +1111,25 @@ static int ipmr_cache_report(const struct mr_table *mrt,
msg = (struct igmpmsg *)skb_network_header(skb);
msg->im_vif = vifi;
msg->im_vif_hi = vifi >> 8;
- ipv4_pktinfo_prepare(mroute_sk, pkt, false);
- memcpy(skb->cb, pkt->cb, sizeof(skb->cb));
- /* Add our header */
- igmp = skb_put(skb, sizeof(struct igmphdr));
+ /* Add our header.
+ * Note that code, csum and group fields are cleared.
+ */
+ igmp = skb_put_zero(skb, sizeof(struct igmphdr));
igmp->type = assert;
msg->im_msgtype = assert;
- igmp->code = 0;
ip_hdr(skb)->tot_len = htons(skb->len); /* Fix the length */
skb->transport_header = skb->network_header;
}
+ memset(skb->cb, 0, sizeof(skb->cb));
+ /* Route queries from ipmr_get_route() carry NETLINK_CB, not IPCB. */
+ if (ip_hdr(pkt)->version) {
+ memcpy(pkt_cb, pkt->cb, sizeof(pkt_cb));
+ ipv4_pktinfo_prepare(mroute_sk, pkt, false);
+ *PKTINFO_SKB_CB(skb) = *PKTINFO_SKB_CB(pkt);
+ memcpy(pkt->cb, pkt_cb, sizeof(pkt_cb));
+ }
+
igmpmsg_netlink_event(mrt, skb);
/* Deliver to mrouted */
@@ -2046,7 +2061,8 @@ static void ip_mr_forward(struct net *net, struct mr_table *mrt,
}
/* Wrong interface: drop packet and (maybe) send PIM assert. */
- if (rcu_access_pointer(mrt->vif_table[vif].dev) != dev) {
+ if (true_vifi < 0 ||
+ rcu_access_pointer(mrt->vif_table[vif].dev) != dev) {
if (rt_is_output_route(skb_rtable(skb))) {
/* It is our own packet, looped back.
* Very complicated situation...
diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c
index 8b3d4e8ee455..d2db09bc7e29 100644
--- a/net/ipv4/syncookies.c
+++ b/net/ipv4/syncookies.c
@@ -205,7 +205,7 @@ struct sock *tcp_get_cookie_sock(struct sock *sk, struct sk_buff *skb,
child = icsk->icsk_af_ops->syn_recv_sock(sk, skb, req, dst,
NULL, &own_req, NULL);
if (child) {
- refcount_set(&req->rsk_refcnt, 1);
+ refcount_set_release(&req->rsk_refcnt, 1);
sock_rps_save_rxhash(child, skb);
if (rsk_drop_req(req)) {
diff --git a/net/ipv4/tcp_fastopen.c b/net/ipv4/tcp_fastopen.c
index 444306af444a..dbd97df1b442 100644
--- a/net/ipv4/tcp_fastopen.c
+++ b/net/ipv4/tcp_fastopen.c
@@ -278,7 +278,7 @@ static struct sock *tcp_fastopen_create_child(struct sock *sk,
tcp_reset_xmit_timer(child, ICSK_TIME_RETRANS,
req->timeout, false);
- refcount_set(&req->rsk_refcnt, 2);
+ refcount_set_release(&req->rsk_refcnt, 2);
sk_mark_napi_id_set(child, skb);
diff --git a/net/ipv4/xfrm4_output.c b/net/ipv4/xfrm4_output.c
index 0ae67d537499..fffa3c4f456e 100644
--- a/net/ipv4/xfrm4_output.c
+++ b/net/ipv4/xfrm4_output.c
@@ -38,9 +38,12 @@ int xfrm4_output(struct net *net, struct sock *sk, struct sk_buff *skb)
void xfrm4_local_error(struct sk_buff *skb, u32 mtu)
{
+ struct sock *sk = skb_to_full_sk(skb);
struct iphdr *hdr;
+ if (!sk)
+ return;
+
hdr = skb->encapsulation ? inner_ip_hdr(skb) : ip_hdr(skb);
- ip_local_error(skb->sk, EMSGSIZE, hdr->daddr,
- inet_sk(skb->sk)->inet_dport, mtu);
+ ip_local_error(sk, EMSGSIZE, hdr->daddr, inet_sk(sk)->inet_dport, mtu);
}
diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c
index 527ae9fc1c41..8f651cafcb2a 100644
--- a/net/ipv6/addrconf.c
+++ b/net/ipv6/addrconf.c
@@ -1484,7 +1484,8 @@ static int ipv6_create_tempaddr(struct inet6_ifaddr *ifp, bool block)
age = (now - tmp_tstamp + ADDRCONF_TIMER_FUZZ_MINUS) / HZ;
if (cfg.preferred_lft <= regen_advance + age) {
cfg.preferred_lft = regen_advance + age + 1;
- if (cfg.preferred_lft > cfg.valid_lft ||
+ if (cfg.preferred_lft <= regen_advance + age ||
+ cfg.preferred_lft > cfg.valid_lft ||
cfg.preferred_lft > if_public_preferred_lft) {
in6_ifa_put(ifp);
in6_dev_put(idev);
diff --git a/net/ipv6/datagram.c b/net/ipv6/datagram.c
index 97c655c26056..7edf30d8098d 100644
--- a/net/ipv6/datagram.c
+++ b/net/ipv6/datagram.c
@@ -148,6 +148,7 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr *uaddr,
__be32 fl6_flowlabel = 0;
__be32 old_fl6_flowlabel;
__be16 old_dport;
+ int old_bound_dev_if;
int addr_type;
int err;
@@ -218,6 +219,8 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr *uaddr,
goto out;
}
+ old_bound_dev_if = sk->sk_bound_dev_if;
+
if (__ipv6_addr_needs_scope_id(addr_type)) {
if (addr_len >= sizeof(struct sockaddr_in6) &&
usin->sin6_scope_id) {
@@ -260,6 +263,7 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr *uaddr,
sk->sk_v6_daddr = old_daddr;
np->flow_label = old_fl6_flowlabel;
inet->inet_dport = old_dport;
+ WRITE_ONCE(sk->sk_bound_dev_if, old_bound_dev_if);
goto out;
}
diff --git a/net/ipv6/inet6_hashtables.c b/net/ipv6/inet6_hashtables.c
index f4f39a20e951..bc3a496a47db 100644
--- a/net/ipv6/inet6_hashtables.c
+++ b/net/ipv6/inet6_hashtables.c
@@ -70,7 +70,7 @@ struct sock *__inet6_lookup_established(const struct net *net,
continue;
if (!inet6_match(net, sk, saddr, daddr, ports, dif, sdif))
continue;
- if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
goto out;
if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) ||
diff --git a/net/ipv6/ip6_gre.c b/net/ipv6/ip6_gre.c
index c14c76b78b98..17c5e9440bfa 100644
--- a/net/ipv6/ip6_gre.c
+++ b/net/ipv6/ip6_gre.c
@@ -1863,6 +1863,7 @@ static int ip6erspan_tap_init(struct net_device *dev)
dev->mtu -= 8;
dev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
+ dev->lltx = true;
ip6erspan_tnl_link_config(tunnel, 1);
netdev_hold(dev, &tunnel->dev_tracker, GFP_KERNEL);
diff --git a/net/ipv6/ip6mr.c b/net/ipv6/ip6mr.c
index 7b60e4b2a817..f61035619cac 100644
--- a/net/ipv6/ip6mr.c
+++ b/net/ipv6/ip6mr.c
@@ -620,7 +620,7 @@ static netdev_tx_t reg_vif_xmit(struct sk_buff *skb,
.flowi6_mark = skb->mark,
};
- if (!pskb_inet_may_pull(skb))
+ if (skb->protocol != htons(ETH_P_IPV6) || !pskb_inet_may_pull(skb))
goto tx_err;
if (ip6mr_fib_lookup(net, &fl6, &mrt) < 0)
@@ -2173,7 +2173,8 @@ static void ip6_mr_forward(struct net *net, struct mr_table *mrt,
/*
* Wrong interface: drop packet and (maybe) send PIM assert.
*/
- if (rcu_access_pointer(mrt->vif_table[vif].dev) != dev) {
+ if (true_vifi < 0 ||
+ rcu_access_pointer(mrt->vif_table[vif].dev) != dev) {
atomic_long_inc(&c->_c.mfc_un.res.wrong_if);
if (true_vifi >= 0 && mrt->mroute_do_assert &&
diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c
index b796d84db5ce..5ecbb9c775be 100644
--- a/net/ipv6/raw.c
+++ b/net/ipv6/raw.c
@@ -653,6 +653,9 @@ static int rawv6_send_hdrinc(struct sock *sk, struct msghdr *msg, int length,
if (unlikely(!skb))
return 0;
+ /* l3mdev_ip6_out() may have replaced the skb dst. */
+ rt = dst_rt6_info(skb_dst(skb));
+
/* Acquire rcu_read_lock() in case we need to use rt->rt6i_idev
* in the error path. Since skb has been freed, the dst could
* have been queued for deletion.
diff --git a/net/l2tp/l2tp_core.c b/net/l2tp/l2tp_core.c
index fd78928e9be2..e78e8ea93bb7 100644
--- a/net/l2tp/l2tp_core.c
+++ b/net/l2tp/l2tp_core.c
@@ -315,10 +315,19 @@ EXPORT_SYMBOL_GPL(l2tp_v2_session_get);
struct l2tp_session *l2tp_session_get(const struct net *net, struct sock *sk, int pver,
u32 tunnel_id, u32 session_id)
{
+ struct l2tp_session *session;
+
if (pver == L2TP_HDR_VER_2)
return l2tp_v2_session_get(net, tunnel_id, session_id);
- else
- return l2tp_v3_session_get(net, sk, session_id);
+
+ /* l2tp_v3_session_get() doesn't check sk if the ID hasn't collided */
+ session = l2tp_v3_session_get(net, sk, session_id);
+ if (session && session->tunnel->sock != sk) {
+ l2tp_session_put(session);
+ return NULL;
+ }
+
+ return session;
}
EXPORT_SYMBOL_GPL(l2tp_session_get);
diff --git a/net/mac80211/tdls.c b/net/mac80211/tdls.c
index 1dde37ad8bb2..83ebd9ab2e9b 100644
--- a/net/mac80211/tdls.c
+++ b/net/mac80211/tdls.c
@@ -2017,7 +2017,7 @@ void
ieee80211_process_tdls_channel_switch(struct ieee80211_sub_if_data *sdata,
struct sk_buff *skb)
{
- struct ieee80211_tdls_data *tf = (void *)skb->data;
+ struct ieee80211_tdls_data *tf;
struct wiphy *wiphy = sdata->local->hw.wiphy;
lockdep_assert_wiphy(wiphy);
@@ -2033,6 +2033,7 @@ ieee80211_process_tdls_channel_switch(struct ieee80211_sub_if_data *sdata,
* The packet/size was already validated by mac80211 Rx path, only look
* at the action type.
*/
+ tf = (void *)skb->data;
switch (tf->action_code) {
case WLAN_TDLS_CHANNEL_SWITCH_REQUEST:
ieee80211_process_tdls_channel_switch_req(sdata, skb);
diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
index 63867702ab08..85c90f7c24d8 100644
--- a/net/mac80211/tx.c
+++ b/net/mac80211/tx.c
@@ -269,7 +269,6 @@ ieee80211_tx_h_dynamic_ps(struct ieee80211_tx_data *tx)
static ieee80211_tx_result debug_noinline
ieee80211_tx_h_check_assoc(struct ieee80211_tx_data *tx)
{
-
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)tx->skb->data;
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(tx->skb);
bool assoc = false;
@@ -277,23 +276,6 @@ ieee80211_tx_h_check_assoc(struct ieee80211_tx_data *tx)
if (unlikely(info->flags & IEEE80211_TX_CTL_INJECTED))
return TX_CONTINUE;
- if (unlikely(test_bit(SCAN_SW_SCANNING, &tx->local->scanning)) &&
- test_bit(SDATA_STATE_OFFCHANNEL, &tx->sdata->state) &&
- !ieee80211_is_probe_req(hdr->frame_control) &&
- !ieee80211_is_any_nullfunc(hdr->frame_control))
- /*
- * When software scanning only nullfunc frames (to notify
- * the sleep state to the AP) and probe requests (for the
- * active scan) are allowed, all other frames should not be
- * sent and we should not get here, but if we do
- * nonetheless, drop them to avoid sending them
- * off-channel. See the link below and
- * ieee80211_start_scan() for more.
- *
- * http://article.gmane.org/gmane.linux.kernel.wireless.general/30089
- */
- return TX_DROP;
-
if (tx->sdata->vif.type == NL80211_IFTYPE_OCB)
return TX_CONTINUE;
@@ -806,6 +788,33 @@ ieee80211_tx_h_rate_ctrl(struct ieee80211_tx_data *tx)
return TX_CONTINUE;
}
+static ieee80211_tx_result debug_noinline
+ieee80211_tx_h_check_offchannel(struct ieee80211_tx_data *tx)
+{
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)tx->skb->data;
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(tx->skb);
+
+ if (unlikely(info->flags & IEEE80211_TX_CTL_INJECTED))
+ return TX_CONTINUE;
+
+ if (unlikely(test_bit(SCAN_SW_SCANNING, &tx->local->scanning)) &&
+ test_bit(SDATA_STATE_OFFCHANNEL, &tx->sdata->state) &&
+ !ieee80211_is_probe_req(hdr->frame_control) &&
+ !ieee80211_is_any_nullfunc(hdr->frame_control)) {
+ /*
+ * When software scanning only nullfunc frames (to notify
+ * the sleep state to the AP) and probe requests (for the
+ * active scan) are allowed, all other frames should not be
+ * sent and we should not get here, but if we do
+ * nonetheless, drop them to avoid sending them
+ * off-channel. See __ieee80211_start_scan() for more.
+ */
+ return TX_DROP;
+ }
+
+ return TX_CONTINUE;
+}
+
static __le16 ieee80211_tx_next_seq(struct sta_info *sta, int tid)
{
u16 *seq = &sta->tid_seq[tid];
@@ -1115,6 +1124,7 @@ static bool ieee80211_tx_prep_agg(struct ieee80211_tx_data *tx,
struct sk_buff *purge_skb = NULL;
if (test_bit(HT_AGG_STATE_OPERATIONAL, &tid_tx->state)) {
+ info->flags |= IEEE80211_TX_CTL_AMPDU;
reset_agg_timer = true;
} else if (test_bit(HT_AGG_STATE_WANT_START, &tid_tx->state)) {
/*
@@ -1146,6 +1156,7 @@ static bool ieee80211_tx_prep_agg(struct ieee80211_tx_data *tx,
if (!tid_tx) {
/* do nothing, let packet pass through */
} else if (test_bit(HT_AGG_STATE_OPERATIONAL, &tid_tx->state)) {
+ info->flags |= IEEE80211_TX_CTL_AMPDU;
reset_agg_timer = true;
} else {
queued = true;
@@ -1852,6 +1863,7 @@ static int invoke_tx_handlers_late(struct ieee80211_tx_data *tx)
goto txh_done;
}
+ CALL_TXH(ieee80211_tx_h_check_offchannel);
CALL_TXH(ieee80211_tx_h_michael_mic_add);
CALL_TXH(ieee80211_tx_h_sequence);
CALL_TXH(ieee80211_tx_h_fragment);
@@ -4691,10 +4703,6 @@ static void ieee80211_8023_xmit(struct ieee80211_sub_if_data *sdata,
queue = ieee80211_select_queue(sdata, sta, skb);
skb_set_queue_mapping(skb, queue);
- if (unlikely(test_bit(SCAN_SW_SCANNING, &local->scanning)) &&
- test_bit(SDATA_STATE_OFFCHANNEL, &sdata->state))
- goto out_free;
-
skb = skb_share_check(skb, GFP_ATOMIC);
if (unlikely(!skb))
return;
@@ -4771,11 +4779,6 @@ static void ieee80211_8023_xmit(struct ieee80211_sub_if_data *sdata,
ieee80211_tpt_led_trig_tx(local, len);
ieee80211_tx_8023(sdata, skb, sta, false);
-
- return;
-
-out_free:
- kfree_skb(skb);
}
static bool ieee80211_check_mcast_offload(struct ieee80211_sub_if_data *sdata,
diff --git a/net/mptcp/pm_kernel.c b/net/mptcp/pm_kernel.c
index fa34a333a311..cf635b84fff4 100644
--- a/net/mptcp/pm_kernel.c
+++ b/net/mptcp/pm_kernel.c
@@ -881,6 +881,7 @@ static int mptcp_pm_nl_create_listen_socket(struct sock *sk,
* old status is known (TCP_CLOSE).
*/
inet_sk_state_store(newsk, TCP_LISTEN);
+ sock_set_flag(newsk, SOCK_RCU_FREE);
lock_sock(ssk);
WRITE_ONCE(mptcp_subflow_ctx(ssk)->pm_listener, true);
err = __inet_listen_sk(ssk, backlog);
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index df09dcfd15ce..fdd4c7d87775 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -3410,6 +3410,24 @@ static void mptcp_copy_ip_options(struct sock *newsk, const struct sock *sk)
rcu_read_unlock();
}
+static void mptcp_sk_clone_destroy(struct sock *nsk)
+{
+ struct mptcp_sock *msk = mptcp_sk(nsk);
+
+ mptcp_release_sched(msk);
+ mptcp_set_state(nsk, TCP_CLOSE);
+ /* inet_csk_prepare_forced_close() clears TCP sock_ops state via
+ * tcp_sk(), but nsk is an MPTCP master socket; keep the inet-level
+ * destroy preparation here.
+ */
+ bh_unlock_sock(nsk);
+ sock_put(nsk);
+ sock_set_flag(nsk, SOCK_DEAD);
+ tcp_orphan_count_inc();
+ inet_sk(nsk)->inet_num = 0;
+ inet_csk_destroy_sock(nsk);
+}
+
struct sock *mptcp_sk_clone_init(const struct sock *sk,
const struct mptcp_options_received *mp_opt,
struct sock *ssk,
@@ -3469,11 +3487,6 @@ struct sock *mptcp_sk_clone_init(const struct sock *sk,
list_add(&subflow->node, &msk->conn_list);
sock_hold(ssk);
- /* new mpc subflow takes ownership of the newly
- * created mptcp socket
- */
- mptcp_token_accept(subflow_req, msk);
-
/* set msk addresses early to ensure mptcp_pm_get_local_id()
* uses the correct data
*/
@@ -3482,6 +3495,14 @@ struct sock *mptcp_sk_clone_init(const struct sock *sk,
mptcp_rcv_space_init(msk, ssk);
msk->rcvq_space.time = mptcp_stamp();
+ if (!mptcp_token_accept(subflow_req, msk)) {
+ list_del_init(&subflow->node);
+ WRITE_ONCE(msk->first, NULL);
+ sock_put(ssk);
+ mptcp_sk_clone_destroy(nsk);
+ return NULL;
+ }
+
if (mp_opt->suboptions & OPTION_MPTCP_MPC_ACK)
__mptcp_subflow_fully_established(msk, subflow, mp_opt);
bh_unlock_sock(nsk);
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index 184831b5b624..1b6335408b17 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -1033,9 +1033,11 @@ static inline void mptcp_token_init_request(struct request_sock *req)
}
int mptcp_token_new_request(struct request_sock *req);
+void mptcp_token_move_request(struct request_sock *req,
+ struct request_sock *new_req);
void mptcp_token_destroy_request(struct request_sock *req);
int mptcp_token_new_connect(struct sock *ssk);
-void mptcp_token_accept(struct mptcp_subflow_request_sock *r,
+bool mptcp_token_accept(struct mptcp_subflow_request_sock *r,
struct mptcp_sock *msk);
bool mptcp_token_exists(u32 token);
struct mptcp_sock *mptcp_token_get_sock(struct net *net, u32 token);
diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c
index e020960f093e..0f8b7396f446 100644
--- a/net/mptcp/subflow.c
+++ b/net/mptcp/subflow.c
@@ -78,6 +78,8 @@ void mptcp_subflow_reqsk_clone(struct request_sock *req,
}
new_subflow_req->msk = msk;
+
+ mptcp_token_move_request(req, new_req);
}
static void subflow_generate_hmac(u64 key1, u64 key2, u32 nonce1, u32 nonce2,
@@ -380,6 +382,10 @@ static void subflow_prep_synack(const struct sock *sk, struct request_sock *req,
struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
struct inet_request_sock *ireq = inet_rsk(req);
+ /* MP_JOIN cannot use TFO, do not send a cookie in the SYN/ACK */
+ if (foc && mptcp_subflow_rsk(req)->mp_join)
+ foc->len = -1;
+
/* clear tstamp_ok, as needed depending on cookie */
if (foc && foc->len > -1)
ireq->tstamp_ok = 0;
@@ -889,6 +895,13 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
fallback = true;
} else if (subflow_req->mp_join) {
+ /* a SYN skb here comes from TFO, which MP_JOIN cannot use: just
+ * fall back to the regular path.
+ */
+ if ((TCP_SKB_CB(skb)->tcp_flags & (TCPHDR_SYN | TCPHDR_ACK)) ==
+ TCPHDR_SYN)
+ return NULL;
+
mptcp_get_options(skb, &mp_opt);
if (!(mp_opt.suboptions & OPTION_MPTCP_MPJ_ACK))
fallback = true;
@@ -920,8 +933,10 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
if (ctx->mp_capable) {
ctx->conn = mptcp_sk_clone_init(listener->conn, &mp_opt, child, req);
- if (!ctx->conn)
+ if (!ctx->conn) {
+ fallback = true;
goto fallback;
+ }
ctx->subflow_id = 1;
owner = mptcp_sk(ctx->conn);
diff --git a/net/mptcp/token.c b/net/mptcp/token.c
index 5bb924534387..d5735d3ac793 100644
--- a/net/mptcp/token.c
+++ b/net/mptcp/token.c
@@ -180,6 +180,43 @@ int mptcp_token_new_connect(struct sock *ssk)
return 0;
}
+/**
+ * mptcp_token_move_request - move request token ownership to a clone
+ * @req: original request socket
+ * @new_req: cloned request socket
+ *
+ * Move the token hash entry from the original request to its clone.
+ */
+void mptcp_token_move_request(struct request_sock *req,
+ struct request_sock *new_req)
+{
+ struct mptcp_subflow_request_sock *subflow_req = mptcp_subflow_rsk(req);
+ struct mptcp_subflow_request_sock *new_subflow_req;
+ struct mptcp_subflow_request_sock *pos;
+ struct token_bucket *bucket;
+
+ new_subflow_req = mptcp_subflow_rsk(new_req);
+
+ if (hlist_nulls_unhashed_lockless(&subflow_req->token_node)) {
+ mptcp_token_init_request(new_req);
+ return;
+ }
+
+ bucket = token_bucket(subflow_req->token);
+ spin_lock_bh(&bucket->lock);
+ if (hlist_nulls_unhashed(&subflow_req->token_node)) {
+ mptcp_token_init_request(new_req);
+ } else {
+ pos = __token_lookup_req(bucket, subflow_req->token);
+ if (pos == subflow_req)
+ hlist_nulls_replace_init_rcu(&subflow_req->token_node,
+ &new_subflow_req->token_node);
+ else
+ mptcp_token_init_request(new_req);
+ }
+ spin_unlock_bh(&bucket->lock);
+}
+
/**
* mptcp_token_accept - replace a req sk with full sock in token hash
* @req: the request socket to be removed
@@ -187,24 +224,36 @@ int mptcp_token_new_connect(struct sock *ssk)
*
* Called when a SYN packet creates a new logical connection, i.e.
* is not a join request.
+ *
+ * Return: true on success.
*/
-void mptcp_token_accept(struct mptcp_subflow_request_sock *req,
+bool mptcp_token_accept(struct mptcp_subflow_request_sock *req,
struct mptcp_sock *msk)
{
struct mptcp_subflow_request_sock *pos;
struct sock *sk = (struct sock *)msk;
struct token_bucket *bucket;
+ bool ret = false;
- sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
bucket = token_bucket(req->token);
spin_lock_bh(&bucket->lock);
+ if (hlist_nulls_unhashed(&req->token_node))
+ goto unlock;
- /* pedantic lookup check for the moved token */
pos = __token_lookup_req(bucket, req->token);
- if (!WARN_ON_ONCE(pos != req))
- hlist_nulls_del_init_rcu(&req->token_node);
+ if (pos != req)
+ goto unlock;
+
+ hlist_nulls_del_init_rcu(&req->token_node);
__sk_nulls_add_node_rcu((struct sock *)msk, &bucket->msk_chain);
+ ret = true;
+
+unlock:
spin_unlock_bh(&bucket->lock);
+ if (ret)
+ sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
+
+ return ret;
}
bool mptcp_token_exists(u32 token)
@@ -353,16 +402,21 @@ void mptcp_token_destroy_request(struct request_sock *req)
struct mptcp_subflow_request_sock *pos;
struct token_bucket *bucket;
- if (hlist_nulls_unhashed(&subflow_req->token_node))
+ if (hlist_nulls_unhashed_lockless(&subflow_req->token_node))
return;
bucket = token_bucket(subflow_req->token);
spin_lock_bh(&bucket->lock);
+ if (hlist_nulls_unhashed(&subflow_req->token_node))
+ goto unlock;
+
pos = __token_lookup_req(bucket, subflow_req->token);
- if (!WARN_ON_ONCE(pos != subflow_req)) {
+ if (pos == subflow_req) {
hlist_nulls_del_init_rcu(&pos->token_node);
bucket->chain_len--;
}
+
+unlock:
spin_unlock_bh(&bucket->lock);
}
diff --git a/net/mptcp/token_test.c b/net/mptcp/token_test.c
index 4fc39fa2e262..be9acce8a567 100644
--- a/net/mptcp/token_test.c
+++ b/net/mptcp/token_test.c
@@ -99,7 +99,7 @@ static void mptcp_token_test_accept(struct kunit *test)
KUNIT_ASSERT_EQ(test, 0,
mptcp_token_new_request((struct request_sock *)req));
msk->token = req->token;
- mptcp_token_accept(req, msk);
+ KUNIT_EXPECT_TRUE(test, mptcp_token_accept(req, msk));
KUNIT_EXPECT_PTR_EQ(test, msk, mptcp_token_get_sock(&init_net, msk->token));
/* this is now a no-op */
@@ -122,7 +122,7 @@ static void mptcp_token_test_destroyed(struct kunit *test)
KUNIT_ASSERT_EQ(test, 0,
mptcp_token_new_request((struct request_sock *)req));
msk->token = req->token;
- mptcp_token_accept(req, msk);
+ KUNIT_EXPECT_TRUE(test, mptcp_token_accept(req, msk));
/* simulate race on removal */
refcount_set(&sk->sk_refcnt, 0);
diff --git a/net/netrom/af_netrom.c b/net/netrom/af_netrom.c
index c9433b1a2872..86c0fca77905 100644
--- a/net/netrom/af_netrom.c
+++ b/net/netrom/af_netrom.c
@@ -96,16 +96,22 @@ static void nr_remove_socket(struct sock *sk)
}
/*
- * Kill all bound sockets on a dropped device.
+ * Kill all bound sockets on a dropped device. Each socket is disconnected
+ * under its lock, which its timers and nr_rx_frame() hold while they work
+ * on it; nr_info_show() takes the two locks in the same order.
*/
static void nr_kill_by_device(struct net_device *dev)
{
struct sock *s;
spin_lock_bh(&nr_list_lock);
- sk_for_each(s, &nr_list)
- if (nr_sk(s)->device == dev)
- nr_disconnect(s, ENETUNREACH);
+ sk_for_each(s, &nr_list) {
+ if (nr_sk(s)->device != dev)
+ continue;
+ bh_lock_sock(s);
+ nr_disconnect(s, ENETUNREACH);
+ bh_unlock_sock(s);
+ }
spin_unlock_bh(&nr_list_lock);
}
@@ -208,29 +214,40 @@ static struct sock *nr_find_peer(unsigned char index, unsigned char id,
}
/*
- * Find next free circuit ID.
+ * Give a socket the next free circuit ID. The search and the assignment
+ * are done under nr_list_lock, which the caller holds, with the socket
+ * already on nr_list: two sockets cannot pick the same ID.
*/
-static unsigned short nr_find_next_circuit(void)
+static void nr_assign_circuit(struct nr_sock *nr)
{
unsigned short id = circuit;
unsigned char i, j;
- struct sock *sk;
+ struct sock *s;
+ bool used;
+
+ lockdep_assert_held(&nr_list_lock);
for (;;) {
i = id / 256;
j = id % 256;
if (i != 0 && j != 0) {
- sk = nr_find_socket(i, j, NULL);
- if (!sk)
+ used = false;
+ sk_for_each(s, &nr_list)
+ if (nr_sk(s)->my_index == i && nr_sk(s)->my_id == j) {
+ used = true;
+ break;
+ }
+ if (!used)
break;
- sock_put(sk);
}
id++;
}
- return id;
+ nr->my_index = i;
+ nr->my_id = j;
+ circuit = id + 1;
}
/*
@@ -459,8 +476,7 @@ static int nr_create(struct net *net, struct socket *sock, int protocol,
msecs_to_jiffies(READ_ONCE(sysctl_netrom_transport_timeout));
nr->t2 =
msecs_to_jiffies(READ_ONCE(sysctl_netrom_transport_acknowledge_delay));
- nr->n2 =
- msecs_to_jiffies(READ_ONCE(sysctl_netrom_transport_maximum_tries));
+ nr->n2 = READ_ONCE(sysctl_netrom_transport_maximum_tries);
nr->t4 =
msecs_to_jiffies(READ_ONCE(sysctl_netrom_transport_busy_delay));
nr->idle =
@@ -710,14 +726,10 @@ static int nr_connect(struct socket *sock, struct sockaddr *uaddr,
nr->dest_addr = addr->sax25_call;
- release_sock(sk);
- circuit = nr_find_next_circuit();
- lock_sock(sk);
-
- nr->my_index = circuit / 256;
- nr->my_id = circuit % 256;
-
- circuit++;
+ /* The socket is on nr_list now, so others see the ID it gets */
+ spin_lock_bh(&nr_list_lock);
+ nr_assign_circuit(nr);
+ spin_unlock_bh(&nr_list_lock);
/* Move to connecting socket, start sending Connect Requests */
sock->state = SS_CONNECTING;
@@ -949,10 +961,16 @@ int nr_rx_frame(struct sk_buff *skb, struct net_device *dev)
bh_lock_sock(sk);
skb_reset_transport_header(skb);
- if (frametype == NR_CONNACK && skb->len == 22)
- nr_sk(sk)->bpqext = 1;
- else
- nr_sk(sk)->bpqext = 0;
+ /*
+ * Only the circuit set-up frames tell whether the peer uses
+ * the BPQ extension: a CONNACK of 22 bytes, or a CONNREQ of
+ * 37 bytes (repeated, for a circuit already accepted). The
+ * other frames say nothing about it.
+ */
+ if (frametype == NR_CONNACK)
+ nr_sk(sk)->bpqext = (skb->len == 22);
+ else if (frametype == NR_CONNREQ)
+ nr_sk(sk)->bpqext = (skb->len == 37);
ret = nr_process_rx_frame(sk, skb);
bh_unlock_sock(sk);
@@ -1014,23 +1032,29 @@ int nr_rx_frame(struct sk_buff *skb, struct net_device *dev)
nr_make->your_index = circuit_index;
nr_make->your_id = circuit_id;
+ /*
+ * Put the new socket on nr_list as it gets its circuit ID, under the
+ * same lock, so that no other socket can be given the same ID.
+ * nr_list_lock is not taken under a socket lock (nr_info_show() takes
+ * them the other way round).
+ */
bh_unlock_sock(sk);
- circuit = nr_find_next_circuit();
+ spin_lock_bh(&nr_list_lock);
+ nr_assign_circuit(nr_make);
+ sk_add_node(make, &nr_list);
+ spin_unlock_bh(&nr_list_lock);
bh_lock_sock(sk);
- nr_make->my_index = circuit / 256;
- nr_make->my_id = circuit % 256;
-
- circuit++;
-
/* Window negotiation */
- if (window < nr_make->window)
- nr_make->window = window;
+ nr_negotiate_window(nr_make, window);
- /* L4 timeout negotiation */
+ /*
+ * L4 timeout negotiation. Ignore a proposal shorter than the
+ * smallest transport timeout the sysctl accepts (5 s), such as 0.
+ */
if (skb->len == 37) {
timeout = skb->data[36] * 256 + skb->data[35];
- if (timeout * HZ < nr_make->t1)
+ if (timeout >= 5 && timeout * HZ < nr_make->t1)
nr_make->t1 = timeout * HZ;
nr_make->bpqext = 1;
} else {
@@ -1054,8 +1078,6 @@ int nr_rx_frame(struct sk_buff *skb, struct net_device *dev)
bh_unlock_sock(sk);
sock_put(sk);
- nr_insert_socket(make);
-
nr_start_heartbeat(make);
nr_start_idletimer(make);
@@ -1163,9 +1185,9 @@ static int nr_sendmsg(struct socket *sock, struct msghdr *msg, size_t len)
goto out;
}
- nr_output(sk, skb); /* Shove it onto the queue */
-
- err = len;
+ err = nr_output(sk, skb); /* Shove it onto the queue */
+ if (!err)
+ err = len;
out:
release_sock(sk);
return err;
@@ -1214,10 +1236,13 @@ static int nr_recvmsg(struct socket *sock, struct msghdr *msg, size_t size,
}
if (sax != NULL) {
+ /*
+ * The NET/ROM headers were pulled when the frame was queued:
+ * give the node at the other end of the circuit.
+ */
memset(sax, 0, sizeof(*sax));
sax->sax25_family = AF_NETROM;
- skb_copy_from_linear_data_offset(skb, 7, sax->sax25_call.ax25_call,
- AX25_ADDR_LEN);
+ sax->sax25_call = nr_sk(sk)->dest_addr;
msg->msg_namelen = sizeof(*sax);
}
diff --git a/net/netrom/nr_in.c b/net/netrom/nr_in.c
index b90a47dcf165..805881e5d6c0 100644
--- a/net/netrom/nr_in.c
+++ b/net/netrom/nr_in.c
@@ -67,26 +67,44 @@ static int nr_queue_rx_frame(struct sock *sk, struct sk_buff *skb, int more)
return 0;
}
- if (!more && nr->fraglen > 0) { /* End of fragment */
- nr->fraglen += skb->len;
- skb_queue_tail(&nr->frag_queue, skb);
-
- if ((skbn = alloc_skb(nr->fraglen, GFP_ATOMIC)) == NULL)
- return 1;
+ if (nr->fraglen > 0) { /* End of fragment */
+ skbn = alloc_skb(nr->fraglen + skb->len, GFP_ATOMIC);
+ if (!skbn)
+ goto busy;
skb_reset_transport_header(skbn);
- while ((skbo = skb_dequeue(&nr->frag_queue)) != NULL) {
+ skb_queue_walk(&nr->frag_queue, skbo)
skb_copy_from_linear_data(skbo,
skb_put(skbn, skbo->len),
skbo->len);
- kfree_skb(skbo);
- }
+ skb_copy_from_linear_data(skb, skb_put(skbn, skb->len),
+ skb->len);
+ }
+
+ if (sock_queue_rcv_skb(sk, skbn) != 0) {
+ if (skbn != skb)
+ kfree_skb(skbn);
+ goto busy;
+ }
+ if (skbn != skb) {
+ /* The whole message is delivered: drop its pieces */
+ skb_queue_purge(&nr->frag_queue);
nr->fraglen = 0;
+ kfree_skb(skb);
}
- return sock_queue_rcv_skb(sk, skbn);
+ return 0;
+
+busy:
+ /*
+ * Not delivered: give the frame back as it came, headers included,
+ * and leave the earlier fragments where they are, so that the
+ * caller can keep it on reseq_queue and deliver it later.
+ */
+ skb_push(skb, NR_NETWORK_LEN + NR_TRANSPORT_LEN);
+ return 1;
}
/*
@@ -111,7 +129,7 @@ static int nr_state1_machine(struct sock *sk, struct sk_buff *skb,
nr->vl = 0;
nr->state = NR_STATE_3;
nr->n2count = 0;
- nr->window = skb->data[20];
+ nr_negotiate_window(nr, skb->data[20]);
sk->sk_state = TCP_ESTABLISHED;
if (!sock_flag(sk, SOCK_DEAD))
sk->sk_state_change(sk);
@@ -176,7 +194,7 @@ static int nr_state3_machine(struct sock *sk, struct sk_buff *skb, int frametype
struct sk_buff *skbn;
unsigned short save_vr;
unsigned short nr, ns;
- int queued = 0;
+ int queued = 0, more;
nr = skb->data[18];
@@ -258,7 +276,9 @@ static int nr_state3_machine(struct sock *sk, struct sk_buff *skb, int frametype
while ((skbn = skb_dequeue(&nrom->reseq_queue)) != NULL) {
ns = skbn->data[17];
if (ns == nrom->vr) {
- if (nr_queue_rx_frame(sk, skbn, frametype & NR_MORE_FLAG) == 0) {
+ /* The flags of this frame, not of the one just received */
+ more = skbn->data[19] & NR_MORE_FLAG;
+ if (nr_queue_rx_frame(sk, skbn, more) == 0) {
nrom->vr = (nrom->vr + 1) % NR_MODULUS;
} else {
nrom->condition |= NR_COND_OWN_RX_BUSY;
diff --git a/net/netrom/nr_out.c b/net/netrom/nr_out.c
index 2b3cbceb0b52..150cb4263d31 100644
--- a/net/netrom/nr_out.c
+++ b/net/netrom/nr_out.c
@@ -28,11 +28,19 @@
/*
* This is where all NET/ROM frames pass, except for IP-over-NET/ROM which
* cannot be fragmented in this manner.
+ *
+ * A message too long for one frame is cut into fragments built on a list
+ * of our own, and only put on the write queue once all of them exist: a
+ * message that cannot be fragmented whole (no memory, or a signal while
+ * waiting for send buffer space) is not sent at all, and the error is
+ * returned. Consumes skb in all cases.
*/
-void nr_output(struct sock *sk, struct sk_buff *skb)
+int nr_output(struct sock *sk, struct sk_buff *skb)
{
+ struct sk_buff_head frags;
struct sk_buff *skbn;
unsigned char transport[NR_TRANSPORT_LEN];
+ unsigned long flags;
int err, frontlen, len;
if (skb->len - NR_TRANSPORT_LEN > NR_MAX_PACKET_SIZE) {
@@ -42,10 +50,13 @@ void nr_output(struct sock *sk, struct sk_buff *skb)
frontlen = skb_headroom(skb);
+ __skb_queue_head_init(&frags);
+
while (skb->len > 0) {
if ((skbn = sock_alloc_send_skb(sk, frontlen + NR_MAX_PACKET_SIZE, 0, &err)) == NULL) {
+ __skb_queue_purge(&frags);
kfree_skb(skb);
- return;
+ return err;
}
skb_reserve(skbn, frontlen);
@@ -63,15 +74,21 @@ void nr_output(struct sock *sk, struct sk_buff *skb)
if (skb->len > 0)
skbn->data[4] |= NR_MORE_FLAG;
- skb_queue_tail(&sk->sk_write_queue, skbn); /* Throw it on the queue */
+ __skb_queue_tail(&frags, skbn);
}
kfree_skb(skb);
+
+ spin_lock_irqsave(&sk->sk_write_queue.lock, flags);
+ skb_queue_splice_tail_init(&frags, &sk->sk_write_queue);
+ spin_unlock_irqrestore(&sk->sk_write_queue.lock, flags);
} else {
skb_queue_tail(&sk->sk_write_queue, skb); /* Throw it on the queue */
}
nr_kick(sk);
+
+ return 0;
}
/*
diff --git a/net/netrom/nr_route.c b/net/netrom/nr_route.c
index 288c4a91ea9b..7fbacc5fee84 100644
--- a/net/netrom/nr_route.c
+++ b/net/netrom/nr_route.c
@@ -344,6 +344,24 @@ static void nr_remove_neigh_locked(struct nr_neigh *nr_neigh)
nr_neigh_put(nr_neigh);
}
+/*
+ * Remove route i of a node: move the routes after it up, and keep which on
+ * the route it selected. If that route is the one removed, which now
+ * selects the next one, as after a link failure. The caller drops the
+ * route's use of its neighbour. Called with nr_route_lock held.
+ */
+static void nr_node_remove_route_locked(struct nr_node *nr_node, int i)
+{
+ lockdep_assert_held(&nr_route_lock);
+
+ if (i < nr_node->which)
+ nr_node->which--;
+
+ nr_node->count--;
+ for (; i < nr_node->count; i++)
+ nr_node->routes[i] = nr_node->routes[i + 1];
+}
+
/*
* "Delete" a node. Strictly speaking remove a route to a node. The node
* is only deleted if no routes are left to it.
@@ -373,22 +391,10 @@ static int nr_del_node(ax25_address *callsign, ax25_address *neighbour, struct n
nr_neigh_route_put_locked(nr_neigh);
nr_neigh_put(nr_neigh);
- nr_node->count--;
+ nr_node_remove_route_locked(nr_node, i);
- if (nr_node->count == 0) {
+ if (nr_node->count == 0)
nr_remove_node_locked(nr_node);
- } else {
- switch (i) {
- case 0:
- nr_node->routes[0] = nr_node->routes[1];
- fallthrough;
- case 1:
- nr_node->routes[1] = nr_node->routes[2];
- fallthrough;
- case 2:
- break;
- }
- }
/* Drop the reference taken by __nr_node_get() */
nr_node_put(nr_node);
ret = 0;
@@ -472,34 +478,24 @@ static int nr_dec_obs(void)
spin_lock_bh(&nr_route_lock);
nr_node_for_each_safe(s, nodet, &nr_node_list) {
- for (i = 0; i < s->count; i++) {
+ /* i only moves on when route i stays: the next one moves up */
+ for (i = 0; i < s->count; ) {
switch (s->routes[i].obs_count) {
case 0: /* A locked entry */
+ i++;
break;
case 1: /* From 1 -> 0 */
nr_neigh = s->routes[i].neighbour;
nr_neigh_route_put_locked(nr_neigh);
-
- s->count--;
-
- switch (i) {
- case 0:
- s->routes[0] = s->routes[1];
- fallthrough;
- case 1:
- s->routes[1] = s->routes[2];
- break;
- case 2:
- break;
- }
+ nr_node_remove_route_locked(s, i);
break;
default:
s->routes[i].obs_count--;
+ i++;
break;
-
}
}
@@ -525,23 +521,14 @@ void nr_rt_device_down(struct net_device *dev)
nr_neigh_for_each_safe(s, nodet, &nr_neigh_list) {
if (s->dev == dev) {
nr_node_for_each_safe(t, node2t, &nr_node_list) {
- for (i = 0; i < t->count; i++) {
+ for (i = 0; i < t->count; ) {
if (t->routes[i].neighbour == s) {
s->count--;
nr_neigh_put(s);
- t->count--;
-
- switch (i) {
- case 0:
- t->routes[0] = t->routes[1];
- fallthrough;
- case 1:
- t->routes[1] = t->routes[2];
- break;
- case 2:
- break;
- }
+ nr_node_remove_route_locked(t, i);
+ } else {
+ i++;
}
}
@@ -774,11 +761,16 @@ int nr_route_frame(struct sk_buff *skb, ax25_cb *ax25)
nr_dest = (ax25_address *)(skb->data + 7);
if (ax25 != NULL) {
- ret = nr_add_node(nr_src, "", &ax25->dest_addr, ax25->digipeat,
- ax25->ax25_dev->dev, 0,
- READ_ONCE(sysctl_netrom_obsolescence_count_initialiser));
- if (ret)
- return ret;
+ /*
+ * A frame that cannot be noted here (it claims to come from
+ * one of our own callsigns, or memory is short) is dropped.
+ * Return 0, not the error: AX.25 takes any other value as
+ * "frame consumed" and would never free it.
+ */
+ if (nr_add_node(nr_src, "", &ax25->dest_addr, ax25->digipeat,
+ ax25->ax25_dev->dev, 0,
+ READ_ONCE(sysctl_netrom_obsolescence_count_initialiser)))
+ return 0;
}
if ((dev = nr_dev_get(nr_dest)) != NULL) { /* Its for me */
@@ -793,10 +785,12 @@ int nr_route_frame(struct sk_buff *skb, ax25_cb *ax25)
if (!READ_ONCE(sysctl_netrom_routing_control) && ax25 != NULL)
return 0;
- /* Its Time-To-Live has expired */
- if (skb->data[14] == 1) {
+ /*
+ * Its Time-To-Live has expired. A frame received with a TTL of 0
+ * has expired too: decremented below, it would leave with 255.
+ */
+ if (skb->data[14] <= 1)
return 0;
- }
nr_node = nr_node_get(nr_dest);
if (nr_node == NULL)
@@ -848,8 +842,14 @@ int nr_route_frame(struct sk_buff *skb, ax25_cb *ax25)
spin_unlock_bh(&nr_route_lock);
nr_node_put(nr_node);
+ /*
+ * ax25_send_frame() only returns NULL before taking the frame:
+ * free our copy then; the caller frees the original.
+ */
if (ret)
kfree_skb(oskb);
+ else
+ kfree_skb(skb);
return ret;
}
diff --git a/net/rds/recv.c b/net/rds/recv.c
index 4b12fa4f1e78..c400cc5ea360 100644
--- a/net/rds/recv.c
+++ b/net/rds/recv.c
@@ -343,7 +343,8 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr,
rds_stats_inc(s_recv_ping);
rds_send_pong(cp, inc->i_hdr.h_sport);
/* if this is a handshake ping, start multipath if necessary */
- if (RDS_HS_PROBE(be16_to_cpu(inc->i_hdr.h_sport),
+ if (conn->c_trans->t_mp_capable &&
+ RDS_HS_PROBE(be16_to_cpu(inc->i_hdr.h_sport),
be16_to_cpu(inc->i_hdr.h_dport))) {
rds_recv_hs_exthdrs(&inc->i_hdr, cp->cp_conn);
rds_start_mprds(cp->cp_conn);
@@ -351,7 +352,8 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr,
goto out;
}
- if (be16_to_cpu(inc->i_hdr.h_dport) == RDS_FLAG_PROBE_PORT &&
+ if (conn->c_trans->t_mp_capable &&
+ be16_to_cpu(inc->i_hdr.h_dport) == RDS_FLAG_PROBE_PORT &&
inc->i_hdr.h_sport == 0) {
rds_recv_hs_exthdrs(&inc->i_hdr, cp->cp_conn);
/* if this is a handshake pong, start multipath if necessary */
diff --git a/net/rose/af_rose.c b/net/rose/af_rose.c
index 4912138bb2f6..fbf0e27ce898 100644
--- a/net/rose/af_rose.c
+++ b/net/rose/af_rose.c
@@ -278,7 +278,7 @@ static void rose_insert_socket(struct sock *sk)
/*
* Find a socket that wants to accept the Call Request we just
- * received.
+ * received. A reference is taken on the socket found.
*/
static struct sock *rose_find_listener(rose_address *addr, ax25_address *call)
{
@@ -304,48 +304,73 @@ static struct sock *rose_find_listener(rose_address *addr, ax25_address *call)
}
s = NULL;
found:
+ if (s)
+ sock_hold(s);
spin_unlock_bh(&rose_list_lock);
return s;
}
/*
* Find a connected ROSE socket given my LCI and device.
+ * The caller holds rose_list_lock.
*/
-struct sock *rose_find_socket(unsigned int lci, struct rose_neigh *neigh)
+static struct sock *__rose_find_socket(unsigned int lci, struct rose_neigh *neigh)
{
struct sock *s;
- spin_lock_bh(&rose_list_lock);
sk_for_each(s, &rose_list) {
struct rose_sock *rose = rose_sk(s);
if (rose->lci == lci && rose->neighbour == neigh)
- goto found;
+ return s;
}
- s = NULL;
-found:
+
+ return NULL;
+}
+
+/*
+ * Find a connected ROSE socket given my LCI and device. A reference is
+ * taken on the socket found: it can be released and destroyed as soon
+ * as rose_list_lock is dropped.
+ */
+struct sock *rose_find_socket(unsigned int lci, struct rose_neigh *neigh)
+{
+ struct sock *s;
+
+ spin_lock_bh(&rose_list_lock);
+ s = __rose_find_socket(lci, neigh);
+ if (s)
+ sock_hold(s);
spin_unlock_bh(&rose_list_lock);
return s;
}
/*
* Find a unique LCI for a given device.
+ * The caller holds rose_route_list_lock.
*/
unsigned int rose_new_lci(struct rose_neigh *neigh)
{
+ unsigned int res = 0;
int lci;
+ spin_lock_bh(&rose_list_lock);
if (neigh->dce_mode) {
for (lci = 1; lci <= sysctl_rose_maximum_vcs; lci++)
- if (rose_find_socket(lci, neigh) == NULL && rose_route_free_lci(lci, neigh) == NULL)
- return lci;
+ if (!__rose_find_socket(lci, neigh) && !rose_route_free_lci(lci, neigh)) {
+ res = lci;
+ break;
+ }
} else {
for (lci = sysctl_rose_maximum_vcs; lci > 0; lci--)
- if (rose_find_socket(lci, neigh) == NULL && rose_route_free_lci(lci, neigh) == NULL)
- return lci;
+ if (!__rose_find_socket(lci, neigh) && !rose_route_free_lci(lci, neigh)) {
+ res = lci;
+ break;
+ }
}
+ spin_unlock_bh(&rose_list_lock);
- return 0;
+ return res;
}
/*
@@ -673,6 +698,7 @@ static struct sock *rose_make_new(struct sock *osk)
static int rose_release(struct socket *sock)
{
struct sock *sk = sock->sk;
+ struct rose_neigh *neigh;
struct rose_sock *rose;
if (sk == NULL) return 0;
@@ -691,10 +717,21 @@ static int rose_release(struct socket *sock)
break;
case ROSE_STATE_2:
- rose_neigh_put(rose->neighbour);
- release_sock(sk);
+ /*
+ * Disconnect before the socket lock is dropped: frames put on
+ * the backlog meanwhile, a Clear Confirmation for one, then
+ * find it in state 0 instead of dropping the neighbour
+ * reference a second time. rose_kill_by_neigh() does not take
+ * the socket lock: take the neighbour under rose_list_lock, as
+ * it does, so that only one of them drops the reference.
+ */
rose_disconnect(sk, 0, -1, -1);
- lock_sock(sk);
+ spin_lock_bh(&rose_list_lock);
+ neigh = rose->neighbour;
+ rose->neighbour = NULL;
+ spin_unlock_bh(&rose_list_lock);
+ if (neigh)
+ rose_neigh_put(neigh);
rose_destroy_socket(sk);
break;
@@ -854,20 +891,6 @@ static int rose_connect(struct socket *sock, struct sockaddr *uaddr, int addr_le
sk->sk_state = TCP_CLOSE;
sock->state = SS_UNCONNECTED;
- rose->neighbour = rose_get_neigh(&addr->srose_addr, &cause,
- &diagnostic, 0);
- if (!rose->neighbour) {
- err = -ENETUNREACH;
- goto out_release;
- }
-
- rose->lci = rose_new_lci(rose->neighbour);
- if (!rose->lci) {
- err = -ENETUNREACH;
- rose_neigh_put(rose->neighbour);
- goto out_release;
- }
-
if (sock_flag(sk, SOCK_ZAPPED)) { /* Must bind first - autobinding in this may or may not work */
struct net_device *dev;
@@ -876,14 +899,12 @@ static int rose_connect(struct socket *sock, struct sockaddr *uaddr, int addr_le
dev = rose_dev_first();
if (!dev) {
err = -ENETUNREACH;
- rose_neigh_put(rose->neighbour);
goto out_release;
}
user = ax25_findbyuid(current_euid());
if (!user) {
err = -EINVAL;
- rose_neigh_put(rose->neighbour);
dev_put(dev);
goto out_release;
}
@@ -897,6 +918,21 @@ static int rose_connect(struct socket *sock, struct sockaddr *uaddr, int addr_le
rose_insert_socket(sk); /* Finish the bind */
}
+
+ rose->neighbour = rose_get_neigh(&addr->srose_addr, &cause,
+ &diagnostic, 0);
+ if (!rose->neighbour) {
+ err = -ENETUNREACH;
+ goto out_release;
+ }
+
+ /* The socket is on the list now, so others see the LCI it gets */
+ if (!rose_assign_lci(sk)) {
+ err = -ENETUNREACH;
+ rose_neigh_put(rose->neighbour);
+ goto out_release;
+ }
+
rose->dest_addr = addr->srose_addr;
rose->dest_call = addr->srose_call;
rose->rand = ((long)rose & 0xFFFF) + rose->lci;
@@ -1103,12 +1139,31 @@ int rose_rx_call_request(struct sk_buff *skb, struct net_device *dev, struct ros
}
sk = rose_find_listener(&facilities.source_addr, &facilities.source_call);
+ if (!sk) {
+ rose_transmit_clear_request(neigh, lci, ROSE_NETWORK_CONGESTION, 120);
+ return 0;
+ }
+
+ /*
+ * The listener may be closed by rose_release() at any time. It is
+ * marked dead before rose_destroy_socket() empties its queue, so
+ * once it is locked a listener that is not dead will see the new
+ * socket on its queue, either in rose_accept() or when it is
+ * destroyed.
+ */
+ bh_lock_sock(sk);
/*
* We can't accept the Call Request.
*/
- if (sk == NULL || sk_acceptq_is_full(sk) ||
- (make = rose_make_new(sk)) == NULL) {
+ if (sock_flag(sk, SOCK_DEAD) || sk->sk_state != TCP_LISTEN ||
+ sk_acceptq_is_full(sk))
+ make = NULL;
+ else
+ make = rose_make_new(sk);
+ if (!make) {
+ bh_unlock_sock(sk);
+ sock_put(sk);
rose_transmit_clear_request(neigh, lci, ROSE_NETWORK_CONGESTION, 120);
return 0;
}
@@ -1164,8 +1219,9 @@ int rose_rx_call_request(struct sk_buff *skb, struct net_device *dev, struct ros
rose_start_heartbeat(make);
- if (!sock_flag(sk, SOCK_DEAD))
- sk->sk_data_ready(sk);
+ sk->sk_data_ready(sk);
+ bh_unlock_sock(sk);
+ sock_put(sk);
return 1;
}
diff --git a/net/rose/rose_in.c b/net/rose/rose_in.c
index 4eec54b24afb..befb4c55e459 100644
--- a/net/rose/rose_in.c
+++ b/net/rose/rose_in.c
@@ -75,17 +75,24 @@ static int rose_state1_machine(struct sock *sk, struct sk_buff *skb, int framety
static int rose_state2_machine(struct sock *sk, struct sk_buff *skb, int frametype)
{
struct rose_sock *rose = rose_sk(sk);
+ /*
+ * A call whose Call Request timed out (T1) is cleared from state 2
+ * while connect() is still waiting for it: report the timeout, or
+ * connect() and SO_ERROR would find no error on a call that never
+ * got through.
+ */
+ int reason = sk->sk_state == TCP_SYN_SENT ? ETIMEDOUT : 0;
switch (frametype) {
case ROSE_CLEAR_REQUEST:
rose_write_internal(sk, ROSE_CLEAR_CONFIRMATION);
- rose_disconnect(sk, 0, skb->data[3], skb->data[4]);
+ rose_disconnect(sk, reason, skb->data[3], skb->data[4]);
rose_neigh_put(rose->neighbour);
rose->neighbour = NULL;
break;
case ROSE_CLEAR_CONFIRMATION:
- rose_disconnect(sk, 0, -1, -1);
+ rose_disconnect(sk, reason, -1, -1);
rose_neigh_put(rose->neighbour);
rose->neighbour = NULL;
break;
diff --git a/net/rose/rose_link.c b/net/rose/rose_link.c
index 1fe48d58726f..912d26f1fe35 100644
--- a/net/rose/rose_link.c
+++ b/net/rose/rose_link.c
@@ -27,29 +27,66 @@
static void rose_ftimer_expiry(struct timer_list *);
static void rose_t0timer_expiry(struct timer_list *);
-static void rose_transmit_restart_confirmation(struct rose_neigh *neigh);
-static void rose_transmit_restart_request(struct rose_neigh *neigh);
+/*
+ * neigh->ax25 is replaced when a frame is sent or the link is looked up,
+ * from process context, softirq and the neighbour timers, and cleared by
+ * rose_link_failed(). Each holder of the pointer owns a reference on the
+ * ax25_cb: swap the pointer under this lock, then drop the reference that
+ * was taken off, so that it is dropped once and none is lost.
+ */
+static DEFINE_SPINLOCK(rose_neigh_ax25_lock);
-void rose_start_ftimer(struct rose_neigh *neigh)
+static void rose_neigh_set_ax25(struct rose_neigh *neigh, ax25_cb *ax25)
{
- timer_delete(&neigh->ftimer);
+ ax25_cb *old;
- neigh->ftimer.function = rose_ftimer_expiry;
- neigh->ftimer.expires =
- jiffies + msecs_to_jiffies(sysctl_rose_link_fail_timeout);
+ spin_lock_bh(&rose_neigh_ax25_lock);
+ old = neigh->ax25;
+ neigh->ax25 = ax25;
+ spin_unlock_bh(&rose_neigh_ax25_lock);
- add_timer(&neigh->ftimer);
+ if (old)
+ ax25_cb_put(old);
}
-static void rose_start_t0timer(struct rose_neigh *neigh)
+/*
+ * Clear neigh->ax25 if it still is @ax25, and drop its reference.
+ */
+bool rose_neigh_clear_ax25(struct rose_neigh *neigh, ax25_cb *ax25)
{
- timer_delete(&neigh->t0timer);
+ bool found;
- neigh->t0timer.function = rose_t0timer_expiry;
- neigh->t0timer.expires =
- jiffies + msecs_to_jiffies(sysctl_rose_restart_request_timeout);
+ spin_lock_bh(&rose_neigh_ax25_lock);
+ found = neigh->ax25 == ax25;
+ if (found)
+ neigh->ax25 = NULL;
+ spin_unlock_bh(&rose_neigh_ax25_lock);
+
+ if (found)
+ ax25_cb_put(ax25);
+
+ return found;
+}
- add_timer(&neigh->t0timer);
+static void rose_transmit_restart_confirmation(struct rose_neigh *neigh);
+static void rose_transmit_restart_request(struct rose_neigh *neigh);
+
+void rose_neigh_init_timers(struct rose_neigh *neigh)
+{
+ timer_setup(&neigh->ftimer, rose_ftimer_expiry, 0);
+ timer_setup(&neigh->t0timer, rose_t0timer_expiry, 0);
+}
+
+void rose_start_ftimer(struct rose_neigh *neigh)
+{
+ mod_timer(&neigh->ftimer,
+ jiffies + msecs_to_jiffies(sysctl_rose_link_fail_timeout));
+}
+
+static void rose_start_t0timer(struct rose_neigh *neigh)
+{
+ mod_timer(&neigh->t0timer,
+ jiffies + msecs_to_jiffies(sysctl_rose_restart_request_timeout));
}
void rose_stop_ftimer(struct rose_neigh *neigh)
@@ -95,19 +132,17 @@ static void rose_t0timer_expiry(struct timer_list *t)
static int rose_send_frame(struct sk_buff *skb, struct rose_neigh *neigh)
{
const ax25_address *rose_call;
- ax25_cb *ax25s;
+ ax25_cb *ax25;
if (ax25cmp(&rose_callsign, &null_ax25_address) == 0)
rose_call = (const ax25_address *)neigh->dev->dev_addr;
else
rose_call = &rose_callsign;
- ax25s = neigh->ax25;
- neigh->ax25 = ax25_send_frame(skb, 260, rose_call, &neigh->callsign, neigh->digipeat, neigh->dev);
- if (ax25s)
- ax25_cb_put(ax25s);
+ ax25 = ax25_send_frame(skb, 260, rose_call, &neigh->callsign, neigh->digipeat, neigh->dev);
+ rose_neigh_set_ax25(neigh, ax25);
- return neigh->ax25 != NULL;
+ return !!ax25;
}
/*
@@ -118,19 +153,17 @@ static int rose_send_frame(struct sk_buff *skb, struct rose_neigh *neigh)
static int rose_link_up(struct rose_neigh *neigh)
{
const ax25_address *rose_call;
- ax25_cb *ax25s;
+ ax25_cb *ax25;
if (ax25cmp(&rose_callsign, &null_ax25_address) == 0)
rose_call = (const ax25_address *)neigh->dev->dev_addr;
else
rose_call = &rose_callsign;
- ax25s = neigh->ax25;
- neigh->ax25 = ax25_find_cb(rose_call, &neigh->callsign, neigh->digipeat, neigh->dev);
- if (ax25s)
- ax25_cb_put(ax25s);
+ ax25 = ax25_find_cb(rose_call, &neigh->callsign, neigh->digipeat, neigh->dev);
+ rose_neigh_set_ax25(neigh, ax25);
- return neigh->ax25 != NULL;
+ return !!ax25;
}
/*
diff --git a/net/rose/rose_loopback.c b/net/rose/rose_loopback.c
index 4e4a473910cc..6c5c35030e67 100644
--- a/net/rose/rose_loopback.c
+++ b/net/rose/rose_loopback.c
@@ -109,6 +109,7 @@ static void rose_loopback_timer(struct timer_list *unused)
if (sk) {
if (rose_rcv_frame(sk, skb) == 0)
kfree_skb(skb);
+ sock_put(sk);
continue;
}
diff --git a/net/rose/rose_route.c b/net/rose/rose_route.c
index 6a1de0055e28..00fbe47fffe0 100644
--- a/net/rose/rose_route.c
+++ b/net/rose/rose_route.c
@@ -102,8 +102,7 @@ static int __must_check rose_add_node(struct rose_route_struct *rose_route,
skb_queue_head_init(&rose_neigh->queue);
- timer_setup(&rose_neigh->ftimer, NULL, 0);
- timer_setup(&rose_neigh->t0timer, NULL, 0);
+ rose_neigh_init_timers(rose_neigh);
if (rose_route->ndigis != 0) {
rose_neigh->digipeat =
@@ -446,8 +445,7 @@ void rose_add_loopback_neigh(void)
skb_queue_head_init(&sn->queue);
- timer_setup(&sn->ftimer, NULL, 0);
- timer_setup(&sn->t0timer, NULL, 0);
+ rose_neigh_init_timers(sn);
spin_lock_bh(&rose_neigh_list_lock);
sn->next = rose_neigh_list;
@@ -726,6 +724,8 @@ struct rose_route *rose_route_free_lci(unsigned int lci, struct rose_neigh *neig
{
struct rose_route *rose_route;
+ lockdep_assert_held(&rose_route_list_lock);
+
for (rose_route = rose_route_list; rose_route != NULL; rose_route = rose_route->next)
if ((rose_route->neigh1 == neigh && rose_route->lci1 == lci) ||
(rose_route->neigh2 == neigh && rose_route->lci2 == lci))
@@ -734,6 +734,23 @@ struct rose_route *rose_route_free_lci(unsigned int lci, struct rose_neigh *neig
return NULL;
}
+/*
+ * Give a socket placing a call a free LCI on its neighbour. As for a
+ * through route in rose_route_frame(), the search and the assignment
+ * are done under rose_route_list_lock, so two calls cannot pick the
+ * same LCI. The socket must already be on the socket list.
+ */
+unsigned int rose_assign_lci(struct sock *sk)
+{
+ struct rose_sock *rose = rose_sk(sk);
+
+ spin_lock_bh(&rose_route_list_lock);
+ rose->lci = rose_new_lci(rose->neighbour);
+ spin_unlock_bh(&rose_route_list_lock);
+
+ return rose->lci;
+}
+
/*
* Find a neighbour or a route given a ROSE address.
*/
@@ -919,14 +936,13 @@ void rose_link_failed(ax25_cb *ax25, int reason)
spin_lock_bh(&rose_neigh_list_lock);
rose_neigh = rose_neigh_list;
while (rose_neigh != NULL) {
- if (rose_neigh->ax25 == ax25)
+ if (READ_ONCE(rose_neigh->ax25) == ax25)
break;
rose_neigh = rose_neigh->next;
}
if (rose_neigh != NULL) {
- rose_neigh->ax25 = NULL;
- ax25_cb_put(ax25);
+ rose_neigh_clear_ax25(rose_neigh, ax25);
rose_del_route_by_neigh(rose_neigh);
rose_kill_by_neigh(rose_neigh);
@@ -1037,25 +1053,34 @@ int rose_route_frame(struct sk_buff *skb, ax25_cb *ax25)
if (frametype == ROSE_CALL_REQUEST) {
struct rose_sock *rose = rose_sk(sk);
- /* Remove an existing unused socket */
- rose_clear_queues(sk);
- rose->cause = ROSE_NETWORK_CONGESTION;
- rose->diagnostic = 0;
- rose_neigh_put(rose->neighbour);
- rose->neighbour = NULL;
- rose->lci = 0;
- rose->state = ROSE_STATE_0;
- sk->sk_state = TCP_CLOSE;
- sk->sk_err = 0;
- sk->sk_shutdown |= SEND_SHUTDOWN;
- if (!sock_flag(sk, SOCK_DEAD)) {
- sk->sk_state_change(sk);
- sock_set_flag(sk, SOCK_DEAD);
+ /*
+ * Remove an existing unused socket, unless it was
+ * disconnected or reaped since it was found.
+ */
+ bh_lock_sock(sk);
+ if (rose->lci == lci && rose->neighbour == rose_neigh) {
+ rose_clear_queues(sk);
+ rose->cause = ROSE_NETWORK_CONGESTION;
+ rose->diagnostic = 0;
+ rose_neigh_put(rose->neighbour);
+ rose->neighbour = NULL;
+ rose->lci = 0;
+ rose->state = ROSE_STATE_0;
+ sk->sk_state = TCP_CLOSE;
+ sk->sk_err = 0;
+ sk->sk_shutdown |= SEND_SHUTDOWN;
+ if (!sock_flag(sk, SOCK_DEAD)) {
+ sk->sk_state_change(sk);
+ sock_set_flag(sk, SOCK_DEAD);
+ }
}
+ bh_unlock_sock(sk);
+ sock_put(sk);
}
else {
skb_reset_transport_header(skb);
res = rose_rcv_frame(sk, skb);
+ sock_put(sk);
goto out;
}
}
diff --git a/net/rose/sysctl_net_rose.c b/net/rose/sysctl_net_rose.c
index d801315b7083..a82735c34871 100644
--- a/net/rose/sysctl_net_rose.c
+++ b/net/rose/sysctl_net_rose.c
@@ -9,13 +9,14 @@
#include <net/ax25.h>
#include <net/rose.h>
-static int min_timer[] = {1 * HZ};
-static int max_timer[] = {300 * HZ};
-static int min_idle[] = {0 * HZ};
-static int max_idle[] = {65535 * HZ};
+/* The timers are given in milliseconds */
+static int min_timer[] = {1 * MSEC_PER_SEC};
+static int max_timer[] = {300 * MSEC_PER_SEC};
+static int min_idle[] = {0 * MSEC_PER_SEC};
+static int max_idle[] = {65535 * MSEC_PER_SEC};
static int min_route[1], max_route[] = {1};
-static int min_ftimer[] = {60 * HZ};
-static int max_ftimer[] = {600 * HZ};
+static int min_ftimer[] = {60 * MSEC_PER_SEC};
+static int max_ftimer[] = {600 * MSEC_PER_SEC};
static int min_maxvcs[] = {1}, max_maxvcs[] = {254};
static int min_window[] = {1}, max_window[] = {7};
diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
index 26085d31be47..e2dc86758888 100644
--- a/net/sched/sch_generic.c
+++ b/net/sched/sch_generic.c
@@ -71,9 +71,34 @@ static void qdisc_maybe_clear_missed(struct Qdisc *q,
#define SKB_XOFF_MAGIC ((struct sk_buff *)1UL)
+/* Re-cap a stale queue_mapping; an ONETXQUEUE qdisc pins q->dev_queue. */
+static u16 qdisc_cap_skb_tx_queue(struct Qdisc *q, struct sk_buff *skb)
+{
+ u16 queue = skb_get_queue_mapping(skb);
+ struct net_device *dev = qdisc_dev(q);
+ u16 capped;
+
+ if (q->flags & TCQ_F_ONETXQUEUE)
+ capped = q->dev_queue - netdev_get_tx_queue(dev, 0);
+ else
+ capped = netdev_cap_txqueue(dev, queue);
+
+ if (unlikely(capped != queue))
+ skb_set_queue_mapping(skb, capped);
+
+ return capped;
+}
+
+static struct netdev_queue *skb_cap_tx_queue(struct Qdisc *q,
+ struct sk_buff *skb)
+{
+ return netdev_get_tx_queue(qdisc_dev(q),
+ qdisc_cap_skb_tx_queue(q, skb));
+}
+
static inline struct sk_buff *__skb_dequeue_bad_txq(struct Qdisc *q)
{
- const struct netdev_queue *txq = q->dev_queue;
+ const struct netdev_queue *txq;
spinlock_t *lock = NULL;
struct sk_buff *skb;
@@ -85,7 +110,8 @@ static inline struct sk_buff *__skb_dequeue_bad_txq(struct Qdisc *q)
skb = skb_peek(&q->skb_bad_txq);
if (skb) {
/* check the reason of requeuing without tx lock first */
- txq = skb_get_tx_queue(txq->dev, skb);
+ /* re-cap: real_num_tx_queues may have shrunk */
+ txq = skb_cap_tx_queue(q, skb);
if (!netif_xmit_frozen_or_stopped(txq)) {
skb = __skb_dequeue(&q->skb_bad_txq);
if (qdisc_is_percpu_stats(q)) {
@@ -199,6 +225,8 @@ static void try_bulk_dequeue_skb(struct Qdisc *q,
if (!nskb)
break;
+ /* Every element of an ONETXQUEUE qdisc is pinned to q->dev_queue. */
+ qdisc_cap_skb_tx_queue(q, nskb);
bytelimit -= nskb->len; /* covers GSO len */
skb->next = nskb;
skb = nskb;
@@ -216,7 +244,6 @@ static void try_bulk_dequeue_skb_slow(struct Qdisc *q,
struct sk_buff *skb,
int *packets)
{
- int mapping = skb_get_queue_mapping(skb);
struct sk_buff *nskb;
int cnt = 0;
@@ -224,7 +251,9 @@ static void try_bulk_dequeue_skb_slow(struct Qdisc *q,
nskb = q->dequeue(q);
if (!nskb)
break;
- if (unlikely(skb_get_queue_mapping(nskb) != mapping)) {
+ /* A tc BPF store may have changed the mapping; cap it. */
+ if (unlikely(qdisc_cap_skb_tx_queue(q, nskb) !=
+ skb_get_queue_mapping(skb))) {
qdisc_enqueue_skb_bad_txq(q, nskb);
break;
}
@@ -269,7 +298,7 @@ static struct sk_buff *dequeue_skb(struct Qdisc *q, bool *validate,
if (xfrm_offload(skb))
*validate = true;
/* check the reason of requeuing without tx lock first */
- txq = skb_get_tx_queue(txq->dev, skb);
+ txq = skb_cap_tx_queue(q, skb);
if (!netif_xmit_frozen_or_stopped(txq)) {
skb = __skb_dequeue(&q->gso_skb);
if (qdisc_is_percpu_stats(q)) {
@@ -305,6 +334,8 @@ static struct sk_buff *dequeue_skb(struct Qdisc *q, bool *validate,
skb = q->dequeue(q);
if (skb) {
bulk:
+ /* cap the head; the bulk helpers cap every appended element */
+ qdisc_cap_skb_tx_queue(q, skb);
if (qdisc_may_bulk(q))
try_bulk_dequeue_skb(q, skb, txq, packets, budget);
else
diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c
index 7d93690cb436..d82b04827c2c 100644
--- a/net/sched/sch_qfq.c
+++ b/net/sched/sch_qfq.c
@@ -517,11 +517,14 @@ static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
sch_tree_lock(sch);
qfq_init_agg(q, new_agg, lmax, weight);
}
+ if (existing && new_agg == cl->agg)
+ goto unlock;
if (existing)
qfq_deact_rm_from_agg(q, cl);
else
qdisc_class_hash_insert(&q->clhash, &cl->common);
qfq_add_to_agg(q, new_agg, cl);
+unlock:
sch_tree_unlock(sch);
qdisc_class_hash_grow(sch, &q->clhash);
diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index 99a711d0479c..f9eeb6296f1d 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -631,6 +631,11 @@ static int taprio_enqueue(struct sk_buff *skb, struct Qdisc *sch,
int queue;
queue = skb_get_queue_mapping(skb);
+ if (unlikely(queue >= qdisc_dev(sch)->real_num_tx_queues)) {
+ /* A tc BPF program may have stored an out-of-range mapping. */
+ queue = netdev_cap_txqueue(qdisc_dev(sch), queue);
+ skb_set_queue_mapping(skb, queue);
+ }
child = q->qdiscs[queue];
if (unlikely(child == &noop_qdisc))
diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c
index 04716666b29b..317ce9ee3254 100644
--- a/net/sctp/sm_make_chunk.c
+++ b/net/sctp/sm_make_chunk.c
@@ -94,7 +94,7 @@ static void sctp_control_release_owner(struct sk_buff *skb)
}
}
-static void sctp_control_set_owner_w(struct sctp_chunk *chunk)
+void sctp_control_set_owner_w(struct sctp_chunk *chunk)
{
struct sctp_association *asoc = chunk->asoc;
struct sk_buff *skb = chunk->skb;
@@ -107,7 +107,8 @@ static void sctp_control_set_owner_w(struct sctp_chunk *chunk)
* For now don't do anything for now.
*/
if (chunk->auth) {
- chunk->shkey = asoc->shkey;
+ if (!chunk->shkey)
+ chunk->shkey = asoc->shkey;
sctp_auth_shkey_hold(chunk->shkey);
}
skb->sk = asoc ? asoc->base.sk : NULL;
@@ -136,6 +137,7 @@ int sctp_init_cause(struct sctp_chunk *chunk, __be16 cause_code,
size_t paylen)
{
struct sctp_errhdr err;
+ int chunklen, padlen;
__u16 len;
/* Cause code constants are now defined in network order. */
@@ -143,7 +145,13 @@ int sctp_init_cause(struct sctp_chunk *chunk, __be16 cause_code,
len = sizeof(err) + paylen;
err.length = htons(len);
- if (skb_tailroom(chunk->skb) < len)
+ /* sctp_addto_chunk() pads the chunk up to a 4-byte boundary before
+ * appending the cause, so account for that padding here as well.
+ */
+ chunklen = ntohs(chunk->chunk_hdr->length);
+ padlen = SCTP_PAD4(chunklen) - chunklen;
+
+ if (skb_tailroom(chunk->skb) < padlen + len)
return -ENOSPC;
chunk->subh.err_hdr = sctp_addto_chunk(chunk, sizeof(err), &err);
diff --git a/net/sctp/socket.c b/net/sctp/socket.c
index 8c9970352952..5cbc364a5357 100644
--- a/net/sctp/socket.c
+++ b/net/sctp/socket.c
@@ -148,25 +148,47 @@ static inline void sctp_set_owner_w(struct sctp_chunk *chunk)
static void sctp_clear_owner_w(struct sctp_chunk *chunk)
{
+ /* Keep the shkey alive until the new owner takes its reference. */
+ if (chunk->shkey)
+ sctp_auth_shkey_hold(chunk->shkey);
skb_orphan(chunk->skb);
}
+static void sctp_set_owner_w_migrate(struct sctp_chunk *chunk)
+{
+ struct sctp_shared_key *shkey = chunk->shkey;
+
+ if (chunk->msg)
+ sctp_set_owner_w(chunk);
+ else
+ sctp_control_set_owner_w(chunk);
+
+ if (shkey)
+ sctp_auth_shkey_release(shkey);
+}
+
+static void sctp_process_tx_chunk(struct sctp_association *asoc,
+ struct sctp_chunk *chunk, bool clear,
+ void (*cb)(struct sctp_chunk *))
+{
+ if ((clear && asoc->base.sk == chunk->skb->sk) ||
+ (!clear && asoc->base.sk != chunk->skb->sk))
+ cb(chunk);
+}
+
#define traverse_and_process() \
do { \
msg = chunk->msg; \
if (msg == prev_msg) \
continue; \
- list_for_each_entry(c, &msg->chunks, frag_list) { \
- if ((clear && asoc->base.sk == c->skb->sk) || \
- (!clear && asoc->base.sk != c->skb->sk)) \
- cb(c); \
- } \
+ list_for_each_entry(c, &msg->chunks, frag_list) \
+ sctp_process_tx_chunk(asoc, c, clear, cb); \
prev_msg = msg; \
} while (0)
-static void sctp_for_each_tx_datachunk(struct sctp_association *asoc,
- bool clear,
- void (*cb)(struct sctp_chunk *))
+static void sctp_for_each_tx_chunk(struct sctp_association *asoc,
+ bool clear,
+ void (*cb)(struct sctp_chunk *))
{
struct sctp_datamsg *msg, *prev_msg = NULL;
@@ -189,6 +211,21 @@ static void sctp_for_each_tx_datachunk(struct sctp_association *asoc,
list_for_each_entry(chunk, &q->out_chunk_list, list)
traverse_and_process();
+
+ list_for_each_entry(chunk, &q->control_chunk_list, list)
+ sctp_process_tx_chunk(asoc, chunk, clear, cb);
+
+ list_for_each_entry(chunk, &asoc->asconf_ack_list, transmitted_list)
+ sctp_process_tx_chunk(asoc, chunk, clear, cb);
+
+ list_for_each_entry(chunk, &asoc->addip_chunk_list, list)
+ sctp_process_tx_chunk(asoc, chunk, clear, cb);
+
+ if (asoc->strreset_chunk)
+ sctp_process_tx_chunk(asoc, asoc->strreset_chunk, clear, cb);
+
+ if (asoc->addip_last_asconf)
+ sctp_process_tx_chunk(asoc, asoc->addip_last_asconf, clear, cb);
}
static void sctp_for_each_rx_skb(struct sctp_association *asoc, struct sock *sk,
@@ -9667,9 +9704,9 @@ static int sctp_sock_migrate(struct sock *oldsk, struct sock *newsk,
* paths won't try to lock it and then oldsk.
*/
lock_sock_nested(newsk, SINGLE_DEPTH_NESTING);
- sctp_for_each_tx_datachunk(assoc, true, sctp_clear_owner_w);
+ sctp_for_each_tx_chunk(assoc, true, sctp_clear_owner_w);
sctp_assoc_migrate(assoc, newsk);
- sctp_for_each_tx_datachunk(assoc, false, sctp_set_owner_w);
+ sctp_for_each_tx_chunk(assoc, false, sctp_set_owner_w_migrate);
/* If the association on the newsk is already closed before accept()
* is called, set RCV_SHUTDOWN flag.
diff --git a/net/smc/smc_rx.c b/net/smc/smc_rx.c
index 24524dfd5518..4e2b156a5245 100644
--- a/net/smc/smc_rx.c
+++ b/net/smc/smc_rx.c
@@ -286,6 +286,27 @@ int smc_rx_wait(struct smc_sock *smc, long *timeo, size_t peeked,
return rc;
}
+/* Wait for the pipe to release the spliced data. Unlike smc_rx_wait()
+ * it does not end on shutdown or sk_err, the data is already in the
+ * RMB; smc_conn_kill() wakes it and the caller checks conn->killed.
+ */
+static void smc_rx_wait_splice(struct smc_sock *smc, long *timeo)
+{
+ DEFINE_WAIT_FUNC(wait, woken_wake_function);
+ struct smc_connection *conn = &smc->conn;
+ struct sock *sk = &smc->sk;
+
+ sk_set_bit(SOCKWQ_ASYNC_WAITDATA, sk);
+ add_wait_queue(sk_sleep(sk), &wait);
+ /* pairs with the smp_mb() in wq_has_sleeper(): the release drops
+ * splice_pending, then looks for a sleeper
+ */
+ smp_mb();
+ sk_wait_event(sk, timeo, !atomic_read(&conn->splice_pending), &wait);
+ remove_wait_queue(sk_sleep(sk), &wait);
+ sk_clear_bit(SOCKWQ_ASYNC_WAITDATA, sk);
+}
+
static int smc_rx_recv_urg(struct smc_sock *smc, struct msghdr *msg, int len,
int flags)
{
@@ -443,7 +464,23 @@ int smc_rx_recvmsg(struct smc_sock *smc, struct msghdr *msg,
/* could be just 1 byte, even after waiting on data above */
readable = smc_rx_data_available(conn, peeked_bytes);
splbytes = atomic_read(&conn->splice_pending);
- if (!readable || (msg && splbytes)) {
+ if (msg && splbytes) {
+ /* a pipe holds bytes ahead of these; the copy path
+ * skips the timeout and signal checks
+ */
+ if (!read_remaining)
+ break;
+ if (!timeo || signal_pending(current)) {
+ if (!read_done)
+ read_done = timeo ?
+ sock_intr_errno(timeo) :
+ -EAGAIN;
+ break;
+ }
+ smc_rx_wait_splice(smc, &timeo);
+ continue;
+ }
+ if (!readable) {
if (splbytes)
func = smc_rx_data_available_and_no_splice_pend;
else
diff --git a/net/sunrpc/rpcb_clnt.c b/net/sunrpc/rpcb_clnt.c
index 2d74c660de66..9b90350820be 100644
--- a/net/sunrpc/rpcb_clnt.c
+++ b/net/sunrpc/rpcb_clnt.c
@@ -400,7 +400,8 @@ static struct rpc_clnt *rpcb_create(struct net *net, const char *nodename,
return rpc_create(&args);
}
-static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, struct rpc_message *msg, bool is_set)
+static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt,
+ struct rpc_message *msg, bool is_set)
{
int flags = RPC_TASK_NOCONNECT;
int error, result = 0;
@@ -410,8 +411,23 @@ static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, stru
msg->rpc_resp = &result;
error = rpc_call_sync(clnt, msg, flags);
- if (error < 0)
- return error;
+ if (error < 0) {
+ switch (error) {
+ /* rpcbind answered; the reply itself carries the error */
+ case -EPROTONOSUPPORT:
+ case -EPFNOSUPPORT:
+ case -EOPNOTSUPP:
+ case -EACCES:
+ /* the call never made it onto the wire */
+ case -ENOMEM:
+ case -EMSGSIZE:
+ /* the caller is going away; this says nothing about rpcbind */
+ case -ERESTARTSYS:
+ return error;
+ }
+ /* anything else, we assume that rpcbind isn't functional */
+ return -EIO;
+ }
if (!result)
return -EACCES;
diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c
index 61c3c3a74328..a3933f86f2d4 100644
--- a/net/sunrpc/svc.c
+++ b/net/sunrpc/svc.c
@@ -1207,10 +1207,40 @@ int svc_generic_rpcbind_set(struct net *net,
error = svc_rpcbind_set_version(net, progp, version,
family, proto, port);
+ /* -EIO means no answer, not a refusal, so vs_rpcb_optnl must keep it. */
+ if (error == -EIO)
+ return error;
+
return (vers->vs_rpcb_optnl) ? 0 : error;
}
EXPORT_SYMBOL_GPL(svc_generic_rpcbind_set);
+/**
+ * svc_rpcb_failure_count - local rpcbind calls for @serv that got no answer
+ * @serv: RPC service to query
+ *
+ * svc_register() adds one for each of its calls that got no answer. A reply
+ * that refuses one entry does not count, because rpcbind answered and the
+ * next entry may still succeed.
+ *
+ * The count is kept per serv rather than per net. The local rpcbind client
+ * is per-net and lockd shares it, but a count that another service can move
+ * says nothing about this serv's own calls.
+ *
+ * This is for callers that cannot see the svc_register() return, because a
+ * transport class sits in between. Such a caller reads the count before it
+ * starts and compares as it goes, so there is no state to reset between
+ * operations. The count never resets, and callers must not attach meaning
+ * to the value itself.
+ *
+ * Return: the number of unanswered calls since this serv was created.
+ */
+unsigned int svc_rpcb_failure_count(struct svc_serv *serv)
+{
+ return atomic_read(&serv->sv_rpcb_failures);
+}
+EXPORT_SYMBOL_GPL(svc_rpcb_failure_count);
+
/**
* svc_register - register an RPC service with the local portmapper
* @serv: svc_serv struct for the service to register
@@ -1221,10 +1251,11 @@ EXPORT_SYMBOL_GPL(svc_generic_rpcbind_set);
*
* Service is registered for any address in the passed-in protocol family
*/
-int svc_register(const struct svc_serv *serv, struct net *net,
+int svc_register(struct svc_serv *serv, struct net *net,
const int family, const unsigned short proto,
const unsigned short port)
{
+ bool noanswer = false;
unsigned int p, i;
int error = 0;
@@ -1236,10 +1267,16 @@ int svc_register(const struct svc_serv *serv, struct net *net,
struct svc_program *progp = &serv->sv_programs[p];
for (i = 0; i < progp->pg_nvers; i++) {
+ const struct svc_version *vers = progp->pg_vers[i];
int ret;
ret = progp->pg_rpcbind_set(net, progp, i,
family, proto, port);
+ if (ret == -EIO) {
+ noanswer = true;
+ if (vers && vers->vs_rpcb_optnl)
+ ret = 0;
+ }
if (ret < 0) {
printk(KERN_WARNING "svc: failed to register "
"%sv%u RPC service (errno %d).\n",
@@ -1251,6 +1288,9 @@ int svc_register(const struct svc_serv *serv, struct net *net,
}
}
+ if (noanswer)
+ atomic_inc(&serv->sv_rpcb_failures);
+
return error;
}
diff --git a/net/wireless/reg.c b/net/wireless/reg.c
index 15e56dfad00f..3f685b1396ff 100644
--- a/net/wireless/reg.c
+++ b/net/wireless/reg.c
@@ -691,6 +691,9 @@ static bool valid_rule(const u8 *data, unsigned int size, u16 rule_ptr)
/* mandatory fields */
if (rule->len < offsetofend(struct fwdb_rule, max_bw))
return false;
+ /* the rule must lie entirely within the database */
+ if ((u8 *)rule + rule->len > data + size)
+ return false;
if (rule->len >= offsetofend(struct fwdb_rule, wmm_ptr)) {
u32 wmm_ptr = be16_to_cpu(rule->wmm_ptr) << 2;
struct fwdb_wmm_rule *wmm;
diff --git a/net/xfrm/espintcp.c b/net/xfrm/espintcp.c
index 26c0a46491a4..e36a637b4ce7 100644
--- a/net/xfrm/espintcp.c
+++ b/net/xfrm/espintcp.c
@@ -437,7 +437,9 @@ static void espintcp_destruct(struct sock *sk)
bool tcp_is_ulp_esp(struct sock *sk)
{
- return sk->sk_prot == &espintcp_prot || sk->sk_prot == &espintcp6_prot;
+ const struct proto *prot = READ_ONCE(sk->sk_prot);
+
+ return prot == &espintcp_prot || prot == &espintcp6_prot;
}
EXPORT_SYMBOL_GPL(tcp_is_ulp_esp);
@@ -469,34 +471,41 @@ static int espintcp_init_sk(struct sock *sk)
__sk_dst_reset(sk);
- strp_check_rcv(&ctx->strp);
skb_queue_head_init(&ctx->ike_queue);
skb_queue_head_init(&ctx->out_queue);
+ INIT_WORK(&ctx->work, espintcp_tx_work);
+
+ ctx->saved_data_ready = sk->sk_data_ready;
+ ctx->saved_write_space = sk->sk_write_space;
+ ctx->saved_destruct = sk->sk_destruct;
+
+ /* avoid using task_frag */
+ sk->sk_allocation = GFP_ATOMIC;
+ sk->sk_use_task_frag = false;
+
+ rcu_assign_pointer(icsk->icsk_ulp_data, ctx);
+
+ /* pairs with smp_rmb() in espintcp_getctx() */
+ smp_wmb();
+
+ WRITE_ONCE(sk->sk_data_ready, espintcp_data_ready);
+ WRITE_ONCE(sk->sk_write_space, espintcp_write_space);
+ sk->sk_destruct = espintcp_destruct;
if (sk->sk_family == AF_INET) {
- sk->sk_prot = &espintcp_prot;
- sk->sk_socket->ops = &espintcp_ops;
+ WRITE_ONCE(sk->sk_prot, &espintcp_prot);
+ WRITE_ONCE(sk->sk_socket->ops, &espintcp_ops);
} else {
mutex_lock(&tcpv6_prot_mutex);
if (!espintcp6_prot.recvmsg)
build_protos(&espintcp6_prot, &espintcp6_ops, sk->sk_prot, sk->sk_socket->ops);
mutex_unlock(&tcpv6_prot_mutex);
- sk->sk_prot = &espintcp6_prot;
- sk->sk_socket->ops = &espintcp6_ops;
+ WRITE_ONCE(sk->sk_prot, &espintcp6_prot);
+ WRITE_ONCE(sk->sk_socket->ops, &espintcp6_ops);
}
- ctx->saved_data_ready = sk->sk_data_ready;
- ctx->saved_write_space = sk->sk_write_space;
- ctx->saved_destruct = sk->sk_destruct;
- sk->sk_data_ready = espintcp_data_ready;
- sk->sk_write_space = espintcp_write_space;
- sk->sk_destruct = espintcp_destruct;
- rcu_assign_pointer(icsk->icsk_ulp_data, ctx);
- INIT_WORK(&ctx->work, espintcp_tx_work);
- /* avoid using task_frag */
- sk->sk_allocation = GFP_ATOMIC;
- sk->sk_use_task_frag = false;
+ strp_check_rcv(&ctx->strp);
return 0;
diff --git a/net/xfrm/xfrm_interface_core.c b/net/xfrm/xfrm_interface_core.c
index 688306bf62c5..9835fbe63ccb 100644
--- a/net/xfrm/xfrm_interface_core.c
+++ b/net/xfrm/xfrm_interface_core.c
@@ -531,8 +531,13 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev)
memset(&fl, 0, sizeof(fl));
+ if (!pskb_inet_may_pull(skb))
+ goto tx_err;
+
switch (skb->protocol) {
case htons(ETH_P_IPV6):
+ if (ipv6_hdr(skb)->version != 6)
+ goto tx_err;
memset(IP6CB(skb), 0, sizeof(*IP6CB(skb)));
xfrm_decode_session(dev_net(dev), skb, &fl, AF_INET6);
if (!dst) {
@@ -548,6 +553,9 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev)
}
break;
case htons(ETH_P_IP):
+ if (ip_hdr(skb)->version != 4 || ip_hdr(skb)->ihl < 5 ||
+ !pskb_network_may_pull(skb, ip_hdr(skb)->ihl * 4))
+ goto tx_err;
memset(IPCB(skb), 0, sizeof(*IPCB(skb)));
xfrm_decode_session(dev_net(dev), skb, &fl, AF_INET);
if (!dst) {
diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
index 25f9cbd3ce6a..c13382bcc658 100644
--- a/net/xfrm/xfrm_policy.c
+++ b/net/xfrm/xfrm_policy.c
@@ -2157,6 +2157,7 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type,
struct xfrm_pol_inexact_bin *bin;
struct xfrm_policy *pol, *ret;
struct hlist_head *chain;
+ unsigned int inexact_sequence;
unsigned int sequence;
int err;
@@ -2191,12 +2192,18 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type,
goto skip_inexact;
bin = xfrm_policy_inexact_lookup_rcu(net, type, family, dir, if_id);
- if (!bin || !xfrm_policy_find_inexact_candidates(&cand, bin, saddr,
- daddr))
+ if (!bin)
+ goto skip_inexact;
+
+ inexact_sequence = read_seqcount_begin(&bin->count);
+ if (!xfrm_policy_find_inexact_candidates(&cand, bin, saddr, daddr))
goto skip_inexact;
pol = xfrm_policy_eval_candidates(&cand, ret, fl, type,
family, if_id);
+ if (read_seqcount_retry(&bin->count, inexact_sequence))
+ goto retry;
+
if (pol) {
ret = pol;
if (IS_ERR(pol))
diff --git a/rust/kernel/bug.rs b/rust/kernel/bug.rs
index 53dbdaec3560..95bfd195c986 100644
--- a/rust/kernel/bug.rs
+++ b/rust/kernel/bug.rs
@@ -79,13 +79,19 @@ macro_rules! warn_flags {
#[cfg(all(CONFIG_BUG, CONFIG_UML))]
macro_rules! warn_flags {
($flags:expr) => {
- // SAFETY: It is always safe to call `warn_slowpath_fmt()`
- // with a valid null-terminated string.
+ if false {
+ _ = $flags;
+ }
+
+ // SAFETY:
+ // - The file name is a null-terminated string with `'static` lifetime.
+ // - `TAINT_WARN` is a valid taint number.
+ // - `fmt` can be NULL, and NULL is passed.
unsafe {
$crate::bindings::warn_slowpath_fmt(
$crate::c_str!(::core::file!()).as_char_ptr(),
line!() as $crate::ffi::c_int,
- $flags as $crate::ffi::c_uint,
+ $crate::bindings::TAINT_WARN,
::core::ptr::null(),
);
}
diff --git a/rust/kernel/cpufreq.rs b/rust/kernel/cpufreq.rs
index be312f523ebe..dbd1ea1e9be7 100644
--- a/rust/kernel/cpufreq.rs
+++ b/rust/kernel/cpufreq.rs
@@ -695,8 +695,8 @@ fn clear_data<T: ForeignOwnable>(&mut self) -> Option<T> {
impl<'a> PolicyCpu<'a> {
fn from_cpu(cpu: CpuId) -> Result<Self> {
// SAFETY: It is safe to call `cpufreq_cpu_get` for any valid CPU.
- let ptr =
- NonNull::new(unsafe { bindings::cpufreq_cpu_get(u32::from(cpu)) }).ok_or(ENODEV)?;
+ let ptr = unsafe { bindings::cpufreq_cpu_get(u32::from(cpu)) };
+ let ptr = NonNull::new(ptr).ok_or(ENODEV)?;
Ok(Self(
// SAFETY: `ptr` is non-NULL and `cpufreq_cpu_get()` took a reference on it, so it is
diff --git a/sound/core/compress_offload.c b/sound/core/compress_offload.c
index e1b2da84f778..b965eabf9684 100644
--- a/sound/core/compress_offload.c
+++ b/sound/core/compress_offload.c
@@ -1399,17 +1399,17 @@ static long snd_compr_ioctl_compat(struct file *file, unsigned int cmd,
#endif
static const struct file_operations snd_compr_file_ops = {
- .owner = THIS_MODULE,
- .open = snd_compr_open,
- .release = snd_compr_free,
- .write = snd_compr_write,
- .read = snd_compr_read,
- .unlocked_ioctl = snd_compr_ioctl,
+ .owner = THIS_MODULE,
+ .open = snd_compr_open,
+ .release = snd_compr_free,
+ .write = snd_compr_write,
+ .read = snd_compr_read,
+ .unlocked_ioctl = snd_compr_ioctl,
#ifdef CONFIG_COMPAT
- .compat_ioctl = snd_compr_ioctl_compat,
+ .compat_ioctl = snd_compr_ioctl_compat,
#endif
- .mmap = snd_compr_mmap,
- .poll = snd_compr_poll,
+ .mmap = snd_compr_mmap,
+ .poll = snd_compr_poll,
};
static int snd_compress_dev_register(struct snd_device *device)
diff --git a/sound/core/control.c b/sound/core/control.c
index 1d13352afed2..750bb3f52d26 100644
--- a/sound/core/control.c
+++ b/sound/core/control.c
@@ -2269,16 +2269,15 @@ EXPORT_SYMBOL_GPL(snd_ctl_disconnect_layer);
* INIT PART
*/
-static const struct file_operations snd_ctl_f_ops =
-{
- .owner = THIS_MODULE,
- .read = snd_ctl_read,
- .open = snd_ctl_open,
- .release = snd_ctl_release,
- .poll = snd_ctl_poll,
- .unlocked_ioctl = snd_ctl_ioctl,
- .compat_ioctl = snd_ctl_ioctl_compat,
- .fasync = snd_ctl_fasync,
+static const struct file_operations snd_ctl_f_ops = {
+ .owner = THIS_MODULE,
+ .read = snd_ctl_read,
+ .open = snd_ctl_open,
+ .release = snd_ctl_release,
+ .poll = snd_ctl_poll,
+ .unlocked_ioctl = snd_ctl_ioctl,
+ .compat_ioctl = snd_ctl_ioctl_compat,
+ .fasync = snd_ctl_fasync,
};
/* call lops under rwsems; called from snd_ctl_dev_*() below() */
diff --git a/sound/core/hwdep.c b/sound/core/hwdep.c
index 09200df2932c..4334162bc0b2 100644
--- a/sound/core/hwdep.c
+++ b/sound/core/hwdep.c
@@ -212,6 +212,7 @@ static int snd_hwdep_dsp_load(struct snd_hwdep *hw,
return -ENXIO;
if (info->index >= 32)
return -EINVAL;
+ guard(mutex)(&hw->dsp_mutex);
/* check whether the dsp was already loaded */
if (hw->dsp_loaded & (1u << info->index))
return -EBUSY;
@@ -323,18 +324,17 @@ static int snd_hwdep_control_ioctl(struct snd_card *card,
*/
-static const struct file_operations snd_hwdep_f_ops =
-{
- .owner = THIS_MODULE,
- .llseek = snd_hwdep_llseek,
- .read = snd_hwdep_read,
- .write = snd_hwdep_write,
- .open = snd_hwdep_open,
- .release = snd_hwdep_release,
- .poll = snd_hwdep_poll,
- .unlocked_ioctl = snd_hwdep_ioctl,
- .compat_ioctl = snd_hwdep_ioctl_compat,
- .mmap = snd_hwdep_mmap,
+static const struct file_operations snd_hwdep_f_ops = {
+ .owner = THIS_MODULE,
+ .llseek = snd_hwdep_llseek,
+ .read = snd_hwdep_read,
+ .write = snd_hwdep_write,
+ .open = snd_hwdep_open,
+ .release = snd_hwdep_release,
+ .poll = snd_hwdep_poll,
+ .unlocked_ioctl = snd_hwdep_ioctl,
+ .compat_ioctl = snd_hwdep_ioctl_compat,
+ .mmap = snd_hwdep_mmap,
};
static void snd_hwdep_free(struct snd_hwdep *hwdep)
@@ -381,6 +381,7 @@ int snd_hwdep_new(struct snd_card *card, char *id, int device,
init_waitqueue_head(&hwdep->open_wait);
mutex_init(&hwdep->open_mutex);
+ mutex_init(&hwdep->dsp_mutex);
hwdep->card = card;
hwdep->device = device;
if (id)
diff --git a/sound/core/init.c b/sound/core/init.c
index 2bf98de846f6..779729738b12 100644
--- a/sound/core/init.c
+++ b/sound/core/init.c
@@ -461,20 +461,19 @@ static int snd_disconnect_fasync(int fd, struct file *file, int on)
return -ENODEV;
}
-static const struct file_operations snd_shutdown_f_ops =
-{
- .owner = THIS_MODULE,
- .llseek = snd_disconnect_llseek,
- .read = snd_disconnect_read,
- .write = snd_disconnect_write,
- .release = snd_disconnect_release,
- .poll = snd_disconnect_poll,
- .unlocked_ioctl = snd_disconnect_ioctl,
+static const struct file_operations snd_shutdown_f_ops = {
+ .owner = THIS_MODULE,
+ .llseek = snd_disconnect_llseek,
+ .read = snd_disconnect_read,
+ .write = snd_disconnect_write,
+ .release = snd_disconnect_release,
+ .poll = snd_disconnect_poll,
+ .unlocked_ioctl = snd_disconnect_ioctl,
#ifdef CONFIG_COMPAT
- .compat_ioctl = snd_disconnect_ioctl,
+ .compat_ioctl = snd_disconnect_ioctl,
#endif
- .mmap = snd_disconnect_mmap,
- .fasync = snd_disconnect_fasync
+ .mmap = snd_disconnect_mmap,
+ .fasync = snd_disconnect_fasync
};
/**
diff --git a/sound/core/jack.c b/sound/core/jack.c
index 93e357a23f17..71607d904e06 100644
--- a/sound/core/jack.c
+++ b/sound/core/jack.c
@@ -306,41 +306,41 @@ static ssize_t jack_type_read(struct file *file,
}
static const struct file_operations jack_type_fops = {
- .open = simple_open,
- .read = jack_type_read,
- .llseek = default_llseek,
+ .open = simple_open,
+ .read = jack_type_read,
+ .llseek = default_llseek,
};
#endif
static const struct file_operations sw_inject_enable_fops = {
- .open = simple_open,
- .read = sw_inject_enable_read,
- .write = sw_inject_enable_write,
- .llseek = default_llseek,
+ .open = simple_open,
+ .read = sw_inject_enable_read,
+ .write = sw_inject_enable_write,
+ .llseek = default_llseek,
};
static const struct file_operations jackin_inject_fops = {
- .open = simple_open,
- .write = jackin_inject_write,
- .llseek = default_llseek,
+ .open = simple_open,
+ .write = jackin_inject_write,
+ .llseek = default_llseek,
};
static const struct file_operations jack_kctl_id_fops = {
- .open = simple_open,
- .read = jack_kctl_id_read,
- .llseek = default_llseek,
+ .open = simple_open,
+ .read = jack_kctl_id_read,
+ .llseek = default_llseek,
};
static const struct file_operations jack_kctl_mask_bits_fops = {
- .open = simple_open,
- .read = jack_kctl_mask_bits_read,
- .llseek = default_llseek,
+ .open = simple_open,
+ .read = jack_kctl_mask_bits_read,
+ .llseek = default_llseek,
};
static const struct file_operations jack_kctl_status_fops = {
- .open = simple_open,
- .read = jack_kctl_status_read,
- .llseek = default_llseek,
+ .open = simple_open,
+ .read = jack_kctl_status_read,
+ .llseek = default_llseek,
};
static int snd_jack_debugfs_add_inject_node(struct snd_jack *jack,
diff --git a/sound/core/oss/mixer_oss.c b/sound/core/oss/mixer_oss.c
index aa98caaaea3c..a7d66ea7c795 100644
--- a/sound/core/oss/mixer_oss.c
+++ b/sound/core/oss/mixer_oss.c
@@ -407,13 +407,12 @@ static long snd_mixer_oss_ioctl_compat(struct file *file, unsigned int cmd,
* REGISTRATION PART
*/
-static const struct file_operations snd_mixer_oss_f_ops =
-{
- .owner = THIS_MODULE,
- .open = snd_mixer_oss_open,
- .release = snd_mixer_oss_release,
- .unlocked_ioctl = snd_mixer_oss_ioctl,
- .compat_ioctl = snd_mixer_oss_ioctl_compat,
+static const struct file_operations snd_mixer_oss_f_ops = {
+ .owner = THIS_MODULE,
+ .open = snd_mixer_oss_open,
+ .release = snd_mixer_oss_release,
+ .unlocked_ioctl = snd_mixer_oss_ioctl,
+ .compat_ioctl = snd_mixer_oss_ioctl_compat,
};
/*
diff --git a/sound/core/oss/pcm_oss.c b/sound/core/oss/pcm_oss.c
index 59d5153d1113..6d49451b730c 100644
--- a/sound/core/oss/pcm_oss.c
+++ b/sound/core/oss/pcm_oss.c
@@ -3120,17 +3120,16 @@ static inline void snd_pcm_oss_proc_done(struct snd_pcm *pcm)
* ENTRY functions
*/
-static const struct file_operations snd_pcm_oss_f_reg =
-{
- .owner = THIS_MODULE,
- .read = snd_pcm_oss_read,
- .write = snd_pcm_oss_write,
- .open = snd_pcm_oss_open,
- .release = snd_pcm_oss_release,
- .poll = snd_pcm_oss_poll,
- .unlocked_ioctl = snd_pcm_oss_ioctl,
- .compat_ioctl = snd_pcm_oss_ioctl_compat,
- .mmap = snd_pcm_oss_mmap,
+static const struct file_operations snd_pcm_oss_f_reg = {
+ .owner = THIS_MODULE,
+ .read = snd_pcm_oss_read,
+ .write = snd_pcm_oss_write,
+ .open = snd_pcm_oss_open,
+ .release = snd_pcm_oss_release,
+ .poll = snd_pcm_oss_poll,
+ .unlocked_ioctl = snd_pcm_oss_ioctl,
+ .compat_ioctl = snd_pcm_oss_ioctl_compat,
+ .mmap = snd_pcm_oss_mmap,
};
static void register_oss_dsp(struct snd_pcm *pcm, int index)
diff --git a/sound/core/pcm_native.c b/sound/core/pcm_native.c
index 304b7e4965e3..553fad5300e1 100644
--- a/sound/core/pcm_native.c
+++ b/sound/core/pcm_native.c
@@ -4228,29 +4228,29 @@ static unsigned long snd_pcm_get_unmapped_area(struct file *file,
const struct file_operations snd_pcm_f_ops[2] = {
{
- .owner = THIS_MODULE,
- .write = snd_pcm_write,
- .write_iter = snd_pcm_writev,
- .open = snd_pcm_playback_open,
- .release = snd_pcm_release,
- .poll = snd_pcm_poll,
- .unlocked_ioctl = snd_pcm_ioctl,
- .compat_ioctl = snd_pcm_ioctl_compat,
- .mmap = snd_pcm_mmap,
- .fasync = snd_pcm_fasync,
- .get_unmapped_area = snd_pcm_get_unmapped_area,
+ .owner = THIS_MODULE,
+ .write = snd_pcm_write,
+ .write_iter = snd_pcm_writev,
+ .open = snd_pcm_playback_open,
+ .release = snd_pcm_release,
+ .poll = snd_pcm_poll,
+ .unlocked_ioctl = snd_pcm_ioctl,
+ .compat_ioctl = snd_pcm_ioctl_compat,
+ .mmap = snd_pcm_mmap,
+ .fasync = snd_pcm_fasync,
+ .get_unmapped_area = snd_pcm_get_unmapped_area,
},
{
- .owner = THIS_MODULE,
- .read = snd_pcm_read,
- .read_iter = snd_pcm_readv,
- .open = snd_pcm_capture_open,
- .release = snd_pcm_release,
- .poll = snd_pcm_poll,
- .unlocked_ioctl = snd_pcm_ioctl,
- .compat_ioctl = snd_pcm_ioctl_compat,
- .mmap = snd_pcm_mmap,
- .fasync = snd_pcm_fasync,
- .get_unmapped_area = snd_pcm_get_unmapped_area,
+ .owner = THIS_MODULE,
+ .read = snd_pcm_read,
+ .read_iter = snd_pcm_readv,
+ .open = snd_pcm_capture_open,
+ .release = snd_pcm_release,
+ .poll = snd_pcm_poll,
+ .unlocked_ioctl = snd_pcm_ioctl,
+ .compat_ioctl = snd_pcm_ioctl_compat,
+ .mmap = snd_pcm_mmap,
+ .fasync = snd_pcm_fasync,
+ .get_unmapped_area = snd_pcm_get_unmapped_area,
}
};
diff --git a/sound/core/rawmidi.c b/sound/core/rawmidi.c
index 7508d99c1e44..63c8978c9875 100644
--- a/sound/core/rawmidi.c
+++ b/sound/core/rawmidi.c
@@ -1784,14 +1784,14 @@ static void snd_rawmidi_proc_info_read(struct snd_info_entry *entry,
*/
static const struct file_operations snd_rawmidi_f_ops = {
- .owner = THIS_MODULE,
- .read = snd_rawmidi_read,
- .write = snd_rawmidi_write,
- .open = snd_rawmidi_open,
- .release = snd_rawmidi_release,
- .poll = snd_rawmidi_poll,
- .unlocked_ioctl = snd_rawmidi_ioctl,
- .compat_ioctl = snd_rawmidi_ioctl_compat,
+ .owner = THIS_MODULE,
+ .read = snd_rawmidi_read,
+ .write = snd_rawmidi_write,
+ .open = snd_rawmidi_open,
+ .release = snd_rawmidi_release,
+ .poll = snd_rawmidi_poll,
+ .unlocked_ioctl = snd_rawmidi_ioctl,
+ .compat_ioctl = snd_rawmidi_ioctl_compat,
};
static int snd_rawmidi_alloc_substreams(struct snd_rawmidi *rmidi,
diff --git a/sound/core/seq/oss/seq_oss.c b/sound/core/seq/oss/seq_oss.c
index 02d30d8b6c3a..4ecb43b6ad3b 100644
--- a/sound/core/seq/oss/seq_oss.c
+++ b/sound/core/seq/oss/seq_oss.c
@@ -206,17 +206,16 @@ odev_poll(struct file *file, poll_table * wait)
* registration of sequencer minor device
*/
-static const struct file_operations seq_oss_f_ops =
-{
- .owner = THIS_MODULE,
- .read = odev_read,
- .write = odev_write,
- .open = odev_open,
- .release = odev_release,
- .poll = odev_poll,
- .unlocked_ioctl = odev_ioctl,
- .compat_ioctl = odev_ioctl_compat,
- .llseek = noop_llseek,
+static const struct file_operations seq_oss_f_ops = {
+ .owner = THIS_MODULE,
+ .read = odev_read,
+ .write = odev_write,
+ .open = odev_open,
+ .release = odev_release,
+ .poll = odev_poll,
+ .unlocked_ioctl = odev_ioctl,
+ .compat_ioctl = odev_ioctl_compat,
+ .llseek = noop_llseek,
};
static int __init
diff --git a/sound/core/seq/seq_clientmgr.c b/sound/core/seq/seq_clientmgr.c
index e73c1becb6e1..9a92a3760258 100644
--- a/sound/core/seq/seq_clientmgr.c
+++ b/sound/core/seq/seq_clientmgr.c
@@ -2661,16 +2661,15 @@ void snd_seq_info_clients_read(struct snd_info_entry *entry,
* REGISTRATION PART
*/
-static const struct file_operations snd_seq_f_ops =
-{
- .owner = THIS_MODULE,
- .read = snd_seq_read,
- .write = snd_seq_write,
- .open = snd_seq_open,
- .release = snd_seq_release,
- .poll = snd_seq_poll,
- .unlocked_ioctl = snd_seq_ioctl,
- .compat_ioctl = snd_seq_ioctl_compat,
+static const struct file_operations snd_seq_f_ops = {
+ .owner = THIS_MODULE,
+ .read = snd_seq_read,
+ .write = snd_seq_write,
+ .open = snd_seq_open,
+ .release = snd_seq_release,
+ .poll = snd_seq_poll,
+ .unlocked_ioctl = snd_seq_ioctl,
+ .compat_ioctl = snd_seq_ioctl_compat,
};
static struct device *seq_dev;
diff --git a/sound/core/sound.c b/sound/core/sound.c
index 7980b60f4ba0..5ce042db4755 100644
--- a/sound/core/sound.c
+++ b/sound/core/sound.c
@@ -167,11 +167,10 @@ static int snd_open(struct inode *inode, struct file *file)
return err;
}
-static const struct file_operations snd_fops =
-{
- .owner = THIS_MODULE,
- .open = snd_open,
- .llseek = noop_llseek,
+static const struct file_operations snd_fops = {
+ .owner = THIS_MODULE,
+ .open = snd_open,
+ .llseek = noop_llseek,
};
#ifdef CONFIG_SND_DYNAMIC_MINORS
diff --git a/sound/core/timer.c b/sound/core/timer.c
index c5f0e895ea02..64bffc21295d 100644
--- a/sound/core/timer.c
+++ b/sound/core/timer.c
@@ -2101,9 +2101,10 @@ static long snd_utimer_ioctl(struct file *file, unsigned int ioctl, unsigned lon
}
static const struct file_operations snd_utimer_fops = {
- .llseek = noop_llseek,
- .release = snd_utimer_release,
- .unlocked_ioctl = snd_utimer_ioctl,
+ .llseek = noop_llseek,
+ .release = snd_utimer_release,
+ .unlocked_ioctl = snd_utimer_ioctl,
+ .compat_ioctl = snd_utimer_ioctl,
};
static int snd_utimer_start(struct snd_timer *t)
@@ -2455,16 +2456,15 @@ static __poll_t snd_timer_user_poll(struct file *file, poll_table * wait)
#define snd_timer_user_ioctl_compat NULL
#endif
-static const struct file_operations snd_timer_f_ops =
-{
- .owner = THIS_MODULE,
- .read = snd_timer_user_read,
- .open = snd_timer_user_open,
- .release = snd_timer_user_release,
- .poll = snd_timer_user_poll,
- .unlocked_ioctl = snd_timer_user_ioctl,
- .compat_ioctl = snd_timer_user_ioctl_compat,
- .fasync = snd_timer_user_fasync,
+static const struct file_operations snd_timer_f_ops = {
+ .owner = THIS_MODULE,
+ .read = snd_timer_user_read,
+ .open = snd_timer_user_open,
+ .release = snd_timer_user_release,
+ .poll = snd_timer_user_poll,
+ .unlocked_ioctl = snd_timer_user_ioctl,
+ .compat_ioctl = snd_timer_user_ioctl_compat,
+ .fasync = snd_timer_user_fasync,
};
/* unregister the system timer */
diff --git a/sound/core/timer_compat.c b/sound/core/timer_compat.c
index 4ae9eaeb5afb..7b648edcb9cb 100644
--- a/sound/core/timer_compat.c
+++ b/sound/core/timer_compat.c
@@ -98,6 +98,7 @@ static long __snd_timer_user_ioctl_compat(struct file *file, unsigned int cmd,
case SNDRV_TIMER_IOCTL_PAUSE:
case SNDRV_TIMER_IOCTL_PAUSE_OLD:
case SNDRV_TIMER_IOCTL_NEXT_DEVICE:
+ case SNDRV_TIMER_IOCTL_CREATE:
return __snd_timer_user_ioctl(file, cmd, (unsigned long)argp, true);
case SNDRV_TIMER_IOCTL_GPARAMS32:
return snd_timer_user_gparams_compat(file, argp);
diff --git a/sound/core/ump_convert.c b/sound/core/ump_convert.c
index 0a7972d91ec5..ecbd894f4c83 100644
--- a/sound/core/ump_convert.c
+++ b/sound/core/ump_convert.c
@@ -69,7 +69,7 @@ static u32 upscale_14_to_32bit(u16 src)
static int cvt_ump_system_to_legacy(u32 data, unsigned char *buf)
{
buf[0] = ump_message_status_channel(data);
- switch (ump_message_status_code(data)) {
+ switch (buf[0]) {
case UMP_SYSTEM_STATUS_MIDI_TIME_CODE:
case UMP_SYSTEM_STATUS_SONG_SELECT:
buf[1] = (data >> 8) & 0x7f;
diff --git a/sound/hda/codecs/realtek/alc269.c b/sound/hda/codecs/realtek/alc269.c
index 404a93f552cb..224eff304473 100644
--- a/sound/hda/codecs/realtek/alc269.c
+++ b/sound/hda/codecs/realtek/alc269.c
@@ -3937,6 +3937,7 @@ enum {
ALC236_FIXUP_HP_LIMIT_INT_MIC_BOOST,
ALC287_FIXUP_LEGION_15IMHG05_SPEAKERS,
ALC287_FIXUP_LEGION_15IMHG05_AUTOMUTE,
+ ALC287_FIXUP_LEGION_S7_15IMH05_AMP,
ALC287_FIXUP_YOGA7_14ITL_SPEAKERS,
ALC298_FIXUP_LENOVO_C940_DUET7,
ALC287_FIXUP_LENOVO_YOGA_BOOK_9I,
@@ -3951,6 +3952,7 @@ enum {
ALC287_FIXUP_LEGION_16ACHG6,
ALC287_FIXUP_CS35L41_I2C_2,
ALC287_FIXUP_CS35L41_I2C_2_HP_GPIO_LED,
+ ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS,
ALC287_FIXUP_CS35L41_I2C_4,
ALC245_FIXUP_CS35L41_SPI_1,
ALC245_FIXUP_CS35L41_SPI_2,
@@ -4058,6 +4060,53 @@ static void alc700_fixup_intel_hades_canyon(struct hda_codec *codec,
alc_update_coef_idx(codec, 0x1b, 0x0400, 0x0400);
}
+static const struct hda_verb legion_amp_verbs[] = {
+ /* left speaker */
+ { 0x20, AC_VERB_SET_COEF_INDEX, 0x24 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x41 },
+ { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0xc },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x1a },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
+ { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x2 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
+
+ /* right speaker */
+ { 0x20, AC_VERB_SET_COEF_INDEX, 0x24 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x42 },
+ { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0xc },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x2a },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
+ { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x2 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
+ {}
+};
+
+static void alc287_fixup_legion_s7_amp(struct hda_codec *codec,
+ const struct hda_fixup *fix, int action)
+{
+ if (action != HDA_FIXUP_ACT_INIT)
+ return;
+
+ snd_hda_sequence_write(codec, legion_amp_verbs);
+ /*
+ * The amplifiers accept the setup only a few ms after the first
+ * write (8 ms is the measured minimum on the test machine), so
+ * keep a margin.
+ */
+ msleep(20);
+ snd_hda_sequence_write(codec, legion_amp_verbs);
+}
+
static const struct hda_fixup alc269_fixups[] = {
[ALC233_FIXUP_WUJIE_SPEAKERS] = {
.type = HDA_FIXUP_PINS,
@@ -6054,41 +6103,13 @@ static const struct hda_fixup alc269_fixups[] = {
},
[ALC287_FIXUP_LEGION_15IMHG05_SPEAKERS] = {
.type = HDA_FIXUP_VERBS,
- //.v.verbs = legion_15imhg05_coefs,
- .v.verbs = (const struct hda_verb[]) {
- // set left speaker Legion 7i.
- { 0x20, AC_VERB_SET_COEF_INDEX, 0x24 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x41 },
-
- { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0xc },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x1a },
- { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
-
- { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x2 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
-
- // set right speaker Legion 7i.
- { 0x20, AC_VERB_SET_COEF_INDEX, 0x24 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x42 },
-
- { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0xc },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x2a },
- { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
-
- { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x2 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
- {}
- },
+ .v.verbs = legion_amp_verbs,
+ .chained = true,
+ .chain_id = ALC287_FIXUP_LEGION_15IMHG05_AUTOMUTE,
+ },
+ [ALC287_FIXUP_LEGION_S7_15IMH05_AMP] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc287_fixup_legion_s7_amp,
.chained = true,
.chain_id = ALC287_FIXUP_LEGION_15IMHG05_AUTOMUTE,
},
@@ -6212,6 +6233,12 @@ static const struct hda_fixup alc269_fixups[] = {
.chained = true,
.chain_id = ALC285_FIXUP_HP_MUTE_LED,
},
+ [ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = cs35l41_fixup_i2c_two,
+ .chained = true,
+ .chain_id = ALC245_FIXUP_HP_X360_MUTE_LEDS,
+ },
[ALC287_FIXUP_CS35L41_I2C_4] = {
.type = HDA_FIXUP_FUNC,
.v.func = cs35l41_fixup_i2c_four,
@@ -6710,6 +6737,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1028, 0x097e, "Dell Precision", ALC289_FIXUP_DUAL_SPK),
SND_PCI_QUIRK(0x1028, 0x098d, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1028, 0x09bf, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1028, 0x09d1, "Alienware m17 R3", ALC289_FIXUP_DUAL_SPK),
SND_PCI_QUIRK(0x1028, 0x0a2e, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC),
SND_PCI_QUIRK(0x1028, 0x0a30, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC),
SND_PCI_QUIRK(0x1028, 0x0a38, "Dell Latitude 7520", ALC269_FIXUP_DELL4_MIC_NO_PRESENCE_QUIET),
@@ -7108,10 +7136,10 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x103c, 0x8d92, "HP ZBook Firefly 16 G12", ALC285_FIXUP_HP_GPIO_LED),
SND_PCI_QUIRK(0x103c, 0x8dcd, "HP Victus 15-fa2xxx", ALC245_FIXUP_HP_MUTE_LED_COEFBIT),
SND_PCI_QUIRK(0x103c, 0x8d9b, "HP 17 Turbine OmniBook 7 UMA", ALC287_FIXUP_CS35L41_I2C_2),
- SND_PCI_QUIRK(0x103c, 0x8d9c, "HP 17 Turbine OmniBook 7 DIS", ALC287_FIXUP_CS35L41_I2C_2),
+ SND_PCI_QUIRK(0x103c, 0x8d9c, "HP 17 Turbine OmniBook 7 DIS", ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS),
SND_PCI_QUIRK(0x103c, 0x8d9d, "HP 17 Turbine OmniBook X UMA", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8d9e, "HP 17 Turbine OmniBook X DIS", ALC287_FIXUP_CS35L41_I2C_2),
- SND_PCI_QUIRK(0x103c, 0x8d9f, "HP 14 Cadet (x360)", ALC287_FIXUP_CS35L41_I2C_2),
+ SND_PCI_QUIRK(0x103c, 0x8d9f, "HP 14 Cadet (x360)", ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS),
SND_PCI_QUIRK(0x103c, 0x8da0, "HP 16 Clipper OmniBook 7(X360)", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8da1, "HP 16 Clipper OmniBook X", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8da7, "HP 14 Enstrom OmniBook X", ALC287_FIXUP_CS35L41_I2C_2),
@@ -7543,7 +7571,12 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x17aa, 0x3801, "Lenovo Yoga9 14IAP7", ALC287_FIXUP_YOGA9_14IAP7_BASS_SPK_PIN),
HDA_CODEC_QUIRK(0x17aa, 0x3802, "DuetITL 2021", ALC287_FIXUP_YOGA7_14ITL_SPEAKERS),
SND_PCI_QUIRK(0x17aa, 0x3802, "Lenovo Yoga Pro 9 14IRP8", ALC287_FIXUP_TAS2781_I2C),
- SND_PCI_QUIRK(0x17aa, 0x3811, "Legion S7 15IMH05", ALC287_FIXUP_LEGION_15IMHG05_SPEAKERS),
+ /* Yoga Pro 9 16IMH9 and Legion 7 16ITHG6 share PCI SSID 17aa:3811
+ * with Legion S7 15IMH05; use codec SSID to distinguish them
+ */
+ HDA_CODEC_QUIRK(0x17aa, 0x38d6, "Lenovo Yoga Pro 9 16IMH9", ALC287_FIXUP_TAS2781_I2C),
+ HDA_CODEC_QUIRK(0x17aa, 0x3855, "Legion 7 16ITHG6", ALC287_FIXUP_LEGION_16ITHG6),
+ SND_PCI_QUIRK(0x17aa, 0x3811, "Legion S7 15IMH05", ALC287_FIXUP_LEGION_S7_15IMH05_AMP),
SND_PCI_QUIRK(0x17aa, 0x3813, "Legion 7i 15IMHG05", ALC287_FIXUP_LEGION_15IMHG05_SPEAKERS),
SND_PCI_QUIRK(0x17aa, 0x3818, "Lenovo C940 / Yoga Duet 7", ALC298_FIXUP_LENOVO_C940_DUET7),
SND_PCI_QUIRK(0x17aa, 0x3819, "Lenovo 13s Gen2 ITL", ALC287_FIXUP_13S_GEN2_SPEAKERS),
diff --git a/sound/hda/codecs/side-codecs/cs35l41_hda.c b/sound/hda/codecs/side-codecs/cs35l41_hda.c
index de2fd6471f05..d9f687967be5 100644
--- a/sound/hda/codecs/side-codecs/cs35l41_hda.c
+++ b/sound/hda/codecs/side-codecs/cs35l41_hda.c
@@ -1921,7 +1921,6 @@ int cs35l41_hda_parse_acpi(struct cs35l41_hda *cs35l41, struct device *physdev,
hw_cfg->valid = false;
hw_cfg->gpio1.valid = false;
hw_cfg->gpio2.valid = false;
- acpi_dev_put(cs35l41->dacpi);
return ret;
}
@@ -1929,7 +1928,6 @@ int cs35l41_hda_parse_acpi(struct cs35l41_hda *cs35l41, struct device *physdev,
static int cs35l41_hda_read_acpi(struct cs35l41_hda *cs35l41, const char *hid, int id)
{
struct acpi_device *adev;
- struct device *physdev;
struct spi_device *spi;
const char *sub;
int ret;
@@ -1941,11 +1939,10 @@ static int cs35l41_hda_read_acpi(struct cs35l41_hda *cs35l41, const char *hid, i
}
cs35l41->dacpi = adev;
- physdev = get_device(acpi_get_first_physical_node(adev));
- if (!physdev) {
- acpi_dev_put(adev);
+ struct device *physdev __free(put_device) =
+ get_device(acpi_get_first_physical_node(adev));
+ if (!physdev)
return -ENODEV;
- }
sub = acpi_get_subsystem_id(ACPI_HANDLE(physdev));
if (IS_ERR(sub))
@@ -1959,13 +1956,9 @@ static int cs35l41_hda_read_acpi(struct cs35l41_hda *cs35l41, const char *hid, i
}
ret = cs35l41_hda_parse_acpi(cs35l41, physdev, id);
- if (ret) {
- put_device(physdev);
+ if (ret)
return ret;
- }
out:
- put_device(physdev);
-
cs35l41->bypass_fw = false;
if (cs35l41->control_bus == SPI) {
spi = to_spi_device(cs35l41->dev);
@@ -2004,8 +1997,10 @@ int cs35l41_hda_probe(struct device *dev, const char *device_name, int id, int i
dev_set_drvdata(dev, cs35l41);
ret = cs35l41_hda_read_acpi(cs35l41, device_name, id);
- if (ret)
- return dev_err_probe(cs35l41->dev, ret, "Platform not supported\n");
+ if (ret) {
+ dev_err_probe(cs35l41->dev, ret, "Platform not supported\n");
+ goto err_acpi;
+ }
if (IS_ERR(cs35l41->reset_gpio)) {
ret = PTR_ERR(cs35l41->reset_gpio);
@@ -2092,6 +2087,7 @@ int cs35l41_hda_probe(struct device *dev, const char *device_name, int id, int i
err:
if (cs35l41_safe_reset(cs35l41->regmap, cs35l41->hw_cfg.bst_type))
gpiod_set_value_cansleep(cs35l41->reset_gpio, 0);
+err_acpi:
gpiod_put(cs35l41->reset_gpio);
gpiod_put(cs35l41->cs_gpio);
acpi_dev_put(cs35l41->dacpi);
diff --git a/sound/hda/codecs/side-codecs/tas2781_hda.c b/sound/hda/codecs/side-codecs/tas2781_hda.c
index 00628e0b5c3e..dc4e42fd8848 100644
--- a/sound/hda/codecs/side-codecs/tas2781_hda.c
+++ b/sound/hda/codecs/side-codecs/tas2781_hda.c
@@ -50,6 +50,31 @@ static void cali_cnv(unsigned char *data, unsigned int base, int offset)
(u32 *)®_data, TASDEV_CALIB_N);
}
+/*
+ * V1 data without CRC is only trusted if every expected device has a
+ * populated, self-consistent entry: R0 (Q27) * 1/R0 (Q31) must be
+ * close to 1.0 in Q58.
+ */
+static bool tas2781_v1_nocrc_valid(const unsigned int *v, int ndev)
+{
+ const u64 one = 1ULL << 58;
+ int i;
+
+ if (ndev <= 0 || ndev > 4)
+ return false;
+
+ for (i = 0; i < ndev; i++) {
+ const unsigned int *e = &v[i * 5];
+ u64 prod = (u64)e[0] * e[1];
+
+ if (!e[0] || !e[1] ||
+ prod < one - one / 10 || prod > one + one / 10)
+ return false;
+ }
+
+ return true;
+}
+
static void tas2781_apply_calib(struct tasdevice_priv *p)
{
struct calidata *cali_data = &p->cali_data;
@@ -145,7 +170,7 @@ static void tas2781_apply_calib(struct tasdevice_priv *p)
* but leave TimeStamp and CRC zeroed.
*/
if (!tmp_val[20] && !tmp_val[21] &&
- tmp_val[0] && tmp_val[1]) {
+ tas2781_v1_nocrc_valid(tmp_val, p->ndev)) {
dev_warn(p->dev,
"%s: V1 data without CRC, accepting\n",
__func__);
diff --git a/sound/pci/cs46xx/dsp_spos.c b/sound/pci/cs46xx/dsp_spos.c
index ba290f3a50e4..144c87097ae8 100644
--- a/sound/pci/cs46xx/dsp_spos.c
+++ b/sound/pci/cs46xx/dsp_spos.c
@@ -1106,6 +1106,8 @@ int cs46xx_dsp_scb_and_task_init (struct snd_cs46xx *chip)
null_scb.entry_point = null_algorithm->address;
ins->the_null_scb = cs46xx_dsp_create_scb(chip, "nullSCB", (u32 *)&null_scb, NULL_SCB_ADDR);
+ if (!ins->the_null_scb)
+ return -ENOMEM;
ins->the_null_scb->task_entry = null_algorithm;
ins->the_null_scb->sub_list_ptr = ins->the_null_scb;
ins->the_null_scb->next_scb_ptr = ins->the_null_scb;
diff --git a/sound/pci/cs46xx/dsp_spos_scb_lib.c b/sound/pci/cs46xx/dsp_spos_scb_lib.c
index 32ed415bf427..b7c41f2bc312 100644
--- a/sound/pci/cs46xx/dsp_spos_scb_lib.c
+++ b/sound/pci/cs46xx/dsp_spos_scb_lib.c
@@ -274,7 +274,8 @@ _dsp_create_generic_scb (struct snd_cs46xx *chip, char * name, u32 * scb_data, u
dev_dbg(chip->card->dev, "dsp_spos: creating SCB <%s>\n", name);
scb = cs46xx_dsp_create_scb(chip,name,scb_data,dest);
-
+ if (!scb)
+ return NULL;
scb->sub_list_ptr = ins->the_null_scb;
scb->next_scb_ptr = ins->the_null_scb;
diff --git a/sound/soc/amd/acp-config.c b/sound/soc/amd/acp-config.c
index 653e8ce94608..7ed461c1d519 100644
--- a/sound/soc/amd/acp-config.c
+++ b/sound/soc/amd/acp-config.c
@@ -23,6 +23,44 @@
static int acp_quirk_data;
+static const struct dmi_system_id acp70_acpi_flag_override_table[] = {
+ {
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "ASUSTeK COMPUTER INC."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "HN7306EA"),
+ },
+ },
+ {
+ /* ASUS Zenbook S16 UM5606GA (Strix Point, ACP 7.0) */
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "ASUSTeK COMPUTER INC."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "Zenbook S16 UM5606GA"),
+ },
+ },
+ {
+ /* Lenovo Yoga Pro 7 15ASH11 (Strix Halo, ACP 7.0) */
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "LENOVO"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "83W5"),
+ },
+ },
+ {
+ /* HP OmniBook X Flip 16-cc0xxx */
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "HP"),
+ DMI_MATCH(DMI_BOARD_NAME, "8EA2"),
+ },
+ },
+ {
+ /* Dell 14S DS14265 */
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "Dell 14S DS14265"),
+ },
+ },
+ {}
+};
+
static const struct config_entry config_table[] = {
{
.flags = FLAG_AMD_SOF,
@@ -200,8 +238,11 @@ int snd_amd_acp_find_config(struct pci_dev *pci)
*/
if (!pci->revision)
return 0;
- else if (pci->revision >= ACP_7_0_REV)
+ else if (pci->revision >= ACP_7_0_REV) {
+ if (dmi_check_system(acp70_acpi_flag_override_table))
+ return 0;
return snd_amd_acp_acpi_find_config(pci);
+ }
for (i = 0; i < ARRAY_SIZE(config_table); i++, table++) {
if (table->device != device)
diff --git a/sound/soc/amd/yc/acp6x-mach.c b/sound/soc/amd/yc/acp6x-mach.c
index 209ac43f9e31..f7507446494b 100644
--- a/sound/soc/amd/yc/acp6x-mach.c
+++ b/sound/soc/amd/yc/acp6x-mach.c
@@ -682,6 +682,13 @@ static const struct dmi_system_id yc_acp_quirk_table[] = {
DMI_MATCH(DMI_BOARD_NAME, "8BD6"),
}
},
+ {
+ .driver_data = &acp6x_card,
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "HP"),
+ DMI_MATCH(DMI_BOARD_NAME, "8EE3"),
+ }
+ },
{
.driver_data = &acp6x_card,
.matches = {
diff --git a/sound/soc/codecs/hdac_hda.c b/sound/soc/codecs/hdac_hda.c
index ac28b3cadb69..ca2d3130cc4c 100644
--- a/sound/soc/codecs/hdac_hda.c
+++ b/sound/soc/codecs/hdac_hda.c
@@ -636,12 +636,35 @@ static const struct snd_soc_component_driver hdac_hda_hdmi_codec = {
.endianness = 1,
};
+static void hdac_hda_restore_dev_type(void *data)
+{
+ struct hdac_device *hdev = data;
+
+ hdev->type = HDA_DEV_LEGACY;
+}
+
static int hdac_hda_dev_probe(struct hdac_device *hdev)
{
- struct hdac_hda_priv *hda_pvt = dev_get_drvdata(&hdev->dev);
+ struct hdac_hda_priv *hda_pvt;
struct hdac_ext_link *hlink;
int ret;
+ hda_pvt = devm_kzalloc(&hdev->dev, sizeof(*hda_pvt), GFP_KERNEL);
+ if (!hda_pvt)
+ return -ENOMEM;
+
+ hda_pvt->codec = dev_to_hda_codec(&hdev->dev);
+ hda_pvt->dev_index = hdev->addr;
+ /* Intel iDisp (HDMI/DP) codec */
+ if (((hdev->vendor_id >> 16) & 0xFFFF) == 0x8086)
+ hda_pvt->need_display_power = true;
+ dev_set_drvdata(&hdev->dev, hda_pvt);
+
+ /* Allow the legacy codec driver to match again after unbind */
+ ret = devm_add_action(&hdev->dev, hdac_hda_restore_dev_type, hdev);
+ if (ret)
+ return ret;
+
/* hold the ref while we probe */
hlink = snd_hdac_ext_bus_get_hlink_by_name(hdev->bus, dev_name(&hdev->dev));
if (!hlink) {
diff --git a/sound/soc/qcom/qdsp6/q6apm-dai.c b/sound/soc/qcom/qdsp6/q6apm-dai.c
index 54c1a33f9f93..4ea2c9869d5a 100644
--- a/sound/soc/qcom/qdsp6/q6apm-dai.c
+++ b/sound/soc/qcom/qdsp6/q6apm-dai.c
@@ -442,11 +442,15 @@ static snd_pcm_uframes_t q6apm_dai_pointer(struct snd_soc_component *component,
struct q6apm_dai_rtd *prtd = runtime->private_data;
snd_pcm_uframes_t ptr;
- ptr = q6apm_get_hw_pointer(prtd->graph, substream->stream) * runtime->period_size;
- if (ptr)
- return ptr - 1;
+ /*
+ * The hw pointer counts the period buffers the DSP has returned in
+ * this lap, 1..periods. A full lap is the start of the ring again:
+ * report 0 rather than one frame less, so that the position of a
+ * drained buffer reaches appl_ptr and ALSA sees the underrun.
+ */
+ ptr = q6apm_get_hw_pointer(prtd->graph, substream->stream) % runtime->periods;
- return 0;
+ return ptr * runtime->period_size;
}
static int q6apm_dai_hw_params(struct snd_soc_component *component,
diff --git a/sound/soc/qcom/qdsp6/q6apm.c b/sound/soc/qcom/qdsp6/q6apm.c
index 50d5d2143daa..7894d4430ff1 100644
--- a/sound/soc/qcom/qdsp6/q6apm.c
+++ b/sound/soc/qcom/qdsp6/q6apm.c
@@ -765,8 +765,6 @@ static int apm_probe(gpr_device_t *gdev)
if (!apm)
return -ENOMEM;
- dev_set_drvdata(dev, apm);
-
mutex_init(&apm->lock);
apm->dev = dev;
apm->gdev = gdev;
@@ -780,6 +778,7 @@ static int apm_probe(gpr_device_t *gdev)
idr_init(&apm->modules_idr);
+ dev_set_drvdata(dev, apm);
g_apm = apm;
q6apm_get_apm_state(apm);
@@ -820,6 +819,9 @@ static int apm_callback(struct gpr_resp_pkt *data, void *priv, int op)
struct gpr_ibasic_rsp_result_t *result;
struct gpr_hdr *hdr = &data->hdr;
+ if (!apm)
+ return 0;
+
result = data->payload;
switch (hdr->opcode) {
diff --git a/sound/soc/qcom/qdsp6/q6prm.c b/sound/soc/qcom/qdsp6/q6prm.c
index 81554d202658..22e2a174a387 100644
--- a/sound/soc/qcom/qdsp6/q6prm.c
+++ b/sound/soc/qcom/qdsp6/q6prm.c
@@ -197,6 +197,9 @@ static int prm_callback(struct gpr_resp_pkt *data, void *priv, int op)
struct gpr_ibasic_rsp_result_t *result;
struct gpr_hdr *hdr = &data->hdr;
+ if (!prm)
+ return 0;
+
switch (hdr->opcode) {
case PRM_CMD_RSP_REQUEST_HW_RSC:
case PRM_CMD_RSP_RELEASE_HW_RSC:
diff --git a/sound/soc/sof/intel/hda-codec.c b/sound/soc/sof/intel/hda-codec.c
index 37674ea452d6..2f5a1db23535 100644
--- a/sound/soc/sof/intel/hda-codec.c
+++ b/sound/soc/sof/intel/hda-codec.c
@@ -15,8 +15,6 @@
#include "hda.h"
#if IS_ENABLED(CONFIG_SND_SOC_SOF_HDA_AUDIO_CODEC)
-#include "../../codecs/hdac_hda.h"
-
#define CODEC_PROBE_RETRIES 3
#define IDISP_VID_INTEL 0x80860000
@@ -148,7 +146,6 @@ static struct hda_codec *hda_codec_device_init(struct hdac_bus *bus, int addr, i
/* probe individual codec */
static int hda_codec_probe(struct snd_sof_dev *sdev, int address)
{
- struct hdac_hda_priv *hda_priv;
struct hda_bus *hbus = sof_to_hbus(sdev);
struct hda_codec *codec;
u32 hda_cmd = (address << 28) | (AC_NODE_ROOT << 20) |
@@ -168,27 +165,15 @@ static int hda_codec_probe(struct snd_sof_dev *sdev, int address)
dev_dbg(sdev->dev, "HDA codec #%d probed OK: response: %x\n",
address, resp);
- hda_priv = devm_kzalloc(sdev->dev, sizeof(*hda_priv), GFP_KERNEL);
- if (!hda_priv)
- return -ENOMEM;
-
codec = hda_codec_device_init(&hbus->core, address, HDA_DEV_LEGACY);
ret = PTR_ERR_OR_ZERO(codec);
if (ret < 0)
return ret;
- hda_priv->codec = codec;
- hda_priv->dev_index = address;
- dev_set_drvdata(&codec->core.dev, hda_priv);
-
- if ((resp & 0xFFFF0000) == IDISP_VID_INTEL) {
- if (!hbus->core.audio_component) {
- dev_dbg(sdev->dev,
- "iDisp hw present but no driver\n");
- ret = -ENOENT;
- goto out;
- }
- hda_priv->need_display_power = true;
+ if ((resp & 0xFFFF0000) == IDISP_VID_INTEL && !hbus->core.audio_component) {
+ dev_dbg(sdev->dev, "iDisp hw present but no driver\n");
+ ret = -ENOENT;
+ goto out;
}
if (is_generic_config(hbus))
diff --git a/sound/soc/stm/stm32_i2s.c b/sound/soc/stm/stm32_i2s.c
index 6ca21780f21d..ab06d13a59b9 100644
--- a/sound/soc/stm/stm32_i2s.c
+++ b/sound/soc/stm/stm32_i2s.c
@@ -1192,7 +1192,7 @@ static int stm32_i2s_parse_dt(struct platform_device *pdev,
struct device_node *np = pdev->dev.of_node;
struct reset_control *rst;
struct resource *res;
- int irq, ret;
+ int ret;
if (!np)
return -ENODEV;
@@ -1238,18 +1238,6 @@ static int stm32_i2s_parse_dt(struct platform_device *pdev,
return ret;
}
- /* Get irqs */
- irq = platform_get_irq(pdev, 0);
- if (irq < 0)
- return irq;
-
- ret = devm_request_irq(&pdev->dev, irq, stm32_i2s_isr, 0,
- dev_name(&pdev->dev), i2s);
- if (ret) {
- dev_err(&pdev->dev, "irq request returned %d\n", ret);
- return ret;
- }
-
/* Reset */
rst = devm_reset_control_get_optional_exclusive(&pdev->dev, NULL);
if (IS_ERR(rst))
@@ -1274,7 +1262,7 @@ static int stm32_i2s_probe(struct platform_device *pdev)
{
struct stm32_i2s_data *i2s;
u32 val;
- int ret;
+ int irq, ret;
i2s = devm_kzalloc(&pdev->dev, sizeof(*i2s), GFP_KERNEL);
if (!i2s)
@@ -1300,9 +1288,18 @@ static int stm32_i2s_probe(struct platform_device *pdev)
return dev_err_probe(&pdev->dev, PTR_ERR(i2s->regmap),
"Regmap init error\n");
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ return irq;
+
+ ret = devm_request_irq(&pdev->dev, irq, stm32_i2s_isr, 0,
+ dev_name(&pdev->dev), i2s);
+ if (ret)
+ return ret;
+
ret = snd_dmaengine_pcm_register(&pdev->dev, &stm32_i2s_pcm_config, 0);
if (ret)
- return dev_err_probe(&pdev->dev, ret, "PCM DMA register error\n");
+ return ret;
ret = snd_soc_register_component(&pdev->dev, &stm32_i2s_component,
i2s->dai_drv, 1);
diff --git a/sound/usb/bcd2000/bcd2000.c b/sound/usb/bcd2000/bcd2000.c
index 2bd49bf82748..0fd086cac955 100644
--- a/sound/usb/bcd2000/bcd2000.c
+++ b/sound/usb/bcd2000/bcd2000.c
@@ -183,10 +183,24 @@ static int bcd2000_midi_output_open(struct snd_rawmidi_substream *substream)
static int bcd2000_midi_output_close(struct snd_rawmidi_substream *substream)
{
struct bcd2000 *bcd2k = substream->rmidi->private_data;
+ struct urb *urb = NULL;
- if (bcd2k->midi_out_active && bcd2k->midi_out_urb) {
- usb_kill_urb(bcd2k->midi_out_urb);
- bcd2k->midi_out_active = 0;
+ /*
+ * Reference the URB under the lock so that it stays valid until
+ * usb_kill_urb() returns, even if a disconnect frees it in the
+ * meantime. usb_kill_urb() cannot run under the lock, as the
+ * completion handler takes the same lock.
+ */
+ scoped_guard(spinlock_irqsave, &bcd2k->midi_lock) {
+ if (bcd2k->midi_out_active && bcd2k->midi_out_urb) {
+ urb = usb_get_urb(bcd2k->midi_out_urb);
+ bcd2k->midi_out_active = 0;
+ }
+ }
+
+ if (urb) {
+ usb_kill_urb(urb);
+ usb_put_urb(urb);
}
return 0;
diff --git a/sound/usb/caiaq/input.c b/sound/usb/caiaq/input.c
index c12eeb971000..6a5adb8d1e6a 100644
--- a/sound/usb/caiaq/input.c
+++ b/sound/usb/caiaq/input.c
@@ -830,6 +830,8 @@ int snd_usb_caiaq_input_init(struct snd_usb_caiaqdev *cdev)
return 0;
exit_free_idev:
+ usb_free_urb(cdev->ep4_in_urb);
+ cdev->ep4_in_urb = NULL;
input_free_device(input);
cdev->input_dev = NULL;
return ret;
diff --git a/sound/usb/line6/capture.c b/sound/usb/line6/capture.c
index 46acd2f44c44..f59ec9c4a1a5 100644
--- a/sound/usb/line6/capture.c
+++ b/sound/usb/line6/capture.c
@@ -94,6 +94,18 @@ void line6_capture_copy(struct snd_line6_pcm *line6pcm, char *fbuf, int fsize)
if (runtime == NULL)
return;
+ /*
+ * snd_line6_capture_open() constrains the buffer to hold at least
+ * two packets, so this cannot happen for a conforming device. Drop
+ * the packet rather than overrun the single wrap below.
+ */
+ if (unlikely(frames > runtime->buffer_size)) {
+ dev_err_ratelimited(line6pcm->line6->ifcdev,
+ "packet of %d frames exceeds %lu-frame buffer, dropping\n",
+ frames, runtime->buffer_size);
+ return;
+ }
+
if (line6pcm->in.pos_done + frames > runtime->buffer_size) {
/*
The transferred area goes over buffer boundary,
@@ -231,6 +243,23 @@ static int snd_line6_capture_open(struct snd_pcm_substream *substream)
if (err < 0)
return err;
+ /*
+ * line6_capture_copy() wraps the ring buffer once per packet and
+ * advances pos_done by the packet, which only works when the buffer
+ * holds more than one packet: with exactly one, pos_done gains a
+ * buffer and loses it again on every packet. The packet size comes
+ * from the device, so bound the buffer by two packets here. The
+ * impulse-response path copies a playback URB into this ring, so
+ * take the larger of the two endpoints.
+ */
+ err = snd_pcm_hw_constraint_minmax(runtime,
+ SNDRV_PCM_HW_PARAM_BUFFER_BYTES,
+ 2 * max(line6pcm->max_packet_size_in,
+ line6pcm->max_packet_size_out),
+ UINT_MAX);
+ if (err < 0)
+ return err;
+
line6_pcm_acquire(line6pcm, LINE6_STREAM_CAPTURE_HELPER, false);
runtime->hw = line6pcm->properties->capture_hw;
diff --git a/sound/usb/midi.c b/sound/usb/midi.c
index 787956252e80..c404412ae53b 100644
--- a/sound/usb/midi.c
+++ b/sound/usb/midi.c
@@ -113,6 +113,7 @@ struct snd_usb_midi {
unsigned int opened[2];
unsigned char disconnected;
unsigned char input_running;
+ unsigned char keep_input_running;
struct snd_kcontrol *roland_load_ctl;
};
@@ -1145,41 +1146,42 @@ static int substream_open(struct snd_rawmidi_substream *substream, int dir,
int open)
{
struct snd_usb_midi *umidi = substream->rmidi->private_data;
- struct snd_kcontrol *ctl;
+ struct snd_ctl_elem_id ctl_id;
+ bool activate_ctl = false;
+ bool active;
guard(rwsem_read)(&umidi->disc_rwsem);
if (umidi->disconnected)
return open ? -ENODEV : 0;
- guard(mutex)(&umidi->mutex);
- if (open) {
- if (!umidi->opened[0] && !umidi->opened[1]) {
- if (umidi->roland_load_ctl) {
- ctl = umidi->roland_load_ctl;
- ctl->vd[0].access |=
- SNDRV_CTL_ELEM_ACCESS_INACTIVE;
- snd_ctl_notify(umidi->card,
- SNDRV_CTL_EVENT_MASK_INFO, &ctl->id);
- update_roland_altsetting(umidi);
+ scoped_guard(mutex, &umidi->mutex) {
+ if (open) {
+ if (!umidi->opened[0] && !umidi->opened[1]) {
+ if (umidi->roland_load_ctl) {
+ ctl_id = umidi->roland_load_ctl->id;
+ activate_ctl = true;
+ active = false;
+ update_roland_altsetting(umidi);
+ }
}
- }
- umidi->opened[dir]++;
- if (umidi->opened[1])
- snd_usbmidi_input_start(&umidi->list);
- } else {
- umidi->opened[dir]--;
- if (!umidi->opened[1])
- snd_usbmidi_input_stop(&umidi->list);
- if (!umidi->opened[0] && !umidi->opened[1]) {
- if (umidi->roland_load_ctl) {
- ctl = umidi->roland_load_ctl;
- ctl->vd[0].access &=
- ~SNDRV_CTL_ELEM_ACCESS_INACTIVE;
- snd_ctl_notify(umidi->card,
- SNDRV_CTL_EVENT_MASK_INFO, &ctl->id);
+ umidi->opened[dir]++;
+ if (umidi->opened[1])
+ snd_usbmidi_input_start(&umidi->list);
+ } else {
+ umidi->opened[dir]--;
+ if (!umidi->opened[1] && !umidi->keep_input_running)
+ snd_usbmidi_input_stop(&umidi->list);
+ if (!umidi->opened[0] && !umidi->opened[1]) {
+ if (umidi->roland_load_ctl) {
+ ctl_id = umidi->roland_load_ctl->id;
+ activate_ctl = true;
+ active = true;
+ }
}
}
}
+ if (activate_ctl)
+ snd_ctl_activate_id(umidi->card, &ctl_id, active);
return 0;
}
@@ -2258,6 +2260,10 @@ static int snd_usbmidi_detect_roland(struct snd_usb_midi *umidi,
struct usb_host_interface *hostif;
u8 *cs_desc;
+ /* TD-11 becomes unusable if input isn't serviced continuously */
+ if (umidi->usb_id == USB_ID(0x0582, 0x0151))
+ umidi->keep_input_running = 1;
+
intf = umidi->iface;
if (!intf)
return -ENOENT;
@@ -2465,7 +2471,7 @@ void snd_usbmidi_input_start(struct list_head *p)
int i;
umidi = list_entry(p, struct snd_usb_midi, list);
- if (umidi->input_running || !umidi->opened[1])
+ if (umidi->input_running || (!umidi->opened[1] && !umidi->keep_input_running))
return;
for (i = 0; i < MIDI_MAX_ENDPOINTS; ++i)
snd_usbmidi_input_start_ep(umidi, umidi->endpoints[i].in);
@@ -2640,6 +2646,10 @@ int __snd_usbmidi_create(struct snd_card *card,
if (err < 0)
goto exit;
+ /* start input for devices that need continuous servicing */
+ if (umidi->keep_input_running)
+ snd_usbmidi_input_start(&umidi->list);
+
usb_autopm_get_interface_no_resume(umidi->iface);
list_add_tail(&umidi->list, midi_list);
diff --git a/sound/usb/misc/ua101.c b/sound/usb/misc/ua101.c
index 426171aa6dd2..c1d081231d7c 100644
--- a/sound/usb/misc/ua101.c
+++ b/sound/usb/misc/ua101.c
@@ -282,13 +282,29 @@ static void playback_work(struct work_struct *work)
/* fill packet with data or silence */
urb->urb.iso_frame_desc[0].length =
frames * ua->playback.frame_bytes;
- if (test_bit(ALSA_PLAYBACK_RUNNING, &ua->states))
+ if (test_bit(ALSA_PLAYBACK_RUNNING, &ua->states)) {
+ /*
+ * The packet size comes from the capture stream,
+ * i.e. from the device, while the ring buffer can
+ * be as small as two tiny periods. The single wrap
+ * in copy_playback_data() is only valid for
+ * frames <= buffer_size, so copy at most one
+ * buffer and send silence for the rest.
+ */
+ unsigned int copy_frames = min(frames,
+ ua->playback.substream->runtime->buffer_size);
+
do_period_elapsed |= copy_playback_data(&ua->playback,
&urb->urb,
- frames);
- else
+ copy_frames);
+ memset(urb->urb.transfer_buffer +
+ copy_frames * ua->playback.frame_bytes, 0,
+ (frames - copy_frames) *
+ ua->playback.frame_bytes);
+ } else {
memset(urb->urb.transfer_buffer, 0,
urb->urb.iso_frame_desc[0].length);
+ }
/* and off you go ... */
err = usb_submit_urb(&urb->urb, GFP_ATOMIC);
@@ -317,6 +333,13 @@ static bool copy_capture_data(struct ua101_stream *stream, struct urb *urb,
runtime = stream->substream->runtime;
frame_bytes = stream->frame_bytes;
+ /*
+ * A packet can carry up to max_packet_bytes, the device's
+ * wMaxPacketSize, which may exceed the whole ring buffer; keep what
+ * fits so the single wrap below stays valid.
+ */
+ if (unlikely(frames > runtime->buffer_size))
+ frames = runtime->buffer_size;
dest = runtime->dma_area + stream->buffer_pos * frame_bytes;
if (stream->buffer_pos + frames <= runtime->buffer_size) {
memcpy(dest, urb->transfer_buffer, frames * frame_bytes);
diff --git a/sound/usb/usx2y/us122l.c b/sound/usb/usx2y/us122l.c
index aaeae099b5f2..46dbf9ea1fb7 100644
--- a/sound/usb/usx2y/us122l.c
+++ b/sound/usb/usx2y/us122l.c
@@ -129,19 +129,26 @@ static const struct vm_operations_struct usb_stream_hwdep_vm_ops = {
static int usb_stream_hwdep_open(struct snd_hwdep *hw, struct file *file)
{
struct us122l *us122l = hw->private_data;
- struct usb_interface *iface;
+ struct usb_interface *iface, *iface0 = NULL;
if (hw->used >= 2)
return -EBUSY;
- if (!us122l->first)
- us122l->first = file;
-
if (us122l->is_us144) {
- iface = usb_ifnum_to_if(us122l->dev, 0);
- usb_autopm_get_interface(iface);
+ iface0 = usb_ifnum_to_if(us122l->dev, 0);
+ if (!iface0)
+ return -ENODEV;
}
+
iface = usb_ifnum_to_if(us122l->dev, 1);
+ if (!iface)
+ return -ENODEV;
+
+ if (!us122l->first)
+ us122l->first = file;
+
+ if (iface0)
+ usb_autopm_get_interface(iface0);
usb_autopm_get_interface(iface);
return 0;
}
@@ -153,10 +160,12 @@ static int usb_stream_hwdep_release(struct snd_hwdep *hw, struct file *file)
if (us122l->is_us144) {
iface = usb_ifnum_to_if(us122l->dev, 0);
- usb_autopm_put_interface(iface);
+ if (iface)
+ usb_autopm_put_interface(iface);
}
iface = usb_ifnum_to_if(us122l->dev, 1);
- usb_autopm_put_interface(iface);
+ if (iface)
+ usb_autopm_put_interface(iface);
if (us122l->first == file)
us122l->first = NULL;
guard(mutex)(&us122l->mutex);
@@ -179,6 +188,8 @@ static int usb_stream_hwdep_mmap(struct snd_hwdep *hw,
offset = area->vm_pgoff << PAGE_SHIFT;
guard(mutex)(&us122l->mutex);
s = us122l->sk.s;
+ if (!s)
+ return -ENODEV;
read = offset < s->read_size;
if (read) {
if (area->vm_flags & VM_WRITE)
diff --git a/sound/usb/usx2y/usX2Yhwdep.c b/sound/usb/usx2y/usX2Yhwdep.c
index 7f153a2fcdd8..3800c5b5887c 100644
--- a/sound/usb/usx2y/usX2Yhwdep.c
+++ b/sound/usb/usx2y/usX2Yhwdep.c
@@ -179,6 +179,9 @@ static int snd_usx2y_hwdep_dsp_load(struct snd_hwdep *hw,
int lret, err;
char *buf;
+ if (priv->chip_status & USX2Y_STAT_CHIP_HUP)
+ return -ENODEV;
+
buf = memdup_user(dsp->image, dsp->length);
if (IS_ERR(buf))
return PTR_ERR(buf);
@@ -201,11 +204,13 @@ static int snd_usx2y_hwdep_dsp_load(struct snd_hwdep *hw,
err = usx2y_in04_init(priv);
if (err) {
dev_err(&dev->dev, "usx2y_in04_init error\n");
+ priv->chip_status |= USX2Y_STAT_CHIP_HUP;
return err;
}
err = usx2y_create_alsa_devices(hw->card);
if (err) {
dev_err(&dev->dev, "usx2y_create_alsa_devices error %i\n", err);
+ priv->chip_status |= USX2Y_STAT_CHIP_HUP;
return err;
}
priv->chip_status |= USX2Y_STAT_CHIP_INIT;
diff --git a/sound/usb/usx2y/usb_stream.c b/sound/usb/usx2y/usb_stream.c
index 3122cf653273..7d621280fba3 100644
--- a/sound/usb/usx2y/usb_stream.c
+++ b/sound/usb/usx2y/usb_stream.c
@@ -682,6 +682,10 @@ int usb_stream_start(struct usb_stream_kernel *sk)
s->sync_packet = -1;
s->inpacket_head = -1;
sk->iso_frame_balance = 0;
+ sk->idle_inurb = sk->inurb[USB_STREAM_NURBS - 2];
+ sk->idle_outurb = sk->outurb[USB_STREAM_NURBS - 2];
+ sk->completed_inurb = sk->inurb[USB_STREAM_NURBS - 1];
+ sk->completed_outurb = sk->outurb[USB_STREAM_NURBS - 1];
++try;
for (u = 0; u < 2; u++) {
struct urb *inurb = sk->inurb[u];
@@ -739,11 +743,6 @@ int usb_stream_start(struct usb_stream_kernel *sk)
return -EFAULT;
}
- sk->idle_inurb = sk->inurb[USB_STREAM_NURBS - 2];
- sk->idle_outurb = sk->outurb[USB_STREAM_NURBS - 2];
- sk->completed_inurb = sk->inurb[USB_STREAM_NURBS - 1];
- sk->completed_outurb = sk->outurb[USB_STREAM_NURBS - 1];
-
/* wait, check */
{
int wait_ms = 3000;
diff --git a/sound/usb/usx2y/usbusx2yaudio.c b/sound/usb/usx2y/usbusx2yaudio.c
index 1ea181e9bb28..f035add1e1b5 100644
--- a/sound/usb/usx2y/usbusx2yaudio.c
+++ b/sound/usb/usx2y/usbusx2yaudio.c
@@ -51,6 +51,18 @@ static int usx2y_urb_capt_retire(struct snd_usx2y_substream *subs)
dev_dbg(&usx2y->dev->dev, "%s: 0 == len ERROR!\n", __func__);
continue;
}
+ /*
+ * snd_usx2y_pcm_open() constrains the buffer to hold at least
+ * two URBs, so this cannot happen for a device that keeps its
+ * packets at 50 frames. Drop the packet rather than exceed the
+ * single wrap below.
+ */
+ if (unlikely(len > runtime->buffer_size)) {
+ dev_err_ratelimited(&usx2y->dev->dev,
+ "%s: packet of %i frames exceeds %lu-frame buffer, dropping\n",
+ __func__, len, runtime->buffer_size);
+ continue;
+ }
/* copy a data chunk */
if ((hwptr_done + len) > runtime->buffer_size) {
@@ -106,14 +118,31 @@ static int usx2y_urb_play_prepare(struct snd_usx2y_substream *subs,
__func__, counts);
return -EPIPE;
}
+ if (counts * usx2y->stride > subs->maxpacksize) {
+ dev_err_ratelimited(&usx2y->dev->dev,
+ "%s: packet of %i bytes exceeds maxpacksize %u\n",
+ __func__, counts * usx2y->stride,
+ subs->maxpacksize);
+ return -EPIPE;
+ }
/* set up descriptor */
urb->iso_frame_desc[pack].offset = pack ?
urb->iso_frame_desc[pack - 1].offset +
urb->iso_frame_desc[pack - 1].length :
0;
- urb->iso_frame_desc[pack].length = cap_urb->iso_frame_desc[pack].actual_length;
+ urb->iso_frame_desc[pack].length = counts * usx2y->stride;
}
if (atomic_read(&subs->state) >= STATE_PRERUNNING) {
+ /*
+ * The copy below wraps once, which needs count <= buffer_size.
+ * snd_usx2y_pcm_open() guarantees it; guard anyway.
+ */
+ if (unlikely(count > runtime->buffer_size)) {
+ dev_err_ratelimited(&usx2y->dev->dev,
+ "%s: URB of %i frames exceeds %lu-frame buffer, stopping\n",
+ __func__, count, runtime->buffer_size);
+ return -EPIPE;
+ }
if (subs->hwptr + count > runtime->buffer_size) {
/* err, the transferred area goes over buffer boundary.
* copy the data to the temp buffer.
@@ -890,6 +919,18 @@ static int snd_usx2y_pcm_open(struct snd_pcm_substream *substream)
runtime->private_data = subs;
subs->pcm_substream = substream;
snd_pcm_hw_constraint_minmax(runtime, SNDRV_PCM_HW_PARAM_PERIOD_TIME, 1000, 200000);
+ /*
+ * Capture wraps the ring buffer once per packet and playback once
+ * per URB of nr_of_packs() packets of at most 50 frames (see
+ * usx2y_urb_play_prepare()). Playback needs the buffer to hold more
+ * than one URB and capture more than one packet; use the URB bound
+ * for both, and two URBs rather than one, because the position only
+ * advances when a URB is less than the whole buffer. The bound is in
+ * frames and does not depend on the altsetting, which is selected
+ * in prepare.
+ */
+ snd_pcm_hw_constraint_minmax(runtime, SNDRV_PCM_HW_PARAM_BUFFER_SIZE,
+ 2 * nr_of_packs() * 50, UINT_MAX);
return 0;
}
diff --git a/sound/x86/intel_hdmi_audio.c b/sound/x86/intel_hdmi_audio.c
index 8e2a007311d3..21ee9c915480 100644
--- a/sound/x86/intel_hdmi_audio.c
+++ b/sound/x86/intel_hdmi_audio.c
@@ -475,7 +475,8 @@ static void had_build_channel_allocation_map(struct snd_intelhad *intelhaddata)
kfree(intelhaddata->chmap->chmap);
intelhaddata->chmap->chmap = NULL;
- chmap = kzalloc(sizeof(*chmap), GFP_KERNEL);
+ /* info->chmap must be zero-terminated; allocate a spare terminator entry */
+ chmap = kzalloc_objs(*chmap, 2);
if (!chmap)
return;
@@ -575,6 +576,12 @@ static int had_register_chmap_ctls(struct snd_intelhad *intelhaddata,
intelhaddata->chmap->kctl->info = had_chmap_ctl_info;
intelhaddata->chmap->kctl->get = had_chmap_ctl_get;
intelhaddata->chmap->chmap = NULL;
+
+ /* drop the inherited TLV read: it walks chmap without our mutex */
+ intelhaddata->chmap->kctl->tlv.c = NULL;
+ intelhaddata->chmap->kctl->vd[0].access &=
+ ~(SNDRV_CTL_ELEM_ACCESS_TLV_READ | SNDRV_CTL_ELEM_ACCESS_TLV_CALLBACK);
+
return 0;
}
diff --git a/tools/perf/tests/tool_pmu.c b/tools/perf/tests/tool_pmu.c
index 1e900ef92e37..5dcb747a6391 100644
--- a/tools/perf/tests/tool_pmu.c
+++ b/tools/perf/tests/tool_pmu.c
@@ -1,8 +1,16 @@
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+#include <errno.h>
+#include <fcntl.h>
+#include <string.h>
+#include <unistd.h>
+#include <internal/xyarray.h>
+#include "cpumap.h"
#include "debug.h"
#include "evlist.h"
+#include "evsel.h"
#include "parse-events.h"
#include "tests.h"
+#include "thread_map.h"
#include "tool_pmu.h"
static int do_test(enum tool_pmu_event ev, bool with_pmu)
@@ -99,9 +107,80 @@ static int test__tool_pmu_with_pmu(struct test_suite *test __maybe_unused,
return TEST_OK;
}
+static int test__tool_pmu_incremental_open_unwind(struct test_suite *test __maybe_unused,
+ int subtest __maybe_unused)
+{
+ struct evlist *evlist = evlist__new();
+ struct parse_events_error err;
+ struct perf_cpu_map *cpus = NULL;
+ struct perf_thread_map *threads = NULL;
+ struct evsel *evsel;
+ int ret = TEST_FAIL, fd0 = -1;
+
+ if (!evlist)
+ return TEST_FAIL;
+
+ parse_events_error__init(&err);
+ if (parse_events(evlist, "tool/user_time/", &err)) {
+ parse_events_error__exit(&err);
+ evlist__delete(evlist);
+ return TEST_FAIL;
+ }
+ parse_events_error__exit(&err);
+
+ evsel = evlist__first(evlist);
+ cpus = perf_cpu_map__new("0,1");
+ threads = thread_map__new_by_tid(getpid());
+ if (!cpus || !threads)
+ goto out;
+
+ /* Step 1: Open CPU index 0 successfully */
+ if (evsel__open_per_cpu_and_thread(evsel, cpus, 0, threads) < 0) {
+ pr_debug("Failed to open CPU index 0\n");
+ goto out;
+ }
+
+ fd0 = (*(int *)xyarray__entry(evsel->core.fd, 0, 0));
+ if (fd0 < 0 || fcntl(fd0, F_GETFD) < 0) {
+ pr_debug("CPU index 0 FD is invalid (%d)\n", fd0);
+ goto out;
+ }
+
+ /* Step 2: Intentionally trigger failure on CPU index 1 */
+ evsel->core.attr.sample_period = 1; /* Not supported for tool PMU -> -EINVAL */
+ if (evsel__open_per_cpu_and_thread(evsel, cpus, 1, threads) >= 0) {
+ pr_debug("Unexpected success opening CPU index 1 with sample_period=1\n");
+ goto out;
+ }
+
+ /*
+ * Step 3: Check that CPU index 0's FD was NOT destroyed by CPU index 1's unwind.
+ */
+ if ((*(int *)xyarray__entry(evsel->core.fd, 0, 0)) != fd0) {
+ pr_debug("FAILED: CPU 0 FD overwritten: FD(evsel, 0, 0)=%d, expected %d\n",
+ (*(int *)xyarray__entry(evsel->core.fd, 0, 0)), fd0);
+ goto out;
+ }
+
+ if (fcntl(fd0, F_GETFD) < 0) {
+ pr_debug("FAILED: CPU 0 FD %d was closed by error unwind! errno=%d (%s)\n",
+ fd0, errno, strerror(errno));
+ goto out;
+ }
+
+ ret = TEST_OK;
+out:
+ evsel__close(evsel);
+ perf_cpu_map__put(cpus);
+ perf_thread_map__put(threads);
+ evlist__delete(evlist);
+ return ret;
+}
+
static struct test_case tests__tool_pmu[] = {
TEST_CASE("Parsing without PMU name", tool_pmu_without_pmu),
TEST_CASE("Parsing with PMU name", tool_pmu_with_pmu),
+ TEST_CASE("Incremental open error unwind boundary", tool_pmu_incremental_open_unwind),
{ .name = NULL, }
};
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index e01103443b23..654e9bbd66f9 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -2642,6 +2642,19 @@ static bool evsel__detect_missing_features(struct evsel *evsel, struct perf_cpu
return false;
}
+static void evsel__close_fds(struct evsel *evsel, int start_cpu_map_idx,
+ int cpu_map_idx, int thread, int nthreads)
+{
+ do {
+ while (--thread >= 0) {
+ if (FD(evsel, cpu_map_idx, thread) >= 0)
+ close(FD(evsel, cpu_map_idx, thread));
+ FD(evsel, cpu_map_idx, thread) = -1;
+ }
+ thread = nthreads;
+ } while (--cpu_map_idx >= start_cpu_map_idx);
+}
+
static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,
struct perf_thread_map *threads,
int start_cpu_map_idx, int end_cpu_map_idx)
@@ -2792,8 +2805,14 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,
if (err == -EMFILE && rlimit__increase_nofile(&set_rlimit))
goto retry_open;
- if (err == -EINVAL && evsel__detect_missing_features(evsel, cpu))
+ if (err == -EINVAL && evsel__detect_missing_features(evsel, cpu)) {
+ /*
+ * The restart opens the events again, so close what this
+ * attempt opened before its entries are overwritten.
+ */
+ evsel__close_fds(evsel, start_cpu_map_idx, idx, thread, nthreads);
goto fallback_missing_features;
+ }
if (evsel__precise_ip_fallback(evsel))
goto retry_open;
@@ -2803,14 +2822,7 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,
threads->err_thread = thread;
old_errno = errno;
- do {
- while (--thread >= 0) {
- if (FD(evsel, idx, thread) >= 0)
- close(FD(evsel, idx, thread));
- FD(evsel, idx, thread) = -1;
- }
- thread = nthreads;
- } while (--idx >= 0);
+ evsel__close_fds(evsel, start_cpu_map_idx, idx, thread, nthreads);
errno = old_errno;
out:
if (err)
diff --git a/tools/perf/util/hwmon_pmu.c b/tools/perf/util/hwmon_pmu.c
index b24cdc329944..7c1ac23e4132 100644
--- a/tools/perf/util/hwmon_pmu.c
+++ b/tools/perf/util/hwmon_pmu.c
@@ -803,7 +803,7 @@ int evsel__hwmon_pmu_open(struct evsel *evsel,
FD(evsel, idx, thread) = -1;
}
thread = nthreads;
- } while (--idx >= 0);
+ } while (--idx >= start_cpu_map_idx);
close(dir);
return err;
}
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index a65a4d9f8cc8..2fa9670bc51f 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -2075,9 +2075,11 @@ static PyObject *pyrf__parse_events(PyObject *self, PyObject *args)
evlist__init(&evlist, cpus, threads);
if (parse_events(&evlist, input, &err)) {
parse_events_error__print(&err, input);
+ parse_events_error__exit(&err);
PyErr_SetFromErrno(PyExc_OSError);
return NULL;
}
+ parse_events_error__exit(&err);
result = pyrf_evlist__from_evlist(&evlist);
evlist__exit(&evlist);
return result;
diff --git a/tools/perf/util/tool_pmu.c b/tools/perf/util/tool_pmu.c
index f075098488ba..601fbf6ae409 100644
--- a/tools/perf/util/tool_pmu.c
+++ b/tools/perf/util/tool_pmu.c
@@ -296,7 +296,7 @@ int evsel__tool_pmu_open(struct evsel *evsel,
FD(evsel, idx, thread) = -1;
}
thread = nthreads;
- } while (--idx >= 0);
+ } while (--idx >= start_cpu_map_idx);
errno = old_errno;
return err;
}
diff --git a/tools/testing/selftests/landlock/audit_test.c b/tools/testing/selftests/landlock/audit_test.c
index dcf63b2f822e..aa6df1b4d42e 100644
--- a/tools/testing/selftests/landlock/audit_test.c
+++ b/tools/testing/selftests/landlock/audit_test.c
@@ -31,7 +31,7 @@ static int matches_log_signal(struct __test_metadata *const _metadata,
log_match_len =
snprintf(log_match, sizeof(log_match), log_template, opid);
- if (log_match_len > sizeof(log_match))
+ if (log_match_len >= sizeof(log_match))
return -E2BIG;
return audit_match_record(audit_fd, AUDIT_LANDLOCK_ACCESS, log_match,
diff --git a/tools/testing/selftests/landlock/net_test.c b/tools/testing/selftests/landlock/net_test.c
index 8f3a780d354b..822c23425d15 100644
--- a/tools/testing/selftests/landlock/net_test.c
+++ b/tools/testing/selftests/landlock/net_test.c
@@ -2067,7 +2067,7 @@ static int matches_log_tcp(const int audit_fd, const char *const blockers,
log_match_len = snprintf(log_match, sizeof(log_match), log_template,
blockers, dir_addr, addr, dir_port);
- if (log_match_len > sizeof(log_match))
+ if (log_match_len >= sizeof(log_match))
return -E2BIG;
return audit_match_record(audit_fd, AUDIT_LANDLOCK_ACCESS, log_match,
@@ -2192,7 +2192,7 @@ static int matches_log_tcp_bound(int audit_fd, const char *const addr,
log_match_len = snprintf(log_match, sizeof(log_match), log_template,
addr, lport, addr, dport);
- if (log_match_len > sizeof(log_match))
+ if (log_match_len >= sizeof(log_match))
return -E2BIG;
return audit_match_record(audit_fd, AUDIT_LANDLOCK_ACCESS, log_match,
diff --git a/tools/testing/selftests/landlock/ptrace_test.c b/tools/testing/selftests/landlock/ptrace_test.c
index 4e356334ecb7..22133c5ae6c5 100644
--- a/tools/testing/selftests/landlock/ptrace_test.c
+++ b/tools/testing/selftests/landlock/ptrace_test.c
@@ -446,7 +446,7 @@ static int matches_log_ptrace(struct __test_metadata *const _metadata,
log_match_len =
snprintf(log_match, sizeof(log_match), log_template, opid);
- if (log_match_len > sizeof(log_match))
+ if (log_match_len >= sizeof(log_match))
return -E2BIG;
return audit_match_record(audit_fd, AUDIT_LANDLOCK_ACCESS, log_match,
diff --git a/tools/testing/selftests/net/fib_nexthops.sh b/tools/testing/selftests/net/fib_nexthops.sh
index 7e450fb9d65b..db65dd4abd3f 100755
--- a/tools/testing/selftests/net/fib_nexthops.sh
+++ b/tools/testing/selftests/net/fib_nexthops.sh
@@ -423,6 +423,14 @@ stop_ip_monitor()
return $rc
}
+fdb_entry_aged()
+{
+ local mac=$1
+ local dev=$2
+
+ ! $BRIDGE fdb get to $mac dev $dev self &> /dev/null
+}
+
check_nexthop_fdb_support()
{
$IP nexthop help 2>&1 | grep -q fdb
@@ -527,14 +535,18 @@ ipv6_fdb_grp_fcnal()
log_test $? 0 "Fdb mac add with nexthop group and static state"
run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic"
- log_test $? 255 "Fdb mac add with nexthop group and dynamic state"
+ log_test $? 0 "Fdb mac add with nexthop group and dynamic state"
+
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self dynamic extern_learn"
+ log_test $? 0 "Fdb mac add with nexthop group and extern_learn"
- run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self"
- run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic"
- log_test $? 255 "Fdb mac replace with nexthop group and dynamic state"
+ run_cmd "$IP link set dev vx10 up type vxlan ageing 1"
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:18 dev vx10 dst 2001:db8:91::11 self dynamic"
+ slowwait 20 fdb_entry_aged 02:02:00:00:00:18 vx10
+ log_test $? 0 "Fdb mac without nexthop group aged out"
- run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic"
- log_test $? 255 "Fdb mac append with nexthop group and dynamic state"
+ run_cmd "$BRIDGE fdb get to 02:02:00:00:00:16 dev vx10 self"
+ log_test $? 0 "Fdb mac with nexthop group not aged out"
run_cmd "$IP -6 ro add 2001:db8:101::1/128 nhid 66"
log_test $? 2 "Route add with fdb nexthop"
@@ -641,14 +653,18 @@ ipv4_fdb_grp_fcnal()
log_test $? 0 "Fdb mac add with nexthop group and static state"
run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic"
- log_test $? 255 "Fdb mac add with nexthop group and dynamic state"
+ log_test $? 0 "Fdb mac add with nexthop group and dynamic state"
+
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self dynamic extern_learn"
+ log_test $? 0 "Fdb mac add with nexthop group and extern_learn"
- run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self"
- run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic"
- log_test $? 255 "Fdb mac replace with nexthop group and dynamic state"
+ run_cmd "$IP link set dev vx10 up type vxlan ageing 1"
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:18 dev vx10 dst 10.0.0.3 self dynamic"
+ slowwait 20 fdb_entry_aged 02:02:00:00:00:18 vx10
+ log_test $? 0 "Fdb mac without nexthop group aged out"
- run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic"
- log_test $? 255 "Fdb mac append with nexthop group and dynamic state"
+ run_cmd "$BRIDGE fdb get to 02:02:00:00:00:16 dev vx10 self"
+ log_test $? 0 "Fdb mac with nexthop group not aged out"
run_cmd "$IP ro add 172.16.0.0/22 nhid 16"
log_test $? 2 "Route add with fdb nexthop"
diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h
index f64064ee4858..b736528815bd 100644
--- a/tools/testing/vma/vma_internal.h
+++ b/tools/testing/vma/vma_internal.h
@@ -802,6 +802,8 @@ static inline bool is_mergeable_vm_userfaultfd_ctx(struct vm_area_struct *vma,
return true;
}
+#define vma_has_uffd_without_event_remap(vma) false
+
static inline bool anon_vma_name_eq(struct anon_vma_name *anon_name1,
struct anon_vma_name *anon_name2)
{
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 2ab7bb46a371..74f7b7ee2879 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -563,15 +563,14 @@ int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args)
return __kvm_gmem_create(kvm, size, flags);
}
-int kvm_gmem_bind(struct kvm *kvm, struct kvm_memory_slot *slot,
- unsigned int fd, u64 offset)
+int kvm_gmem_prepare_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot,
+ unsigned int fd, u64 offset)
{
u64 size = slot->npages << PAGE_SHIFT;
- unsigned long start, end;
struct kvm_gmem *gmem;
struct inode *inode;
struct file *file;
- int r = -EINVAL;
+
BUILD_BUG_ON(sizeof(gpa_t) != sizeof(offset));
BUILD_BUG_ON(sizeof(gfn_t) != sizeof(slot->gmem.pgoff));
@@ -595,93 +594,93 @@ int kvm_gmem_bind(struct kvm *kvm, struct kvm_memory_slot *slot,
if (!PAGE_ALIGNED(offset) || offset + size > i_size_read(inode))
goto err;
- filemap_invalidate_lock(inode->i_mapping);
-
- start = offset >> PAGE_SHIFT;
- end = start + slot->npages;
-
- if (!xa_empty(&gmem->bindings) &&
- xa_find(&gmem->bindings, &start, end - 1, XA_PRESENT)) {
- filemap_invalidate_unlock(inode->i_mapping);
- goto err;
- }
-
/*
* memslots of flag KVM_MEM_GUEST_MEMFD are immutable to change, so
* kvm_gmem_bind() must occur on a new memslot. Because the memslot
* is not visible yet, kvm_gmem_get_pfn() is guaranteed to see the file.
*/
WRITE_ONCE(slot->gmem.file, file);
- slot->gmem.pgoff = start;
+ slot->gmem.pgoff = offset >> PAGE_SHIFT;
if (kvm_gmem_supports_mmap(inode))
slot->flags |= KVM_MEMSLOT_GMEM_ONLY;
- r = xa_err(xa_store_range(&gmem->bindings, start, end - 1, slot, GFP_KERNEL));
- if (r) {
- xa_store_range(&gmem->bindings, start, end - 1, NULL, GFP_KERNEL);
- slot->gmem.file = NULL;
- slot->gmem.pgoff = 0;
- slot->flags &= ~KVM_MEMSLOT_GMEM_ONLY;
- }
- filemap_invalidate_unlock(inode->i_mapping);
-
/*
- * Drop the reference to the file, even on success. The file pins KVM,
- * not the other way 'round. Active bindings are invalidated if the
- * file is closed before memslots are destroyed.
+ * Gift the caller a reference to the file. The reference will be
+ * dropped after bindings are established, or if installing the new
+ * memslot ultimately fails.
*/
+ return 0;
+
err:
fput(file);
- return r;
+ return -EINVAL;
}
-static void __kvm_gmem_unbind(struct kvm_memory_slot *slot, struct kvm_gmem *gmem)
+int kvm_gmem_commit_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot)
{
- unsigned long start = slot->gmem.pgoff;
- unsigned long end = start + slot->npages;
+ struct kvm_gmem *gmem = slot->gmem.file->private_data;
+ struct inode *inode = file_inode(slot->gmem.file);
+ unsigned long start, end;
+ int r;
- xa_store_range(&gmem->bindings, start, end - 1, NULL, GFP_KERNEL);
+ if (WARN_ON_ONCE(slot->gmem.file->f_op != &kvm_gmem_fops))
+ return -EIO;
- /*
- * synchronize_srcu(&kvm->srcu) ensured that kvm_gmem_get_pfn()
- * cannot see this memslot.
- */
- WRITE_ONCE(slot->gmem.file, NULL);
+ filemap_invalidate_lock(inode->i_mapping);
+
+ start = slot->gmem.pgoff;
+ end = start + slot->npages;
+
+ if (!xa_empty(&gmem->bindings) &&
+ xa_find(&gmem->bindings, &start, end - 1, XA_PRESENT)) {
+ filemap_invalidate_unlock(inode->i_mapping);
+ return -EEXIST;
+ }
+
+ r = xa_err(xa_store_range(&gmem->bindings, start, end - 1, slot, GFP_KERNEL));
+ if (r)
+ xa_store_range(&gmem->bindings, start, end - 1, NULL, GFP_KERNEL);
+
+ filemap_invalidate_unlock(inode->i_mapping);
+
+ return r;
}
void kvm_gmem_unbind(struct kvm_memory_slot *slot)
{
- struct file *file;
+ struct file *file = slot->gmem.file;
+ unsigned long start = slot->gmem.pgoff;
+ unsigned long end = start + slot->npages;
+ struct kvm_gmem *gmem;
/*
* Nothing to do if the underlying file was _already_ closed, as
* kvm_gmem_release() invalidates and nullifies all bindings.
*/
- if (!slot->gmem.file)
+ if (!file)
return;
- file = kvm_gmem_get_file(slot);
-
/*
* However, if the file is _being_ closed, then the bindings need to be
* removed as kvm_gmem_release() might not run until after the memslot
- * is freed. Note, modifying the bindings is safe even though the file
- * is dying as kvm_gmem_release() nullifies slot->gmem.file under
- * slots_lock, and only puts its reference to KVM after destroying all
- * bindings. I.e. reaching this point means kvm_gmem_release() hasn't
- * yet destroyed the bindings or freed the gmem_file, and can't do so
- * until the caller drops slots_lock.
+ * is freed. Modifying the bindings is safe even if the file is dying
+ * as kvm_gmem_release() nullifies slot->gmem.file under slots_lock,
+ * and only puts its reference to KVM after destroying all bindings.
+ * I.e. reaching this point means kvm_gmem_release() hasn't destroyed
+ * the bindings or freed the gmem_file and can't do so until the caller
+ * drops slots_lock, so there's no need to verify the file is live.
*/
- if (!file) {
- __kvm_gmem_unbind(slot, slot->gmem.file->private_data);
- return;
- }
+ gmem = file->private_data;
filemap_invalidate_lock(file->f_mapping);
- __kvm_gmem_unbind(slot, file->private_data);
- filemap_invalidate_unlock(file->f_mapping);
+ xa_store_range(&gmem->bindings, start, end - 1, NULL, GFP_KERNEL);
- fput(file);
+ /*
+ * synchronize_srcu(&kvm->srcu) ensured that kvm_gmem_get_pfn()
+ * cannot see this memslot.
+ */
+ WRITE_ONCE(slot->gmem.file, NULL);
+ filemap_invalidate_unlock(file->f_mapping);
}
/* Returns a locked folio on success. */
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 23e7d1bb80b0..723463fe6038 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -1748,6 +1748,12 @@ static void kvm_commit_memory_region(struct kvm *kvm,
kvm_free_memslot(kvm, old);
break;
case KVM_MR_MOVE:
+ /*
+ * Moving a guest_memfd memslot isn't supported, and will never
+ * be supported.
+ */
+ WARN_ON_ONCE(old->flags & KVM_MEM_GUEST_MEMFD);
+ fallthrough;
case KVM_MR_FLAGS_ONLY:
/*
* Free the dirty bitmap as needed; the below check encompasses
@@ -1756,6 +1762,15 @@ static void kvm_commit_memory_region(struct kvm *kvm,
if (old->dirty_bitmap && !new->dirty_bitmap)
kvm_destroy_dirty_bitmap(old);
+ /*
+ * TODO: Drop the WARN and do the unbind() call only for MOVE
+ * when dirty logging guest_memfd memslots is supported. Until
+ * then, flags-only changes on guest_memfd slots should also be
+ * impossible; unbind the old memslot for defense-in-depth.
+ */
+ if (WARN_ON_ONCE(old->flags & KVM_MEM_GUEST_MEMFD))
+ kvm_gmem_unbind(old);
+
/*
* The final quirk. Free the detached, old slot, but only its
* memory, not any metadata. Metadata, including arch specific
@@ -1940,6 +1955,15 @@ static int kvm_set_memslot(struct kvm *kvm,
if (r)
goto err;
+ if (change == KVM_MR_CREATE && (new->flags & KVM_MEM_GUEST_MEMFD)) {
+ r = kvm_gmem_commit_memory_region(kvm, new);
+ if (r) {
+ kvm_arch_free_memslot(kvm, new);
+ kvm_destroy_dirty_bitmap(new);
+ goto err;
+ }
+ }
+
/*
* For DELETE and MOVE, the working slot is now active as the INVALID
* version of the old slot. MOVE is particularly special as it reuses
@@ -2112,21 +2136,30 @@ static int kvm_set_memory_region(struct kvm *kvm,
new->npages = npages;
new->flags = mem->flags;
new->userspace_addr = mem->userspace_addr;
- if (mem->flags & KVM_MEM_GUEST_MEMFD) {
- r = kvm_gmem_bind(kvm, new, mem->guest_memfd, mem->guest_memfd_offset);
+ if (change == KVM_MR_CREATE && (mem->flags & KVM_MEM_GUEST_MEMFD)) {
+ r = kvm_gmem_prepare_memory_region(kvm, new, mem->guest_memfd,
+ mem->guest_memfd_offset);
if (r)
goto out;
}
r = kvm_set_memslot(kvm, old, new, change);
+
+ /*
+ * Drop the reference to the gmem file, even on success. The file pins
+ * KVM, not the other way 'round. Active bindings are invalidated if
+ * the file is closed before memslots are destroyed.
+ */
+#ifdef CONFIG_KVM_GUEST_MEMFD
+ if (change == KVM_MR_CREATE && (mem->flags & KVM_MEM_GUEST_MEMFD))
+ fput(new->gmem.file);
+#endif
+
if (r)
- goto out_unbind;
+ goto out;
return 0;
-out_unbind:
- if (mem->flags & KVM_MEM_GUEST_MEMFD)
- kvm_gmem_unbind(new);
out:
kfree(new);
return r;
diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h
index 0e68675a8a63..9db0a64d9d54 100644
--- a/virt/kvm/kvm_mm.h
+++ b/virt/kvm/kvm_mm.h
@@ -73,8 +73,9 @@ static inline void gfn_to_pfn_cache_invalidate_start(struct kvm *kvm,
#ifdef CONFIG_KVM_GUEST_MEMFD
void kvm_gmem_init(struct module *module);
int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args);
-int kvm_gmem_bind(struct kvm *kvm, struct kvm_memory_slot *slot,
- unsigned int fd, u64 offset);
+int kvm_gmem_prepare_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot,
+ unsigned int fd, u64 offset);
+int kvm_gmem_commit_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot);
void kvm_gmem_unbind(struct kvm_memory_slot *slot);
#else
static inline void kvm_gmem_init(struct module *module)
@@ -82,9 +83,16 @@ static inline void kvm_gmem_init(struct module *module)
}
-static inline int kvm_gmem_bind(struct kvm *kvm,
- struct kvm_memory_slot *slot,
- unsigned int fd, u64 offset)
+static inline int kvm_gmem_prepare_memory_region(struct kvm *kvm,
+ struct kvm_memory_slot *slot,
+ unsigned int fd, u64 offset)
+{
+ WARN_ON_ONCE(1);
+ return -EIO;
+}
+
+static inline int kvm_gmem_commit_memory_region(struct kvm *kvm,
+ struct kvm_memory_slot *slot)
{
WARN_ON_ONCE(1);
return -EIO;
^ permalink raw reply related [flat|nested] 2+ messages in thread