From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org, akpm@linux-foundation.org,
torvalds@linux-foundation.org, stable@vger.kernel.org
Cc: lwn@lwn.net, jslaby@suse.cz,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Subject: Re: Linux 6.12.95
Date: Sat, 4 Jul 2026 13:59:01 +0200 [thread overview]
Message-ID: <2026070401-issue-antibody-a2a6@gregkh> (raw)
In-Reply-To: <2026070401-polygon-mobilize-50fa@gregkh>
diff --git a/Documentation/networking/bonding.rst b/Documentation/networking/bonding.rst
index e774b48de9f5..94ef39457a62 100644
--- a/Documentation/networking/bonding.rst
+++ b/Documentation/networking/bonding.rst
@@ -193,6 +193,15 @@ ad_actor_sys_prio
This parameter has effect only in 802.3ad mode and is available through
SysFs interface.
+actor_port_prio
+
+ In an AD system, this specifies the port priority. The allowed range
+ is 1 - 65535. If the value is not specified, it takes 255 as the
+ default value.
+
+ This parameter has effect only in 802.3ad mode and is available through
+ netlink interface.
+
ad_actor_system
In an AD system, this specifies the mac-address for the actor in
@@ -562,6 +571,12 @@ lacp_rate
The default is slow.
+broadcast_neighbor
+
+ Option specifying whether to broadcast ARP/ND packets to all
+ active slaves. This option has no effect in modes other than
+ 802.3ad mode. The default is off (0).
+
max_bonds
Specifies the number of bonding devices to create for this
diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst
index e4be1378ba26..ff578a770598 100644
--- a/Documentation/userspace-api/ioctl/ioctl-number.rst
+++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
@@ -66,330 +66,329 @@ This table lists ioctls visible from user land for Linux/x86. It contains
most drivers up to 2.6.31, but I know I am missing some. There has been
no attempt to list non-X86 architectures or ioctls from drivers/staging/.
-==== ===== ======================================================= ================================================================
-Code Seq# Include File Comments
+==== ===== ========================================================= ================================================================
+Code Seq# Include File Comments
(hex)
-==== ===== ======================================================= ================================================================
-0x00 00-1F linux/fs.h conflict!
-0x00 00-1F scsi/scsi_ioctl.h conflict!
-0x00 00-1F linux/fb.h conflict!
-0x00 00-1F linux/wavefront.h conflict!
+==== ===== ========================================================= ================================================================
+0x00 00-1F linux/fs.h conflict!
+0x00 00-1F scsi/scsi_ioctl.h conflict!
+0x00 00-1F linux/fb.h conflict!
+0x00 00-1F linux/wavefront.h conflict!
0x02 all linux/fd.h
0x03 all linux/hdreg.h
-0x04 D2-DC linux/umsdos_fs.h Dead since 2.6.11, but don't reuse these.
+0x04 D2-DC linux/umsdos_fs.h Dead since 2.6.11, but don't reuse these.
0x06 all linux/lp.h
0x07 9F-D0 linux/vmw_vmci_defs.h, uapi/linux/vm_sockets.h
0x09 all linux/raid/md_u.h
0x10 00-0F drivers/char/s390/vmcp.h
0x10 10-1F arch/s390/include/uapi/sclp_ctl.h
0x10 20-2F arch/s390/include/uapi/asm/hypfs.h
-0x12 all linux/fs.h BLK* ioctls
+0x12 all linux/fs.h BLK* ioctls
linux/blkpg.h
-0x15 all linux/fs.h FS_IOC_* ioctls
-0x1b all InfiniBand Subsystem
- <http://infiniband.sourceforge.net/>
+0x15 all linux/fs.h FS_IOC_* ioctls
+0x1b all InfiniBand Subsystem
+ <http://infiniband.sourceforge.net/>
0x20 all drivers/cdrom/cm206.h
0x22 all scsi/sg.h
-0x3E 00-0F linux/counter.h <mailto:linux-iio@vger.kernel.org>
+0x3E 00-0F linux/counter.h <mailto:linux-iio@vger.kernel.org>
'!' 00-1F uapi/linux/seccomp.h
-'#' 00-3F IEEE 1394 Subsystem
- Block for the entire subsystem
+'#' 00-3F IEEE 1394 Subsystem
+ Block for the entire subsystem
'$' 00-0F linux/perf_counter.h, linux/perf_event.h
-'%' 00-0F include/uapi/linux/stm.h System Trace Module subsystem
- <mailto:alexander.shishkin@linux.intel.com>
+'%' 00-0F include/uapi/linux/stm.h System Trace Module subsystem
+ <mailto:alexander.shishkin@linux.intel.com>
'&' 00-07 drivers/firewire/nosy-user.h
-'*' 00-1F uapi/linux/user_events.h User Events Subsystem
- <mailto:linux-trace-kernel@vger.kernel.org>
-'1' 00-1F linux/timepps.h PPS kit from Ulrich Windl
- <ftp://ftp.de.kernel.org/pub/linux/daemons/ntp/PPS/>
+'*' 00-1F uapi/linux/user_events.h User Events Subsystem
+ <mailto:linux-trace-kernel@vger.kernel.org>
+'1' 00-1F linux/timepps.h PPS kit from Ulrich Windl
+ <ftp://ftp.de.kernel.org/pub/linux/daemons/ntp/PPS/>
'2' 01-04 linux/i2o.h
-'3' 00-0F drivers/s390/char/raw3270.h conflict!
-'3' 00-1F linux/suspend_ioctls.h, conflict!
+'3' 00-0F drivers/s390/char/raw3270.h conflict!
+'3' 00-1F linux/suspend_ioctls.h, conflict!
kernel/power/user.c
-'8' all SNP8023 advanced NIC card
- <mailto:mcr@solidum.com>
+'8' all SNP8023 advanced NIC card
+ <mailto:mcr@solidum.com>
';' 64-7F linux/vfio.h
';' 80-FF linux/iommufd.h
-'=' 00-3f uapi/linux/ptp_clock.h <mailto:richardcochran@gmail.com>
-'@' 00-0F linux/radeonfb.h conflict!
-'@' 00-0F drivers/video/aty/aty128fb.c conflict!
-'A' 00-1F linux/apm_bios.h conflict!
-'A' 00-0F linux/agpgart.h, conflict!
+'=' 00-3f uapi/linux/ptp_clock.h <mailto:richardcochran@gmail.com>
+'@' 00-0F linux/radeonfb.h conflict!
+'@' 00-0F drivers/video/aty/aty128fb.c conflict!
+'A' 00-1F linux/apm_bios.h conflict!
+'A' 00-0F linux/agpgart.h, conflict!
drivers/char/agp/compat_ioctl.h
-'A' 00-7F sound/asound.h conflict!
-'B' 00-1F linux/cciss_ioctl.h conflict!
-'B' 00-0F include/linux/pmu.h conflict!
-'B' C0-FF advanced bbus <mailto:maassen@uni-freiburg.de>
-'B' 00-0F xen/xenbus_dev.h conflict!
-'C' all linux/soundcard.h conflict!
-'C' 01-2F linux/capi.h conflict!
-'C' F0-FF drivers/net/wan/cosa.h conflict!
+'A' 00-7F sound/asound.h conflict!
+'B' 00-1F linux/cciss_ioctl.h conflict!
+'B' 00-0F include/linux/pmu.h conflict!
+'B' C0-FF advanced bbus <mailto:maassen@uni-freiburg.de>
+'B' 00-0F xen/xenbus_dev.h conflict!
+'C' all linux/soundcard.h conflict!
+'C' 01-2F linux/capi.h conflict!
+'C' F0-FF drivers/net/wan/cosa.h conflict!
'D' all arch/s390/include/asm/dasd.h
-'D' 40-5F drivers/scsi/dpt/dtpi_ioctl.h Dead since 2022
+'D' 40-5F drivers/scsi/dpt/dtpi_ioctl.h Dead since 2022
'D' 05 drivers/scsi/pmcraid.h
-'E' all linux/input.h conflict!
-'E' 00-0F xen/evtchn.h conflict!
-'F' all linux/fb.h conflict!
-'F' 01-02 drivers/scsi/pmcraid.h conflict!
-'F' 20 drivers/video/fsl-diu-fb.h conflict!
-'F' 20 linux/ivtvfb.h conflict!
-'F' 20 linux/matroxfb.h conflict!
-'F' 20 drivers/video/aty/atyfb_base.c conflict!
-'F' 00-0F video/da8xx-fb.h conflict!
-'F' 80-8F linux/arcfb.h conflict!
-'F' DD video/sstfb.h conflict!
-'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict!
-'G' 00-0F xen/gntalloc.h, xen/gntdev.h conflict!
-'H' 00-7F linux/hiddev.h conflict!
-'H' 00-0F linux/hidraw.h conflict!
-'H' 01 linux/mei.h conflict!
-'H' 02 linux/mei.h conflict!
-'H' 03 linux/mei.h conflict!
-'H' 00-0F sound/asound.h conflict!
-'H' 20-40 sound/asound_fm.h conflict!
-'H' 80-8F sound/sfnt_info.h conflict!
-'H' 10-8F sound/emu10k1.h conflict!
-'H' 10-1F sound/sb16_csp.h conflict!
-'H' 10-1F sound/hda_hwdep.h conflict!
-'H' 40-4F sound/hdspm.h conflict!
-'H' 40-4F sound/hdsp.h conflict!
+'E' all linux/input.h conflict!
+'E' 00-0F xen/evtchn.h conflict!
+'F' all linux/fb.h conflict!
+'F' 01-02 drivers/scsi/pmcraid.h conflict!
+'F' 20 drivers/video/fsl-diu-fb.h conflict!
+'F' 20 linux/ivtvfb.h conflict!
+'F' 20 linux/matroxfb.h conflict!
+'F' 20 drivers/video/aty/atyfb_base.c conflict!
+'F' 00-0F video/da8xx-fb.h conflict!
+'F' 80-8F linux/arcfb.h conflict!
+'F' DD video/sstfb.h conflict!
+'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict!
+'G' 00-0F xen/gntalloc.h, xen/gntdev.h conflict!
+'H' 00-7F linux/hiddev.h conflict!
+'H' 00-0F linux/hidraw.h conflict!
+'H' 01 linux/mei.h conflict!
+'H' 02 linux/mei.h conflict!
+'H' 03 linux/mei.h conflict!
+'H' 00-0F sound/asound.h conflict!
+'H' 20-40 sound/asound_fm.h conflict!
+'H' 80-8F sound/sfnt_info.h conflict!
+'H' 10-8F sound/emu10k1.h conflict!
+'H' 10-1F sound/sb16_csp.h conflict!
+'H' 10-1F sound/hda_hwdep.h conflict!
+'H' 40-4F sound/hdspm.h conflict!
+'H' 40-4F sound/hdsp.h conflict!
'H' 90 sound/usb/usx2y/usb_stream.h
-'H' 00-0F uapi/misc/habanalabs.h conflict!
+'H' 00-0F uapi/misc/habanalabs.h conflict!
'H' A0 uapi/linux/usb/cdc-wdm.h
-'H' C0-F0 net/bluetooth/hci.h conflict!
-'H' C0-DF net/bluetooth/hidp/hidp.h conflict!
-'H' C0-DF net/bluetooth/cmtp/cmtp.h conflict!
-'H' C0-DF net/bluetooth/bnep/bnep.h conflict!
-'H' F1 linux/hid-roccat.h <mailto:erazor_de@users.sourceforge.net>
+'H' C0-F0 net/bluetooth/hci.h conflict!
+'H' C0-DF net/bluetooth/hidp/hidp.h conflict!
+'H' C0-DF net/bluetooth/cmtp/cmtp.h conflict!
+'H' C0-DF net/bluetooth/bnep/bnep.h conflict!
+'H' F1 linux/hid-roccat.h <mailto:erazor_de@users.sourceforge.net>
'H' F8-FA sound/firewire.h
-'I' all linux/isdn.h conflict!
-'I' 00-0F drivers/isdn/divert/isdn_divert.h conflict!
-'I' 40-4F linux/mISDNif.h conflict!
+'I' all linux/isdn.h conflict!
+'I' 00-0F drivers/isdn/divert/isdn_divert.h conflict!
+'I' 40-4F linux/mISDNif.h conflict!
'K' all linux/kd.h
-'L' 00-1F linux/loop.h conflict!
-'L' 10-1F drivers/scsi/mpt3sas/mpt3sas_ctl.h conflict!
-'L' E0-FF linux/ppdd.h encrypted disk device driver
- <http://linux01.gwdg.de/~alatham/ppdd.html>
-'M' all linux/soundcard.h conflict!
-'M' 01-16 mtd/mtd-abi.h conflict!
+'L' 00-1F linux/loop.h conflict!
+'L' 10-1F drivers/scsi/mpt3sas/mpt3sas_ctl.h conflict!
+'L' E0-FF linux/ppdd.h encrypted disk device driver
+ <http://linux01.gwdg.de/~alatham/ppdd.html>
+'M' all linux/soundcard.h conflict!
+'M' 01-16 mtd/mtd-abi.h conflict!
and drivers/mtd/mtdchar.c
'M' 01-03 drivers/scsi/megaraid/megaraid_sas.h
-'M' 00-0F drivers/video/fsl-diu-fb.h conflict!
+'M' 00-0F drivers/video/fsl-diu-fb.h conflict!
'N' 00-1F drivers/usb/scanner.h
'N' 40-7F drivers/block/nvme.c
-'N' 80-8F uapi/linux/ntsync.h NT synchronization primitives
- <mailto:wine-devel@winehq.org>
-'O' 00-06 mtd/ubi-user.h UBI
-'P' all linux/soundcard.h conflict!
-'P' 60-6F sound/sscape_ioctl.h conflict!
-'P' 00-0F drivers/usb/class/usblp.c conflict!
-'P' 01-09 drivers/misc/pci_endpoint_test.c conflict!
-'P' 00-0F xen/privcmd.h conflict!
-'P' 00-05 linux/tps6594_pfsm.h conflict!
+'N' 80-8F uapi/linux/ntsync.h NT synchronization primitives
+ <mailto:wine-devel@winehq.org>
+'O' 00-06 mtd/ubi-user.h UBI
+'P' all linux/soundcard.h conflict!
+'P' 60-6F sound/sscape_ioctl.h conflict!
+'P' 00-0F drivers/usb/class/usblp.c conflict!
+'P' 01-09 drivers/misc/pci_endpoint_test.c conflict!
+'P' 00-0F xen/privcmd.h conflict!
+'P' 00-05 linux/tps6594_pfsm.h conflict!
'Q' all linux/soundcard.h
-'R' 00-1F linux/random.h conflict!
-'R' 01 linux/rfkill.h conflict!
+'R' 00-1F linux/random.h conflict!
+'R' 01 linux/rfkill.h conflict!
'R' 20-2F linux/trace_mmap.h
'R' C0-DF net/bluetooth/rfcomm.h
'R' E0 uapi/linux/fsl_mc.h
-'S' all linux/cdrom.h conflict!
-'S' 80-81 scsi/scsi_ioctl.h conflict!
-'S' 82-FF scsi/scsi.h conflict!
-'S' 00-7F sound/asequencer.h conflict!
-'T' all linux/soundcard.h conflict!
-'T' 00-AF sound/asound.h conflict!
-'T' all arch/x86/include/asm/ioctls.h conflict!
-'T' C0-DF linux/if_tun.h conflict!
-'U' all sound/asound.h conflict!
-'U' 00-CF linux/uinput.h conflict!
+'S' all linux/cdrom.h conflict!
+'S' 80-81 scsi/scsi_ioctl.h conflict!
+'S' 82-FF scsi/scsi.h conflict!
+'S' 00-7F sound/asequencer.h conflict!
+'T' all linux/soundcard.h conflict!
+'T' 00-AF sound/asound.h conflict!
+'T' all arch/x86/include/asm/ioctls.h conflict!
+'T' C0-DF linux/if_tun.h conflict!
+'U' all sound/asound.h conflict!
+'U' 00-CF linux/uinput.h conflict!
'U' 00-EF linux/usbdevice_fs.h
'U' C0-CF drivers/bluetooth/hci_uart.h
-'V' all linux/vt.h conflict!
-'V' all linux/videodev2.h conflict!
-'V' C0 linux/ivtvfb.h conflict!
-'V' C0 linux/ivtv.h conflict!
-'V' C0 media/si4713.h conflict!
-'W' 00-1F linux/watchdog.h conflict!
-'W' 00-1F linux/wanrouter.h conflict! (pre 3.9)
-'W' 00-3F sound/asound.h conflict!
+'V' all linux/vt.h conflict!
+'V' all linux/videodev2.h conflict!
+'V' C0 linux/ivtvfb.h conflict!
+'V' C0 linux/ivtv.h conflict!
+'V' C0 media/si4713.h conflict!
+'W' 00-1F linux/watchdog.h conflict!
+'W' 00-1F linux/wanrouter.h conflict! (pre 3.9)
+'W' 00-3F sound/asound.h conflict!
'W' 40-5F drivers/pci/switch/switchtec.c
'W' 60-61 linux/watch_queue.h
-'X' all fs/xfs/xfs_fs.h, conflict!
+'X' all fs/xfs/xfs_fs.h, conflict!
fs/xfs/linux-2.6/xfs_ioctl32.h,
include/linux/falloc.h,
linux/fs.h,
-'X' all fs/ocfs2/ocfs_fs.h conflict!
-'X' 01 linux/pktcdvd.h conflict!
+'X' all fs/ocfs2/ocfs_fs.h conflict!
+'X' 01 linux/pktcdvd.h conflict!
'Z' 14-15 drivers/message/fusion/mptctl.h
-'[' 00-3F linux/usb/tmc.h USB Test and Measurement Devices
- <mailto:gregkh@linuxfoundation.org>
-'a' all linux/atm*.h, linux/sonet.h ATM on linux
- <http://lrcwww.epfl.ch/>
-'a' 00-0F drivers/crypto/qat/qat_common/adf_cfg_common.h conflict! qat driver
-'b' 00-FF conflict! bit3 vme host bridge
- <mailto:natalia@nikhefk.nikhef.nl>
-'b' 00-0F linux/dma-buf.h conflict!
-'c' 00-7F linux/comstats.h conflict!
-'c' 00-7F linux/coda.h conflict!
-'c' 00-1F linux/chio.h conflict!
-'c' 80-9F arch/s390/include/asm/chsc.h conflict!
+'[' 00-3F linux/usb/tmc.h USB Test and Measurement Devices
+ <mailto:gregkh@linuxfoundation.org>
+'a' all linux/atm*.h, linux/sonet.h ATM on linux
+ <http://lrcwww.epfl.ch/>
+'b' 00-FF conflict! bit3 vme host bridge
+ <mailto:natalia@nikhefk.nikhef.nl>
+'b' 00-0F linux/dma-buf.h conflict!
+'c' 00-7F linux/comstats.h conflict!
+'c' 00-7F linux/coda.h conflict!
+'c' 00-1F linux/chio.h conflict!
+'c' 80-9F arch/s390/include/asm/chsc.h conflict!
'c' A0-AF arch/x86/include/asm/msr.h conflict!
-'d' 00-FF linux/char/drm/drm.h conflict!
-'d' 02-40 pcmcia/ds.h conflict!
+'d' 00-FF linux/char/drm/drm.h conflict!
+'d' 02-40 pcmcia/ds.h conflict!
'd' F0-FF linux/digi1.h
-'e' all linux/digi1.h conflict!
-'f' 00-1F linux/ext2_fs.h conflict!
-'f' 00-1F linux/ext3_fs.h conflict!
-'f' 00-0F fs/jfs/jfs_dinode.h conflict!
-'f' 00-0F fs/ext4/ext4.h conflict!
-'f' 00-0F linux/fs.h conflict!
-'f' 00-0F fs/ocfs2/ocfs2_fs.h conflict!
+'e' all linux/digi1.h conflict!
+'f' 00-1F linux/ext2_fs.h conflict!
+'f' 00-1F linux/ext3_fs.h conflict!
+'f' 00-0F fs/jfs/jfs_dinode.h conflict!
+'f' 00-0F fs/ext4/ext4.h conflict!
+'f' 00-0F linux/fs.h conflict!
+'f' 00-0F fs/ocfs2/ocfs2_fs.h conflict!
'f' 13-27 linux/fscrypt.h
'f' 81-8F linux/fsverity.h
'g' 00-0F linux/usb/gadgetfs.h
'g' 20-2F linux/usb/g_printer.h
-'h' 00-7F conflict! Charon filesystem
- <mailto:zapman@interlan.net>
-'h' 00-1F linux/hpet.h conflict!
+'h' 00-7F conflict! Charon filesystem
+ <mailto:zapman@interlan.net>
+'h' 00-1F linux/hpet.h conflict!
'h' 80-8F fs/hfsplus/ioctl.c
-'i' 00-3F linux/i2o-dev.h conflict!
-'i' 0B-1F linux/ipmi.h conflict!
+'i' 00-3F linux/i2o-dev.h conflict!
+'i' 0B-1F linux/ipmi.h conflict!
'i' 80-8F linux/i8k.h
-'i' 90-9F `linux/iio/*.h` IIO
+'i' 90-9F `linux/iio/*.h` IIO
'j' 00-3F linux/joystick.h
-'k' 00-0F linux/spi/spidev.h conflict!
-'k' 00-05 video/kyro.h conflict!
-'k' 10-17 linux/hsi/hsi_char.h HSI character device
-'l' 00-3F linux/tcfs_fs.h transparent cryptographic file system
- <http://web.archive.org/web/%2A/http://mikonos.dia.unisa.it/tcfs>
-'l' 40-7F linux/udf_fs_i.h in development:
- <https://github.com/pali/udftools>
-'m' 00-09 linux/mmtimer.h conflict!
-'m' all linux/mtio.h conflict!
-'m' all linux/soundcard.h conflict!
-'m' all linux/synclink.h conflict!
-'m' 00-19 drivers/message/fusion/mptctl.h conflict!
-'m' 00 drivers/scsi/megaraid/megaraid_ioctl.h conflict!
+'k' 00-0F linux/spi/spidev.h conflict!
+'k' 00-05 video/kyro.h conflict!
+'k' 10-17 linux/hsi/hsi_char.h HSI character device
+'l' 00-3F linux/tcfs_fs.h transparent cryptographic file system
+ <http://web.archive.org/web/%2A/http://mikonos.dia.unisa.it/tcfs>
+'l' 40-7F linux/udf_fs_i.h in development:
+ <https://github.com/pali/udftools>
+'m' 00-09 linux/mmtimer.h conflict!
+'m' all linux/mtio.h conflict!
+'m' all linux/soundcard.h conflict!
+'m' all linux/synclink.h conflict!
+'m' 00-19 drivers/message/fusion/mptctl.h conflict!
+'m' 00 drivers/scsi/megaraid/megaraid_ioctl.h conflict!
'n' 00-7F linux/ncp_fs.h and fs/ncpfs/ioctl.c
-'n' 80-8F uapi/linux/nilfs2_api.h NILFS2
-'n' E0-FF linux/matroxfb.h matroxfb
-'o' 00-1F fs/ocfs2/ocfs2_fs.h OCFS2
-'o' 00-03 mtd/ubi-user.h conflict! (OCFS2 and UBI overlaps)
-'o' 40-41 mtd/ubi-user.h UBI
-'o' 01-A1 `linux/dvb/*.h` DVB
-'p' 00-0F linux/phantom.h conflict! (OpenHaptics needs this)
-'p' 00-1F linux/rtc.h conflict!
+'n' 80-8F uapi/linux/nilfs2_api.h NILFS2
+'n' E0-FF linux/matroxfb.h matroxfb
+'o' 00-1F fs/ocfs2/ocfs2_fs.h OCFS2
+'o' 00-03 mtd/ubi-user.h conflict! (OCFS2 and UBI overlaps)
+'o' 40-41 mtd/ubi-user.h UBI
+'o' 01-A1 `linux/dvb/*.h` DVB
+'p' 00-0F linux/phantom.h conflict! (OpenHaptics needs this)
+'p' 00-1F linux/rtc.h conflict!
'p' 40-7F linux/nvram.h
-'p' 80-9F linux/ppdev.h user-space parport
- <mailto:tim@cyberelk.net>
-'p' A1-A5 linux/pps.h LinuxPPS
- <mailto:giometti@linux.it>
+'p' 80-9F linux/ppdev.h user-space parport
+ <mailto:tim@cyberelk.net>
+'p' A1-A5 linux/pps.h LinuxPPS
+ <mailto:giometti@linux.it>
'q' 00-1F linux/serio.h
-'q' 80-FF linux/telephony.h Internet PhoneJACK, Internet LineJACK
- linux/ixjuser.h <http://web.archive.org/web/%2A/http://www.quicknet.net>
+'q' 80-FF linux/telephony.h Internet PhoneJACK, Internet LineJACK
+ linux/ixjuser.h <http://web.archive.org/web/%2A/http://www.quicknet.net>
'r' 00-1F linux/msdos_fs.h and fs/fat/dir.c
's' all linux/cdk.h
't' 00-7F linux/ppp-ioctl.h
't' 80-8F linux/isdn_ppp.h
-'t' 90-91 linux/toshiba.h toshiba and toshiba_acpi SMM
-'u' 00-1F linux/smb_fs.h gone
-'u' 00-2F linux/ublk_cmd.h conflict!
-'u' 20-3F linux/uvcvideo.h USB video class host driver
-'u' 40-4f linux/udmabuf.h userspace dma-buf misc device
-'v' 00-1F linux/ext2_fs.h conflict!
-'v' 00-1F linux/fs.h conflict!
-'v' 00-0F linux/sonypi.h conflict!
-'v' 00-0F media/v4l2-subdev.h conflict!
-'v' 20-27 arch/powerpc/include/uapi/asm/vas-api.h VAS API
-'v' C0-FF linux/meye.h conflict!
-'w' all CERN SCI driver
-'y' 00-1F packet based user level communications
- <mailto:zapman@interlan.net>
-'z' 00-3F CAN bus card conflict!
- <mailto:hdstich@connectu.ulm.circular.de>
-'z' 40-7F CAN bus card conflict!
- <mailto:oe@port.de>
-'z' 10-4F drivers/s390/crypto/zcrypt_api.h conflict!
+'t' 90-91 linux/toshiba.h toshiba and toshiba_acpi SMM
+'u' 00-1F linux/smb_fs.h gone
+'u' 00-2F linux/ublk_cmd.h conflict!
+'u' 20-3F linux/uvcvideo.h USB video class host driver
+'u' 40-4f linux/udmabuf.h userspace dma-buf misc device
+'v' 00-1F linux/ext2_fs.h conflict!
+'v' 00-1F linux/fs.h conflict!
+'v' 00-0F linux/sonypi.h conflict!
+'v' 00-0F media/v4l2-subdev.h conflict!
+'v' 20-27 arch/powerpc/include/uapi/asm/vas-api.h VAS API
+'v' C0-FF linux/meye.h conflict!
+'w' all CERN SCI driver
+'y' 00-1F packet based user level communications
+ <mailto:zapman@interlan.net>
+'z' 00-3F CAN bus card conflict!
+ <mailto:hdstich@connectu.ulm.circular.de>
+'z' 40-7F CAN bus card conflict!
+ <mailto:oe@port.de>
+'z' 10-4F drivers/s390/crypto/zcrypt_api.h conflict!
'|' 00-7F linux/media.h
0x80 00-1F linux/fb.h
0x81 00-1F linux/vduse.h
0x89 00-06 arch/x86/include/asm/sockios.h
0x89 0B-DF linux/sockios.h
-0x89 E0-EF linux/sockios.h SIOCPROTOPRIVATE range
-0x89 F0-FF linux/sockios.h SIOCDEVPRIVATE range
+0x89 E0-EF linux/sockios.h SIOCPROTOPRIVATE range
+0x89 F0-FF linux/sockios.h SIOCDEVPRIVATE range
0x8A 00-1F linux/eventpoll.h
0x8B all linux/wireless.h
-0x8C 00-3F WiNRADiO driver
- <http://www.winradio.com.au/>
+0x8C 00-3F WiNRADiO driver
+ <http://www.winradio.com.au/>
0x90 00 drivers/cdrom/sbpcd.h
0x92 00-0F drivers/usb/mon/mon_bin.c
0x93 60-7F linux/auto_fs.h
-0x94 all fs/btrfs/ioctl.h Btrfs filesystem
- and linux/fs.h some lifted to vfs/generic
-0x97 00-7F fs/ceph/ioctl.h Ceph file system
-0x99 00-0F 537-Addinboard driver
- <mailto:buk@buks.ipn.de>
-0xA0 all linux/sdp/sdp.h Industrial Device Project
- <mailto:kenji@bitgate.com>
-0xA1 0 linux/vtpm_proxy.h TPM Emulator Proxy Driver
-0xA2 all uapi/linux/acrn.h ACRN hypervisor
-0xA3 80-8F Port ACL in development:
- <mailto:tlewis@mindspring.com>
+0x94 all fs/btrfs/ioctl.h Btrfs filesystem
+ and linux/fs.h some lifted to vfs/generic
+0x97 00-7F fs/ceph/ioctl.h Ceph file system
+0x99 00-0F 537-Addinboard driver
+ <mailto:buk@buks.ipn.de>
+0xA0 all linux/sdp/sdp.h Industrial Device Project
+ <mailto:kenji@bitgate.com>
+0xA1 0 linux/vtpm_proxy.h TPM Emulator Proxy Driver
+0xA2 all uapi/linux/acrn.h ACRN hypervisor
+0xA3 80-8F Port ACL in development:
+ <mailto:tlewis@mindspring.com>
0xA3 90-9F linux/dtlk.h
-0xA4 00-1F uapi/linux/tee.h Generic TEE subsystem
-0xA4 00-1F uapi/asm/sgx.h <mailto:linux-sgx@vger.kernel.org>
-0xA5 01-05 linux/surface_aggregator/cdev.h Microsoft Surface Platform System Aggregator
- <mailto:luzmaximilian@gmail.com>
-0xA5 20-2F linux/surface_aggregator/dtx.h Microsoft Surface DTX driver
- <mailto:luzmaximilian@gmail.com>
+0xA4 00-1F uapi/linux/tee.h Generic TEE subsystem
+0xA4 00-1F uapi/asm/sgx.h <mailto:linux-sgx@vger.kernel.org>
+0xA5 01-05 linux/surface_aggregator/cdev.h Microsoft Surface Platform System Aggregator
+ <mailto:luzmaximilian@gmail.com>
+0xA5 20-2F linux/surface_aggregator/dtx.h Microsoft Surface DTX driver
+ <mailto:luzmaximilian@gmail.com>
0xAA 00-3F linux/uapi/linux/userfaultfd.h
0xAB 00-1F linux/nbd.h
0xAC 00-1F linux/raw.h
-0xAD 00 Netfilter device in development:
- <mailto:rusty@rustcorp.com.au>
-0xAE 00-1F linux/kvm.h Kernel-based Virtual Machine
- <mailto:kvm@vger.kernel.org>
-0xAE 40-FF linux/kvm.h Kernel-based Virtual Machine
- <mailto:kvm@vger.kernel.org>
-0xAE 20-3F linux/nitro_enclaves.h Nitro Enclaves
-0xAF 00-1F linux/fsl_hypervisor.h Freescale hypervisor
-0xB0 all RATIO devices in development:
- <mailto:vgo@ratio.de>
-0xB1 00-1F PPPoX
- <mailto:mostrows@styx.uwaterloo.ca>
-0xB2 00 arch/powerpc/include/uapi/asm/papr-vpd.h powerpc/pseries VPD API
- <mailto:linuxppc-dev>
-0xB2 01-02 arch/powerpc/include/uapi/asm/papr-sysparm.h powerpc/pseries system parameter API
- <mailto:linuxppc-dev>
+0xAD 00 Netfilter device in development:
+ <mailto:rusty@rustcorp.com.au>
+0xAE 00-1F linux/kvm.h Kernel-based Virtual Machine
+ <mailto:kvm@vger.kernel.org>
+0xAE 40-FF linux/kvm.h Kernel-based Virtual Machine
+ <mailto:kvm@vger.kernel.org>
+0xAE 20-3F linux/nitro_enclaves.h Nitro Enclaves
+0xAF 00-1F linux/fsl_hypervisor.h Freescale hypervisor
+0xB0 all RATIO devices in development:
+ <mailto:vgo@ratio.de>
+0xB1 00-1F PPPoX
+ <mailto:mostrows@styx.uwaterloo.ca>
+0xB2 00 arch/powerpc/include/uapi/asm/papr-vpd.h powerpc/pseries VPD API
+ <mailto:linuxppc-dev@lists.ozlabs.org>
+0xB2 01-02 arch/powerpc/include/uapi/asm/papr-sysparm.h powerpc/pseries system parameter API
+ <mailto:linuxppc-dev@lists.ozlabs.org>
0xB3 00 linux/mmc/ioctl.h
-0xB4 00-0F linux/gpio.h <mailto:linux-gpio@vger.kernel.org>
-0xB5 00-0F uapi/linux/rpmsg.h <mailto:linux-remoteproc@vger.kernel.org>
+0xB4 00-0F linux/gpio.h <mailto:linux-gpio@vger.kernel.org>
+0xB5 00-0F uapi/linux/rpmsg.h <mailto:linux-remoteproc@vger.kernel.org>
0xB6 all linux/fpga-dfl.h
-0xB7 all uapi/linux/remoteproc_cdev.h <mailto:linux-remoteproc@vger.kernel.org>
-0xB7 all uapi/linux/nsfs.h <mailto:Andrei Vagin <avagin@openvz.org>>
-0xB8 01-02 uapi/misc/mrvl_cn10k_dpi.h Marvell CN10K DPI driver
+0xB7 all uapi/linux/remoteproc_cdev.h <mailto:linux-remoteproc@vger.kernel.org>
+0xB7 all uapi/linux/nsfs.h <mailto:Andrei Vagin <avagin@openvz.org>>
+0xB8 01-02 uapi/misc/mrvl_cn10k_dpi.h Marvell CN10K DPI driver
0xC0 00-0F linux/usb/iowarrior.h
0xCA 00-0F uapi/misc/cxl.h
0xCA 10-2F uapi/misc/ocxl.h
0xCA 80-BF uapi/scsi/cxlflash_ioctl.h
-0xCB 00-1F CBM serial IEC bus in development:
- <mailto:michael.klein@puffin.lb.shuttle.de>
-0xCC 00-0F drivers/misc/ibmvmc.h pseries VMC driver
+0xCB 00-1F CBM serial IEC bus in development:
+ <mailto:michael.klein@puffin.lb.shuttle.de>
+0xCC 00-0F drivers/misc/ibmvmc.h pseries VMC driver
0xCD 01 linux/reiserfs_fs.h
-0xCE 01-02 uapi/linux/cxl_mem.h Compute Express Link Memory Devices
+0xCE 01-02 uapi/linux/cxl_mem.h Compute Express Link Memory Devices
0xCF 02 fs/smb/client/cifs_ioctl.h
0xDB 00-0F drivers/char/mwave/mwavepub.h
-0xDD 00-3F ZFCP device driver see drivers/s390/scsi/
- <mailto:aherrman@de.ibm.com>
+0xDD 00-3F ZFCP device driver see drivers/s390/scsi/
+ <mailto:aherrman@de.ibm.com>
0xE5 00-3F linux/fuse.h
-0xEC 00-01 drivers/platform/chrome/cros_ec_dev.h ChromeOS EC driver
-0xEE 00-09 uapi/linux/pfrut.h Platform Firmware Runtime Update and Telemetry
-0xF3 00-3F drivers/usb/misc/sisusbvga/sisusb.h sisfb (in development)
- <mailto:thomas@winischhofer.net>
-0xF6 all LTTng Linux Trace Toolkit Next Generation
- <mailto:mathieu.desnoyers@efficios.com>
-0xF8 all arch/x86/include/uapi/asm/amd_hsmp.h AMD HSMP EPYC system management interface driver
- <mailto:nchatrad@amd.com>
+0xEC 00-01 drivers/platform/chrome/cros_ec_dev.h ChromeOS EC driver
+0xEE 00-09 uapi/linux/pfrut.h Platform Firmware Runtime Update and Telemetry
+0xF3 00-3F drivers/usb/misc/sisusbvga/sisusb.h sisfb (in development)
+ <mailto:thomas@winischhofer.net>
+0xF6 all LTTng Linux Trace Toolkit Next Generation
+ <mailto:mathieu.desnoyers@efficios.com>
+0xF8 all arch/x86/include/uapi/asm/amd_hsmp.h AMD HSMP EPYC system management interface driver
+ <mailto:nchatrad@amd.com>
0xFD all linux/dm-ioctl.h
0xFE all linux/isst_if.h
-==== ===== ======================================================= ================================================================
+==== ===== ========================================================= ================================================================
diff --git a/Makefile b/Makefile
index 3f5933a5c7e7..bcb9d2fd69a6 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
VERSION = 6
PATCHLEVEL = 12
-SUBLEVEL = 94
+SUBLEVEL = 95
EXTRAVERSION =
NAME = Baby Opossum Posse
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 012ec170232e..dd98fa97aad0 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -212,6 +212,7 @@ config ARM64
if DYNAMIC_FTRACE_WITH_ARGS
select HAVE_SAMPLE_FTRACE_DIRECT
select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
+ select HAVE_BUILDTIME_MCOUNT_SORT
select HAVE_EFFICIENT_UNALIGNED_ACCESS
select HAVE_GUP_FAST
select HAVE_FTRACE_MCOUNT_RECORD
diff --git a/arch/loongarch/kernel/smp.c b/arch/loongarch/kernel/smp.c
index d96065dbe779..cf1f377ec622 100644
--- a/arch/loongarch/kernel/smp.c
+++ b/arch/loongarch/kernel/smp.c
@@ -650,6 +650,7 @@ static void stop_this_cpu(void *dummy)
set_cpu_online(smp_processor_id(), false);
calculate_cpu_foreign_map();
local_irq_disable();
+ rcutree_report_cpu_dead();
while (true);
}
diff --git a/arch/mips/dec/prom/console.c b/arch/mips/dec/prom/console.c
index 31a8441d8431..b4f0dba3fa20 100644
--- a/arch/mips/dec/prom/console.c
+++ b/arch/mips/dec/prom/console.c
@@ -2,8 +2,9 @@
/*
* DECstation PROM-based early console support.
*
- * Copyright (C) 2004, 2007 Maciej W. Rozycki
+ * Copyright (C) 2004, 2007, 2026 Maciej W. Rozycki
*/
+#include <linux/bug.h>
#include <linux/console.h>
#include <linux/init.h>
#include <linux/kernel.h>
@@ -14,9 +15,11 @@
static void __init prom_console_write(struct console *con, const char *s,
unsigned int c)
{
- char buf[81];
+ static char buf[81] __initdata = { 0 };
unsigned int chunk = sizeof(buf) - 1;
+ BUG_ON((long)buf != (int)(long)buf);
+
while (c > 0) {
if (chunk > c)
chunk = c;
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index 39e193cad2b9..a95814e088a2 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -20,6 +20,7 @@
#include <linux/sched/mm.h>
#include <linux/cpumask.h>
#include <linux/cpu.h>
+#include <linux/rcupdate.h>
#include <linux/err.h>
#include <linux/ftrace.h>
#include <linux/irqdomain.h>
@@ -411,6 +412,7 @@ static void stop_this_cpu(void *dummy)
set_cpu_online(smp_processor_id(), false);
calculate_cpu_foreign_map();
local_irq_disable();
+ rcutree_report_cpu_dead();
while (1);
}
diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h
index b59ffeb668d6..a62604fab1aa 100644
--- a/arch/riscv/include/asm/cacheflush.h
+++ b/arch/riscv/include/asm/cacheflush.h
@@ -43,20 +43,23 @@ do { \
#ifdef CONFIG_64BIT
extern u64 new_vmalloc[NR_CPUS / sizeof(u64) + 1];
extern char _end[];
+static inline void mark_new_valid_map(void)
+{
+ int i;
+
+ /*
+ * We don't care if concurrently a cpu resets this value since
+ * the only place this can happen is in handle_exception() where
+ * an sfence.vma is emitted.
+ */
+ for (i = 0; i < ARRAY_SIZE(new_vmalloc); ++i)
+ new_vmalloc[i] = -1ULL;
+}
#define flush_cache_vmap flush_cache_vmap
static inline void flush_cache_vmap(unsigned long start, unsigned long end)
{
- if (is_vmalloc_or_module_addr((void *)start)) {
- int i;
-
- /*
- * We don't care if concurrently a cpu resets this value since
- * the only place this can happen is in handle_exception() where
- * an sfence.vma is emitted.
- */
- for (i = 0; i < ARRAY_SIZE(new_vmalloc); ++i)
- new_vmalloc[i] = -1ULL;
- }
+ if (is_vmalloc_or_module_addr((void *)start))
+ mark_new_valid_map();
}
#define flush_cache_vmap_early(start, end) local_flush_tlb_kernel_range(start, end)
#endif
diff --git a/arch/riscv/include/asm/kfence.h b/arch/riscv/include/asm/kfence.h
index d08bf7fb3aee..29cb3a6ee113 100644
--- a/arch/riscv/include/asm/kfence.h
+++ b/arch/riscv/include/asm/kfence.h
@@ -6,6 +6,7 @@
#include <linux/kfence.h>
#include <linux/pfn.h>
#include <asm-generic/pgalloc.h>
+#include <asm/cacheflush.h>
#include <asm/pgtable.h>
static inline bool arch_kfence_init_pool(void)
@@ -17,10 +18,12 @@ static inline bool kfence_protect_page(unsigned long addr, bool protect)
{
pte_t *pte = virt_to_kpte(addr);
- if (protect)
+ if (protect) {
set_pte(pte, __pte(pte_val(ptep_get(pte)) & ~_PAGE_PRESENT));
- else
+ } else {
set_pte(pte, __pte(pte_val(ptep_get(pte)) | _PAGE_PRESENT));
+ mark_new_valid_map();
+ }
preempt_disable();
local_flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index af483de85dfc..5efa5584303e 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -106,8 +106,10 @@ SYM_CODE_START(handle_exception)
#ifdef CONFIG_64BIT
/*
- * The RISC-V kernel does not eagerly emit a sfence.vma after each
- * new vmalloc mapping, which may result in exceptions:
+ * The RISC-V kernel does not flush TLBs on all CPUS after each new
+ * vmalloc mapping or kfence_unprotect(), which may result in
+ * exceptions:
+ *
* - if the uarch caches invalid entries, the new mapping would not be
* observed by the page table walker and an invalidation is needed.
* - if the uarch does not cache invalid entries, a reordered access
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index 0d9c62b1dd44..f85c51241b3d 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -1838,6 +1838,11 @@ static bool hv_is_vp_in_sparse_set(u32 vp_id, u64 valid_bank_mask, u64 sparse_ba
int valid_bit_nr = vp_id / HV_VCPUS_PER_SPARSE_BANK;
unsigned long sbank;
+ BUILD_BUG_ON(BITS_PER_TYPE(valid_bank_mask) != HV_MAX_SPARSE_VCPU_BANKS);
+
+ if (valid_bit_nr >= HV_MAX_SPARSE_VCPU_BANKS)
+ return false;
+
if (!test_bit(valid_bit_nr, (unsigned long *)&valid_bank_mask))
return false;
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index d288c60ae200..aab26f90c285 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -2329,13 +2329,15 @@ static struct kvm_mmu_page *kvm_mmu_get_child_sp(struct kvm_vcpu *vcpu,
u64 *sptep, gfn_t gfn,
bool direct, unsigned int access)
{
- union kvm_mmu_page_role role;
+ union kvm_mmu_page_role role = kvm_mmu_child_role(sptep, direct, access);
- if (is_shadow_present_pte(*sptep) && !is_large_pte(*sptep) &&
- spte_to_child_sp(*sptep) && spte_to_child_sp(*sptep)->gfn == gfn)
+ if (is_shadow_present_pte(*sptep) &&
+ !is_large_pte(*sptep) &&
+ spte_to_child_sp(*sptep) &&
+ spte_to_child_sp(*sptep)->gfn == gfn &&
+ spte_to_child_sp(*sptep)->role.word == role.word)
return ERR_PTR(-EEXIST);
- role = kvm_mmu_child_role(sptep, direct, access);
return kvm_mmu_get_shadow_page(vcpu, gfn, role);
}
@@ -6950,13 +6952,19 @@ static bool kvm_mmu_zap_collapsible_spte(struct kvm *kvm,
sp = sptep_to_sp(sptep);
/*
- * We cannot do huge page mapping for indirect shadow pages,
- * which are found on the last rmap (level = 1) when not using
- * tdp; such shadow pages are synced with the page table in
- * the guest, and the guest page table is using 4K page size
- * mapping if the indirect sp has level = 1.
+ * Direct shadow page can be replaced by a hugepage if the host
+ * mapping level allows it and the memslot maps all of the host
+ * hugepage. Note! If the memslot maps only part of the
+ * hugepage, sp->gfn may be below slot->base_gfn, and querying
+ * the max mapping level would cause an out-of-bounds lpage_info
+ * access. So the gfn bounds check *must* be done first.
+ *
+ * Indirect shadow pages are created when the guest page tables
+ * are using 4K pages. Since the host mapping is always
+ * constrained by the page size in the guest, indirect shadow
+ * pages are never collapsible.
*/
- if (sp->role.direct &&
+ if (sp->role.direct && is_gfn_in_memslot(slot, sp->gfn) &&
sp->role.level < kvm_mmu_max_mapping_level(kvm, slot, sp->gfn,
PG_LEVEL_NUM)) {
kvm_zap_one_rmap_spte(kvm, rmap_head, sptep);
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 115c59c86f44..ccfa4924196a 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1280,6 +1280,7 @@ static int sev_dbg_crypt(struct kvm *kvm, struct kvm_sev_cmd *argp, bool dec)
s_off = vaddr & ~PAGE_MASK;
d_off = dst_vaddr & ~PAGE_MASK;
len = min_t(size_t, (PAGE_SIZE - s_off), size);
+ len = min_t(size_t, len, PAGE_SIZE - d_off);
if (dec)
ret = __sev_dbg_decrypt_user(kvm,
@@ -3168,37 +3169,6 @@ void sev_guest_memory_reclaimed(struct kvm *kvm)
wbinvd_on_all_cpus();
}
-void sev_free_vcpu(struct kvm_vcpu *vcpu)
-{
- struct vcpu_svm *svm;
-
- if (!sev_es_guest(vcpu->kvm))
- return;
-
- svm = to_svm(vcpu);
-
- /*
- * If it's an SNP guest, then the VMSA was marked in the RMP table as
- * a guest-owned page. Transition the page to hypervisor state before
- * releasing it back to the system.
- */
- if (sev_snp_guest(vcpu->kvm)) {
- u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT;
-
- if (kvm_rmp_make_shared(vcpu->kvm, pfn, PG_LEVEL_4K))
- goto skip_vmsa_free;
- }
-
- if (vcpu->arch.guest_state_protected)
- sev_flush_encrypted_page(vcpu, svm->sev_es.vmsa);
-
- __free_page(virt_to_page(svm->sev_es.vmsa));
-
-skip_vmsa_free:
- if (svm->sev_es.ghcb_sa_free)
- kvfree(svm->sev_es.ghcb_sa);
-}
-
static void dump_ghcb(struct vcpu_svm *svm)
{
struct ghcb *ghcb = svm->sev_es.ghcb;
@@ -3443,6 +3413,20 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm)
return 1;
}
+static void __sev_es_unmap_ghcb(struct vcpu_svm *svm)
+{
+ if (svm->sev_es.ghcb_sa_free) {
+ kvfree(svm->sev_es.ghcb_sa);
+ svm->sev_es.ghcb_sa = NULL;
+ svm->sev_es.ghcb_sa_free = false;
+ }
+
+ if (svm->sev_es.ghcb) {
+ kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map, true);
+ svm->sev_es.ghcb = NULL;
+ }
+}
+
void sev_es_unmap_ghcb(struct vcpu_svm *svm)
{
/* Clear any indication that the vCPU is in a type of AP Reset Hold */
@@ -3461,18 +3445,41 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm)
svm->sev_es.ghcb_sa_sync = false;
}
- if (svm->sev_es.ghcb_sa_free) {
- kvfree(svm->sev_es.ghcb_sa);
- svm->sev_es.ghcb_sa = NULL;
- svm->sev_es.ghcb_sa_free = false;
- }
-
trace_kvm_vmgexit_exit(svm->vcpu.vcpu_id, svm->sev_es.ghcb);
sev_es_sync_to_ghcb(svm);
- kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map, true);
- svm->sev_es.ghcb = NULL;
+ __sev_es_unmap_ghcb(svm);
+}
+
+void sev_free_vcpu(struct kvm_vcpu *vcpu)
+{
+ struct vcpu_svm *svm;
+
+ if (!sev_es_guest(vcpu->kvm))
+ return;
+
+ svm = to_svm(vcpu);
+
+ /*
+ * If it's an SNP guest, then the VMSA was marked in the RMP table as
+ * a guest-owned page. Transition the page to hypervisor state before
+ * releasing it back to the system.
+ */
+ if (sev_snp_guest(vcpu->kvm)) {
+ u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT;
+
+ if (kvm_rmp_make_shared(vcpu->kvm, pfn, PG_LEVEL_4K))
+ goto skip_vmsa_free;
+ }
+
+ if (vcpu->arch.guest_state_protected)
+ sev_flush_encrypted_page(vcpu, svm->sev_es.vmsa);
+
+ __free_page(virt_to_page(svm->sev_es.vmsa));
+
+skip_vmsa_free:
+ __sev_es_unmap_ghcb(svm);
}
void pre_sev_run(struct vcpu_svm *svm, int cpu)
@@ -4347,6 +4354,16 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
exit_code = kvm_ghcb_get_sw_exit_code(control);
switch (exit_code) {
case SVM_VMGEXIT_MMIO_READ:
+ if (!control->exit_info_2)
+ return 1;
+
+ if (to_kvm_sev_info(vcpu->kvm)->ghcb_version >= 2 &&
+ control->exit_info_2 > 8) {
+ ghcb_set_sw_exit_info_1(svm->sev_es.ghcb, 2);
+ ghcb_set_sw_exit_info_2(svm->sev_es.ghcb, GHCB_ERR_INVALID_INPUT);
+ return 1;
+ }
+
ret = setup_vmgexit_scratch(svm, true, control->exit_info_2);
if (ret)
break;
@@ -4357,6 +4374,16 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
svm->sev_es.ghcb_sa);
break;
case SVM_VMGEXIT_MMIO_WRITE:
+ if (!control->exit_info_2)
+ return 1;
+
+ if (to_kvm_sev_info(vcpu->kvm)->ghcb_version >= 2 &&
+ control->exit_info_2 > 8) {
+ ghcb_set_sw_exit_info_1(svm->sev_es.ghcb, 2);
+ ghcb_set_sw_exit_info_2(svm->sev_es.ghcb, GHCB_ERR_INVALID_INPUT);
+ return 1;
+ }
+
ret = setup_vmgexit_scratch(svm, false, control->exit_info_2);
if (ret)
break;
@@ -4439,6 +4466,11 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
control->exit_info_1, control->exit_info_2);
ret = -EINVAL;
break;
+ case SVM_EXIT_IOIO:
+ if (!((control->exit_info_1 & SVM_IOIO_SIZE_MASK) >> SVM_IOIO_SIZE_SHIFT))
+ return 1;
+
+ fallthrough;
default:
ret = svm_invoke_exit_handler(vcpu, exit_code);
}
@@ -4459,6 +4491,9 @@ int sev_es_string_io(struct vcpu_svm *svm, int size, unsigned int port, int in)
if (unlikely(check_mul_overflow(count, size, &bytes)))
return -EINVAL;
+ if (!bytes)
+ return 1;
+
r = setup_vmgexit_scratch(svm, in, bytes);
if (r)
return r;
diff --git a/block/bdev.c b/block/bdev.c
index e7daca6565ea..ea4268a308cc 100644
--- a/block/bdev.c
+++ b/block/bdev.c
@@ -414,15 +414,10 @@ EXPORT_SYMBOL_GPL(blockdev_superblock);
void __init bdev_cache_init(void)
{
- int err;
-
bdev_cachep = kmem_cache_create("bdev_cache", sizeof(struct bdev_inode),
0, (SLAB_HWCACHE_ALIGN|SLAB_RECLAIM_ACCOUNT|
SLAB_ACCOUNT|SLAB_PANIC),
init_once);
- err = register_filesystem(&bd_type);
- if (err)
- panic("Cannot register bdev pseudo-fs");
blockdev_mnt = kern_mount(&bd_type);
if (IS_ERR(blockdev_mnt))
panic("Cannot create bdev pseudo-fs");
diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c
index a0fbb427a7a6..d95104f0fcdc 100644
--- a/block/blk-cgroup.c
+++ b/block/blk-cgroup.c
@@ -164,20 +164,10 @@ static void blkg_free(struct blkcg_gq *blkg)
static void __blkg_release(struct rcu_head *rcu)
{
struct blkcg_gq *blkg = container_of(rcu, struct blkcg_gq, rcu_head);
- struct blkcg *blkcg = blkg->blkcg;
- int cpu;
#ifdef CONFIG_BLK_CGROUP_PUNT_BIO
WARN_ON(!bio_list_empty(&blkg->async_bios));
#endif
- /*
- * Flush all the non-empty percpu lockless lists before releasing
- * us, given these stat belongs to us.
- *
- * blkg_stat_lock is for serializing blkg stat update
- */
- for_each_possible_cpu(cpu)
- __blkcg_rstat_flush(blkcg, cpu);
/* release the blkcg and parent blkg refs this blkg has been holding */
css_put(&blkg->blkcg->css);
@@ -195,6 +185,17 @@ static void __blkg_release(struct rcu_head *rcu)
static void blkg_release(struct percpu_ref *ref)
{
struct blkcg_gq *blkg = container_of(ref, struct blkcg_gq, refcnt);
+ struct blkcg *blkcg = blkg->blkcg;
+ int cpu;
+
+ /*
+ * Flush all the non-empty percpu lockless lists before releasing
+ * us, given these stat belongs to us.
+ *
+ * blkg_stat_lock is for serializing blkg stat update
+ */
+ for_each_possible_cpu(cpu)
+ __blkcg_rstat_flush(blkcg, cpu);
call_rcu(&blkg->rcu_head, __blkg_release);
}
diff --git a/drivers/acpi/scan.c b/drivers/acpi/scan.c
index 0774cc692110..4b9c81fbdc3b 100644
--- a/drivers/acpi/scan.c
+++ b/drivers/acpi/scan.c
@@ -5,6 +5,7 @@
#define pr_fmt(fmt) "ACPI: " fmt
+#include <linux/async.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/slab.h>
@@ -2364,46 +2365,34 @@ static int acpi_dev_get_next_consumer_dev_cb(struct acpi_dep_data *dep, void *da
return 0;
}
-struct acpi_scan_clear_dep_work {
- struct work_struct work;
- struct acpi_device *adev;
-};
-
-static void acpi_scan_clear_dep_fn(struct work_struct *work)
+static void acpi_scan_clear_dep_fn(void *dev, async_cookie_t cookie)
{
- struct acpi_scan_clear_dep_work *cdw;
-
- cdw = container_of(work, struct acpi_scan_clear_dep_work, work);
+ struct acpi_device *adev = to_acpi_device(dev);
acpi_scan_lock_acquire();
- acpi_bus_attach(cdw->adev, (void *)true);
+ acpi_bus_attach(adev, (void *)true);
acpi_scan_lock_release();
- acpi_dev_put(cdw->adev);
- kfree(cdw);
+ acpi_dev_put(adev);
}
static bool acpi_scan_clear_dep_queue(struct acpi_device *adev)
{
- struct acpi_scan_clear_dep_work *cdw;
-
if (adev->dep_unmet)
return false;
- cdw = kmalloc(sizeof(*cdw), GFP_KERNEL);
- if (!cdw)
- return false;
-
- cdw->adev = adev;
- INIT_WORK(&cdw->work, acpi_scan_clear_dep_fn);
/*
- * Since the work function may block on the lock until the entire
- * initial enumeration of devices is complete, put it into the unbound
- * workqueue.
+ * Async schedule the deferred acpi_scan_clear_dep_fn() since:
+ * - acpi_bus_attach() needs to hold acpi_scan_lock which cannot
+ * be acquired under acpi_dep_list_lock (held here)
+ * - the deferred work at boot stage is ensured to be finished
+ * before userspace init task by the async_synchronize_full()
+ * barrier
+ *
+ * Use _nocall variant since it'll return on failure instead of
+ * run the function synchronously.
*/
- queue_work(system_unbound_wq, &cdw->work);
-
- return true;
+ return async_schedule_dev_nocall(acpi_scan_clear_dep_fn, &adev->dev);
}
static void acpi_scan_delete_dep_data(struct acpi_dep_data *dep)
diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 948e77da1dc3..2f7e8bab3d10 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -794,7 +794,6 @@ static int add_memory_block(unsigned long block_id, unsigned long state,
mem->start_section_nr = block_id * sections_per_block;
mem->state = state;
mem->nid = NUMA_NO_NODE;
- mem->altmap = altmap;
INIT_LIST_HEAD(&mem->group_next);
#ifndef CONFIG_NUMA
@@ -812,6 +811,8 @@ static int add_memory_block(unsigned long block_id, unsigned long state,
if (ret)
return ret;
+ mem->altmap = altmap;
+
if (group) {
mem->group = group;
list_add(&mem->group_next, &group->memory_blocks);
diff --git a/drivers/char/agp/amd64-agp.c b/drivers/char/agp/amd64-agp.c
index 8e41731d3642..c9d7cefa5192 100644
--- a/drivers/char/agp/amd64-agp.c
+++ b/drivers/char/agp/amd64-agp.c
@@ -546,7 +546,7 @@ static int agp_amd64_probe(struct pci_dev *pdev,
/* Fill in the mode register */
pci_read_config_dword(pdev, bridge->capndx+PCI_AGP_STATUS, &bridge->mode);
- if (cache_nbs(pdev, cap_ptr) == -1) {
+ if (cache_nbs(pdev, cap_ptr) < 0) {
agp_put_bridge(bridge);
return -ENODEV;
}
diff --git a/drivers/crypto/intel/qat/qat_common/adf_cfg.c b/drivers/crypto/intel/qat/qat_common/adf_cfg.c
index b0fc453fa3fb..97273da40b43 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_cfg.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_cfg.c
@@ -103,16 +103,6 @@ static void adf_cfg_section_del_all(struct list_head *head);
static void adf_cfg_section_del_all_except(struct list_head *head,
const char *section_name);
-void adf_cfg_del_all(struct adf_accel_dev *accel_dev)
-{
- struct adf_cfg_device_data *dev_cfg_data = accel_dev->cfg;
-
- down_write(&dev_cfg_data->lock);
- adf_cfg_section_del_all(&dev_cfg_data->sec_list);
- up_write(&dev_cfg_data->lock);
- clear_bit(ADF_STATUS_CONFIGURED, &accel_dev->status);
-}
-
void adf_cfg_del_all_except(struct adf_accel_dev *accel_dev,
const char *section_name)
{
diff --git a/drivers/crypto/intel/qat/qat_common/adf_cfg.h b/drivers/crypto/intel/qat/qat_common/adf_cfg.h
index 2afa6f0d15c5..108032b39242 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_cfg.h
+++ b/drivers/crypto/intel/qat/qat_common/adf_cfg.h
@@ -34,7 +34,6 @@ void adf_cfg_dev_remove(struct adf_accel_dev *accel_dev);
void adf_cfg_dev_dbgfs_add(struct adf_accel_dev *accel_dev);
void adf_cfg_dev_dbgfs_rm(struct adf_accel_dev *accel_dev);
int adf_cfg_section_add(struct adf_accel_dev *accel_dev, const char *name);
-void adf_cfg_del_all(struct adf_accel_dev *accel_dev);
void adf_cfg_del_all_except(struct adf_accel_dev *accel_dev,
const char *section_name);
int adf_cfg_add_key_value_param(struct adf_accel_dev *accel_dev,
diff --git a/drivers/crypto/intel/qat/qat_common/adf_cfg_common.h b/drivers/crypto/intel/qat/qat_common/adf_cfg_common.h
index 89df3888d7ea..e642438b381d 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_cfg_common.h
+++ b/drivers/crypto/intel/qat/qat_common/adf_cfg_common.h
@@ -4,18 +4,11 @@
#define ADF_CFG_COMMON_H_
#include <linux/types.h>
-#include <linux/ioctl.h>
#define ADF_CFG_MAX_STR_LEN 64
#define ADF_CFG_MAX_KEY_LEN_IN_BYTES ADF_CFG_MAX_STR_LEN
#define ADF_CFG_MAX_VAL_LEN_IN_BYTES ADF_CFG_MAX_STR_LEN
#define ADF_CFG_MAX_SECTION_LEN_IN_BYTES ADF_CFG_MAX_STR_LEN
-#define ADF_CFG_BASE_DEC 10
-#define ADF_CFG_BASE_HEX 16
-#define ADF_CFG_ALL_DEVICES 0xFE
-#define ADF_CFG_NO_DEVICE 0xFF
-#define ADF_CFG_AFFINITY_WHATEVER 0xFF
-#define MAX_DEVICE_NAME_SIZE 32
#define ADF_MAX_DEVICES (32 * 32)
#define ADF_DEVS_ARRAY_SIZE BITS_TO_LONGS(ADF_MAX_DEVICES)
@@ -49,29 +42,4 @@ enum adf_device_type {
DEV_4XXX,
DEV_420XX,
};
-
-struct adf_dev_status_info {
- enum adf_device_type type;
- __u32 accel_id;
- __u32 instance_id;
- __u8 num_ae;
- __u8 num_accel;
- __u8 num_logical_accel;
- __u8 banks_per_accel;
- __u8 state;
- __u8 bus;
- __u8 dev;
- __u8 fun;
- char name[MAX_DEVICE_NAME_SIZE];
-};
-
-#define ADF_CTL_IOC_MAGIC 'a'
-#define IOCTL_CONFIG_SYS_RESOURCE_PARAMETERS _IOW(ADF_CTL_IOC_MAGIC, 0, \
- struct adf_user_cfg_ctl_data)
-#define IOCTL_STOP_ACCEL_DEV _IOW(ADF_CTL_IOC_MAGIC, 1, \
- struct adf_user_cfg_ctl_data)
-#define IOCTL_START_ACCEL_DEV _IOW(ADF_CTL_IOC_MAGIC, 2, \
- struct adf_user_cfg_ctl_data)
-#define IOCTL_STATUS_ACCEL_DEV _IOW(ADF_CTL_IOC_MAGIC, 3, __u32)
-#define IOCTL_GET_NUM_DEVICES _IOW(ADF_CTL_IOC_MAGIC, 4, __s32)
#endif
diff --git a/drivers/crypto/intel/qat/qat_common/adf_cfg_user.h b/drivers/crypto/intel/qat/qat_common/adf_cfg_user.h
deleted file mode 100644
index 421f4fb8b4dd..000000000000
--- a/drivers/crypto/intel/qat/qat_common/adf_cfg_user.h
+++ /dev/null
@@ -1,38 +0,0 @@
-/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0-only) */
-/* Copyright(c) 2014 - 2020 Intel Corporation */
-#ifndef ADF_CFG_USER_H_
-#define ADF_CFG_USER_H_
-
-#include "adf_cfg_common.h"
-#include "adf_cfg_strings.h"
-
-struct adf_user_cfg_key_val {
- char key[ADF_CFG_MAX_KEY_LEN_IN_BYTES];
- char val[ADF_CFG_MAX_VAL_LEN_IN_BYTES];
- union {
- struct adf_user_cfg_key_val *next;
- __u64 padding3;
- };
- enum adf_cfg_val_type type;
-} __packed;
-
-struct adf_user_cfg_section {
- char name[ADF_CFG_MAX_SECTION_LEN_IN_BYTES];
- union {
- struct adf_user_cfg_key_val *params;
- __u64 padding1;
- };
- union {
- struct adf_user_cfg_section *next;
- __u64 padding3;
- };
-} __packed;
-
-struct adf_user_cfg_ctl_data {
- union {
- struct adf_user_cfg_section *config_section;
- __u64 padding;
- };
- __u8 device_id;
-} __packed;
-#endif
diff --git a/drivers/crypto/intel/qat/qat_common/adf_common_drv.h b/drivers/crypto/intel/qat/qat_common/adf_common_drv.h
index 25c940b06c36..fc2fea0d3ada 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_common_drv.h
+++ b/drivers/crypto/intel/qat/qat_common/adf_common_drv.h
@@ -68,11 +68,8 @@ int adf_devmgr_add_dev(struct adf_accel_dev *accel_dev,
void adf_devmgr_rm_dev(struct adf_accel_dev *accel_dev,
struct adf_accel_dev *pf);
struct list_head *adf_devmgr_get_head(void);
-struct adf_accel_dev *adf_devmgr_get_dev_by_id(u32 id);
struct adf_accel_dev *adf_devmgr_get_first(void);
struct adf_accel_dev *adf_devmgr_pci_to_accel_dev(struct pci_dev *pci_dev);
-int adf_devmgr_verify_id(u32 id);
-void adf_devmgr_get_num_dev(u32 *num);
int adf_devmgr_in_reset(struct adf_accel_dev *accel_dev);
int adf_dev_started(struct adf_accel_dev *accel_dev);
int adf_dev_restarting_notify(struct adf_accel_dev *accel_dev);
diff --git a/drivers/crypto/intel/qat/qat_common/adf_ctl_drv.c b/drivers/crypto/intel/qat/qat_common/adf_ctl_drv.c
index 70fa0f6497a9..36d5ebb3efab 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_ctl_drv.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_ctl_drv.c
@@ -2,422 +2,13 @@
/* Copyright(c) 2014 - 2020 Intel Corporation */
#include <crypto/algapi.h>
+#include <linux/errno.h>
#include <linux/module.h>
-#include <linux/mutex.h>
-#include <linux/slab.h>
-#include <linux/fs.h>
-#include <linux/bitops.h>
-#include <linux/pci.h>
-#include <linux/cdev.h>
-#include <linux/uaccess.h>
-#include "adf_accel_devices.h"
#include "adf_common_drv.h"
-#include "adf_cfg.h"
-#include "adf_cfg_common.h"
-#include "adf_cfg_user.h"
-
-#define ADF_CFG_MAX_SECTION 512
-#define ADF_CFG_MAX_KEY_VAL 256
-
-#define DEVICE_NAME "qat_adf_ctl"
-
-static DEFINE_MUTEX(adf_ctl_lock);
-static long adf_ctl_ioctl(struct file *fp, unsigned int cmd, unsigned long arg);
-
-static const struct file_operations adf_ctl_ops = {
- .owner = THIS_MODULE,
- .unlocked_ioctl = adf_ctl_ioctl,
- .compat_ioctl = compat_ptr_ioctl,
-};
-
-static const struct class adf_ctl_class = {
- .name = DEVICE_NAME,
-};
-
-struct adf_ctl_drv_info {
- unsigned int major;
- struct cdev drv_cdev;
-};
-
-static struct adf_ctl_drv_info adf_ctl_drv;
-
-static void adf_chr_drv_destroy(void)
-{
- device_destroy(&adf_ctl_class, MKDEV(adf_ctl_drv.major, 0));
- cdev_del(&adf_ctl_drv.drv_cdev);
- class_unregister(&adf_ctl_class);
- unregister_chrdev_region(MKDEV(adf_ctl_drv.major, 0), 1);
-}
-
-static int adf_chr_drv_create(void)
-{
- dev_t dev_id;
- struct device *drv_device;
- int ret;
-
- if (alloc_chrdev_region(&dev_id, 0, 1, DEVICE_NAME)) {
- pr_err("QAT: unable to allocate chrdev region\n");
- return -EFAULT;
- }
-
- ret = class_register(&adf_ctl_class);
- if (ret)
- goto err_chrdev_unreg;
-
- adf_ctl_drv.major = MAJOR(dev_id);
- cdev_init(&adf_ctl_drv.drv_cdev, &adf_ctl_ops);
- if (cdev_add(&adf_ctl_drv.drv_cdev, dev_id, 1)) {
- pr_err("QAT: cdev add failed\n");
- goto err_class_destr;
- }
-
- drv_device = device_create(&adf_ctl_class, NULL,
- MKDEV(adf_ctl_drv.major, 0),
- NULL, DEVICE_NAME);
- if (IS_ERR(drv_device)) {
- pr_err("QAT: failed to create device\n");
- goto err_cdev_del;
- }
- return 0;
-err_cdev_del:
- cdev_del(&adf_ctl_drv.drv_cdev);
-err_class_destr:
- class_unregister(&adf_ctl_class);
-err_chrdev_unreg:
- unregister_chrdev_region(dev_id, 1);
- return -EFAULT;
-}
-
-static int adf_ctl_alloc_resources(struct adf_user_cfg_ctl_data **ctl_data,
- unsigned long arg)
-{
- struct adf_user_cfg_ctl_data *cfg_data;
-
- cfg_data = kzalloc(sizeof(*cfg_data), GFP_KERNEL);
- if (!cfg_data)
- return -ENOMEM;
-
- /* Initialize device id to NO DEVICE as 0 is a valid device id */
- cfg_data->device_id = ADF_CFG_NO_DEVICE;
-
- if (copy_from_user(cfg_data, (void __user *)arg, sizeof(*cfg_data))) {
- pr_err("QAT: failed to copy from user cfg_data.\n");
- kfree(cfg_data);
- return -EIO;
- }
-
- *ctl_data = cfg_data;
- return 0;
-}
-
-static int adf_add_key_value_data(struct adf_accel_dev *accel_dev,
- const char *section,
- const struct adf_user_cfg_key_val *key_val)
-{
- if (key_val->type == ADF_HEX) {
- long *ptr = (long *)key_val->val;
- long val = *ptr;
-
- if (adf_cfg_add_key_value_param(accel_dev, section,
- key_val->key, (void *)val,
- key_val->type)) {
- dev_err(&GET_DEV(accel_dev),
- "failed to add hex keyvalue.\n");
- return -EFAULT;
- }
- } else {
- if (adf_cfg_add_key_value_param(accel_dev, section,
- key_val->key, key_val->val,
- key_val->type)) {
- dev_err(&GET_DEV(accel_dev),
- "failed to add keyvalue.\n");
- return -EFAULT;
- }
- }
- return 0;
-}
-
-static int adf_copy_key_value_data(struct adf_accel_dev *accel_dev,
- struct adf_user_cfg_ctl_data *ctl_data)
-{
- struct adf_user_cfg_key_val key_val;
- struct adf_user_cfg_key_val *params_head;
- struct adf_user_cfg_section section, *section_head;
- int i, j;
-
- section_head = ctl_data->config_section;
-
- for (i = 0; section_head && i < ADF_CFG_MAX_SECTION; i++) {
- if (copy_from_user(§ion, (void __user *)section_head,
- sizeof(*section_head))) {
- dev_err(&GET_DEV(accel_dev),
- "failed to copy section info\n");
- goto out_err;
- }
-
- if (adf_cfg_section_add(accel_dev, section.name)) {
- dev_err(&GET_DEV(accel_dev),
- "failed to add section.\n");
- goto out_err;
- }
-
- params_head = section.params;
-
- for (j = 0; params_head && j < ADF_CFG_MAX_KEY_VAL; j++) {
- if (copy_from_user(&key_val, (void __user *)params_head,
- sizeof(key_val))) {
- dev_err(&GET_DEV(accel_dev),
- "Failed to copy keyvalue.\n");
- goto out_err;
- }
- if (adf_add_key_value_data(accel_dev, section.name,
- &key_val)) {
- goto out_err;
- }
- params_head = key_val.next;
- }
- section_head = section.next;
- }
- return 0;
-out_err:
- adf_cfg_del_all(accel_dev);
- return -EFAULT;
-}
-
-static int adf_ctl_ioctl_dev_config(struct file *fp, unsigned int cmd,
- unsigned long arg)
-{
- int ret;
- struct adf_user_cfg_ctl_data *ctl_data;
- struct adf_accel_dev *accel_dev;
-
- ret = adf_ctl_alloc_resources(&ctl_data, arg);
- if (ret)
- return ret;
-
- accel_dev = adf_devmgr_get_dev_by_id(ctl_data->device_id);
- if (!accel_dev) {
- ret = -EFAULT;
- goto out;
- }
-
- if (adf_dev_started(accel_dev)) {
- ret = -EFAULT;
- goto out;
- }
-
- if (adf_copy_key_value_data(accel_dev, ctl_data)) {
- ret = -EFAULT;
- goto out;
- }
- set_bit(ADF_STATUS_CONFIGURED, &accel_dev->status);
-out:
- kfree(ctl_data);
- return ret;
-}
-
-static int adf_ctl_is_device_in_use(int id)
-{
- struct adf_accel_dev *dev;
-
- list_for_each_entry(dev, adf_devmgr_get_head(), list) {
- if (id == dev->accel_id || id == ADF_CFG_ALL_DEVICES) {
- if (adf_devmgr_in_reset(dev) || adf_dev_in_use(dev)) {
- dev_info(&GET_DEV(dev),
- "device qat_dev%d is busy\n",
- dev->accel_id);
- return -EBUSY;
- }
- }
- }
- return 0;
-}
-
-static void adf_ctl_stop_devices(u32 id)
-{
- struct adf_accel_dev *accel_dev;
-
- list_for_each_entry(accel_dev, adf_devmgr_get_head(), list) {
- if (id == accel_dev->accel_id || id == ADF_CFG_ALL_DEVICES) {
- if (!adf_dev_started(accel_dev))
- continue;
-
- /* First stop all VFs */
- if (!accel_dev->is_vf)
- continue;
-
- adf_dev_down(accel_dev);
- }
- }
-
- list_for_each_entry(accel_dev, adf_devmgr_get_head(), list) {
- if (id == accel_dev->accel_id || id == ADF_CFG_ALL_DEVICES) {
- if (!adf_dev_started(accel_dev))
- continue;
-
- adf_dev_down(accel_dev);
- }
- }
-}
-
-static int adf_ctl_ioctl_dev_stop(struct file *fp, unsigned int cmd,
- unsigned long arg)
-{
- int ret;
- struct adf_user_cfg_ctl_data *ctl_data;
-
- ret = adf_ctl_alloc_resources(&ctl_data, arg);
- if (ret)
- return ret;
-
- if (adf_devmgr_verify_id(ctl_data->device_id)) {
- pr_err("QAT: Device %d not found\n", ctl_data->device_id);
- ret = -ENODEV;
- goto out;
- }
-
- ret = adf_ctl_is_device_in_use(ctl_data->device_id);
- if (ret)
- goto out;
-
- if (ctl_data->device_id == ADF_CFG_ALL_DEVICES)
- pr_info("QAT: Stopping all acceleration devices.\n");
- else
- pr_info("QAT: Stopping acceleration device qat_dev%d.\n",
- ctl_data->device_id);
-
- adf_ctl_stop_devices(ctl_data->device_id);
-
-out:
- kfree(ctl_data);
- return ret;
-}
-
-static int adf_ctl_ioctl_dev_start(struct file *fp, unsigned int cmd,
- unsigned long arg)
-{
- int ret;
- struct adf_user_cfg_ctl_data *ctl_data;
- struct adf_accel_dev *accel_dev;
-
- ret = adf_ctl_alloc_resources(&ctl_data, arg);
- if (ret)
- return ret;
-
- ret = -ENODEV;
- accel_dev = adf_devmgr_get_dev_by_id(ctl_data->device_id);
- if (!accel_dev)
- goto out;
-
- dev_info(&GET_DEV(accel_dev),
- "Starting acceleration device qat_dev%d.\n",
- ctl_data->device_id);
-
- ret = adf_dev_up(accel_dev, false);
-
- if (ret) {
- dev_err(&GET_DEV(accel_dev), "Failed to start qat_dev%d\n",
- ctl_data->device_id);
- adf_dev_down(accel_dev);
- }
-out:
- kfree(ctl_data);
- return ret;
-}
-
-static int adf_ctl_ioctl_get_num_devices(struct file *fp, unsigned int cmd,
- unsigned long arg)
-{
- u32 num_devices = 0;
-
- adf_devmgr_get_num_dev(&num_devices);
- if (copy_to_user((void __user *)arg, &num_devices, sizeof(num_devices)))
- return -EFAULT;
-
- return 0;
-}
-
-static int adf_ctl_ioctl_get_status(struct file *fp, unsigned int cmd,
- unsigned long arg)
-{
- struct adf_hw_device_data *hw_data;
- struct adf_dev_status_info dev_info;
- struct adf_accel_dev *accel_dev;
-
- if (copy_from_user(&dev_info, (void __user *)arg,
- sizeof(struct adf_dev_status_info))) {
- pr_err("QAT: failed to copy from user.\n");
- return -EFAULT;
- }
-
- accel_dev = adf_devmgr_get_dev_by_id(dev_info.accel_id);
- if (!accel_dev)
- return -ENODEV;
-
- hw_data = accel_dev->hw_device;
- dev_info.state = adf_dev_started(accel_dev) ? DEV_UP : DEV_DOWN;
- dev_info.num_ae = hw_data->get_num_aes(hw_data);
- dev_info.num_accel = hw_data->get_num_accels(hw_data);
- dev_info.num_logical_accel = hw_data->num_logical_accel;
- dev_info.banks_per_accel = hw_data->num_banks
- / hw_data->num_logical_accel;
- strscpy(dev_info.name, hw_data->dev_class->name, sizeof(dev_info.name));
- dev_info.instance_id = hw_data->instance_id;
- dev_info.type = hw_data->dev_class->type;
- dev_info.bus = accel_to_pci_dev(accel_dev)->bus->number;
- dev_info.dev = PCI_SLOT(accel_to_pci_dev(accel_dev)->devfn);
- dev_info.fun = PCI_FUNC(accel_to_pci_dev(accel_dev)->devfn);
-
- if (copy_to_user((void __user *)arg, &dev_info,
- sizeof(struct adf_dev_status_info))) {
- dev_err(&GET_DEV(accel_dev), "failed to copy status.\n");
- return -EFAULT;
- }
- return 0;
-}
-
-static long adf_ctl_ioctl(struct file *fp, unsigned int cmd, unsigned long arg)
-{
- int ret;
-
- if (mutex_lock_interruptible(&adf_ctl_lock))
- return -EFAULT;
-
- switch (cmd) {
- case IOCTL_CONFIG_SYS_RESOURCE_PARAMETERS:
- ret = adf_ctl_ioctl_dev_config(fp, cmd, arg);
- break;
-
- case IOCTL_STOP_ACCEL_DEV:
- ret = adf_ctl_ioctl_dev_stop(fp, cmd, arg);
- break;
-
- case IOCTL_START_ACCEL_DEV:
- ret = adf_ctl_ioctl_dev_start(fp, cmd, arg);
- break;
-
- case IOCTL_GET_NUM_DEVICES:
- ret = adf_ctl_ioctl_get_num_devices(fp, cmd, arg);
- break;
-
- case IOCTL_STATUS_ACCEL_DEV:
- ret = adf_ctl_ioctl_get_status(fp, cmd, arg);
- break;
- default:
- pr_err_ratelimited("QAT: Invalid ioctl %d\n", cmd);
- ret = -EFAULT;
- break;
- }
- mutex_unlock(&adf_ctl_lock);
- return ret;
-}
static int __init adf_register_ctl_device_driver(void)
{
- if (adf_chr_drv_create())
- goto err_chr_dev;
-
if (adf_init_misc_wq())
goto err_misc_wq;
@@ -449,15 +40,11 @@ static int __init adf_register_ctl_device_driver(void)
err_aer:
adf_exit_misc_wq();
err_misc_wq:
- adf_chr_drv_destroy();
-err_chr_dev:
- mutex_destroy(&adf_ctl_lock);
return -EFAULT;
}
static void __exit adf_unregister_ctl_device_driver(void)
{
- adf_chr_drv_destroy();
adf_exit_misc_wq();
adf_exit_aer();
adf_exit_vf_wq();
@@ -465,7 +52,6 @@ static void __exit adf_unregister_ctl_device_driver(void)
qat_crypto_unregister();
qat_compression_unregister();
adf_clean_vf_map(false);
- mutex_destroy(&adf_ctl_lock);
}
module_init(adf_register_ctl_device_driver);
diff --git a/drivers/crypto/intel/qat/qat_common/adf_dev_mgr.c b/drivers/crypto/intel/qat/qat_common/adf_dev_mgr.c
index 96ddd1c419c4..9013a04f425b 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_dev_mgr.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_dev_mgr.c
@@ -45,19 +45,6 @@ static struct vf_id_map *adf_find_vf(u32 bdf)
return NULL;
}
-static int adf_get_vf_real_id(u32 fake)
-{
- struct list_head *itr;
-
- list_for_each(itr, &vfs_table) {
- struct vf_id_map *ptr =
- list_entry(itr, struct vf_id_map, list);
- if (ptr->fake_id == fake)
- return ptr->id;
- }
- return -1;
-}
-
/**
* adf_clean_vf_map() - Cleans VF id mappings
* @vf: flag indicating whether mappings is cleaned
@@ -314,63 +301,6 @@ struct adf_accel_dev *adf_devmgr_pci_to_accel_dev(struct pci_dev *pci_dev)
}
EXPORT_SYMBOL_GPL(adf_devmgr_pci_to_accel_dev);
-struct adf_accel_dev *adf_devmgr_get_dev_by_id(u32 id)
-{
- struct list_head *itr;
- int real_id;
-
- mutex_lock(&table_lock);
- real_id = adf_get_vf_real_id(id);
- if (real_id < 0)
- goto unlock;
-
- id = real_id;
-
- list_for_each(itr, &accel_table) {
- struct adf_accel_dev *ptr =
- list_entry(itr, struct adf_accel_dev, list);
- if (ptr->accel_id == id) {
- mutex_unlock(&table_lock);
- return ptr;
- }
- }
-unlock:
- mutex_unlock(&table_lock);
- return NULL;
-}
-
-int adf_devmgr_verify_id(u32 id)
-{
- if (id == ADF_CFG_ALL_DEVICES)
- return 0;
-
- if (adf_devmgr_get_dev_by_id(id))
- return 0;
-
- return -ENODEV;
-}
-
-static int adf_get_num_dettached_vfs(void)
-{
- struct list_head *itr;
- int vfs = 0;
-
- mutex_lock(&table_lock);
- list_for_each(itr, &vfs_table) {
- struct vf_id_map *ptr =
- list_entry(itr, struct vf_id_map, list);
- if (ptr->bdf != ~0 && !ptr->attached)
- vfs++;
- }
- mutex_unlock(&table_lock);
- return vfs;
-}
-
-void adf_devmgr_get_num_dev(u32 *num)
-{
- *num = num_devices - adf_get_num_dettached_vfs();
-}
-
/**
* adf_dev_in_use() - Check whether accel_dev is currently in use
* @accel_dev: Pointer to acceleration device.
diff --git a/drivers/fpga/of-fpga-region.c b/drivers/fpga/of-fpga-region.c
index efc35dad6aaa..cbf6cece2f80 100644
--- a/drivers/fpga/of-fpga-region.c
+++ b/drivers/fpga/of-fpga-region.c
@@ -168,11 +168,10 @@ static int child_regions_with_firmware(struct device_node *overlay)
fpga_region_of_match);
}
- of_node_put(child_region);
-
if (ret)
pr_err("firmware-name not allowed in child FPGA region: %pOF",
child_region);
+ of_node_put(child_region);
return ret;
}
diff --git a/drivers/gpio/gpiolib.c b/drivers/gpio/gpiolib.c
index 5c8cd8165696..878f9ab4a098 100644
--- a/drivers/gpio/gpiolib.c
+++ b/drivers/gpio/gpiolib.c
@@ -785,13 +785,15 @@ static const struct device_type gpio_dev_type = {
#define gcdev_unregister(gdev) device_del(&(gdev)->dev)
#endif
+/*
+ * An initial reference count has been held in gpiochip_add_data_with_key().
+ * The caller should drop the reference via gpio_device_put() on errors.
+ */
static int gpiochip_setup_dev(struct gpio_device *gdev)
{
struct fwnode_handle *fwnode = dev_fwnode(&gdev->dev);
int ret;
- device_initialize(&gdev->dev);
-
/*
* If fwnode doesn't belong to another device, it's safe to clear its
* initialized flag.
@@ -859,9 +861,11 @@ static void gpiochip_setup_devs(void)
list_for_each_entry_srcu(gdev, &gpio_devices, list,
srcu_read_lock_held(&gpio_devices_srcu)) {
ret = gpiochip_setup_dev(gdev);
- if (ret)
+ if (ret) {
+ gpio_device_put(gdev);
dev_err(&gdev->dev,
"Failed to initialize gpio device (%d)\n", ret);
+ }
}
}
@@ -883,6 +887,21 @@ void *gpiochip_get_data(struct gpio_chip *gc)
}
EXPORT_SYMBOL_GPL(gpiochip_get_data);
+/*
+ * If the calling driver provides the specific firmware node,
+ * use it. Otherwise use the one from the parent device, if any.
+ */
+static struct fwnode_handle *gpiochip_choose_fwnode(struct gpio_chip *gc)
+{
+ if (gc->fwnode)
+ return gc->fwnode;
+
+ if (gc->parent)
+ return dev_fwnode(gc->parent);
+
+ return NULL;
+}
+
int gpiochip_get_ngpios(struct gpio_chip *gc, struct device *dev)
{
u32 ngpios = gc->ngpio;
@@ -924,80 +943,73 @@ int gpiochip_add_data_with_key(struct gpio_chip *gc, void *data,
struct gpio_device *gdev;
unsigned int desc_index;
int base = 0;
- int ret = 0;
+ int ret;
- /*
- * First: allocate and populate the internal stat container, and
- * set up the struct device.
- */
gdev = kzalloc(sizeof(*gdev), GFP_KERNEL);
if (!gdev)
return -ENOMEM;
-
- gdev->dev.type = &gpio_dev_type;
- gdev->dev.bus = &gpio_bus_type;
- gdev->dev.parent = gc->parent;
- rcu_assign_pointer(gdev->chip, gc);
-
gc->gpiodev = gdev;
gpiochip_set_data(gc, data);
- /*
- * If the calling driver did not initialize firmware node,
- * do it here using the parent device, if any.
- */
- if (gc->fwnode)
- device_set_node(&gdev->dev, gc->fwnode);
- else if (gc->parent)
- device_set_node(&gdev->dev, dev_fwnode(gc->parent));
-
- gdev->id = ida_alloc(&gpio_ida, GFP_KERNEL);
- if (gdev->id < 0) {
- ret = gdev->id;
+ ret = ida_alloc(&gpio_ida, GFP_KERNEL);
+ if (ret < 0)
goto err_free_gdev;
- }
+ gdev->id = ret;
- ret = dev_set_name(&gdev->dev, GPIOCHIP_NAME "%d", gdev->id);
+ ret = init_srcu_struct(&gdev->srcu);
if (ret)
goto err_free_ida;
+ rcu_assign_pointer(gdev->chip, gc);
- if (gc->parent && gc->parent->driver)
- gdev->owner = gc->parent->driver->owner;
- else if (gc->owner)
- /* TODO: remove chip->owner */
- gdev->owner = gc->owner;
- else
- gdev->owner = THIS_MODULE;
+ ret = init_srcu_struct(&gdev->desc_srcu);
+ if (ret)
+ goto err_cleanup_gdev_srcu;
+
+ ret = dev_set_name(&gdev->dev, GPIOCHIP_NAME "%d", gdev->id);
+ if (ret)
+ goto err_cleanup_desc_srcu;
+
+ device_initialize(&gdev->dev);
+ /*
+ * After this point any allocated resources to `gdev` will be
+ * free():ed by gpiodev_release(). If you add new resources
+ * then make sure they get free():ed there.
+ */
+ gdev->dev.type = &gpio_dev_type;
+ gdev->dev.bus = &gpio_bus_type;
+ gdev->dev.parent = gc->parent;
+ device_set_node(&gdev->dev, gpiochip_choose_fwnode(gc));
ret = gpiochip_get_ngpios(gc, &gdev->dev);
if (ret)
- goto err_free_dev_name;
+ goto err_put_device;
+ gdev->ngpio = gc->ngpio;
gdev->descs = kcalloc(gc->ngpio, sizeof(*gdev->descs), GFP_KERNEL);
if (!gdev->descs) {
ret = -ENOMEM;
- goto err_free_dev_name;
+ goto err_put_device;
}
gdev->label = kstrdup_const(gc->label ?: "unknown", GFP_KERNEL);
if (!gdev->label) {
ret = -ENOMEM;
- goto err_free_descs;
+ goto err_put_device;
}
- gdev->ngpio = gc->ngpio;
gdev->can_sleep = gc->can_sleep;
-
BLOCKING_INIT_NOTIFIER_HEAD(&gdev->line_state_notifier);
BLOCKING_INIT_NOTIFIER_HEAD(&gdev->device_notifier);
-
- ret = init_srcu_struct(&gdev->srcu);
- if (ret)
- goto err_free_label;
-
- ret = init_srcu_struct(&gdev->desc_srcu);
- if (ret)
- goto err_cleanup_gdev_srcu;
+#ifdef CONFIG_PINCTRL
+ INIT_LIST_HEAD(&gdev->pin_ranges);
+#endif
+ if (gc->parent && gc->parent->driver)
+ gdev->owner = gc->parent->driver->owner;
+ else if (gc->owner)
+ /* TODO: remove chip->owner */
+ gdev->owner = gc->owner;
+ else
+ gdev->owner = THIS_MODULE;
scoped_guard(mutex, &gpio_devices_lock) {
/*
@@ -1013,7 +1025,7 @@ int gpiochip_add_data_with_key(struct gpio_chip *gc, void *data,
if (base < 0) {
ret = base;
base = 0;
- goto err_cleanup_desc_srcu;
+ goto err_put_device;
}
/*
@@ -1033,14 +1045,10 @@ int gpiochip_add_data_with_key(struct gpio_chip *gc, void *data,
ret = gpiodev_add_to_list_unlocked(gdev);
if (ret) {
chip_err(gc, "GPIO integer space overlap, cannot add chip\n");
- goto err_cleanup_desc_srcu;
+ goto err_put_device;
}
}
-#ifdef CONFIG_PINCTRL
- INIT_LIST_HEAD(&gdev->pin_ranges);
-#endif
-
if (gc->names)
gpiochip_set_desc_names(gc);
@@ -1121,25 +1129,19 @@ int gpiochip_add_data_with_key(struct gpio_chip *gc, void *data,
scoped_guard(mutex, &gpio_devices_lock)
list_del_rcu(&gdev->list);
synchronize_srcu(&gpio_devices_srcu);
- if (gdev->dev.release) {
- /* release() has been registered by gpiochip_setup_dev() */
- gpio_device_put(gdev);
- goto err_print_message;
- }
+err_put_device:
+ gpio_device_put(gdev);
+ goto err_print_message;
+
err_cleanup_desc_srcu:
cleanup_srcu_struct(&gdev->desc_srcu);
err_cleanup_gdev_srcu:
cleanup_srcu_struct(&gdev->srcu);
-err_free_label:
- kfree_const(gdev->label);
-err_free_descs:
- kfree(gdev->descs);
-err_free_dev_name:
- kfree(dev_name(&gdev->dev));
err_free_ida:
ida_free(&gpio_ida, gdev->id);
err_free_gdev:
kfree(gdev);
+
err_print_message:
/* failures here can mean systems won't boot... */
if (ret != -EPROBE_DEFER) {
@@ -2874,7 +2876,7 @@ EXPORT_SYMBOL_GPL(gpiod_direction_output);
*/
int gpiod_enable_hw_timestamp_ns(struct gpio_desc *desc, unsigned long flags)
{
- int ret = 0;
+ int ret;
VALIDATE_DESC(desc);
@@ -2907,7 +2909,7 @@ EXPORT_SYMBOL_GPL(gpiod_enable_hw_timestamp_ns);
*/
int gpiod_disable_hw_timestamp_ns(struct gpio_desc *desc, unsigned long flags)
{
- int ret = 0;
+ int ret;
VALIDATE_DESC(desc);
diff --git a/drivers/gpu/drm/v3d/v3d_drv.h b/drivers/gpu/drm/v3d/v3d_drv.h
index d4b0549205c2..b6e11968fba4 100644
--- a/drivers/gpu/drm/v3d/v3d_drv.h
+++ b/drivers/gpu/drm/v3d/v3d_drv.h
@@ -59,6 +59,9 @@ struct v3d_queue_state {
/* Stores the GPU stats for this queue in the global context. */
struct v3d_stats stats;
+
+ /* Currently active job for this queue */
+ struct v3d_job *active_job;
};
/* Performance monitor object. The perform lifetime is controlled by userspace
@@ -147,10 +150,6 @@ struct v3d_dev {
struct work_struct overflow_mem_work;
- struct v3d_bin_job *bin_job;
- struct v3d_render_job *render_job;
- struct v3d_tfu_job *tfu_job;
- struct v3d_csd_job *csd_job;
struct v3d_cpu_job *cpu_job;
struct v3d_queue_state queue[V3D_MAX_QUEUES];
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index 6b6ba7a68fcb..cf3b93101429 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -304,16 +304,15 @@ void
v3d_gem_destroy(struct drm_device *dev)
{
struct v3d_dev *v3d = to_v3d_dev(dev);
+ enum v3d_queue q;
v3d_sched_fini(v3d);
/* Waiting for jobs to finish would need to be done before
* unregistering V3D.
*/
- WARN_ON(v3d->bin_job);
- WARN_ON(v3d->render_job);
- WARN_ON(v3d->tfu_job);
- WARN_ON(v3d->csd_job);
+ for (q = 0; q < V3D_MAX_QUEUES; q++)
+ WARN_ON(v3d->queue[q].active_job);
drm_mm_takedown(&v3d->mm);
diff --git a/drivers/gpu/drm/v3d/v3d_irq.c b/drivers/gpu/drm/v3d/v3d_irq.c
index b98e1a4b33c7..2464ea4d935d 100644
--- a/drivers/gpu/drm/v3d/v3d_irq.c
+++ b/drivers/gpu/drm/v3d/v3d_irq.c
@@ -42,6 +42,8 @@ v3d_overflow_mem_work(struct work_struct *work)
container_of(work, struct v3d_dev, overflow_mem_work);
struct drm_device *dev = &v3d->drm;
struct v3d_bo *bo = v3d_bo_create(dev, NULL /* XXX: GMP */, 256 * 1024);
+ struct v3d_queue_state *queue = &v3d->queue[V3D_BIN];
+ struct v3d_bin_job *bin_job;
struct drm_gem_object *obj;
unsigned long irqflags;
@@ -61,13 +63,15 @@ v3d_overflow_mem_work(struct work_struct *work)
* some binner pool anyway.
*/
spin_lock_irqsave(&v3d->job_lock, irqflags);
- if (!v3d->bin_job) {
+ bin_job = (struct v3d_bin_job *)queue->active_job;
+
+ if (!bin_job) {
spin_unlock_irqrestore(&v3d->job_lock, irqflags);
goto out;
}
drm_gem_object_get(obj);
- list_add_tail(&bo->unref_head, &v3d->bin_job->render->unref_list);
+ list_add_tail(&bo->unref_head, &bin_job->render->unref_list);
spin_unlock_irqrestore(&v3d->job_lock, irqflags);
v3d_mmu_flush_all(v3d);
@@ -79,6 +83,20 @@ v3d_overflow_mem_work(struct work_struct *work)
drm_gem_object_put(obj);
}
+static void
+v3d_irq_signal_fence(struct v3d_dev *v3d, enum v3d_queue q,
+ void (*trace_irq)(struct drm_device *, uint64_t))
+{
+ struct v3d_queue_state *queue = &v3d->queue[q];
+ struct v3d_fence *fence = to_v3d_fence(queue->active_job->irq_fence);
+
+ v3d_job_update_stats(queue->active_job, q);
+ trace_irq(&v3d->drm, fence->seqno);
+
+ queue->active_job = NULL;
+ dma_fence_signal(&fence->base);
+}
+
static irqreturn_t
v3d_irq(int irq, void *arg)
{
@@ -102,41 +120,17 @@ v3d_irq(int irq, void *arg)
}
if (intsts & V3D_INT_FLDONE) {
- struct v3d_fence *fence =
- to_v3d_fence(v3d->bin_job->base.irq_fence);
-
- v3d_job_update_stats(&v3d->bin_job->base, V3D_BIN);
- trace_v3d_bcl_irq(&v3d->drm, fence->seqno);
-
- v3d->bin_job = NULL;
- dma_fence_signal(&fence->base);
-
+ v3d_irq_signal_fence(v3d, V3D_BIN, trace_v3d_bcl_irq);
status = IRQ_HANDLED;
}
if (intsts & V3D_INT_FRDONE) {
- struct v3d_fence *fence =
- to_v3d_fence(v3d->render_job->base.irq_fence);
-
- v3d_job_update_stats(&v3d->render_job->base, V3D_RENDER);
- trace_v3d_rcl_irq(&v3d->drm, fence->seqno);
-
- v3d->render_job = NULL;
- dma_fence_signal(&fence->base);
-
+ v3d_irq_signal_fence(v3d, V3D_RENDER, trace_v3d_rcl_irq);
status = IRQ_HANDLED;
}
if (intsts & V3D_INT_CSDDONE(v3d->ver)) {
- struct v3d_fence *fence =
- to_v3d_fence(v3d->csd_job->base.irq_fence);
-
- v3d_job_update_stats(&v3d->csd_job->base, V3D_CSD);
- trace_v3d_csd_irq(&v3d->drm, fence->seqno);
-
- v3d->csd_job = NULL;
- dma_fence_signal(&fence->base);
-
+ v3d_irq_signal_fence(v3d, V3D_CSD, trace_v3d_csd_irq);
status = IRQ_HANDLED;
}
@@ -168,15 +162,7 @@ v3d_hub_irq(int irq, void *arg)
V3D_WRITE(V3D_HUB_INT_CLR, intsts);
if (intsts & V3D_HUB_INT_TFUC) {
- struct v3d_fence *fence =
- to_v3d_fence(v3d->tfu_job->base.irq_fence);
-
- v3d_job_update_stats(&v3d->tfu_job->base, V3D_TFU);
- trace_v3d_tfu_irq(&v3d->drm, fence->seqno);
-
- v3d->tfu_job = NULL;
- dma_fence_signal(&fence->base);
-
+ v3d_irq_signal_fence(v3d, V3D_TFU, trace_v3d_tfu_irq);
status = IRQ_HANDLED;
}
diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c
index 8e3c8ffc2a42..f2bac920af89 100644
--- a/drivers/gpu/drm/v3d/v3d_sched.c
+++ b/drivers/gpu/drm/v3d/v3d_sched.c
@@ -208,14 +208,18 @@ static struct dma_fence *v3d_bin_job_run(struct drm_sched_job *sched_job)
struct dma_fence *fence;
unsigned long irqflags;
- if (unlikely(job->base.base.s_fence->finished.error))
+ if (unlikely(job->base.base.s_fence->finished.error)) {
+ spin_lock_irqsave(&v3d->job_lock, irqflags);
+ v3d->queue[V3D_BIN].active_job = NULL;
+ spin_unlock_irqrestore(&v3d->job_lock, irqflags);
return NULL;
+ }
/* Lock required around bin_job update vs
* v3d_overflow_mem_work().
*/
spin_lock_irqsave(&v3d->job_lock, irqflags);
- v3d->bin_job = job;
+ v3d->queue[V3D_BIN].active_job = &job->base;
/* Clear out the overflow allocation, so we don't
* reuse the overflow attached to a previous job.
*/
@@ -263,10 +267,12 @@ static struct dma_fence *v3d_render_job_run(struct drm_sched_job *sched_job)
struct drm_device *dev = &v3d->drm;
struct dma_fence *fence;
- if (unlikely(job->base.base.s_fence->finished.error))
+ if (unlikely(job->base.base.s_fence->finished.error)) {
+ v3d->queue[V3D_RENDER].active_job = NULL;
return NULL;
+ }
- v3d->render_job = job;
+ v3d->queue[V3D_RENDER].active_job = &job->base;
/* Can we avoid this flush? We need to be careful of
* scheduling, though -- imagine job0 rendering to texture and
@@ -309,10 +315,12 @@ v3d_tfu_job_run(struct drm_sched_job *sched_job)
struct drm_device *dev = &v3d->drm;
struct dma_fence *fence;
- if (unlikely(job->base.base.s_fence->finished.error))
+ if (unlikely(job->base.base.s_fence->finished.error)) {
+ v3d->queue[V3D_TFU].active_job = NULL;
return NULL;
+ }
- v3d->tfu_job = job;
+ v3d->queue[V3D_TFU].active_job = &job->base;
fence = v3d_fence_create(v3d, V3D_TFU);
if (IS_ERR(fence))
@@ -355,10 +363,22 @@ v3d_csd_job_run(struct drm_sched_job *sched_job)
struct dma_fence *fence;
int i, csd_cfg0_reg;
- if (unlikely(job->base.base.s_fence->finished.error))
+ if (unlikely(job->base.base.s_fence->finished.error)) {
+ v3d->queue[V3D_CSD].active_job = NULL;
+ return NULL;
+ }
+
+ /* The HW interprets a workgroup size of 0 as 65536; however, the
+ * user-space driver exposes a maximum of 65535. Therefore, a 0 in
+ * any dimension means that we have no workgroups and the compute
+ * shader should not be dispatched.
+ */
+ if (!V3D_GET_FIELD(job->args.cfg[0], V3D_CSD_QUEUED_CFG0_NUM_WGS_X) ||
+ !V3D_GET_FIELD(job->args.cfg[1], V3D_CSD_QUEUED_CFG1_NUM_WGS_Y) ||
+ !V3D_GET_FIELD(job->args.cfg[2], V3D_CSD_QUEUED_CFG2_NUM_WGS_Z))
return NULL;
- v3d->csd_job = job;
+ v3d->queue[V3D_CSD].active_job = &job->base;
v3d_invalidate_caches(v3d);
@@ -408,13 +428,13 @@ v3d_rewrite_csd_job_wg_counts_from_indirect(struct v3d_cpu_job *job)
wg_counts = (uint32_t *)(bo->vaddr + indirect_csd->offset);
- if (wg_counts[0] == 0 || wg_counts[1] == 0 || wg_counts[2] == 0)
- goto unmap_bo;
-
args->cfg[0] = wg_counts[0] << V3D_CSD_CFG012_WG_COUNT_SHIFT;
args->cfg[1] = wg_counts[1] << V3D_CSD_CFG012_WG_COUNT_SHIFT;
args->cfg[2] = wg_counts[2] << V3D_CSD_CFG012_WG_COUNT_SHIFT;
+ if (wg_counts[0] == 0 || wg_counts[1] == 0 || wg_counts[2] == 0)
+ goto unmap_bo;
+
num_batches = DIV_ROUND_UP(indirect_csd->wg_size, 16) *
(wg_counts[0] * wg_counts[1] * wg_counts[2]);
diff --git a/drivers/gpu/drm/xe/display/xe_display.c b/drivers/gpu/drm/xe/display/xe_display.c
index e164e2d71e11..de1fd4dff0e8 100644
--- a/drivers/gpu/drm/xe/display/xe_display.c
+++ b/drivers/gpu/drm/xe/display/xe_display.c
@@ -148,6 +148,15 @@ int xe_display_init_noirq(struct xe_device *xe)
intel_display_driver_early_probe(xe);
+ intel_display_device_info_runtime_init(xe);
+
+ /* Display may have been disabled at runtime init */
+ if (!has_display(xe)) {
+ xe->info.probe_display = false;
+ unset_display_features(xe);
+ return 0;
+ }
+
/* Early display init.. */
intel_opregion_setup(display);
@@ -159,8 +168,6 @@ int xe_display_init_noirq(struct xe_device *xe)
intel_bw_init_hw(xe);
- intel_display_device_info_runtime_init(xe);
-
err = intel_display_driver_probe_noirq(xe);
if (err) {
intel_opregion_cleanup(display);
diff --git a/drivers/hv/hv_kvp.c b/drivers/hv/hv_kvp.c
index 77017d951826..8019ef2cdcf2 100644
--- a/drivers/hv/hv_kvp.c
+++ b/drivers/hv/hv_kvp.c
@@ -93,7 +93,7 @@ static void kvp_send_key(struct work_struct *dummy);
static void kvp_respond_to_host(struct hv_kvp_msg *msg, int error);
static void kvp_timeout_func(struct work_struct *dummy);
static void kvp_host_handshake_func(struct work_struct *dummy);
-static void kvp_register(int);
+static int kvp_register(int);
static DECLARE_DELAYED_WORK(kvp_timeout_work, kvp_timeout_func);
static DECLARE_DELAYED_WORK(kvp_host_handshake_work, kvp_host_handshake_func);
@@ -127,24 +127,26 @@ static void kvp_register_done(void)
hv_poll_channel(kvp_transaction.recv_channel, kvp_poll_wrapper);
}
-static void
+static int
kvp_register(int reg_value)
{
struct hv_kvp_msg *kvp_msg;
char *version;
+ int ret;
kvp_msg = kzalloc(sizeof(*kvp_msg), GFP_KERNEL);
+ if (!kvp_msg)
+ return -ENOMEM;
- if (kvp_msg) {
- version = kvp_msg->body.kvp_register.version;
- kvp_msg->kvp_hdr.operation = reg_value;
- strcpy(version, HV_DRV_VERSION);
+ version = kvp_msg->body.kvp_register.version;
+ kvp_msg->kvp_hdr.operation = reg_value;
+ strcpy(version, HV_DRV_VERSION);
- hvutil_transport_send(hvt, kvp_msg, sizeof(*kvp_msg),
- kvp_register_done);
- kfree(kvp_msg);
- }
+ ret = hvutil_transport_send(hvt, kvp_msg, sizeof(*kvp_msg),
+ kvp_register_done);
+ kfree(kvp_msg);
+ return ret;
}
static void kvp_timeout_func(struct work_struct *dummy)
@@ -186,9 +188,8 @@ static int kvp_handle_handshake(struct hv_kvp_msg *msg)
*/
pr_debug("KVP: userspace daemon ver. %d connected\n",
msg->kvp_hdr.operation);
- kvp_register(dm_reg_value);
- return 0;
+ return kvp_register(dm_reg_value);
}
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 128d732665bf..0191562e606d 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -2205,8 +2205,8 @@ static acpi_status vmbus_walk_resources(struct acpi_resource *res, void *ctx)
return AE_NO_MEMORY;
/* If this range overlaps the virtual TPM, truncate it. */
- if (end > VTPM_BASE_ADDRESS && start < VTPM_BASE_ADDRESS)
- end = VTPM_BASE_ADDRESS;
+ if (end >= VTPM_BASE_ADDRESS && start < VTPM_BASE_ADDRESS)
+ end = VTPM_BASE_ADDRESS - 1;
new_res->name = "hyperv mmio";
new_res->flags = IORESOURCE_MEM;
@@ -2273,6 +2273,7 @@ static void vmbus_mmio_remove(void)
static void __maybe_unused vmbus_reserve_fb(void)
{
resource_size_t start = 0, size;
+ resource_size_t low_mmio_base;
struct pci_dev *pdev;
if (efi_enabled(EFI_BOOT)) {
@@ -2280,6 +2281,24 @@ static void __maybe_unused vmbus_reserve_fb(void)
if (IS_ENABLED(CONFIG_SYSFB)) {
start = screen_info.lfb_base;
size = max_t(__u32, screen_info.lfb_size, 0x800000);
+
+ low_mmio_base = hyperv_mmio->start;
+ if (!low_mmio_base || upper_32_bits(low_mmio_base) ||
+ (start && start < low_mmio_base)) {
+ pr_warn("Unexpected low mmio base %pa\n", &low_mmio_base);
+ } else {
+ /*
+ * If the kdump/kexec or CVM kernel's lfb_base
+ * is 0, fall back to the low mmio base.
+ */
+ if (!start)
+ start = low_mmio_base;
+ /*
+ * Reserve half of the space below 4GB for high
+ * resolutions, but cap the reservation to 128MB.
+ */
+ size = min((SZ_4G - start) / 2, SZ_128M);
+ }
}
} else {
/* Gen1 VM: get FB base from PCI */
@@ -2300,8 +2319,10 @@ static void __maybe_unused vmbus_reserve_fb(void)
pci_dev_put(pdev);
}
- if (!start)
+ if (!start) {
+ pr_warn("Unexpected framebuffer mmio base of zero\n");
return;
+ }
/*
* Make a claim for the frame buffer in the resource tree under the
@@ -2311,6 +2332,8 @@ static void __maybe_unused vmbus_reserve_fb(void)
*/
for (; !fb_mmio && (size >= 0x100000); size >>= 1)
fb_mmio = __request_region(hyperv_mmio, start, size, fb_mmio_name, 0);
+
+ pr_info("hv_mmio=%pR,%pR fb=%pR\n", hyperv_mmio, hyperv_mmio->sibling, fb_mmio);
}
/**
diff --git a/drivers/i2c/i2c-core-base.c b/drivers/i2c/i2c-core-base.c
index 75d30861ffe2..461758b9293c 100644
--- a/drivers/i2c/i2c-core-base.c
+++ b/drivers/i2c/i2c-core-base.c
@@ -1559,6 +1559,10 @@ static int i2c_register_adapter(struct i2c_adapter *adap)
pm_suspend_ignore_children(&adap->dev, true);
pm_runtime_enable(&adap->dev);
+ mutex_lock(&core_lock);
+ idr_replace(&i2c_adapter_idr, adap, adap->nr);
+ mutex_unlock(&core_lock);
+
res = device_add(&adap->dev);
if (res) {
pr_err("adapter '%s': can't register device (%d)\n", adap->name, res);
@@ -1617,7 +1621,7 @@ static int __i2c_add_numbered_adapter(struct i2c_adapter *adap)
int id;
mutex_lock(&core_lock);
- id = idr_alloc(&i2c_adapter_idr, adap, adap->nr, adap->nr + 1, GFP_KERNEL);
+ id = idr_alloc(&i2c_adapter_idr, NULL, adap->nr, adap->nr + 1, GFP_KERNEL);
mutex_unlock(&core_lock);
if (WARN(id < 0, "couldn't get idr"))
return id == -ENOSPC ? -EBUSY : id;
@@ -1653,7 +1657,7 @@ int i2c_add_adapter(struct i2c_adapter *adapter)
}
mutex_lock(&core_lock);
- id = idr_alloc(&i2c_adapter_idr, adapter,
+ id = idr_alloc(&i2c_adapter_idr, NULL,
__i2c_first_dynamic_bus_num, 0, GFP_KERNEL);
mutex_unlock(&core_lock);
if (WARN(id < 0, "couldn't get idr"))
diff --git a/drivers/i2c/i2c-stub.c b/drivers/i2c/i2c-stub.c
index 09e7b7bf4c5f..a6e2ce4360ca 100644
--- a/drivers/i2c/i2c-stub.c
+++ b/drivers/i2c/i2c-stub.c
@@ -214,6 +214,11 @@ static s32 stub_xfer(struct i2c_adapter *adap, u16 addr, unsigned short flags,
* We ignore banks here, because banked chips don't use I2C
* block transfers
*/
+ if (data->block[0] == 0 ||
+ data->block[0] > I2C_SMBUS_BLOCK_MAX) {
+ ret = -EINVAL;
+ break;
+ }
if (data->block[0] > 256 - command) /* Avoid overrun */
data->block[0] = 256 - command;
len = data->block[0];
diff --git a/drivers/iio/adc/ti-ads1298.c b/drivers/iio/adc/ti-ads1298.c
index d00cd169e8df..00d93fc043e2 100644
--- a/drivers/iio/adc/ti-ads1298.c
+++ b/drivers/iio/adc/ti-ads1298.c
@@ -279,6 +279,7 @@ static const u8 ads1298_pga_settings[] = { 6, 1, 2, 3, 4, 8, 12 };
static int ads1298_get_scale(struct ads1298_private *priv,
int channel, int *val, int *val2)
{
+ unsigned int pga_idx;
int ret;
unsigned int regval;
u8 gain;
@@ -302,7 +303,11 @@ static int ads1298_get_scale(struct ads1298_private *priv,
if (ret)
return ret;
- gain = ads1298_pga_settings[FIELD_GET(ADS1298_MASK_CH_PGA, regval)];
+ pga_idx = FIELD_GET(ADS1298_MASK_CH_PGA, regval);
+ if (pga_idx >= ARRAY_SIZE(ads1298_pga_settings))
+ return -EINVAL;
+
+ gain = ads1298_pga_settings[pga_idx];
*val /= gain; /* Full scale is VREF / gain */
*val2 = ADS1298_BITS_PER_SAMPLE - 1; /* Signed, hence the -1 */
diff --git a/drivers/iio/light/bh1780.c b/drivers/iio/light/bh1780.c
index 475f44954f61..f478f12640d5 100644
--- a/drivers/iio/light/bh1780.c
+++ b/drivers/iio/light/bh1780.c
@@ -109,10 +109,10 @@ static int bh1780_read_raw(struct iio_dev *indio_dev,
case IIO_LIGHT:
pm_runtime_get_sync(&bh1780->client->dev);
value = bh1780_read_word(bh1780, BH1780_REG_DLOW);
- if (value < 0)
- return value;
pm_runtime_mark_last_busy(&bh1780->client->dev);
pm_runtime_put_autosuspend(&bh1780->client->dev);
+ if (value < 0)
+ return value;
*val = value;
return IIO_VAL_INT;
diff --git a/drivers/iio/light/veml6075.c b/drivers/iio/light/veml6075.c
index 859891e8f115..876abac9acde 100644
--- a/drivers/iio/light/veml6075.c
+++ b/drivers/iio/light/veml6075.c
@@ -100,7 +100,7 @@ static const struct iio_chan_spec veml6075_channels[] = {
static int veml6075_request_measurement(struct veml6075_data *data)
{
- int ret, conf, int_time;
+ int ret, conf, int_time, int_index;
ret = regmap_read(data->regmap, VEML6075_CMD_CONF, &conf);
if (ret < 0)
@@ -117,7 +117,11 @@ static int veml6075_request_measurement(struct veml6075_data *data)
* time for all possible configurations. Using a 1.50 factor simplifies
* operations and ensures reliability under all circumstances.
*/
- int_time = veml6075_it_ms[FIELD_GET(VEML6075_CONF_IT, conf)];
+ int_index = FIELD_GET(VEML6075_CONF_IT, conf);
+ if (int_index >= ARRAY_SIZE(veml6075_it_ms))
+ return -EINVAL;
+
+ int_time = veml6075_it_ms[int_index];
msleep(int_time + (int_time / 2));
/* shutdown again, data registers are still accessible */
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index c1587845f280..7ad5d88358b0 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -4231,7 +4231,7 @@ int bnxt_re_alloc_ucontext(struct ib_ucontext *ctx, struct ib_udata *udata)
uctx->rdev = rdev;
- uctx->shpg = (void *)__get_free_page(GFP_KERNEL);
+ uctx->shpg = (void *)get_zeroed_page(GFP_KERNEL);
if (!uctx->shpg) {
rc = -ENOMEM;
goto fail;
diff --git a/drivers/irqchip/irq-imgpdc.c b/drivers/irqchip/irq-imgpdc.c
index b42ed68acfa6..602cea5e52d2 100644
--- a/drivers/irqchip/irq-imgpdc.c
+++ b/drivers/irqchip/irq-imgpdc.c
@@ -378,6 +378,7 @@ static int pdc_intc_probe(struct platform_device *pdev)
dev_err(&pdev->dev, "cannot add IRQ domain\n");
return -ENOMEM;
}
+ priv->domain->flags |= IRQ_DOMAIN_FLAG_DESTROY_GC;
/*
* Set up 2 generic irq chips with 2 chip types.
@@ -465,6 +466,11 @@ static void pdc_intc_remove(struct platform_device *pdev)
{
struct pdc_intc_priv *priv = platform_get_drvdata(pdev);
+ for (unsigned int i = 0; i < priv->nr_perips; ++i)
+ irq_set_chained_handler_and_data(priv->perip_irqs[i], NULL, NULL);
+
+ irq_set_chained_handler_and_data(priv->syswake_irq, NULL, NULL);
+
irq_domain_remove(priv->domain);
}
diff --git a/drivers/media/test-drivers/vidtv/vidtv_mux.c b/drivers/media/test-drivers/vidtv/vidtv_mux.c
index 7dad97881fdb..00ae52dbc96e 100644
--- a/drivers/media/test-drivers/vidtv/vidtv_mux.c
+++ b/drivers/media/test-drivers/vidtv/vidtv_mux.c
@@ -101,7 +101,8 @@ static int vidtv_mux_pid_ctx_init(struct vidtv_mux *m)
/* add a ctx for all PMT sections */
while (p) {
pid = vidtv_psi_get_pat_program_pid(p);
- vidtv_mux_create_pid_ctx_once(m, pid);
+ if (!vidtv_mux_create_pid_ctx_once(m, pid))
+ goto free;
p = p->next;
}
@@ -170,6 +171,9 @@ static u32 vidtv_mux_push_si(struct vidtv_mux *m)
nit_ctx = vidtv_mux_get_pid_ctx(m, VIDTV_NIT_PID);
eit_ctx = vidtv_mux_get_pid_ctx(m, VIDTV_EIT_PID);
+ if (!pat_ctx || !sdt_ctx || !nit_ctx || !eit_ctx)
+ return 0;
+
pat_args.offset = m->mux_buf_offset;
pat_args.continuity_counter = &pat_ctx->cc;
@@ -186,6 +190,8 @@ static u32 vidtv_mux_push_si(struct vidtv_mux *m)
}
pmt_ctx = vidtv_mux_get_pid_ctx(m, pmt_pid);
+ if (!pmt_ctx)
+ continue;
pmt_args.offset = m->mux_buf_offset;
pmt_args.pmt = m->si.pmt_secs[i];
diff --git a/drivers/mtd/spi-nor/macronix.c b/drivers/mtd/spi-nor/macronix.c
index ea6be95e75a5..6127565372c5 100644
--- a/drivers/mtd/spi-nor/macronix.c
+++ b/drivers/mtd/spi-nor/macronix.c
@@ -28,8 +28,26 @@ mx25l25635_post_bfpt_fixups(struct spi_nor *nor,
return 0;
}
+static int
+macronix_qpp4b_post_sfdp_fixups(struct spi_nor *nor)
+{
+ /* PP_1_1_4_4B is supported but missing in 4BAIT. */
+ struct spi_nor_flash_parameter *params = nor->params;
+
+ params->hwcaps.mask |= SNOR_HWCAPS_PP_1_1_4;
+ spi_nor_set_pp_settings(¶ms->page_programs[SNOR_CMD_PP_1_1_4],
+ SPINOR_OP_PP_1_1_4_4B, SNOR_PROTO_1_1_4);
+
+ return 0;
+}
+
static const struct spi_nor_fixups mx25l25635_fixups = {
.post_bfpt = mx25l25635_post_bfpt_fixups,
+ .post_sfdp = macronix_qpp4b_post_sfdp_fixups,
+};
+
+static const struct spi_nor_fixups macronix_qpp4b_fixups = {
+ .post_sfdp = macronix_qpp4b_post_sfdp_fixups,
};
static const struct flash_info macronix_nor_parts[] = {
@@ -85,11 +103,17 @@ static const struct flash_info macronix_nor_parts[] = {
.size = SZ_64M,
.no_sfdp_flags = SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ,
.fixup_flags = SPI_NOR_4B_OPCODES,
+ .fixups = ¯onix_qpp4b_fixups,
}, {
.id = SNOR_ID(0xc2, 0x20, 0x1b),
.name = "mx66l1g45g",
.size = SZ_128M,
.no_sfdp_flags = SECT_4K | SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ,
+ .fixups = ¯onix_qpp4b_fixups,
+ }, {
+ /* MX66L2G45G */
+ .id = SNOR_ID(0xc2, 0x20, 0x1c),
+ .fixups = ¯onix_qpp4b_fixups,
}, {
.id = SNOR_ID(0xc2, 0x23, 0x14),
.name = "mx25v8035f",
@@ -137,18 +161,25 @@ static const struct flash_info macronix_nor_parts[] = {
.size = SZ_64M,
.no_sfdp_flags = SECT_4K | SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ,
.fixup_flags = SPI_NOR_4B_OPCODES,
+ .fixups = ¯onix_qpp4b_fixups,
}, {
.id = SNOR_ID(0xc2, 0x25, 0x3a),
.name = "mx66u51235f",
.size = SZ_64M,
.no_sfdp_flags = SECT_4K | SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ,
.fixup_flags = SPI_NOR_4B_OPCODES,
+ .fixups = ¯onix_qpp4b_fixups,
+ }, {
+ /* MX66U1G45G */
+ .id = SNOR_ID(0xc2, 0x25, 0x3b),
+ .fixups = ¯onix_qpp4b_fixups,
}, {
.id = SNOR_ID(0xc2, 0x25, 0x3c),
.name = "mx66u2g45g",
.size = SZ_256M,
.no_sfdp_flags = SECT_4K | SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ,
.fixup_flags = SPI_NOR_4B_OPCODES,
+ .fixups = ¯onix_qpp4b_fixups,
}, {
.id = SNOR_ID(0xc2, 0x26, 0x18),
.name = "mx25l12855e",
diff --git a/drivers/net/bonding/bond_3ad.c b/drivers/net/bonding/bond_3ad.c
index 4c2560ae8866..b777902ed850 100644
--- a/drivers/net/bonding/bond_3ad.c
+++ b/drivers/net/bonding/bond_3ad.c
@@ -436,6 +436,7 @@ static void __ad_actor_update_port(struct port *port)
port->actor_system = BOND_AD_INFO(bond).system.sys_mac_addr;
port->actor_system_priority = BOND_AD_INFO(bond).system.sys_priority;
+ port->actor_port_priority = SLAVE_AD_INFO(port->slave)->port_priority;
}
/* Conversions */
@@ -990,6 +991,7 @@ static int ad_marker_send(struct port *port, struct bond_marker *marker)
static void ad_mux_machine(struct port *port, bool *update_slave_arr)
{
struct bonding *bond = __get_bond_by_port(port);
+ struct aggregator *aggregator;
mux_states_t last_state;
/* keep current State Machine state to compare later if it was
@@ -997,6 +999,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
*/
last_state = port->sm_mux_state;
+ aggregator = rcu_dereference(port->aggregator);
if (port->sm_vars & AD_PORT_BEGIN) {
port->sm_mux_state = AD_MUX_DETACHED;
} else {
@@ -1016,7 +1019,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
* cycle to update ready variable, we check
* READY_N and update READY here
*/
- __set_agg_ports_ready(port->aggregator, __agg_ports_are_ready(port->aggregator));
+ __set_agg_ports_ready(aggregator, __agg_ports_are_ready(aggregator));
port->sm_mux_state = AD_MUX_DETACHED;
break;
}
@@ -1031,7 +1034,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
* update ready variable, we check READY_N and update
* READY here
*/
- __set_agg_ports_ready(port->aggregator, __agg_ports_are_ready(port->aggregator));
+ __set_agg_ports_ready(aggregator, __agg_ports_are_ready(aggregator));
/* if the wait_while_timer expired, and the port is
* in READY state, move to ATTACHED state
@@ -1047,7 +1050,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
if ((port->sm_vars & AD_PORT_SELECTED) &&
(port->partner_oper.port_state & LACP_STATE_SYNCHRONIZATION) &&
!__check_agg_selection_timer(port)) {
- if (port->aggregator->is_active) {
+ if (aggregator->is_active) {
int state = AD_MUX_COLLECTING_DISTRIBUTING;
if (!bond->params.coupled_control)
@@ -1063,9 +1066,9 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
* cycle to update ready variable, we check
* READY_N and update READY here
*/
- __set_agg_ports_ready(port->aggregator, __agg_ports_are_ready(port->aggregator));
+ __set_agg_ports_ready(aggregator, __agg_ports_are_ready(aggregator));
port->sm_mux_state = AD_MUX_DETACHED;
- } else if (port->aggregator->is_active) {
+ } else if (aggregator->is_active) {
port->actor_oper_port_state |=
LACP_STATE_SYNCHRONIZATION;
}
@@ -1076,7 +1079,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
* sure that a collecting distributing
* port in an active aggregator is enabled
*/
- if (port->aggregator->is_active &&
+ if (aggregator->is_active &&
!__port_is_collecting_distributing(port)) {
__enable_port(port);
*update_slave_arr = true;
@@ -1095,7 +1098,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
*/
struct slave *slave = port->slave;
- if (port->aggregator->is_active &&
+ if (aggregator->is_active &&
bond_is_slave_rx_disabled(slave)) {
ad_enable_collecting(port);
*update_slave_arr = true;
@@ -1115,8 +1118,8 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
* sure that a collecting distributing
* port in an active aggregator is enabled
*/
- if (port->aggregator &&
- port->aggregator->is_active &&
+ if (aggregator &&
+ aggregator->is_active &&
!__port_is_collecting_distributing(port)) {
__enable_port(port);
*update_slave_arr = true;
@@ -1148,7 +1151,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
port->sm_mux_timer_counter = __ad_timer_to_ticks(AD_WAIT_WHILE_TIMER, 0);
break;
case AD_MUX_ATTACHED:
- if (port->aggregator->is_active)
+ if (aggregator->is_active)
port->actor_oper_port_state |=
LACP_STATE_SYNCHRONIZATION;
else
@@ -1345,8 +1348,8 @@ static void ad_churn_machine(struct port *port)
{
if (port->sm_vars & AD_PORT_CHURNED) {
port->sm_vars &= ~AD_PORT_CHURNED;
- port->sm_churn_actor_state = AD_CHURN_MONITOR;
- port->sm_churn_partner_state = AD_CHURN_MONITOR;
+ WRITE_ONCE(port->sm_churn_actor_state, AD_CHURN_MONITOR);
+ WRITE_ONCE(port->sm_churn_partner_state, AD_CHURN_MONITOR);
port->sm_churn_actor_timer_counter =
__ad_timer_to_ticks(AD_ACTOR_CHURN_TIMER, 0);
port->sm_churn_partner_timer_counter =
@@ -1357,20 +1360,22 @@ static void ad_churn_machine(struct port *port)
!(--port->sm_churn_actor_timer_counter) &&
port->sm_churn_actor_state == AD_CHURN_MONITOR) {
if (port->actor_oper_port_state & LACP_STATE_SYNCHRONIZATION) {
- port->sm_churn_actor_state = AD_NO_CHURN;
+ WRITE_ONCE(port->sm_churn_actor_state, AD_NO_CHURN);
} else {
- port->churn_actor_count++;
- port->sm_churn_actor_state = AD_CHURN;
+ WRITE_ONCE(port->churn_actor_count,
+ port->churn_actor_count + 1);
+ WRITE_ONCE(port->sm_churn_actor_state, AD_CHURN);
}
}
if (port->sm_churn_partner_timer_counter &&
!(--port->sm_churn_partner_timer_counter) &&
port->sm_churn_partner_state == AD_CHURN_MONITOR) {
if (port->partner_oper.port_state & LACP_STATE_SYNCHRONIZATION) {
- port->sm_churn_partner_state = AD_NO_CHURN;
+ WRITE_ONCE(port->sm_churn_partner_state, AD_NO_CHURN);
} else {
- port->churn_partner_count++;
- port->sm_churn_partner_state = AD_CHURN;
+ WRITE_ONCE(port->churn_partner_count,
+ port->churn_partner_count + 1);
+ WRITE_ONCE(port->sm_churn_partner_state, AD_CHURN);
}
}
}
@@ -1521,9 +1526,9 @@ static void ad_port_selection_logic(struct port *port, bool *update_slave_arr)
bond = __get_bond_by_port(port);
/* if the port is connected to other aggregator, detach it */
- if (port->aggregator) {
+ temp_aggregator = rcu_dereference(port->aggregator);
+ if (temp_aggregator) {
/* detach the port from its former aggregator */
- temp_aggregator = port->aggregator;
for (curr_port = temp_aggregator->lag_ports; curr_port;
last_port = curr_port,
curr_port = curr_port->next_port_in_aggregator) {
@@ -1546,7 +1551,7 @@ static void ad_port_selection_logic(struct port *port, bool *update_slave_arr)
/* clear the port's relations to this
* aggregator
*/
- port->aggregator = NULL;
+ RCU_INIT_POINTER(port->aggregator, NULL);
port->next_port_in_aggregator = NULL;
port->actor_port_aggregator_identifier = 0;
@@ -1569,7 +1574,7 @@ static void ad_port_selection_logic(struct port *port, bool *update_slave_arr)
port->slave->bond->dev->name,
port->slave->dev->name,
port->actor_port_number,
- port->aggregator->aggregator_identifier);
+ temp_aggregator->aggregator_identifier);
}
}
/* search on all aggregators for a suitable aggregator for this port */
@@ -1593,15 +1598,15 @@ static void ad_port_selection_logic(struct port *port, bool *update_slave_arr)
)
) {
/* attach to the founded aggregator */
- port->aggregator = aggregator;
+ rcu_assign_pointer(port->aggregator, aggregator);
port->actor_port_aggregator_identifier =
- port->aggregator->aggregator_identifier;
+ aggregator->aggregator_identifier;
port->next_port_in_aggregator = aggregator->lag_ports;
- port->aggregator->num_of_ports++;
+ aggregator->num_of_ports++;
aggregator->lag_ports = port;
slave_dbg(bond->dev, slave->dev, "Port %d joined LAG %d (existing LAG)\n",
port->actor_port_number,
- port->aggregator->aggregator_identifier);
+ aggregator->aggregator_identifier);
/* mark this port as selected */
port->sm_vars |= AD_PORT_SELECTED;
@@ -1616,39 +1621,40 @@ static void ad_port_selection_logic(struct port *port, bool *update_slave_arr)
if (!found) {
if (free_aggregator) {
/* assign port a new aggregator */
- port->aggregator = free_aggregator;
port->actor_port_aggregator_identifier =
- port->aggregator->aggregator_identifier;
+ free_aggregator->aggregator_identifier;
/* update the new aggregator's parameters
* if port was responsed from the end-user
*/
if (port->actor_oper_port_key & AD_DUPLEX_KEY_MASKS)
/* if port is full duplex */
- port->aggregator->is_individual = false;
+ free_aggregator->is_individual = false;
else
- port->aggregator->is_individual = true;
+ free_aggregator->is_individual = true;
- port->aggregator->actor_admin_aggregator_key =
+ free_aggregator->actor_admin_aggregator_key =
port->actor_admin_port_key;
- port->aggregator->actor_oper_aggregator_key =
+ free_aggregator->actor_oper_aggregator_key =
port->actor_oper_port_key;
- port->aggregator->partner_system =
+ free_aggregator->partner_system =
port->partner_oper.system;
- port->aggregator->partner_system_priority =
+ free_aggregator->partner_system_priority =
port->partner_oper.system_priority;
- port->aggregator->partner_oper_aggregator_key = port->partner_oper.key;
- port->aggregator->receive_state = 1;
- port->aggregator->transmit_state = 1;
- port->aggregator->lag_ports = port;
- port->aggregator->num_of_ports++;
+ free_aggregator->partner_oper_aggregator_key = port->partner_oper.key;
+ free_aggregator->receive_state = 1;
+ free_aggregator->transmit_state = 1;
+ free_aggregator->lag_ports = port;
+ free_aggregator->num_of_ports++;
+
+ rcu_assign_pointer(port->aggregator, free_aggregator);
/* mark this port as selected */
port->sm_vars |= AD_PORT_SELECTED;
slave_dbg(bond->dev, port->slave->dev, "Port %d joined LAG %d (new LAG)\n",
port->actor_port_number,
- port->aggregator->aggregator_identifier);
+ free_aggregator->aggregator_identifier);
} else {
slave_err(bond->dev, port->slave->dev,
"Port %d did not find a suitable aggregator\n",
@@ -1660,13 +1666,12 @@ static void ad_port_selection_logic(struct port *port, bool *update_slave_arr)
* in all aggregator's ports, else set ready=FALSE in all
* aggregator's ports
*/
- __set_agg_ports_ready(port->aggregator,
- __agg_ports_are_ready(port->aggregator));
+ aggregator = rcu_dereference(port->aggregator);
+ __set_agg_ports_ready(aggregator, __agg_ports_are_ready(aggregator));
- aggregator = __get_first_agg(port);
- ad_agg_selection_logic(aggregator, update_slave_arr);
+ ad_agg_selection_logic(__get_first_agg(port), update_slave_arr);
- if (!port->aggregator->is_active)
+ if (!aggregator->is_active)
port->actor_oper_port_state &= ~LACP_STATE_SYNCHRONIZATION;
}
@@ -2020,13 +2025,15 @@ static void ad_initialize_port(struct port *port, const struct bond_params *bond
*/
static void ad_enable_collecting(struct port *port)
{
- if (port->aggregator->is_active) {
+ struct aggregator *aggregator = rcu_dereference(port->aggregator);
+
+ if (aggregator->is_active) {
struct slave *slave = port->slave;
slave_dbg(slave->bond->dev, slave->dev,
"Enabling collecting on port %d (LAG %d)\n",
port->actor_port_number,
- port->aggregator->aggregator_identifier);
+ aggregator->aggregator_identifier);
__enable_collecting_port(port);
}
}
@@ -2038,11 +2045,13 @@ static void ad_enable_collecting(struct port *port)
*/
static void ad_disable_distributing(struct port *port, bool *update_slave_arr)
{
- if (port->aggregator && __agg_has_partner(port->aggregator)) {
+ struct aggregator *aggregator = rcu_dereference(port->aggregator);
+
+ if (aggregator && __agg_has_partner(aggregator)) {
slave_dbg(port->slave->bond->dev, port->slave->dev,
"Disabling distributing on port %d (LAG %d)\n",
port->actor_port_number,
- port->aggregator->aggregator_identifier);
+ aggregator->aggregator_identifier);
__disable_distributing_port(port);
/* Slave array needs an update */
*update_slave_arr = true;
@@ -2059,11 +2068,13 @@ static void ad_disable_distributing(struct port *port, bool *update_slave_arr)
static void ad_enable_collecting_distributing(struct port *port,
bool *update_slave_arr)
{
- if (port->aggregator->is_active) {
+ struct aggregator *aggregator = rcu_dereference(port->aggregator);
+
+ if (aggregator->is_active) {
slave_dbg(port->slave->bond->dev, port->slave->dev,
"Enabling port %d (LAG %d)\n",
port->actor_port_number,
- port->aggregator->aggregator_identifier);
+ aggregator->aggregator_identifier);
__enable_port(port);
/* Slave array needs update */
*update_slave_arr = true;
@@ -2078,11 +2089,13 @@ static void ad_enable_collecting_distributing(struct port *port,
static void ad_disable_collecting_distributing(struct port *port,
bool *update_slave_arr)
{
- if (port->aggregator && __agg_has_partner(port->aggregator)) {
+ struct aggregator *aggregator = rcu_dereference(port->aggregator);
+
+ if (aggregator && __agg_has_partner(aggregator)) {
slave_dbg(port->slave->bond->dev, port->slave->dev,
"Disabling port %d (LAG %d)\n",
port->actor_port_number,
- port->aggregator->aggregator_identifier);
+ aggregator->aggregator_identifier);
__disable_port(port);
/* Slave array needs an update */
*update_slave_arr = true;
@@ -2195,6 +2208,9 @@ void bond_3ad_bind_slave(struct slave *slave)
ad_initialize_port(port, &bond->params);
+ /* Port priority is initialized. Update it to slave's ad info */
+ SLAVE_AD_INFO(slave)->port_priority = port->actor_port_priority;
+
port->slave = slave;
port->actor_port_number = SLAVE_AD_INFO(slave)->id;
/* key is determined according to the link speed, duplex and
@@ -2319,7 +2335,7 @@ void bond_3ad_unbind_slave(struct slave *slave)
*/
for (temp_port = aggregator->lag_ports; temp_port;
temp_port = temp_port->next_port_in_aggregator) {
- temp_port->aggregator = new_aggregator;
+ rcu_assign_pointer(temp_port->aggregator, new_aggregator);
temp_port->actor_port_aggregator_identifier = new_aggregator->aggregator_identifier;
}
@@ -2788,15 +2804,16 @@ int bond_3ad_set_carrier(struct bonding *bond)
int __bond_3ad_get_active_agg_info(struct bonding *bond,
struct ad_info *ad_info)
{
- struct aggregator *aggregator = NULL;
+ struct aggregator *aggregator = NULL, *tmp;
struct list_head *iter;
struct slave *slave;
struct port *port;
bond_for_each_slave_rcu(bond, slave, iter) {
port = &(SLAVE_AD_INFO(slave)->port);
- if (port->aggregator && port->aggregator->is_active) {
- aggregator = port->aggregator;
+ tmp = rcu_dereference(port->aggregator);
+ if (tmp && tmp->is_active) {
+ aggregator = tmp;
break;
}
}
diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index 7b8555f6102a..69e79b22e319 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -211,6 +211,8 @@ atomic_t netpoll_block_tx = ATOMIC_INIT(0);
unsigned int bond_net_id __read_mostly;
+DEFINE_STATIC_KEY_FALSE(bond_bcast_neigh_enabled);
+
static const struct flow_dissector_key flow_keys_bonding_keys[] = {
{
.key_id = FLOW_DISSECTOR_KEY_CONTROL,
@@ -1468,7 +1470,7 @@ static void bond_poll_controller(struct net_device *bond_dev)
if (BOND_MODE(bond) == BOND_MODE_8023AD) {
struct aggregator *agg =
- SLAVE_AD_INFO(slave)->port.aggregator;
+ rcu_dereference(SLAVE_AD_INFO(slave)->port.aggregator);
if (agg &&
agg->aggregator_identifier != ad_info.aggregator_id)
@@ -2388,7 +2390,9 @@ int bond_enslave(struct net_device *bond_dev, struct net_device *slave_dev,
bpf_prog_inc(bond->xdp_prog);
}
- if (bond_mode_can_use_xmit_hash(bond))
+ /* broadcast mode uses the all_slaves to loop through slaves. */
+ if (bond_mode_can_use_xmit_hash(bond) ||
+ BOND_MODE(bond) == BOND_MODE_BROADCAST)
bond_update_slave_arr(bond, NULL);
bond_xdp_set_features(bond_dev);
@@ -2531,7 +2535,8 @@ static int __bond_release_one(struct net_device *bond_dev,
bond_upper_dev_unlink(bond, slave);
- if (bond_mode_can_use_xmit_hash(bond))
+ if (bond_mode_can_use_xmit_hash(bond) ||
+ BOND_MODE(bond) == BOND_MODE_BROADCAST)
bond_update_slave_arr(bond, slave);
slave_info(bond_dev, slave_dev, "Releasing %s interface\n",
@@ -4445,6 +4450,9 @@ static int bond_open(struct net_device *bond_dev)
bond_for_each_slave(bond, slave, iter)
dev_mc_add(slave->dev, lacpdu_mcast_addr);
+
+ if (bond->params.broadcast_neighbor)
+ static_branch_inc(&bond_bcast_neigh_enabled);
}
if (bond_mode_can_use_xmit_hash(bond))
@@ -4468,6 +4476,10 @@ static int bond_close(struct net_device *bond_dev)
if (bond_is_lb(bond))
bond_alb_deinitialize(bond);
+ if (BOND_MODE(bond) == BOND_MODE_8023AD &&
+ bond->params.broadcast_neighbor)
+ static_branch_dec(&bond_bcast_neigh_enabled);
+
if (bond_uses_primary(bond)) {
rcu_read_lock();
slave = rcu_dereference(bond->curr_active_slave);
@@ -5179,13 +5191,18 @@ static void bond_set_slave_arr(struct bonding *bond,
{
struct bond_up_slave *usable, *all;
- usable = rtnl_dereference(bond->usable_slaves);
- rcu_assign_pointer(bond->usable_slaves, usable_slaves);
- kfree_rcu(usable, rcu);
-
all = rtnl_dereference(bond->all_slaves);
rcu_assign_pointer(bond->all_slaves, all_slaves);
kfree_rcu(all, rcu);
+
+ if (BOND_MODE(bond) == BOND_MODE_BROADCAST) {
+ kfree_rcu(usable_slaves, rcu);
+ return;
+ }
+
+ usable = rtnl_dereference(bond->usable_slaves);
+ rcu_assign_pointer(bond->usable_slaves, usable_slaves);
+ kfree_rcu(usable, rcu);
}
static void bond_reset_slave_arr(struct bonding *bond)
@@ -5235,15 +5252,16 @@ int bond_update_slave_arr(struct bonding *bond, struct slave *skipslave)
spin_unlock_bh(&bond->mode_lock);
agg_id = ad_info.aggregator_id;
}
+ rcu_read_lock();
bond_for_each_slave(bond, slave, iter) {
if (skipslave == slave)
continue;
all_slaves->arr[all_slaves->count++] = slave;
if (BOND_MODE(bond) == BOND_MODE_8023AD) {
- struct aggregator *agg;
+ const struct aggregator *agg;
- agg = SLAVE_AD_INFO(slave)->port.aggregator;
+ agg = rcu_dereference(SLAVE_AD_INFO(slave)->port.aggregator);
if (!agg || agg->aggregator_identifier != agg_id)
continue;
}
@@ -5255,6 +5273,7 @@ int bond_update_slave_arr(struct bonding *bond, struct slave *skipslave)
usable_slaves->arr[usable_slaves->count++] = slave;
}
+ rcu_read_unlock();
bond_set_slave_arr(bond, usable_slaves, all_slaves);
return ret;
@@ -5304,6 +5323,37 @@ static struct slave *bond_xdp_xmit_3ad_xor_slave_get(struct bonding *bond,
return slaves->arr[hash % count];
}
+static bool bond_should_broadcast_neighbor(struct sk_buff *skb,
+ struct net_device *dev)
+{
+ struct bonding *bond = netdev_priv(dev);
+ struct {
+ struct ipv6hdr ip6;
+ struct icmp6hdr icmp6;
+ } *combined, _combined;
+
+ if (!static_branch_unlikely(&bond_bcast_neigh_enabled))
+ return false;
+
+ if (!bond->params.broadcast_neighbor)
+ return false;
+
+ if (skb->protocol == htons(ETH_P_ARP))
+ return true;
+
+ if (skb->protocol == htons(ETH_P_IPV6)) {
+ combined = skb_header_pointer(skb, skb_mac_header_len(skb),
+ sizeof(_combined),
+ &_combined);
+ if (combined && combined->ip6.nexthdr == NEXTHDR_ICMP &&
+ (combined->icmp6.icmp6_type == NDISC_NEIGHBOUR_SOLICITATION ||
+ combined->icmp6.icmp6_type == NDISC_NEIGHBOUR_ADVERTISEMENT))
+ return true;
+ }
+
+ return false;
+}
+
/* Use this Xmit function for 3AD as well as XOR modes. The current
* usable slave array is formed in the control path. The xmit function
* just calculates hash and sends the packet out.
@@ -5323,9 +5373,12 @@ static netdev_tx_t bond_3ad_xor_xmit(struct sk_buff *skb,
return bond_tx_drop(dev, skb);
}
-/* in broadcast mode, we send everything to all usable interfaces. */
+/* in broadcast mode, we send everything to all or usable slave interfaces.
+ * under rcu_read_lock when this function is called.
+ */
static netdev_tx_t bond_xmit_broadcast(struct sk_buff *skb,
- struct net_device *bond_dev)
+ struct net_device *bond_dev,
+ bool all_slaves)
{
struct bonding *bond = netdev_priv(bond_dev);
struct bond_up_slave *slaves;
@@ -5333,7 +5386,10 @@ static netdev_tx_t bond_xmit_broadcast(struct sk_buff *skb,
bool skb_used = false;
int slaves_count, i;
- slaves = rcu_dereference(bond->all_slaves);
+ if (all_slaves)
+ slaves = rcu_dereference(bond->all_slaves);
+ else
+ slaves = rcu_dereference(bond->usable_slaves);
slaves_count = slaves ? READ_ONCE(slaves->count) : 0;
for (i = 0; i < slaves_count; i++) {
@@ -5575,10 +5631,13 @@ static netdev_tx_t __bond_start_xmit(struct sk_buff *skb, struct net_device *dev
case BOND_MODE_ACTIVEBACKUP:
return bond_xmit_activebackup(skb, dev);
case BOND_MODE_8023AD:
+ if (bond_should_broadcast_neighbor(skb, dev))
+ return bond_xmit_broadcast(skb, dev, false);
+ fallthrough;
case BOND_MODE_XOR:
return bond_3ad_xor_xmit(skb, dev);
case BOND_MODE_BROADCAST:
- return bond_xmit_broadcast(skb, dev);
+ return bond_xmit_broadcast(skb, dev, true);
case BOND_MODE_ALB:
return bond_alb_xmit(skb, dev);
case BOND_MODE_TLB:
@@ -6454,6 +6513,7 @@ static int __init bond_check_params(struct bond_params *params)
eth_zero_addr(params->ad_actor_system);
params->ad_user_port_key = ad_user_port_key;
params->coupled_control = 1;
+ params->broadcast_neighbor = 0;
if (packets_per_slave > 0) {
params->reciprocal_packets_per_slave =
reciprocal_value(packets_per_slave);
diff --git a/drivers/net/bonding/bond_netlink.c b/drivers/net/bonding/bond_netlink.c
index 2a6a424806aa..d7fc781b642c 100644
--- a/drivers/net/bonding/bond_netlink.c
+++ b/drivers/net/bonding/bond_netlink.c
@@ -28,6 +28,9 @@ static size_t bond_get_slave_size(const struct net_device *bond_dev,
nla_total_size(sizeof(u8)) + /* IFLA_BOND_SLAVE_AD_ACTOR_OPER_PORT_STATE */
nla_total_size(sizeof(u16)) + /* IFLA_BOND_SLAVE_AD_PARTNER_OPER_PORT_STATE */
nla_total_size(sizeof(s32)) + /* IFLA_BOND_SLAVE_PRIO */
+ nla_total_size(sizeof(u16)) + /* IFLA_BOND_SLAVE_ACTOR_PORT_PRIO */
+ nla_total_size(sizeof(u8)) + /* IFLA_BOND_SLAVE_AD_CHURN_ACTOR_STATE */
+ nla_total_size(sizeof(u8)) + /* IFLA_BOND_SLAVE_AD_CHURN_PARTNER_STATE */
0;
}
@@ -63,24 +66,39 @@ static int bond_fill_slave_info(struct sk_buff *skb,
const struct port *ad_port;
ad_port = &SLAVE_AD_INFO(slave)->port;
- agg = SLAVE_AD_INFO(slave)->port.aggregator;
+ rcu_read_lock();
+ agg = rcu_dereference(SLAVE_AD_INFO(slave)->port.aggregator);
if (agg) {
if (nla_put_u16(skb, IFLA_BOND_SLAVE_AD_AGGREGATOR_ID,
agg->aggregator_identifier))
- goto nla_put_failure;
+ goto nla_put_failure_rcu;
if (nla_put_u8(skb,
IFLA_BOND_SLAVE_AD_ACTOR_OPER_PORT_STATE,
ad_port->actor_oper_port_state))
- goto nla_put_failure;
+ goto nla_put_failure_rcu;
if (nla_put_u16(skb,
IFLA_BOND_SLAVE_AD_PARTNER_OPER_PORT_STATE,
ad_port->partner_oper.port_state))
- goto nla_put_failure;
+ goto nla_put_failure_rcu;
+
+ if (nla_put_u8(skb, IFLA_BOND_SLAVE_AD_CHURN_ACTOR_STATE,
+ READ_ONCE(ad_port->sm_churn_actor_state)))
+ goto nla_put_failure_rcu;
+ if (nla_put_u8(skb, IFLA_BOND_SLAVE_AD_CHURN_PARTNER_STATE,
+ READ_ONCE(ad_port->sm_churn_partner_state)))
+ goto nla_put_failure_rcu;
}
+ rcu_read_unlock();
+
+ if (nla_put_u16(skb, IFLA_BOND_SLAVE_ACTOR_PORT_PRIO,
+ SLAVE_AD_INFO(slave)->port_priority))
+ goto nla_put_failure;
}
return 0;
+nla_put_failure_rcu:
+ rcu_read_unlock();
nla_put_failure:
return -EMSGSIZE;
}
@@ -129,6 +147,7 @@ static const struct nla_policy bond_policy[IFLA_BOND_MAX + 1] = {
static const struct nla_policy bond_slave_policy[IFLA_BOND_SLAVE_MAX + 1] = {
[IFLA_BOND_SLAVE_QUEUE_ID] = { .type = NLA_U16 },
[IFLA_BOND_SLAVE_PRIO] = { .type = NLA_S32 },
+ [IFLA_BOND_SLAVE_ACTOR_PORT_PRIO] = { .type = NLA_U16 },
};
static int bond_validate(struct nlattr *tb[], struct nlattr *data[],
@@ -179,6 +198,16 @@ static int bond_slave_changelink(struct net_device *bond_dev,
return err;
}
+ if (data[IFLA_BOND_SLAVE_ACTOR_PORT_PRIO]) {
+ u16 ad_prio = nla_get_u16(data[IFLA_BOND_SLAVE_ACTOR_PORT_PRIO]);
+
+ bond_opt_slave_initval(&newval, &slave_dev, ad_prio);
+ err = __bond_opt_set(bond, BOND_OPT_ACTOR_PORT_PRIO, &newval,
+ data[IFLA_BOND_SLAVE_ACTOR_PORT_PRIO], extack);
+ if (err)
+ return err;
+ }
+
return 0;
}
diff --git a/drivers/net/bonding/bond_options.c b/drivers/net/bonding/bond_options.c
index 33af81a55a45..4d01f1081956 100644
--- a/drivers/net/bonding/bond_options.c
+++ b/drivers/net/bonding/bond_options.c
@@ -79,6 +79,8 @@ static int bond_option_tlb_dynamic_lb_set(struct bonding *bond,
const struct bond_opt_value *newval);
static int bond_option_ad_actor_sys_prio_set(struct bonding *bond,
const struct bond_opt_value *newval);
+static int bond_option_actor_port_prio_set(struct bonding *bond,
+ const struct bond_opt_value *newval);
static int bond_option_ad_actor_system_set(struct bonding *bond,
const struct bond_opt_value *newval);
static int bond_option_ad_user_port_key_set(struct bonding *bond,
@@ -87,6 +89,8 @@ static int bond_option_missed_max_set(struct bonding *bond,
const struct bond_opt_value *newval);
static int bond_option_coupled_control_set(struct bonding *bond,
const struct bond_opt_value *newval);
+static int bond_option_broadcast_neigh_set(struct bonding *bond,
+ const struct bond_opt_value *newval);
static const struct bond_opt_value bond_mode_tbl[] = {
{ "balance-rr", BOND_MODE_ROUNDROBIN, BOND_VALFLAG_DEFAULT},
@@ -240,6 +244,12 @@ static const struct bond_opt_value bond_coupled_control_tbl[] = {
{ NULL, -1, 0},
};
+static const struct bond_opt_value bond_broadcast_neigh_tbl[] = {
+ { "off", 0, BOND_VALFLAG_DEFAULT},
+ { "on", 1, 0},
+ { NULL, -1, 0}
+};
+
static const struct bond_option bond_opts[BOND_OPT_LAST] = {
[BOND_OPT_MODE] = {
.id = BOND_OPT_MODE,
@@ -476,6 +486,13 @@ static const struct bond_option bond_opts[BOND_OPT_LAST] = {
.values = bond_ad_actor_sys_prio_tbl,
.set = bond_option_ad_actor_sys_prio_set,
},
+ [BOND_OPT_ACTOR_PORT_PRIO] = {
+ .id = BOND_OPT_ACTOR_PORT_PRIO,
+ .name = "actor_port_prio",
+ .unsuppmodes = BOND_MODE_ALL_EX(BIT(BOND_MODE_8023AD)),
+ .flags = BOND_OPTFLAG_RAWVAL,
+ .set = bond_option_actor_port_prio_set,
+ },
[BOND_OPT_AD_ACTOR_SYSTEM] = {
.id = BOND_OPT_AD_ACTOR_SYSTEM,
.name = "ad_actor_system",
@@ -513,6 +530,14 @@ static const struct bond_option bond_opts[BOND_OPT_LAST] = {
.flags = BOND_OPTFLAG_IFDOWN,
.values = bond_coupled_control_tbl,
.set = bond_option_coupled_control_set,
+ },
+ [BOND_OPT_BROADCAST_NEIGH] = {
+ .id = BOND_OPT_BROADCAST_NEIGH,
+ .name = "broadcast_neighbor",
+ .desc = "Broadcast neighbor packets to all active slaves",
+ .unsuppmodes = BOND_MODE_ALL_EX(BIT(BOND_MODE_8023AD)),
+ .values = bond_broadcast_neigh_tbl,
+ .set = bond_option_broadcast_neigh_set,
}
};
@@ -894,6 +919,13 @@ static int bond_option_mode_set(struct bonding *bond,
bond->params.arp_validate = BOND_ARP_VALIDATE_NONE;
bond->params.mode = newval->value;
+ /* When changing mode, the bond device is down, we may reduce
+ * the bond_bcast_neigh_enabled in bond_close() if broadcast_neighbor
+ * enabled in 8023ad mode. Therefore, only clear broadcast_neighbor
+ * to 0.
+ */
+ bond->params.broadcast_neighbor = 0;
+
if (bond->dev->reg_state == NETREG_REGISTERED) {
bool update = false;
@@ -1796,6 +1828,26 @@ static int bond_option_ad_actor_sys_prio_set(struct bonding *bond,
return 0;
}
+static int bond_option_actor_port_prio_set(struct bonding *bond,
+ const struct bond_opt_value *newval)
+{
+ struct slave *slave;
+
+ slave = bond_slave_get_rtnl(newval->slave_dev);
+ if (!slave) {
+ netdev_dbg(bond->dev, "%s called on NULL slave\n", __func__);
+ return -ENODEV;
+ }
+
+ netdev_dbg(newval->slave_dev, "Setting actor_port_prio to %llu\n",
+ newval->value);
+
+ SLAVE_AD_INFO(slave)->port_priority = newval->value;
+ bond_3ad_update_ad_actor_settings(bond);
+
+ return 0;
+}
+
static int bond_option_ad_actor_system_set(struct bonding *bond,
const struct bond_opt_value *newval)
{
@@ -1843,3 +1895,22 @@ static int bond_option_coupled_control_set(struct bonding *bond,
bond->params.coupled_control = newval->value;
return 0;
}
+
+static int bond_option_broadcast_neigh_set(struct bonding *bond,
+ const struct bond_opt_value *newval)
+{
+ if (bond->params.broadcast_neighbor == newval->value)
+ return 0;
+
+ bond->params.broadcast_neighbor = newval->value;
+ if (bond->dev->flags & IFF_UP) {
+ if (bond->params.broadcast_neighbor)
+ static_branch_inc(&bond_bcast_neigh_enabled);
+ else
+ static_branch_dec(&bond_bcast_neigh_enabled);
+ }
+
+ netdev_dbg(bond->dev, "Setting broadcast_neighbor to %s (%llu)\n",
+ newval->string, newval->value);
+ return 0;
+}
diff --git a/drivers/net/bonding/bond_procfs.c b/drivers/net/bonding/bond_procfs.c
index 7edf72ec816a..62a5b40b43a2 100644
--- a/drivers/net/bonding/bond_procfs.c
+++ b/drivers/net/bonding/bond_procfs.c
@@ -187,6 +187,7 @@ static void bond_info_show_master(struct seq_file *seq)
}
}
+/* Note: runs under rcu_read_lock() */
static void bond_info_show_slave(struct seq_file *seq,
const struct slave *slave)
{
@@ -213,19 +214,19 @@ static void bond_info_show_slave(struct seq_file *seq,
if (BOND_MODE(bond) == BOND_MODE_8023AD) {
const struct port *port = &SLAVE_AD_INFO(slave)->port;
- const struct aggregator *agg = port->aggregator;
+ const struct aggregator *agg = rcu_dereference(port->aggregator);
if (agg) {
seq_printf(seq, "Aggregator ID: %d\n",
agg->aggregator_identifier);
seq_printf(seq, "Actor Churn State: %s\n",
- bond_3ad_churn_desc(port->sm_churn_actor_state));
+ bond_3ad_churn_desc(READ_ONCE(port->sm_churn_actor_state)));
seq_printf(seq, "Partner Churn State: %s\n",
- bond_3ad_churn_desc(port->sm_churn_partner_state));
+ bond_3ad_churn_desc(READ_ONCE(port->sm_churn_partner_state)));
seq_printf(seq, "Actor Churned Count: %d\n",
- port->churn_actor_count);
+ READ_ONCE(port->churn_actor_count));
seq_printf(seq, "Partner Churned Count: %d\n",
- port->churn_partner_count);
+ READ_ONCE(port->churn_partner_count));
if (capable(CAP_NET_ADMIN)) {
seq_puts(seq, "details actor lacp pdu:\n");
diff --git a/drivers/net/bonding/bond_sysfs_slave.c b/drivers/net/bonding/bond_sysfs_slave.c
index 36d0e8440b5b..fc6fe7181789 100644
--- a/drivers/net/bonding/bond_sysfs_slave.c
+++ b/drivers/net/bonding/bond_sysfs_slave.c
@@ -62,10 +62,15 @@ static ssize_t ad_aggregator_id_show(struct slave *slave, char *buf)
const struct aggregator *agg;
if (BOND_MODE(slave->bond) == BOND_MODE_8023AD) {
- agg = SLAVE_AD_INFO(slave)->port.aggregator;
- if (agg)
- return sysfs_emit(buf, "%d\n",
- agg->aggregator_identifier);
+ rcu_read_lock();
+ agg = rcu_dereference(SLAVE_AD_INFO(slave)->port.aggregator);
+ if (agg) {
+ ssize_t res = sysfs_emit(buf, "%d\n",
+ agg->aggregator_identifier);
+ rcu_read_unlock();
+ return res;
+ }
+ rcu_read_unlock();
}
return sysfs_emit(buf, "N/A\n");
@@ -78,7 +83,7 @@ static ssize_t ad_actor_oper_port_state_show(struct slave *slave, char *buf)
if (BOND_MODE(slave->bond) == BOND_MODE_8023AD) {
ad_port = &SLAVE_AD_INFO(slave)->port;
- if (ad_port->aggregator)
+ if (rcu_access_pointer(ad_port->aggregator))
return sysfs_emit(buf, "%u\n",
ad_port->actor_oper_port_state);
}
@@ -93,7 +98,7 @@ static ssize_t ad_partner_oper_port_state_show(struct slave *slave, char *buf)
if (BOND_MODE(slave->bond) == BOND_MODE_8023AD) {
ad_port = &SLAVE_AD_INFO(slave)->port;
- if (ad_port->aggregator)
+ if (rcu_access_pointer(ad_port->aggregator))
return sysfs_emit(buf, "%u\n",
ad_port->partner_oper.port_state);
}
diff --git a/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.c b/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.c
index f3bea196a8f9..1c118381a0bf 100644
--- a/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.c
+++ b/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.c
@@ -210,8 +210,8 @@ static void rmnet_dellink(struct net_device *dev, struct list_head *head)
ep = rmnet_get_endpoint(real_port, mux_id);
if (ep) {
hlist_del_init_rcu(&ep->hlnode);
- rmnet_vnd_dellink(mux_id, real_port, ep);
- kfree(ep);
+ real_port->nr_rmnet_devs--;
+ kfree_rcu(ep, rcu);
}
netdev_upper_dev_unlink(real_dev, dev);
@@ -235,9 +235,9 @@ static void rmnet_force_unassociate_device(struct net_device *real_dev)
hash_for_each_safe(port->muxed_ep, bkt_ep, tmp_ep, ep, hlnode) {
unregister_netdevice_queue(ep->egress_dev, &list);
netdev_upper_dev_unlink(real_dev, ep->egress_dev);
- rmnet_vnd_dellink(ep->mux_id, port, ep);
hlist_del_init_rcu(&ep->hlnode);
- kfree(ep);
+ port->nr_rmnet_devs--;
+ kfree_rcu(ep, rcu);
}
rmnet_unregister_real_device(real_dev);
unregister_netdevice_many(&list);
diff --git a/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.h b/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.h
index ed112d51ac5a..f50fae1c6bdd 100644
--- a/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.h
+++ b/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.h
@@ -18,6 +18,7 @@ struct rmnet_endpoint {
u8 mux_id;
struct net_device *egress_dev;
struct hlist_node hlnode;
+ struct rcu_head rcu;
};
struct rmnet_egress_agg_params {
diff --git a/drivers/net/wan/hdlc_ppp.c b/drivers/net/wan/hdlc_ppp.c
index 37a3c989cba1..27b771146656 100644
--- a/drivers/net/wan/hdlc_ppp.c
+++ b/drivers/net/wan/hdlc_ppp.c
@@ -621,7 +621,6 @@ static void ppp_start(struct net_device *dev)
struct proto *proto = &ppp->protos[i];
proto->dev = dev;
- timer_setup(&proto->timer, ppp_timer, 0);
proto->state = CLOSED;
}
ppp->protos[IDX_LCP].pid = PID_LCP;
@@ -641,6 +640,15 @@ static void ppp_close(struct net_device *dev)
ppp_tx_flush();
}
+static void ppp_timer_release(struct net_device *dev)
+{
+ struct ppp *ppp = get_ppp(dev);
+ int i;
+
+ for (i = 0; i < IDX_COUNT; i++)
+ timer_shutdown_sync(&ppp->protos[i].timer);
+}
+
static struct hdlc_proto proto = {
.start = ppp_start,
.stop = ppp_stop,
@@ -649,6 +657,7 @@ static struct hdlc_proto proto = {
.ioctl = ppp_ioctl,
.netif_rx = ppp_rx,
.module = THIS_MODULE,
+ .detach = ppp_timer_release,
};
static const struct header_ops ppp_header_ops = {
@@ -659,7 +668,7 @@ static int ppp_ioctl(struct net_device *dev, struct if_settings *ifs)
{
hdlc_device *hdlc = dev_to_hdlc(dev);
struct ppp *ppp;
- int result;
+ int i, result;
switch (ifs->type) {
case IF_GET_PROTO:
@@ -687,6 +696,8 @@ static int ppp_ioctl(struct net_device *dev, struct if_settings *ifs)
return result;
ppp = get_ppp(dev);
+ for (i = 0; i < IDX_COUNT; i++)
+ timer_setup(&ppp->protos[i].timer, ppp_timer, 0);
spin_lock_init(&ppp->lock);
ppp->req_timeout = 2;
ppp->cr_retries = 10;
diff --git a/drivers/net/wireless/ath/ath11k/dp.c b/drivers/net/wireless/ath/ath11k/dp.c
index f124b7329e1a..06b574ff950e 100644
--- a/drivers/net/wireless/ath/ath11k/dp.c
+++ b/drivers/net/wireless/ath/ath11k/dp.c
@@ -1040,6 +1040,7 @@ void ath11k_dp_free(struct ath11k_base *ab)
idr_destroy(&dp->tx_ring[i].txbuf_idr);
spin_unlock_bh(&dp->tx_ring[i].tx_idr_lock);
kfree(dp->tx_ring[i].tx_status);
+ dp->tx_ring[i].tx_status = NULL;
}
/* Deinit any SOC level resource */
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ptp.c b/drivers/net/wireless/intel/iwlwifi/mvm/ptp.c
index e89259de6f4c..273a6718c907 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/ptp.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/ptp.c
@@ -316,11 +316,11 @@ void iwl_mvm_ptp_remove(struct iwl_mvm *mvm)
mvm->ptp_data.ptp_clock_info.name,
ptp_clock_index(mvm->ptp_data.ptp_clock));
+ cancel_delayed_work_sync(&mvm->ptp_data.dwork);
ptp_clock_unregister(mvm->ptp_data.ptp_clock);
mvm->ptp_data.ptp_clock = NULL;
memset(&mvm->ptp_data.ptp_clock_info, 0,
sizeof(mvm->ptp_data.ptp_clock_info));
mvm->ptp_data.last_gp2 = 0;
- cancel_delayed_work_sync(&mvm->ptp_data.dwork);
}
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/usb.c b/drivers/net/wireless/mediatek/mt76/mt76x2/usb.c
index 96cecc576a98..427b294423d3 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/usb.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/usb.c
@@ -16,6 +16,7 @@ static const struct usb_device_id mt76x2u_device_table[] = {
{ USB_DEVICE(0x0e8d, 0x7612) }, /* Aukey USBAC1200 - Alfa AWUS036ACM */
{ USB_DEVICE(0x057c, 0x8503) }, /* Avm FRITZ!WLAN AC860 */
{ USB_DEVICE(0x7392, 0xb711) }, /* Edimax EW 7722 UAC */
+ { USB_DEVICE(0x056e, 0x400a) }, /* ELECOM WDC-867SU3S */
{ USB_DEVICE(0x0e8d, 0x7632) }, /* HC-M7662BU1 */
{ USB_DEVICE(0x0471, 0x2126) }, /* LiteOn WN4516R module, nonstandard USB connector */
{ USB_DEVICE(0x0471, 0x7600) }, /* LiteOn WN4519R module, nonstandard USB connector */
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/init.c b/drivers/net/wireless/mediatek/mt76/mt7921/init.c
index 4bd533c4ba9a..276dfb9c26e0 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/init.c
@@ -137,6 +137,13 @@ mt7921_regd_notifier(struct wiphy *wiphy,
dev->mt76.region = request->dfs_region;
dev->country_ie_env = request->country_ie_env;
+ if (request->initiator == NL80211_REGDOM_SET_BY_USER) {
+ if (dev->mt76.alpha2[0] == '0' && dev->mt76.alpha2[1] == '0')
+ wiphy->regulatory_flags &= ~REGULATORY_COUNTRY_IE_IGNORE;
+ else
+ wiphy->regulatory_flags |= REGULATORY_COUNTRY_IE_IGNORE;
+ }
+
if (pm->suspended)
return;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/main.c b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
index a93ae4e44f16..99561094640f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
@@ -365,12 +365,15 @@ void mt7921_roc_abort_sync(struct mt792x_dev *dev)
{
struct mt792x_phy *phy = &dev->phy;
+ if (!test_and_clear_bit(MT76_STATE_ROC, &phy->mt76->state))
+ return;
+
del_timer_sync(&phy->roc_timer);
- cancel_work_sync(&phy->roc_work);
- if (test_and_clear_bit(MT76_STATE_ROC, &phy->mt76->state))
- ieee80211_iterate_active_interfaces(mt76_hw(dev),
- IEEE80211_IFACE_ITER_RESUME_ALL,
- mt7921_roc_iter, (void *)phy);
+ cancel_work(&phy->roc_work);
+
+ ieee80211_iterate_interfaces(mt76_hw(dev),
+ IEEE80211_IFACE_ITER_RESUME_ALL,
+ mt7921_roc_iter, (void *)phy);
}
EXPORT_SYMBOL_GPL(mt7921_roc_abort_sync);
@@ -881,6 +884,7 @@ void mt7921_mac_sta_remove(struct mt76_dev *mdev, struct ieee80211_vif *vif,
struct mt792x_dev *dev = container_of(mdev, struct mt792x_dev, mt76);
struct mt792x_sta *msta = (struct mt792x_sta *)sta->drv_priv;
+ mt7921_roc_abort_sync(dev);
mt76_connac_free_pending_tx_skbs(&dev->pm, &msta->deflink.wcid);
mt76_connac_pm_wake(&dev->mphy, &dev->pm);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/main.c b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
index 5fc95c862364..b1f54cb54e17 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
@@ -1190,6 +1190,9 @@ mt7925_mac_sta_remove_links(struct mt792x_dev *dev, struct ieee80211_vif *vif,
if (vif->type == NL80211_IFTYPE_AP)
break;
+ if (vif->type == NL80211_IFTYPE_STATION && sta->tdls)
+ continue;
+
link_sta = mt792x_sta_to_link_sta(vif, sta, link_id);
if (!link_sta)
continue;
diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/trx.h b/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/trx.h
index 1155365348f3..d5de09d75f45 100644
--- a/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/trx.h
+++ b/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/trx.h
@@ -291,7 +291,7 @@ static inline int get_rx_desc_paggr(__le32 *__pdesc)
static inline int get_rx_status_desc_rpt_sel(__le32 *__pdesc)
{
- return le32_get_bits(*(__pdesc + 1), BIT(28));
+ return le32_get_bits(*(__pdesc + 2), BIT(28));
}
static inline int get_rx_desc_rxmcs(__le32 *__pdesc)
diff --git a/drivers/net/wireless/realtek/rtw88/tx.c b/drivers/net/wireless/realtek/rtw88/tx.c
index dae7ca148865..2e66140c6482 100644
--- a/drivers/net/wireless/realtek/rtw88/tx.c
+++ b/drivers/net/wireless/realtek/rtw88/tx.c
@@ -191,6 +191,7 @@ void rtw_tx_report_purge_timer(struct timer_list *t)
void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn)
{
struct rtw_tx_report *tx_report = &rtwdev->tx_report;
+ unsigned long timeout = RTW_TX_PROBE_TIMEOUT;
unsigned long flags;
u8 *drv_data;
@@ -202,7 +203,11 @@ void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn)
__skb_queue_tail(&tx_report->queue, skb);
spin_unlock_irqrestore(&tx_report->q_lock, flags);
- mod_timer(&tx_report->purge_timer, jiffies + RTW_TX_PROBE_TIMEOUT);
+ if (rtwdev->chip->id == RTW_CHIP_TYPE_8723D &&
+ rtwdev->hci.type == RTW_HCI_TYPE_USB)
+ timeout = msecs_to_jiffies(2500);
+
+ mod_timer(&tx_report->purge_timer, jiffies + timeout);
}
EXPORT_SYMBOL(rtw_tx_report_enqueue);
diff --git a/drivers/net/wireless/realtek/rtw88/usb.c b/drivers/net/wireless/realtek/rtw88/usb.c
index 52b83191f325..b3e1e9a0fda6 100644
--- a/drivers/net/wireless/realtek/rtw88/usb.c
+++ b/drivers/net/wireless/realtek/rtw88/usb.c
@@ -398,6 +398,7 @@ static bool rtw_usb_tx_agg_skb(struct rtw_usb *rtwusb, struct sk_buff_head *list
int agg_num = 0;
unsigned int align_next = 0;
u8 qsel;
+ int ret;
if (skb_queue_empty(list))
return false;
@@ -455,7 +456,13 @@ static bool rtw_usb_tx_agg_skb(struct rtw_usb *rtwusb, struct sk_buff_head *list
tx_desc = (struct rtw_tx_desc *)skb_head->data;
qsel = le32_get_bits(tx_desc->w1, RTW_TX_DESC_W1_QSEL);
- rtw_usb_write_port(rtwdev, qsel, skb_head, rtw_usb_write_port_tx_complete, txcb);
+ ret = rtw_usb_write_port(rtwdev, qsel, skb_head,
+ rtw_usb_write_port_tx_complete, txcb);
+ if (ret) {
+ ieee80211_purge_tx_queue(rtwdev->hw, &txcb->tx_ack_queue);
+ kfree(txcb);
+ return false;
+ }
return true;
}
@@ -517,8 +524,10 @@ static int rtw_usb_write_data(struct rtw_dev *rtwdev,
ret = rtw_usb_write_port(rtwdev, qsel, skb,
rtw_usb_write_port_complete, skb);
- if (unlikely(ret))
+ if (unlikely(ret)) {
rtw_err(rtwdev, "failed to do USB write, ret=%d\n", ret);
+ dev_kfree_skb_any(skb);
+ }
return ret;
}
diff --git a/drivers/ntb/hw/epf/ntb_hw_epf.c b/drivers/ntb/hw/epf/ntb_hw_epf.c
index 2b51156e01b0..f248c03134a2 100644
--- a/drivers/ntb/hw/epf/ntb_hw_epf.c
+++ b/drivers/ntb/hw/epf/ntb_hw_epf.c
@@ -646,7 +646,8 @@ static void ntb_epf_deinit_pci(struct ntb_epf_dev *ndev)
struct pci_dev *pdev = ndev->ntb.pdev;
pci_iounmap(pdev, ndev->ctrl_reg);
- pci_iounmap(pdev, ndev->peer_spad_reg);
+ if (ndev->barno_map[BAR_PEER_SPAD] != ndev->barno_map[BAR_CONFIG])
+ pci_iounmap(pdev, ndev->peer_spad_reg);
pci_iounmap(pdev, ndev->db_reg);
pci_release_regions(pdev);
diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c
index ae0f36e270ba..5d27cd149f51 100644
--- a/drivers/pci/controller/dwc/pcie-qcom.c
+++ b/drivers/pci/controller/dwc/pcie-qcom.c
@@ -329,20 +329,15 @@ static void qcom_pcie_clear_aspm_l0s(struct dw_pcie *pci)
dw_pcie_dbi_ro_wr_dis(pci);
}
-static void qcom_pcie_set_slot_nccs(struct dw_pcie *pci)
+static void qcom_pcie_clear_hpc(struct dw_pcie *pci)
{
u16 offset = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP);
u32 val;
dw_pcie_dbi_ro_wr_en(pci);
- /*
- * Qcom PCIe Root Ports do not support generating command completion
- * notifications for the Hot-Plug commands. So set the NCCS field to
- * avoid waiting for the completions.
- */
val = readl(pci->dbi_base + offset + PCI_EXP_SLTCAP);
- val |= PCI_EXP_SLTCAP_NCCS;
+ val &= ~PCI_EXP_SLTCAP_HPC;
writel(val, pci->dbi_base + offset + PCI_EXP_SLTCAP);
dw_pcie_dbi_ro_wr_dis(pci);
@@ -537,7 +532,7 @@ static int qcom_pcie_post_init_2_1_0(struct qcom_pcie *pcie)
writel(CFG_BRIDGE_SB_INIT,
pci->dbi_base + AXI_MSTR_RESP_COMP_CTRL1);
- qcom_pcie_set_slot_nccs(pcie->pci);
+ qcom_pcie_clear_hpc(pcie->pci);
return 0;
}
@@ -617,7 +612,7 @@ static int qcom_pcie_post_init_1_0_0(struct qcom_pcie *pcie)
writel(val, pcie->parf + PARF_AXI_MSTR_WR_ADDR_HALT);
}
- qcom_pcie_set_slot_nccs(pcie->pci);
+ qcom_pcie_clear_hpc(pcie->pci);
return 0;
}
@@ -710,7 +705,7 @@ static int qcom_pcie_post_init_2_3_2(struct qcom_pcie *pcie)
val |= EN;
writel(val, pcie->parf + PARF_AXI_MSTR_WR_ADDR_HALT_V2);
- qcom_pcie_set_slot_nccs(pcie->pci);
+ qcom_pcie_clear_hpc(pcie->pci);
return 0;
}
@@ -1014,7 +1009,7 @@ static int qcom_pcie_post_init_2_7_0(struct qcom_pcie *pcie)
writel(WR_NO_SNOOP_OVERIDE_EN | RD_NO_SNOOP_OVERIDE_EN,
pcie->parf + PARF_NO_SNOOP_OVERIDE);
- qcom_pcie_set_slot_nccs(pcie->pci);
+ qcom_pcie_clear_hpc(pcie->pci);
return 0;
}
diff --git a/drivers/power/reset/linkstation-poweroff.c b/drivers/power/reset/linkstation-poweroff.c
index 02f5fdb8ffc4..e56d75bfcc43 100644
--- a/drivers/power/reset/linkstation-poweroff.c
+++ b/drivers/power/reset/linkstation-poweroff.c
@@ -163,10 +163,10 @@ static int __init linkstation_poweroff_init(void)
dn = of_find_matching_node(NULL, ls_poweroff_of_match);
if (!dn)
return -ENODEV;
- of_node_put(dn);
match = of_match_node(ls_poweroff_of_match, dn);
cfg = match->data;
+ of_node_put(dn);
dn = of_find_node_by_name(NULL, cfg->mdio_node_name);
if (!dn)
diff --git a/drivers/power/sequencing/core.c b/drivers/power/sequencing/core.c
index 70b92e1c3ac5..50aed0a5e8b2 100644
--- a/drivers/power/sequencing/core.c
+++ b/drivers/power/sequencing/core.c
@@ -990,8 +990,9 @@ static void *pwrseq_debugfs_seq_start(struct seq_file *seq, loff_t *pos)
ctx.index = *pos;
/*
- * We're holding the lock for the entire printout so no need to fiddle
- * with device reference count.
+ * Hold the lock for the entire printout to prevent device removal.
+ * Reference counts are managed by start()/next()/stop() as required
+ * by the seq_file contract.
*/
down_read(&pwrseq_sem);
@@ -999,7 +1000,7 @@ static void *pwrseq_debugfs_seq_start(struct seq_file *seq, loff_t *pos)
if (!ctx.index)
return NULL;
- return ctx.dev;
+ return get_device(ctx.dev);
}
static void *pwrseq_debugfs_seq_next(struct seq_file *seq, void *data,
@@ -1009,8 +1010,9 @@ static void *pwrseq_debugfs_seq_next(struct seq_file *seq, void *data,
++*pos;
- struct device *next __free(put_device) =
- bus_find_next_device(&pwrseq_bus, curr);
+ struct device *next = bus_find_next_device(&pwrseq_bus, curr);
+
+ put_device(curr);
return next;
}
@@ -1059,6 +1061,8 @@ static int pwrseq_debugfs_seq_show(struct seq_file *seq, void *data)
static void pwrseq_debugfs_seq_stop(struct seq_file *seq, void *data)
{
+ if (data)
+ put_device(data);
up_read(&pwrseq_sem);
}
diff --git a/drivers/regulator/core.c b/drivers/regulator/core.c
index 078d3dc50aa3..54fd06891bdc 100644
--- a/drivers/regulator/core.c
+++ b/drivers/regulator/core.c
@@ -2116,8 +2116,16 @@ static int regulator_resolve_supply(struct regulator_dev *rdev)
if (rdev->use_count) {
ret = regulator_enable(rdev->supply);
if (ret < 0) {
- _regulator_put(rdev->supply);
+ struct regulator *supply;
+
+ regulator_lock_two(rdev, rdev->supply->rdev, &ww_ctx);
+
+ supply = rdev->supply;
rdev->supply = NULL;
+
+ regulator_unlock_two(rdev, supply->rdev, &ww_ctx);
+
+ regulator_put(supply);
goto out;
}
}
diff --git a/drivers/rpmsg/rpmsg_char.c b/drivers/rpmsg/rpmsg_char.c
index 96fcdd2d7093..d918cb30db9b 100644
--- a/drivers/rpmsg/rpmsg_char.c
+++ b/drivers/rpmsg/rpmsg_char.c
@@ -104,6 +104,9 @@ static int rpmsg_ept_cb(struct rpmsg_device *rpdev, void *buf, int len,
struct rpmsg_eptdev *eptdev = priv;
struct sk_buff *skb;
+ if (!eptdev)
+ return 0;
+
skb = alloc_skb(len, GFP_ATOMIC);
if (!skb)
return -ENOMEM;
@@ -124,6 +127,9 @@ static int rpmsg_ept_flow_cb(struct rpmsg_device *rpdev, void *priv, bool enable
{
struct rpmsg_eptdev *eptdev = priv;
+ if (!eptdev)
+ return 0;
+
eptdev->remote_flow_restricted = enable;
eptdev->remote_flow_updated = true;
@@ -490,6 +496,7 @@ static int rpmsg_chrdev_probe(struct rpmsg_device *rpdev)
struct rpmsg_channel_info chinfo;
struct rpmsg_eptdev *eptdev;
struct device *dev = &rpdev->dev;
+ int ret;
memcpy(chinfo.name, rpdev->id.name, RPMSG_NAME_SIZE);
chinfo.src = rpdev->src;
@@ -502,13 +509,17 @@ static int rpmsg_chrdev_probe(struct rpmsg_device *rpdev)
/* Set the default_ept to the rpmsg device endpoint */
eptdev->default_ept = rpdev->ept;
+ ret = rpmsg_chrdev_eptdev_add(eptdev, chinfo);
+
+ if (ret)
+ return ret;
/*
* The rpmsg_ept_cb uses *priv parameter to get its rpmsg_eptdev context.
- * Storedit in default_ept *priv field.
+ * Stored it in default_ept *priv field.
*/
eptdev->default_ept->priv = eptdev;
- return rpmsg_chrdev_eptdev_add(eptdev, chinfo);
+ return 0;
}
static void rpmsg_chrdev_remove(struct rpmsg_device *rpdev)
diff --git a/drivers/staging/rtl8723bs/core/rtw_mlme.c b/drivers/staging/rtl8723bs/core/rtw_mlme.c
index 3563017acc31..d4b6e748f11f 100644
--- a/drivers/staging/rtl8723bs/core/rtw_mlme.c
+++ b/drivers/staging/rtl8723bs/core/rtw_mlme.c
@@ -449,8 +449,11 @@ static void update_current_network(struct adapter *adapter, struct wlan_bssid_ex
if ((check_fwstate(pmlmepriv, _FW_LINKED) == true) && (is_same_network(&(pmlmepriv->cur_network.network), pnetwork, 0))) {
update_network(&(pmlmepriv->cur_network.network), pnetwork, adapter, true);
+ if (pmlmepriv->cur_network.network.ie_length < sizeof(struct ndis_802_11_fix_ie))
+ return;
+
rtw_update_protection(adapter, (pmlmepriv->cur_network.network.ies) + sizeof(struct ndis_802_11_fix_ie),
- pmlmepriv->cur_network.network.ie_length);
+ pmlmepriv->cur_network.network.ie_length - sizeof(struct ndis_802_11_fix_ie));
}
}
@@ -1070,8 +1073,11 @@ static void rtw_joinbss_update_network(struct adapter *padapter, struct wlan_net
break;
}
+ if (cur_network->network.ie_length < sizeof(struct ndis_802_11_fix_ie))
+ return;
+
rtw_update_protection(padapter, (cur_network->network.ies) + sizeof(struct ndis_802_11_fix_ie),
- (cur_network->network.ie_length));
+ (cur_network->network.ie_length - sizeof(struct ndis_802_11_fix_ie)));
rtw_update_ht_cap(padapter, cur_network->network.ies, cur_network->network.ie_length, (u8) cur_network->network.configuration.ds_config);
}
diff --git a/drivers/tty/serial/8250/8250_dw.c b/drivers/tty/serial/8250/8250_dw.c
index f17dc3de020c..7b2e72bfeb75 100644
--- a/drivers/tty/serial/8250/8250_dw.c
+++ b/drivers/tty/serial/8250/8250_dw.c
@@ -655,8 +655,10 @@ static int dw8250_probe(struct platform_device *pdev)
*/
if (data->clk) {
err = clk_notifier_register(data->clk, &data->clk_notifier);
- if (err)
+ if (err) {
+ serial8250_unregister_port(data->data.line);
return dev_err_probe(dev, err, "Failed to set the clock notifier\n");
+ }
queue_work(system_unbound_wq, &data->clk_work);
}
diff --git a/drivers/tty/serial/qcom_geni_serial.c b/drivers/tty/serial/qcom_geni_serial.c
index c09526e2d423..cf705b01e8bd 100644
--- a/drivers/tty/serial/qcom_geni_serial.c
+++ b/drivers/tty/serial/qcom_geni_serial.c
@@ -867,12 +867,9 @@ static void qcom_geni_serial_handle_rx_dma(struct uart_port *uport, bool drop)
port->rx_dma_addr = 0;
rx_in = readl(uport->membase + SE_DMA_RX_LEN_IN);
- if (!rx_in) {
- dev_warn(uport->dev, "serial engine reports 0 RX bytes in!\n");
- return;
- }
-
- if (!drop)
+ if (!rx_in)
+ dev_warn_ratelimited(uport->dev, "serial engine reports 0 RX bytes in!\n");
+ else if (!drop)
handle_rx_uart(uport, rx_in);
ret = geni_se_rx_dma_prep(&port->se, port->rx_buf,
diff --git a/drivers/tty/vt/vc_screen.c b/drivers/tty/vt/vc_screen.c
index 79b33d998d43..253663515355 100644
--- a/drivers/tty/vt/vc_screen.c
+++ b/drivers/tty/vt/vc_screen.c
@@ -699,7 +699,7 @@ vcs_write(struct file *file, const char __user *buf, size_t count, loff_t *ppos)
}
*ppos += written;
ret = written;
- if (written)
+ if (written && vc)
vcs_scr_updated(vc);
unlock_out:
diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
index 86b9a371645b..a689b390fce3 100644
--- a/drivers/video/fbdev/core/fbmem.c
+++ b/drivers/video/fbdev/core/fbmem.c
@@ -680,6 +680,18 @@ int fb_new_modelist(struct fb_info *info)
if (list_empty(&info->modelist))
return 1;
+ /*
+ * The new modelist may not contain the current mode (info->var), and
+ * fbcon_new_modelist() below only re-points consoles mapped to this
+ * framebuffer. Add the current mode here so info->var keeps a match
+ * even when fbcon is unbound.
+ */
+ if (!fb_match_mode(&info->var, &info->modelist)) {
+ fb_var_to_videomode(&mode, &info->var);
+ if (fb_add_videomode(&mode, &info->modelist))
+ return 1;
+ }
+
fbcon_new_modelist(info);
return 0;
diff --git a/drivers/video/fbdev/core/fbsysfs.c b/drivers/video/fbdev/core/fbsysfs.c
index 1b3c9958ef5c..10c3cfb959f4 100644
--- a/drivers/video/fbdev/core/fbsysfs.c
+++ b/drivers/video/fbdev/core/fbsysfs.c
@@ -11,6 +11,7 @@
#include <linux/major.h>
#include "fb_internal.h"
+#include "fbcon.h"
#define FB_SYSFS_FLAG_ATTR 1
@@ -113,8 +114,15 @@ static ssize_t store_modes(struct device *device,
if (fb_new_modelist(fb_info)) {
fb_destroy_modelist(&fb_info->modelist);
list_splice(&old_list, &fb_info->modelist);
- } else
+ } else {
+ /*
+ * fb_display[i].mode and fb_info->mode both point into the old
+ * list. Clear them before it is freed.
+ */
+ fbcon_delete_modelist(&old_list);
+ fb_info->mode = NULL;
fb_destroy_modelist(&old_list);
+ }
unlock_fb_info(fb_info);
console_unlock();
diff --git a/drivers/video/fbdev/core/modedb.c b/drivers/video/fbdev/core/modedb.c
index 7196b055f2bd..7428d58e6ca8 100644
--- a/drivers/video/fbdev/core/modedb.c
+++ b/drivers/video/fbdev/core/modedb.c
@@ -258,7 +258,7 @@ static const struct fb_videomode modedb[] = {
FB_VMODE_DOUBLE },
/* 1920x1080 @ 60 Hz, 67.3 kHz hsync */
- { NULL, 60, 1920, 1080, 6734, 148, 88, 36, 4, 44, 5, 0,
+ { NULL, 60, 1920, 1080, 6734, 148, 88, 36, 4, 44, 5,
FB_SYNC_HOR_HIGH_ACT | FB_SYNC_VERT_HIGH_ACT,
FB_VMODE_NONINTERLACED },
@@ -625,7 +625,7 @@ int fb_find_mode(struct fb_var_screeninfo *var,
const struct fb_videomode *default_mode,
unsigned int default_bpp)
{
- char *mode_option_buf = NULL;
+ char *mode_option_buf __free(kfree) = NULL;
int i;
/* Set up defaults */
@@ -723,7 +723,6 @@ int fb_find_mode(struct fb_var_screeninfo *var,
res_specified = 1;
}
done:
- kfree(mode_option_buf);
if (cvt) {
struct fb_videomode cvt_mode;
int ret;
diff --git a/fs/backing-file.c b/fs/backing-file.c
index 09a9be945d45..53690754810f 100644
--- a/fs/backing-file.c
+++ b/fs/backing-file.c
@@ -12,6 +12,7 @@
#include <linux/backing-file.h>
#include <linux/splice.h>
#include <linux/mm.h>
+#include <linux/security.h>
#include "internal.h"
@@ -29,19 +30,20 @@
* returned file into a container structure that also stores the stacked
* file's path, which can be retrieved using backing_file_user_path().
*/
-struct file *backing_file_open(const struct path *user_path, int flags,
+struct file *backing_file_open(const struct file *user_file, int flags,
const struct path *real_path,
const struct cred *cred)
{
+ const struct path *user_path = &user_file->f_path;
struct file *f;
int error;
- f = alloc_empty_backing_file(flags, cred);
+ f = alloc_empty_backing_file(flags, cred, user_file);
if (IS_ERR(f))
return f;
path_get(user_path);
- *backing_file_user_path(f) = *user_path;
+ backing_file_set_user_path(f, user_path);
error = vfs_open(real_path, f);
if (error) {
fput(f);
@@ -52,20 +54,21 @@ struct file *backing_file_open(const struct path *user_path, int flags,
}
EXPORT_SYMBOL_GPL(backing_file_open);
-struct file *backing_tmpfile_open(const struct path *user_path, int flags,
+struct file *backing_tmpfile_open(const struct file *user_file, int flags,
const struct path *real_parentpath,
umode_t mode, const struct cred *cred)
{
struct mnt_idmap *real_idmap = mnt_idmap(real_parentpath->mnt);
+ const struct path *user_path = &user_file->f_path;
struct file *f;
int error;
- f = alloc_empty_backing_file(flags, cred);
+ f = alloc_empty_backing_file(flags, cred, user_file);
if (IS_ERR(f))
return f;
path_get(user_path);
- *backing_file_user_path(f) = *user_path;
+ backing_file_set_user_path(f, user_path);
error = vfs_tmpfile(real_idmap, real_parentpath, f, mode);
if (error) {
fput(f);
@@ -326,6 +329,7 @@ EXPORT_SYMBOL_GPL(backing_file_splice_write);
int backing_file_mmap(struct file *file, struct vm_area_struct *vma,
struct backing_file_ctx *ctx)
{
+ struct file *user_file = vma->vm_file;
const struct cred *old_cred;
int ret;
@@ -339,6 +343,12 @@ int backing_file_mmap(struct file *file, struct vm_area_struct *vma,
vma_set_file(vma, file);
old_cred = override_creds(ctx->cred);
+ ret = security_mmap_backing_file(vma, file, user_file);
+ if (ret) {
+ revert_creds(old_cred);
+ return ret;
+ }
+
ret = call_mmap(vma->vm_file, vma);
revert_creds(old_cred);
diff --git a/fs/dlm/lockspace.c b/fs/dlm/lockspace.c
index 7b4b6977dcd6..ee11a70def92 100644
--- a/fs/dlm/lockspace.c
+++ b/fs/dlm/lockspace.c
@@ -576,7 +576,7 @@ static int new_lockspace(const char *name, const char *cluster,
lockspace to start running (via sysfs) in dlm_ls_start(). */
error = do_uevent(ls, 1);
- if (error)
+ if (error < 0)
goto out_recoverd;
/* wait until recovery is successful or failed */
diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index a860cb54658a..22605fbc12de 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -798,36 +798,47 @@ static void ep_free(struct eventpoll *ep)
}
/*
- * Removes a "struct epitem" from the eventpoll RB tree and deallocates
- * all the associated resources. Must be called with "mtx" held.
- * If the dying flag is set, do the removal only if force is true.
- * This prevents ep_clear_and_put() from dropping all the ep references
- * while running concurrently with eventpoll_release_file().
- * Returns true if the eventpoll can be disposed.
+ * The ffd.file pointer may be in the process of being torn down due to
+ * being closed, but we may not have finished eventpoll_release() yet.
+ *
+ * Normally, even with the atomic_long_inc_not_zero, the file may have
+ * been free'd and then gotten re-allocated to something else (since
+ * files are not RCU-delayed, they are SLAB_TYPESAFE_BY_RCU).
+ *
+ * But for epoll, users hold the ep->mtx mutex, and as such any file in
+ * the process of being free'd will block in eventpoll_release_file()
+ * and thus the underlying file allocation will not be free'd, and the
+ * file re-use cannot happen.
+ *
+ * For the same reason we can avoid a rcu_read_lock() around the
+ * operation - 'ffd.file' cannot go away even if the refcount has
+ * reached zero (but we must still not call out to ->poll() functions
+ * etc).
*/
-static bool __ep_remove(struct eventpoll *ep, struct epitem *epi, bool force)
+static struct file *epi_fget(const struct epitem *epi)
{
- struct file *file = epi->ffd.file;
- struct epitems_head *to_free;
- struct hlist_head *head;
+ struct file *file;
- lockdep_assert_irqs_enabled();
+ file = epi->ffd.file;
+ if (!atomic_long_inc_not_zero(&file->f_count))
+ file = NULL;
+ return file;
+}
- /*
- * Removes poll wait queue hooks.
- */
- ep_unregister_pollwait(ep, epi);
+/*
+ * Called with &file->f_lock held,
+ * returns with it released
+ */
+static void ep_remove_file(struct eventpoll *ep, struct epitem *epi,
+ struct file *file)
+{
+ struct epitems_head *to_free = NULL;
+ struct hlist_head *head = file->f_ep;
- /* Remove the current item from the list of epoll hooks */
- spin_lock(&file->f_lock);
- if (epi->dying && !force) {
- spin_unlock(&file->f_lock);
- return false;
- }
+ lockdep_assert_held(&ep->mtx);
+ lockdep_assert_held(&file->f_lock);
- to_free = NULL;
- head = file->f_ep;
- if (head->first == &epi->fllink && !epi->fllink.next) {
+ if (hlist_is_singular_node(&epi->fllink, head)) {
/* See eventpoll_release() for details. */
WRITE_ONCE(file->f_ep, NULL);
if (!is_file_epoll(file)) {
@@ -840,6 +851,11 @@ static bool __ep_remove(struct eventpoll *ep, struct epitem *epi, bool force)
hlist_del_rcu(&epi->fllink);
spin_unlock(&file->f_lock);
free_ephead(to_free);
+}
+
+static bool ep_remove_epi(struct eventpoll *ep, struct epitem *epi)
+{
+ lockdep_assert_held(&ep->mtx);
rb_erase_cached(&epi->rbn, &ep->rbr);
@@ -865,9 +881,31 @@ static bool __ep_remove(struct eventpoll *ep, struct epitem *epi, bool force)
/*
* ep_remove variant for callers owing an additional reference to the ep
*/
-static void ep_remove_safe(struct eventpoll *ep, struct epitem *epi)
+static void ep_remove(struct eventpoll *ep, struct epitem *epi)
{
- if (__ep_remove(ep, epi, false))
+ struct file *file __free(fput) = NULL;
+
+ lockdep_assert_irqs_enabled();
+ lockdep_assert_held(&ep->mtx);
+
+ ep_unregister_pollwait(ep, epi);
+
+ /* cheap sync with eventpoll_release_file() */
+ if (unlikely(READ_ONCE(epi->dying)))
+ return;
+
+ /*
+ * If we manage to grab a reference it means we're not in
+ * eventpoll_release_file() and aren't going to be.
+ */
+ file = epi_fget(epi);
+ if (!file)
+ return;
+
+ spin_lock(&file->f_lock);
+ ep_remove_file(ep, epi, file);
+
+ if (ep_remove_epi(ep, epi))
WARN_ON_ONCE(ep_refcount_dec_and_test(ep));
}
@@ -894,7 +932,7 @@ static void ep_clear_and_put(struct eventpoll *ep)
/*
* Walks through the whole tree and try to free each "struct epitem".
- * Note that ep_remove_safe() will not remove the epitem in case of a
+ * Note that ep_remove() will not remove the epitem in case of a
* racing eventpoll_release_file(); the latter will do the removal.
* At this point we are sure no poll callbacks will be lingering around.
* Since we still own a reference to the eventpoll struct, the loop can't
@@ -903,7 +941,7 @@ static void ep_clear_and_put(struct eventpoll *ep)
for (rbp = rb_first_cached(&ep->rbr); rbp; rbp = next) {
next = rb_next(rbp);
epi = rb_entry(rbp, struct epitem, rbn);
- ep_remove_safe(ep, epi);
+ ep_remove(ep, epi);
cond_resched();
}
@@ -983,34 +1021,6 @@ static __poll_t __ep_eventpoll_poll(struct file *file, poll_table *wait, int dep
return res;
}
-/*
- * The ffd.file pointer may be in the process of being torn down due to
- * being closed, but we may not have finished eventpoll_release() yet.
- *
- * Normally, even with the atomic_long_inc_not_zero, the file may have
- * been free'd and then gotten re-allocated to something else (since
- * files are not RCU-delayed, they are SLAB_TYPESAFE_BY_RCU).
- *
- * But for epoll, users hold the ep->mtx mutex, and as such any file in
- * the process of being free'd will block in eventpoll_release_file()
- * and thus the underlying file allocation will not be free'd, and the
- * file re-use cannot happen.
- *
- * For the same reason we can avoid a rcu_read_lock() around the
- * operation - 'ffd.file' cannot go away even if the refcount has
- * reached zero (but we must still not call out to ->poll() functions
- * etc).
- */
-static struct file *epi_fget(const struct epitem *epi)
-{
- struct file *file;
-
- file = epi->ffd.file;
- if (!atomic_long_inc_not_zero(&file->f_count))
- file = NULL;
- return file;
-}
-
/*
* Differs from ep_eventpoll_poll() in that internal callers already have
* the ep->mtx so we need to start from depth=1, such that mutex_lock_nested()
@@ -1099,7 +1109,7 @@ void eventpoll_release_file(struct file *file)
spin_lock(&file->f_lock);
if (file->f_ep && file->f_ep->first) {
epi = hlist_entry(file->f_ep->first, struct epitem, fllink);
- epi->dying = true;
+ WRITE_ONCE(epi->dying, true);
spin_unlock(&file->f_lock);
/*
@@ -1108,7 +1118,13 @@ void eventpoll_release_file(struct file *file)
*/
ep = epi->ep;
mutex_lock(&ep->mtx);
- dispose = __ep_remove(ep, epi, true);
+
+ ep_unregister_pollwait(ep, epi);
+
+ spin_lock(&file->f_lock);
+ ep_remove_file(ep, epi, file);
+ dispose = ep_remove_epi(ep, epi);
+
mutex_unlock(&ep->mtx);
if (dispose && ep_refcount_dec_and_test(ep))
@@ -1590,21 +1606,21 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event,
mutex_unlock(&tep->mtx);
/*
- * ep_remove_safe() calls in the later error paths can't lead to
+ * ep_remove() calls in the later error paths can't lead to
* ep_free() as the ep file itself still holds an ep reference.
*/
ep_get(ep);
/* now check if we've created too many backpaths */
if (unlikely(full_check && reverse_path_check())) {
- ep_remove_safe(ep, epi);
+ ep_remove(ep, epi);
return -EINVAL;
}
if (epi->event.events & EPOLLWAKEUP) {
error = ep_create_wakeup_source(epi);
if (error) {
- ep_remove_safe(ep, epi);
+ ep_remove(ep, epi);
return error;
}
}
@@ -1628,7 +1644,7 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event,
* high memory pressure.
*/
if (unlikely(!epq.epi)) {
- ep_remove_safe(ep, epi);
+ ep_remove(ep, epi);
return -ENOMEM;
}
@@ -2317,7 +2333,7 @@ int do_epoll_ctl(int epfd, int op, int fd, struct epoll_event *epds,
* The eventpoll itself is still alive: the refcount
* can't go to zero here.
*/
- ep_remove_safe(ep, epi);
+ ep_remove(ep, epi);
error = 0;
} else {
error = -ENOENT;
diff --git a/fs/exfat/dir.c b/fs/exfat/dir.c
index 1c428f7f83f5..a3bdee0461e0 100644
--- a/fs/exfat/dir.c
+++ b/fs/exfat/dir.c
@@ -1100,12 +1100,12 @@ int exfat_find_dir_entry(struct super_block *sb, struct exfat_inode_info *ei,
continue;
}
- brelse(bh);
if (entry_type == TYPE_EXTEND) {
unsigned short entry_uniname[16], unichar;
if (step != DIRENT_STEP_NAME ||
name_len >= MAX_NAME_LENGTH) {
+ brelse(bh);
step = DIRENT_STEP_FILE;
continue;
}
@@ -1116,6 +1116,7 @@ int exfat_find_dir_entry(struct super_block *sb, struct exfat_inode_info *ei,
uniname += EXFAT_FILE_NAME_LEN;
len = exfat_extract_uni_name(ep, entry_uniname);
+ brelse(bh);
name_len += len;
unichar = *(uniname+len);
@@ -1134,6 +1135,7 @@ int exfat_find_dir_entry(struct super_block *sb, struct exfat_inode_info *ei,
continue;
}
+ brelse(bh);
if (entry_type &
(TYPE_CRITICAL_SEC | TYPE_BENIGN_SEC)) {
if (step == DIRENT_STEP_SECD) {
diff --git a/fs/f2fs/acl.c b/fs/f2fs/acl.c
index 8bffdeccdbc3..e45f68254075 100644
--- a/fs/f2fs/acl.c
+++ b/fs/f2fs/acl.c
@@ -46,6 +46,7 @@ static inline int f2fs_acl_count(size_t size)
static struct posix_acl *f2fs_acl_from_disk(const char *value, size_t size)
{
int i, count;
+ int err = -EINVAL;
struct posix_acl *acl;
struct f2fs_acl_header *hdr = (struct f2fs_acl_header *)value;
struct f2fs_acl_entry *entry = (struct f2fs_acl_entry *)(hdr + 1);
@@ -69,8 +70,11 @@ static struct posix_acl *f2fs_acl_from_disk(const char *value, size_t size)
for (i = 0; i < count; i++) {
- if ((char *)entry > end)
+ if (unlikely((char *)entry +
+ sizeof(struct f2fs_acl_entry_short) > end)) {
+ err = -EFSCORRUPTED;
goto fail;
+ }
acl->a_entries[i].e_tag = le16_to_cpu(entry->e_tag);
acl->a_entries[i].e_perm = le16_to_cpu(entry->e_perm);
@@ -85,6 +89,11 @@ static struct posix_acl *f2fs_acl_from_disk(const char *value, size_t size)
break;
case ACL_USER:
+ if (unlikely((char *)entry +
+ sizeof(struct f2fs_acl_entry) > end)) {
+ err = -EFSCORRUPTED;
+ goto fail;
+ }
acl->a_entries[i].e_uid =
make_kuid(&init_user_ns,
le32_to_cpu(entry->e_id));
@@ -92,6 +101,11 @@ static struct posix_acl *f2fs_acl_from_disk(const char *value, size_t size)
sizeof(struct f2fs_acl_entry));
break;
case ACL_GROUP:
+ if (unlikely((char *)entry +
+ sizeof(struct f2fs_acl_entry) > end)) {
+ err = -EFSCORRUPTED;
+ goto fail;
+ }
acl->a_entries[i].e_gid =
make_kgid(&init_user_ns,
le32_to_cpu(entry->e_id));
@@ -107,7 +121,7 @@ static struct posix_acl *f2fs_acl_from_disk(const char *value, size_t size)
return acl;
fail:
posix_acl_release(acl);
- return ERR_PTR(-EINVAL);
+ return ERR_PTR(err);
}
static void *f2fs_acl_to_disk(struct f2fs_sb_info *sbi,
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 234444794b55..faa350c0da93 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -3566,6 +3566,7 @@ static int prepare_atomic_write_begin(struct f2fs_sb_info *sbi,
pgoff_t index = folio->index;
int err = 0;
block_t ori_blk_addr = NULL_ADDR;
+ bool cow_has_reserved_block = false;
/* If pos is beyond the end of file, reserve a new block in COW inode */
if ((pos & PAGE_MASK) >= i_size_read(inode))
@@ -3575,9 +3576,11 @@ static int prepare_atomic_write_begin(struct f2fs_sb_info *sbi,
err = __find_data_block(cow_inode, index, blk_addr);
if (err) {
return err;
- } else if (*blk_addr != NULL_ADDR) {
+ } else if (__is_valid_data_blkaddr(*blk_addr)) {
*use_cow = true;
return 0;
+ } else if (*blk_addr == NEW_ADDR) {
+ cow_has_reserved_block = true;
}
if (is_inode_flag_set(inode, FI_ATOMIC_REPLACE))
@@ -3590,10 +3593,13 @@ static int prepare_atomic_write_begin(struct f2fs_sb_info *sbi,
reserve_block:
/* Finally, we should reserve a new block in COW inode for the update */
- err = __reserve_data_block(cow_inode, index, blk_addr, node_changed);
- if (err)
- return err;
- inc_atomic_write_cnt(inode);
+ if (!cow_has_reserved_block) {
+ err = __reserve_data_block(cow_inode, index, blk_addr,
+ node_changed);
+ if (err)
+ return err;
+ inc_atomic_write_cnt(inode);
+ }
if (ori_blk_addr != NULL_ADDR)
*blk_addr = ori_blk_addr;
diff --git a/fs/f2fs/extent_cache.c b/fs/f2fs/extent_cache.c
index e8119d51fb4b..d0acbd6e2123 100644
--- a/fs/f2fs/extent_cache.c
+++ b/fs/f2fs/extent_cache.c
@@ -86,10 +86,9 @@ static bool __may_extent_tree(struct inode *inode, enum extent_type type)
if (!__init_may_extent_tree(inode, type))
return false;
- if (is_inode_flag_set(inode, FI_NO_EXTENT))
- return false;
-
if (type == EX_READ) {
+ if (is_inode_flag_set(inode, FI_NO_EXTENT))
+ return false;
if (is_inode_flag_set(inode, FI_COMPRESSED_FILE) &&
!f2fs_sb_has_readonly(F2FS_I_SB(inode)))
return false;
@@ -602,14 +601,10 @@ static unsigned int __destroy_extent_node(struct inode *inode,
while (atomic_read(&et->node_cnt)) {
write_lock(&et->lock);
- if (!is_inode_flag_set(inode, FI_NO_EXTENT))
- set_inode_flag(inode, FI_NO_EXTENT);
node_cnt += __free_extent_tree(sbi, et, nr_shrink);
write_unlock(&et->lock);
}
- f2fs_bug_on(sbi, atomic_read(&et->node_cnt));
-
return node_cnt;
}
@@ -639,12 +634,12 @@ static void __update_extent_tree_range(struct inode *inode,
write_lock(&et->lock);
- if (is_inode_flag_set(inode, FI_NO_EXTENT)) {
- write_unlock(&et->lock);
- return;
- }
-
if (type == EX_READ) {
+ if (is_inode_flag_set(inode, FI_NO_EXTENT)) {
+ write_unlock(&et->lock);
+ return;
+ }
+
prev = et->largest;
dei.len = 0;
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 3a56ab6a22c0..5f69bdca95c2 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -1838,8 +1838,15 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
if (f2fs_is_pinned_file(inode)) {
block_t sec_blks = CAP_BLKS_PER_SEC(sbi);
- block_t sec_len = roundup(map.m_len, sec_blks);
+ block_t sec_len;
+ if (map.m_lblk % sec_blks) {
+ map.m_lblk = rounddown(map.m_lblk, sec_blks);
+ map.m_len = pg_end - map.m_lblk;
+ if (off_end)
+ map.m_len++;
+ }
+ sec_len = roundup(map.m_len, sec_blks);
map.m_len = sec_blks;
next_alloc:
f2fs_down_write(&sbi->pin_sem);
diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c
index 9180693b933f..67c800802141 100644
--- a/fs/f2fs/inode.c
+++ b/fs/f2fs/inode.c
@@ -541,8 +541,13 @@ static int do_read_inode(struct inode *inode)
static bool is_meta_ino(struct f2fs_sb_info *sbi, unsigned int ino)
{
- return ino == F2FS_NODE_INO(sbi) || ino == F2FS_META_INO(sbi) ||
- ino == F2FS_COMPRESS_INO(sbi);
+ if (ino == F2FS_NODE_INO(sbi) || ino == F2FS_META_INO(sbi))
+ return true;
+#ifdef CONFIG_F2FS_FS_COMPRESSION
+ if (test_opt(sbi, COMPRESS_CACHE) && ino == F2FS_COMPRESS_INO(sbi))
+ return true;
+#endif
+ return false;
}
struct inode *f2fs_iget(struct super_block *sb, unsigned long ino)
diff --git a/fs/fhandle.c b/fs/fhandle.c
index 38d803a28ab9..bbb5857c77b9 100644
--- a/fs/fhandle.c
+++ b/fs/fhandle.c
@@ -242,6 +242,19 @@ static int do_handle_to_path(struct file_handle *handle, struct path *path,
return 0;
}
+static bool capable_wrt_mount(struct mount *mount)
+{
+ struct mnt_namespace *mnt_ns;
+
+ /*
+ * For ->mnt_ns access.
+ * The following READ_ONCE() is semantically rcu_dereference().
+ */
+ guard(rcu)();
+ mnt_ns = READ_ONCE(mount->mnt_ns);
+ return ns_capable(mnt_ns->user_ns, CAP_SYS_ADMIN);
+}
+
/*
* Allow relaxed permissions of file handles if the caller has the
* ability to mount the filesystem or create a bind-mount of the
@@ -273,8 +286,7 @@ static inline bool may_decode_fh(struct handle_to_path_ctx *ctx,
if (ns_capable(root->mnt->mnt_sb->s_user_ns, CAP_SYS_ADMIN))
ctx->flags = HANDLE_CHECK_PERMS;
else if (is_mounted(root->mnt) &&
- ns_capable(real_mount(root->mnt)->mnt_ns->user_ns,
- CAP_SYS_ADMIN) &&
+ capable_wrt_mount(real_mount(root->mnt)) &&
!has_locked_children(real_mount(root->mnt), root->dentry))
ctx->flags = HANDLE_CHECK_PERMS | HANDLE_CHECK_SUBTREE;
else
diff --git a/fs/file_table.c b/fs/file_table.c
index 2a08bc93b0b9..f78bcff79f1a 100644
--- a/fs/file_table.c
+++ b/fs/file_table.c
@@ -46,21 +46,40 @@ static struct percpu_counter nr_files __cacheline_aligned_in_smp;
struct backing_file {
struct file file;
struct path user_path;
+#ifdef CONFIG_SECURITY
+ void *security;
+#endif
};
-static inline struct backing_file *backing_file(struct file *f)
-{
- return container_of(f, struct backing_file, file);
-}
+#define backing_file(f) container_of(f, struct backing_file, file)
-struct path *backing_file_user_path(struct file *f)
+struct path *backing_file_user_path(const struct file *f)
{
return &backing_file(f)->user_path;
}
EXPORT_SYMBOL_GPL(backing_file_user_path);
+void backing_file_set_user_path(struct file *f, const struct path *path)
+{
+ backing_file(f)->user_path = *path;
+}
+EXPORT_SYMBOL_GPL(backing_file_set_user_path);
+
+#ifdef CONFIG_SECURITY
+void *backing_file_security(const struct file *f)
+{
+ return backing_file(f)->security;
+}
+
+void backing_file_set_security(struct file *f, void *security)
+{
+ backing_file(f)->security = security;
+}
+#endif /* CONFIG_SECURITY */
+
static inline void backing_file_free(struct backing_file *ff)
{
+ security_backing_file_free(&ff->file);
path_put(&ff->user_path);
kfree(ff);
}
@@ -259,10 +278,12 @@ struct file *alloc_empty_file_noaccount(int flags, const struct cred *cred)
return f;
}
-static int init_backing_file(struct backing_file *ff)
+static int init_backing_file(struct backing_file *ff,
+ const struct file *user_file)
{
memset(&ff->user_path, 0, sizeof(ff->user_path));
- return 0;
+ backing_file_set_security(&ff->file, NULL);
+ return security_backing_file_alloc(&ff->file, user_file);
}
/*
@@ -272,7 +293,8 @@ static int init_backing_file(struct backing_file *ff)
* This is only for kernel internal use, and the allocate file must not be
* installed into file tables or such.
*/
-struct file *alloc_empty_backing_file(int flags, const struct cred *cred)
+struct file *alloc_empty_backing_file(int flags, const struct cred *cred,
+ const struct file *user_file)
{
struct backing_file *ff;
int error;
@@ -289,7 +311,7 @@ struct file *alloc_empty_backing_file(int flags, const struct cred *cred)
/* The f_mode flags must be set before fput(). */
ff->file.f_mode |= FMODE_BACKING | FMODE_NOACCOUNT;
- error = init_backing_file(ff);
+ error = init_backing_file(ff, user_file);
if (unlikely(error)) {
fput(&ff->file);
return ERR_PTR(error);
diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c
index 8f4a2ff56cc3..f7e305c9ad4e 100644
--- a/fs/fuse/dev.c
+++ b/fs/fuse/dev.c
@@ -891,6 +891,10 @@ static int fuse_try_move_page(struct fuse_copy_state *cs, struct page **pagep)
if (WARN_ON(folio_test_mlocked(oldfolio)))
goto out_fallback_unlock;
+ err = lock_request(cs->req);
+ if (err)
+ goto out_fallback_unlock;
+
replace_page_cache_folio(oldfolio, newfolio);
folio_get(newfolio);
@@ -904,20 +908,7 @@ static int fuse_try_move_page(struct fuse_copy_state *cs, struct page **pagep)
*/
pipe_buf_release(cs->pipe, buf);
- err = 0;
- spin_lock(&cs->req->waitq.lock);
- if (test_bit(FR_ABORTED, &cs->req->flags))
- err = -ENOENT;
- else
- *pagep = &newfolio->page;
- spin_unlock(&cs->req->waitq.lock);
-
- if (err) {
- folio_unlock(newfolio);
- folio_put(newfolio);
- goto out_put_old;
- }
-
+ *pagep = &newfolio->page;
folio_unlock(oldfolio);
/* Drop ref for ap->pages[] array */
folio_put(oldfolio);
@@ -1769,6 +1760,10 @@ static int fuse_retrieve(struct fuse_mount *fm, struct inode *inode,
page = find_get_page(mapping, index);
if (!page)
break;
+ if (!PageUptodate(page)) {
+ put_page(page);
+ break;
+ }
this_num = min_t(unsigned, num, PAGE_SIZE - offset);
ap->pages[ap->num_pages] = page;
diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index 23afaadd5f3e..1f71ef3ead7a 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -373,8 +373,14 @@ void fuse_file_release(struct inode *inode, struct fuse_file *ff,
* aio and closes the fd before the aio completes. Since aio takes its
* own ref to the file, the IO completion has to drop the ref, which is
* how the fuse server can end up closing its clients' files.
+ *
+ * Exception is virtio-fs, which is not affected by the above (server is
+ * on host, cannot close open files in guest). Virtio-fs needs sync
+ * release, because the num_waiting mechanism to wait for all requests
+ * before commencing with fs shutdown doesn't work if submounts are
+ * used.
*/
- fuse_file_put(ff, false);
+ fuse_file_put(ff, ff->fm->fc->auto_submounts);
}
void fuse_release_common(struct file *file, bool isdir)
diff --git a/fs/fuse/passthrough.c b/fs/fuse/passthrough.c
index 6bfd09dda9e3..140e150be0de 100644
--- a/fs/fuse/passthrough.c
+++ b/fs/fuse/passthrough.c
@@ -326,7 +326,7 @@ struct fuse_backing *fuse_passthrough_open(struct file *file,
goto out;
/* Allocate backing file per fuse file to store fuse path */
- backing_file = backing_file_open(&file->f_path, file->f_flags,
+ backing_file = backing_file_open(file, file->f_flags,
&fb->file->f_path, fb->cred);
err = PTR_ERR(backing_file);
if (IS_ERR(backing_file)) {
diff --git a/fs/gfs2/super.c b/fs/gfs2/super.c
index e6f8be03190c..3d5d4b844621 100644
--- a/fs/gfs2/super.c
+++ b/fs/gfs2/super.c
@@ -652,6 +652,7 @@ static void gfs2_put_super(struct super_block *sb)
gfs2_delete_debugfs_file(sdp);
gfs2_sys_fs_del(sdp);
+ rcu_barrier();
free_sbd(sdp);
}
diff --git a/fs/internal.h b/fs/internal.h
index 8c1b7acbbe8f..997acc721f61 100644
--- a/fs/internal.h
+++ b/fs/internal.h
@@ -99,7 +99,9 @@ extern void chroot_fs_refs(const struct path *, const struct path *);
*/
struct file *alloc_empty_file(int flags, const struct cred *cred);
struct file *alloc_empty_file_noaccount(int flags, const struct cred *cred);
-struct file *alloc_empty_backing_file(int flags, const struct cred *cred);
+struct file *alloc_empty_backing_file(int flags, const struct cred *cred,
+ const struct file *user_file);
+void backing_file_set_user_path(struct file *f, const struct path *path);
static inline void file_put_write_access(struct file *file)
{
diff --git a/fs/mount.h b/fs/mount.h
index 179f690a0c72..c3987f1d1749 100644
--- a/fs/mount.h
+++ b/fs/mount.h
@@ -58,7 +58,15 @@ struct mount {
struct list_head mnt_slave_list;/* list of slave mounts */
struct list_head mnt_slave; /* slave list entry */
struct mount *mnt_master; /* slave is on master->mnt_slave_list */
- struct mnt_namespace *mnt_ns; /* containing namespace */
+ /*
+ * Containing namespace (active or deactivating, non-refcounted).
+ * Normally protected by namespace_sem.
+ * Can also be accessed locklessly under RCU. RCU readers can't rely on
+ * the namespace still being active, but implicitly hold a passive
+ * reference (because an RCU delay happens between a namespace being
+ * deactivated and the corresponding passive refcount drop).
+ */
+ struct mnt_namespace *mnt_ns;
struct mountpoint *mnt_mp; /* where is it mounted */
union {
struct hlist_node mnt_mp_list; /* list mounts with the same mountpoint */
diff --git a/fs/namespace.c b/fs/namespace.c
index df6d6a2c6b89..94c06c842902 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -1132,7 +1132,7 @@ static void mnt_add_to_ns(struct mnt_namespace *ns, struct mount *mnt)
struct rb_node *parent = NULL;
WARN_ON(mnt_ns_attached(mnt));
- mnt->mnt_ns = ns;
+ WRITE_ONCE(mnt->mnt_ns, ns);
while (*link) {
parent = *link;
if (mnt->mnt_id_unique < node_to_mount(parent)->mnt_id_unique)
@@ -1493,7 +1493,7 @@ EXPORT_SYMBOL(mntget);
void mnt_make_shortterm(struct vfsmount *mnt)
{
if (mnt)
- real_mount(mnt)->mnt_ns = NULL;
+ WRITE_ONCE(real_mount(mnt)->mnt_ns, NULL);
}
/**
@@ -1805,7 +1805,7 @@ static void umount_tree(struct mount *mnt, enum umount_tree_flags how)
ns->nr_mounts--;
__touch_mnt_namespace(ns);
}
- p->mnt_ns = NULL;
+ WRITE_ONCE(p->mnt_ns, NULL);
if (how & UMOUNT_SYNC)
p->mnt.mnt_flags |= MNT_SYNC_UMOUNT;
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 5fe20936e145..8bdde6cc873f 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -1054,6 +1054,7 @@ struct nfs_server *nfs_alloc_server(void)
server->io_stats = nfs_alloc_iostats();
if (!server->io_stats) {
+ ida_free(&s_sysfs_ids, server->s_sysfs_id);
kfree(server);
return NULL;
}
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 116499e0f5ce..5928d1cb1de2 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -2217,11 +2217,11 @@ pnfs_update_layout(struct inode *ino,
dprintk("%s wait for layoutreturn\n", __func__);
lseg = ERR_PTR(pnfs_prepare_to_retry_layoutget(lo));
if (!IS_ERR(lseg)) {
- pnfs_put_layout_hdr(lo);
dprintk("%s retrying\n", __func__);
trace_pnfs_update_layout(ino, pos, count, iomode, lo,
lseg,
PNFS_UPDATE_LAYOUT_RETRY);
+ pnfs_put_layout_hdr(lo);
goto lookup_again;
}
trace_pnfs_update_layout(ino, pos, count, iomode, lo, lseg,
diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c
index dd688d17b5b9..39507b2e259b 100644
--- a/fs/nfs/pnfs_nfs.c
+++ b/fs/nfs/pnfs_nfs.c
@@ -1055,14 +1055,14 @@ nfs4_decode_mp_ds_addr(struct net *net, struct xdr_stream *xdr, gfp_t gfp_flags)
/* r_netid */
nlen = xdr_stream_decode_string_dup(xdr, &netid, XDR_MAX_NETOBJ,
gfp_flags);
- if (unlikely(nlen < 0))
+ if (unlikely(nlen <= 0))
goto out_err;
/* r_addr: ip/ip6addr with port in dec octets - see RFC 5665 */
/* port is ".ABC.DEF", 8 chars max */
rlen = xdr_stream_decode_string_dup(xdr, &buf, INET6_ADDRSTRLEN +
IPV6_SCOPE_ID_LEN + 8, gfp_flags);
- if (unlikely(rlen < 0))
+ if (unlikely(rlen <= 0))
goto out_free_netid;
/* replace port '.' with '-' */
diff --git a/fs/nfsd/nfs2acl.c b/fs/nfsd/nfs2acl.c
index 0ac538c76180..76305b86c1a9 100644
--- a/fs/nfsd/nfs2acl.c
+++ b/fs/nfsd/nfs2acl.c
@@ -131,10 +131,7 @@ static __be32 nfsacld_proc_setacl(struct svc_rqst *rqstp)
resp->status = fh_getattr(fh, &resp->stat);
out:
- /* argp->acl_{access,default} may have been allocated in
- nfssvc_decode_setaclargs. */
- posix_acl_release(argp->acl_access);
- posix_acl_release(argp->acl_default);
+ /* argp->acl_{access,default} are released in nfsaclsvc_release_setacl. */
return rpc_success;
out_drop_lock:
@@ -310,6 +307,16 @@ static void nfsaclsvc_release_access(struct svc_rqst *rqstp)
fh_put(&resp->fh);
}
+static void nfsaclsvc_release_setacl(struct svc_rqst *rqstp)
+{
+ struct nfsd3_setaclargs *argp = rqstp->rq_argp;
+ struct nfsd_attrstat *resp = rqstp->rq_resp;
+
+ fh_put(&resp->fh);
+ posix_acl_release(argp->acl_access);
+ posix_acl_release(argp->acl_default);
+}
+
#define ST 1 /* status*/
#define AT 21 /* attributes */
#define pAT (1+AT) /* post attributes - conditional */
@@ -343,7 +350,7 @@ static const struct svc_procedure nfsd_acl_procedures2[5] = {
.pc_func = nfsacld_proc_setacl,
.pc_decode = nfsaclsvc_decode_setaclargs,
.pc_encode = nfssvc_encode_attrstatres,
- .pc_release = nfssvc_release_attrstat,
+ .pc_release = nfsaclsvc_release_setacl,
.pc_argsize = sizeof(struct nfsd3_setaclargs),
.pc_argzero = sizeof(struct nfsd3_setaclargs),
.pc_ressize = sizeof(struct nfsd_attrstat),
diff --git a/fs/nfsd/nfs3acl.c b/fs/nfsd/nfs3acl.c
index 7b5433bd3019..e87731380be8 100644
--- a/fs/nfsd/nfs3acl.c
+++ b/fs/nfsd/nfs3acl.c
@@ -118,10 +118,7 @@ static __be32 nfsd3_proc_setacl(struct svc_rqst *rqstp)
out_errno:
resp->status = nfserrno(error);
out:
- /* argp->acl_{access,default} may have been allocated in
- nfs3svc_decode_setaclargs. */
- posix_acl_release(argp->acl_access);
- posix_acl_release(argp->acl_default);
+ /* argp->acl_{access,default} are released in nfs3svc_release_setacl. */
return rpc_success;
}
@@ -223,6 +220,16 @@ static void nfs3svc_release_getacl(struct svc_rqst *rqstp)
posix_acl_release(resp->acl_default);
}
+static void nfs3svc_release_setacl(struct svc_rqst *rqstp)
+{
+ struct nfsd3_setaclargs *argp = rqstp->rq_argp;
+ struct nfsd3_attrstat *resp = rqstp->rq_resp;
+
+ fh_put(&resp->fh);
+ posix_acl_release(argp->acl_access);
+ posix_acl_release(argp->acl_default);
+}
+
#define ST 1 /* status*/
#define AT 21 /* attributes */
#define pAT (1+AT) /* post attributes - conditional */
@@ -256,7 +263,7 @@ static const struct svc_procedure nfsd_acl_procedures3[3] = {
.pc_func = nfsd3_proc_setacl,
.pc_decode = nfs3svc_decode_setaclargs,
.pc_encode = nfs3svc_encode_setaclres,
- .pc_release = nfs3svc_release_fhandle,
+ .pc_release = nfs3svc_release_setacl,
.pc_argsize = sizeof(struct nfsd3_setaclargs),
.pc_argzero = sizeof(struct nfsd3_setaclargs),
.pc_ressize = sizeof(struct nfsd3_attrstat),
diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
index 1c8fcb04b3cd..075715998648 100644
--- a/fs/nfsd/nfs4recover.c
+++ b/fs/nfsd/nfs4recover.c
@@ -818,7 +818,8 @@ __cld_pipe_inprogress_downcall(const struct cld_msg_v2 __user *cmsg,
if (IS_ERR(name.data))
return PTR_ERR(name.data);
name.len = namelen;
- get_user(princhashlen, &ci->cc_princhash.cp_len);
+ if (get_user(princhashlen, &ci->cc_princhash.cp_len))
+ return -EFAULT;
if (princhashlen > 0) {
princhash.data = memdup_user(
&ci->cc_princhash.cp_data,
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 2d9174729782..1ce66acc9b6b 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -4971,6 +4971,7 @@ find_or_alloc_open_stateowner(unsigned int strhashval, struct nfsd4_open *open,
/* Replace unconfirmed owners without checking for replay. */
release_openowner(oo);
oo = NULL;
+ goto retry;
}
if (oo) {
if (new)
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 8471371c6888..9c8767cab51d 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -1849,10 +1849,11 @@ static __be32 nfsd4_decode_secinfo_no_name(struct nfsd4_compoundargs *argp,
union nfsd4_op_u *u)
{
struct nfsd4_secinfo_no_name *sin = &u->secinfo_no_name;
+
+ sin->sin_exp = NULL;
if (xdr_stream_decode_u32(argp->xdr, &sin->sin_style) < 0)
return nfserr_bad_xdr;
- sin->sin_exp = NULL;
return nfs_ok;
}
diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c
index 08c8babfdd75..67d09fcee669 100644
--- a/fs/nfsd/vfs.c
+++ b/fs/nfsd/vfs.c
@@ -1203,8 +1203,10 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf,
nfsd_stats_io_write_add(nn, exp, *cnt);
fsnotify_modify(file);
host_err = filemap_check_wb_err(file->f_mapping, since);
- if (host_err < 0)
+ if (host_err < 0) {
+ commit_reset_write_verifier(nn, rqstp, host_err);
goto out_nfserr;
+ }
if (stable && fhp->fh_use_wgather) {
host_err = wait_for_concurrent_writes(file);
@@ -1370,6 +1372,8 @@ nfsd_commit(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf,
nfsd_copy_write_verifier(verf, nn);
err2 = filemap_check_wb_err(nf->nf_file->f_mapping,
since);
+ if (err2 < 0)
+ commit_reset_write_verifier(nn, rqstp, err2);
err = nfserrno(err2);
break;
case -EINVAL:
diff --git a/fs/ntfs3/xattr.c b/fs/ntfs3/xattr.c
index 6861c09d66d7..287534823346 100644
--- a/fs/ntfs3/xattr.c
+++ b/fs/ntfs3/xattr.c
@@ -829,6 +829,12 @@ static int ntfs_getxattr(const struct xattr_handler *handler, struct dentry *de,
return err;
}
+static bool ntfs_is_reserved_lxattr(const char *name)
+{
+ return !strcmp(name, "$LXUID") || !strcmp(name, "$LXGID") ||
+ !strcmp(name, "$LXMOD") || !strcmp(name, "$LXDEV");
+}
+
/*
* ntfs_setxattr - inode_operations::setxattr
*/
@@ -933,6 +939,12 @@ static noinline int ntfs_setxattr(const struct xattr_handler *handler,
goto out;
}
+ /* Do not allow non privileged users to change $LXUID/$LXGID... */
+ if (ntfs_is_reserved_lxattr(name) && !capable(CAP_SYS_ADMIN)) {
+ err = -EPERM;
+ goto out;
+ }
+
/* Deal with NTFS extended attribute. */
err = ntfs_set_ea(inode, name, strlen(name), value, size, flags, 0,
NULL);
diff --git a/fs/ocfs2/suballoc.c b/fs/ocfs2/suballoc.c
index e93fc842bb20..a752ea59a167 100644
--- a/fs/ocfs2/suballoc.c
+++ b/fs/ocfs2/suballoc.c
@@ -205,8 +205,16 @@ static int ocfs2_validate_gd_parent(struct super_block *sb,
int resize)
{
unsigned int max_bits;
+ unsigned int max_bitmap_bits;
+ unsigned int max_bitmap_size;
+ int suballocator;
struct ocfs2_group_desc *gd = (struct ocfs2_group_desc *)bh->b_data;
+ suballocator = le64_to_cpu(di->i_blkno) != OCFS2_SB(sb)->bitmap_blkno;
+ max_bitmap_size = ocfs2_group_bitmap_size(sb, suballocator,
+ OCFS2_SB(sb)->s_feature_incompat);
+ max_bitmap_bits = max_bitmap_size * 8;
+
if (di->i_blkno != gd->bg_parent_dinode) {
do_error("Group descriptor #%llu has bad parent pointer (%llu, expected %llu)\n",
(unsigned long long)bh->b_blocknr,
@@ -214,6 +222,20 @@ static int ocfs2_validate_gd_parent(struct super_block *sb,
(unsigned long long)le64_to_cpu(di->i_blkno));
}
+ if (le16_to_cpu(gd->bg_size) > max_bitmap_size) {
+ do_error("Group descriptor #%llu has bitmap size %u but physical max of %u\n",
+ (unsigned long long)bh->b_blocknr,
+ le16_to_cpu(gd->bg_size),
+ max_bitmap_size);
+ }
+
+ if (le16_to_cpu(gd->bg_bits) > max_bitmap_bits) {
+ do_error("Group descriptor #%llu has bit count %u but physical max of %u\n",
+ (unsigned long long)bh->b_blocknr,
+ le16_to_cpu(gd->bg_bits),
+ max_bitmap_bits);
+ }
+
max_bits = le16_to_cpu(di->id2.i_chain.cl_cpg) * le16_to_cpu(di->id2.i_chain.cl_bpc);
if (le16_to_cpu(gd->bg_bits) > max_bits) {
do_error("Group descriptor #%llu has bit count of %u\n",
diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c
index ab65e98a1def..1c8009bf194b 100644
--- a/fs/overlayfs/dir.c
+++ b/fs/overlayfs/dir.c
@@ -1320,7 +1320,7 @@ static int ovl_create_tmpfile(struct file *file, struct dentry *dentry,
goto out_revert_creds;
ovl_path_upper(dentry->d_parent, &realparentpath);
- realfile = backing_tmpfile_open(&file->f_path, flags, &realparentpath,
+ realfile = backing_tmpfile_open(file, flags, &realparentpath,
mode, current_cred());
err = PTR_ERR_OR_ZERO(realfile);
pr_debug("tmpfile/open(%pd2, 0%o) = %i\n", realparentpath.dentry, mode, err);
diff --git a/fs/overlayfs/file.c b/fs/overlayfs/file.c
index 94095058da34..3765e1defa19 100644
--- a/fs/overlayfs/file.c
+++ b/fs/overlayfs/file.c
@@ -47,8 +47,7 @@ static struct file *ovl_open_realfile(const struct file *file,
} else {
if (!inode_owner_or_capable(real_idmap, realinode))
flags &= ~O_NOATIME;
-
- realfile = backing_file_open(file_user_path((struct file *) file),
+ realfile = backing_file_open(file,
flags, realpath, current_cred());
}
revert_creds(old_cred);
diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c
index eb8ab4c8c7f4..162bcd46100e 100644
--- a/fs/smb/server/smb2pdu.c
+++ b/fs/smb/server/smb2pdu.c
@@ -612,6 +612,11 @@ int smb2_check_user_session(struct ksmbd_work *work)
sess_id, work->sess->id);
return -EINVAL;
}
+ if (work->sess->state != SMB2_SESSION_VALID) {
+ pr_err("compound request on a non-valid session (state %d)\n",
+ work->sess->state);
+ return -EINVAL;
+ }
return 1;
}
diff --git a/fs/smb/server/smbacl.c b/fs/smb/server/smbacl.c
index e3c512675c63..f5864dd1dd5f 100644
--- a/fs/smb/server/smbacl.c
+++ b/fs/smb/server/smbacl.c
@@ -1480,7 +1480,9 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
break;
aces_size -= ace_size;
- if (ace->sid.num_subauth > SID_MAX_SUB_AUTHORITIES)
+ if (ace->sid.num_subauth > SID_MAX_SUB_AUTHORITIES ||
+ ace_size < offsetof(struct smb_ace, sid) + CIFS_SID_BASE_SIZE +
+ sizeof(__le32) * ace->sid.num_subauth)
break;
if (!compare_sids(&sid, &ace->sid) ||
diff --git a/fs/xfs/libxfs/xfs_ag_resv.c b/fs/xfs/libxfs/xfs_ag_resv.c
index 216423df939e..4886fad98155 100644
--- a/fs/xfs/libxfs/xfs_ag_resv.c
+++ b/fs/xfs/libxfs/xfs_ag_resv.c
@@ -91,9 +91,9 @@ xfs_ag_resv_critical(
trace_xfs_ag_resv_critical(pag, type, avail);
/* Critically low if less than 10% or max btree height remains. */
- return XFS_TEST_ERROR(avail < orig / 10 ||
- avail < pag->pag_mount->m_agbtree_maxlevels,
- pag->pag_mount, XFS_ERRTAG_AG_RESV_CRITICAL);
+ return avail < orig / 10 ||
+ avail < pag->pag_mount->m_agbtree_maxlevels ||
+ XFS_TEST_ERROR(pag->pag_mount, XFS_ERRTAG_AG_RESV_CRITICAL);
}
/*
@@ -201,7 +201,7 @@ __xfs_ag_resv_init(
return -EINVAL;
}
- if (XFS_TEST_ERROR(false, mp, XFS_ERRTAG_AG_RESV_FAIL))
+ if (XFS_TEST_ERROR(mp, XFS_ERRTAG_AG_RESV_FAIL))
error = -ENOSPC;
else
error = xfs_dec_fdblocks(mp, hidden_space, true);
diff --git a/fs/xfs/libxfs/xfs_alloc.c b/fs/xfs/libxfs/xfs_alloc.c
index 22bdbb3e9980..3633f4d28c3b 100644
--- a/fs/xfs/libxfs/xfs_alloc.c
+++ b/fs/xfs/libxfs/xfs_alloc.c
@@ -3312,7 +3312,7 @@ xfs_agf_read_verify(
xfs_verifier_error(bp, -EFSBADCRC, __this_address);
else {
fa = xfs_agf_verify(bp);
- if (XFS_TEST_ERROR(fa, mp, XFS_ERRTAG_ALLOC_READ_AGF))
+ if (fa || XFS_TEST_ERROR(mp, XFS_ERRTAG_ALLOC_READ_AGF))
xfs_verifier_error(bp, -EFSCORRUPTED, fa);
}
}
@@ -3986,8 +3986,7 @@ __xfs_free_extent(
ASSERT(len != 0);
ASSERT(type != XFS_AG_RESV_AGFL);
- if (XFS_TEST_ERROR(false, mp,
- XFS_ERRTAG_FREE_EXTENT))
+ if (XFS_TEST_ERROR(mp, XFS_ERRTAG_FREE_EXTENT))
return -EIO;
error = xfs_free_extent_fix_freelist(tp, pag, &agbp);
diff --git a/fs/xfs/libxfs/xfs_attr_leaf.c b/fs/xfs/libxfs/xfs_attr_leaf.c
index bfcac9036c11..afdd3fc0a8be 100644
--- a/fs/xfs/libxfs/xfs_attr_leaf.c
+++ b/fs/xfs/libxfs/xfs_attr_leaf.c
@@ -1225,7 +1225,7 @@ xfs_attr3_leaf_to_node(
trace_xfs_attr_leaf_to_node(args);
- if (XFS_TEST_ERROR(false, mp, XFS_ERRTAG_ATTR_LEAF_TO_NODE)) {
+ if (XFS_TEST_ERROR(mp, XFS_ERRTAG_ATTR_LEAF_TO_NODE)) {
error = -EIO;
goto out;
}
diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index 36dd08d13293..b2b5a36ead97 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -3766,8 +3766,7 @@ xfs_bmap_btalloc(
/* Trim the allocation back to the maximum an AG can fit. */
args.maxlen = min(ap->length, mp->m_ag_max_usable);
- if (unlikely(XFS_TEST_ERROR(false, mp,
- XFS_ERRTAG_BMAP_ALLOC_MINLEN_EXTENT)))
+ if (unlikely(XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAP_ALLOC_MINLEN_EXTENT)))
error = xfs_bmap_exact_minlen_extent_alloc(ap, &args);
else if ((ap->datatype & XFS_ALLOC_USERDATA) &&
xfs_inode_is_filestream(ap->ip))
@@ -3953,7 +3952,7 @@ xfs_bmapi_read(
}
if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) ||
- XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) {
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) {
xfs_bmap_mark_sick(ip, whichfork);
return -EFSCORRUPTED;
}
@@ -4442,7 +4441,7 @@ xfs_bmapi_write(
(XFS_BMAPI_PREALLOC | XFS_BMAPI_ZERO));
if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) ||
- XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) {
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) {
xfs_bmap_mark_sick(ip, whichfork);
return -EFSCORRUPTED;
}
@@ -4785,7 +4784,7 @@ xfs_bmapi_remap(
(XFS_BMAPI_ATTRFORK | XFS_BMAPI_PREALLOC));
if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) ||
- XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) {
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) {
xfs_bmap_mark_sick(ip, whichfork);
return -EFSCORRUPTED;
}
@@ -5873,7 +5872,7 @@ xfs_bmap_collapse_extents(
int logflags = 0;
if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) ||
- XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) {
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) {
xfs_bmap_mark_sick(ip, whichfork);
return -EFSCORRUPTED;
}
@@ -5988,7 +5987,7 @@ xfs_bmap_insert_extents(
int logflags = 0;
if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) ||
- XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) {
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) {
xfs_bmap_mark_sick(ip, whichfork);
return -EFSCORRUPTED;
}
@@ -6092,7 +6091,7 @@ xfs_bmap_split_extent(
int i = 0;
if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) ||
- XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) {
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) {
xfs_bmap_mark_sick(ip, whichfork);
return -EFSCORRUPTED;
}
@@ -6257,7 +6256,7 @@ xfs_bmap_finish_one(
trace_xfs_bmap_deferred(bi);
- if (XFS_TEST_ERROR(false, tp->t_mountp, XFS_ERRTAG_BMAP_FINISH_ONE))
+ if (XFS_TEST_ERROR(tp->t_mountp, XFS_ERRTAG_BMAP_FINISH_ONE))
return -EIO;
switch (bi->bi_type) {
diff --git a/fs/xfs/libxfs/xfs_btree.c b/fs/xfs/libxfs/xfs_btree.c
index 134d87b3489a..f0ddaa136a0b 100644
--- a/fs/xfs/libxfs/xfs_btree.c
+++ b/fs/xfs/libxfs/xfs_btree.c
@@ -300,7 +300,7 @@ xfs_btree_check_block(
fa = __xfs_btree_check_block(cur, block, level, bp);
if (XFS_IS_CORRUPT(mp, fa != NULL) ||
- XFS_TEST_ERROR(false, mp, xfs_btree_block_errtag(cur))) {
+ XFS_TEST_ERROR(mp, xfs_btree_block_errtag(cur))) {
if (bp)
trace_xfs_btree_corrupt(bp, _RET_IP_);
xfs_btree_mark_sick(cur);
diff --git a/fs/xfs/libxfs/xfs_da_btree.c b/fs/xfs/libxfs/xfs_da_btree.c
index 723a0643b838..90f7fc219fcc 100644
--- a/fs/xfs/libxfs/xfs_da_btree.c
+++ b/fs/xfs/libxfs/xfs_da_btree.c
@@ -565,7 +565,7 @@ xfs_da3_split(
trace_xfs_da_split(state->args);
- if (XFS_TEST_ERROR(false, state->mp, XFS_ERRTAG_DA_LEAF_SPLIT))
+ if (XFS_TEST_ERROR(state->mp, XFS_ERRTAG_DA_LEAF_SPLIT))
return -EIO;
/*
diff --git a/fs/xfs/libxfs/xfs_dir2.c b/fs/xfs/libxfs/xfs_dir2.c
index 202468223bf9..945285060e65 100644
--- a/fs/xfs/libxfs/xfs_dir2.c
+++ b/fs/xfs/libxfs/xfs_dir2.c
@@ -223,7 +223,7 @@ xfs_dir_ino_validate(
bool ino_ok = xfs_verify_dir_ino(mp, ino);
if (XFS_IS_CORRUPT(mp, !ino_ok) ||
- XFS_TEST_ERROR(false, mp, XFS_ERRTAG_DIR_INO_VALIDATE)) {
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_DIR_INO_VALIDATE)) {
xfs_warn(mp, "Invalid inode number 0x%Lx",
(unsigned long long) ino);
return -EFSCORRUPTED;
diff --git a/fs/xfs/libxfs/xfs_exchmaps.c b/fs/xfs/libxfs/xfs_exchmaps.c
index 2021396651de..f9df18a84f65 100644
--- a/fs/xfs/libxfs/xfs_exchmaps.c
+++ b/fs/xfs/libxfs/xfs_exchmaps.c
@@ -616,7 +616,7 @@ xfs_exchmaps_finish_one(
return error;
}
- if (XFS_TEST_ERROR(false, tp->t_mountp, XFS_ERRTAG_EXCHMAPS_FINISH_ONE))
+ if (XFS_TEST_ERROR(tp->t_mountp, XFS_ERRTAG_EXCHMAPS_FINISH_ONE))
return -EIO;
/* If we still have work to do, ask for a new transaction. */
@@ -880,7 +880,7 @@ xmi_ensure_delta_nextents(
&new_nextents))
return -EFBIG;
- if (XFS_TEST_ERROR(false, mp, XFS_ERRTAG_REDUCE_MAX_IEXTENTS) &&
+ if (XFS_TEST_ERROR(mp, XFS_ERRTAG_REDUCE_MAX_IEXTENTS) &&
new_nextents > 10)
return -EFBIG;
diff --git a/fs/xfs/libxfs/xfs_ialloc.c b/fs/xfs/libxfs/xfs_ialloc.c
index 8223464e23e7..0fdb2a30e50d 100644
--- a/fs/xfs/libxfs/xfs_ialloc.c
+++ b/fs/xfs/libxfs/xfs_ialloc.c
@@ -2690,7 +2690,7 @@ xfs_agi_read_verify(
xfs_verifier_error(bp, -EFSBADCRC, __this_address);
else {
fa = xfs_agi_verify(bp);
- if (XFS_TEST_ERROR(fa, mp, XFS_ERRTAG_IALLOC_READ_AGI))
+ if (fa || XFS_TEST_ERROR(mp, XFS_ERRTAG_IALLOC_READ_AGI))
xfs_verifier_error(bp, -EFSCORRUPTED, fa);
}
}
diff --git a/fs/xfs/libxfs/xfs_inode_buf.c b/fs/xfs/libxfs/xfs_inode_buf.c
index 79babeac9d75..63f4abd87076 100644
--- a/fs/xfs/libxfs/xfs_inode_buf.c
+++ b/fs/xfs/libxfs/xfs_inode_buf.c
@@ -60,8 +60,8 @@ xfs_inode_buf_verify(
di_ok = xfs_verify_magic16(bp, dip->di_magic) &&
xfs_dinode_good_version(mp, dip->di_version) &&
xfs_verify_agino_or_null(bp->b_pag, unlinked_ino);
- if (unlikely(XFS_TEST_ERROR(!di_ok, mp,
- XFS_ERRTAG_ITOBP_INOTOBP))) {
+ if (unlikely(!di_ok ||
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_ITOBP_INOTOBP))) {
if (readahead) {
bp->b_flags &= ~XBF_DONE;
xfs_buf_ioerror(bp, -EIO);
diff --git a/fs/xfs/libxfs/xfs_inode_fork.c b/fs/xfs/libxfs/xfs_inode_fork.c
index 1158ca48626b..78af39caa0f1 100644
--- a/fs/xfs/libxfs/xfs_inode_fork.c
+++ b/fs/xfs/libxfs/xfs_inode_fork.c
@@ -795,8 +795,7 @@ xfs_iext_count_extend(
if (nr_exts < ifp->if_nextents)
return -EFBIG;
- if (XFS_TEST_ERROR(false, mp, XFS_ERRTAG_REDUCE_MAX_IEXTENTS) &&
- nr_exts > 10)
+ if (XFS_TEST_ERROR(mp, XFS_ERRTAG_REDUCE_MAX_IEXTENTS) && nr_exts > 10)
return -EFBIG;
if (nr_exts > xfs_iext_max_nextents(has_large, whichfork)) {
diff --git a/fs/xfs/libxfs/xfs_refcount.c b/fs/xfs/libxfs/xfs_refcount.c
index 198b84117df1..e26a2472c25d 100644
--- a/fs/xfs/libxfs/xfs_refcount.c
+++ b/fs/xfs/libxfs/xfs_refcount.c
@@ -1073,8 +1073,7 @@ xfs_refcount_still_have_space(
* refcount continue update "error" has been injected.
*/
if (cur->bc_refc.nr_ops > 2 &&
- XFS_TEST_ERROR(false, cur->bc_mp,
- XFS_ERRTAG_REFCOUNT_CONTINUE_UPDATE))
+ XFS_TEST_ERROR(cur->bc_mp, XFS_ERRTAG_REFCOUNT_CONTINUE_UPDATE))
return false;
if (cur->bc_refc.nr_ops == 0)
@@ -1353,7 +1352,7 @@ xfs_refcount_finish_one(
trace_xfs_refcount_deferred(mp, ri);
- if (XFS_TEST_ERROR(false, mp, XFS_ERRTAG_REFCOUNT_FINISH_ONE))
+ if (XFS_TEST_ERROR(mp, XFS_ERRTAG_REFCOUNT_FINISH_ONE))
return -EIO;
/*
diff --git a/fs/xfs/libxfs/xfs_rmap.c b/fs/xfs/libxfs/xfs_rmap.c
index 6ef4687b3aba..0fbec9134cb5 100644
--- a/fs/xfs/libxfs/xfs_rmap.c
+++ b/fs/xfs/libxfs/xfs_rmap.c
@@ -2579,7 +2579,7 @@ xfs_rmap_finish_one(
trace_xfs_rmap_deferred(mp, ri);
- if (XFS_TEST_ERROR(false, mp, XFS_ERRTAG_RMAP_FINISH_ONE))
+ if (XFS_TEST_ERROR(mp, XFS_ERRTAG_RMAP_FINISH_ONE))
return -EIO;
/*
diff --git a/fs/xfs/scrub/cow_repair.c b/fs/xfs/scrub/cow_repair.c
index 4de3f0f40f48..337bdd524b01 100644
--- a/fs/xfs/scrub/cow_repair.c
+++ b/fs/xfs/scrub/cow_repair.c
@@ -297,18 +297,15 @@ xrep_cow_find_bad(
* on the debugging knob, replace everything in the CoW fork.
*/
if ((sc->sm->sm_flags & XFS_SCRUB_IFLAG_FORCE_REBUILD) ||
- XFS_TEST_ERROR(false, sc->mp, XFS_ERRTAG_FORCE_SCRUB_REPAIR)) {
+ XFS_TEST_ERROR(sc->mp, XFS_ERRTAG_FORCE_SCRUB_REPAIR))
error = xrep_cow_mark_file_range(xc, xc->irec.br_startblock,
xc->irec.br_blockcount);
- if (error)
- return error;
- }
out_sa:
xchk_ag_free(sc, &sc->sa);
out_pag:
xfs_perag_put(pag);
- return 0;
+ return error;
}
/*
diff --git a/fs/xfs/scrub/repair.c b/fs/xfs/scrub/repair.c
index c19abc687072..4d763e7b6446 100644
--- a/fs/xfs/scrub/repair.c
+++ b/fs/xfs/scrub/repair.c
@@ -990,7 +990,7 @@ xrep_will_attempt(
return true;
/* Let debug users force us into the repair routines. */
- if (XFS_TEST_ERROR(false, sc->mp, XFS_ERRTAG_FORCE_SCRUB_REPAIR))
+ if (XFS_TEST_ERROR(sc->mp, XFS_ERRTAG_FORCE_SCRUB_REPAIR))
return true;
/* Metadata is corrupt or failed cross-referencing. */
diff --git a/fs/xfs/xfs_attr_item.c b/fs/xfs/xfs_attr_item.c
index 6a2ee93d5fff..b81eafc9543b 100644
--- a/fs/xfs/xfs_attr_item.c
+++ b/fs/xfs/xfs_attr_item.c
@@ -490,7 +490,7 @@ xfs_attr_finish_item(
/* Reset trans after EAGAIN cycle since the transaction is new */
args->trans = tp;
- if (XFS_TEST_ERROR(false, args->dp->i_mount, XFS_ERRTAG_LARP)) {
+ if (XFS_TEST_ERROR(args->dp->i_mount, XFS_ERRTAG_LARP)) {
error = -EIO;
goto out;
}
diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c
index afcdfe317b7e..1f6b2888bd41 100644
--- a/fs/xfs/xfs_buf.c
+++ b/fs/xfs/xfs_buf.c
@@ -1498,7 +1498,7 @@ xfs_buf_bio_end_io(
if (!bio->bi_status &&
(bp->b_flags & XBF_WRITE) && (bp->b_flags & XBF_ASYNC) &&
- XFS_TEST_ERROR(false, bp->b_mount, XFS_ERRTAG_BUF_IOERROR))
+ XFS_TEST_ERROR(bp->b_mount, XFS_ERRTAG_BUF_IOERROR))
bio->bi_status = BLK_STS_IOERR;
/*
@@ -2451,7 +2451,7 @@ void xfs_buf_set_ref(struct xfs_buf *bp, int lru_ref)
* This allows userspace to disrupt buffer caching for debug/testing
* purposes.
*/
- if (XFS_TEST_ERROR(false, bp->b_mount, XFS_ERRTAG_BUF_LRU_REF))
+ if (XFS_TEST_ERROR(bp->b_mount, XFS_ERRTAG_BUF_LRU_REF))
lru_ref = 0;
atomic_set(&bp->b_lru_ref, lru_ref);
diff --git a/fs/xfs/xfs_error.c b/fs/xfs/xfs_error.c
index 78cdc5064a8c..3b41cc591e92 100644
--- a/fs/xfs/xfs_error.c
+++ b/fs/xfs/xfs_error.c
@@ -292,7 +292,6 @@ xfs_errortag_enabled(
bool
xfs_errortag_test(
struct xfs_mount *mp,
- const char *expression,
const char *file,
int line,
unsigned int error_tag)
@@ -318,8 +317,8 @@ xfs_errortag_test(
return false;
xfs_warn_ratelimited(mp,
-"Injecting error (%s) at file %s, line %d, on filesystem \"%s\"",
- expression, file, line, mp->m_super->s_id);
+"Injecting error at file %s, line %d, on filesystem \"%s\"",
+ file, line, mp->m_super->s_id);
return true;
}
diff --git a/fs/xfs/xfs_error.h b/fs/xfs/xfs_error.h
index 0b9c5ba8a598..ff7792988b8a 100644
--- a/fs/xfs/xfs_error.h
+++ b/fs/xfs/xfs_error.h
@@ -41,10 +41,10 @@ extern void xfs_inode_verifier_error(struct xfs_inode *ip, int error,
#ifdef DEBUG
extern int xfs_errortag_init(struct xfs_mount *mp);
extern void xfs_errortag_del(struct xfs_mount *mp);
-extern bool xfs_errortag_test(struct xfs_mount *mp, const char *expression,
- const char *file, int line, unsigned int error_tag);
-#define XFS_TEST_ERROR(expr, mp, tag) \
- ((expr) || xfs_errortag_test((mp), #expr, __FILE__, __LINE__, (tag)))
+bool xfs_errortag_test(struct xfs_mount *mp, const char *file, int line,
+ unsigned int error_tag);
+#define XFS_TEST_ERROR(mp, tag) \
+ xfs_errortag_test((mp), __FILE__, __LINE__, (tag))
bool xfs_errortag_enabled(struct xfs_mount *mp, unsigned int tag);
#define XFS_ERRORTAG_DELAY(mp, tag) \
do { \
@@ -66,7 +66,7 @@ extern int xfs_errortag_clearall(struct xfs_mount *mp);
#else
#define xfs_errortag_init(mp) (0)
#define xfs_errortag_del(mp)
-#define XFS_TEST_ERROR(expr, mp, tag) (expr)
+#define XFS_TEST_ERROR(mp, tag) (false)
#define XFS_ERRORTAG_DELAY(mp, tag) ((void)0)
#define xfs_errortag_set(mp, tag, val) (ENOSYS)
#define xfs_errortag_add(mp, tag) (ENOSYS)
diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
index ed09b4a3084e..266fcc789098 100644
--- a/fs/xfs/xfs_inode.c
+++ b/fs/xfs/xfs_inode.c
@@ -2367,37 +2367,35 @@ xfs_iflush(
* error handling as the caller will shutdown and fail the buffer.
*/
error = -EFSCORRUPTED;
- if (XFS_TEST_ERROR(dip->di_magic != cpu_to_be16(XFS_DINODE_MAGIC),
- mp, XFS_ERRTAG_IFLUSH_1)) {
+ if (dip->di_magic != cpu_to_be16(XFS_DINODE_MAGIC) ||
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_IFLUSH_1)) {
xfs_alert_tag(mp, XFS_PTAG_IFLUSH,
"%s: Bad inode %llu magic number 0x%x, ptr "PTR_FMT,
__func__, ip->i_ino, be16_to_cpu(dip->di_magic), dip);
goto flush_out;
}
if (S_ISREG(VFS_I(ip)->i_mode)) {
- if (XFS_TEST_ERROR(
- ip->i_df.if_format != XFS_DINODE_FMT_EXTENTS &&
- ip->i_df.if_format != XFS_DINODE_FMT_BTREE,
- mp, XFS_ERRTAG_IFLUSH_3)) {
+ if ((ip->i_df.if_format != XFS_DINODE_FMT_EXTENTS &&
+ ip->i_df.if_format != XFS_DINODE_FMT_BTREE) ||
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_IFLUSH_3)) {
xfs_alert_tag(mp, XFS_PTAG_IFLUSH,
"%s: Bad regular inode %llu, ptr "PTR_FMT,
__func__, ip->i_ino, ip);
goto flush_out;
}
} else if (S_ISDIR(VFS_I(ip)->i_mode)) {
- if (XFS_TEST_ERROR(
- ip->i_df.if_format != XFS_DINODE_FMT_EXTENTS &&
- ip->i_df.if_format != XFS_DINODE_FMT_BTREE &&
- ip->i_df.if_format != XFS_DINODE_FMT_LOCAL,
- mp, XFS_ERRTAG_IFLUSH_4)) {
+ if ((ip->i_df.if_format != XFS_DINODE_FMT_EXTENTS &&
+ ip->i_df.if_format != XFS_DINODE_FMT_BTREE &&
+ ip->i_df.if_format != XFS_DINODE_FMT_LOCAL) ||
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_IFLUSH_4)) {
xfs_alert_tag(mp, XFS_PTAG_IFLUSH,
"%s: Bad directory inode %llu, ptr "PTR_FMT,
__func__, ip->i_ino, ip);
goto flush_out;
}
}
- if (XFS_TEST_ERROR(ip->i_df.if_nextents + xfs_ifork_nextents(&ip->i_af) >
- ip->i_nblocks, mp, XFS_ERRTAG_IFLUSH_5)) {
+ if (ip->i_df.if_nextents + xfs_ifork_nextents(&ip->i_af) >
+ ip->i_nblocks || XFS_TEST_ERROR(mp, XFS_ERRTAG_IFLUSH_5)) {
xfs_alert_tag(mp, XFS_PTAG_IFLUSH,
"%s: detected corrupt incore inode %llu, "
"total extents = %llu nblocks = %lld, ptr "PTR_FMT,
@@ -2406,8 +2404,8 @@ xfs_iflush(
ip->i_nblocks, ip);
goto flush_out;
}
- if (XFS_TEST_ERROR(ip->i_forkoff > mp->m_sb.sb_inodesize,
- mp, XFS_ERRTAG_IFLUSH_6)) {
+ if (ip->i_forkoff > mp->m_sb.sb_inodesize ||
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_IFLUSH_6)) {
xfs_alert_tag(mp, XFS_PTAG_IFLUSH,
"%s: bad inode %llu, forkoff 0x%x, ptr "PTR_FMT,
__func__, ip->i_ino, ip->i_forkoff, ip);
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index 6335b122486f..3483b3728b7b 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -993,7 +993,7 @@ xfs_buffered_write_iomap_begin(
return error;
if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(&ip->i_df)) ||
- XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) {
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) {
xfs_bmap_mark_sick(ip, XFS_DATA_FORK);
error = -EFSCORRUPTED;
goto out_unlock;
diff --git a/fs/xfs/xfs_log.c b/fs/xfs/xfs_log.c
index bf9245b1b4c6..e275dd265faf 100644
--- a/fs/xfs/xfs_log.c
+++ b/fs/xfs/xfs_log.c
@@ -968,8 +968,8 @@ xfs_log_unmount_write(
* counters will be recalculated. Refer to xlog_check_unmount_rec for
* more details.
*/
- if (XFS_TEST_ERROR(xfs_fs_has_sickness(mp, XFS_SICK_FS_COUNTERS), mp,
- XFS_ERRTAG_FORCE_SUMMARY_RECALC)) {
+ if (xfs_fs_has_sickness(mp, XFS_SICK_FS_COUNTERS) ||
+ XFS_TEST_ERROR(mp, XFS_ERRTAG_FORCE_SUMMARY_RECALC)) {
xfs_alert(mp, "%s: will fix summary counters at next mount",
__func__);
return;
@@ -1239,7 +1239,7 @@ xlog_ioend_work(
/*
* Race to shutdown the filesystem if we see an error.
*/
- if (XFS_TEST_ERROR(error, log->l_mp, XFS_ERRTAG_IODONE_IOERR)) {
+ if (error || XFS_TEST_ERROR(log->l_mp, XFS_ERRTAG_IODONE_IOERR)) {
xfs_alert(log->l_mp, "log I/O error %d", error);
xlog_force_shutdown(log, SHUTDOWN_LOG_IO_ERROR);
}
@@ -1848,7 +1848,7 @@ xlog_sync(
* detects the bad CRC and attempts to recover.
*/
#ifdef DEBUG
- if (XFS_TEST_ERROR(false, log->l_mp, XFS_ERRTAG_LOG_BAD_CRC)) {
+ if (XFS_TEST_ERROR(log->l_mp, XFS_ERRTAG_LOG_BAD_CRC)) {
iclog->ic_header.h_crc &= cpu_to_le32(0xAAAAAAAA);
iclog->ic_fail_crc = true;
xfs_warn(log->l_mp,
diff --git a/fs/xfs/xfs_trans_ail.c b/fs/xfs/xfs_trans_ail.c
index 85641af916a0..62d764205206 100644
--- a/fs/xfs/xfs_trans_ail.c
+++ b/fs/xfs/xfs_trans_ail.c
@@ -385,7 +385,7 @@ xfsaild_push_item(
* If log item pinning is enabled, skip the push and track the item as
* pinned. This can help induce head-behind-tail conditions.
*/
- if (XFS_TEST_ERROR(false, ailp->ail_log->l_mp, XFS_ERRTAG_LOG_ITEM_PIN))
+ if (XFS_TEST_ERROR(ailp->ail_log->l_mp, XFS_ERRTAG_LOG_ITEM_PIN))
return XFS_ITEM_PINNED;
/*
diff --git a/include/keys/request_key_auth-type.h b/include/keys/request_key_auth-type.h
index 36b89a933310..01e42ee5f409 100644
--- a/include/keys/request_key_auth-type.h
+++ b/include/keys/request_key_auth-type.h
@@ -9,12 +9,14 @@
#define _KEYS_REQUEST_KEY_AUTH_TYPE_H
#include <linux/key.h>
+#include <linux/refcount.h>
/*
* Authorisation record for request_key().
*/
struct request_key_auth {
struct rcu_head rcu;
+ refcount_t usage;
struct key *target_key;
struct key *dest_keyring;
const struct cred *cred;
diff --git a/include/linux/backing-file.h b/include/linux/backing-file.h
index 2eed0ffb5e8f..cd18acd7ac5b 100644
--- a/include/linux/backing-file.h
+++ b/include/linux/backing-file.h
@@ -19,10 +19,10 @@ struct backing_file_ctx {
void (*end_write)(struct file *, loff_t, ssize_t);
};
-struct file *backing_file_open(const struct path *user_path, int flags,
+struct file *backing_file_open(const struct file *user_file, int flags,
const struct path *real_path,
const struct cred *cred);
-struct file *backing_tmpfile_open(const struct path *user_path, int flags,
+struct file *backing_tmpfile_open(const struct file *user_file, int flags,
const struct path *real_parentpath,
umode_t mode, const struct cred *cred);
ssize_t backing_file_read_iter(struct file *file, struct iov_iter *iter,
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 11d0a1b8daa2..33f5b16c0fd9 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -1105,16 +1105,12 @@ static inline void blk_flush_plug(struct blk_plug *plug, bool async)
__blk_flush_plug(plug, async);
}
-/*
- * tsk == current here
- */
-static inline void blk_plug_invalidate_ts(struct task_struct *tsk)
+static __always_inline void blk_plug_invalidate_ts(void)
{
- struct blk_plug *plug = tsk->plug;
-
- if (plug)
- plug->cur_ktime = 0;
- current->flags &= ~PF_BLOCK_TS;
+ if (unlikely(current->flags & PF_BLOCK_TS)) {
+ current->plug->cur_ktime = 0;
+ current->flags &= ~PF_BLOCK_TS;
+ }
}
int blkdev_issue_flush(struct block_device *bdev);
@@ -1140,7 +1136,7 @@ static inline void blk_flush_plug(struct blk_plug *plug, bool async)
{
}
-static inline void blk_plug_invalidate_ts(struct task_struct *tsk)
+static inline void blk_plug_invalidate_ts(void)
{
}
diff --git a/include/linux/err.h b/include/linux/err.h
index 1d60aa86db53..281115ff0425 100644
--- a/include/linux/err.h
+++ b/include/linux/err.h
@@ -36,7 +36,7 @@
*
* Return: A pointer with @error encoded within its value.
*/
-static inline void * __must_check ERR_PTR(long error)
+static __always_inline void * __must_check ERR_PTR(long error)
{
return (void *) error;
}
@@ -52,7 +52,7 @@ static inline void * __must_check ERR_PTR(long error)
* @ptr: An error pointer.
* Return: The error code within @ptr.
*/
-static inline long __must_check PTR_ERR(__force const void *ptr)
+static __always_inline long __must_check PTR_ERR(__force const void *ptr)
{
return (long) ptr;
}
@@ -65,7 +65,7 @@ static inline long __must_check PTR_ERR(__force const void *ptr)
* @ptr: The pointer to check.
* Return: true if @ptr is an error pointer, false otherwise.
*/
-static inline bool __must_check IS_ERR(__force const void *ptr)
+static __always_inline bool __must_check IS_ERR(__force const void *ptr)
{
return IS_ERR_VALUE((unsigned long)ptr);
}
@@ -79,7 +79,7 @@ static inline bool __must_check IS_ERR(__force const void *ptr)
*
* Like IS_ERR(), but also returns true for a null pointer.
*/
-static inline bool __must_check IS_ERR_OR_NULL(__force const void *ptr)
+static __always_inline bool __must_check IS_ERR_OR_NULL(__force const void *ptr)
{
return unlikely(!ptr) || IS_ERR_VALUE((unsigned long)ptr);
}
@@ -91,7 +91,7 @@ static inline bool __must_check IS_ERR_OR_NULL(__force const void *ptr)
* Explicitly cast an error-valued pointer to another pointer type in such a
* way as to make it clear that's what's going on.
*/
-static inline void * __must_check ERR_CAST(__force const void *ptr)
+static __always_inline void * __must_check ERR_CAST(__force const void *ptr)
{
/* cast away the const */
return (void *) ptr;
@@ -114,7 +114,7 @@ static inline void * __must_check ERR_CAST(__force const void *ptr)
*
* Return: The error code within @ptr if it is an error pointer; 0 otherwise.
*/
-static inline int __must_check PTR_ERR_OR_ZERO(__force const void *ptr)
+static __always_inline int __must_check PTR_ERR_OR_ZERO(__force const void *ptr)
{
if (IS_ERR(ptr))
return PTR_ERR(ptr);
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 87720e1b5419..0eb43147dc87 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -2738,7 +2738,20 @@ struct file *dentry_open(const struct path *path, int flags,
const struct cred *creds);
struct file *dentry_create(const struct path *path, int flags, umode_t mode,
const struct cred *cred);
-struct path *backing_file_user_path(struct file *f);
+struct path *backing_file_user_path(const struct file *f);
+
+#ifdef CONFIG_SECURITY
+void *backing_file_security(const struct file *f);
+void backing_file_set_security(struct file *f, void *security);
+#else
+static inline void *backing_file_security(const struct file *f)
+{
+ return NULL;
+}
+static inline void backing_file_set_security(struct file *f, void *security)
+{
+}
+#endif /* CONFIG_SECURITY */
/*
* When mmapping a file on a stackable filesystem (e.g., overlayfs), the file
@@ -2750,14 +2763,14 @@ struct path *backing_file_user_path(struct file *f);
* by fstat() on that same fd.
*/
/* Get the path to display in /proc/<pid>/maps */
-static inline const struct path *file_user_path(struct file *f)
+static inline const struct path *file_user_path(const struct file *f)
{
if (unlikely(f->f_mode & FMODE_BACKING))
return backing_file_user_path(f);
return &f->f_path;
}
/* Get the inode whose inode number to display in /proc/<pid>/maps */
-static inline const struct inode *file_user_inode(struct file *f)
+static inline const struct inode *file_user_inode(const struct file *f)
{
if (unlikely(f->f_mode & FMODE_BACKING))
return d_inode(backing_file_user_path(f)->dentry);
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 04b81e2166d5..b4235e99f0a9 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -1745,6 +1745,11 @@ int kvm_request_irq_source_id(struct kvm *kvm);
void kvm_free_irq_source_id(struct kvm *kvm, int irq_source_id);
bool kvm_arch_irqfd_allowed(struct kvm *kvm, struct kvm_irqfd *args);
+static inline bool is_gfn_in_memslot(const struct kvm_memory_slot *slot, gfn_t gfn)
+{
+ return gfn >= slot->base_gfn && gfn < slot->base_gfn + slot->npages;
+}
+
/*
* Returns a pointer to the memslot if it contains gfn.
* Otherwise returns NULL.
@@ -1755,7 +1760,7 @@ try_get_memslot(struct kvm_memory_slot *slot, gfn_t gfn)
if (!slot)
return NULL;
- if (gfn >= slot->base_gfn && gfn < slot->base_gfn + slot->npages)
+ if (is_gfn_in_memslot(slot, gfn))
return slot;
else
return NULL;
diff --git a/include/linux/lsm_audit.h b/include/linux/lsm_audit.h
index 97a8b21eb033..c0a2839253fa 100644
--- a/include/linux/lsm_audit.h
+++ b/include/linux/lsm_audit.h
@@ -93,7 +93,7 @@ struct common_audit_data {
#endif
char *kmod_name;
struct lsm_ioctlop_audit *op;
- struct file *file;
+ const struct file *file;
struct lsm_ibpkey_audit *ibpkey;
struct lsm_ibendport_audit *ibendport;
int reason;
diff --git a/include/linux/lsm_hook_defs.h b/include/linux/lsm_hook_defs.h
index 9eca013aa5e1..addb34abffa1 100644
--- a/include/linux/lsm_hook_defs.h
+++ b/include/linux/lsm_hook_defs.h
@@ -188,6 +188,9 @@ LSM_HOOK(int, 0, file_permission, struct file *file, int mask)
LSM_HOOK(int, 0, file_alloc_security, struct file *file)
LSM_HOOK(void, LSM_RET_VOID, file_release, struct file *file)
LSM_HOOK(void, LSM_RET_VOID, file_free_security, struct file *file)
+LSM_HOOK(int, 0, backing_file_alloc, struct file *backing_file,
+ const struct file *user_file)
+LSM_HOOK(void, LSM_RET_VOID, backing_file_free, struct file *backing_file)
LSM_HOOK(int, 0, file_ioctl, struct file *file, unsigned int cmd,
unsigned long arg)
LSM_HOOK(int, 0, file_ioctl_compat, struct file *file, unsigned int cmd,
@@ -195,6 +198,8 @@ LSM_HOOK(int, 0, file_ioctl_compat, struct file *file, unsigned int cmd,
LSM_HOOK(int, 0, mmap_addr, unsigned long addr)
LSM_HOOK(int, 0, mmap_file, struct file *file, unsigned long reqprot,
unsigned long prot, unsigned long flags)
+LSM_HOOK(int, 0, mmap_backing_file, struct vm_area_struct *vma,
+ struct file *backing_file, struct file *user_file)
LSM_HOOK(int, 0, file_mprotect, struct vm_area_struct *vma,
unsigned long reqprot, unsigned long prot)
LSM_HOOK(int, 0, file_lock, struct file *file, unsigned int cmd)
diff --git a/include/linux/lsm_hooks.h b/include/linux/lsm_hooks.h
index 090d1d3e19fe..0876cf11e200 100644
--- a/include/linux/lsm_hooks.h
+++ b/include/linux/lsm_hooks.h
@@ -104,6 +104,7 @@ struct security_hook_list {
struct lsm_blob_sizes {
int lbs_cred;
int lbs_file;
+ int lbs_backing_file;
int lbs_ib;
int lbs_inode;
int lbs_sock;
diff --git a/include/linux/security.h b/include/linux/security.h
index 2c6db949ad1a..e4300f2ff11b 100644
--- a/include/linux/security.h
+++ b/include/linux/security.h
@@ -421,11 +421,17 @@ int security_file_permission(struct file *file, int mask);
int security_file_alloc(struct file *file);
void security_file_release(struct file *file);
void security_file_free(struct file *file);
+int security_backing_file_alloc(struct file *backing_file,
+ const struct file *user_file);
+void security_backing_file_free(struct file *backing_file);
int security_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
int security_file_ioctl_compat(struct file *file, unsigned int cmd,
unsigned long arg);
int security_mmap_file(struct file *file, unsigned long prot,
unsigned long flags);
+int security_mmap_backing_file(struct vm_area_struct *vma,
+ struct file *backing_file,
+ struct file *user_file);
int security_mmap_addr(unsigned long addr);
int security_file_mprotect(struct vm_area_struct *vma, unsigned long reqprot,
unsigned long prot);
@@ -1065,6 +1071,15 @@ static inline void security_file_release(struct file *file)
static inline void security_file_free(struct file *file)
{ }
+static inline int security_backing_file_alloc(struct file *backing_file,
+ const struct file *user_file)
+{
+ return 0;
+}
+
+static inline void security_backing_file_free(struct file *backing_file)
+{ }
+
static inline int security_file_ioctl(struct file *file, unsigned int cmd,
unsigned long arg)
{
@@ -1084,6 +1099,13 @@ static inline int security_mmap_file(struct file *file, unsigned long prot,
return 0;
}
+static inline int security_mmap_backing_file(struct vm_area_struct *vma,
+ struct file *backing_file,
+ struct file *user_file)
+{
+ return 0;
+}
+
static inline int security_mmap_addr(unsigned long addr)
{
return cap_mmap_addr(addr);
diff --git a/include/linux/skmsg.h b/include/linux/skmsg.h
index 5581e7263c50..ce8464324d42 100644
--- a/include/linux/skmsg.h
+++ b/include/linux/skmsg.h
@@ -4,6 +4,7 @@
#ifndef _LINUX_SKMSG_H
#define _LINUX_SKMSG_H
+#include <linux/bitops.h>
#include <linux/bpf.h>
#include <linux/filter.h>
#include <linux/scatterlist.h>
@@ -199,11 +200,14 @@ static inline void sk_msg_xfer(struct sk_msg *dst, struct sk_msg *src,
int which, u32 size)
{
dst->sg.data[which] = src->sg.data[which];
+ __assign_bit(which, dst->sg.copy, test_bit(which, src->sg.copy));
dst->sg.data[which].length = size;
dst->sg.size += size;
src->sg.size -= size;
src->sg.data[which].length -= size;
src->sg.data[which].offset += size;
+ if (!src->sg.data[which].length)
+ __clear_bit(which, src->sg.copy);
}
static inline void sk_msg_xfer_full(struct sk_msg *dst, struct sk_msg *src)
@@ -273,16 +277,19 @@ static inline void sk_msg_page_add(struct sk_msg *msg, struct page *page,
static inline void sk_msg_sg_copy(struct sk_msg *msg, u32 i, bool copy_state)
{
do {
- if (copy_state)
- __set_bit(i, msg->sg.copy);
- else
- __clear_bit(i, msg->sg.copy);
+ __assign_bit(i, msg->sg.copy, copy_state);
sk_msg_iter_var_next(i);
if (i == msg->sg.end)
break;
} while (1);
}
+static inline void sk_msg_sg_copy_assign(struct sk_msg *dst, u32 dst_i,
+ const struct sk_msg *src, u32 src_i)
+{
+ __assign_bit(dst_i, dst->sg.copy, test_bit(src_i, src->sg.copy));
+}
+
static inline void sk_msg_sg_copy_set(struct sk_msg *msg, u32 start)
{
sk_msg_sg_copy(msg, start, true);
diff --git a/include/net/bond_3ad.h b/include/net/bond_3ad.h
index dba369a2cf27..4ee2ccf46279 100644
--- a/include/net/bond_3ad.h
+++ b/include/net/bond_3ad.h
@@ -242,7 +242,7 @@ typedef struct port {
churn_state_t sm_churn_actor_state;
churn_state_t sm_churn_partner_state;
struct slave *slave; /* pointer to the bond slave that this port belongs to */
- struct aggregator *aggregator; /* pointer to an aggregator that this port related to */
+ struct aggregator __rcu *aggregator; /* pointer to an aggregator that this port related to */
struct port *next_port_in_aggregator; /* Next port on the linked list of the parent aggregator */
u32 transaction_id; /* continuous number for identification of Marker PDU's; */
struct lacpdu lacpdu; /* the lacpdu that will be sent for this port */
@@ -274,6 +274,7 @@ struct ad_slave_info {
struct port port; /* 802.3ad port structure */
struct bond_3ad_stats stats;
u16 id;
+ u16 port_priority;
};
static inline const char *bond_3ad_churn_desc(churn_state_t state)
diff --git a/include/net/bond_options.h b/include/net/bond_options.h
index 18687ccf0638..e6eedf23aea1 100644
--- a/include/net/bond_options.h
+++ b/include/net/bond_options.h
@@ -77,6 +77,8 @@ enum {
BOND_OPT_NS_TARGETS,
BOND_OPT_PRIO,
BOND_OPT_COUPLED_CONTROL,
+ BOND_OPT_BROADCAST_NEIGH,
+ BOND_OPT_ACTOR_PORT_PRIO,
BOND_OPT_LAST
};
diff --git a/include/net/bonding.h b/include/net/bonding.h
index 66940d41d485..0d9c1eb40d12 100644
--- a/include/net/bonding.h
+++ b/include/net/bonding.h
@@ -115,6 +115,8 @@ static inline int is_netpoll_tx_blocked(struct net_device *dev)
#define is_netpoll_tx_blocked(dev) (0)
#endif
+DECLARE_STATIC_KEY_FALSE(bond_bcast_neigh_enabled);
+
struct bond_params {
int mode;
int xmit_policy;
@@ -149,6 +151,7 @@ struct bond_params {
struct in6_addr ns_targets[BOND_MAX_NS_TARGETS];
#endif
int coupled_control;
+ int broadcast_neighbor;
/* 2 bytes of padding : see ether_addr_equal_64bits() */
u8 ad_actor_system[ETH_ALEN + 2];
diff --git a/include/net/phonet/pn_dev.h b/include/net/phonet/pn_dev.h
index e9dc8dca5817..6b2102b4ece3 100644
--- a/include/net/phonet/pn_dev.h
+++ b/include/net/phonet/pn_dev.h
@@ -38,7 +38,7 @@ int phonet_address_add(struct net_device *dev, u8 addr);
int phonet_address_del(struct net_device *dev, u8 addr);
u8 phonet_address_get(struct net_device *dev, u8 addr);
int phonet_address_lookup(struct net *net, u8 addr);
-void phonet_address_notify(int event, struct net_device *dev, u8 addr);
+void phonet_address_notify(struct net *net, int event, u32 ifindex, u8 addr);
int phonet_route_add(struct net_device *dev, u8 daddr);
int phonet_route_del(struct net_device *dev, u8 daddr);
diff --git a/include/net/rtnetlink.h b/include/net/rtnetlink.h
index 2d3eb7cb4dff..7c057611f4ab 100644
--- a/include/net/rtnetlink.h
+++ b/include/net/rtnetlink.h
@@ -212,6 +212,8 @@ int rtnl_configure_link(struct net_device *dev, const struct ifinfomsg *ifm,
int rtnl_nla_parse_ifinfomsg(struct nlattr **tb, const struct nlattr *nla_peer,
struct netlink_ext_ack *exterr);
struct net *rtnl_get_net_ns_capable(struct sock *sk, int netnsid);
+bool rtnl_dev_link_net_capable(const struct net_device *dev,
+ const struct net *link_net);
#define MODULE_ALIAS_RTNL_LINK(kind) MODULE_ALIAS("rtnl-link-" kind)
diff --git a/include/net/sock.h b/include/net/sock.h
index 0d77a87929f9..dffbaaa7fe49 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -2040,7 +2040,7 @@ static inline int sk_rx_queue_get(const struct sock *sk)
static inline void sk_set_socket(struct sock *sk, struct socket *sock)
{
- sk->sk_socket = sock;
+ WRITE_ONCE(sk->sk_socket, sock);
}
static inline wait_queue_head_t *sk_sleep(struct sock *sk)
diff --git a/include/net/udp_tunnel.h b/include/net/udp_tunnel.h
index 6e2c5c77031f..8ed36ec520d7 100644
--- a/include/net/udp_tunnel.h
+++ b/include/net/udp_tunnel.h
@@ -152,13 +152,13 @@ void udp_tunnel_xmit_skb(struct rtable *rt, struct sock *sk, struct sk_buff *skb
__be16 df, __be16 src_port, __be16 dst_port,
bool xnet, bool nocheck);
-int udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk,
- struct sk_buff *skb,
- struct net_device *dev,
- const struct in6_addr *saddr,
- const struct in6_addr *daddr,
- __u8 prio, __u8 ttl, __be32 label,
- __be16 src_port, __be16 dst_port, bool nocheck);
+void udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk,
+ struct sk_buff *skb,
+ struct net_device *dev,
+ const struct in6_addr *saddr,
+ const struct in6_addr *daddr,
+ __u8 prio, __u8 ttl, __be32 label,
+ __be16 src_port, __be16 dst_port, bool nocheck);
void udp_tunnel_sock_release(struct socket *sock);
diff --git a/include/uapi/linux/if_link.h b/include/uapi/linux/if_link.h
index 2acc7687e017..087048d620f3 100644
--- a/include/uapi/linux/if_link.h
+++ b/include/uapi/linux/if_link.h
@@ -1551,6 +1551,9 @@ enum {
IFLA_BOND_SLAVE_AD_ACTOR_OPER_PORT_STATE,
IFLA_BOND_SLAVE_AD_PARTNER_OPER_PORT_STATE,
IFLA_BOND_SLAVE_PRIO,
+ IFLA_BOND_SLAVE_ACTOR_PORT_PRIO,
+ IFLA_BOND_SLAVE_AD_CHURN_ACTOR_STATE,
+ IFLA_BOND_SLAVE_AD_CHURN_PARTNER_STATE,
__IFLA_BOND_SLAVE_MAX,
};
diff --git a/io_uring/net.c b/io_uring/net.c
index 8eb0ebdc6a72..f2f2ae1037e9 100644
--- a/io_uring/net.c
+++ b/io_uring/net.c
@@ -1719,7 +1719,7 @@ int io_socket(struct io_kiocb *req, unsigned int issue_flags)
int io_connect_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
{
struct io_connect *conn = io_kiocb_to_cmd(req, struct io_connect);
- struct io_async_msghdr *io;
+ struct sockaddr_storage *addr;
if (sqe->len || sqe->buf_index || sqe->rw_flags || sqe->splice_fd_in)
return -EINVAL;
@@ -1728,17 +1728,17 @@ int io_connect_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
conn->addr_len = READ_ONCE(sqe->addr2);
conn->in_progress = conn->seen_econnaborted = false;
- io = io_msg_alloc_async(req);
- if (unlikely(!io))
+ if (io_alloc_async_data(req))
return -ENOMEM;
+ addr = req->async_data;
- return move_addr_to_kernel(conn->addr, conn->addr_len, &io->addr);
+ return move_addr_to_kernel(conn->addr, conn->addr_len, addr);
}
int io_connect(struct io_kiocb *req, unsigned int issue_flags)
{
struct io_connect *connect = io_kiocb_to_cmd(req, struct io_connect);
- struct io_async_msghdr *io = req->async_data;
+ struct sockaddr_storage *addr = req->async_data;
unsigned file_flags;
int ret;
bool force_nonblock = issue_flags & IO_URING_F_NONBLOCK;
@@ -1752,8 +1752,7 @@ int io_connect(struct io_kiocb *req, unsigned int issue_flags)
file_flags = force_nonblock ? O_NONBLOCK : 0;
- ret = __sys_connect_file(req->file, &io->addr, connect->addr_len,
- file_flags);
+ ret = __sys_connect_file(req->file, addr, connect->addr_len, file_flags);
if ((ret == -EAGAIN || ret == -EINPROGRESS || ret == -ECONNABORTED)
&& force_nonblock) {
if (ret == -EINPROGRESS) {
@@ -1782,7 +1781,6 @@ int io_connect(struct io_kiocb *req, unsigned int issue_flags)
out:
if (ret < 0)
req_set_fail(req);
- io_req_msg_cleanup(req, issue_flags);
io_req_set_res(req, ret, 0);
return IOU_OK;
}
@@ -1792,15 +1790,15 @@ int io_connect(struct io_kiocb *req, unsigned int issue_flags)
* which in turn end up in mnt_want_write() which will grab the fs
* percpu start write sem. This can trigger a lockdep warning.
*/
-static int io_bind_file_create(const struct io_async_msghdr *io, int addr_len)
+static int io_bind_file_create(const struct sockaddr_storage *addr, int addr_len)
{
const struct sockaddr_un *sun;
- if (io->addr.ss_family != AF_UNIX)
+ if (addr->ss_family != AF_UNIX)
return 0;
if (addr_len <= offsetof(struct sockaddr_un, sun_path))
return 0;
- sun = (const struct sockaddr_un *) &io->addr;
+ sun = (const struct sockaddr_un *) addr;
return sun->sun_path[0] != '\0';
}
@@ -1808,7 +1806,7 @@ int io_bind_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
{
struct io_bind *bind = io_kiocb_to_cmd(req, struct io_bind);
struct sockaddr __user *uaddr;
- struct io_async_msghdr *io;
+ struct sockaddr_storage *addr;
int ret;
if (sqe->len || sqe->buf_index || sqe->rw_flags || sqe->splice_fd_in)
@@ -1817,21 +1815,23 @@ int io_bind_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
uaddr = u64_to_user_ptr(READ_ONCE(sqe->addr));
bind->addr_len = READ_ONCE(sqe->addr2);
- io = io_msg_alloc_async(req);
- if (unlikely(!io))
+ if (io_alloc_async_data(req))
return -ENOMEM;
- ret = move_addr_to_kernel(uaddr, bind->addr_len, &io->addr);
+ addr = req->async_data;
+
+ ret = move_addr_to_kernel(uaddr, bind->addr_len, addr);
if (unlikely(ret))
return ret;
- if (io_bind_file_create(io, bind->addr_len))
+ if (io_bind_file_create(addr, bind->addr_len))
req->flags |= REQ_F_FORCE_ASYNC;
return 0;
}
+
int io_bind(struct io_kiocb *req, unsigned int issue_flags)
{
struct io_bind *bind = io_kiocb_to_cmd(req, struct io_bind);
- struct io_async_msghdr *io = req->async_data;
+ struct sockaddr_storage *addr = req->async_data;
struct socket *sock;
int ret;
@@ -1839,7 +1839,7 @@ int io_bind(struct io_kiocb *req, unsigned int issue_flags)
if (unlikely(!sock))
return -ENOTSOCK;
- ret = __sys_bind_socket(sock, &io->addr, bind->addr_len);
+ ret = __sys_bind_socket(sock, addr, bind->addr_len);
if (ret < 0)
req_set_fail(req);
io_req_set_res(req, ret, 0);
diff --git a/io_uring/opdef.c b/io_uring/opdef.c
index 5dc1cba158a0..bbb62d2ab2a3 100644
--- a/io_uring/opdef.c
+++ b/io_uring/opdef.c
@@ -205,7 +205,7 @@ const struct io_issue_def io_issue_defs[] = {
.unbound_nonreg_file = 1,
.pollout = 1,
#if defined(CONFIG_NET)
- .async_size = sizeof(struct io_async_msghdr),
+ .async_size = sizeof(struct sockaddr_storage),
.prep = io_connect_prep,
.issue = io_connect,
#else
@@ -501,7 +501,7 @@ const struct io_issue_def io_issue_defs[] = {
.needs_file = 1,
.prep = io_bind_prep,
.issue = io_bind,
- .async_size = sizeof(struct io_async_msghdr),
+ .async_size = sizeof(struct sockaddr_storage),
#else
.prep = io_eopnotsupp_prep,
#endif
diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
index 1ebf40badbf6..9b89732f2cb4 100644
--- a/kernel/bpf/cgroup.c
+++ b/kernel/bpf/cgroup.c
@@ -1774,7 +1774,7 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head,
kfree(ctx.cur_val);
if (ret == 1 && ctx.new_updated) {
- kfree(*buf);
+ kvfree(*buf);
*buf = ctx.new_val;
*pcount = ctx.new_len;
} else {
diff --git a/kernel/fork.c b/kernel/fork.c
index c4955cffcb6f..c81b45509e8a 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -2450,6 +2450,7 @@ __latent_entropy struct task_struct *copy_process(
#ifdef CONFIG_BLOCK
p->plug = NULL;
+ p->flags &= ~PF_BLOCK_TS;
#endif
futex_init_task(p);
diff --git a/kernel/futex/pi.c b/kernel/futex/pi.c
index b4cdda1ea130..ccd5d19dea68 100644
--- a/kernel/futex/pi.c
+++ b/kernel/futex/pi.c
@@ -922,6 +922,7 @@ int futex_lock_pi(u32 __user *uaddr, unsigned int flags, ktime_t *time, int tryl
struct rt_mutex_waiter rt_waiter;
struct futex_hash_bucket *hb;
struct futex_q q = futex_q_init;
+ DEFINE_WAKE_Q(wake_q);
int res, ret;
if (!IS_ENABLED(CONFIG_FUTEX_PI))
@@ -1019,8 +1020,11 @@ int futex_lock_pi(u32 __user *uaddr, unsigned int flags, ktime_t *time, int tryl
* such that futex_unlock_pi() is guaranteed to observe the waiter when
* it sees the futex_q::pi_state.
*/
- ret = __rt_mutex_start_proxy_lock(&q.pi_state->pi_mutex, &rt_waiter, current);
+ ret = __rt_mutex_start_proxy_lock(&q.pi_state->pi_mutex, &rt_waiter, current, &wake_q);
+ preempt_disable();
raw_spin_unlock_irq(&q.pi_state->pi_mutex.wait_lock);
+ wake_up_q(&wake_q);
+ preempt_enable();
if (ret) {
if (ret == 1)
diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c
index cbae8c0b89ab..6c94da061ec2 100644
--- a/kernel/locking/mutex.c
+++ b/kernel/locking/mutex.c
@@ -575,6 +575,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas
struct lockdep_map *nest_lock, unsigned long ip,
struct ww_acquire_ctx *ww_ctx, const bool use_ww_ctx)
{
+ DEFINE_WAKE_Q(wake_q);
struct mutex_waiter waiter;
struct ww_mutex *ww;
int ret;
@@ -625,7 +626,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas
*/
if (__mutex_trylock(lock)) {
if (ww_ctx)
- __ww_mutex_check_waiters(lock, ww_ctx);
+ __ww_mutex_check_waiters(lock, ww_ctx, &wake_q);
goto skip_wait;
}
@@ -645,7 +646,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas
* Add in stamp order, waking up waiters that must kill
* themselves.
*/
- ret = __ww_mutex_add_waiter(&waiter, lock, ww_ctx);
+ ret = __ww_mutex_add_waiter(&waiter, lock, ww_ctx, &wake_q);
if (ret)
goto err_early_kill;
}
@@ -681,6 +682,10 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas
}
raw_spin_unlock(&lock->wait_lock);
+ /* Make sure we do wakeups before calling schedule */
+ wake_up_q(&wake_q);
+ wake_q_init(&wake_q);
+
schedule_preempt_disabled();
first = __mutex_waiter_is_first(lock, &waiter);
@@ -714,7 +719,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas
*/
if (!ww_ctx->is_wait_die &&
!__mutex_waiter_is_first(lock, &waiter))
- __ww_mutex_check_waiters(lock, ww_ctx);
+ __ww_mutex_check_waiters(lock, ww_ctx, &wake_q);
}
__mutex_remove_waiter(lock, &waiter);
@@ -730,6 +735,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas
ww_mutex_lock_acquired(ww, ww_ctx);
raw_spin_unlock(&lock->wait_lock);
+ wake_up_q(&wake_q);
preempt_enable();
return 0;
@@ -741,6 +747,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas
raw_spin_unlock(&lock->wait_lock);
debug_mutex_free_waiter(&waiter);
mutex_release(&lock->dep_map, ip);
+ wake_up_q(&wake_q);
preempt_enable();
return ret;
}
@@ -951,9 +958,10 @@ static noinline void __sched __mutex_unlock_slowpath(struct mutex *lock, unsigne
if (owner & MUTEX_FLAG_HANDOFF)
__mutex_handoff(lock, next);
+ preempt_disable();
raw_spin_unlock(&lock->wait_lock);
-
wake_up_q(&wake_q);
+ preempt_enable();
}
#ifndef CONFIG_DEBUG_LOCK_ALLOC
diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c
index 66f9fb25d4be..9b4bd06dd9ac 100644
--- a/kernel/locking/rtmutex.c
+++ b/kernel/locking/rtmutex.c
@@ -34,13 +34,15 @@
static inline int __ww_mutex_add_waiter(struct rt_mutex_waiter *waiter,
struct rt_mutex *lock,
- struct ww_acquire_ctx *ww_ctx)
+ struct ww_acquire_ctx *ww_ctx,
+ struct wake_q_head *wake_q)
{
return 0;
}
static inline void __ww_mutex_check_waiters(struct rt_mutex *lock,
- struct ww_acquire_ctx *ww_ctx)
+ struct ww_acquire_ctx *ww_ctx,
+ struct wake_q_head *wake_q)
{
}
@@ -1201,7 +1203,8 @@ static int __sched task_blocks_on_rt_mutex(struct rt_mutex_base *lock,
struct rt_mutex_waiter *waiter,
struct task_struct *task,
struct ww_acquire_ctx *ww_ctx,
- enum rtmutex_chainwalk chwalk)
+ enum rtmutex_chainwalk chwalk,
+ struct wake_q_head *wake_q)
{
struct task_struct *owner = rt_mutex_owner(lock);
struct rt_mutex_waiter *top_waiter = waiter;
@@ -1245,7 +1248,7 @@ static int __sched task_blocks_on_rt_mutex(struct rt_mutex_base *lock,
/* Check whether the waiter should back out immediately */
rtm = container_of(lock, struct rt_mutex, rtmutex);
- res = __ww_mutex_add_waiter(waiter, rtm, ww_ctx);
+ res = __ww_mutex_add_waiter(waiter, rtm, ww_ctx, wake_q);
if (res) {
raw_spin_lock(&task->pi_lock);
rt_mutex_dequeue(lock, waiter);
@@ -1544,6 +1547,9 @@ static void __sched remove_waiter(struct rt_mutex_base *lock,
lockdep_assert_held(&lock->wait_lock);
+ if (!waiter_task) /* never enqueued */
+ return;
+
scoped_guard(raw_spinlock, &waiter_task->pi_lock) {
rt_mutex_dequeue(lock, waiter);
waiter_task->pi_blocked_on = NULL;
@@ -1677,12 +1683,14 @@ static void __sched rt_mutex_handle_deadlock(int res, int detect_deadlock,
* @state: The task state for sleeping
* @chwalk: Indicator whether full or partial chainwalk is requested
* @waiter: Initializer waiter for blocking
+ * @wake_q: The wake_q to wake tasks after we release the wait_lock
*/
static int __sched __rt_mutex_slowlock(struct rt_mutex_base *lock,
struct ww_acquire_ctx *ww_ctx,
unsigned int state,
enum rtmutex_chainwalk chwalk,
- struct rt_mutex_waiter *waiter)
+ struct rt_mutex_waiter *waiter,
+ struct wake_q_head *wake_q)
{
struct rt_mutex *rtm = container_of(lock, struct rt_mutex, rtmutex);
struct ww_mutex *ww = ww_container_of(rtm);
@@ -1693,7 +1701,7 @@ static int __sched __rt_mutex_slowlock(struct rt_mutex_base *lock,
/* Try to acquire the lock again: */
if (try_to_take_rt_mutex(lock, current, NULL)) {
if (build_ww_mutex() && ww_ctx) {
- __ww_mutex_check_waiters(rtm, ww_ctx);
+ __ww_mutex_check_waiters(rtm, ww_ctx, wake_q);
ww_mutex_lock_acquired(ww, ww_ctx);
}
return 0;
@@ -1703,7 +1711,7 @@ static int __sched __rt_mutex_slowlock(struct rt_mutex_base *lock,
trace_contention_begin(lock, LCB_F_RT);
- ret = task_blocks_on_rt_mutex(lock, waiter, current, ww_ctx, chwalk);
+ ret = task_blocks_on_rt_mutex(lock, waiter, current, ww_ctx, chwalk, wake_q);
if (likely(!ret))
ret = rt_mutex_slowlock_block(lock, ww_ctx, state, NULL, waiter);
@@ -1711,7 +1719,7 @@ static int __sched __rt_mutex_slowlock(struct rt_mutex_base *lock,
/* acquired the lock */
if (build_ww_mutex() && ww_ctx) {
if (!ww_ctx->is_wait_die)
- __ww_mutex_check_waiters(rtm, ww_ctx);
+ __ww_mutex_check_waiters(rtm, ww_ctx, wake_q);
ww_mutex_lock_acquired(ww, ww_ctx);
}
} else {
@@ -1733,7 +1741,8 @@ static int __sched __rt_mutex_slowlock(struct rt_mutex_base *lock,
static inline int __rt_mutex_slowlock_locked(struct rt_mutex_base *lock,
struct ww_acquire_ctx *ww_ctx,
- unsigned int state)
+ unsigned int state,
+ struct wake_q_head *wake_q)
{
struct rt_mutex_waiter waiter;
int ret;
@@ -1742,7 +1751,7 @@ static inline int __rt_mutex_slowlock_locked(struct rt_mutex_base *lock,
waiter.ww_ctx = ww_ctx;
ret = __rt_mutex_slowlock(lock, ww_ctx, state, RT_MUTEX_MIN_CHAINWALK,
- &waiter);
+ &waiter, wake_q);
debug_rt_mutex_free_waiter(&waiter);
return ret;
@@ -1758,6 +1767,7 @@ static int __sched rt_mutex_slowlock(struct rt_mutex_base *lock,
struct ww_acquire_ctx *ww_ctx,
unsigned int state)
{
+ DEFINE_WAKE_Q(wake_q);
unsigned long flags;
int ret;
@@ -1779,8 +1789,11 @@ static int __sched rt_mutex_slowlock(struct rt_mutex_base *lock,
* irqsave/restore variants.
*/
raw_spin_lock_irqsave(&lock->wait_lock, flags);
- ret = __rt_mutex_slowlock_locked(lock, ww_ctx, state);
+ ret = __rt_mutex_slowlock_locked(lock, ww_ctx, state, &wake_q);
+ preempt_disable();
raw_spin_unlock_irqrestore(&lock->wait_lock, flags);
+ wake_up_q(&wake_q);
+ preempt_enable();
rt_mutex_post_schedule();
return ret;
@@ -1806,8 +1819,10 @@ static __always_inline int __rt_mutex_lock(struct rt_mutex_base *lock,
/**
* rtlock_slowlock_locked - Slow path lock acquisition for RT locks
* @lock: The underlying RT mutex
+ * @wake_q: The wake_q to wake tasks after we release the wait_lock
*/
-static void __sched rtlock_slowlock_locked(struct rt_mutex_base *lock)
+static void __sched rtlock_slowlock_locked(struct rt_mutex_base *lock,
+ struct wake_q_head *wake_q)
{
struct rt_mutex_waiter waiter;
struct task_struct *owner;
@@ -1824,7 +1839,7 @@ static void __sched rtlock_slowlock_locked(struct rt_mutex_base *lock)
trace_contention_begin(lock, LCB_F_RT);
- task_blocks_on_rt_mutex(lock, &waiter, current, NULL, RT_MUTEX_MIN_CHAINWALK);
+ task_blocks_on_rt_mutex(lock, &waiter, current, NULL, RT_MUTEX_MIN_CHAINWALK, wake_q);
for (;;) {
/* Try to acquire the lock again */
@@ -1835,7 +1850,11 @@ static void __sched rtlock_slowlock_locked(struct rt_mutex_base *lock)
owner = rt_mutex_owner(lock);
else
owner = NULL;
+ preempt_disable();
raw_spin_unlock_irq(&lock->wait_lock);
+ wake_up_q(wake_q);
+ wake_q_init(wake_q);
+ preempt_enable();
if (!owner || !rtmutex_spin_on_owner(lock, &waiter, owner))
schedule_rtlock();
@@ -1860,10 +1879,14 @@ static void __sched rtlock_slowlock_locked(struct rt_mutex_base *lock)
static __always_inline void __sched rtlock_slowlock(struct rt_mutex_base *lock)
{
unsigned long flags;
+ DEFINE_WAKE_Q(wake_q);
raw_spin_lock_irqsave(&lock->wait_lock, flags);
- rtlock_slowlock_locked(lock);
+ rtlock_slowlock_locked(lock, &wake_q);
+ preempt_disable();
raw_spin_unlock_irqrestore(&lock->wait_lock, flags);
+ wake_up_q(&wake_q);
+ preempt_enable();
}
#endif /* RT_MUTEX_BUILD_SPINLOCKS */
diff --git a/kernel/locking/rtmutex_api.c b/kernel/locking/rtmutex_api.c
index a6974d044593..091b109acdc5 100644
--- a/kernel/locking/rtmutex_api.c
+++ b/kernel/locking/rtmutex_api.c
@@ -275,6 +275,7 @@ void __sched rt_mutex_proxy_unlock(struct rt_mutex_base *lock)
* @lock: the rt_mutex to take
* @waiter: the pre-initialized rt_mutex_waiter
* @task: the task to prepare
+ * @wake_q: the wake_q to wake tasks after we release the wait_lock
*
* Starts the rt_mutex acquire; it enqueues the @waiter and does deadlock
* detection. It does not wait, see rt_mutex_wait_proxy_lock() for that.
@@ -291,7 +292,8 @@ void __sched rt_mutex_proxy_unlock(struct rt_mutex_base *lock)
*/
int __sched __rt_mutex_start_proxy_lock(struct rt_mutex_base *lock,
struct rt_mutex_waiter *waiter,
- struct task_struct *task)
+ struct task_struct *task,
+ struct wake_q_head *wake_q)
{
int ret;
@@ -302,7 +304,7 @@ int __sched __rt_mutex_start_proxy_lock(struct rt_mutex_base *lock,
/* We enforce deadlock detection for futexes */
ret = task_blocks_on_rt_mutex(lock, waiter, task, NULL,
- RT_MUTEX_FULL_CHAINWALK);
+ RT_MUTEX_FULL_CHAINWALK, wake_q);
if (ret && !rt_mutex_owner(lock)) {
/*
@@ -341,12 +343,16 @@ int __sched rt_mutex_start_proxy_lock(struct rt_mutex_base *lock,
struct task_struct *task)
{
int ret;
+ DEFINE_WAKE_Q(wake_q);
raw_spin_lock_irq(&lock->wait_lock);
- ret = __rt_mutex_start_proxy_lock(lock, waiter, task);
- if (unlikely(ret))
+ ret = __rt_mutex_start_proxy_lock(lock, waiter, task, &wake_q);
+ if (unlikely(ret < 0))
remove_waiter(lock, waiter);
+ preempt_disable();
raw_spin_unlock_irq(&lock->wait_lock);
+ wake_up_q(&wake_q);
+ preempt_enable();
return ret;
}
diff --git a/kernel/locking/rtmutex_common.h b/kernel/locking/rtmutex_common.h
index 1162e07cdaea..c38a2d2d4a7e 100644
--- a/kernel/locking/rtmutex_common.h
+++ b/kernel/locking/rtmutex_common.h
@@ -83,7 +83,8 @@ extern void rt_mutex_init_proxy_locked(struct rt_mutex_base *lock,
extern void rt_mutex_proxy_unlock(struct rt_mutex_base *lock);
extern int __rt_mutex_start_proxy_lock(struct rt_mutex_base *lock,
struct rt_mutex_waiter *waiter,
- struct task_struct *task);
+ struct task_struct *task,
+ struct wake_q_head *);
extern int rt_mutex_start_proxy_lock(struct rt_mutex_base *lock,
struct rt_mutex_waiter *waiter,
struct task_struct *task);
diff --git a/kernel/locking/rwbase_rt.c b/kernel/locking/rwbase_rt.c
index 34a59569db6b..9f4322c07486 100644
--- a/kernel/locking/rwbase_rt.c
+++ b/kernel/locking/rwbase_rt.c
@@ -69,6 +69,7 @@ static int __sched __rwbase_read_lock(struct rwbase_rt *rwb,
unsigned int state)
{
struct rt_mutex_base *rtm = &rwb->rtmutex;
+ DEFINE_WAKE_Q(wake_q);
int ret;
rwbase_pre_schedule();
@@ -110,7 +111,7 @@ static int __sched __rwbase_read_lock(struct rwbase_rt *rwb,
* For rwlocks this returns 0 unconditionally, so the below
* !ret conditionals are optimized out.
*/
- ret = rwbase_rtmutex_slowlock_locked(rtm, state);
+ ret = rwbase_rtmutex_slowlock_locked(rtm, state, &wake_q);
/*
* On success the rtmutex is held, so there can't be a writer
@@ -121,7 +122,12 @@ static int __sched __rwbase_read_lock(struct rwbase_rt *rwb,
*/
if (!ret)
atomic_inc(&rwb->readers);
+
+ preempt_disable();
raw_spin_unlock_irq(&rtm->wait_lock);
+ wake_up_q(&wake_q);
+ preempt_enable();
+
if (!ret)
rwbase_rtmutex_unlock(rtm);
diff --git a/kernel/locking/rwsem.c b/kernel/locking/rwsem.c
index 2bbb6eca5144..2ddb827e3bea 100644
--- a/kernel/locking/rwsem.c
+++ b/kernel/locking/rwsem.c
@@ -1413,8 +1413,8 @@ static inline void __downgrade_write(struct rw_semaphore *sem)
#define rwbase_rtmutex_lock_state(rtm, state) \
__rt_mutex_lock(rtm, state)
-#define rwbase_rtmutex_slowlock_locked(rtm, state) \
- __rt_mutex_slowlock_locked(rtm, NULL, state)
+#define rwbase_rtmutex_slowlock_locked(rtm, state, wq) \
+ __rt_mutex_slowlock_locked(rtm, NULL, state, wq)
#define rwbase_rtmutex_unlock(rtm) \
__rt_mutex_unlock(rtm)
diff --git a/kernel/locking/spinlock_rt.c b/kernel/locking/spinlock_rt.c
index 38e292454fcc..014143934e00 100644
--- a/kernel/locking/spinlock_rt.c
+++ b/kernel/locking/spinlock_rt.c
@@ -162,9 +162,10 @@ rwbase_rtmutex_lock_state(struct rt_mutex_base *rtm, unsigned int state)
}
static __always_inline int
-rwbase_rtmutex_slowlock_locked(struct rt_mutex_base *rtm, unsigned int state)
+rwbase_rtmutex_slowlock_locked(struct rt_mutex_base *rtm, unsigned int state,
+ struct wake_q_head *wake_q)
{
- rtlock_slowlock_locked(rtm);
+ rtlock_slowlock_locked(rtm, wake_q);
return 0;
}
diff --git a/kernel/locking/ww_mutex.h b/kernel/locking/ww_mutex.h
index 76d204b7d29c..a54bd16d0f17 100644
--- a/kernel/locking/ww_mutex.h
+++ b/kernel/locking/ww_mutex.h
@@ -275,7 +275,7 @@ __ww_ctx_less(struct ww_acquire_ctx *a, struct ww_acquire_ctx *b)
*/
static bool
__ww_mutex_die(struct MUTEX *lock, struct MUTEX_WAITER *waiter,
- struct ww_acquire_ctx *ww_ctx)
+ struct ww_acquire_ctx *ww_ctx, struct wake_q_head *wake_q)
{
if (!ww_ctx->is_wait_die)
return false;
@@ -284,7 +284,7 @@ __ww_mutex_die(struct MUTEX *lock, struct MUTEX_WAITER *waiter,
#ifndef WW_RT
debug_mutex_wake_waiter(lock, waiter);
#endif
- wake_up_process(waiter->task);
+ wake_q_add(wake_q, waiter->task);
}
return true;
@@ -299,7 +299,8 @@ __ww_mutex_die(struct MUTEX *lock, struct MUTEX_WAITER *waiter,
*/
static bool __ww_mutex_wound(struct MUTEX *lock,
struct ww_acquire_ctx *ww_ctx,
- struct ww_acquire_ctx *hold_ctx)
+ struct ww_acquire_ctx *hold_ctx,
+ struct wake_q_head *wake_q)
{
struct task_struct *owner = __ww_mutex_owner(lock);
@@ -331,7 +332,7 @@ static bool __ww_mutex_wound(struct MUTEX *lock,
* wakeup pending to re-read the wounded state.
*/
if (owner != current)
- wake_up_process(owner);
+ wake_q_add(wake_q, owner);
return true;
}
@@ -352,7 +353,8 @@ static bool __ww_mutex_wound(struct MUTEX *lock,
* The current task must not be on the wait list.
*/
static void
-__ww_mutex_check_waiters(struct MUTEX *lock, struct ww_acquire_ctx *ww_ctx)
+__ww_mutex_check_waiters(struct MUTEX *lock, struct ww_acquire_ctx *ww_ctx,
+ struct wake_q_head *wake_q)
{
struct MUTEX_WAITER *cur;
@@ -364,8 +366,8 @@ __ww_mutex_check_waiters(struct MUTEX *lock, struct ww_acquire_ctx *ww_ctx)
if (!cur->ww_ctx)
continue;
- if (__ww_mutex_die(lock, cur, ww_ctx) ||
- __ww_mutex_wound(lock, cur->ww_ctx, ww_ctx))
+ if (__ww_mutex_die(lock, cur, ww_ctx, wake_q) ||
+ __ww_mutex_wound(lock, cur->ww_ctx, ww_ctx, wake_q))
break;
}
}
@@ -377,6 +379,8 @@ __ww_mutex_check_waiters(struct MUTEX *lock, struct ww_acquire_ctx *ww_ctx)
static __always_inline void
ww_mutex_set_context_fastpath(struct ww_mutex *lock, struct ww_acquire_ctx *ctx)
{
+ DEFINE_WAKE_Q(wake_q);
+
ww_mutex_lock_acquired(lock, ctx);
/*
@@ -405,8 +409,11 @@ ww_mutex_set_context_fastpath(struct ww_mutex *lock, struct ww_acquire_ctx *ctx)
* die or wound us.
*/
lock_wait_lock(&lock->base);
- __ww_mutex_check_waiters(&lock->base, ctx);
+ __ww_mutex_check_waiters(&lock->base, ctx, &wake_q);
+ preempt_disable();
unlock_wait_lock(&lock->base);
+ wake_up_q(&wake_q);
+ preempt_enable();
}
static __always_inline int
@@ -488,7 +495,8 @@ __ww_mutex_check_kill(struct MUTEX *lock, struct MUTEX_WAITER *waiter,
static inline int
__ww_mutex_add_waiter(struct MUTEX_WAITER *waiter,
struct MUTEX *lock,
- struct ww_acquire_ctx *ww_ctx)
+ struct ww_acquire_ctx *ww_ctx,
+ struct wake_q_head *wake_q)
{
struct MUTEX_WAITER *cur, *pos = NULL;
bool is_wait_die;
@@ -532,7 +540,7 @@ __ww_mutex_add_waiter(struct MUTEX_WAITER *waiter,
pos = cur;
/* Wait-Die: ensure younger waiters die. */
- __ww_mutex_die(lock, cur, ww_ctx);
+ __ww_mutex_die(lock, cur, ww_ctx, wake_q);
}
__ww_waiter_add(lock, waiter, pos);
@@ -550,7 +558,7 @@ __ww_mutex_add_waiter(struct MUTEX_WAITER *waiter,
* such that either we or the fastpath will wound @ww->ctx.
*/
smp_mb();
- __ww_mutex_wound(lock, ww_ctx, ww->ctx);
+ __ww_mutex_wound(lock, ww_ctx, ww->ctx, wake_q);
}
return 0;
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 1b1ddd24cb22..49dbb52842c9 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5226,6 +5226,12 @@ static struct rq *finish_task_switch(struct task_struct *prev)
*/
kmap_local_sched_in();
+ /*
+ * Any cached block-layer timestamp (plug->cur_ktime) is stale now,
+ * invalidate it.
+ */
+ blk_plug_invalidate_ts();
+
fire_sched_in_preempt_notifiers(current);
/*
* When switching through a kernel thread, the loop in
@@ -6785,12 +6791,10 @@ static inline void sched_submit_work(struct task_struct *tsk)
static void sched_update_worker(struct task_struct *tsk)
{
- if (tsk->flags & (PF_WQ_WORKER | PF_IO_WORKER | PF_BLOCK_TS)) {
- if (tsk->flags & PF_BLOCK_TS)
- blk_plug_invalidate_ts(tsk);
+ if (tsk->flags & (PF_WQ_WORKER | PF_IO_WORKER)) {
if (tsk->flags & PF_WQ_WORKER)
wq_worker_running(tsk);
- else if (tsk->flags & PF_IO_WORKER)
+ else
io_wq_worker_running(tsk);
}
}
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index 4a44451efbcc..41c874fbd6fa 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -2943,6 +2943,10 @@ int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
if (!is_kprobe_multi(prog))
return -EINVAL;
+ /* kprobe_multi is not allowed to be sleepable. */
+ if (prog->sleepable)
+ return -EINVAL;
+
flags = attr->link_create.kprobe_multi.flags;
if (flags & ~BPF_F_KPROBE_MULTI_RETURN)
return -EINVAL;
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index c8a6c0bb907e..4a486f7457e2 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -1122,7 +1122,6 @@ struct ftrace_page {
};
#define ENTRY_SIZE sizeof(struct dyn_ftrace)
-#define ENTRIES_PER_PAGE (PAGE_SIZE / ENTRY_SIZE)
static struct ftrace_page *ftrace_pages_start;
static struct ftrace_page *ftrace_pages;
@@ -3754,7 +3753,8 @@ static int ftrace_update_code(struct module *mod, struct ftrace_page *new_pgs)
return 0;
}
-static int ftrace_allocate_records(struct ftrace_page *pg, int count)
+static int ftrace_allocate_records(struct ftrace_page *pg, int count,
+ unsigned long *num_pages)
{
int order;
int pages;
@@ -3764,7 +3764,7 @@ static int ftrace_allocate_records(struct ftrace_page *pg, int count)
return -EINVAL;
/* We want to fill as much as possible, with no empty pages */
- pages = DIV_ROUND_UP(count, ENTRIES_PER_PAGE);
+ pages = DIV_ROUND_UP(count * ENTRY_SIZE, PAGE_SIZE);
order = fls(pages) - 1;
again:
@@ -3779,6 +3779,7 @@ static int ftrace_allocate_records(struct ftrace_page *pg, int count)
}
ftrace_number_of_pages += 1 << order;
+ *num_pages += 1 << order;
ftrace_number_of_groups++;
cnt = (PAGE_SIZE << order) / ENTRY_SIZE;
@@ -3807,12 +3808,14 @@ static void ftrace_free_pages(struct ftrace_page *pages)
}
static struct ftrace_page *
-ftrace_allocate_pages(unsigned long num_to_init)
+ftrace_allocate_pages(unsigned long num_to_init, unsigned long *num_pages)
{
struct ftrace_page *start_pg;
struct ftrace_page *pg;
int cnt;
+ *num_pages = 0;
+
if (!num_to_init)
return NULL;
@@ -3826,7 +3829,7 @@ ftrace_allocate_pages(unsigned long num_to_init)
* waste as little space as possible.
*/
for (;;) {
- cnt = ftrace_allocate_records(pg, num_to_init);
+ cnt = ftrace_allocate_records(pg, num_to_init, num_pages);
if (cnt < 0)
goto free_pages;
@@ -7049,6 +7052,7 @@ static int ftrace_process_locs(struct module *mod,
unsigned long *p;
unsigned long addr;
unsigned long flags = 0; /* Shut up gcc */
+ unsigned long pages;
int ret = -ENOMEM;
count = end - start;
@@ -7068,7 +7072,7 @@ static int ftrace_process_locs(struct module *mod,
test_is_sorted(start, count);
}
- start_pg = ftrace_allocate_pages(count);
+ start_pg = ftrace_allocate_pages(count, &pages);
if (!start_pg)
return -ENOMEM;
@@ -7100,7 +7104,9 @@ static int ftrace_process_locs(struct module *mod,
pg = start_pg;
while (p < end) {
unsigned long end_offset;
- addr = ftrace_call_adjust(*p++);
+
+ addr = *p++;
+
/*
* Some architecture linkers will pad between
* the different mcount_loc sections of different
@@ -7112,6 +7118,19 @@ static int ftrace_process_locs(struct module *mod,
continue;
}
+ /*
+ * If this is core kernel, make sure the address is in core
+ * or inittext, as weak functions get zeroed and KASLR can
+ * move them to something other than zero. It just will not
+ * move it to an area where kernel text is.
+ */
+ if (!mod && !(is_kernel_text(addr) || is_kernel_inittext(addr))) {
+ skipped++;
+ continue;
+ }
+
+ addr = ftrace_call_adjust(addr);
+
end_offset = (pg->index+1) * sizeof(pg->records[0]);
if (end_offset > PAGE_SIZE << pg->order) {
/* We should have allocated enough */
@@ -7151,11 +7170,41 @@ static int ftrace_process_locs(struct module *mod,
/* We should have used all pages unless we skipped some */
if (pg_unuse) {
- WARN_ON(!skipped);
+ unsigned long pg_remaining, remaining = 0;
+ long skip;
+
+ /* Count the number of entries unused and compare it to skipped. */
+ pg_remaining = (PAGE_SIZE << pg->order) / ENTRY_SIZE - pg->index;
+
+ if (!WARN(skipped < pg_remaining, "Extra allocated pages for ftrace")) {
+
+ skip = skipped - pg_remaining;
+
+ for (pg = pg_unuse; pg && skip > 0; pg = pg->next) {
+ remaining += 1 << pg->order;
+ skip -= (PAGE_SIZE << pg->order) / ENTRY_SIZE;
+ }
+
+ pages -= remaining;
+
+ /*
+ * Check to see if the number of pages remaining would
+ * just fit the number of entries skipped.
+ */
+ WARN(pg || skip > 0, "Extra allocated pages for ftrace: %lu with %lu skipped",
+ remaining, skipped);
+ }
/* Need to synchronize with ftrace_location_range() */
synchronize_rcu();
ftrace_free_pages(pg_unuse);
}
+
+ if (!mod) {
+ count -= skipped;
+ pr_info("ftrace: allocating %ld entries in %ld pages\n",
+ count, pages);
+ }
+
return ret;
}
@@ -7810,9 +7859,6 @@ void __init ftrace_init(void)
goto failed;
}
- pr_info("ftrace: allocating %ld entries in %ld pages\n",
- count, DIV_ROUND_UP(count, ENTRIES_PER_PAGE));
-
ret = ftrace_process_locs(NULL,
__start_mcount_loc,
__stop_mcount_loc);
diff --git a/lib/debugobjects.c b/lib/debugobjects.c
index 932e2d8dbd9b..7abd909c8076 100644
--- a/lib/debugobjects.c
+++ b/lib/debugobjects.c
@@ -596,21 +596,59 @@ static struct debug_obj *lookup_object_or_alloc(void *addr, struct debug_bucket
return NULL;
}
-static void debug_objects_fill_pool(void)
+static inline bool debug_objects_is_pi_blocked_on(void)
+{
+#ifdef CONFIG_RT_MUTEXES
+ return current->pi_blocked_on != NULL;
+#else
+ return false;
+#endif
+}
+
+static inline bool can_fill_pool(void)
{
/*
- * On RT enabled kernels the pool refill must happen in preemptible
- * context -- for !RT kernels we rely on the fact that spinlock_t and
- * raw_spinlock_t are basically the same type and this lock-type
- * inversion works just fine.
+ * On !RT enabled kernels there are no restrictions and spinlock_t and
+ * raw_spinlock_t are the same types.
+ */
+ if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+ return true;
+
+ /*
+ * On RT enabled kernels, the task must not be blocked on a lock as
+ * that could corrupt the PI state when blocking on a lock in the
+ * allocation path.
*/
- if (!IS_ENABLED(CONFIG_PREEMPT_RT) || preemptible()) {
+ if (debug_objects_is_pi_blocked_on())
+ return false;
+
+ /*
+ * On RT enabled kernels the pool refill should happen in preemptible
+ * context.
+ */
+ if (preemptible())
+ return true;
+
+ /*
+ * Though during system boot before scheduling is set up, preemption is
+ * disabled and the pool can get exhausted. Before scheduling is active
+ * a task cannot be blocked on a sleeping lock, but it might hold a lock
+ * and if interrupted then hard interrupt context might run into a lock
+ * inversion. So exclude hard interrupt context from allocations before
+ * scheduling is active.
+ */
+ return system_state < SYSTEM_SCHEDULING && !in_hardirq();
+}
+
+static void debug_objects_fill_pool(void)
+{
+ if (can_fill_pool()) {
/*
* Annotate away the spinlock_t inside raw_spinlock_t warning
- * by temporarily raising the wait-type to WAIT_SLEEP, matching
- * the preemptible() condition above.
+ * by temporarily raising the wait-type to LD_WAIT_CONFIG, matching
+ * the preemptible() condition in can_fill_pool().
*/
- static DEFINE_WAIT_OVERRIDE_MAP(fill_pool_map, LD_WAIT_SLEEP);
+ static DEFINE_WAIT_OVERRIDE_MAP(fill_pool_map, LD_WAIT_CONFIG);
lock_map_acquire_try(&fill_pool_map);
fill_pool();
lock_map_release(&fill_pool_map);
diff --git a/net/9p/client.c b/net/9p/client.c
index 52a5497cfca7..2f6baeb47500 100644
--- a/net/9p/client.c
+++ b/net/9p/client.c
@@ -1215,7 +1215,8 @@ struct p9_fid *p9_client_walk(struct p9_fid *oldfid, uint16_t nwname,
clunk_fid:
kfree(wqids);
- p9_fid_put(fid);
+ if (fid != oldfid)
+ p9_fid_put(fid);
fid = NULL;
error:
diff --git a/net/batman-adv/bat_iv_ogm.c b/net/batman-adv/bat_iv_ogm.c
index b37c9fb178ae..5dd3e1f281ba 100644
--- a/net/batman-adv/bat_iv_ogm.c
+++ b/net/batman-adv/bat_iv_ogm.c
@@ -310,14 +310,23 @@ batadv_iv_ogm_aggr_packet(int buff_pos, int packet_len,
const struct batadv_ogm_packet *ogm_packet)
{
int next_buff_pos = 0;
+ u16 tvlv_len;
/* check if there is enough space for the header */
next_buff_pos += buff_pos + sizeof(*ogm_packet);
if (next_buff_pos > packet_len)
return false;
+ tvlv_len = ntohs(ogm_packet->tvlv_len);
+
+ /* the fields of an aggregated OGM are accessed assuming (at least)
+ * 2-byte alignment, so a following OGM must start at an even offset.
+ */
+ if (tvlv_len & 1)
+ return false;
+
/* check if there is enough space for the optional TVLV */
- next_buff_pos += ntohs(ogm_packet->tvlv_len);
+ next_buff_pos += tvlv_len;
return next_buff_pos <= packet_len;
}
diff --git a/net/batman-adv/bat_v.c b/net/batman-adv/bat_v.c
index d35479c465e2..e13b39121f2d 100644
--- a/net/batman-adv/bat_v.c
+++ b/net/batman-adv/bat_v.c
@@ -819,6 +819,7 @@ void batadv_v_hardif_init(struct batadv_hard_iface *hard_iface)
hard_iface->bat_v.aggr_len = 0;
skb_queue_head_init(&hard_iface->bat_v.aggr_list);
+ hard_iface->bat_v.aggr_list_enabled = false;
INIT_DELAYED_WORK(&hard_iface->bat_v.aggr_wq,
batadv_v_ogm_aggr_work);
}
diff --git a/net/batman-adv/bat_v_ogm.c b/net/batman-adv/bat_v_ogm.c
index 8cfc3944dcfd..c5c4d33cb198 100644
--- a/net/batman-adv/bat_v_ogm.c
+++ b/net/batman-adv/bat_v_ogm.c
@@ -254,11 +254,18 @@ static void batadv_v_ogm_queue_on_if(struct batadv_priv *bat_priv,
}
spin_lock_bh(&hard_iface->bat_v.aggr_list.lock);
+ if (!hard_iface->bat_v.aggr_list_enabled) {
+ kfree_skb(skb);
+ goto unlock;
+ }
+
if (!batadv_v_ogm_queue_left(skb, hard_iface))
batadv_v_ogm_aggr_send(bat_priv, hard_iface);
hard_iface->bat_v.aggr_len += batadv_v_ogm_len(skb);
__skb_queue_tail(&hard_iface->bat_v.aggr_list, skb);
+
+unlock:
spin_unlock_bh(&hard_iface->bat_v.aggr_list.lock);
}
@@ -421,6 +428,10 @@ int batadv_v_ogm_iface_enable(struct batadv_hard_iface *hard_iface)
{
struct batadv_priv *bat_priv = netdev_priv(hard_iface->soft_iface);
+ spin_lock_bh(&hard_iface->bat_v.aggr_list.lock);
+ hard_iface->bat_v.aggr_list_enabled = true;
+ spin_unlock_bh(&hard_iface->bat_v.aggr_list.lock);
+
batadv_v_ogm_start_queue_timer(hard_iface);
batadv_v_ogm_start_timer(bat_priv);
@@ -436,6 +447,7 @@ void batadv_v_ogm_iface_disable(struct batadv_hard_iface *hard_iface)
cancel_delayed_work_sync(&hard_iface->bat_v.aggr_wq);
spin_lock_bh(&hard_iface->bat_v.aggr_list.lock);
+ hard_iface->bat_v.aggr_list_enabled = false;
batadv_v_ogm_aggr_list_free(hard_iface);
spin_unlock_bh(&hard_iface->bat_v.aggr_list.lock);
}
@@ -841,14 +853,23 @@ batadv_v_ogm_aggr_packet(int buff_pos, int packet_len,
const struct batadv_ogm2_packet *ogm2_packet)
{
int next_buff_pos = 0;
+ u16 tvlv_len;
/* check if there is enough space for the header */
next_buff_pos += buff_pos + sizeof(*ogm2_packet);
if (next_buff_pos > packet_len)
return false;
+ tvlv_len = ntohs(ogm2_packet->tvlv_len);
+
+ /* the fields of an aggregated OGMv2 are accessed assuming (at least)
+ * 2-byte alignment, so a following OGMv2 must start at an even offset.
+ */
+ if (tvlv_len & 1)
+ return false;
+
/* check if there is enough space for the optional TVLV */
- next_buff_pos += ntohs(ogm2_packet->tvlv_len);
+ next_buff_pos += tvlv_len;
return next_buff_pos <= packet_len;
}
diff --git a/net/batman-adv/bridge_loop_avoidance.c b/net/batman-adv/bridge_loop_avoidance.c
index 15aeb07285e6..cc6c14cac062 100644
--- a/net/batman-adv/bridge_loop_avoidance.c
+++ b/net/batman-adv/bridge_loop_avoidance.c
@@ -512,7 +512,7 @@ batadv_bla_get_backbone_gw(struct batadv_priv *bat_priv, const u8 *orig,
return NULL;
entry->vid = vid;
- entry->lasttime = jiffies;
+ WRITE_ONCE(entry->lasttime, jiffies);
entry->crc = BATADV_BLA_CRC_INIT;
entry->bat_priv = bat_priv;
spin_lock_init(&entry->crc_lock);
@@ -580,7 +580,7 @@ batadv_bla_update_own_backbone_gw(struct batadv_priv *bat_priv,
if (unlikely(!backbone_gw))
return;
- backbone_gw->lasttime = jiffies;
+ WRITE_ONCE(backbone_gw->lasttime, jiffies);
batadv_backbone_gw_put(backbone_gw);
}
@@ -714,7 +714,7 @@ static void batadv_bla_add_claim(struct batadv_priv *bat_priv,
ether_addr_copy(claim->addr, mac);
spin_lock_init(&claim->backbone_lock);
claim->vid = vid;
- claim->lasttime = jiffies;
+ WRITE_ONCE(claim->lasttime, jiffies);
kref_get(&backbone_gw->refcount);
claim->backbone_gw = backbone_gw;
kref_init(&claim->refcount);
@@ -736,7 +736,7 @@ static void batadv_bla_add_claim(struct batadv_priv *bat_priv,
return;
}
} else {
- claim->lasttime = jiffies;
+ WRITE_ONCE(claim->lasttime, jiffies);
if (claim->backbone_gw == backbone_gw)
/* no need to register a new backbone */
goto claim_free_ref;
@@ -769,7 +769,7 @@ static void batadv_bla_add_claim(struct batadv_priv *bat_priv,
spin_lock_bh(&backbone_gw->crc_lock);
backbone_gw->crc ^= crc16(0, claim->addr, ETH_ALEN);
spin_unlock_bh(&backbone_gw->crc_lock);
- backbone_gw->lasttime = jiffies;
+ WRITE_ONCE(backbone_gw->lasttime, jiffies);
claim_free_ref:
batadv_claim_put(claim);
@@ -858,7 +858,7 @@ static bool batadv_handle_announce(struct batadv_priv *bat_priv, u8 *an_addr,
return true;
/* handle as ANNOUNCE frame */
- backbone_gw->lasttime = jiffies;
+ WRITE_ONCE(backbone_gw->lasttime, jiffies);
crc = ntohs(*((__force __be16 *)(&an_addr[4])));
batadv_dbg(BATADV_DBG_BLA, bat_priv,
@@ -1253,7 +1253,7 @@ static void batadv_bla_purge_backbone_gw(struct batadv_priv *bat_priv, int now)
head, hash_entry) {
if (now)
goto purge_now;
- if (!batadv_has_timed_out(backbone_gw->lasttime,
+ if (!batadv_has_timed_out(READ_ONCE(backbone_gw->lasttime),
BATADV_BLA_BACKBONE_TIMEOUT))
continue;
@@ -1334,7 +1334,7 @@ static void batadv_bla_purge_claims(struct batadv_priv *bat_priv,
primary_if->net_dev->dev_addr))
goto skip;
- if (!batadv_has_timed_out(claim->lasttime,
+ if (!batadv_has_timed_out(READ_ONCE(claim->lasttime),
BATADV_BLA_CLAIM_TIMEOUT))
goto skip;
@@ -1494,7 +1494,7 @@ static void batadv_bla_periodic_work(struct work_struct *work)
eth_random_addr(bat_priv->bla.loopdetect_addr);
bat_priv->bla.loopdetect_addr[0] = 0xba;
bat_priv->bla.loopdetect_addr[1] = 0xbe;
- bat_priv->bla.loopdetect_lasttime = jiffies;
+ WRITE_ONCE(bat_priv->bla.loopdetect_lasttime, jiffies);
atomic_set(&bat_priv->bla.loopdetect_next,
BATADV_BLA_LOOPDETECT_PERIODS);
@@ -1515,7 +1515,7 @@ static void batadv_bla_periodic_work(struct work_struct *work)
primary_if->net_dev->dev_addr))
continue;
- backbone_gw->lasttime = jiffies;
+ WRITE_ONCE(backbone_gw->lasttime, jiffies);
batadv_bla_send_announce(bat_priv, backbone_gw);
if (send_loopdetect)
@@ -1900,7 +1900,7 @@ batadv_bla_loopdetect_check(struct batadv_priv *bat_priv, struct sk_buff *skb,
/* If the packet came too late, don't forward it on the mesh
* but don't consider that as loop. It might be a coincidence.
*/
- if (batadv_has_timed_out(bat_priv->bla.loopdetect_lasttime,
+ if (batadv_has_timed_out(READ_ONCE(bat_priv->bla.loopdetect_lasttime),
BATADV_BLA_LOOPDETECT_TIMEOUT))
return true;
@@ -2016,7 +2016,7 @@ bool batadv_bla_rx(struct batadv_priv *bat_priv, struct sk_buff *skb,
if (own_claim) {
/* ... allow it in any case */
- claim->lasttime = jiffies;
+ WRITE_ONCE(claim->lasttime, jiffies);
goto allow;
}
@@ -2118,7 +2118,7 @@ bool batadv_bla_tx(struct batadv_priv *bat_priv, struct sk_buff *skb,
/* if yes, the client has roamed and we have
* to unclaim it.
*/
- if (batadv_has_timed_out(claim->lasttime, 100)) {
+ if (batadv_has_timed_out(READ_ONCE(claim->lasttime), 100)) {
/* only unclaim if the last claim entry is
* older than 100 ms to make sure we really
* have a roaming client here.
@@ -2372,7 +2372,7 @@ batadv_bla_backbone_dump_entry(struct sk_buff *msg, u32 portid,
backbone_crc = backbone_gw->crc;
spin_unlock_bh(&backbone_gw->crc_lock);
- msecs = jiffies_to_msecs(jiffies - backbone_gw->lasttime);
+ msecs = jiffies_to_msecs(jiffies - READ_ONCE(backbone_gw->lasttime));
if (is_own)
if (nla_put_flag(msg, BATADV_ATTR_BLA_OWN)) {
diff --git a/net/batman-adv/distributed-arp-table.c b/net/batman-adv/distributed-arp-table.c
index 4d53e7c3ea54..bd5477263ecd 100644
--- a/net/batman-adv/distributed-arp-table.c
+++ b/net/batman-adv/distributed-arp-table.c
@@ -215,10 +215,13 @@ static void batadv_dat_purge(struct work_struct *work)
*/
static bool batadv_compare_dat(const struct hlist_node *node, const void *data2)
{
- const void *data1 = container_of(node, struct batadv_dat_entry,
- hash_entry);
+ const struct batadv_dat_entry *entry1;
+ const struct batadv_dat_entry *entry2;
- return memcmp(data1, data2, sizeof(__be32)) == 0;
+ entry1 = container_of(node, struct batadv_dat_entry, hash_entry);
+ entry2 = data2;
+
+ return entry1->ip == entry2->ip && entry1->vid == entry2->vid;
}
/**
@@ -345,6 +348,9 @@ batadv_dat_entry_hash_find(struct batadv_priv *bat_priv, __be32 ip,
if (dat_entry->ip != ip)
continue;
+ if (dat_entry->vid != vid)
+ continue;
+
if (!kref_get_unless_zero(&dat_entry->refcount))
continue;
diff --git a/net/batman-adv/fragmentation.c b/net/batman-adv/fragmentation.c
index fd7cb789ae9a..375c94991213 100644
--- a/net/batman-adv/fragmentation.c
+++ b/net/batman-adv/fragmentation.c
@@ -384,6 +384,8 @@ bool batadv_frag_skb_buffer(struct sk_buff **skb,
* @skb: skb to forward
* @recv_if: interface that the skb is received on
* @orig_node_src: originator that the skb is received from
+ * @rx_result: set to NET_RX_SUCCESS when the fragment was forwarded and
+ * NET_RX_DROP when it was dropped; only valid when true is returned
*
* Look up the next-hop of the fragments payload and check if the merged packet
* will exceed the MTU towards the next-hop. If so, the fragment is forwarded
@@ -393,7 +395,8 @@ bool batadv_frag_skb_buffer(struct sk_buff **skb,
*/
bool batadv_frag_skb_fwd(struct sk_buff *skb,
struct batadv_hard_iface *recv_if,
- struct batadv_orig_node *orig_node_src)
+ struct batadv_orig_node *orig_node_src,
+ int *rx_result)
{
struct batadv_priv *bat_priv = netdev_priv(recv_if->soft_iface);
struct batadv_neigh_node *neigh_node = NULL;
@@ -412,12 +415,29 @@ bool batadv_frag_skb_fwd(struct sk_buff *skb,
*/
total_size = ntohs(packet->total_size);
if (total_size > neigh_node->if_incoming->net_dev->mtu) {
+ if (packet->ttl < 2) {
+ kfree_skb(skb);
+ *rx_result = NET_RX_DROP;
+ ret = true;
+ goto out;
+ }
+
+ if (skb_cow(skb, ETH_HLEN) < 0) {
+ kfree_skb(skb);
+ *rx_result = NET_RX_DROP;
+ ret = true;
+ goto out;
+ }
+
+ packet = (struct batadv_frag_packet *)skb->data;
+
batadv_inc_counter(bat_priv, BATADV_CNT_FRAG_FWD);
batadv_add_counter(bat_priv, BATADV_CNT_FRAG_FWD_BYTES,
skb->len + ETH_HLEN);
packet->ttl--;
batadv_send_unicast_skb(skb, neigh_node);
+ *rx_result = NET_RX_SUCCESS;
ret = true;
}
diff --git a/net/batman-adv/fragmentation.h b/net/batman-adv/fragmentation.h
index dbf0871f8703..51e281027ab6 100644
--- a/net/batman-adv/fragmentation.h
+++ b/net/batman-adv/fragmentation.h
@@ -19,7 +19,8 @@ void batadv_frag_purge_orig(struct batadv_orig_node *orig,
bool (*check_cb)(struct batadv_frag_table_entry *));
bool batadv_frag_skb_fwd(struct sk_buff *skb,
struct batadv_hard_iface *recv_if,
- struct batadv_orig_node *orig_node_src);
+ struct batadv_orig_node *orig_node_src,
+ int *rx_result);
bool batadv_frag_skb_buffer(struct sk_buff **skb,
struct batadv_orig_node *orig_node);
int batadv_frag_send_packet(struct sk_buff *skb,
diff --git a/net/batman-adv/netlink.c b/net/batman-adv/netlink.c
index 9362cd9d6f3d..a7522015224a 100644
--- a/net/batman-adv/netlink.c
+++ b/net/batman-adv/netlink.c
@@ -936,9 +936,15 @@ static int batadv_netlink_set_hardif(struct sk_buff *skb,
#ifdef CONFIG_BATMAN_ADV_BATMAN_V
if (info->attrs[BATADV_ATTR_ELP_INTERVAL]) {
+ u32 elp_interval;
+
attr = info->attrs[BATADV_ATTR_ELP_INTERVAL];
+ elp_interval = nla_get_u32(attr);
+
+ elp_interval = min_t(u32, elp_interval, INT_MAX);
+ elp_interval = max_t(u32, elp_interval, BATADV_JITTER);
- atomic_set(&hard_iface->bat_v.elp_interval, nla_get_u32(attr));
+ atomic_set(&hard_iface->bat_v.elp_interval, elp_interval);
}
if (info->attrs[BATADV_ATTR_THROUGHPUT_OVERRIDE]) {
diff --git a/net/batman-adv/routing.c b/net/batman-adv/routing.c
index f1061985149f..f828d8143f75 100644
--- a/net/batman-adv/routing.c
+++ b/net/batman-adv/routing.c
@@ -8,6 +8,7 @@
#include "main.h"
#include <linux/atomic.h>
+#include <linux/build_bug.h>
#include <linux/byteorder/generic.h>
#include <linux/compiler.h>
#include <linux/errno.h>
@@ -205,6 +206,59 @@ bool batadv_check_management_packet(struct sk_buff *skb,
return true;
}
+/**
+ * batadv_skb_decrement_ttl() - decrement ttl in a batman-adv header, csum-safe
+ * @skb: the received packet with @skb->data pointing to the batman-adv header
+ *
+ * Supports the following packet types, all of which carry the TTL at offset 2:
+ *
+ * - batadv_ogm_packet
+ * - batadv_ogm2_packet
+ * - batadv_icmp_header
+ * - batadv_icmp_packet
+ * - batadv_icmp_tp_packet
+ * - batadv_icmp_packet_rr
+ * - batadv_unicast_packet
+ * - batadv_frag_packet
+ * - batadv_bcast_packet
+ * - batadv_mcast_packet
+ * - batadv_coded_packet
+ * - batadv_unicast_tvlv_packet
+ *
+ * Return: true if the packet may be forwarded (ttl decremented),
+ * false if it must be dropped (ttl would expire)
+ */
+static bool batadv_skb_decrement_ttl(struct sk_buff *skb)
+{
+ static const size_t ttl_offset = 2;
+ u8 *ttl_pos;
+
+ BUILD_BUG_ON(offsetof(struct batadv_ogm_packet, ttl) != ttl_offset);
+ BUILD_BUG_ON(offsetof(struct batadv_ogm2_packet, ttl) != ttl_offset);
+ BUILD_BUG_ON(offsetof(struct batadv_icmp_header, ttl) != ttl_offset);
+ BUILD_BUG_ON(offsetof(struct batadv_icmp_packet, ttl) != ttl_offset);
+ BUILD_BUG_ON(offsetof(struct batadv_icmp_tp_packet, ttl) != ttl_offset);
+ BUILD_BUG_ON(offsetof(struct batadv_icmp_packet_rr, ttl) != ttl_offset);
+ BUILD_BUG_ON(offsetof(struct batadv_unicast_packet, ttl) != ttl_offset);
+ BUILD_BUG_ON(offsetof(struct batadv_frag_packet, ttl) != ttl_offset);
+ BUILD_BUG_ON(offsetof(struct batadv_bcast_packet, ttl) != ttl_offset);
+ BUILD_BUG_ON(offsetof(struct batadv_mcast_packet, ttl) != ttl_offset);
+ BUILD_BUG_ON(offsetof(struct batadv_coded_packet, ttl) != ttl_offset);
+ BUILD_BUG_ON(offsetof(struct batadv_unicast_tvlv_packet, ttl) != ttl_offset);
+
+ ttl_pos = skb->data + ttl_offset;
+
+ /* would expire on this hop -> drop, leave header + csum untouched */
+ if (*ttl_pos < 2)
+ return false;
+
+ skb_postpull_rcsum(skb, ttl_pos, 1);
+ (*ttl_pos)--;
+ skb_postpush_rcsum(skb, ttl_pos, 1);
+
+ return true;
+}
+
/**
* batadv_recv_my_icmp_packet() - receive an icmp packet locally
* @bat_priv: the bat priv with all the soft interface information
@@ -1121,10 +1175,9 @@ int batadv_recv_frag_packet(struct sk_buff *skb,
/* Route the fragment if it is not for us and too big to be merged. */
if (!batadv_is_my_mac(bat_priv, frag_packet->dest) &&
- batadv_frag_skb_fwd(skb, recv_if, orig_node_src)) {
+ batadv_frag_skb_fwd(skb, recv_if, orig_node_src, &ret)) {
/* skb was consumed */
skb = NULL;
- ret = NET_RX_SUCCESS;
goto put_orig_node;
}
@@ -1198,7 +1251,13 @@ int batadv_recv_bcast_packet(struct sk_buff *skb,
if (batadv_is_my_mac(bat_priv, bcast_packet->orig))
goto free_skb;
- if (bcast_packet->ttl-- < 2)
+ /* create a copy of the skb, if needed, to modify it. */
+ if (skb_cow(skb, ETH_HLEN) < 0)
+ goto free_skb;
+
+ bcast_packet = (struct batadv_bcast_packet *)skb->data;
+
+ if (!batadv_skb_decrement_ttl(skb))
goto free_skb;
orig_node = batadv_orig_hash_find(bat_priv, bcast_packet->orig);
@@ -1305,7 +1364,7 @@ int batadv_recv_mcast_packet(struct sk_buff *skb,
goto free_skb;
mcast_packet = (struct batadv_mcast_packet *)skb->data;
- if (mcast_packet->ttl-- < 2)
+ if (!batadv_skb_decrement_ttl(skb))
goto free_skb;
tvlv_buff = (unsigned char *)(skb->data + hdr_size);
@@ -1314,6 +1373,12 @@ int batadv_recv_mcast_packet(struct sk_buff *skb,
if (tvlv_buff_len > skb->len - hdr_size)
goto free_skb;
+ /* the fields of an multicast payload are accessed assuming (at least)
+ * 2-byte alignment, so a following packet must start at an even offset.
+ */
+ if (tvlv_buff_len & 1)
+ goto free_skb;
+
ret = batadv_tvlv_containers_process(bat_priv, BATADV_MCAST, NULL, skb,
tvlv_buff, tvlv_buff_len);
if (ret >= 0) {
diff --git a/net/batman-adv/tp_meter.c b/net/batman-adv/tp_meter.c
index dfc337454992..50b83e0438a0 100644
--- a/net/batman-adv/tp_meter.c
+++ b/net/batman-adv/tp_meter.c
@@ -87,6 +87,11 @@
#define BATADV_TP_PLEN (BATADV_TP_PACKET_LEN - ETH_HLEN - \
sizeof(struct batadv_unicast_packet))
+/**
+ * BATADV_TP_MAX_UNACKED - maximum number of packets a receiver didn't yet ack
+ */
+#define BATADV_TP_MAX_UNACKED 100
+
static u8 batadv_tp_prerandom[4096] __read_mostly;
/**
@@ -154,9 +159,12 @@ static void batadv_tp_update_cwnd(struct batadv_tp_vars *tp_vars, u32 mss)
return;
}
+ /* prevent overflow in (mss * mss) << 3 */
+ mss = min_t(u32, mss, (1U << 14) - 1);
+
/* increment CWND at least of 1 (section 3.1 of RFC5681) */
tp_vars->dec_cwnd += max_t(u32, 1U << 3,
- ((mss * mss) << 6) / (tp_vars->cwnd << 3));
+ ((mss * mss) << 3) / tp_vars->cwnd);
if (tp_vars->dec_cwnd < (mss << 3)) {
spin_unlock_bh(&tp_vars->cwnd_lock);
return;
@@ -730,7 +738,7 @@ static void batadv_tp_recv_ack(struct batadv_priv *bat_priv,
if (atomic_read(&tp_vars->dup_acks) != 3)
goto out;
- if (recv_ack >= tp_vars->recover)
+ if (!batadv_seq_before(tp_vars->recover, recv_ack))
goto out;
/* if this is the third duplicate ACK do Fast Retransmit */
@@ -817,10 +825,15 @@ static void batadv_tp_recv_ack(struct batadv_priv *bat_priv,
static bool batadv_tp_avail(struct batadv_tp_vars *tp_vars,
size_t payload_len)
{
+ u32 last_sent = READ_ONCE(tp_vars->last_sent);
u32 win_left, win_limit;
win_limit = atomic_read(&tp_vars->last_acked) + tp_vars->cwnd;
- win_left = win_limit - tp_vars->last_sent;
+
+ if (batadv_seq_before(last_sent, win_limit))
+ win_left = win_limit - last_sent;
+ else
+ win_left = 0;
return win_left >= payload_len;
}
@@ -1045,6 +1058,7 @@ void batadv_tp_start(struct batadv_priv *bat_priv, const u8 *dst,
tp_vars->icmp_uid = icmp_uid;
tp_vars->last_sent = BATADV_TP_FIRST_SEQ;
+ atomic_set(&tp_vars->dup_acks, 0);
atomic_set(&tp_vars->last_acked, BATADV_TP_FIRST_SEQ);
tp_vars->fast_recovery = false;
tp_vars->recover = BATADV_TP_FIRST_SEQ;
@@ -1054,6 +1068,8 @@ void batadv_tp_start(struct batadv_priv *bat_priv, const u8 *dst,
* soft_interface, hence its MTU
*/
tp_vars->cwnd = BATADV_TP_PLEN * 3;
+ tp_vars->dec_cwnd = 0;
+
/* at the beginning initialise the SS threshold to the biggest possible
* window size, hence the AWND size
*/
@@ -1085,21 +1101,21 @@ void batadv_tp_start(struct batadv_priv *bat_priv, const u8 *dst,
tp_vars->prerandom_offset = 0;
spin_lock_init(&tp_vars->prerandom_lock);
- kref_get(&tp_vars->refcount);
- hlist_add_head_rcu(&tp_vars->list, &bat_priv->tp_list);
- spin_unlock_bh(&bat_priv->tp_list_lock);
-
tp_vars->test_length = test_length;
if (!tp_vars->test_length)
tp_vars->test_length = BATADV_TP_DEF_TEST_LENGTH;
+ /* init work item for finished tp tests */
+ INIT_DELAYED_WORK(&tp_vars->finish_work, batadv_tp_sender_finish);
+
+ kref_get(&tp_vars->refcount);
+ hlist_add_head_rcu(&tp_vars->list, &bat_priv->tp_list);
+ spin_unlock_bh(&bat_priv->tp_list_lock);
+
batadv_dbg(BATADV_DBG_TP_METER, bat_priv,
"Meter: starting throughput meter towards %pM (length=%ums)\n",
dst, test_length);
- /* init work item for finished tp tests */
- INIT_DELAYED_WORK(&tp_vars->finish_work, batadv_tp_sender_finish);
-
/* start tp kthread. This way the write() call issued from userspace can
* happily return and avoid to block
*/
@@ -1167,7 +1183,7 @@ static void batadv_tp_receiver_shutdown(struct timer_list *t)
bat_priv = tp_vars->bat_priv;
/* if there is recent activity rearm the timer */
- if (!batadv_has_timed_out(tp_vars->last_recv_time,
+ if (!batadv_has_timed_out(READ_ONCE(tp_vars->last_recv_time),
BATADV_TP_RECV_TIMEOUT)) {
/* reset the receiver shutdown timer */
batadv_tp_reset_receiver_timer(tp_vars);
@@ -1184,6 +1200,7 @@ static void batadv_tp_receiver_shutdown(struct timer_list *t)
list_for_each_entry_safe(un, safe, &tp_vars->unacked_list, list) {
list_del(&un->list);
kfree(un);
+ tp_vars->unacked_count--;
}
spin_unlock_bh(&tp_vars->unacked_lock);
@@ -1267,7 +1284,8 @@ static int batadv_tp_send_ack(struct batadv_priv *bat_priv, const u8 *dst,
/**
* batadv_tp_handle_out_of_order() - store an out of order packet
* @tp_vars: the private data of the current TP meter session
- * @skb: the buffer containing the received packet
+ * @seqno: sequence number of new received packet
+ * @payload_len: length of the received packet
*
* Store the out of order packet in the unacked list for late processing. This
* packets are kept in this list so that they can be ACKed at once as soon as
@@ -1276,28 +1294,24 @@ static int batadv_tp_send_ack(struct batadv_priv *bat_priv, const u8 *dst,
* Return: true if the packed has been successfully processed, false otherwise
*/
static bool batadv_tp_handle_out_of_order(struct batadv_tp_vars *tp_vars,
- const struct sk_buff *skb)
+ u32 seqno, u32 payload_len)
+ __must_hold(&tp_vars->unacked_lock)
{
- const struct batadv_icmp_tp_packet *icmp;
struct batadv_tp_unacked *un, *new;
- u32 payload_len;
bool added = false;
new = kmalloc(sizeof(*new), GFP_ATOMIC);
if (unlikely(!new))
return false;
- icmp = (struct batadv_icmp_tp_packet *)skb->data;
-
- new->seqno = ntohl(icmp->seqno);
- payload_len = skb->len - sizeof(struct batadv_unicast_packet);
+ new->seqno = seqno;
new->len = payload_len;
- spin_lock_bh(&tp_vars->unacked_lock);
/* if the list is empty immediately attach this new object */
if (list_empty(&tp_vars->unacked_list)) {
list_add(&new->list, &tp_vars->unacked_list);
- goto out;
+ tp_vars->unacked_count++;
+ return true;
}
/* otherwise loop over the list and either drop the packet because this
@@ -1325,17 +1339,26 @@ static bool batadv_tp_handle_out_of_order(struct batadv_tp_vars *tp_vars,
* one is attached _after_ it. In this way the list is kept in
* ascending order
*/
- list_add_tail(&new->list, &un->list);
+ list_add(&new->list, &un->list);
added = true;
+ tp_vars->unacked_count++;
break;
}
/* received packet with smallest seqno out of order; add it to front */
- if (!added)
+ if (!added) {
list_add(&new->list, &tp_vars->unacked_list);
+ tp_vars->unacked_count++;
+ }
-out:
- spin_unlock_bh(&tp_vars->unacked_lock);
+ /* remove the last (biggest) unacked seqno when list is too large */
+ if (tp_vars->unacked_count > BATADV_TP_MAX_UNACKED) {
+ un = list_last_entry(&tp_vars->unacked_list,
+ struct batadv_tp_unacked, list);
+ list_del(&un->list);
+ kfree(un);
+ tp_vars->unacked_count--;
+ }
return true;
}
@@ -1346,6 +1369,7 @@ static bool batadv_tp_handle_out_of_order(struct batadv_tp_vars *tp_vars,
* @tp_vars: the private data of the current TP meter session
*/
static void batadv_tp_ack_unordered(struct batadv_tp_vars *tp_vars)
+ __must_hold(&tp_vars->unacked_lock)
{
struct batadv_tp_unacked *un, *safe;
u32 to_ack;
@@ -1353,7 +1377,6 @@ static void batadv_tp_ack_unordered(struct batadv_tp_vars *tp_vars)
/* go through the unacked packet list and possibly ACK them as
* well
*/
- spin_lock_bh(&tp_vars->unacked_lock);
list_for_each_entry_safe(un, safe, &tp_vars->unacked_list, list) {
/* the list is ordered, therefore it is possible to stop as soon
* there is a gap between the last acked seqno and the seqno of
@@ -1369,8 +1392,8 @@ static void batadv_tp_ack_unordered(struct batadv_tp_vars *tp_vars)
list_del(&un->list);
kfree(un);
+ tp_vars->unacked_count--;
}
- spin_unlock_bh(&tp_vars->unacked_lock);
}
/**
@@ -1392,8 +1415,10 @@ batadv_tp_init_recv(struct batadv_priv *bat_priv,
tp_vars = batadv_tp_list_find_session(bat_priv, icmp->orig,
icmp->session, BATADV_TP_RECEIVER);
- if (tp_vars)
+ if (tp_vars) {
+ WRITE_ONCE(tp_vars->last_recv_time, jiffies);
goto out_unlock;
+ }
if (!atomic_add_unless(&bat_priv->tp_num, 1, BATADV_TP_MAX_NUM)) {
batadv_dbg(BATADV_DBG_TP_METER, bat_priv,
@@ -1417,12 +1442,15 @@ batadv_tp_init_recv(struct batadv_priv *bat_priv,
spin_lock_init(&tp_vars->unacked_lock);
INIT_LIST_HEAD(&tp_vars->unacked_list);
+ tp_vars->unacked_count = 0;
kref_get(&tp_vars->refcount);
- hlist_add_head_rcu(&tp_vars->list, &bat_priv->tp_list);
+ timer_setup(&tp_vars->timer, batadv_tp_receiver_shutdown, 0);
+
+ WRITE_ONCE(tp_vars->last_recv_time, jiffies);
kref_get(&tp_vars->refcount);
- timer_setup(&tp_vars->timer, batadv_tp_receiver_shutdown, 0);
+ hlist_add_head_rcu(&tp_vars->list, &bat_priv->tp_list);
batadv_tp_reset_receiver_timer(tp_vars);
@@ -1444,7 +1472,8 @@ static void batadv_tp_recv_msg(struct batadv_priv *bat_priv,
{
const struct batadv_icmp_tp_packet *icmp;
struct batadv_tp_vars *tp_vars;
- size_t packet_size;
+ u32 payload_len;
+ u32 to_ack;
u32 seqno;
icmp = (struct batadv_icmp_tp_packet *)skb->data;
@@ -1469,41 +1498,49 @@ static void batadv_tp_recv_msg(struct batadv_priv *bat_priv,
icmp->orig);
goto out;
}
+
+ WRITE_ONCE(tp_vars->last_recv_time, jiffies);
}
- tp_vars->last_recv_time = jiffies;
+ spin_lock_bh(&tp_vars->unacked_lock);
/* if the packet is a duplicate, it may be the case that an ACK has been
* lost. Resend the ACK
*/
- if (batadv_seq_before(seqno, tp_vars->last_recv))
+ payload_len = skb->len - sizeof(struct batadv_unicast_packet);
+ to_ack = seqno + payload_len;
+ if (batadv_seq_before(to_ack, tp_vars->last_recv))
goto send_ack;
/* if the packet is out of order enqueue it */
- if (ntohl(icmp->seqno) != tp_vars->last_recv) {
+ if (batadv_seq_before(tp_vars->last_recv, seqno)) {
/* exit immediately (and do not send any ACK) if the packet has
* not been enqueued correctly
*/
- if (!batadv_tp_handle_out_of_order(tp_vars, skb))
+ if (!batadv_tp_handle_out_of_order(tp_vars, seqno, payload_len)) {
+ spin_unlock_bh(&tp_vars->unacked_lock);
goto out;
+ }
/* send a duplicate ACK */
goto send_ack;
}
/* if everything was fine count the ACKed bytes */
- packet_size = skb->len - sizeof(struct batadv_unicast_packet);
- tp_vars->last_recv += packet_size;
+ tp_vars->last_recv = to_ack;
/* check if this ordered message filled a gap.... */
batadv_tp_ack_unordered(tp_vars);
send_ack:
+ to_ack = tp_vars->last_recv;
+ spin_unlock_bh(&tp_vars->unacked_lock);
+
/* send the ACK. If the received packet was out of order, the ACK that
* is going to be sent is a duplicate (the sender will count them and
* possibly enter Fast Retransmit as soon as it has reached 3)
*/
- batadv_tp_send_ack(bat_priv, icmp->orig, tp_vars->last_recv,
+ batadv_tp_send_ack(bat_priv, icmp->orig, to_ack,
icmp->timestamp, icmp->session, icmp->uid);
out:
batadv_tp_vars_put(tp_vars);
diff --git a/net/batman-adv/translation-table.c b/net/batman-adv/translation-table.c
index 7041cd69e200..2a7caab5c197 100644
--- a/net/batman-adv/translation-table.c
+++ b/net/batman-adv/translation-table.c
@@ -485,6 +485,9 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv,
if (!batadv_compare_eth(entry->change.addr, common->addr))
continue;
+ if (entry->change.vid != tt_change_node->change.vid)
+ continue;
+
/* DEL+ADD in the same orig interval have no effect and can be
* removed to avoid silly behaviour on the receiver side. The
* other way around (ADD+DEL) can happen in case of roaming of
@@ -3514,6 +3517,7 @@ static void batadv_tt_roam_purge(struct batadv_priv *bat_priv)
* batadv_tt_check_roam_count() - check if a client has roamed too frequently
* @bat_priv: the bat priv with all the soft interface information
* @client: mac address of the roaming client
+ * @vid: VLAN identifier
*
* This function checks whether the client already reached the
* maximum number of possible roaming phases. In this case the ROAMING_ADV
@@ -3521,7 +3525,7 @@ static void batadv_tt_roam_purge(struct batadv_priv *bat_priv)
*
* Return: true if the ROAMING_ADV can be sent, false otherwise
*/
-static bool batadv_tt_check_roam_count(struct batadv_priv *bat_priv, u8 *client)
+static bool batadv_tt_check_roam_count(struct batadv_priv *bat_priv, u8 *client, u16 vid)
{
struct batadv_tt_roam_node *tt_roam_node;
bool ret = false;
@@ -3534,6 +3538,9 @@ static bool batadv_tt_check_roam_count(struct batadv_priv *bat_priv, u8 *client)
if (!batadv_compare_eth(tt_roam_node->addr, client))
continue;
+ if (tt_roam_node->vid != vid)
+ continue;
+
if (batadv_has_timed_out(tt_roam_node->first_time,
BATADV_ROAMING_MAX_TIME))
continue;
@@ -3555,6 +3562,7 @@ static bool batadv_tt_check_roam_count(struct batadv_priv *bat_priv, u8 *client)
atomic_set(&tt_roam_node->counter,
BATADV_ROAMING_MAX_COUNT - 1);
ether_addr_copy(tt_roam_node->addr, client);
+ tt_roam_node->vid = vid;
list_add(&tt_roam_node->list, &bat_priv->tt.roam_list);
ret = true;
@@ -3591,7 +3599,7 @@ static void batadv_send_roam_adv(struct batadv_priv *bat_priv, u8 *client,
/* before going on we have to check whether the client has
* already roamed to us too many times
*/
- if (!batadv_tt_check_roam_count(bat_priv, client))
+ if (!batadv_tt_check_roam_count(bat_priv, client, vid))
goto out;
batadv_dbg(BATADV_DBG_TT, bat_priv,
diff --git a/net/batman-adv/tvlv.c b/net/batman-adv/tvlv.c
index 8d6b017c433c..cc66e231ccb8 100644
--- a/net/batman-adv/tvlv.c
+++ b/net/batman-adv/tvlv.c
@@ -398,7 +398,6 @@ static int batadv_tvlv_call_handler(struct batadv_priv *bat_priv,
tvlv_handler->ogm_handler(bat_priv, orig_node,
BATADV_NO_FLAGS,
tvlv_value, tvlv_value_len);
- tvlv_handler->flags |= BATADV_TVLV_HANDLER_OGM_CALLED;
break;
case BATADV_UNICAST_TVLV:
if (!skb)
@@ -430,6 +429,48 @@ static int batadv_tvlv_call_handler(struct batadv_priv *bat_priv,
return NET_RX_SUCCESS;
}
+/**
+ * batadv_tvlv_containers_contain() - check if a tvlv buffer holds a container
+ * @tvlv_value: tvlv content
+ * @tvlv_value_len: tvlv content length
+ * @type: tvlv container type to look for
+ * @version: tvlv container version to look for
+ *
+ * Return: true if a container of the given type and version is present in the
+ * tvlv buffer, false otherwise.
+ */
+static bool batadv_tvlv_containers_contain(void *tvlv_value,
+ u16 tvlv_value_len, u8 type,
+ u8 version)
+{
+ struct batadv_tvlv_hdr *tvlv_hdr;
+ u16 tvlv_value_cont_len;
+
+ while (tvlv_value_len >= sizeof(*tvlv_hdr)) {
+ tvlv_hdr = tvlv_value;
+ tvlv_value_cont_len = ntohs(tvlv_hdr->len);
+ tvlv_value = tvlv_hdr + 1;
+ tvlv_value_len -= sizeof(*tvlv_hdr);
+
+ if (tvlv_value_cont_len > tvlv_value_len)
+ break;
+
+ /* the next tvlv header is accessed assuming (at least) 2-byte
+ * alignment, so it must start at an even offset.
+ */
+ if (tvlv_value_cont_len & 1)
+ break;
+
+ if (tvlv_hdr->type == type && tvlv_hdr->version == version)
+ return true;
+
+ tvlv_value = (u8 *)tvlv_value + tvlv_value_cont_len;
+ tvlv_value_len -= tvlv_value_cont_len;
+ }
+
+ return false;
+}
+
/**
* batadv_tvlv_containers_process() - parse the given tvlv buffer to call the
* appropriate handlers
@@ -449,7 +490,9 @@ int batadv_tvlv_containers_process(struct batadv_priv *bat_priv,
struct sk_buff *skb, void *tvlv_value,
u16 tvlv_value_len)
{
+ u16 tvlv_value_start_len = tvlv_value_len;
struct batadv_tvlv_handler *tvlv_handler;
+ void *tvlv_value_start = tvlv_value;
struct batadv_tvlv_hdr *tvlv_hdr;
u16 tvlv_value_cont_len;
u8 cifnotfound = BATADV_TVLV_HANDLER_OGM_CIFNOTFND;
@@ -464,6 +507,12 @@ int batadv_tvlv_containers_process(struct batadv_priv *bat_priv,
if (tvlv_value_cont_len > tvlv_value_len)
break;
+ /* the next tvlv header is accessed assuming (at least) 2-byte
+ * alignment, so it must start at an even offset.
+ */
+ if (tvlv_value_cont_len & 1)
+ break;
+
tvlv_handler = batadv_tvlv_handler_get(bat_priv,
tvlv_hdr->type,
tvlv_hdr->version);
@@ -487,12 +536,20 @@ int batadv_tvlv_containers_process(struct batadv_priv *bat_priv,
if (!tvlv_handler->ogm_handler)
continue;
- if ((tvlv_handler->flags & BATADV_TVLV_HANDLER_OGM_CIFNOTFND) &&
- !(tvlv_handler->flags & BATADV_TVLV_HANDLER_OGM_CALLED))
- tvlv_handler->ogm_handler(bat_priv, orig_node,
- cifnotfound, NULL, 0);
+ if (!(tvlv_handler->flags & BATADV_TVLV_HANDLER_OGM_CIFNOTFND))
+ continue;
+
+ /* if the corresponding container was present then the handler
+ * was already called from the loop above
+ */
+ if (batadv_tvlv_containers_contain(tvlv_value_start,
+ tvlv_value_start_len,
+ tvlv_handler->type,
+ tvlv_handler->version))
+ continue;
- tvlv_handler->flags &= ~BATADV_TVLV_HANDLER_OGM_CALLED;
+ tvlv_handler->ogm_handler(bat_priv, orig_node,
+ cifnotfound, NULL, 0);
}
rcu_read_unlock();
diff --git a/net/batman-adv/types.h b/net/batman-adv/types.h
index f703d266780d..0022bee14574 100644
--- a/net/batman-adv/types.h
+++ b/net/batman-adv/types.h
@@ -130,6 +130,12 @@ struct batadv_hard_iface_bat_v {
/** @aggr_list: queue for to be aggregated OGM packets */
struct sk_buff_head aggr_list;
+ /**
+ * @aggr_list_enabled: aggr_list is active and new skbs can be
+ * enqueued. Protected by aggr_list.lock after initialization
+ */
+ bool aggr_list_enabled:1;
+
/** @aggr_len: size of the OGM aggregate (excluding ethernet header) */
unsigned int aggr_len;
@@ -1555,9 +1561,12 @@ struct batadv_tp_vars {
/** @unacked_list: list of unacked packets (meta-info only) */
struct list_head unacked_list;
- /** @unacked_lock: protect unacked_list */
+ /** @unacked_lock: protect unacked_list + &batadv_tp_receiver.last_recv */
spinlock_t unacked_lock;
+ /** @unacked_count: number of unacked entries */
+ size_t unacked_count;
+
/** @last_recv_time: time (jiffies) a msg was received */
unsigned long last_recv_time;
@@ -2056,6 +2065,9 @@ struct batadv_tt_roam_node {
/** @addr: mac address of the client in the roaming phase */
u8 addr[ETH_ALEN];
+ /** @vid: VLAN identifier */
+ u16 vid;
+
/**
* @counter: number of allowed roaming events per client within a single
* OGM interval (changes are committed with each OGM)
@@ -2471,13 +2483,6 @@ enum batadv_tvlv_handler_flags {
* will call this handler even if its type was not found (with no data)
*/
BATADV_TVLV_HANDLER_OGM_CIFNOTFND = BIT(1),
-
- /**
- * @BATADV_TVLV_HANDLER_OGM_CALLED: interval tvlv handling flag - the
- * API marks a handler as being called, so it won't be called if the
- * BATADV_TVLV_HANDLER_OGM_CIFNOTFND flag was set
- */
- BATADV_TVLV_HANDLER_OGM_CALLED = BIT(2),
};
#endif /* _NET_BATMAN_ADV_TYPES_H_ */
diff --git a/net/core/filter.c b/net/core/filter.c
index 3d71a5907253..b03621405d06 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -2740,11 +2740,13 @@ BPF_CALL_4(bpf_msg_pull_data, struct sk_msg *, msg, u32, start,
poffset += len;
sge->length = 0;
put_page(sg_page(sge));
+ __clear_bit(i, msg->sg.copy);
sk_msg_iter_var_next(i);
} while (i != last_sge);
sg_set_page(&msg->sg.data[first_sge], page, copy, 0);
+ __clear_bit(first_sge, msg->sg.copy);
/* To repair sg ring we need to shift entries. If we only
* had a single entry though we can just replace it and
@@ -2770,9 +2772,11 @@ BPF_CALL_4(bpf_msg_pull_data, struct sk_msg *, msg, u32, start,
break;
msg->sg.data[i] = msg->sg.data[move_from];
+ sk_msg_sg_copy_assign(msg, i, msg, move_from);
msg->sg.data[move_from].length = 0;
msg->sg.data[move_from].page_link = 0;
msg->sg.data[move_from].offset = 0;
+ __clear_bit(move_from, msg->sg.copy);
sk_msg_iter_var_next(i);
} while (1);
@@ -2801,6 +2805,7 @@ BPF_CALL_4(bpf_msg_push_data, struct sk_msg *, msg, u32, start,
{
struct scatterlist sge, nsge, nnsge, rsge = {0}, *psge;
u32 new, i = 0, l = 0, space, copy = 0, offset = 0;
+ bool sge_copy, nsge_copy, nnsge_copy, rsge_copy = false;
u8 *raw, *to, *from;
struct page *page;
@@ -2873,6 +2878,7 @@ BPF_CALL_4(bpf_msg_push_data, struct sk_msg *, msg, u32, start,
sk_msg_iter_var_prev(i);
psge = sk_msg_elem(msg, i);
rsge = sk_msg_elem_cpy(msg, i);
+ rsge_copy = test_bit(i, msg->sg.copy);
psge->length = start - offset;
rsge.length -= psge->length;
@@ -2897,24 +2903,32 @@ BPF_CALL_4(bpf_msg_push_data, struct sk_msg *, msg, u32, start,
/* Shift one or two slots as needed */
sge = sk_msg_elem_cpy(msg, new);
+ sge_copy = test_bit(new, msg->sg.copy);
sg_unmark_end(&sge);
nsge = sk_msg_elem_cpy(msg, i);
+ nsge_copy = test_bit(i, msg->sg.copy);
if (rsge.length) {
sk_msg_iter_var_next(i);
nnsge = sk_msg_elem_cpy(msg, i);
+ nnsge_copy = test_bit(i, msg->sg.copy);
sk_msg_iter_next(msg, end);
}
while (i != msg->sg.end) {
msg->sg.data[i] = sge;
+ __assign_bit(i, msg->sg.copy, sge_copy);
sge = nsge;
+ sge_copy = nsge_copy;
sk_msg_iter_var_next(i);
if (rsge.length) {
nsge = nnsge;
+ nsge_copy = nnsge_copy;
nnsge = sk_msg_elem_cpy(msg, i);
+ nnsge_copy = test_bit(i, msg->sg.copy);
} else {
nsge = sk_msg_elem_cpy(msg, i);
+ nsge_copy = test_bit(i, msg->sg.copy);
}
}
@@ -2928,6 +2942,7 @@ BPF_CALL_4(bpf_msg_push_data, struct sk_msg *, msg, u32, start,
get_page(sg_page(&rsge));
sk_msg_iter_var_next(new);
msg->sg.data[new] = rsge;
+ __assign_bit(new, msg->sg.copy, rsge_copy);
}
sk_msg_reset_curr(msg);
@@ -2955,25 +2970,33 @@ static void sk_msg_shift_left(struct sk_msg *msg, int i)
prev = i;
sk_msg_iter_var_next(i);
msg->sg.data[prev] = msg->sg.data[i];
+ sk_msg_sg_copy_assign(msg, prev, msg, i);
} while (i != msg->sg.end);
sk_msg_iter_prev(msg, end);
+ __clear_bit(msg->sg.end, msg->sg.copy);
}
static void sk_msg_shift_right(struct sk_msg *msg, int i)
{
struct scatterlist tmp, sge;
+ bool tmp_copy, sge_copy;
sk_msg_iter_next(msg, end);
sge = sk_msg_elem_cpy(msg, i);
+ sge_copy = test_bit(i, msg->sg.copy);
sk_msg_iter_var_next(i);
tmp = sk_msg_elem_cpy(msg, i);
+ tmp_copy = test_bit(i, msg->sg.copy);
while (i != msg->sg.end) {
msg->sg.data[i] = sge;
+ __assign_bit(i, msg->sg.copy, sge_copy);
sk_msg_iter_var_next(i);
sge = tmp;
+ sge_copy = tmp_copy;
tmp = sk_msg_elem_cpy(msg, i);
+ tmp_copy = test_bit(i, msg->sg.copy);
}
}
@@ -3033,6 +3056,8 @@ BPF_CALL_4(bpf_msg_pop_data, struct sk_msg *, msg, u32, start,
struct scatterlist *nsge, *sge = sk_msg_elem(msg, i);
int a = start - offset;
int b = sge->length - pop - a;
+ u32 sge_i = i;
+ bool sge_copy = test_bit(i, msg->sg.copy);
sk_msg_iter_var_next(i);
@@ -3045,6 +3070,7 @@ BPF_CALL_4(bpf_msg_pop_data, struct sk_msg *, msg, u32, start,
sg_set_page(nsge,
sg_page(sge),
b, sge->offset + pop + a);
+ __assign_bit(i, msg->sg.copy, sge_copy);
} else {
struct page *page, *orig;
u8 *to, *from;
@@ -3061,6 +3087,7 @@ BPF_CALL_4(bpf_msg_pop_data, struct sk_msg *, msg, u32, start,
memcpy(to, from, a);
memcpy(to + a, from + a + pop, b);
sg_set_page(sge, page, a + b, 0);
+ __clear_bit(sge_i, msg->sg.copy);
put_page(orig);
}
pop = 0;
diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index 2176cd1bc765..118612325ce9 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c
@@ -2205,6 +2205,14 @@ struct net *rtnl_get_net_ns_capable(struct sock *sk, int netnsid)
}
EXPORT_SYMBOL_GPL(rtnl_get_net_ns_capable);
+bool rtnl_dev_link_net_capable(const struct net_device *dev,
+ const struct net *link_net)
+{
+ return net_eq(link_net, dev_net(dev)) ||
+ ns_capable(link_net->user_ns, CAP_NET_ADMIN);
+}
+EXPORT_SYMBOL_GPL(rtnl_dev_link_net_capable);
+
static int rtnl_valid_dump_ifinfo_req(const struct nlmsghdr *nlh,
bool strict_check, struct nlattr **tb,
struct netlink_ext_ack *extack)
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 4be699bd3a17..fede3aa3ddbc 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -5525,15 +5525,28 @@ static void __skb_complete_tx_timestamp(struct sk_buff *skb,
static bool skb_may_tx_timestamp(struct sock *sk, bool tsonly)
{
- bool ret;
+ struct socket *sock;
+ struct file *file;
+ bool ret = false;
if (likely(READ_ONCE(sysctl_tstamp_allow_data) || tsonly))
return true;
- read_lock_bh(&sk->sk_callback_lock);
- ret = sk->sk_socket && sk->sk_socket->file &&
- file_ns_capable(sk->sk_socket->file, &init_user_ns, CAP_NET_RAW);
- read_unlock_bh(&sk->sk_callback_lock);
+ /* The sk pointer remains valid as long as the skb is. The sk_socket and
+ * file pointer may become NULL if the socket is closed. Both structures
+ * (including file->cred) are RCU freed which means they can be accessed
+ * within a RCU read section.
+ */
+ rcu_read_lock();
+ sock = READ_ONCE(sk->sk_socket);
+ if (!sock)
+ goto out;
+ file = READ_ONCE(sock->file);
+ if (!file)
+ goto out;
+ ret = file_ns_capable(file, &init_user_ns, CAP_NET_RAW);
+out:
+ rcu_read_unlock();
return ret;
}
diff --git a/net/core/skmsg.c b/net/core/skmsg.c
index fbaee3d09990..7e68b4fdbfce 100644
--- a/net/core/skmsg.c
+++ b/net/core/skmsg.c
@@ -66,6 +66,7 @@ int sk_msg_alloc(struct sock *sk, struct sk_msg *msg, int len,
sge = &msg->sg.data[msg->sg.end];
sg_unmark_end(sge);
sg_set_page(sge, pfrag->page, use, orig_offset);
+ __clear_bit(msg->sg.end, msg->sg.copy);
get_page(pfrag->page);
sk_msg_iter_next(msg, end);
}
@@ -186,6 +187,7 @@ static int sk_msg_free_elem(struct sock *sk, struct sk_msg *msg, u32 i,
sk_mem_uncharge(sk, len);
put_page(sg_page(sge));
}
+ __clear_bit(i, msg->sg.copy);
memset(sge, 0, sizeof(*sge));
return len;
}
diff --git a/net/ipv4/ip_gre.c b/net/ipv4/ip_gre.c
index 084556b03a2e..ffad16b023ad 100644
--- a/net/ipv4/ip_gre.c
+++ b/net/ipv4/ip_gre.c
@@ -1446,6 +1446,9 @@ static int ipgre_changelink(struct net_device *dev, struct nlattr *tb[],
__u32 fwmark = t->fwmark;
int err;
+ if (!rtnl_dev_link_net_capable(dev, t->net))
+ return -EPERM;
+
err = ipgre_newlink_encap_setup(dev, data);
if (err)
return err;
@@ -1475,6 +1478,9 @@ static int erspan_changelink(struct net_device *dev, struct nlattr *tb[],
__u32 fwmark = t->fwmark;
int err;
+ if (!rtnl_dev_link_net_capable(dev, t->net))
+ return -EPERM;
+
err = ipgre_newlink_encap_setup(dev, data);
if (err)
return err;
diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c
index ba51fc42531c..41031c5c4303 100644
--- a/net/ipv4/ip_output.c
+++ b/net/ipv4/ip_output.c
@@ -1118,8 +1118,8 @@ static int __ip_append_data(struct sock *sk,
!(rt->dst.dev->features & NETIF_F_SG)))
alloclen = fraglen;
else {
- alloclen = fragheaderlen + transhdrlen;
- pagedlen = datalen - transhdrlen;
+ alloclen = fragheaderlen + transhdrlen + fraggap;
+ pagedlen = datalen - transhdrlen - fraggap;
}
alloclen += alloc_extra;
@@ -1166,10 +1166,10 @@ static int __ip_append_data(struct sock *sk,
}
copy = datalen - transhdrlen - fraggap - pagedlen;
- /* [!] NOTE: copy will be negative if pagedlen>0
- * because then the equation reduces to -fraggap.
- */
- if (copy > 0 && getfrag(from, data + transhdrlen, offset, copy, fraggap, skb) < 0) {
+ if (copy > 0 &&
+ INDIRECT_CALL_1(getfrag, ip_generic_getfrag,
+ from, data + transhdrlen, offset,
+ copy, fraggap, skb) < 0) {
err = -EFAULT;
kfree_skb(skb);
goto error;
@@ -1213,8 +1213,9 @@ static int __ip_append_data(struct sock *sk,
unsigned int off;
off = skb->len;
- if (getfrag(from, skb_put(skb, copy),
- offset, copy, off, skb) < 0) {
+ if (INDIRECT_CALL_1(getfrag, ip_generic_getfrag,
+ from, skb_put(skb, copy),
+ offset, copy, off, skb) < 0) {
__skb_trim(skb, off);
err = -EFAULT;
goto error;
@@ -1254,7 +1255,8 @@ static int __ip_append_data(struct sock *sk,
get_page(pfrag->page);
}
copy = min_t(int, copy, pfrag->size - pfrag->offset);
- if (getfrag(from,
+ if (INDIRECT_CALL_1(getfrag, ip_generic_getfrag,
+ from,
page_address(pfrag->page) + pfrag->offset,
offset, copy, skb->len, skb) < 0)
goto error_efault;
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index be38712265c3..a7b1fe194fb1 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -1777,6 +1777,10 @@ static int tcp_ao_delete_key(struct sock *sk, struct tcp_ao_info *ao_info,
* them and we can just free all resources in RCU fashion.
*/
if (del_async) {
+ if (ao_info->current_key == key)
+ WRITE_ONCE(ao_info->current_key, NULL);
+ if (ao_info->rnext_key == key)
+ WRITE_ONCE(ao_info->rnext_key, NULL);
atomic_sub(tcp_ao_sizeof_key(key), &sk->sk_omem_alloc);
call_rcu(&key->rcu, tcp_ao_key_free_rcu);
return 0;
diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
index 31021dc40e52..50b41be5d38d 100644
--- a/net/ipv6/ip6_output.c
+++ b/net/ipv6/ip6_output.c
@@ -1633,8 +1633,8 @@ static int __ip6_append_data(struct sock *sk,
!(rt->dst.dev->features & NETIF_F_SG)))
alloclen = fraglen;
else {
- alloclen = fragheaderlen + transhdrlen;
- pagedlen = datalen - transhdrlen;
+ alloclen = fragheaderlen + transhdrlen + fraggap;
+ pagedlen = datalen - transhdrlen - fraggap;
}
alloclen += alloc_extra;
@@ -1649,10 +1649,7 @@ static int __ip6_append_data(struct sock *sk,
fraglen = datalen + fragheaderlen;
copy = datalen - transhdrlen - fraggap - pagedlen;
- /* [!] NOTE: copy may be negative if pagedlen>0
- * because then the equation may reduces to -fraggap.
- */
- if (copy < 0 && !(flags & MSG_SPLICE_PAGES)) {
+ if (copy < 0) {
err = -EINVAL;
goto error;
}
@@ -1698,8 +1695,9 @@ static int __ip6_append_data(struct sock *sk,
pskb_trim_unique(skb_prev, maxfraglen);
}
if (copy > 0 &&
- getfrag(from, data + transhdrlen, offset,
- copy, fraggap, skb) < 0) {
+ INDIRECT_CALL_1(getfrag, ip_generic_getfrag,
+ from, data + transhdrlen, offset,
+ copy, fraggap, skb) < 0) {
err = -EFAULT;
kfree_skb(skb);
goto error;
@@ -1743,8 +1741,9 @@ static int __ip6_append_data(struct sock *sk,
unsigned int off;
off = skb->len;
- if (getfrag(from, skb_put(skb, copy),
- offset, copy, off, skb) < 0) {
+ if (INDIRECT_CALL_1(getfrag, ip_generic_getfrag,
+ from, skb_put(skb, copy),
+ offset, copy, off, skb) < 0) {
__skb_trim(skb, off);
err = -EFAULT;
goto error;
@@ -1784,7 +1783,8 @@ static int __ip6_append_data(struct sock *sk,
get_page(pfrag->page);
}
copy = min_t(int, copy, pfrag->size - pfrag->offset);
- if (getfrag(from,
+ if (INDIRECT_CALL_1(getfrag, ip_generic_getfrag,
+ from,
page_address(pfrag->page) + pfrag->offset,
offset, copy, skb->len, skb) < 0)
goto error_efault;
diff --git a/net/ipv6/ip6_udp_tunnel.c b/net/ipv6/ip6_udp_tunnel.c
index 2acf1bb93fc0..f22eff2ba77c 100644
--- a/net/ipv6/ip6_udp_tunnel.c
+++ b/net/ipv6/ip6_udp_tunnel.c
@@ -74,13 +74,13 @@ int udp_sock_create6(struct net *net, struct udp_port_cfg *cfg,
}
EXPORT_SYMBOL_GPL(udp_sock_create6);
-int udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk,
- struct sk_buff *skb,
- struct net_device *dev,
- const struct in6_addr *saddr,
- const struct in6_addr *daddr,
- __u8 prio, __u8 ttl, __be32 label,
- __be16 src_port, __be16 dst_port, bool nocheck)
+void udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk,
+ struct sk_buff *skb,
+ struct net_device *dev,
+ const struct in6_addr *saddr,
+ const struct in6_addr *daddr,
+ __u8 prio, __u8 ttl, __be32 label,
+ __be16 src_port, __be16 dst_port, bool nocheck)
{
struct udphdr *uh;
struct ipv6hdr *ip6h;
@@ -109,7 +109,6 @@ int udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk,
ip6h->saddr = *saddr;
ip6tunnel_xmit(sk, skb, dev);
- return 0;
}
EXPORT_SYMBOL_GPL(udp_tunnel6_xmit_skb);
diff --git a/net/ipv6/ip6_vti.c b/net/ipv6/ip6_vti.c
index 6fe696939d04..e0e6e67a25e0 100644
--- a/net/ipv6/ip6_vti.c
+++ b/net/ipv6/ip6_vti.c
@@ -1159,6 +1159,7 @@ static int __net_init vti6_init_net(struct net *net)
goto err_alloc_dev;
dev_net_set(ip6n->fb_tnl_dev, net);
ip6n->fb_tnl_dev->rtnl_link_ops = &vti6_link_ops;
+ ip6n->fb_tnl_dev->netns_local = true;
err = vti6_fb_tnl_dev_init(ip6n->fb_tnl_dev);
if (err < 0)
diff --git a/net/mac802154/llsec.c b/net/mac802154/llsec.c
index f13b07ebfb98..09a47104b577 100644
--- a/net/mac802154/llsec.c
+++ b/net/mac802154/llsec.c
@@ -710,6 +710,7 @@ int mac802154_llsec_encrypt(struct mac802154_llsec *sec, struct sk_buff *skb)
{
struct ieee802154_hdr hdr;
int rc, authlen, hlen;
+ struct sk_buff *trailer;
struct mac802154_llsec_key *key;
u32 frame_ctr;
@@ -769,6 +770,12 @@ int mac802154_llsec_encrypt(struct mac802154_llsec *sec, struct sk_buff *skb)
skb->mac_len = ieee802154_hdr_push(skb, &hdr);
skb_reset_mac_header(skb);
+ rc = skb_cow_data(skb, 0, &trailer);
+ if (rc < 0) {
+ llsec_key_put(key);
+ return rc;
+ }
+
rc = llsec_do_encrypt(skb, sec, &hdr, key);
llsec_key_put(key);
@@ -908,6 +915,13 @@ llsec_do_decrypt(struct sk_buff *skb, const struct mac802154_llsec *sec,
const struct ieee802154_hdr *hdr,
struct mac802154_llsec_key *key, __le64 dev_addr)
{
+ struct sk_buff *trailer;
+ int err;
+
+ err = skb_cow_data(skb, 0, &trailer);
+ if (err < 0)
+ return err;
+
if (hdr->sec.level == IEEE802154_SCF_SECLEVEL_ENC)
return llsec_do_decrypt_unauth(skb, sec, hdr, key, dev_addr);
else
diff --git a/net/phonet/pn_dev.c b/net/phonet/pn_dev.c
index cde671d29d5d..3f020c362d6c 100644
--- a/net/phonet/pn_dev.c
+++ b/net/phonet/pn_dev.c
@@ -98,11 +98,14 @@ static void phonet_device_destroy(struct net_device *dev)
mutex_unlock(&pndevs->lock);
if (pnd) {
+ struct net *net = dev_net(dev);
+ u32 ifindex = dev->ifindex;
u8 addr;
for_each_set_bit(addr, pnd->addrs, 64)
- phonet_address_notify(RTM_DELADDR, dev, addr);
- kfree(pnd);
+ phonet_address_notify(net, RTM_DELADDR, ifindex, addr);
+
+ kfree_rcu(pnd, rcu);
}
}
@@ -244,8 +247,9 @@ static int phonet_device_autoconf(struct net_device *dev)
ret = phonet_address_add(dev, req.ifr_phonet_autoconf.device);
if (ret)
return ret;
- phonet_address_notify(RTM_NEWADDR, dev,
- req.ifr_phonet_autoconf.device);
+
+ phonet_address_notify(dev_net(dev), RTM_NEWADDR, dev->ifindex,
+ req.ifr_phonet_autoconf.device);
return 0;
}
diff --git a/net/phonet/pn_netlink.c b/net/phonet/pn_netlink.c
index 894e5c72d6bf..23097085ad38 100644
--- a/net/phonet/pn_netlink.c
+++ b/net/phonet/pn_netlink.c
@@ -19,10 +19,10 @@
/* Device address handling */
-static int fill_addr(struct sk_buff *skb, struct net_device *dev, u8 addr,
+static int fill_addr(struct sk_buff *skb, u32 ifindex, u8 addr,
u32 portid, u32 seq, int event);
-void phonet_address_notify(int event, struct net_device *dev, u8 addr)
+void phonet_address_notify(struct net *net, int event, u32 ifindex, u8 addr)
{
struct sk_buff *skb;
int err = -ENOBUFS;
@@ -31,17 +31,18 @@ void phonet_address_notify(int event, struct net_device *dev, u8 addr)
nla_total_size(1), GFP_KERNEL);
if (skb == NULL)
goto errout;
- err = fill_addr(skb, dev, addr, 0, 0, event);
+
+ err = fill_addr(skb, ifindex, addr, 0, 0, event);
if (err < 0) {
WARN_ON(err == -EMSGSIZE);
kfree_skb(skb);
goto errout;
}
- rtnl_notify(skb, dev_net(dev), 0,
- RTNLGRP_PHONET_IFADDR, NULL, GFP_KERNEL);
+
+ rtnl_notify(skb, net, 0, RTNLGRP_PHONET_IFADDR, NULL, GFP_KERNEL);
return;
errout:
- rtnl_set_sk_err(dev_net(dev), RTNLGRP_PHONET_IFADDR, err);
+ rtnl_set_sk_err(net, RTNLGRP_PHONET_IFADDR, err);
}
static const struct nla_policy ifa_phonet_policy[IFA_MAX+1] = {
@@ -88,12 +89,12 @@ static int addr_doit(struct sk_buff *skb, struct nlmsghdr *nlh,
else
err = phonet_address_del(dev, pnaddr);
if (!err)
- phonet_address_notify(nlh->nlmsg_type, dev, pnaddr);
+ phonet_address_notify(net, nlh->nlmsg_type, ifm->ifa_index, pnaddr);
return err;
}
-static int fill_addr(struct sk_buff *skb, struct net_device *dev, u8 addr,
- u32 portid, u32 seq, int event)
+static int fill_addr(struct sk_buff *skb, u32 ifindex, u8 addr,
+ u32 portid, u32 seq, int event)
{
struct ifaddrmsg *ifm;
struct nlmsghdr *nlh;
@@ -107,7 +108,7 @@ static int fill_addr(struct sk_buff *skb, struct net_device *dev, u8 addr,
ifm->ifa_prefixlen = 0;
ifm->ifa_flags = IFA_F_PERMANENT;
ifm->ifa_scope = RT_SCOPE_LINK;
- ifm->ifa_index = dev->ifindex;
+ ifm->ifa_index = ifindex;
if (nla_put_u8(skb, IFA_LOCAL, addr))
goto nla_put_failure;
nlmsg_end(skb, nlh);
@@ -140,7 +141,7 @@ static int getaddr_dumpit(struct sk_buff *skb, struct netlink_callback *cb)
if (addr_idx++ < addr_start_idx)
continue;
- if (fill_addr(skb, pnd->netdev, addr << 2,
+ if (fill_addr(skb, pnd->netdev->ifindex, addr << 2,
NETLINK_CB(cb->skb).portid,
cb->nlh->nlmsg_seq, RTM_NEWADDR) < 0)
goto out;
diff --git a/net/rxrpc/input.c b/net/rxrpc/input.c
index 6a075a7c190d..ca2d40ba7098 100644
--- a/net/rxrpc/input.c
+++ b/net/rxrpc/input.c
@@ -775,9 +775,23 @@ static void rxrpc_input_soft_acks(struct rxrpc_call *call,
rxrpc_seq_t since)
{
struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
- unsigned int i, old_nacks = 0;
+ unsigned int i, old_nacks = 0, nsack;
rxrpc_seq_t lowest_nak = seq + sp->ack.nr_acks;
- u8 *acks = skb->data + sizeof(struct rxrpc_wire_header) + sizeof(struct rxrpc_ackpacket);
+ u8 sack[256] __aligned(sizeof(unsigned long));
+ u8 *acks = sack;
+
+ /* AF_RXRPC assumes that it can access the SACK table directly from
+ * skb->data as a flat buffer, but the skb may be non-linear (e.g. a
+ * fragmented UDP packet) and skb_condense() can silently fail to
+ * linearise it. Copy the SACK table out into a local buffer before
+ * parsing it.
+ */
+ memset(sack, 0, sizeof(sack));
+ nsack = umin(sp->ack.nr_acks, 256);
+ if (skb_copy_bits(skb,
+ sizeof(struct rxrpc_wire_header) + sizeof(struct rxrpc_ackpacket),
+ sack, nsack) < 0)
+ return;
for (i = 0; i < sp->ack.nr_acks; i++) {
if (acks[i] == RXRPC_ACK_TYPE_ACK) {
@@ -934,9 +948,6 @@ static void rxrpc_input_ack(struct rxrpc_call *call, struct sk_buff *skb)
skb_copy_bits(skb, ioffset, &trailer, sizeof(trailer)) < 0)
return rxrpc_proto_abort(call, 0, rxrpc_badmsg_short_ack_trailer);
- if (nr_acks > 0)
- skb_condense(skb);
-
if (call->cong_last_nack) {
since = rxrpc_input_check_prev_ack(call, &summary, first_soft_ack);
rxrpc_free_skb(call->cong_last_nack, rxrpc_skb_put_last_nack);
diff --git a/net/sctp/ipv6.c b/net/sctp/ipv6.c
index 9f835e674c59..b45cc51dfc35 100644
--- a/net/sctp/ipv6.c
+++ b/net/sctp/ipv6.c
@@ -261,9 +261,12 @@ static int sctp_v6_xmit(struct sk_buff *skb, struct sctp_transport *t)
skb_set_inner_ipproto(skb, IPPROTO_SCTP);
label = ip6_make_flowlabel(sock_net(sk), skb, fl6->flowlabel, true, fl6);
- return udp_tunnel6_xmit_skb(dst, sk, skb, NULL, &fl6->saddr,
- &fl6->daddr, tclass, ip6_dst_hoplimit(dst),
- label, sctp_sk(sk)->udp_port, t->encap_port, false);
+ local_bh_disable();
+ udp_tunnel6_xmit_skb(dst, sk, skb, NULL, &fl6->saddr, &fl6->daddr,
+ tclass, ip6_dst_hoplimit(dst), label,
+ sctp_sk(sk)->udp_port, t->encap_port, false);
+ local_bh_enable();
+ return 0;
}
/* Returns the dst cache entry for the given source and destination ip
diff --git a/net/sctp/protocol.c b/net/sctp/protocol.c
index 39ca5403d4d7..6ea15361088b 100644
--- a/net/sctp/protocol.c
+++ b/net/sctp/protocol.c
@@ -1086,9 +1086,11 @@ static inline int sctp_v4_xmit(struct sk_buff *skb, struct sctp_transport *t)
skb_reset_inner_mac_header(skb);
skb_reset_inner_transport_header(skb);
skb_set_inner_ipproto(skb, IPPROTO_SCTP);
+ local_bh_disable();
udp_tunnel_xmit_skb(dst_rtable(dst), sk, skb, fl4->saddr,
fl4->daddr, dscp, ip4_dst_hoplimit(dst), df,
sctp_sk(sk)->udp_port, t->encap_port, false, false);
+ local_bh_enable();
return 0;
}
diff --git a/net/socket.c b/net/socket.c
index 5c5dd9f6605a..723bc3a1ba5c 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -652,7 +652,7 @@ static void __sock_release(struct socket *sock, struct inode *inode)
iput(SOCK_INODE(sock));
return;
}
- sock->file = NULL;
+ WRITE_ONCE(sock->file, NULL);
}
/**
diff --git a/net/tipc/crypto.c b/net/tipc/crypto.c
index d7736d900271..d58db40d6c03 100644
--- a/net/tipc/crypto.c
+++ b/net/tipc/crypto.c
@@ -950,12 +950,20 @@ static int tipc_aead_decrypt(struct net *net, struct tipc_aead *aead,
goto exit;
}
+ /* Get net to avoid freed tipc_crypto when delete namespace */
+ if (!maybe_get_net(net)) {
+ tipc_bearer_put(b);
+ rc = -ENODEV;
+ goto exit;
+ }
+
/* Now, do decrypt */
rc = crypto_aead_decrypt(req);
if (rc == -EINPROGRESS || rc == -EBUSY)
return rc;
tipc_bearer_put(b);
+ put_net(net);
exit:
kfree(ctx);
@@ -993,6 +1001,7 @@ static void tipc_aead_decrypt_done(void *data, int err)
}
tipc_bearer_put(b);
+ put_net(net);
}
static inline int tipc_ehdr_size(struct tipc_ehdr *ehdr)
diff --git a/net/tipc/udp_media.c b/net/tipc/udp_media.c
index 258d6aa4f21a..1b8d6bbf8a8e 100644
--- a/net/tipc/udp_media.c
+++ b/net/tipc/udp_media.c
@@ -172,7 +172,7 @@ static int tipc_udp_xmit(struct net *net, struct sk_buff *skb,
struct udp_media_addr *dst, struct dst_cache *cache)
{
struct dst_entry *ndst;
- int ttl, err = 0;
+ int ttl, err;
local_bh_disable();
ndst = dst_cache_get(cache);
@@ -217,13 +217,13 @@ static int tipc_udp_xmit(struct net *net, struct sk_buff *skb,
dst_cache_set_ip6(cache, ndst, &fl6.saddr);
}
ttl = ip6_dst_hoplimit(ndst);
- err = udp_tunnel6_xmit_skb(ndst, ub->ubsock->sk, skb, NULL,
- &src->ipv6, &dst->ipv6, 0, ttl, 0,
- src->port, dst->port, false);
+ udp_tunnel6_xmit_skb(ndst, ub->ubsock->sk, skb, NULL,
+ &src->ipv6, &dst->ipv6, 0, ttl, 0,
+ src->port, dst->port, false);
#endif
}
local_bh_enable();
- return err;
+ return 0;
tx_error:
local_bh_enable();
diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c
index 129a8c778d32..d3eab47b47ea 100644
--- a/net/tls/tls_sw.c
+++ b/net/tls/tls_sw.c
@@ -623,6 +623,7 @@ static int tls_split_open_record(struct sock *sk, struct tls_rec *from,
struct scatterlist *sge, *osge, *nsge;
u32 orig_size = msg_opl->sg.size;
struct scatterlist tmp = { };
+ u32 tmp_i = 0;
struct sk_msg *msg_npl;
struct tls_rec *new;
int ret;
@@ -644,6 +645,7 @@ static int tls_split_open_record(struct sock *sk, struct tls_rec *from,
if (sge->length > apply) {
u32 len = sge->length - apply;
+ tmp_i = i;
get_page(sg_page(sge));
sg_set_page(&tmp, sg_page(sge), len,
sge->offset + apply);
@@ -675,6 +677,7 @@ static int tls_split_open_record(struct sock *sk, struct tls_rec *from,
nsge = sk_msg_elem(msg_npl, j);
if (tmp.length) {
memcpy(nsge, &tmp, sizeof(*nsge));
+ sk_msg_sg_copy_assign(msg_npl, j, msg_opl, tmp_i);
sk_msg_iter_var_next(j);
nsge = sk_msg_elem(msg_npl, j);
}
@@ -682,6 +685,7 @@ static int tls_split_open_record(struct sock *sk, struct tls_rec *from,
osge = sk_msg_elem(msg_opl, i);
while (osge->length) {
memcpy(nsge, osge, sizeof(*nsge));
+ sk_msg_sg_copy_assign(msg_npl, j, msg_opl, i);
sg_unmark_end(nsge);
sk_msg_iter_var_next(i);
sk_msg_iter_var_next(j);
diff --git a/net/unix/garbage.c b/net/unix/garbage.c
index 1cdb54c61619..fa6983dc3181 100644
--- a/net/unix/garbage.c
+++ b/net/unix/garbage.c
@@ -583,6 +583,8 @@ static void __unix_gc(struct work_struct *work)
struct sk_buff_head hitlist;
struct sk_buff *skb;
+ WRITE_ONCE(gc_in_progress, true);
+
spin_lock(&unix_gc_lock);
if (!unix_graph_maybe_cyclic) {
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index a9b3f34a78d2..94f794ca11ca 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -173,12 +173,14 @@ mksysmap()
sorttable()
{
- ${objtree}/scripts/sorttable ${1}
+ ${NM} -S ${1} > .tmp_vmlinux.nm-sort
+ ${objtree}/scripts/sorttable -s .tmp_vmlinux.nm-sort ${1}
}
cleanup()
{
rm -f .btf.*
+ rm -f .tmp_vmlinux.nm-sort
rm -f System.map
rm -f vmlinux
rm -f vmlinux.map
diff --git a/scripts/sorttable.c b/scripts/sorttable.c
index 83cdb843d92f..deed676bfe38 100644
--- a/scripts/sorttable.c
+++ b/scripts/sorttable.c
@@ -28,6 +28,7 @@
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
+#include <stdbool.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
@@ -64,14 +65,246 @@
#define EM_LOONGARCH 258
#endif
+typedef union {
+ Elf32_Ehdr e32;
+ Elf64_Ehdr e64;
+} Elf_Ehdr;
+
+typedef union {
+ Elf32_Shdr e32;
+ Elf64_Shdr e64;
+} Elf_Shdr;
+
+typedef union {
+ Elf32_Sym e32;
+ Elf64_Sym e64;
+} Elf_Sym;
+
+typedef union {
+ Elf32_Rela e32;
+ Elf64_Rela e64;
+} Elf_Rela;
+
static uint32_t (*r)(const uint32_t *);
static uint16_t (*r2)(const uint16_t *);
static uint64_t (*r8)(const uint64_t *);
static void (*w)(uint32_t, uint32_t *);
-static void (*w2)(uint16_t, uint16_t *);
static void (*w8)(uint64_t, uint64_t *);
typedef void (*table_sort_t)(char *, int);
+static struct elf_funcs {
+ int (*compare_extable)(const void *a, const void *b);
+ uint64_t (*ehdr_shoff)(Elf_Ehdr *ehdr);
+ uint16_t (*ehdr_shstrndx)(Elf_Ehdr *ehdr);
+ uint16_t (*ehdr_shentsize)(Elf_Ehdr *ehdr);
+ uint16_t (*ehdr_shnum)(Elf_Ehdr *ehdr);
+ uint64_t (*shdr_addr)(Elf_Shdr *shdr);
+ uint64_t (*shdr_offset)(Elf_Shdr *shdr);
+ uint64_t (*shdr_size)(Elf_Shdr *shdr);
+ uint64_t (*shdr_entsize)(Elf_Shdr *shdr);
+ uint32_t (*shdr_link)(Elf_Shdr *shdr);
+ uint32_t (*shdr_name)(Elf_Shdr *shdr);
+ uint32_t (*shdr_type)(Elf_Shdr *shdr);
+ uint8_t (*sym_type)(Elf_Sym *sym);
+ uint32_t (*sym_name)(Elf_Sym *sym);
+ uint64_t (*sym_value)(Elf_Sym *sym);
+ uint16_t (*sym_shndx)(Elf_Sym *sym);
+ uint64_t (*rela_offset)(Elf_Rela *rela);
+ uint64_t (*rela_info)(Elf_Rela *rela);
+ uint64_t (*rela_addend)(Elf_Rela *rela);
+ void (*rela_write_addend)(Elf_Rela *rela, uint64_t val);
+} e;
+
+static uint64_t ehdr64_shoff(Elf_Ehdr *ehdr)
+{
+ return r8(&ehdr->e64.e_shoff);
+}
+
+static uint64_t ehdr32_shoff(Elf_Ehdr *ehdr)
+{
+ return r(&ehdr->e32.e_shoff);
+}
+
+static uint64_t ehdr_shoff(Elf_Ehdr *ehdr)
+{
+ return e.ehdr_shoff(ehdr);
+}
+
+#define EHDR_HALF(fn_name) \
+static uint16_t ehdr64_##fn_name(Elf_Ehdr *ehdr) \
+{ \
+ return r2(&ehdr->e64.e_##fn_name); \
+} \
+ \
+static uint16_t ehdr32_##fn_name(Elf_Ehdr *ehdr) \
+{ \
+ return r2(&ehdr->e32.e_##fn_name); \
+} \
+ \
+static uint16_t ehdr_##fn_name(Elf_Ehdr *ehdr) \
+{ \
+ return e.ehdr_##fn_name(ehdr); \
+}
+
+EHDR_HALF(shentsize)
+EHDR_HALF(shstrndx)
+EHDR_HALF(shnum)
+
+#define SHDR_WORD(fn_name) \
+static uint32_t shdr64_##fn_name(Elf_Shdr *shdr) \
+{ \
+ return r(&shdr->e64.sh_##fn_name); \
+} \
+ \
+static uint32_t shdr32_##fn_name(Elf_Shdr *shdr) \
+{ \
+ return r(&shdr->e32.sh_##fn_name); \
+} \
+ \
+static uint32_t shdr_##fn_name(Elf_Shdr *shdr) \
+{ \
+ return e.shdr_##fn_name(shdr); \
+}
+
+#define SHDR_ADDR(fn_name) \
+static uint64_t shdr64_##fn_name(Elf_Shdr *shdr) \
+{ \
+ return r8(&shdr->e64.sh_##fn_name); \
+} \
+ \
+static uint64_t shdr32_##fn_name(Elf_Shdr *shdr) \
+{ \
+ return r(&shdr->e32.sh_##fn_name); \
+} \
+ \
+static uint64_t shdr_##fn_name(Elf_Shdr *shdr) \
+{ \
+ return e.shdr_##fn_name(shdr); \
+}
+
+#define SHDR_WORD(fn_name) \
+static uint32_t shdr64_##fn_name(Elf_Shdr *shdr) \
+{ \
+ return r(&shdr->e64.sh_##fn_name); \
+} \
+ \
+static uint32_t shdr32_##fn_name(Elf_Shdr *shdr) \
+{ \
+ return r(&shdr->e32.sh_##fn_name); \
+} \
+static uint32_t shdr_##fn_name(Elf_Shdr *shdr) \
+{ \
+ return e.shdr_##fn_name(shdr); \
+}
+
+SHDR_ADDR(addr)
+SHDR_ADDR(offset)
+SHDR_ADDR(size)
+SHDR_ADDR(entsize)
+
+SHDR_WORD(link)
+SHDR_WORD(name)
+SHDR_WORD(type)
+
+#define SYM_ADDR(fn_name) \
+static uint64_t sym64_##fn_name(Elf_Sym *sym) \
+{ \
+ return r8(&sym->e64.st_##fn_name); \
+} \
+ \
+static uint64_t sym32_##fn_name(Elf_Sym *sym) \
+{ \
+ return r(&sym->e32.st_##fn_name); \
+} \
+ \
+static uint64_t sym_##fn_name(Elf_Sym *sym) \
+{ \
+ return e.sym_##fn_name(sym); \
+}
+
+#define SYM_WORD(fn_name) \
+static uint32_t sym64_##fn_name(Elf_Sym *sym) \
+{ \
+ return r(&sym->e64.st_##fn_name); \
+} \
+ \
+static uint32_t sym32_##fn_name(Elf_Sym *sym) \
+{ \
+ return r(&sym->e32.st_##fn_name); \
+} \
+ \
+static uint32_t sym_##fn_name(Elf_Sym *sym) \
+{ \
+ return e.sym_##fn_name(sym); \
+}
+
+#define SYM_HALF(fn_name) \
+static uint16_t sym64_##fn_name(Elf_Sym *sym) \
+{ \
+ return r2(&sym->e64.st_##fn_name); \
+} \
+ \
+static uint16_t sym32_##fn_name(Elf_Sym *sym) \
+{ \
+ return r2(&sym->e32.st_##fn_name); \
+} \
+ \
+static uint16_t sym_##fn_name(Elf_Sym *sym) \
+{ \
+ return e.sym_##fn_name(sym); \
+}
+
+static uint8_t sym64_type(Elf_Sym *sym)
+{
+ return ELF64_ST_TYPE(sym->e64.st_info);
+}
+
+static uint8_t sym32_type(Elf_Sym *sym)
+{
+ return ELF32_ST_TYPE(sym->e32.st_info);
+}
+
+static uint8_t sym_type(Elf_Sym *sym)
+{
+ return e.sym_type(sym);
+}
+
+SYM_ADDR(value)
+SYM_WORD(name)
+SYM_HALF(shndx)
+
+#define __maybe_unused __attribute__((__unused__))
+
+#define RELA_ADDR(fn_name) \
+static uint64_t rela64_##fn_name(Elf_Rela *rela) \
+{ \
+ return r8((uint64_t *)&rela->e64.r_##fn_name); \
+} \
+ \
+static uint64_t rela32_##fn_name(Elf_Rela *rela) \
+{ \
+ return r((uint32_t *)&rela->e32.r_##fn_name); \
+} \
+ \
+static uint64_t __maybe_unused rela_##fn_name(Elf_Rela *rela) \
+{ \
+ return e.rela_##fn_name(rela); \
+}
+
+RELA_ADDR(offset)
+RELA_ADDR(info)
+RELA_ADDR(addend)
+
+static void rela64_write_addend(Elf_Rela *rela, uint64_t val)
+{
+ w8(val, (uint64_t *)&rela->e64.r_addend);
+}
+
+static void rela32_write_addend(Elf_Rela *rela, uint64_t val)
+{
+ w(val, (uint32_t *)&rela->e32.r_addend);
+}
+
/*
* Get the whole file as a programming convenience in order to avoid
* malloc+lseek+read+free of many pieces. If successful, then mmap
@@ -146,24 +379,14 @@ static void wbe(uint32_t val, uint32_t *x)
put_unaligned_be32(val, x);
}
-static void w2be(uint16_t val, uint16_t *x)
-{
- put_unaligned_be16(val, x);
-}
-
-static void w8be(uint64_t val, uint64_t *x)
-{
- put_unaligned_be64(val, x);
-}
-
static void wle(uint32_t val, uint32_t *x)
{
put_unaligned_le32(val, x);
}
-static void w2le(uint16_t val, uint16_t *x)
+static void w8be(uint64_t val, uint64_t *x)
{
- put_unaligned_le16(val, x);
+ put_unaligned_be64(val, x);
}
static void w8le(uint64_t val, uint64_t *x)
@@ -195,10 +418,740 @@ static inline unsigned int get_secindex(unsigned int shndx,
return r(&symtab_shndx_start[sym_offs]);
}
-/* 32 bit and 64 bit are very similar */
-#include "sorttable.h"
-#define SORTTABLE_64
-#include "sorttable.h"
+static int compare_extable_32(const void *a, const void *b)
+{
+ Elf32_Addr av = r(a);
+ Elf32_Addr bv = r(b);
+
+ if (av < bv)
+ return -1;
+ return av > bv;
+}
+
+static int compare_extable_64(const void *a, const void *b)
+{
+ Elf64_Addr av = r8(a);
+ Elf64_Addr bv = r8(b);
+
+ if (av < bv)
+ return -1;
+ return av > bv;
+}
+
+static int compare_extable(const void *a, const void *b)
+{
+ return e.compare_extable(a, b);
+}
+
+static inline void *get_index(void *start, int entsize, int index)
+{
+ return start + (entsize * index);
+}
+
+static int extable_ent_size;
+static int long_size;
+
+#define ERRSTR_MAXSZ 256
+
+#ifdef UNWINDER_ORC_ENABLED
+/* ORC unwinder only support X86_64 */
+#include <asm/orc_types.h>
+
+static char g_err[ERRSTR_MAXSZ];
+static int *g_orc_ip_table;
+static struct orc_entry *g_orc_table;
+
+static pthread_t orc_sort_thread;
+
+static inline unsigned long orc_ip(const int *ip)
+{
+ return (unsigned long)ip + *ip;
+}
+
+static int orc_sort_cmp(const void *_a, const void *_b)
+{
+ struct orc_entry *orc_a, *orc_b;
+ const int *a = g_orc_ip_table + *(int *)_a;
+ const int *b = g_orc_ip_table + *(int *)_b;
+ unsigned long a_val = orc_ip(a);
+ unsigned long b_val = orc_ip(b);
+
+ if (a_val > b_val)
+ return 1;
+ if (a_val < b_val)
+ return -1;
+
+ /*
+ * The "weak" section terminator entries need to always be on the left
+ * to ensure the lookup code skips them in favor of real entries.
+ * These terminator entries exist to handle any gaps created by
+ * whitelisted .o files which didn't get objtool generation.
+ */
+ orc_a = g_orc_table + (a - g_orc_ip_table);
+ orc_b = g_orc_table + (b - g_orc_ip_table);
+ if (orc_a->type == ORC_TYPE_UNDEFINED && orc_b->type == ORC_TYPE_UNDEFINED)
+ return 0;
+ return orc_a->type == ORC_TYPE_UNDEFINED ? -1 : 1;
+}
+
+static void *sort_orctable(void *arg)
+{
+ int i;
+ int *idxs = NULL;
+ int *tmp_orc_ip_table = NULL;
+ struct orc_entry *tmp_orc_table = NULL;
+ unsigned int *orc_ip_size = (unsigned int *)arg;
+ unsigned int num_entries = *orc_ip_size / sizeof(int);
+ unsigned int orc_size = num_entries * sizeof(struct orc_entry);
+
+ idxs = (int *)malloc(*orc_ip_size);
+ if (!idxs) {
+ snprintf(g_err, ERRSTR_MAXSZ, "malloc idxs: %s",
+ strerror(errno));
+ pthread_exit(g_err);
+ }
+
+ tmp_orc_ip_table = (int *)malloc(*orc_ip_size);
+ if (!tmp_orc_ip_table) {
+ snprintf(g_err, ERRSTR_MAXSZ, "malloc tmp_orc_ip_table: %s",
+ strerror(errno));
+ pthread_exit(g_err);
+ }
+
+ tmp_orc_table = (struct orc_entry *)malloc(orc_size);
+ if (!tmp_orc_table) {
+ snprintf(g_err, ERRSTR_MAXSZ, "malloc tmp_orc_table: %s",
+ strerror(errno));
+ pthread_exit(g_err);
+ }
+
+ /* initialize indices array, convert ip_table to absolute address */
+ for (i = 0; i < num_entries; i++) {
+ idxs[i] = i;
+ tmp_orc_ip_table[i] = g_orc_ip_table[i] + i * sizeof(int);
+ }
+ memcpy(tmp_orc_table, g_orc_table, orc_size);
+
+ qsort(idxs, num_entries, sizeof(int), orc_sort_cmp);
+
+ for (i = 0; i < num_entries; i++) {
+ if (idxs[i] == i)
+ continue;
+
+ /* convert back to relative address */
+ g_orc_ip_table[i] = tmp_orc_ip_table[idxs[i]] - i * sizeof(int);
+ g_orc_table[i] = tmp_orc_table[idxs[i]];
+ }
+
+ free(idxs);
+ free(tmp_orc_ip_table);
+ free(tmp_orc_table);
+ pthread_exit(NULL);
+}
+#endif
+
+#ifdef MCOUNT_SORT_ENABLED
+
+static int compare_values_64(const void *a, const void *b)
+{
+ uint64_t av = *(uint64_t *)a;
+ uint64_t bv = *(uint64_t *)b;
+
+ if (av < bv)
+ return -1;
+ return av > bv;
+}
+
+static int compare_values_32(const void *a, const void *b)
+{
+ uint32_t av = *(uint32_t *)a;
+ uint32_t bv = *(uint32_t *)b;
+
+ if (av < bv)
+ return -1;
+ return av > bv;
+}
+
+static int (*compare_values)(const void *a, const void *b);
+
+/* Only used for sorting mcount table */
+static void rela_write_addend(Elf_Rela *rela, uint64_t val)
+{
+ e.rela_write_addend(rela, val);
+}
+
+struct func_info {
+ uint64_t addr;
+ uint64_t size;
+};
+
+/* List of functions created by: nm -S vmlinux */
+static struct func_info *function_list;
+static int function_list_size;
+
+/* Allocate functions in 1k blocks */
+#define FUNC_BLK_SIZE 1024
+#define FUNC_BLK_MASK (FUNC_BLK_SIZE - 1)
+
+static int add_field(uint64_t addr, uint64_t size)
+{
+ struct func_info *fi;
+ int fsize = function_list_size;
+
+ if (!(fsize & FUNC_BLK_MASK)) {
+ fsize += FUNC_BLK_SIZE;
+ fi = realloc(function_list, fsize * sizeof(struct func_info));
+ if (!fi)
+ return -1;
+ function_list = fi;
+ }
+ fi = &function_list[function_list_size++];
+ fi->addr = addr;
+ fi->size = size;
+ return 0;
+}
+
+/* Used for when mcount/fentry is before the function entry */
+static int before_func;
+
+/* Only return match if the address lies inside the function size */
+static int cmp_func_addr(const void *K, const void *A)
+{
+ uint64_t key = *(const uint64_t *)K;
+ const struct func_info *a = A;
+
+ if (key + before_func < a->addr)
+ return -1;
+ return key >= a->addr + a->size;
+}
+
+/* Find the function in function list that is bounded by the function size */
+static int find_func(uint64_t key)
+{
+ return bsearch(&key, function_list, function_list_size,
+ sizeof(struct func_info), cmp_func_addr) != NULL;
+}
+
+static int cmp_funcs(const void *A, const void *B)
+{
+ const struct func_info *a = A;
+ const struct func_info *b = B;
+
+ if (a->addr < b->addr)
+ return -1;
+ return a->addr > b->addr;
+}
+
+static int parse_symbols(const char *fname)
+{
+ FILE *fp;
+ char addr_str[20]; /* Only need 17, but round up to next int size */
+ char size_str[20];
+ char type;
+
+ fp = fopen(fname, "r");
+ if (!fp) {
+ perror(fname);
+ return -1;
+ }
+
+ while (fscanf(fp, "%16s %16s %c %*s\n", addr_str, size_str, &type) == 3) {
+ uint64_t addr;
+ uint64_t size;
+
+ /* Only care about functions */
+ if (type != 't' && type != 'T' && type != 'W')
+ continue;
+
+ addr = strtoull(addr_str, NULL, 16);
+ size = strtoull(size_str, NULL, 16);
+ if (add_field(addr, size) < 0)
+ return -1;
+ }
+ fclose(fp);
+
+ qsort(function_list, function_list_size, sizeof(struct func_info), cmp_funcs);
+
+ return 0;
+}
+
+static pthread_t mcount_sort_thread;
+static bool sort_reloc;
+
+static long rela_type;
+
+static char m_err[ERRSTR_MAXSZ];
+
+struct elf_mcount_loc {
+ Elf_Ehdr *ehdr;
+ Elf_Shdr *init_data_sec;
+ uint64_t start_mcount_loc;
+ uint64_t stop_mcount_loc;
+};
+
+/* Fill the array with the content of the relocs */
+static int fill_relocs(void *ptr, uint64_t size, Elf_Ehdr *ehdr, uint64_t start_loc)
+{
+ Elf_Shdr *shdr_start;
+ Elf_Rela *rel;
+ unsigned int shnum;
+ unsigned int count = 0;
+ int shentsize;
+ void *array_end = ptr + size;
+
+ shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr));
+ shentsize = ehdr_shentsize(ehdr);
+
+ shnum = ehdr_shnum(ehdr);
+ if (shnum == SHN_UNDEF)
+ shnum = shdr_size(shdr_start);
+
+ for (int i = 0; i < shnum; i++) {
+ Elf_Shdr *shdr = get_index(shdr_start, shentsize, i);
+ void *end;
+
+ if (shdr_type(shdr) != SHT_RELA)
+ continue;
+
+ rel = (void *)ehdr + shdr_offset(shdr);
+ end = (void *)rel + shdr_size(shdr);
+
+ for (; (void *)rel < end; rel = (void *)rel + shdr_entsize(shdr)) {
+ uint64_t offset = rela_offset(rel);
+
+ if (offset >= start_loc && offset < start_loc + size) {
+ if (ptr + long_size > array_end) {
+ snprintf(m_err, ERRSTR_MAXSZ,
+ "Too many relocations");
+ return -1;
+ }
+
+ /* Make sure this has the correct type */
+ if (rela_info(rel) != rela_type) {
+ snprintf(m_err, ERRSTR_MAXSZ,
+ "rela has type %lx but expected %lx\n",
+ (long)rela_info(rel), rela_type);
+ return -1;
+ }
+
+ if (long_size == 4)
+ *(uint32_t *)ptr = rela_addend(rel);
+ else
+ *(uint64_t *)ptr = rela_addend(rel);
+ ptr += long_size;
+ count++;
+ }
+ }
+ }
+ return count;
+}
+
+/* Put the sorted vals back into the relocation elements */
+static void replace_relocs(void *ptr, uint64_t size, Elf_Ehdr *ehdr, uint64_t start_loc)
+{
+ Elf_Shdr *shdr_start;
+ Elf_Rela *rel;
+ unsigned int shnum;
+ int shentsize;
+
+ shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr));
+ shentsize = ehdr_shentsize(ehdr);
+
+ shnum = ehdr_shnum(ehdr);
+ if (shnum == SHN_UNDEF)
+ shnum = shdr_size(shdr_start);
+
+ for (int i = 0; i < shnum; i++) {
+ Elf_Shdr *shdr = get_index(shdr_start, shentsize, i);
+ void *end;
+
+ if (shdr_type(shdr) != SHT_RELA)
+ continue;
+
+ rel = (void *)ehdr + shdr_offset(shdr);
+ end = (void *)rel + shdr_size(shdr);
+
+ for (; (void *)rel < end; rel = (void *)rel + shdr_entsize(shdr)) {
+ uint64_t offset = rela_offset(rel);
+
+ if (offset >= start_loc && offset < start_loc + size) {
+ if (long_size == 4)
+ rela_write_addend(rel, *(uint32_t *)ptr);
+ else
+ rela_write_addend(rel, *(uint64_t *)ptr);
+ ptr += long_size;
+ }
+ }
+ }
+}
+
+static int fill_addrs(void *ptr, uint64_t size, void *addrs)
+{
+ void *end = ptr + size;
+ int count = 0;
+
+ for (; ptr < end; ptr += long_size, addrs += long_size, count++) {
+ if (long_size == 4)
+ *(uint32_t *)ptr = r(addrs);
+ else
+ *(uint64_t *)ptr = r8(addrs);
+ }
+ return count;
+}
+
+static void replace_addrs(void *ptr, uint64_t size, void *addrs)
+{
+ void *end = ptr + size;
+
+ for (; ptr < end; ptr += long_size, addrs += long_size) {
+ if (long_size == 4)
+ w(*(uint32_t *)ptr, addrs);
+ else
+ w8(*(uint64_t *)ptr, addrs);
+ }
+}
+
+/* Sort the addresses stored between __start_mcount_loc to __stop_mcount_loc in vmlinux */
+static void *sort_mcount_loc(void *arg)
+{
+ struct elf_mcount_loc *emloc = (struct elf_mcount_loc *)arg;
+ uint64_t offset = emloc->start_mcount_loc - shdr_addr(emloc->init_data_sec)
+ + shdr_offset(emloc->init_data_sec);
+ uint64_t size = emloc->stop_mcount_loc - emloc->start_mcount_loc;
+ unsigned char *start_loc = (void *)emloc->ehdr + offset;
+ Elf_Ehdr *ehdr = emloc->ehdr;
+ void *e_msg = NULL;
+ void *vals;
+ int count;
+
+ vals = malloc(long_size * size);
+ if (!vals) {
+ snprintf(m_err, ERRSTR_MAXSZ, "Failed to allocate sort array");
+ pthread_exit(m_err);
+ }
+
+ if (sort_reloc) {
+ count = fill_relocs(vals, size, ehdr, emloc->start_mcount_loc);
+ /* gcc may use relocs to save the addresses, but clang does not. */
+ if (!count) {
+ count = fill_addrs(vals, size, start_loc);
+ sort_reloc = 0;
+ }
+ } else
+ count = fill_addrs(vals, size, start_loc);
+
+ if (count < 0) {
+ e_msg = m_err;
+ goto out;
+ }
+
+ if (count != size / long_size) {
+ snprintf(m_err, ERRSTR_MAXSZ, "Expected %u mcount elements but found %u\n",
+ (int)(size / long_size), count);
+ e_msg = m_err;
+ goto out;
+ }
+
+ /* zero out any locations not found by function list */
+ if (function_list_size) {
+ for (void *ptr = vals; ptr < vals + size; ptr += long_size) {
+ uint64_t key;
+
+ key = long_size == 4 ? *(uint32_t *)ptr : *(uint64_t *)ptr;
+ if (!find_func(key)) {
+ if (long_size == 4)
+ *(uint32_t *)ptr = 0;
+ else
+ *(uint64_t *)ptr = 0;
+ }
+ }
+ }
+
+ compare_values = long_size == 4 ? compare_values_32 : compare_values_64;
+
+ qsort(vals, count, long_size, compare_values);
+
+ if (sort_reloc)
+ replace_relocs(vals, size, ehdr, emloc->start_mcount_loc);
+ else
+ replace_addrs(vals, size, start_loc);
+
+out:
+ free(vals);
+
+ pthread_exit(e_msg);
+}
+
+/* Get the address of __start_mcount_loc and __stop_mcount_loc in System.map */
+static void get_mcount_loc(struct elf_mcount_loc *emloc, Elf_Shdr *symtab_sec,
+ const char *strtab)
+{
+ Elf_Sym *sym, *end_sym;
+ int symentsize = shdr_entsize(symtab_sec);
+ int found = 0;
+
+ sym = (void *)emloc->ehdr + shdr_offset(symtab_sec);
+ end_sym = (void *)sym + shdr_size(symtab_sec);
+
+ while (sym < end_sym) {
+ if (!strcmp(strtab + sym_name(sym), "__start_mcount_loc")) {
+ emloc->start_mcount_loc = sym_value(sym);
+ if (++found == 2)
+ break;
+ } else if (!strcmp(strtab + sym_name(sym), "__stop_mcount_loc")) {
+ emloc->stop_mcount_loc = sym_value(sym);
+ if (++found == 2)
+ break;
+ }
+ sym = (void *)sym + symentsize;
+ }
+
+ if (!emloc->start_mcount_loc) {
+ fprintf(stderr, "get start_mcount_loc error!");
+ return;
+ }
+
+ if (!emloc->stop_mcount_loc) {
+ fprintf(stderr, "get stop_mcount_loc error!");
+ return;
+ }
+}
+#else /* MCOUNT_SORT_ENABLED */
+static inline int parse_symbols(const char *fname) { return 0; }
+#endif
+
+static int do_sort(Elf_Ehdr *ehdr,
+ char const *const fname,
+ table_sort_t custom_sort)
+{
+ int rc = -1;
+ Elf_Shdr *shdr_start;
+ Elf_Shdr *strtab_sec = NULL;
+ Elf_Shdr *symtab_sec = NULL;
+ Elf_Shdr *extab_sec = NULL;
+ Elf_Shdr *string_sec;
+ Elf_Sym *sym;
+ const Elf_Sym *symtab;
+ Elf32_Word *symtab_shndx = NULL;
+ Elf_Sym *sort_needed_sym = NULL;
+ Elf_Shdr *sort_needed_sec;
+ uint32_t *sort_needed_loc;
+ void *sym_start;
+ void *sym_end;
+ const char *secstrings;
+ const char *strtab;
+ char *extab_image;
+ int sort_need_index;
+ int symentsize;
+ int shentsize;
+ int idx;
+ int i;
+ unsigned int shnum;
+ unsigned int shstrndx;
+#ifdef MCOUNT_SORT_ENABLED
+ struct elf_mcount_loc mstruct = {0};
+#endif
+#ifdef UNWINDER_ORC_ENABLED
+ unsigned int orc_ip_size = 0;
+ unsigned int orc_size = 0;
+ unsigned int orc_num_entries = 0;
+#endif
+
+ shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr));
+ shentsize = ehdr_shentsize(ehdr);
+
+ shstrndx = ehdr_shstrndx(ehdr);
+ if (shstrndx == SHN_XINDEX)
+ shstrndx = shdr_link(shdr_start);
+ string_sec = get_index(shdr_start, shentsize, shstrndx);
+ secstrings = (const char *)ehdr + shdr_offset(string_sec);
+
+ shnum = ehdr_shnum(ehdr);
+ if (shnum == SHN_UNDEF)
+ shnum = shdr_size(shdr_start);
+
+ for (i = 0; i < shnum; i++) {
+ Elf_Shdr *shdr = get_index(shdr_start, shentsize, i);
+
+ idx = shdr_name(shdr);
+ if (!strcmp(secstrings + idx, "__ex_table"))
+ extab_sec = shdr;
+ if (!strcmp(secstrings + idx, ".symtab"))
+ symtab_sec = shdr;
+ if (!strcmp(secstrings + idx, ".strtab"))
+ strtab_sec = shdr;
+
+ if (shdr_type(shdr) == SHT_SYMTAB_SHNDX)
+ symtab_shndx = (Elf32_Word *)((const char *)ehdr +
+ shdr_offset(shdr));
+
+#ifdef MCOUNT_SORT_ENABLED
+ /* locate the .init.data section in vmlinux */
+ if (!strcmp(secstrings + idx, ".init.data"))
+ mstruct.init_data_sec = shdr;
+#endif
+
+#ifdef UNWINDER_ORC_ENABLED
+ /* locate the ORC unwind tables */
+ if (!strcmp(secstrings + idx, ".orc_unwind_ip")) {
+ orc_ip_size = shdr_size(shdr);
+ g_orc_ip_table = (int *)((void *)ehdr +
+ shdr_offset(shdr));
+ }
+ if (!strcmp(secstrings + idx, ".orc_unwind")) {
+ orc_size = shdr_size(shdr);
+ g_orc_table = (struct orc_entry *)((void *)ehdr +
+ shdr_offset(shdr));
+ }
+#endif
+ } /* for loop */
+
+#ifdef UNWINDER_ORC_ENABLED
+ if (!g_orc_ip_table || !g_orc_table) {
+ fprintf(stderr,
+ "incomplete ORC unwind tables in file: %s\n", fname);
+ goto out;
+ }
+
+ orc_num_entries = orc_ip_size / sizeof(int);
+ if (orc_ip_size % sizeof(int) != 0 ||
+ orc_size % sizeof(struct orc_entry) != 0 ||
+ orc_num_entries != orc_size / sizeof(struct orc_entry)) {
+ fprintf(stderr,
+ "inconsistent ORC unwind table entries in file: %s\n",
+ fname);
+ goto out;
+ }
+
+ /* create thread to sort ORC unwind tables concurrently */
+ if (pthread_create(&orc_sort_thread, NULL,
+ sort_orctable, &orc_ip_size)) {
+ fprintf(stderr,
+ "pthread_create orc_sort_thread failed '%s': %s\n",
+ strerror(errno), fname);
+ goto out;
+ }
+#endif
+ if (!extab_sec) {
+ fprintf(stderr, "no __ex_table in file: %s\n", fname);
+ goto out;
+ }
+
+ if (!symtab_sec) {
+ fprintf(stderr, "no .symtab in file: %s\n", fname);
+ goto out;
+ }
+
+ if (!strtab_sec) {
+ fprintf(stderr, "no .strtab in file: %s\n", fname);
+ goto out;
+ }
+
+ extab_image = (void *)ehdr + shdr_offset(extab_sec);
+ strtab = (const char *)ehdr + shdr_offset(strtab_sec);
+ symtab = (const Elf_Sym *)((const char *)ehdr + shdr_offset(symtab_sec));
+
+#ifdef MCOUNT_SORT_ENABLED
+ mstruct.ehdr = ehdr;
+ get_mcount_loc(&mstruct, symtab_sec, strtab);
+
+ if (!mstruct.init_data_sec || !mstruct.start_mcount_loc || !mstruct.stop_mcount_loc) {
+ fprintf(stderr,
+ "incomplete mcount's sort in file: %s\n",
+ fname);
+ goto out;
+ }
+
+ /* create thread to sort mcount_loc concurrently */
+ if (pthread_create(&mcount_sort_thread, NULL, &sort_mcount_loc, &mstruct)) {
+ fprintf(stderr,
+ "pthread_create mcount_sort_thread failed '%s': %s\n",
+ strerror(errno), fname);
+ goto out;
+ }
+#endif
+
+ if (custom_sort) {
+ custom_sort(extab_image, shdr_size(extab_sec));
+ } else {
+ int num_entries = shdr_size(extab_sec) / extable_ent_size;
+ qsort(extab_image, num_entries,
+ extable_ent_size, compare_extable);
+ }
+
+ /* find the flag main_extable_sort_needed */
+ sym_start = (void *)ehdr + shdr_offset(symtab_sec);
+ sym_end = sym_start + shdr_size(symtab_sec);
+ symentsize = shdr_entsize(symtab_sec);
+
+ for (sym = sym_start; (void *)sym + symentsize < sym_end;
+ sym = (void *)sym + symentsize) {
+ if (sym_type(sym) != STT_OBJECT)
+ continue;
+ if (!strcmp(strtab + sym_name(sym),
+ "main_extable_sort_needed")) {
+ sort_needed_sym = sym;
+ break;
+ }
+ }
+
+ if (!sort_needed_sym) {
+ fprintf(stderr,
+ "no main_extable_sort_needed symbol in file: %s\n",
+ fname);
+ goto out;
+ }
+
+ sort_need_index = get_secindex(sym_shndx(sym),
+ ((void *)sort_needed_sym - (void *)symtab) / symentsize,
+ symtab_shndx);
+ sort_needed_sec = get_index(shdr_start, shentsize, sort_need_index);
+ sort_needed_loc = (void *)ehdr +
+ shdr_offset(sort_needed_sec) +
+ sym_value(sort_needed_sym) - shdr_addr(sort_needed_sec);
+
+ /* extable has been sorted, clear the flag */
+ w(0, sort_needed_loc);
+ rc = 0;
+
+out:
+#ifdef UNWINDER_ORC_ENABLED
+ if (orc_sort_thread) {
+ void *retval = NULL;
+ /* wait for ORC tables sort done */
+ rc = pthread_join(orc_sort_thread, &retval);
+ if (rc) {
+ fprintf(stderr,
+ "pthread_join failed '%s': %s\n",
+ strerror(errno), fname);
+ } else if (retval) {
+ rc = -1;
+ fprintf(stderr,
+ "failed to sort ORC tables '%s': %s\n",
+ (char *)retval, fname);
+ }
+ }
+#endif
+
+#ifdef MCOUNT_SORT_ENABLED
+ if (mcount_sort_thread) {
+ void *retval = NULL;
+ /* wait for mcount sort done */
+ rc = pthread_join(mcount_sort_thread, &retval);
+ if (rc) {
+ fprintf(stderr,
+ "pthread_join failed '%s': %s\n",
+ strerror(errno), fname);
+ } else if (retval) {
+ rc = -1;
+ fprintf(stderr,
+ "failed to sort mcount '%s': %s\n",
+ (char *)retval, fname);
+ }
+ }
+#endif
+ return rc;
+}
static int compare_relative_table(const void *a, const void *b)
{
@@ -267,17 +1220,15 @@ static void sort_relative_table_with_data(char *extab_image, int image_size)
static int do_file(char const *const fname, void *addr)
{
- int rc = -1;
- Elf32_Ehdr *ehdr = addr;
+ Elf_Ehdr *ehdr = addr;
table_sort_t custom_sort = NULL;
- switch (ehdr->e_ident[EI_DATA]) {
+ switch (ehdr->e32.e_ident[EI_DATA]) {
case ELFDATA2LSB:
r = rle;
r2 = r2le;
r8 = r8le;
w = wle;
- w2 = w2le;
w8 = w8le;
break;
case ELFDATA2MSB:
@@ -285,25 +1236,31 @@ static int do_file(char const *const fname, void *addr)
r2 = r2be;
r8 = r8be;
w = wbe;
- w2 = w2be;
w8 = w8be;
break;
default:
fprintf(stderr, "unrecognized ELF data encoding %d: %s\n",
- ehdr->e_ident[EI_DATA], fname);
+ ehdr->e32.e_ident[EI_DATA], fname);
return -1;
}
- if (memcmp(ELFMAG, ehdr->e_ident, SELFMAG) != 0 ||
- (r2(&ehdr->e_type) != ET_EXEC && r2(&ehdr->e_type) != ET_DYN) ||
- ehdr->e_ident[EI_VERSION] != EV_CURRENT) {
+ if (memcmp(ELFMAG, ehdr->e32.e_ident, SELFMAG) != 0 ||
+ (r2(&ehdr->e32.e_type) != ET_EXEC && r2(&ehdr->e32.e_type) != ET_DYN) ||
+ ehdr->e32.e_ident[EI_VERSION] != EV_CURRENT) {
fprintf(stderr, "unrecognized ET_EXEC/ET_DYN file %s\n", fname);
return -1;
}
- switch (r2(&ehdr->e_machine)) {
- case EM_386:
+ switch (r2(&ehdr->e32.e_machine)) {
case EM_AARCH64:
+#ifdef MCOUNT_SORT_ENABLED
+ sort_reloc = true;
+ rela_type = 0x403;
+ /* arm64 uses patchable function entry placing before function */
+ before_func = 8;
+#endif
+ /* fallthrough */
+ case EM_386:
case EM_LOONGARCH:
case EM_RISCV:
case EM_S390:
@@ -324,40 +1281,93 @@ static int do_file(char const *const fname, void *addr)
break;
default:
fprintf(stderr, "unrecognized e_machine %d %s\n",
- r2(&ehdr->e_machine), fname);
+ r2(&ehdr->e32.e_machine), fname);
return -1;
}
- switch (ehdr->e_ident[EI_CLASS]) {
- case ELFCLASS32:
- if (r2(&ehdr->e_ehsize) != sizeof(Elf32_Ehdr) ||
- r2(&ehdr->e_shentsize) != sizeof(Elf32_Shdr)) {
+ switch (ehdr->e32.e_ident[EI_CLASS]) {
+ case ELFCLASS32: {
+ struct elf_funcs efuncs = {
+ .compare_extable = compare_extable_32,
+ .ehdr_shoff = ehdr32_shoff,
+ .ehdr_shentsize = ehdr32_shentsize,
+ .ehdr_shstrndx = ehdr32_shstrndx,
+ .ehdr_shnum = ehdr32_shnum,
+ .shdr_addr = shdr32_addr,
+ .shdr_offset = shdr32_offset,
+ .shdr_link = shdr32_link,
+ .shdr_size = shdr32_size,
+ .shdr_name = shdr32_name,
+ .shdr_type = shdr32_type,
+ .shdr_entsize = shdr32_entsize,
+ .sym_type = sym32_type,
+ .sym_name = sym32_name,
+ .sym_value = sym32_value,
+ .sym_shndx = sym32_shndx,
+ .rela_offset = rela32_offset,
+ .rela_info = rela32_info,
+ .rela_addend = rela32_addend,
+ .rela_write_addend = rela32_write_addend,
+ };
+
+ e = efuncs;
+ long_size = 4;
+ extable_ent_size = 8;
+
+ if (r2(&ehdr->e32.e_ehsize) != sizeof(Elf32_Ehdr) ||
+ r2(&ehdr->e32.e_shentsize) != sizeof(Elf32_Shdr)) {
fprintf(stderr,
"unrecognized ET_EXEC/ET_DYN file: %s\n", fname);
- break;
+ return -1;
+ }
+
}
- rc = do_sort_32(ehdr, fname, custom_sort);
break;
- case ELFCLASS64:
- {
- Elf64_Ehdr *const ghdr = (Elf64_Ehdr *)ehdr;
- if (r2(&ghdr->e_ehsize) != sizeof(Elf64_Ehdr) ||
- r2(&ghdr->e_shentsize) != sizeof(Elf64_Shdr)) {
+ case ELFCLASS64: {
+ struct elf_funcs efuncs = {
+ .compare_extable = compare_extable_64,
+ .ehdr_shoff = ehdr64_shoff,
+ .ehdr_shentsize = ehdr64_shentsize,
+ .ehdr_shstrndx = ehdr64_shstrndx,
+ .ehdr_shnum = ehdr64_shnum,
+ .shdr_addr = shdr64_addr,
+ .shdr_offset = shdr64_offset,
+ .shdr_link = shdr64_link,
+ .shdr_size = shdr64_size,
+ .shdr_name = shdr64_name,
+ .shdr_type = shdr64_type,
+ .shdr_entsize = shdr64_entsize,
+ .sym_type = sym64_type,
+ .sym_name = sym64_name,
+ .sym_value = sym64_value,
+ .sym_shndx = sym64_shndx,
+ .rela_offset = rela64_offset,
+ .rela_info = rela64_info,
+ .rela_addend = rela64_addend,
+ .rela_write_addend = rela64_write_addend,
+ };
+
+ e = efuncs;
+ long_size = 8;
+ extable_ent_size = 16;
+
+ if (r2(&ehdr->e64.e_ehsize) != sizeof(Elf64_Ehdr) ||
+ r2(&ehdr->e64.e_shentsize) != sizeof(Elf64_Shdr)) {
fprintf(stderr,
"unrecognized ET_EXEC/ET_DYN file: %s\n",
fname);
- break;
+ return -1;
}
- rc = do_sort_64(ghdr, fname, custom_sort);
+
}
break;
default:
fprintf(stderr, "unrecognized ELF class %d %s\n",
- ehdr->e_ident[EI_CLASS], fname);
- break;
+ ehdr->e32.e_ident[EI_CLASS], fname);
+ return -1;
}
- return rc;
+ return do_sort(ehdr, fname, custom_sort);
}
int main(int argc, char *argv[])
@@ -365,14 +1375,29 @@ int main(int argc, char *argv[])
int i, n_error = 0; /* gcc-4.3.0 false positive complaint */
size_t size = 0;
void *addr = NULL;
+ int c;
+
+ while ((c = getopt(argc, argv, "s:")) >= 0) {
+ switch (c) {
+ case 's':
+ if (parse_symbols(optarg) < 0) {
+ fprintf(stderr, "Could not parse %s\n", optarg);
+ return -1;
+ }
+ break;
+ default:
+ fprintf(stderr, "usage: sorttable [-s nm-file] vmlinux...\n");
+ return 0;
+ }
+ }
- if (argc < 2) {
+ if ((argc - optind) < 1) {
fprintf(stderr, "usage: sorttable vmlinux...\n");
return 0;
}
/* Process each file in turn, allowing deep failure. */
- for (i = 1; i < argc; i++) {
+ for (i = optind; i < argc; i++) {
addr = mmap_file(argv[i], &size);
if (!addr) {
++n_error;
diff --git a/scripts/sorttable.h b/scripts/sorttable.h
deleted file mode 100644
index a7c5445baf00..000000000000
--- a/scripts/sorttable.h
+++ /dev/null
@@ -1,500 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-/*
- * sorttable.h
- *
- * Added ORC unwind tables sort support and other updates:
- * Copyright (C) 1999-2019 Alibaba Group Holding Limited. by:
- * Shile Zhang <shile.zhang@linux.alibaba.com>
- *
- * Copyright 2011 - 2012 Cavium, Inc.
- *
- * Some of code was taken out of arch/x86/kernel/unwind_orc.c, written by:
- * Copyright (C) 2017 Josh Poimboeuf <jpoimboe@redhat.com>
- *
- * Some of this code was taken out of recordmcount.h written by:
- *
- * Copyright 2009 John F. Reiser <jreiser@BitWagon.com>. All rights reserved.
- * Copyright 2010 Steven Rostedt <srostedt@redhat.com>, Red Hat Inc.
- */
-
-#undef extable_ent_size
-#undef compare_extable
-#undef get_mcount_loc
-#undef sort_mcount_loc
-#undef elf_mcount_loc
-#undef do_sort
-#undef Elf_Addr
-#undef Elf_Ehdr
-#undef Elf_Shdr
-#undef Elf_Rel
-#undef Elf_Rela
-#undef Elf_Sym
-#undef ELF_R_SYM
-#undef Elf_r_sym
-#undef ELF_R_INFO
-#undef Elf_r_info
-#undef ELF_ST_BIND
-#undef ELF_ST_TYPE
-#undef fn_ELF_R_SYM
-#undef fn_ELF_R_INFO
-#undef uint_t
-#undef _r
-#undef _w
-
-#ifdef SORTTABLE_64
-# define extable_ent_size 16
-# define compare_extable compare_extable_64
-# define get_mcount_loc get_mcount_loc_64
-# define sort_mcount_loc sort_mcount_loc_64
-# define elf_mcount_loc elf_mcount_loc_64
-# define do_sort do_sort_64
-# define Elf_Addr Elf64_Addr
-# define Elf_Ehdr Elf64_Ehdr
-# define Elf_Shdr Elf64_Shdr
-# define Elf_Rel Elf64_Rel
-# define Elf_Rela Elf64_Rela
-# define Elf_Sym Elf64_Sym
-# define ELF_R_SYM ELF64_R_SYM
-# define Elf_r_sym Elf64_r_sym
-# define ELF_R_INFO ELF64_R_INFO
-# define Elf_r_info Elf64_r_info
-# define ELF_ST_BIND ELF64_ST_BIND
-# define ELF_ST_TYPE ELF64_ST_TYPE
-# define fn_ELF_R_SYM fn_ELF64_R_SYM
-# define fn_ELF_R_INFO fn_ELF64_R_INFO
-# define uint_t uint64_t
-# define _r r8
-# define _w w8
-#else
-# define extable_ent_size 8
-# define compare_extable compare_extable_32
-# define get_mcount_loc get_mcount_loc_32
-# define sort_mcount_loc sort_mcount_loc_32
-# define elf_mcount_loc elf_mcount_loc_32
-# define do_sort do_sort_32
-# define Elf_Addr Elf32_Addr
-# define Elf_Ehdr Elf32_Ehdr
-# define Elf_Shdr Elf32_Shdr
-# define Elf_Rel Elf32_Rel
-# define Elf_Rela Elf32_Rela
-# define Elf_Sym Elf32_Sym
-# define ELF_R_SYM ELF32_R_SYM
-# define Elf_r_sym Elf32_r_sym
-# define ELF_R_INFO ELF32_R_INFO
-# define Elf_r_info Elf32_r_info
-# define ELF_ST_BIND ELF32_ST_BIND
-# define ELF_ST_TYPE ELF32_ST_TYPE
-# define fn_ELF_R_SYM fn_ELF32_R_SYM
-# define fn_ELF_R_INFO fn_ELF32_R_INFO
-# define uint_t uint32_t
-# define _r r
-# define _w w
-#endif
-
-#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED)
-/* ORC unwinder only support X86_64 */
-#include <asm/orc_types.h>
-
-#define ERRSTR_MAXSZ 256
-
-char g_err[ERRSTR_MAXSZ];
-int *g_orc_ip_table;
-struct orc_entry *g_orc_table;
-
-pthread_t orc_sort_thread;
-
-static inline unsigned long orc_ip(const int *ip)
-{
- return (unsigned long)ip + *ip;
-}
-
-static int orc_sort_cmp(const void *_a, const void *_b)
-{
- struct orc_entry *orc_a, *orc_b;
- const int *a = g_orc_ip_table + *(int *)_a;
- const int *b = g_orc_ip_table + *(int *)_b;
- unsigned long a_val = orc_ip(a);
- unsigned long b_val = orc_ip(b);
-
- if (a_val > b_val)
- return 1;
- if (a_val < b_val)
- return -1;
-
- /*
- * The "weak" section terminator entries need to always be on the left
- * to ensure the lookup code skips them in favor of real entries.
- * These terminator entries exist to handle any gaps created by
- * whitelisted .o files which didn't get objtool generation.
- */
- orc_a = g_orc_table + (a - g_orc_ip_table);
- orc_b = g_orc_table + (b - g_orc_ip_table);
- if (orc_a->type == ORC_TYPE_UNDEFINED && orc_b->type == ORC_TYPE_UNDEFINED)
- return 0;
- return orc_a->type == ORC_TYPE_UNDEFINED ? -1 : 1;
-}
-
-static void *sort_orctable(void *arg)
-{
- int i;
- int *idxs = NULL;
- int *tmp_orc_ip_table = NULL;
- struct orc_entry *tmp_orc_table = NULL;
- unsigned int *orc_ip_size = (unsigned int *)arg;
- unsigned int num_entries = *orc_ip_size / sizeof(int);
- unsigned int orc_size = num_entries * sizeof(struct orc_entry);
-
- idxs = (int *)malloc(*orc_ip_size);
- if (!idxs) {
- snprintf(g_err, ERRSTR_MAXSZ, "malloc idxs: %s",
- strerror(errno));
- pthread_exit(g_err);
- }
-
- tmp_orc_ip_table = (int *)malloc(*orc_ip_size);
- if (!tmp_orc_ip_table) {
- snprintf(g_err, ERRSTR_MAXSZ, "malloc tmp_orc_ip_table: %s",
- strerror(errno));
- pthread_exit(g_err);
- }
-
- tmp_orc_table = (struct orc_entry *)malloc(orc_size);
- if (!tmp_orc_table) {
- snprintf(g_err, ERRSTR_MAXSZ, "malloc tmp_orc_table: %s",
- strerror(errno));
- pthread_exit(g_err);
- }
-
- /* initialize indices array, convert ip_table to absolute address */
- for (i = 0; i < num_entries; i++) {
- idxs[i] = i;
- tmp_orc_ip_table[i] = g_orc_ip_table[i] + i * sizeof(int);
- }
- memcpy(tmp_orc_table, g_orc_table, orc_size);
-
- qsort(idxs, num_entries, sizeof(int), orc_sort_cmp);
-
- for (i = 0; i < num_entries; i++) {
- if (idxs[i] == i)
- continue;
-
- /* convert back to relative address */
- g_orc_ip_table[i] = tmp_orc_ip_table[idxs[i]] - i * sizeof(int);
- g_orc_table[i] = tmp_orc_table[idxs[i]];
- }
-
- free(idxs);
- free(tmp_orc_ip_table);
- free(tmp_orc_table);
- pthread_exit(NULL);
-}
-#endif
-
-static int compare_extable(const void *a, const void *b)
-{
- Elf_Addr av = _r(a);
- Elf_Addr bv = _r(b);
-
- if (av < bv)
- return -1;
- if (av > bv)
- return 1;
- return 0;
-}
-#ifdef MCOUNT_SORT_ENABLED
-pthread_t mcount_sort_thread;
-
-struct elf_mcount_loc {
- Elf_Ehdr *ehdr;
- Elf_Shdr *init_data_sec;
- uint_t start_mcount_loc;
- uint_t stop_mcount_loc;
-};
-
-/* Sort the addresses stored between __start_mcount_loc to __stop_mcount_loc in vmlinux */
-static void *sort_mcount_loc(void *arg)
-{
- struct elf_mcount_loc *emloc = (struct elf_mcount_loc *)arg;
- uint_t offset = emloc->start_mcount_loc - _r(&(emloc->init_data_sec)->sh_addr)
- + _r(&(emloc->init_data_sec)->sh_offset);
- uint_t count = emloc->stop_mcount_loc - emloc->start_mcount_loc;
- unsigned char *start_loc = (void *)emloc->ehdr + offset;
-
- qsort(start_loc, count/sizeof(uint_t), sizeof(uint_t), compare_extable);
- return NULL;
-}
-
-/* Get the address of __start_mcount_loc and __stop_mcount_loc in System.map */
-static void get_mcount_loc(uint_t *_start, uint_t *_stop)
-{
- FILE *file_start, *file_stop;
- char start_buff[20];
- char stop_buff[20];
- int len = 0;
-
- file_start = popen(" grep start_mcount System.map | awk '{print $1}' ", "r");
- if (!file_start) {
- fprintf(stderr, "get start_mcount_loc error!");
- return;
- }
-
- file_stop = popen(" grep stop_mcount System.map | awk '{print $1}' ", "r");
- if (!file_stop) {
- fprintf(stderr, "get stop_mcount_loc error!");
- pclose(file_start);
- return;
- }
-
- while (fgets(start_buff, sizeof(start_buff), file_start) != NULL) {
- len = strlen(start_buff);
- start_buff[len - 1] = '\0';
- }
- *_start = strtoul(start_buff, NULL, 16);
-
- while (fgets(stop_buff, sizeof(stop_buff), file_stop) != NULL) {
- len = strlen(stop_buff);
- stop_buff[len - 1] = '\0';
- }
- *_stop = strtoul(stop_buff, NULL, 16);
-
- pclose(file_start);
- pclose(file_stop);
-}
-#endif
-static int do_sort(Elf_Ehdr *ehdr,
- char const *const fname,
- table_sort_t custom_sort)
-{
- int rc = -1;
- Elf_Shdr *s, *shdr = (Elf_Shdr *)((char *)ehdr + _r(&ehdr->e_shoff));
- Elf_Shdr *strtab_sec = NULL;
- Elf_Shdr *symtab_sec = NULL;
- Elf_Shdr *extab_sec = NULL;
- Elf_Sym *sym;
- const Elf_Sym *symtab;
- Elf32_Word *symtab_shndx = NULL;
- Elf_Sym *sort_needed_sym = NULL;
- Elf_Shdr *sort_needed_sec;
- Elf_Rel *relocs = NULL;
- int relocs_size = 0;
- uint32_t *sort_needed_loc;
- const char *secstrings;
- const char *strtab;
- char *extab_image;
- int extab_index = 0;
- int i;
- int idx;
- unsigned int shnum;
- unsigned int shstrndx;
-#ifdef MCOUNT_SORT_ENABLED
- struct elf_mcount_loc mstruct = {0};
- uint_t _start_mcount_loc = 0;
- uint_t _stop_mcount_loc = 0;
-#endif
-#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED)
- unsigned int orc_ip_size = 0;
- unsigned int orc_size = 0;
- unsigned int orc_num_entries = 0;
-#endif
-
- shstrndx = r2(&ehdr->e_shstrndx);
- if (shstrndx == SHN_XINDEX)
- shstrndx = r(&shdr[0].sh_link);
- secstrings = (const char *)ehdr + _r(&shdr[shstrndx].sh_offset);
-
- shnum = r2(&ehdr->e_shnum);
- if (shnum == SHN_UNDEF)
- shnum = _r(&shdr[0].sh_size);
-
- for (i = 0, s = shdr; s < shdr + shnum; i++, s++) {
- idx = r(&s->sh_name);
- if (!strcmp(secstrings + idx, "__ex_table")) {
- extab_sec = s;
- extab_index = i;
- }
- if (!strcmp(secstrings + idx, ".symtab"))
- symtab_sec = s;
- if (!strcmp(secstrings + idx, ".strtab"))
- strtab_sec = s;
-
- if ((r(&s->sh_type) == SHT_REL ||
- r(&s->sh_type) == SHT_RELA) &&
- r(&s->sh_info) == extab_index) {
- relocs = (void *)ehdr + _r(&s->sh_offset);
- relocs_size = _r(&s->sh_size);
- }
- if (r(&s->sh_type) == SHT_SYMTAB_SHNDX)
- symtab_shndx = (Elf32_Word *)((const char *)ehdr +
- _r(&s->sh_offset));
-
-#ifdef MCOUNT_SORT_ENABLED
- /* locate the .init.data section in vmlinux */
- if (!strcmp(secstrings + idx, ".init.data")) {
- get_mcount_loc(&_start_mcount_loc, &_stop_mcount_loc);
- mstruct.ehdr = ehdr;
- mstruct.init_data_sec = s;
- mstruct.start_mcount_loc = _start_mcount_loc;
- mstruct.stop_mcount_loc = _stop_mcount_loc;
- }
-#endif
-
-#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED)
- /* locate the ORC unwind tables */
- if (!strcmp(secstrings + idx, ".orc_unwind_ip")) {
- orc_ip_size = s->sh_size;
- g_orc_ip_table = (int *)((void *)ehdr +
- s->sh_offset);
- }
- if (!strcmp(secstrings + idx, ".orc_unwind")) {
- orc_size = s->sh_size;
- g_orc_table = (struct orc_entry *)((void *)ehdr +
- s->sh_offset);
- }
-#endif
- } /* for loop */
-
-#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED)
- if (!g_orc_ip_table || !g_orc_table) {
- fprintf(stderr,
- "incomplete ORC unwind tables in file: %s\n", fname);
- goto out;
- }
-
- orc_num_entries = orc_ip_size / sizeof(int);
- if (orc_ip_size % sizeof(int) != 0 ||
- orc_size % sizeof(struct orc_entry) != 0 ||
- orc_num_entries != orc_size / sizeof(struct orc_entry)) {
- fprintf(stderr,
- "inconsistent ORC unwind table entries in file: %s\n",
- fname);
- goto out;
- }
-
- /* create thread to sort ORC unwind tables concurrently */
- if (pthread_create(&orc_sort_thread, NULL,
- sort_orctable, &orc_ip_size)) {
- fprintf(stderr,
- "pthread_create orc_sort_thread failed '%s': %s\n",
- strerror(errno), fname);
- goto out;
- }
-#endif
-
-#ifdef MCOUNT_SORT_ENABLED
- if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
- fprintf(stderr,
- "incomplete mcount's sort in file: %s\n",
- fname);
- goto out;
- }
-
- /* create thread to sort mcount_loc concurrently */
- if (pthread_create(&mcount_sort_thread, NULL, &sort_mcount_loc, &mstruct)) {
- fprintf(stderr,
- "pthread_create mcount_sort_thread failed '%s': %s\n",
- strerror(errno), fname);
- goto out;
- }
-#endif
- if (!extab_sec) {
- fprintf(stderr, "no __ex_table in file: %s\n", fname);
- goto out;
- }
-
- if (!symtab_sec) {
- fprintf(stderr, "no .symtab in file: %s\n", fname);
- goto out;
- }
-
- if (!strtab_sec) {
- fprintf(stderr, "no .strtab in file: %s\n", fname);
- goto out;
- }
-
- extab_image = (void *)ehdr + _r(&extab_sec->sh_offset);
- strtab = (const char *)ehdr + _r(&strtab_sec->sh_offset);
- symtab = (const Elf_Sym *)((const char *)ehdr +
- _r(&symtab_sec->sh_offset));
-
- if (custom_sort) {
- custom_sort(extab_image, _r(&extab_sec->sh_size));
- } else {
- int num_entries = _r(&extab_sec->sh_size) / extable_ent_size;
- qsort(extab_image, num_entries,
- extable_ent_size, compare_extable);
- }
-
- /* If there were relocations, we no longer need them. */
- if (relocs)
- memset(relocs, 0, relocs_size);
-
- /* find the flag main_extable_sort_needed */
- for (sym = (void *)ehdr + _r(&symtab_sec->sh_offset);
- sym < sym + _r(&symtab_sec->sh_size) / sizeof(Elf_Sym);
- sym++) {
- if (ELF_ST_TYPE(sym->st_info) != STT_OBJECT)
- continue;
- if (!strcmp(strtab + r(&sym->st_name),
- "main_extable_sort_needed")) {
- sort_needed_sym = sym;
- break;
- }
- }
-
- if (!sort_needed_sym) {
- fprintf(stderr,
- "no main_extable_sort_needed symbol in file: %s\n",
- fname);
- goto out;
- }
-
- sort_needed_sec = &shdr[get_secindex(r2(&sym->st_shndx),
- sort_needed_sym - symtab,
- symtab_shndx)];
- sort_needed_loc = (void *)ehdr +
- _r(&sort_needed_sec->sh_offset) +
- _r(&sort_needed_sym->st_value) -
- _r(&sort_needed_sec->sh_addr);
-
- /* extable has been sorted, clear the flag */
- w(0, sort_needed_loc);
- rc = 0;
-
-out:
-#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED)
- if (orc_sort_thread) {
- void *retval = NULL;
- /* wait for ORC tables sort done */
- rc = pthread_join(orc_sort_thread, &retval);
- if (rc) {
- fprintf(stderr,
- "pthread_join failed '%s': %s\n",
- strerror(errno), fname);
- } else if (retval) {
- rc = -1;
- fprintf(stderr,
- "failed to sort ORC tables '%s': %s\n",
- (char *)retval, fname);
- }
- }
-#endif
-
-#ifdef MCOUNT_SORT_ENABLED
- if (mcount_sort_thread) {
- void *retval = NULL;
- /* wait for mcount sort done */
- rc = pthread_join(mcount_sort_thread, &retval);
- if (rc) {
- fprintf(stderr,
- "pthread_join failed '%s': %s\n",
- strerror(errno), fname);
- } else if (retval) {
- rc = -1;
- fprintf(stderr,
- "failed to sort mcount '%s': %s\n",
- (char *)retval, fname);
- }
- }
-#endif
- return rc;
-}
diff --git a/security/apparmor/include/policy_unpack.h b/security/apparmor/include/policy_unpack.h
index e5a95dc4da1f..b9de0fdf9ee5 100644
--- a/security/apparmor/include/policy_unpack.h
+++ b/security/apparmor/include/policy_unpack.h
@@ -163,6 +163,25 @@ aa_get_profile_loaddata(struct aa_loaddata *data)
return data;
}
+/**
+ * aa_get_profile_loaddata_not0 - get a profile reference count if not zero
+ * @data: reference to get a count on
+ *
+ * Like aa_get_profile_loaddata(), but safe to call on an entry that may
+ * be on a list (e.g. ns->rawdata_list) where the last pcount has already
+ * dropped and the deferred cleanup has not yet run.
+ *
+ * Returns: pointer to reference, or %NULL if @data is NULL or its
+ * profile refcount has already reached zero.
+ */
+static inline struct aa_loaddata *
+aa_get_profile_loaddata_not0(struct aa_loaddata *data)
+{
+ if (data && kref_get_unless_zero(&data->pcount))
+ return data;
+ return NULL;
+}
+
void __aa_loaddata_update(struct aa_loaddata *data, long revision);
bool aa_rawdata_eq(struct aa_loaddata *l, struct aa_loaddata *r);
void aa_loaddata_kref(struct kref *kref);
diff --git a/security/apparmor/lsm.c b/security/apparmor/lsm.c
index 0182b09cc8a0..89c79fd241ba 100644
--- a/security/apparmor/lsm.c
+++ b/security/apparmor/lsm.c
@@ -1207,7 +1207,21 @@ static int aa_sock_msg_perm(const char *op, u32 request, struct socket *sock,
static int apparmor_socket_sendmsg(struct socket *sock,
struct msghdr *msg, int size)
{
- return aa_sock_msg_perm(OP_SENDMSG, AA_MAY_SEND, sock, msg, size);
+ int error = aa_sock_msg_perm(OP_SENDMSG, AA_MAY_SEND, sock, msg, size);
+
+ if (error)
+ return error;
+
+ /* TCP fast open carries connect() semantics in sendmsg(); mediate
+ * the implicit connect so it cannot bypass the connect permission.
+ */
+ if ((msg->msg_flags & MSG_FASTOPEN) && msg->msg_name &&
+ (sk_is_tcp(sock->sk) ||
+ (sk_is_inet(sock->sk) && sock->sk->sk_type == SOCK_STREAM &&
+ sock->sk->sk_protocol == IPPROTO_MPTCP)))
+ error = aa_sk_perm(OP_CONNECT, AA_MAY_CONNECT, sock->sk);
+
+ return error;
}
static int apparmor_socket_recvmsg(struct socket *sock,
diff --git a/security/apparmor/policy.c b/security/apparmor/policy.c
index b5ae0314b384..9140b613c0cf 100644
--- a/security/apparmor/policy.c
+++ b/security/apparmor/policy.c
@@ -1175,8 +1175,12 @@ ssize_t aa_replace_profiles(struct aa_ns *policy_ns, struct aa_label *label,
if (aa_rawdata_eq(rawdata_ent, udata)) {
struct aa_loaddata *tmp;
- tmp = aa_get_profile_loaddata(rawdata_ent);
- /* check we didn't fail the race */
+ /*
+ * Entries remain on rawdata_list with
+ * pcount == 0 until do_ploaddata_rmfs()
+ * runs; only take a live profile ref.
+ */
+ tmp = aa_get_profile_loaddata_not0(rawdata_ent);
if (tmp) {
aa_put_profile_loaddata(udata);
udata = tmp;
diff --git a/security/keys/internal.h b/security/keys/internal.h
index 2cffa6dc8255..b7b622bc36a1 100644
--- a/security/keys/internal.h
+++ b/security/keys/internal.h
@@ -208,6 +208,8 @@ extern struct key *request_key_auth_new(struct key *target,
const void *callout_info,
size_t callout_len,
struct key *dest_keyring);
+struct request_key_auth *request_key_auth_get(struct key *authkey);
+void request_key_auth_put(struct request_key_auth *rka);
extern struct key *key_get_instantiation_authkey(key_serial_t target_id);
diff --git a/security/keys/keyctl.c b/security/keys/keyctl.c
index ab927a142f51..11bedca4e1c6 100644
--- a/security/keys/keyctl.c
+++ b/security/keys/keyctl.c
@@ -1197,9 +1197,13 @@ static long keyctl_instantiate_key_common(key_serial_t id,
if (!instkey)
goto error;
- rka = instkey->payload.data[0];
- if (rka->target_key->serial != id)
+ rka = request_key_auth_get(instkey);
+ if (!rka) {
+ ret = -EKEYREVOKED;
goto error;
+ }
+ if (rka->target_key->serial != id)
+ goto error_put_rka;
/* pull the payload in if one was supplied */
payload = NULL;
@@ -1208,7 +1212,7 @@ static long keyctl_instantiate_key_common(key_serial_t id,
ret = -ENOMEM;
payload = kvmalloc(plen, GFP_KERNEL);
if (!payload)
- goto error;
+ goto error_put_rka;
ret = -EFAULT;
if (!copy_from_iter_full(payload, plen, from))
@@ -1234,6 +1238,8 @@ static long keyctl_instantiate_key_common(key_serial_t id,
error2:
kvfree_sensitive(payload, plen);
+error_put_rka:
+ request_key_auth_put(rka);
error:
return ret;
}
@@ -1358,15 +1364,19 @@ long keyctl_reject_key(key_serial_t id, unsigned timeout, unsigned error,
if (!instkey)
goto error;
- rka = instkey->payload.data[0];
- if (rka->target_key->serial != id)
+ rka = request_key_auth_get(instkey);
+ if (!rka) {
+ ret = -EKEYREVOKED;
goto error;
+ }
+ if (rka->target_key->serial != id)
+ goto error_put_rka;
/* find the destination keyring if present (which must also be
* writable) */
ret = get_instantiation_keyring(ringid, rka, &dest_keyring);
if (ret < 0)
- goto error;
+ goto error_put_rka;
/* instantiate the key and link it into a keyring */
ret = key_reject_and_link(rka->target_key, timeout, error,
@@ -1379,6 +1389,8 @@ long keyctl_reject_key(key_serial_t id, unsigned timeout, unsigned error,
if (ret == 0)
keyctl_change_reqkey_auth(NULL);
+error_put_rka:
+ request_key_auth_put(rka);
error:
return ret;
}
diff --git a/security/keys/keyctl_pkey.c b/security/keys/keyctl_pkey.c
index 97bc27bbf079..ba150ee2d4a3 100644
--- a/security/keys/keyctl_pkey.c
+++ b/security/keys/keyctl_pkey.c
@@ -138,28 +138,35 @@ static int keyctl_pkey_params_get_2(const struct keyctl_pkey_params __user *_par
if (uparams.in_len > info.max_dec_size ||
uparams.out_len > info.max_enc_size)
return -EINVAL;
+
+ params->out_len = info.max_enc_size;
break;
case KEYCTL_PKEY_DECRYPT:
if (uparams.in_len > info.max_enc_size ||
uparams.out_len > info.max_dec_size)
return -EINVAL;
+
+ params->out_len = info.max_dec_size;
break;
case KEYCTL_PKEY_SIGN:
if (uparams.in_len > info.max_data_size ||
uparams.out_len > info.max_sig_size)
return -EINVAL;
+
+ params->out_len = info.max_sig_size;
break;
case KEYCTL_PKEY_VERIFY:
if (uparams.in_len > info.max_data_size ||
uparams.in2_len > info.max_sig_size)
return -EINVAL;
+
+ params->out_len = info.max_sig_size;
break;
default:
BUG();
}
params->in_len = uparams.in_len;
- params->out_len = uparams.out_len; /* Note: same as in2_len */
return 0;
}
diff --git a/security/keys/request_key_auth.c b/security/keys/request_key_auth.c
index 8f33cd170e42..bd34317b58cc 100644
--- a/security/keys/request_key_auth.c
+++ b/security/keys/request_key_auth.c
@@ -23,6 +23,7 @@ static void request_key_auth_describe(const struct key *, struct seq_file *);
static void request_key_auth_revoke(struct key *);
static void request_key_auth_destroy(struct key *);
static long request_key_auth_read(const struct key *, char *, size_t);
+static void request_key_auth_rcu_disposal(struct rcu_head *);
/*
* The request-key authorisation key type definition.
@@ -115,6 +116,31 @@ static void free_request_key_auth(struct request_key_auth *rka)
kfree(rka);
}
+/*
+ * Take a reference to the request-key authorisation payload so callers can
+ * drop authkey->sem before doing operations that may sleep.
+ */
+struct request_key_auth *request_key_auth_get(struct key *authkey)
+{
+ struct request_key_auth *rka;
+
+ down_read(&authkey->sem);
+ rka = dereference_key_locked(authkey);
+ if (rka && !test_bit(KEY_FLAG_REVOKED, &authkey->flags))
+ refcount_inc(&rka->usage);
+ else
+ rka = NULL;
+ up_read(&authkey->sem);
+
+ return rka;
+}
+
+void request_key_auth_put(struct request_key_auth *rka)
+{
+ if (rka && refcount_dec_and_test(&rka->usage))
+ call_rcu(&rka->rcu, request_key_auth_rcu_disposal);
+}
+
/*
* Dispose of the request_key_auth record under RCU conditions
*/
@@ -136,8 +162,10 @@ static void request_key_auth_revoke(struct key *key)
struct request_key_auth *rka = dereference_key_locked(key);
kenter("{%d}", key->serial);
+ if (!rka)
+ return;
rcu_assign_keypointer(key, NULL);
- call_rcu(&rka->rcu, request_key_auth_rcu_disposal);
+ request_key_auth_put(rka);
}
/*
@@ -150,7 +178,7 @@ static void request_key_auth_destroy(struct key *key)
kenter("{%d}", key->serial);
if (rka) {
rcu_assign_keypointer(key, NULL);
- call_rcu(&rka->rcu, request_key_auth_rcu_disposal);
+ request_key_auth_put(rka);
}
}
@@ -174,6 +202,7 @@ struct key *request_key_auth_new(struct key *target, const char *op,
rka = kzalloc(sizeof(*rka), GFP_KERNEL);
if (!rka)
goto error;
+ refcount_set(&rka->usage, 1);
rka->callout_info = kmemdup(callout_info, callout_len, GFP_KERNEL);
if (!rka->callout_info)
goto error_free_rka;
diff --git a/security/security.c b/security/security.c
index 6e4deac6ec07..dd6b922c12de 100644
--- a/security/security.c
+++ b/security/security.c
@@ -95,6 +95,7 @@ const char *const lockdown_reasons[LOCKDOWN_CONFIDENTIALITY_MAX + 1] = {
static BLOCKING_NOTIFIER_HEAD(blocking_lsm_notifier_chain);
static struct kmem_cache *lsm_file_cache;
+static struct kmem_cache *lsm_backing_file_cache;
static struct kmem_cache *lsm_inode_cache;
char *lsm_names;
@@ -266,6 +267,7 @@ static void __init lsm_set_blob_sizes(struct lsm_blob_sizes *needed)
lsm_set_blob_size(&needed->lbs_cred, &blob_sizes.lbs_cred);
lsm_set_blob_size(&needed->lbs_file, &blob_sizes.lbs_file);
+ lsm_set_blob_size(&needed->lbs_backing_file, &blob_sizes.lbs_backing_file);
lsm_set_blob_size(&needed->lbs_ib, &blob_sizes.lbs_ib);
/*
* The inode blob gets an rcu_head in addition to
@@ -468,6 +470,7 @@ static void __init ordered_lsm_init(void)
init_debug("cred blob size = %d\n", blob_sizes.lbs_cred);
init_debug("file blob size = %d\n", blob_sizes.lbs_file);
+ init_debug("lsm_backing_file_cache = %d\n", blob_sizes.lbs_backing_file);
init_debug("ib blob size = %d\n", blob_sizes.lbs_ib);
init_debug("inode blob size = %d\n", blob_sizes.lbs_inode);
init_debug("ipc blob size = %d\n", blob_sizes.lbs_ipc);
@@ -490,6 +493,11 @@ static void __init ordered_lsm_init(void)
lsm_file_cache = kmem_cache_create("lsm_file_cache",
blob_sizes.lbs_file, 0,
SLAB_PANIC, NULL);
+ if (blob_sizes.lbs_backing_file)
+ lsm_backing_file_cache = kmem_cache_create(
+ "lsm_backing_file_cache",
+ blob_sizes.lbs_backing_file,
+ 0, SLAB_PANIC, NULL);
if (blob_sizes.lbs_inode)
lsm_inode_cache = kmem_cache_create("lsm_inode_cache",
blob_sizes.lbs_inode, 0,
@@ -666,6 +674,30 @@ int unregister_blocking_lsm_notifier(struct notifier_block *nb)
}
EXPORT_SYMBOL(unregister_blocking_lsm_notifier);
+/**
+ * lsm_backing_file_alloc - allocate a composite backing file blob
+ * @backing_file: the backing file
+ *
+ * Allocate the backing file blob for all the modules.
+ *
+ * Returns 0, or -ENOMEM if memory can't be allocated.
+ */
+static int lsm_backing_file_alloc(struct file *backing_file)
+{
+ void *blob;
+
+ if (!lsm_backing_file_cache) {
+ backing_file_set_security(backing_file, NULL);
+ return 0;
+ }
+
+ blob = kmem_cache_zalloc(lsm_backing_file_cache, GFP_KERNEL);
+ backing_file_set_security(backing_file, blob);
+ if (!blob)
+ return -ENOMEM;
+ return 0;
+}
+
/**
* lsm_blob_alloc - allocate a composite blob
* @dest: the destination for the blob
@@ -2893,6 +2925,57 @@ void security_file_free(struct file *file)
}
}
+/**
+ * security_backing_file_alloc() - Allocate and setup a backing file blob
+ * @backing_file: the backing file
+ * @user_file: the associated user visible file
+ *
+ * Allocate a backing file LSM blob and perform any necessary initialization of
+ * the LSM blob. There will be some operations where the LSM will not have
+ * access to @user_file after this point, so any important state associated
+ * with @user_file that is important to the LSM should be captured in the
+ * backing file's LSM blob.
+ *
+ * LSM's should avoid taking a reference to @user_file in this hook as it will
+ * result in problems later when the system attempts to drop/put the file
+ * references due to a circular dependency.
+ *
+ * Return: Return 0 if the hook is successful, negative values otherwise.
+ */
+int security_backing_file_alloc(struct file *backing_file,
+ const struct file *user_file)
+{
+ int rc;
+
+ rc = lsm_backing_file_alloc(backing_file);
+ if (rc)
+ return rc;
+ rc = call_int_hook(backing_file_alloc, backing_file, user_file);
+ if (unlikely(rc))
+ security_backing_file_free(backing_file);
+
+ return rc;
+}
+
+/**
+ * security_backing_file_free() - Free a backing file blob
+ * @backing_file: the backing file
+ *
+ * Free any LSM state associate with a backing file's LSM blob, including the
+ * blob itself.
+ */
+void security_backing_file_free(struct file *backing_file)
+{
+ void *blob = backing_file_security(backing_file);
+
+ call_void_hook(backing_file_free, backing_file);
+
+ if (blob) {
+ backing_file_set_security(backing_file, NULL);
+ kmem_cache_free(lsm_backing_file_cache, blob);
+ }
+}
+
/**
* security_file_ioctl() - Check if an ioctl is allowed
* @file: associated file
@@ -2981,6 +3064,32 @@ int security_mmap_file(struct file *file, unsigned long prot,
flags);
}
+/**
+ * security_mmap_backing_file - Check if mmap'ing a backing file is allowed
+ * @vma: the vm_area_struct for the mmap'd region
+ * @backing_file: the backing file being mmap'd
+ * @user_file: the user file being mmap'd
+ *
+ * Check permissions for a mmap operation on a stacked filesystem. This hook
+ * is called after the security_mmap_file() and is responsible for authorizing
+ * the mmap on @backing_file. It is important to note that the mmap operation
+ * on @user_file has already been authorized and the @vma->vm_file has been
+ * set to @backing_file.
+ *
+ * Return: Returns 0 if permission is granted.
+ */
+int security_mmap_backing_file(struct vm_area_struct *vma,
+ struct file *backing_file,
+ struct file *user_file)
+{
+ /* recommended by the stackable filesystem devs */
+ if (WARN_ON_ONCE(!(backing_file->f_mode & FMODE_BACKING)))
+ return -EIO;
+
+ return call_int_hook(mmap_backing_file, vma, backing_file, user_file);
+}
+EXPORT_SYMBOL_GPL(security_mmap_backing_file);
+
/**
* security_mmap_addr() - Check if mmap'ing an address is allowed
* @addr: address
diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c
index 8e31d3b60fc6..1b89c8d5fa2f 100644
--- a/security/selinux/hooks.c
+++ b/security/selinux/hooks.c
@@ -1724,49 +1724,72 @@ static inline int file_path_has_perm(const struct cred *cred,
static int bpf_fd_pass(const struct file *file, u32 sid);
#endif
-/* Check whether a task can use an open file descriptor to
- access an inode in a given way. Check access to the
- descriptor itself, and then use dentry_has_perm to
- check a particular permission to the file.
- Access to the descriptor is implicitly granted if it
- has the same SID as the process. If av is zero, then
- access to the file is not checked, e.g. for cases
- where only the descriptor is affected like seek. */
-static int file_has_perm(const struct cred *cred,
- struct file *file,
- u32 av)
+static int __file_has_perm(const struct cred *cred, const struct file *file,
+ u32 av, bool bf_user_file)
+
{
- struct file_security_struct *fsec = selinux_file(file);
- struct inode *inode = file_inode(file);
struct common_audit_data ad;
- u32 sid = cred_sid(cred);
+ struct inode *inode;
+ u32 ssid = cred_sid(cred);
+ u32 tsid_fd;
int rc;
- ad.type = LSM_AUDIT_DATA_FILE;
- ad.u.file = file;
+ if (bf_user_file) {
+ struct backing_file_security_struct *bfsec;
+ const struct path *path;
- if (sid != fsec->sid) {
- rc = avc_has_perm(sid, fsec->sid,
- SECCLASS_FD,
- FD__USE,
- &ad);
+ if (WARN_ON(!(file->f_mode & FMODE_BACKING)))
+ return -EIO;
+
+ bfsec = selinux_backing_file(file);
+ path = backing_file_user_path(file);
+ tsid_fd = bfsec->uf_sid;
+ inode = d_inode(path->dentry);
+
+ ad.type = LSM_AUDIT_DATA_PATH;
+ ad.u.path = *path;
+ } else {
+ struct file_security_struct *fsec = selinux_file(file);
+
+ tsid_fd = fsec->sid;
+ inode = file_inode(file);
+
+ ad.type = LSM_AUDIT_DATA_FILE;
+ ad.u.file = file;
+ }
+
+ if (ssid != tsid_fd) {
+ rc = avc_has_perm(ssid, tsid_fd, SECCLASS_FD, FD__USE, &ad);
if (rc)
- goto out;
+ return rc;
}
#ifdef CONFIG_BPF_SYSCALL
- rc = bpf_fd_pass(file, cred_sid(cred));
+ /* regardless of backing vs user file, use the underlying file here */
+ rc = bpf_fd_pass(file, ssid);
if (rc)
return rc;
#endif
/* av is zero if only checking access to the descriptor. */
- rc = 0;
if (av)
- rc = inode_has_perm(cred, inode, av, &ad);
+ return inode_has_perm(cred, inode, av, &ad);
-out:
- return rc;
+ return 0;
+}
+
+/* Check whether a task can use an open file descriptor to
+ access an inode in a given way. Check access to the
+ descriptor itself, and then use dentry_has_perm to
+ check a particular permission to the file.
+ Access to the descriptor is implicitly granted if it
+ has the same SID as the process. If av is zero, then
+ access to the file is not checked, e.g. for cases
+ where only the descriptor is affected like seek. */
+static inline int file_has_perm(const struct cred *cred,
+ const struct file *file, u32 av)
+{
+ return __file_has_perm(cred, file, av, false);
}
/*
@@ -3653,6 +3676,17 @@ static int selinux_file_alloc_security(struct file *file)
return 0;
}
+static int selinux_backing_file_alloc(struct file *backing_file,
+ const struct file *user_file)
+{
+ struct backing_file_security_struct *bfsec;
+
+ bfsec = selinux_backing_file(backing_file);
+ bfsec->uf_sid = selinux_file(user_file)->sid;
+
+ return 0;
+}
+
/*
* Check whether a task has the ioctl permission and cmd
* operation to an inode.
@@ -3770,42 +3804,55 @@ static int selinux_file_ioctl_compat(struct file *file, unsigned int cmd,
static int default_noexec __ro_after_init;
-static int file_map_prot_check(struct file *file, unsigned long prot, int shared)
+static int __file_map_prot_check(const struct cred *cred,
+ const struct file *file, unsigned long prot,
+ bool shared, bool bf_user_file)
{
- const struct cred *cred = current_cred();
- u32 sid = cred_sid(cred);
- int rc = 0;
+ struct inode *inode = NULL;
+ bool prot_exec = prot & PROT_EXEC;
+ bool prot_write = prot & PROT_WRITE;
+
+ if (file) {
+ if (bf_user_file)
+ inode = d_inode(backing_file_user_path(file)->dentry);
+ else
+ inode = file_inode(file);
+ }
+
+ if (default_noexec && prot_exec &&
+ (!file || IS_PRIVATE(inode) || (!shared && prot_write))) {
+ int rc;
+ u32 sid = cred_sid(cred);
- if (default_noexec &&
- (prot & PROT_EXEC) && (!file || IS_PRIVATE(file_inode(file)) ||
- (!shared && (prot & PROT_WRITE)))) {
/*
- * We are making executable an anonymous mapping or a
- * private file mapping that will also be writable.
- * This has an additional check.
+ * We are making executable an anonymous mapping or a private
+ * file mapping that will also be writable.
*/
- rc = avc_has_perm(sid, sid, SECCLASS_PROCESS,
- PROCESS__EXECMEM, NULL);
+ rc = avc_has_perm(sid, sid, SECCLASS_PROCESS, PROCESS__EXECMEM,
+ NULL);
if (rc)
- goto error;
+ return rc;
}
if (file) {
- /* read access is always possible with a mapping */
+ /* "read" always possible, "write" only if shared */
u32 av = FILE__READ;
-
- /* write access only matters if the mapping is shared */
- if (shared && (prot & PROT_WRITE))
+ if (shared && prot_write)
av |= FILE__WRITE;
-
- if (prot & PROT_EXEC)
+ if (prot_exec)
av |= FILE__EXECUTE;
- return file_has_perm(cred, file, av);
+ return __file_has_perm(cred, file, av, bf_user_file);
}
-error:
- return rc;
+ return 0;
+}
+
+static inline int file_map_prot_check(const struct cred *cred,
+ const struct file *file,
+ unsigned long prot, bool shared)
+{
+ return __file_map_prot_check(cred, file, prot, shared, false);
}
static int selinux_mmap_addr(unsigned long addr)
@@ -3821,36 +3868,80 @@ static int selinux_mmap_addr(unsigned long addr)
return rc;
}
-static int selinux_mmap_file(struct file *file,
- unsigned long reqprot __always_unused,
- unsigned long prot, unsigned long flags)
+static int selinux_mmap_file_common(const struct cred *cred, struct file *file,
+ unsigned long prot, bool shared)
{
- struct common_audit_data ad;
- int rc;
-
if (file) {
+ int rc;
+ struct common_audit_data ad;
+
ad.type = LSM_AUDIT_DATA_FILE;
ad.u.file = file;
- rc = inode_has_perm(current_cred(), file_inode(file),
- FILE__MAP, &ad);
+ rc = inode_has_perm(cred, file_inode(file), FILE__MAP, &ad);
if (rc)
return rc;
}
- return file_map_prot_check(file, prot,
- (flags & MAP_TYPE) == MAP_SHARED);
+ return file_map_prot_check(cred, file, prot, shared);
+}
+
+static int selinux_mmap_file(struct file *file,
+ unsigned long reqprot __always_unused,
+ unsigned long prot, unsigned long flags)
+{
+ return selinux_mmap_file_common(current_cred(), file, prot,
+ (flags & MAP_TYPE) == MAP_SHARED);
+}
+
+/**
+ * selinux_mmap_backing_file - Check mmap permissions on a backing file
+ * @vma: memory region
+ * @backing_file: stacked filesystem backing file
+ * @user_file: user visible file
+ *
+ * This is called after selinux_mmap_file() on stacked filesystems, and it
+ * is this function's responsibility to verify access to @backing_file and
+ * setup the SELinux state for possible later use in the mprotect() code path.
+ *
+ * By the time this function is called, mmap() access to @user_file has already
+ * been authorized and @vma->vm_file has been set to point to @backing_file.
+ *
+ * Return zero on success, negative values otherwise.
+ */
+static int selinux_mmap_backing_file(struct vm_area_struct *vma,
+ struct file *backing_file,
+ struct file *user_file __always_unused)
+{
+ unsigned long prot = 0;
+
+ /* translate vma->vm_flags perms into PROT perms */
+ if (vma->vm_flags & VM_READ)
+ prot |= PROT_READ;
+ if (vma->vm_flags & VM_WRITE)
+ prot |= PROT_WRITE;
+ if (vma->vm_flags & VM_EXEC)
+ prot |= PROT_EXEC;
+
+ return selinux_mmap_file_common(backing_file->f_cred, backing_file,
+ prot, vma->vm_flags & VM_SHARED);
}
static int selinux_file_mprotect(struct vm_area_struct *vma,
unsigned long reqprot __always_unused,
unsigned long prot)
{
+ int rc;
const struct cred *cred = current_cred();
u32 sid = cred_sid(cred);
+ const struct file *file = vma->vm_file;
+ bool backing_file;
+ bool shared = vma->vm_flags & VM_SHARED;
+
+ /* check if we need to trigger the "backing files are awful" mode */
+ backing_file = file && (file->f_mode & FMODE_BACKING);
if (default_noexec &&
(prot & PROT_EXEC) && !(vma->vm_flags & VM_EXEC)) {
- int rc = 0;
/*
* We don't use the vma_is_initial_heap() helper as it has
* a history of problems and is currently broken on systems
@@ -3864,11 +3955,15 @@ static int selinux_file_mprotect(struct vm_area_struct *vma,
vma->vm_end <= vma->vm_mm->brk) {
rc = avc_has_perm(sid, sid, SECCLASS_PROCESS,
PROCESS__EXECHEAP, NULL);
- } else if (!vma->vm_file && (vma_is_initial_stack(vma) ||
+ if (rc)
+ return rc;
+ } else if (!file && (vma_is_initial_stack(vma) ||
vma_is_stack_for_current(vma))) {
rc = avc_has_perm(sid, sid, SECCLASS_PROCESS,
PROCESS__EXECSTACK, NULL);
- } else if (vma->vm_file && vma->anon_vma) {
+ if (rc)
+ return rc;
+ } else if (file && vma->anon_vma) {
/*
* We are making executable a file mapping that has
* had some COW done. Since pages might have been
@@ -3876,13 +3971,29 @@ static int selinux_file_mprotect(struct vm_area_struct *vma,
* modified content. This typically should only
* occur for text relocations.
*/
- rc = file_has_perm(cred, vma->vm_file, FILE__EXECMOD);
+ rc = __file_has_perm(cred, file, FILE__EXECMOD,
+ backing_file);
+ if (rc)
+ return rc;
+ if (backing_file) {
+ rc = file_has_perm(file->f_cred, file,
+ FILE__EXECMOD);
+ if (rc)
+ return rc;
+ }
}
+ }
+
+ rc = __file_map_prot_check(cred, file, prot, shared, backing_file);
+ if (rc)
+ return rc;
+ if (backing_file) {
+ rc = file_map_prot_check(file->f_cred, file, prot, shared);
if (rc)
return rc;
}
- return file_map_prot_check(vma->vm_file, prot, vma->vm_flags&VM_SHARED);
+ return 0;
}
static int selinux_file_lock(struct file *file, unsigned int cmd)
@@ -6960,6 +7071,7 @@ static void selinux_bpf_token_free(struct bpf_token *token)
struct lsm_blob_sizes selinux_blob_sizes __ro_after_init = {
.lbs_cred = sizeof(struct task_security_struct),
.lbs_file = sizeof(struct file_security_struct),
+ .lbs_backing_file = sizeof(struct backing_file_security_struct),
.lbs_inode = sizeof(struct inode_security_struct),
.lbs_ipc = sizeof(struct ipc_security_struct),
.lbs_key = sizeof(struct key_security_struct),
@@ -7165,9 +7277,11 @@ static struct security_hook_list selinux_hooks[] __ro_after_init = {
LSM_HOOK_INIT(file_permission, selinux_file_permission),
LSM_HOOK_INIT(file_alloc_security, selinux_file_alloc_security),
+ LSM_HOOK_INIT(backing_file_alloc, selinux_backing_file_alloc),
LSM_HOOK_INIT(file_ioctl, selinux_file_ioctl),
LSM_HOOK_INIT(file_ioctl_compat, selinux_file_ioctl_compat),
LSM_HOOK_INIT(mmap_file, selinux_mmap_file),
+ LSM_HOOK_INIT(mmap_backing_file, selinux_mmap_backing_file),
LSM_HOOK_INIT(mmap_addr, selinux_mmap_addr),
LSM_HOOK_INIT(file_mprotect, selinux_file_mprotect),
LSM_HOOK_INIT(file_lock, selinux_file_lock),
diff --git a/security/selinux/include/objsec.h b/security/selinux/include/objsec.h
index c88cae81ee4c..dc42282a2c05 100644
--- a/security/selinux/include/objsec.h
+++ b/security/selinux/include/objsec.h
@@ -61,6 +61,10 @@ struct file_security_struct {
u32 pseqno; /* Policy seqno at the time of file open */
};
+struct backing_file_security_struct {
+ u32 uf_sid; /* associated user file fsec->sid */
+};
+
struct superblock_security_struct {
u32 sid; /* SID of file system superblock */
u32 def_sid; /* default SID for labeling */
@@ -159,6 +163,13 @@ static inline struct file_security_struct *selinux_file(const struct file *file)
return file->f_security + selinux_blob_sizes.lbs_file;
}
+static inline struct backing_file_security_struct *
+selinux_backing_file(const struct file *backing_file)
+{
+ void *blob = backing_file_security(backing_file);
+ return blob + selinux_blob_sizes.lbs_backing_file;
+}
+
static inline struct inode_security_struct *
selinux_inode(const struct inode *inode)
{
diff --git a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c
index 960c9323d1e0..4183c2c057a3 100644
--- a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c
+++ b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c
@@ -6,6 +6,7 @@
#include "kprobe_multi_override.skel.h"
#include "kprobe_multi_session.skel.h"
#include "kprobe_multi_session_cookie.skel.h"
+#include "kprobe_multi_sleepable.skel.h"
#include "bpf/libbpf_internal.h"
#include "bpf/hashmap.h"
@@ -216,7 +217,9 @@ static void test_attach_api_syms(void)
static void test_attach_api_fails(void)
{
LIBBPF_OPTS(bpf_kprobe_multi_opts, opts);
+ LIBBPF_OPTS(bpf_test_run_opts, topts);
struct kprobe_multi *skel = NULL;
+ struct kprobe_multi_sleepable *sl_skel = NULL;
struct bpf_link *link = NULL;
unsigned long long addrs[2];
const char *syms[2] = {
@@ -224,7 +227,7 @@ static void test_attach_api_fails(void)
"bpf_fentry_test2",
};
__u64 cookies[2];
- int saved_error;
+ int saved_error, err;
addrs[0] = ksym_get_addr("bpf_fentry_test1");
addrs[1] = ksym_get_addr("bpf_fentry_test2");
@@ -323,9 +326,39 @@ static void test_attach_api_fails(void)
if (!ASSERT_EQ(saved_error, -E2BIG, "fail_6_error"))
goto cleanup;
+ /* fail_9 - sleepable kprobe multi should not attach */
+ sl_skel = kprobe_multi_sleepable__open();
+ if (!ASSERT_OK_PTR(sl_skel, "sleep_skel_open"))
+ goto cleanup;
+
+ sl_skel->bss->user_ptr = sl_skel;
+
+ err = bpf_program__set_flags(sl_skel->progs.handle_kprobe_multi_sleepable,
+ BPF_F_SLEEPABLE);
+ if (!ASSERT_OK(err, "sleep_skel_set_flags"))
+ goto cleanup;
+
+ err = kprobe_multi_sleepable__load(sl_skel);
+ if (!ASSERT_OK(err, "sleep_skel_load"))
+ goto cleanup;
+
+ link = bpf_program__attach_kprobe_multi_opts(sl_skel->progs.handle_kprobe_multi_sleepable,
+ "bpf_fentry_test1", NULL);
+ saved_error = -errno;
+
+ if (!ASSERT_ERR_PTR(link, "fail_9"))
+ goto cleanup;
+
+ if (!ASSERT_EQ(saved_error, -EINVAL, "fail_9_error"))
+ goto cleanup;
+
+ err = bpf_prog_test_run_opts(bpf_program__fd(sl_skel->progs.fentry), &topts);
+ ASSERT_OK(err, "bpf_prog_test_run_opts");
+
cleanup:
bpf_link__destroy(link);
kprobe_multi__destroy(skel);
+ kprobe_multi_sleepable__destroy(sl_skel);
}
static void test_session_skel_api(void)
diff --git a/tools/testing/selftests/bpf/progs/kprobe_multi_sleepable.c b/tools/testing/selftests/bpf/progs/kprobe_multi_sleepable.c
new file mode 100644
index 000000000000..932e1d9c72e2
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/kprobe_multi_sleepable.c
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+void *user_ptr = 0;
+
+SEC("kprobe.multi")
+int handle_kprobe_multi_sleepable(struct pt_regs *ctx)
+{
+ int a, err;
+
+ err = bpf_copy_from_user(&a, sizeof(a), user_ptr);
+ barrier_var(a);
+ return err;
+}
+
+SEC("fentry/bpf_fentry_test1")
+int BPF_PROG(fentry)
+{
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/ptp/testptp.c b/tools/testing/selftests/ptp/testptp.c
index e0aed424fe42..edc08a4433fd 100644
--- a/tools/testing/selftests/ptp/testptp.c
+++ b/tools/testing/selftests/ptp/testptp.c
@@ -147,6 +147,7 @@ static void usage(char *progname)
" -T val set the ptp clock time to 'val' seconds\n"
" -x val get an extended ptp clock time with the desired number of samples (up to %d)\n"
" -X get a ptp clock cross timestamp\n"
+ " -y val pre/post tstamp timebase to use {realtime|monotonic|monotonic-raw}\n"
" -z test combinations of rising/falling external time stamp flags\n",
progname, PTP_MAX_SAMPLES);
}
@@ -191,6 +192,7 @@ int main(int argc, char *argv[])
int readonly = 0;
int settime = 0;
int channel = -1;
+ clockid_t ext_clockid = CLOCK_REALTIME;
int64_t t1, t2, tp;
int64_t interval, offset;
@@ -200,7 +202,7 @@ int main(int argc, char *argv[])
progname = strrchr(argv[0], '/');
progname = progname ? 1+progname : argv[0];
- while (EOF != (c = getopt(argc, argv, "cd:e:f:F:ghH:i:k:lL:n:o:p:P:rsSt:T:w:x:Xz"))) {
+ while (EOF != (c = getopt(argc, argv, "cd:e:f:F:ghH:i:k:lL:n:o:p:P:rsSt:T:w:x:Xy:z"))) {
switch (c) {
case 'c':
capabilities = 1;
@@ -283,6 +285,21 @@ int main(int argc, char *argv[])
case 'X':
getcross = 1;
break;
+ case 'y':
+ if (!strcasecmp(optarg, "realtime"))
+ ext_clockid = CLOCK_REALTIME;
+ else if (!strcasecmp(optarg, "monotonic"))
+ ext_clockid = CLOCK_MONOTONIC;
+ else if (!strcasecmp(optarg, "monotonic-raw"))
+ ext_clockid = CLOCK_MONOTONIC_RAW;
+ else {
+ fprintf(stderr,
+ "type needs to be realtime, monotonic or monotonic-raw; was given %s\n",
+ optarg);
+ return -1;
+ }
+ break;
+
case 'z':
flagtest = 1;
break;
@@ -575,6 +592,7 @@ int main(int argc, char *argv[])
}
soe->n_samples = getextended;
+ soe->clockid = ext_clockid;
if (ioctl(fd, PTP_SYS_OFFSET_EXTENDED, soe)) {
perror("PTP_SYS_OFFSET_EXTENDED");
@@ -583,12 +601,46 @@ int main(int argc, char *argv[])
getextended);
for (i = 0; i < getextended; i++) {
- printf("sample #%2d: system time before: %lld.%09u\n",
- i, soe->ts[i][0].sec, soe->ts[i][0].nsec);
+ switch (ext_clockid) {
+ case CLOCK_REALTIME:
+ printf("sample #%2d: real time before: %lld.%09u\n",
+ i, soe->ts[i][0].sec,
+ soe->ts[i][0].nsec);
+ break;
+ case CLOCK_MONOTONIC:
+ printf("sample #%2d: monotonic time before: %lld.%09u\n",
+ i, soe->ts[i][0].sec,
+ soe->ts[i][0].nsec);
+ break;
+ case CLOCK_MONOTONIC_RAW:
+ printf("sample #%2d: monotonic-raw time before: %lld.%09u\n",
+ i, soe->ts[i][0].sec,
+ soe->ts[i][0].nsec);
+ break;
+ default:
+ break;
+ }
printf(" phc time: %lld.%09u\n",
soe->ts[i][1].sec, soe->ts[i][1].nsec);
- printf(" system time after: %lld.%09u\n",
- soe->ts[i][2].sec, soe->ts[i][2].nsec);
+ switch (ext_clockid) {
+ case CLOCK_REALTIME:
+ printf(" real time after: %lld.%09u\n",
+ soe->ts[i][2].sec,
+ soe->ts[i][2].nsec);
+ break;
+ case CLOCK_MONOTONIC:
+ printf(" monotonic time after: %lld.%09u\n",
+ soe->ts[i][2].sec,
+ soe->ts[i][2].nsec);
+ break;
+ case CLOCK_MONOTONIC_RAW:
+ printf(" monotonic-raw time after: %lld.%09u\n",
+ soe->ts[i][2].sec,
+ soe->ts[i][2].nsec);
+ break;
+ default:
+ break;
+ }
}
}
diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
index ee4d5e4e68d7..c9c9ccdb0e7e 100644
--- a/virt/kvm/eventfd.c
+++ b/virt/kvm/eventfd.c
@@ -24,6 +24,7 @@
#include <linux/slab.h>
#include <linux/seqlock.h>
#include <linux/irqbypass.h>
+#include <linux/unaligned.h>
#include <trace/events/kvm.h>
#include <kvm/iodev.h>
@@ -744,21 +745,18 @@ ioeventfd_in_range(struct _ioeventfd *p, gpa_t addr, int len, const void *val)
return true;
/* otherwise, we have to actually compare the data */
-
- BUG_ON(!IS_ALIGNED((unsigned long)val, len));
-
switch (len) {
case 1:
- _val = *(u8 *)val;
+ _val = get_unaligned((u8 *)val);
break;
case 2:
- _val = *(u16 *)val;
+ _val = get_unaligned((u16 *)val);
break;
case 4:
- _val = *(u32 *)val;
+ _val = get_unaligned((u32 *)val);
break;
case 8:
- _val = *(u64 *)val;
+ _val = get_unaligned((u64 *)val);
break;
default:
return false;
prev parent reply other threads:[~2026-07-04 11:59 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-04 11:59 Linux 6.12.95 Greg Kroah-Hartman
2026-07-04 11:59 ` Greg Kroah-Hartman [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2026070401-issue-antibody-a2a6@gregkh \
--to=gregkh@linuxfoundation.org \
--cc=akpm@linux-foundation.org \
--cc=jslaby@suse.cz \
--cc=linux-kernel@vger.kernel.org \
--cc=lwn@lwn.net \
--cc=stable@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.