From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A15B3A839B; Sat, 4 Jul 2026 11:59:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783166366; cv=none; b=IFQ/O6w9s7eEHZ4Ayhb8tsp2cWU1yHKrk/7POWRSgd5tk0CEhpLqqqcIIPxuWcJQzUj+SLXmxsv7pfIXLjaZJXCrrO+/I/04itAw0NyHm9M6lVctBbYacNOXC9ech1glbVk87k/MDaQPN4XGd5xw2ELZcQpDNORZRvmfPDcJyIM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783166366; c=relaxed/simple; bh=ka6ghKH2Ywog0vvHC86LzPXVs8CbRUFx8gcN8c6h+54=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NkqxPWi8cgas6XojuogZ7YeXHz+dV+XAQB4rziiAb+OwaIbkRLkec3MFH9Q98wCY8u7hym4XaqrdirNqfxGMmMuXCnvLZ/KbixldE1UkcsW56RqpJJ6gJBm4IVOJzxJAF3iY9oRkeGofap94GZhszWzw/fF5P+Kv2MFdpMTx39A= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=oYAQ0nq7; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="oYAQ0nq7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F04B81F00A3A; Sat, 4 Jul 2026 11:58:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1783166341; bh=fGA7YvXhGH3bKbbfOLIdUjas1UCEe9Ndh+d693HdHw4=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=oYAQ0nq7peNhlufuazbJWhou9OkmXZ4Zs2F6P+rXUzP2wE4qqCLCBViTiYZ97bj86 m4HfX47Kq7ak4f2/Eo4BQs7c+5mFXLd1Nx6FmYcSsKKuuIh7yUjHGC0/cmi5i7FkIj J1mCdBaBQCdtE45Ku47EoUr55Vpq0knfoh7WU7yw= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org, akpm@linux-foundation.org, torvalds@linux-foundation.org, stable@vger.kernel.org Cc: lwn@lwn.net, jslaby@suse.cz, Greg Kroah-Hartman Subject: Re: Linux 6.12.95 Date: Sat, 4 Jul 2026 13:59:01 +0200 Message-ID: <2026070401-issue-antibody-a2a6@gregkh> X-Mailer: git-send-email 2.55.0 In-Reply-To: <2026070401-polygon-mobilize-50fa@gregkh> References: <2026070401-polygon-mobilize-50fa@gregkh> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit diff --git a/Documentation/networking/bonding.rst b/Documentation/networking/bonding.rst index e774b48de9f5..94ef39457a62 100644 --- a/Documentation/networking/bonding.rst +++ b/Documentation/networking/bonding.rst @@ -193,6 +193,15 @@ ad_actor_sys_prio This parameter has effect only in 802.3ad mode and is available through SysFs interface. +actor_port_prio + + In an AD system, this specifies the port priority. The allowed range + is 1 - 65535. If the value is not specified, it takes 255 as the + default value. + + This parameter has effect only in 802.3ad mode and is available through + netlink interface. + ad_actor_system In an AD system, this specifies the mac-address for the actor in @@ -562,6 +571,12 @@ lacp_rate The default is slow. +broadcast_neighbor + + Option specifying whether to broadcast ARP/ND packets to all + active slaves. This option has no effect in modes other than + 802.3ad mode. The default is off (0). + max_bonds Specifies the number of bonding devices to create for this diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index e4be1378ba26..ff578a770598 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -66,330 +66,329 @@ This table lists ioctls visible from user land for Linux/x86. It contains most drivers up to 2.6.31, but I know I am missing some. There has been no attempt to list non-X86 architectures or ioctls from drivers/staging/. -==== ===== ======================================================= ================================================================ -Code Seq# Include File Comments +==== ===== ========================================================= ================================================================ +Code Seq# Include File Comments (hex) -==== ===== ======================================================= ================================================================ -0x00 00-1F linux/fs.h conflict! -0x00 00-1F scsi/scsi_ioctl.h conflict! -0x00 00-1F linux/fb.h conflict! -0x00 00-1F linux/wavefront.h conflict! +==== ===== ========================================================= ================================================================ +0x00 00-1F linux/fs.h conflict! +0x00 00-1F scsi/scsi_ioctl.h conflict! +0x00 00-1F linux/fb.h conflict! +0x00 00-1F linux/wavefront.h conflict! 0x02 all linux/fd.h 0x03 all linux/hdreg.h -0x04 D2-DC linux/umsdos_fs.h Dead since 2.6.11, but don't reuse these. +0x04 D2-DC linux/umsdos_fs.h Dead since 2.6.11, but don't reuse these. 0x06 all linux/lp.h 0x07 9F-D0 linux/vmw_vmci_defs.h, uapi/linux/vm_sockets.h 0x09 all linux/raid/md_u.h 0x10 00-0F drivers/char/s390/vmcp.h 0x10 10-1F arch/s390/include/uapi/sclp_ctl.h 0x10 20-2F arch/s390/include/uapi/asm/hypfs.h -0x12 all linux/fs.h BLK* ioctls +0x12 all linux/fs.h BLK* ioctls linux/blkpg.h -0x15 all linux/fs.h FS_IOC_* ioctls -0x1b all InfiniBand Subsystem - +0x15 all linux/fs.h FS_IOC_* ioctls +0x1b all InfiniBand Subsystem + 0x20 all drivers/cdrom/cm206.h 0x22 all scsi/sg.h -0x3E 00-0F linux/counter.h +0x3E 00-0F linux/counter.h '!' 00-1F uapi/linux/seccomp.h -'#' 00-3F IEEE 1394 Subsystem - Block for the entire subsystem +'#' 00-3F IEEE 1394 Subsystem + Block for the entire subsystem '$' 00-0F linux/perf_counter.h, linux/perf_event.h -'%' 00-0F include/uapi/linux/stm.h System Trace Module subsystem - +'%' 00-0F include/uapi/linux/stm.h System Trace Module subsystem + '&' 00-07 drivers/firewire/nosy-user.h -'*' 00-1F uapi/linux/user_events.h User Events Subsystem - -'1' 00-1F linux/timepps.h PPS kit from Ulrich Windl - +'*' 00-1F uapi/linux/user_events.h User Events Subsystem + +'1' 00-1F linux/timepps.h PPS kit from Ulrich Windl + '2' 01-04 linux/i2o.h -'3' 00-0F drivers/s390/char/raw3270.h conflict! -'3' 00-1F linux/suspend_ioctls.h, conflict! +'3' 00-0F drivers/s390/char/raw3270.h conflict! +'3' 00-1F linux/suspend_ioctls.h, conflict! kernel/power/user.c -'8' all SNP8023 advanced NIC card - +'8' all SNP8023 advanced NIC card + ';' 64-7F linux/vfio.h ';' 80-FF linux/iommufd.h -'=' 00-3f uapi/linux/ptp_clock.h -'@' 00-0F linux/radeonfb.h conflict! -'@' 00-0F drivers/video/aty/aty128fb.c conflict! -'A' 00-1F linux/apm_bios.h conflict! -'A' 00-0F linux/agpgart.h, conflict! +'=' 00-3f uapi/linux/ptp_clock.h +'@' 00-0F linux/radeonfb.h conflict! +'@' 00-0F drivers/video/aty/aty128fb.c conflict! +'A' 00-1F linux/apm_bios.h conflict! +'A' 00-0F linux/agpgart.h, conflict! drivers/char/agp/compat_ioctl.h -'A' 00-7F sound/asound.h conflict! -'B' 00-1F linux/cciss_ioctl.h conflict! -'B' 00-0F include/linux/pmu.h conflict! -'B' C0-FF advanced bbus -'B' 00-0F xen/xenbus_dev.h conflict! -'C' all linux/soundcard.h conflict! -'C' 01-2F linux/capi.h conflict! -'C' F0-FF drivers/net/wan/cosa.h conflict! +'A' 00-7F sound/asound.h conflict! +'B' 00-1F linux/cciss_ioctl.h conflict! +'B' 00-0F include/linux/pmu.h conflict! +'B' C0-FF advanced bbus +'B' 00-0F xen/xenbus_dev.h conflict! +'C' all linux/soundcard.h conflict! +'C' 01-2F linux/capi.h conflict! +'C' F0-FF drivers/net/wan/cosa.h conflict! 'D' all arch/s390/include/asm/dasd.h -'D' 40-5F drivers/scsi/dpt/dtpi_ioctl.h Dead since 2022 +'D' 40-5F drivers/scsi/dpt/dtpi_ioctl.h Dead since 2022 'D' 05 drivers/scsi/pmcraid.h -'E' all linux/input.h conflict! -'E' 00-0F xen/evtchn.h conflict! -'F' all linux/fb.h conflict! -'F' 01-02 drivers/scsi/pmcraid.h conflict! -'F' 20 drivers/video/fsl-diu-fb.h conflict! -'F' 20 linux/ivtvfb.h conflict! -'F' 20 linux/matroxfb.h conflict! -'F' 20 drivers/video/aty/atyfb_base.c conflict! -'F' 00-0F video/da8xx-fb.h conflict! -'F' 80-8F linux/arcfb.h conflict! -'F' DD video/sstfb.h conflict! -'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict! -'G' 00-0F xen/gntalloc.h, xen/gntdev.h conflict! -'H' 00-7F linux/hiddev.h conflict! -'H' 00-0F linux/hidraw.h conflict! -'H' 01 linux/mei.h conflict! -'H' 02 linux/mei.h conflict! -'H' 03 linux/mei.h conflict! -'H' 00-0F sound/asound.h conflict! -'H' 20-40 sound/asound_fm.h conflict! -'H' 80-8F sound/sfnt_info.h conflict! -'H' 10-8F sound/emu10k1.h conflict! -'H' 10-1F sound/sb16_csp.h conflict! -'H' 10-1F sound/hda_hwdep.h conflict! -'H' 40-4F sound/hdspm.h conflict! -'H' 40-4F sound/hdsp.h conflict! +'E' all linux/input.h conflict! +'E' 00-0F xen/evtchn.h conflict! +'F' all linux/fb.h conflict! +'F' 01-02 drivers/scsi/pmcraid.h conflict! +'F' 20 drivers/video/fsl-diu-fb.h conflict! +'F' 20 linux/ivtvfb.h conflict! +'F' 20 linux/matroxfb.h conflict! +'F' 20 drivers/video/aty/atyfb_base.c conflict! +'F' 00-0F video/da8xx-fb.h conflict! +'F' 80-8F linux/arcfb.h conflict! +'F' DD video/sstfb.h conflict! +'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict! +'G' 00-0F xen/gntalloc.h, xen/gntdev.h conflict! +'H' 00-7F linux/hiddev.h conflict! +'H' 00-0F linux/hidraw.h conflict! +'H' 01 linux/mei.h conflict! +'H' 02 linux/mei.h conflict! +'H' 03 linux/mei.h conflict! +'H' 00-0F sound/asound.h conflict! +'H' 20-40 sound/asound_fm.h conflict! +'H' 80-8F sound/sfnt_info.h conflict! +'H' 10-8F sound/emu10k1.h conflict! +'H' 10-1F sound/sb16_csp.h conflict! +'H' 10-1F sound/hda_hwdep.h conflict! +'H' 40-4F sound/hdspm.h conflict! +'H' 40-4F sound/hdsp.h conflict! 'H' 90 sound/usb/usx2y/usb_stream.h -'H' 00-0F uapi/misc/habanalabs.h conflict! +'H' 00-0F uapi/misc/habanalabs.h conflict! 'H' A0 uapi/linux/usb/cdc-wdm.h -'H' C0-F0 net/bluetooth/hci.h conflict! -'H' C0-DF net/bluetooth/hidp/hidp.h conflict! -'H' C0-DF net/bluetooth/cmtp/cmtp.h conflict! -'H' C0-DF net/bluetooth/bnep/bnep.h conflict! -'H' F1 linux/hid-roccat.h +'H' C0-F0 net/bluetooth/hci.h conflict! +'H' C0-DF net/bluetooth/hidp/hidp.h conflict! +'H' C0-DF net/bluetooth/cmtp/cmtp.h conflict! +'H' C0-DF net/bluetooth/bnep/bnep.h conflict! +'H' F1 linux/hid-roccat.h 'H' F8-FA sound/firewire.h -'I' all linux/isdn.h conflict! -'I' 00-0F drivers/isdn/divert/isdn_divert.h conflict! -'I' 40-4F linux/mISDNif.h conflict! +'I' all linux/isdn.h conflict! +'I' 00-0F drivers/isdn/divert/isdn_divert.h conflict! +'I' 40-4F linux/mISDNif.h conflict! 'K' all linux/kd.h -'L' 00-1F linux/loop.h conflict! -'L' 10-1F drivers/scsi/mpt3sas/mpt3sas_ctl.h conflict! -'L' E0-FF linux/ppdd.h encrypted disk device driver - -'M' all linux/soundcard.h conflict! -'M' 01-16 mtd/mtd-abi.h conflict! +'L' 00-1F linux/loop.h conflict! +'L' 10-1F drivers/scsi/mpt3sas/mpt3sas_ctl.h conflict! +'L' E0-FF linux/ppdd.h encrypted disk device driver + +'M' all linux/soundcard.h conflict! +'M' 01-16 mtd/mtd-abi.h conflict! and drivers/mtd/mtdchar.c 'M' 01-03 drivers/scsi/megaraid/megaraid_sas.h -'M' 00-0F drivers/video/fsl-diu-fb.h conflict! +'M' 00-0F drivers/video/fsl-diu-fb.h conflict! 'N' 00-1F drivers/usb/scanner.h 'N' 40-7F drivers/block/nvme.c -'N' 80-8F uapi/linux/ntsync.h NT synchronization primitives - -'O' 00-06 mtd/ubi-user.h UBI -'P' all linux/soundcard.h conflict! -'P' 60-6F sound/sscape_ioctl.h conflict! -'P' 00-0F drivers/usb/class/usblp.c conflict! -'P' 01-09 drivers/misc/pci_endpoint_test.c conflict! -'P' 00-0F xen/privcmd.h conflict! -'P' 00-05 linux/tps6594_pfsm.h conflict! +'N' 80-8F uapi/linux/ntsync.h NT synchronization primitives + +'O' 00-06 mtd/ubi-user.h UBI +'P' all linux/soundcard.h conflict! +'P' 60-6F sound/sscape_ioctl.h conflict! +'P' 00-0F drivers/usb/class/usblp.c conflict! +'P' 01-09 drivers/misc/pci_endpoint_test.c conflict! +'P' 00-0F xen/privcmd.h conflict! +'P' 00-05 linux/tps6594_pfsm.h conflict! 'Q' all linux/soundcard.h -'R' 00-1F linux/random.h conflict! -'R' 01 linux/rfkill.h conflict! +'R' 00-1F linux/random.h conflict! +'R' 01 linux/rfkill.h conflict! 'R' 20-2F linux/trace_mmap.h 'R' C0-DF net/bluetooth/rfcomm.h 'R' E0 uapi/linux/fsl_mc.h -'S' all linux/cdrom.h conflict! -'S' 80-81 scsi/scsi_ioctl.h conflict! -'S' 82-FF scsi/scsi.h conflict! -'S' 00-7F sound/asequencer.h conflict! -'T' all linux/soundcard.h conflict! -'T' 00-AF sound/asound.h conflict! -'T' all arch/x86/include/asm/ioctls.h conflict! -'T' C0-DF linux/if_tun.h conflict! -'U' all sound/asound.h conflict! -'U' 00-CF linux/uinput.h conflict! +'S' all linux/cdrom.h conflict! +'S' 80-81 scsi/scsi_ioctl.h conflict! +'S' 82-FF scsi/scsi.h conflict! +'S' 00-7F sound/asequencer.h conflict! +'T' all linux/soundcard.h conflict! +'T' 00-AF sound/asound.h conflict! +'T' all arch/x86/include/asm/ioctls.h conflict! +'T' C0-DF linux/if_tun.h conflict! +'U' all sound/asound.h conflict! +'U' 00-CF linux/uinput.h conflict! 'U' 00-EF linux/usbdevice_fs.h 'U' C0-CF drivers/bluetooth/hci_uart.h -'V' all linux/vt.h conflict! -'V' all linux/videodev2.h conflict! -'V' C0 linux/ivtvfb.h conflict! -'V' C0 linux/ivtv.h conflict! -'V' C0 media/si4713.h conflict! -'W' 00-1F linux/watchdog.h conflict! -'W' 00-1F linux/wanrouter.h conflict! (pre 3.9) -'W' 00-3F sound/asound.h conflict! +'V' all linux/vt.h conflict! +'V' all linux/videodev2.h conflict! +'V' C0 linux/ivtvfb.h conflict! +'V' C0 linux/ivtv.h conflict! +'V' C0 media/si4713.h conflict! +'W' 00-1F linux/watchdog.h conflict! +'W' 00-1F linux/wanrouter.h conflict! (pre 3.9) +'W' 00-3F sound/asound.h conflict! 'W' 40-5F drivers/pci/switch/switchtec.c 'W' 60-61 linux/watch_queue.h -'X' all fs/xfs/xfs_fs.h, conflict! +'X' all fs/xfs/xfs_fs.h, conflict! fs/xfs/linux-2.6/xfs_ioctl32.h, include/linux/falloc.h, linux/fs.h, -'X' all fs/ocfs2/ocfs_fs.h conflict! -'X' 01 linux/pktcdvd.h conflict! +'X' all fs/ocfs2/ocfs_fs.h conflict! +'X' 01 linux/pktcdvd.h conflict! 'Z' 14-15 drivers/message/fusion/mptctl.h -'[' 00-3F linux/usb/tmc.h USB Test and Measurement Devices - -'a' all linux/atm*.h, linux/sonet.h ATM on linux - -'a' 00-0F drivers/crypto/qat/qat_common/adf_cfg_common.h conflict! qat driver -'b' 00-FF conflict! bit3 vme host bridge - -'b' 00-0F linux/dma-buf.h conflict! -'c' 00-7F linux/comstats.h conflict! -'c' 00-7F linux/coda.h conflict! -'c' 00-1F linux/chio.h conflict! -'c' 80-9F arch/s390/include/asm/chsc.h conflict! +'[' 00-3F linux/usb/tmc.h USB Test and Measurement Devices + +'a' all linux/atm*.h, linux/sonet.h ATM on linux + +'b' 00-FF conflict! bit3 vme host bridge + +'b' 00-0F linux/dma-buf.h conflict! +'c' 00-7F linux/comstats.h conflict! +'c' 00-7F linux/coda.h conflict! +'c' 00-1F linux/chio.h conflict! +'c' 80-9F arch/s390/include/asm/chsc.h conflict! 'c' A0-AF arch/x86/include/asm/msr.h conflict! -'d' 00-FF linux/char/drm/drm.h conflict! -'d' 02-40 pcmcia/ds.h conflict! +'d' 00-FF linux/char/drm/drm.h conflict! +'d' 02-40 pcmcia/ds.h conflict! 'd' F0-FF linux/digi1.h -'e' all linux/digi1.h conflict! -'f' 00-1F linux/ext2_fs.h conflict! -'f' 00-1F linux/ext3_fs.h conflict! -'f' 00-0F fs/jfs/jfs_dinode.h conflict! -'f' 00-0F fs/ext4/ext4.h conflict! -'f' 00-0F linux/fs.h conflict! -'f' 00-0F fs/ocfs2/ocfs2_fs.h conflict! +'e' all linux/digi1.h conflict! +'f' 00-1F linux/ext2_fs.h conflict! +'f' 00-1F linux/ext3_fs.h conflict! +'f' 00-0F fs/jfs/jfs_dinode.h conflict! +'f' 00-0F fs/ext4/ext4.h conflict! +'f' 00-0F linux/fs.h conflict! +'f' 00-0F fs/ocfs2/ocfs2_fs.h conflict! 'f' 13-27 linux/fscrypt.h 'f' 81-8F linux/fsverity.h 'g' 00-0F linux/usb/gadgetfs.h 'g' 20-2F linux/usb/g_printer.h -'h' 00-7F conflict! Charon filesystem - -'h' 00-1F linux/hpet.h conflict! +'h' 00-7F conflict! Charon filesystem + +'h' 00-1F linux/hpet.h conflict! 'h' 80-8F fs/hfsplus/ioctl.c -'i' 00-3F linux/i2o-dev.h conflict! -'i' 0B-1F linux/ipmi.h conflict! +'i' 00-3F linux/i2o-dev.h conflict! +'i' 0B-1F linux/ipmi.h conflict! 'i' 80-8F linux/i8k.h -'i' 90-9F `linux/iio/*.h` IIO +'i' 90-9F `linux/iio/*.h` IIO 'j' 00-3F linux/joystick.h -'k' 00-0F linux/spi/spidev.h conflict! -'k' 00-05 video/kyro.h conflict! -'k' 10-17 linux/hsi/hsi_char.h HSI character device -'l' 00-3F linux/tcfs_fs.h transparent cryptographic file system - -'l' 40-7F linux/udf_fs_i.h in development: - -'m' 00-09 linux/mmtimer.h conflict! -'m' all linux/mtio.h conflict! -'m' all linux/soundcard.h conflict! -'m' all linux/synclink.h conflict! -'m' 00-19 drivers/message/fusion/mptctl.h conflict! -'m' 00 drivers/scsi/megaraid/megaraid_ioctl.h conflict! +'k' 00-0F linux/spi/spidev.h conflict! +'k' 00-05 video/kyro.h conflict! +'k' 10-17 linux/hsi/hsi_char.h HSI character device +'l' 00-3F linux/tcfs_fs.h transparent cryptographic file system + +'l' 40-7F linux/udf_fs_i.h in development: + +'m' 00-09 linux/mmtimer.h conflict! +'m' all linux/mtio.h conflict! +'m' all linux/soundcard.h conflict! +'m' all linux/synclink.h conflict! +'m' 00-19 drivers/message/fusion/mptctl.h conflict! +'m' 00 drivers/scsi/megaraid/megaraid_ioctl.h conflict! 'n' 00-7F linux/ncp_fs.h and fs/ncpfs/ioctl.c -'n' 80-8F uapi/linux/nilfs2_api.h NILFS2 -'n' E0-FF linux/matroxfb.h matroxfb -'o' 00-1F fs/ocfs2/ocfs2_fs.h OCFS2 -'o' 00-03 mtd/ubi-user.h conflict! (OCFS2 and UBI overlaps) -'o' 40-41 mtd/ubi-user.h UBI -'o' 01-A1 `linux/dvb/*.h` DVB -'p' 00-0F linux/phantom.h conflict! (OpenHaptics needs this) -'p' 00-1F linux/rtc.h conflict! +'n' 80-8F uapi/linux/nilfs2_api.h NILFS2 +'n' E0-FF linux/matroxfb.h matroxfb +'o' 00-1F fs/ocfs2/ocfs2_fs.h OCFS2 +'o' 00-03 mtd/ubi-user.h conflict! (OCFS2 and UBI overlaps) +'o' 40-41 mtd/ubi-user.h UBI +'o' 01-A1 `linux/dvb/*.h` DVB +'p' 00-0F linux/phantom.h conflict! (OpenHaptics needs this) +'p' 00-1F linux/rtc.h conflict! 'p' 40-7F linux/nvram.h -'p' 80-9F linux/ppdev.h user-space parport - -'p' A1-A5 linux/pps.h LinuxPPS - +'p' 80-9F linux/ppdev.h user-space parport + +'p' A1-A5 linux/pps.h LinuxPPS + 'q' 00-1F linux/serio.h -'q' 80-FF linux/telephony.h Internet PhoneJACK, Internet LineJACK - linux/ixjuser.h +'q' 80-FF linux/telephony.h Internet PhoneJACK, Internet LineJACK + linux/ixjuser.h 'r' 00-1F linux/msdos_fs.h and fs/fat/dir.c 's' all linux/cdk.h 't' 00-7F linux/ppp-ioctl.h 't' 80-8F linux/isdn_ppp.h -'t' 90-91 linux/toshiba.h toshiba and toshiba_acpi SMM -'u' 00-1F linux/smb_fs.h gone -'u' 00-2F linux/ublk_cmd.h conflict! -'u' 20-3F linux/uvcvideo.h USB video class host driver -'u' 40-4f linux/udmabuf.h userspace dma-buf misc device -'v' 00-1F linux/ext2_fs.h conflict! -'v' 00-1F linux/fs.h conflict! -'v' 00-0F linux/sonypi.h conflict! -'v' 00-0F media/v4l2-subdev.h conflict! -'v' 20-27 arch/powerpc/include/uapi/asm/vas-api.h VAS API -'v' C0-FF linux/meye.h conflict! -'w' all CERN SCI driver -'y' 00-1F packet based user level communications - -'z' 00-3F CAN bus card conflict! - -'z' 40-7F CAN bus card conflict! - -'z' 10-4F drivers/s390/crypto/zcrypt_api.h conflict! +'t' 90-91 linux/toshiba.h toshiba and toshiba_acpi SMM +'u' 00-1F linux/smb_fs.h gone +'u' 00-2F linux/ublk_cmd.h conflict! +'u' 20-3F linux/uvcvideo.h USB video class host driver +'u' 40-4f linux/udmabuf.h userspace dma-buf misc device +'v' 00-1F linux/ext2_fs.h conflict! +'v' 00-1F linux/fs.h conflict! +'v' 00-0F linux/sonypi.h conflict! +'v' 00-0F media/v4l2-subdev.h conflict! +'v' 20-27 arch/powerpc/include/uapi/asm/vas-api.h VAS API +'v' C0-FF linux/meye.h conflict! +'w' all CERN SCI driver +'y' 00-1F packet based user level communications + +'z' 00-3F CAN bus card conflict! + +'z' 40-7F CAN bus card conflict! + +'z' 10-4F drivers/s390/crypto/zcrypt_api.h conflict! '|' 00-7F linux/media.h 0x80 00-1F linux/fb.h 0x81 00-1F linux/vduse.h 0x89 00-06 arch/x86/include/asm/sockios.h 0x89 0B-DF linux/sockios.h -0x89 E0-EF linux/sockios.h SIOCPROTOPRIVATE range -0x89 F0-FF linux/sockios.h SIOCDEVPRIVATE range +0x89 E0-EF linux/sockios.h SIOCPROTOPRIVATE range +0x89 F0-FF linux/sockios.h SIOCDEVPRIVATE range 0x8A 00-1F linux/eventpoll.h 0x8B all linux/wireless.h -0x8C 00-3F WiNRADiO driver - +0x8C 00-3F WiNRADiO driver + 0x90 00 drivers/cdrom/sbpcd.h 0x92 00-0F drivers/usb/mon/mon_bin.c 0x93 60-7F linux/auto_fs.h -0x94 all fs/btrfs/ioctl.h Btrfs filesystem - and linux/fs.h some lifted to vfs/generic -0x97 00-7F fs/ceph/ioctl.h Ceph file system -0x99 00-0F 537-Addinboard driver - -0xA0 all linux/sdp/sdp.h Industrial Device Project - -0xA1 0 linux/vtpm_proxy.h TPM Emulator Proxy Driver -0xA2 all uapi/linux/acrn.h ACRN hypervisor -0xA3 80-8F Port ACL in development: - +0x94 all fs/btrfs/ioctl.h Btrfs filesystem + and linux/fs.h some lifted to vfs/generic +0x97 00-7F fs/ceph/ioctl.h Ceph file system +0x99 00-0F 537-Addinboard driver + +0xA0 all linux/sdp/sdp.h Industrial Device Project + +0xA1 0 linux/vtpm_proxy.h TPM Emulator Proxy Driver +0xA2 all uapi/linux/acrn.h ACRN hypervisor +0xA3 80-8F Port ACL in development: + 0xA3 90-9F linux/dtlk.h -0xA4 00-1F uapi/linux/tee.h Generic TEE subsystem -0xA4 00-1F uapi/asm/sgx.h -0xA5 01-05 linux/surface_aggregator/cdev.h Microsoft Surface Platform System Aggregator - -0xA5 20-2F linux/surface_aggregator/dtx.h Microsoft Surface DTX driver - +0xA4 00-1F uapi/linux/tee.h Generic TEE subsystem +0xA4 00-1F uapi/asm/sgx.h +0xA5 01-05 linux/surface_aggregator/cdev.h Microsoft Surface Platform System Aggregator + +0xA5 20-2F linux/surface_aggregator/dtx.h Microsoft Surface DTX driver + 0xAA 00-3F linux/uapi/linux/userfaultfd.h 0xAB 00-1F linux/nbd.h 0xAC 00-1F linux/raw.h -0xAD 00 Netfilter device in development: - -0xAE 00-1F linux/kvm.h Kernel-based Virtual Machine - -0xAE 40-FF linux/kvm.h Kernel-based Virtual Machine - -0xAE 20-3F linux/nitro_enclaves.h Nitro Enclaves -0xAF 00-1F linux/fsl_hypervisor.h Freescale hypervisor -0xB0 all RATIO devices in development: - -0xB1 00-1F PPPoX - -0xB2 00 arch/powerpc/include/uapi/asm/papr-vpd.h powerpc/pseries VPD API - -0xB2 01-02 arch/powerpc/include/uapi/asm/papr-sysparm.h powerpc/pseries system parameter API - +0xAD 00 Netfilter device in development: + +0xAE 00-1F linux/kvm.h Kernel-based Virtual Machine + +0xAE 40-FF linux/kvm.h Kernel-based Virtual Machine + +0xAE 20-3F linux/nitro_enclaves.h Nitro Enclaves +0xAF 00-1F linux/fsl_hypervisor.h Freescale hypervisor +0xB0 all RATIO devices in development: + +0xB1 00-1F PPPoX + +0xB2 00 arch/powerpc/include/uapi/asm/papr-vpd.h powerpc/pseries VPD API + +0xB2 01-02 arch/powerpc/include/uapi/asm/papr-sysparm.h powerpc/pseries system parameter API + 0xB3 00 linux/mmc/ioctl.h -0xB4 00-0F linux/gpio.h -0xB5 00-0F uapi/linux/rpmsg.h +0xB4 00-0F linux/gpio.h +0xB5 00-0F uapi/linux/rpmsg.h 0xB6 all linux/fpga-dfl.h -0xB7 all uapi/linux/remoteproc_cdev.h -0xB7 all uapi/linux/nsfs.h > -0xB8 01-02 uapi/misc/mrvl_cn10k_dpi.h Marvell CN10K DPI driver +0xB7 all uapi/linux/remoteproc_cdev.h +0xB7 all uapi/linux/nsfs.h > +0xB8 01-02 uapi/misc/mrvl_cn10k_dpi.h Marvell CN10K DPI driver 0xC0 00-0F linux/usb/iowarrior.h 0xCA 00-0F uapi/misc/cxl.h 0xCA 10-2F uapi/misc/ocxl.h 0xCA 80-BF uapi/scsi/cxlflash_ioctl.h -0xCB 00-1F CBM serial IEC bus in development: - -0xCC 00-0F drivers/misc/ibmvmc.h pseries VMC driver +0xCB 00-1F CBM serial IEC bus in development: + +0xCC 00-0F drivers/misc/ibmvmc.h pseries VMC driver 0xCD 01 linux/reiserfs_fs.h -0xCE 01-02 uapi/linux/cxl_mem.h Compute Express Link Memory Devices +0xCE 01-02 uapi/linux/cxl_mem.h Compute Express Link Memory Devices 0xCF 02 fs/smb/client/cifs_ioctl.h 0xDB 00-0F drivers/char/mwave/mwavepub.h -0xDD 00-3F ZFCP device driver see drivers/s390/scsi/ - +0xDD 00-3F ZFCP device driver see drivers/s390/scsi/ + 0xE5 00-3F linux/fuse.h -0xEC 00-01 drivers/platform/chrome/cros_ec_dev.h ChromeOS EC driver -0xEE 00-09 uapi/linux/pfrut.h Platform Firmware Runtime Update and Telemetry -0xF3 00-3F drivers/usb/misc/sisusbvga/sisusb.h sisfb (in development) - -0xF6 all LTTng Linux Trace Toolkit Next Generation - -0xF8 all arch/x86/include/uapi/asm/amd_hsmp.h AMD HSMP EPYC system management interface driver - +0xEC 00-01 drivers/platform/chrome/cros_ec_dev.h ChromeOS EC driver +0xEE 00-09 uapi/linux/pfrut.h Platform Firmware Runtime Update and Telemetry +0xF3 00-3F drivers/usb/misc/sisusbvga/sisusb.h sisfb (in development) + +0xF6 all LTTng Linux Trace Toolkit Next Generation + +0xF8 all arch/x86/include/uapi/asm/amd_hsmp.h AMD HSMP EPYC system management interface driver + 0xFD all linux/dm-ioctl.h 0xFE all linux/isst_if.h -==== ===== ======================================================= ================================================================ +==== ===== ========================================================= ================================================================ diff --git a/Makefile b/Makefile index 3f5933a5c7e7..bcb9d2fd69a6 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 VERSION = 6 PATCHLEVEL = 12 -SUBLEVEL = 94 +SUBLEVEL = 95 EXTRAVERSION = NAME = Baby Opossum Posse diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 012ec170232e..dd98fa97aad0 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -212,6 +212,7 @@ config ARM64 if DYNAMIC_FTRACE_WITH_ARGS select HAVE_SAMPLE_FTRACE_DIRECT select HAVE_SAMPLE_FTRACE_DIRECT_MULTI + select HAVE_BUILDTIME_MCOUNT_SORT select HAVE_EFFICIENT_UNALIGNED_ACCESS select HAVE_GUP_FAST select HAVE_FTRACE_MCOUNT_RECORD diff --git a/arch/loongarch/kernel/smp.c b/arch/loongarch/kernel/smp.c index d96065dbe779..cf1f377ec622 100644 --- a/arch/loongarch/kernel/smp.c +++ b/arch/loongarch/kernel/smp.c @@ -650,6 +650,7 @@ static void stop_this_cpu(void *dummy) set_cpu_online(smp_processor_id(), false); calculate_cpu_foreign_map(); local_irq_disable(); + rcutree_report_cpu_dead(); while (true); } diff --git a/arch/mips/dec/prom/console.c b/arch/mips/dec/prom/console.c index 31a8441d8431..b4f0dba3fa20 100644 --- a/arch/mips/dec/prom/console.c +++ b/arch/mips/dec/prom/console.c @@ -2,8 +2,9 @@ /* * DECstation PROM-based early console support. * - * Copyright (C) 2004, 2007 Maciej W. Rozycki + * Copyright (C) 2004, 2007, 2026 Maciej W. Rozycki */ +#include #include #include #include @@ -14,9 +15,11 @@ static void __init prom_console_write(struct console *con, const char *s, unsigned int c) { - char buf[81]; + static char buf[81] __initdata = { 0 }; unsigned int chunk = sizeof(buf) - 1; + BUG_ON((long)buf != (int)(long)buf); + while (c > 0) { if (chunk > c) chunk = c; diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c index 39e193cad2b9..a95814e088a2 100644 --- a/arch/mips/kernel/smp.c +++ b/arch/mips/kernel/smp.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -411,6 +412,7 @@ static void stop_this_cpu(void *dummy) set_cpu_online(smp_processor_id(), false); calculate_cpu_foreign_map(); local_irq_disable(); + rcutree_report_cpu_dead(); while (1); } diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h index b59ffeb668d6..a62604fab1aa 100644 --- a/arch/riscv/include/asm/cacheflush.h +++ b/arch/riscv/include/asm/cacheflush.h @@ -43,20 +43,23 @@ do { \ #ifdef CONFIG_64BIT extern u64 new_vmalloc[NR_CPUS / sizeof(u64) + 1]; extern char _end[]; +static inline void mark_new_valid_map(void) +{ + int i; + + /* + * We don't care if concurrently a cpu resets this value since + * the only place this can happen is in handle_exception() where + * an sfence.vma is emitted. + */ + for (i = 0; i < ARRAY_SIZE(new_vmalloc); ++i) + new_vmalloc[i] = -1ULL; +} #define flush_cache_vmap flush_cache_vmap static inline void flush_cache_vmap(unsigned long start, unsigned long end) { - if (is_vmalloc_or_module_addr((void *)start)) { - int i; - - /* - * We don't care if concurrently a cpu resets this value since - * the only place this can happen is in handle_exception() where - * an sfence.vma is emitted. - */ - for (i = 0; i < ARRAY_SIZE(new_vmalloc); ++i) - new_vmalloc[i] = -1ULL; - } + if (is_vmalloc_or_module_addr((void *)start)) + mark_new_valid_map(); } #define flush_cache_vmap_early(start, end) local_flush_tlb_kernel_range(start, end) #endif diff --git a/arch/riscv/include/asm/kfence.h b/arch/riscv/include/asm/kfence.h index d08bf7fb3aee..29cb3a6ee113 100644 --- a/arch/riscv/include/asm/kfence.h +++ b/arch/riscv/include/asm/kfence.h @@ -6,6 +6,7 @@ #include #include #include +#include #include static inline bool arch_kfence_init_pool(void) @@ -17,10 +18,12 @@ static inline bool kfence_protect_page(unsigned long addr, bool protect) { pte_t *pte = virt_to_kpte(addr); - if (protect) + if (protect) { set_pte(pte, __pte(pte_val(ptep_get(pte)) & ~_PAGE_PRESENT)); - else + } else { set_pte(pte, __pte(pte_val(ptep_get(pte)) | _PAGE_PRESENT)); + mark_new_valid_map(); + } preempt_disable(); local_flush_tlb_kernel_range(addr, addr + PAGE_SIZE); diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index af483de85dfc..5efa5584303e 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -106,8 +106,10 @@ SYM_CODE_START(handle_exception) #ifdef CONFIG_64BIT /* - * The RISC-V kernel does not eagerly emit a sfence.vma after each - * new vmalloc mapping, which may result in exceptions: + * The RISC-V kernel does not flush TLBs on all CPUS after each new + * vmalloc mapping or kfence_unprotect(), which may result in + * exceptions: + * * - if the uarch caches invalid entries, the new mapping would not be * observed by the page table walker and an invalidation is needed. * - if the uarch does not cache invalid entries, a reordered access diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 0d9c62b1dd44..f85c51241b3d 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -1838,6 +1838,11 @@ static bool hv_is_vp_in_sparse_set(u32 vp_id, u64 valid_bank_mask, u64 sparse_ba int valid_bit_nr = vp_id / HV_VCPUS_PER_SPARSE_BANK; unsigned long sbank; + BUILD_BUG_ON(BITS_PER_TYPE(valid_bank_mask) != HV_MAX_SPARSE_VCPU_BANKS); + + if (valid_bit_nr >= HV_MAX_SPARSE_VCPU_BANKS) + return false; + if (!test_bit(valid_bit_nr, (unsigned long *)&valid_bank_mask)) return false; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index d288c60ae200..aab26f90c285 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2329,13 +2329,15 @@ static struct kvm_mmu_page *kvm_mmu_get_child_sp(struct kvm_vcpu *vcpu, u64 *sptep, gfn_t gfn, bool direct, unsigned int access) { - union kvm_mmu_page_role role; + union kvm_mmu_page_role role = kvm_mmu_child_role(sptep, direct, access); - if (is_shadow_present_pte(*sptep) && !is_large_pte(*sptep) && - spte_to_child_sp(*sptep) && spte_to_child_sp(*sptep)->gfn == gfn) + if (is_shadow_present_pte(*sptep) && + !is_large_pte(*sptep) && + spte_to_child_sp(*sptep) && + spte_to_child_sp(*sptep)->gfn == gfn && + spte_to_child_sp(*sptep)->role.word == role.word) return ERR_PTR(-EEXIST); - role = kvm_mmu_child_role(sptep, direct, access); return kvm_mmu_get_shadow_page(vcpu, gfn, role); } @@ -6950,13 +6952,19 @@ static bool kvm_mmu_zap_collapsible_spte(struct kvm *kvm, sp = sptep_to_sp(sptep); /* - * We cannot do huge page mapping for indirect shadow pages, - * which are found on the last rmap (level = 1) when not using - * tdp; such shadow pages are synced with the page table in - * the guest, and the guest page table is using 4K page size - * mapping if the indirect sp has level = 1. + * Direct shadow page can be replaced by a hugepage if the host + * mapping level allows it and the memslot maps all of the host + * hugepage. Note! If the memslot maps only part of the + * hugepage, sp->gfn may be below slot->base_gfn, and querying + * the max mapping level would cause an out-of-bounds lpage_info + * access. So the gfn bounds check *must* be done first. + * + * Indirect shadow pages are created when the guest page tables + * are using 4K pages. Since the host mapping is always + * constrained by the page size in the guest, indirect shadow + * pages are never collapsible. */ - if (sp->role.direct && + if (sp->role.direct && is_gfn_in_memslot(slot, sp->gfn) && sp->role.level < kvm_mmu_max_mapping_level(kvm, slot, sp->gfn, PG_LEVEL_NUM)) { kvm_zap_one_rmap_spte(kvm, rmap_head, sptep); diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 115c59c86f44..ccfa4924196a 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -1280,6 +1280,7 @@ static int sev_dbg_crypt(struct kvm *kvm, struct kvm_sev_cmd *argp, bool dec) s_off = vaddr & ~PAGE_MASK; d_off = dst_vaddr & ~PAGE_MASK; len = min_t(size_t, (PAGE_SIZE - s_off), size); + len = min_t(size_t, len, PAGE_SIZE - d_off); if (dec) ret = __sev_dbg_decrypt_user(kvm, @@ -3168,37 +3169,6 @@ void sev_guest_memory_reclaimed(struct kvm *kvm) wbinvd_on_all_cpus(); } -void sev_free_vcpu(struct kvm_vcpu *vcpu) -{ - struct vcpu_svm *svm; - - if (!sev_es_guest(vcpu->kvm)) - return; - - svm = to_svm(vcpu); - - /* - * If it's an SNP guest, then the VMSA was marked in the RMP table as - * a guest-owned page. Transition the page to hypervisor state before - * releasing it back to the system. - */ - if (sev_snp_guest(vcpu->kvm)) { - u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT; - - if (kvm_rmp_make_shared(vcpu->kvm, pfn, PG_LEVEL_4K)) - goto skip_vmsa_free; - } - - if (vcpu->arch.guest_state_protected) - sev_flush_encrypted_page(vcpu, svm->sev_es.vmsa); - - __free_page(virt_to_page(svm->sev_es.vmsa)); - -skip_vmsa_free: - if (svm->sev_es.ghcb_sa_free) - kvfree(svm->sev_es.ghcb_sa); -} - static void dump_ghcb(struct vcpu_svm *svm) { struct ghcb *ghcb = svm->sev_es.ghcb; @@ -3443,6 +3413,20 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm) return 1; } +static void __sev_es_unmap_ghcb(struct vcpu_svm *svm) +{ + if (svm->sev_es.ghcb_sa_free) { + kvfree(svm->sev_es.ghcb_sa); + svm->sev_es.ghcb_sa = NULL; + svm->sev_es.ghcb_sa_free = false; + } + + if (svm->sev_es.ghcb) { + kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map, true); + svm->sev_es.ghcb = NULL; + } +} + void sev_es_unmap_ghcb(struct vcpu_svm *svm) { /* Clear any indication that the vCPU is in a type of AP Reset Hold */ @@ -3461,18 +3445,41 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm) svm->sev_es.ghcb_sa_sync = false; } - if (svm->sev_es.ghcb_sa_free) { - kvfree(svm->sev_es.ghcb_sa); - svm->sev_es.ghcb_sa = NULL; - svm->sev_es.ghcb_sa_free = false; - } - trace_kvm_vmgexit_exit(svm->vcpu.vcpu_id, svm->sev_es.ghcb); sev_es_sync_to_ghcb(svm); - kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map, true); - svm->sev_es.ghcb = NULL; + __sev_es_unmap_ghcb(svm); +} + +void sev_free_vcpu(struct kvm_vcpu *vcpu) +{ + struct vcpu_svm *svm; + + if (!sev_es_guest(vcpu->kvm)) + return; + + svm = to_svm(vcpu); + + /* + * If it's an SNP guest, then the VMSA was marked in the RMP table as + * a guest-owned page. Transition the page to hypervisor state before + * releasing it back to the system. + */ + if (sev_snp_guest(vcpu->kvm)) { + u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT; + + if (kvm_rmp_make_shared(vcpu->kvm, pfn, PG_LEVEL_4K)) + goto skip_vmsa_free; + } + + if (vcpu->arch.guest_state_protected) + sev_flush_encrypted_page(vcpu, svm->sev_es.vmsa); + + __free_page(virt_to_page(svm->sev_es.vmsa)); + +skip_vmsa_free: + __sev_es_unmap_ghcb(svm); } void pre_sev_run(struct vcpu_svm *svm, int cpu) @@ -4347,6 +4354,16 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu) exit_code = kvm_ghcb_get_sw_exit_code(control); switch (exit_code) { case SVM_VMGEXIT_MMIO_READ: + if (!control->exit_info_2) + return 1; + + if (to_kvm_sev_info(vcpu->kvm)->ghcb_version >= 2 && + control->exit_info_2 > 8) { + ghcb_set_sw_exit_info_1(svm->sev_es.ghcb, 2); + ghcb_set_sw_exit_info_2(svm->sev_es.ghcb, GHCB_ERR_INVALID_INPUT); + return 1; + } + ret = setup_vmgexit_scratch(svm, true, control->exit_info_2); if (ret) break; @@ -4357,6 +4374,16 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu) svm->sev_es.ghcb_sa); break; case SVM_VMGEXIT_MMIO_WRITE: + if (!control->exit_info_2) + return 1; + + if (to_kvm_sev_info(vcpu->kvm)->ghcb_version >= 2 && + control->exit_info_2 > 8) { + ghcb_set_sw_exit_info_1(svm->sev_es.ghcb, 2); + ghcb_set_sw_exit_info_2(svm->sev_es.ghcb, GHCB_ERR_INVALID_INPUT); + return 1; + } + ret = setup_vmgexit_scratch(svm, false, control->exit_info_2); if (ret) break; @@ -4439,6 +4466,11 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu) control->exit_info_1, control->exit_info_2); ret = -EINVAL; break; + case SVM_EXIT_IOIO: + if (!((control->exit_info_1 & SVM_IOIO_SIZE_MASK) >> SVM_IOIO_SIZE_SHIFT)) + return 1; + + fallthrough; default: ret = svm_invoke_exit_handler(vcpu, exit_code); } @@ -4459,6 +4491,9 @@ int sev_es_string_io(struct vcpu_svm *svm, int size, unsigned int port, int in) if (unlikely(check_mul_overflow(count, size, &bytes))) return -EINVAL; + if (!bytes) + return 1; + r = setup_vmgexit_scratch(svm, in, bytes); if (r) return r; diff --git a/block/bdev.c b/block/bdev.c index e7daca6565ea..ea4268a308cc 100644 --- a/block/bdev.c +++ b/block/bdev.c @@ -414,15 +414,10 @@ EXPORT_SYMBOL_GPL(blockdev_superblock); void __init bdev_cache_init(void) { - int err; - bdev_cachep = kmem_cache_create("bdev_cache", sizeof(struct bdev_inode), 0, (SLAB_HWCACHE_ALIGN|SLAB_RECLAIM_ACCOUNT| SLAB_ACCOUNT|SLAB_PANIC), init_once); - err = register_filesystem(&bd_type); - if (err) - panic("Cannot register bdev pseudo-fs"); blockdev_mnt = kern_mount(&bd_type); if (IS_ERR(blockdev_mnt)) panic("Cannot create bdev pseudo-fs"); diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index a0fbb427a7a6..d95104f0fcdc 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -164,20 +164,10 @@ static void blkg_free(struct blkcg_gq *blkg) static void __blkg_release(struct rcu_head *rcu) { struct blkcg_gq *blkg = container_of(rcu, struct blkcg_gq, rcu_head); - struct blkcg *blkcg = blkg->blkcg; - int cpu; #ifdef CONFIG_BLK_CGROUP_PUNT_BIO WARN_ON(!bio_list_empty(&blkg->async_bios)); #endif - /* - * Flush all the non-empty percpu lockless lists before releasing - * us, given these stat belongs to us. - * - * blkg_stat_lock is for serializing blkg stat update - */ - for_each_possible_cpu(cpu) - __blkcg_rstat_flush(blkcg, cpu); /* release the blkcg and parent blkg refs this blkg has been holding */ css_put(&blkg->blkcg->css); @@ -195,6 +185,17 @@ static void __blkg_release(struct rcu_head *rcu) static void blkg_release(struct percpu_ref *ref) { struct blkcg_gq *blkg = container_of(ref, struct blkcg_gq, refcnt); + struct blkcg *blkcg = blkg->blkcg; + int cpu; + + /* + * Flush all the non-empty percpu lockless lists before releasing + * us, given these stat belongs to us. + * + * blkg_stat_lock is for serializing blkg stat update + */ + for_each_possible_cpu(cpu) + __blkcg_rstat_flush(blkcg, cpu); call_rcu(&blkg->rcu_head, __blkg_release); } diff --git a/drivers/acpi/scan.c b/drivers/acpi/scan.c index 0774cc692110..4b9c81fbdc3b 100644 --- a/drivers/acpi/scan.c +++ b/drivers/acpi/scan.c @@ -5,6 +5,7 @@ #define pr_fmt(fmt) "ACPI: " fmt +#include #include #include #include @@ -2364,46 +2365,34 @@ static int acpi_dev_get_next_consumer_dev_cb(struct acpi_dep_data *dep, void *da return 0; } -struct acpi_scan_clear_dep_work { - struct work_struct work; - struct acpi_device *adev; -}; - -static void acpi_scan_clear_dep_fn(struct work_struct *work) +static void acpi_scan_clear_dep_fn(void *dev, async_cookie_t cookie) { - struct acpi_scan_clear_dep_work *cdw; - - cdw = container_of(work, struct acpi_scan_clear_dep_work, work); + struct acpi_device *adev = to_acpi_device(dev); acpi_scan_lock_acquire(); - acpi_bus_attach(cdw->adev, (void *)true); + acpi_bus_attach(adev, (void *)true); acpi_scan_lock_release(); - acpi_dev_put(cdw->adev); - kfree(cdw); + acpi_dev_put(adev); } static bool acpi_scan_clear_dep_queue(struct acpi_device *adev) { - struct acpi_scan_clear_dep_work *cdw; - if (adev->dep_unmet) return false; - cdw = kmalloc(sizeof(*cdw), GFP_KERNEL); - if (!cdw) - return false; - - cdw->adev = adev; - INIT_WORK(&cdw->work, acpi_scan_clear_dep_fn); /* - * Since the work function may block on the lock until the entire - * initial enumeration of devices is complete, put it into the unbound - * workqueue. + * Async schedule the deferred acpi_scan_clear_dep_fn() since: + * - acpi_bus_attach() needs to hold acpi_scan_lock which cannot + * be acquired under acpi_dep_list_lock (held here) + * - the deferred work at boot stage is ensured to be finished + * before userspace init task by the async_synchronize_full() + * barrier + * + * Use _nocall variant since it'll return on failure instead of + * run the function synchronously. */ - queue_work(system_unbound_wq, &cdw->work); - - return true; + return async_schedule_dev_nocall(acpi_scan_clear_dep_fn, &adev->dev); } static void acpi_scan_delete_dep_data(struct acpi_dep_data *dep) diff --git a/drivers/base/memory.c b/drivers/base/memory.c index 948e77da1dc3..2f7e8bab3d10 100644 --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -794,7 +794,6 @@ static int add_memory_block(unsigned long block_id, unsigned long state, mem->start_section_nr = block_id * sections_per_block; mem->state = state; mem->nid = NUMA_NO_NODE; - mem->altmap = altmap; INIT_LIST_HEAD(&mem->group_next); #ifndef CONFIG_NUMA @@ -812,6 +811,8 @@ static int add_memory_block(unsigned long block_id, unsigned long state, if (ret) return ret; + mem->altmap = altmap; + if (group) { mem->group = group; list_add(&mem->group_next, &group->memory_blocks); diff --git a/drivers/char/agp/amd64-agp.c b/drivers/char/agp/amd64-agp.c index 8e41731d3642..c9d7cefa5192 100644 --- a/drivers/char/agp/amd64-agp.c +++ b/drivers/char/agp/amd64-agp.c @@ -546,7 +546,7 @@ static int agp_amd64_probe(struct pci_dev *pdev, /* Fill in the mode register */ pci_read_config_dword(pdev, bridge->capndx+PCI_AGP_STATUS, &bridge->mode); - if (cache_nbs(pdev, cap_ptr) == -1) { + if (cache_nbs(pdev, cap_ptr) < 0) { agp_put_bridge(bridge); return -ENODEV; } diff --git a/drivers/crypto/intel/qat/qat_common/adf_cfg.c b/drivers/crypto/intel/qat/qat_common/adf_cfg.c index b0fc453fa3fb..97273da40b43 100644 --- a/drivers/crypto/intel/qat/qat_common/adf_cfg.c +++ b/drivers/crypto/intel/qat/qat_common/adf_cfg.c @@ -103,16 +103,6 @@ static void adf_cfg_section_del_all(struct list_head *head); static void adf_cfg_section_del_all_except(struct list_head *head, const char *section_name); -void adf_cfg_del_all(struct adf_accel_dev *accel_dev) -{ - struct adf_cfg_device_data *dev_cfg_data = accel_dev->cfg; - - down_write(&dev_cfg_data->lock); - adf_cfg_section_del_all(&dev_cfg_data->sec_list); - up_write(&dev_cfg_data->lock); - clear_bit(ADF_STATUS_CONFIGURED, &accel_dev->status); -} - void adf_cfg_del_all_except(struct adf_accel_dev *accel_dev, const char *section_name) { diff --git a/drivers/crypto/intel/qat/qat_common/adf_cfg.h b/drivers/crypto/intel/qat/qat_common/adf_cfg.h index 2afa6f0d15c5..108032b39242 100644 --- a/drivers/crypto/intel/qat/qat_common/adf_cfg.h +++ b/drivers/crypto/intel/qat/qat_common/adf_cfg.h @@ -34,7 +34,6 @@ void adf_cfg_dev_remove(struct adf_accel_dev *accel_dev); void adf_cfg_dev_dbgfs_add(struct adf_accel_dev *accel_dev); void adf_cfg_dev_dbgfs_rm(struct adf_accel_dev *accel_dev); int adf_cfg_section_add(struct adf_accel_dev *accel_dev, const char *name); -void adf_cfg_del_all(struct adf_accel_dev *accel_dev); void adf_cfg_del_all_except(struct adf_accel_dev *accel_dev, const char *section_name); int adf_cfg_add_key_value_param(struct adf_accel_dev *accel_dev, diff --git a/drivers/crypto/intel/qat/qat_common/adf_cfg_common.h b/drivers/crypto/intel/qat/qat_common/adf_cfg_common.h index 89df3888d7ea..e642438b381d 100644 --- a/drivers/crypto/intel/qat/qat_common/adf_cfg_common.h +++ b/drivers/crypto/intel/qat/qat_common/adf_cfg_common.h @@ -4,18 +4,11 @@ #define ADF_CFG_COMMON_H_ #include -#include #define ADF_CFG_MAX_STR_LEN 64 #define ADF_CFG_MAX_KEY_LEN_IN_BYTES ADF_CFG_MAX_STR_LEN #define ADF_CFG_MAX_VAL_LEN_IN_BYTES ADF_CFG_MAX_STR_LEN #define ADF_CFG_MAX_SECTION_LEN_IN_BYTES ADF_CFG_MAX_STR_LEN -#define ADF_CFG_BASE_DEC 10 -#define ADF_CFG_BASE_HEX 16 -#define ADF_CFG_ALL_DEVICES 0xFE -#define ADF_CFG_NO_DEVICE 0xFF -#define ADF_CFG_AFFINITY_WHATEVER 0xFF -#define MAX_DEVICE_NAME_SIZE 32 #define ADF_MAX_DEVICES (32 * 32) #define ADF_DEVS_ARRAY_SIZE BITS_TO_LONGS(ADF_MAX_DEVICES) @@ -49,29 +42,4 @@ enum adf_device_type { DEV_4XXX, DEV_420XX, }; - -struct adf_dev_status_info { - enum adf_device_type type; - __u32 accel_id; - __u32 instance_id; - __u8 num_ae; - __u8 num_accel; - __u8 num_logical_accel; - __u8 banks_per_accel; - __u8 state; - __u8 bus; - __u8 dev; - __u8 fun; - char name[MAX_DEVICE_NAME_SIZE]; -}; - -#define ADF_CTL_IOC_MAGIC 'a' -#define IOCTL_CONFIG_SYS_RESOURCE_PARAMETERS _IOW(ADF_CTL_IOC_MAGIC, 0, \ - struct adf_user_cfg_ctl_data) -#define IOCTL_STOP_ACCEL_DEV _IOW(ADF_CTL_IOC_MAGIC, 1, \ - struct adf_user_cfg_ctl_data) -#define IOCTL_START_ACCEL_DEV _IOW(ADF_CTL_IOC_MAGIC, 2, \ - struct adf_user_cfg_ctl_data) -#define IOCTL_STATUS_ACCEL_DEV _IOW(ADF_CTL_IOC_MAGIC, 3, __u32) -#define IOCTL_GET_NUM_DEVICES _IOW(ADF_CTL_IOC_MAGIC, 4, __s32) #endif diff --git a/drivers/crypto/intel/qat/qat_common/adf_cfg_user.h b/drivers/crypto/intel/qat/qat_common/adf_cfg_user.h deleted file mode 100644 index 421f4fb8b4dd..000000000000 --- a/drivers/crypto/intel/qat/qat_common/adf_cfg_user.h +++ /dev/null @@ -1,38 +0,0 @@ -/* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0-only) */ -/* Copyright(c) 2014 - 2020 Intel Corporation */ -#ifndef ADF_CFG_USER_H_ -#define ADF_CFG_USER_H_ - -#include "adf_cfg_common.h" -#include "adf_cfg_strings.h" - -struct adf_user_cfg_key_val { - char key[ADF_CFG_MAX_KEY_LEN_IN_BYTES]; - char val[ADF_CFG_MAX_VAL_LEN_IN_BYTES]; - union { - struct adf_user_cfg_key_val *next; - __u64 padding3; - }; - enum adf_cfg_val_type type; -} __packed; - -struct adf_user_cfg_section { - char name[ADF_CFG_MAX_SECTION_LEN_IN_BYTES]; - union { - struct adf_user_cfg_key_val *params; - __u64 padding1; - }; - union { - struct adf_user_cfg_section *next; - __u64 padding3; - }; -} __packed; - -struct adf_user_cfg_ctl_data { - union { - struct adf_user_cfg_section *config_section; - __u64 padding; - }; - __u8 device_id; -} __packed; -#endif diff --git a/drivers/crypto/intel/qat/qat_common/adf_common_drv.h b/drivers/crypto/intel/qat/qat_common/adf_common_drv.h index 25c940b06c36..fc2fea0d3ada 100644 --- a/drivers/crypto/intel/qat/qat_common/adf_common_drv.h +++ b/drivers/crypto/intel/qat/qat_common/adf_common_drv.h @@ -68,11 +68,8 @@ int adf_devmgr_add_dev(struct adf_accel_dev *accel_dev, void adf_devmgr_rm_dev(struct adf_accel_dev *accel_dev, struct adf_accel_dev *pf); struct list_head *adf_devmgr_get_head(void); -struct adf_accel_dev *adf_devmgr_get_dev_by_id(u32 id); struct adf_accel_dev *adf_devmgr_get_first(void); struct adf_accel_dev *adf_devmgr_pci_to_accel_dev(struct pci_dev *pci_dev); -int adf_devmgr_verify_id(u32 id); -void adf_devmgr_get_num_dev(u32 *num); int adf_devmgr_in_reset(struct adf_accel_dev *accel_dev); int adf_dev_started(struct adf_accel_dev *accel_dev); int adf_dev_restarting_notify(struct adf_accel_dev *accel_dev); diff --git a/drivers/crypto/intel/qat/qat_common/adf_ctl_drv.c b/drivers/crypto/intel/qat/qat_common/adf_ctl_drv.c index 70fa0f6497a9..36d5ebb3efab 100644 --- a/drivers/crypto/intel/qat/qat_common/adf_ctl_drv.c +++ b/drivers/crypto/intel/qat/qat_common/adf_ctl_drv.c @@ -2,422 +2,13 @@ /* Copyright(c) 2014 - 2020 Intel Corporation */ #include +#include #include -#include -#include -#include -#include -#include -#include -#include -#include "adf_accel_devices.h" #include "adf_common_drv.h" -#include "adf_cfg.h" -#include "adf_cfg_common.h" -#include "adf_cfg_user.h" - -#define ADF_CFG_MAX_SECTION 512 -#define ADF_CFG_MAX_KEY_VAL 256 - -#define DEVICE_NAME "qat_adf_ctl" - -static DEFINE_MUTEX(adf_ctl_lock); -static long adf_ctl_ioctl(struct file *fp, unsigned int cmd, unsigned long arg); - -static const struct file_operations adf_ctl_ops = { - .owner = THIS_MODULE, - .unlocked_ioctl = adf_ctl_ioctl, - .compat_ioctl = compat_ptr_ioctl, -}; - -static const struct class adf_ctl_class = { - .name = DEVICE_NAME, -}; - -struct adf_ctl_drv_info { - unsigned int major; - struct cdev drv_cdev; -}; - -static struct adf_ctl_drv_info adf_ctl_drv; - -static void adf_chr_drv_destroy(void) -{ - device_destroy(&adf_ctl_class, MKDEV(adf_ctl_drv.major, 0)); - cdev_del(&adf_ctl_drv.drv_cdev); - class_unregister(&adf_ctl_class); - unregister_chrdev_region(MKDEV(adf_ctl_drv.major, 0), 1); -} - -static int adf_chr_drv_create(void) -{ - dev_t dev_id; - struct device *drv_device; - int ret; - - if (alloc_chrdev_region(&dev_id, 0, 1, DEVICE_NAME)) { - pr_err("QAT: unable to allocate chrdev region\n"); - return -EFAULT; - } - - ret = class_register(&adf_ctl_class); - if (ret) - goto err_chrdev_unreg; - - adf_ctl_drv.major = MAJOR(dev_id); - cdev_init(&adf_ctl_drv.drv_cdev, &adf_ctl_ops); - if (cdev_add(&adf_ctl_drv.drv_cdev, dev_id, 1)) { - pr_err("QAT: cdev add failed\n"); - goto err_class_destr; - } - - drv_device = device_create(&adf_ctl_class, NULL, - MKDEV(adf_ctl_drv.major, 0), - NULL, DEVICE_NAME); - if (IS_ERR(drv_device)) { - pr_err("QAT: failed to create device\n"); - goto err_cdev_del; - } - return 0; -err_cdev_del: - cdev_del(&adf_ctl_drv.drv_cdev); -err_class_destr: - class_unregister(&adf_ctl_class); -err_chrdev_unreg: - unregister_chrdev_region(dev_id, 1); - return -EFAULT; -} - -static int adf_ctl_alloc_resources(struct adf_user_cfg_ctl_data **ctl_data, - unsigned long arg) -{ - struct adf_user_cfg_ctl_data *cfg_data; - - cfg_data = kzalloc(sizeof(*cfg_data), GFP_KERNEL); - if (!cfg_data) - return -ENOMEM; - - /* Initialize device id to NO DEVICE as 0 is a valid device id */ - cfg_data->device_id = ADF_CFG_NO_DEVICE; - - if (copy_from_user(cfg_data, (void __user *)arg, sizeof(*cfg_data))) { - pr_err("QAT: failed to copy from user cfg_data.\n"); - kfree(cfg_data); - return -EIO; - } - - *ctl_data = cfg_data; - return 0; -} - -static int adf_add_key_value_data(struct adf_accel_dev *accel_dev, - const char *section, - const struct adf_user_cfg_key_val *key_val) -{ - if (key_val->type == ADF_HEX) { - long *ptr = (long *)key_val->val; - long val = *ptr; - - if (adf_cfg_add_key_value_param(accel_dev, section, - key_val->key, (void *)val, - key_val->type)) { - dev_err(&GET_DEV(accel_dev), - "failed to add hex keyvalue.\n"); - return -EFAULT; - } - } else { - if (adf_cfg_add_key_value_param(accel_dev, section, - key_val->key, key_val->val, - key_val->type)) { - dev_err(&GET_DEV(accel_dev), - "failed to add keyvalue.\n"); - return -EFAULT; - } - } - return 0; -} - -static int adf_copy_key_value_data(struct adf_accel_dev *accel_dev, - struct adf_user_cfg_ctl_data *ctl_data) -{ - struct adf_user_cfg_key_val key_val; - struct adf_user_cfg_key_val *params_head; - struct adf_user_cfg_section section, *section_head; - int i, j; - - section_head = ctl_data->config_section; - - for (i = 0; section_head && i < ADF_CFG_MAX_SECTION; i++) { - if (copy_from_user(§ion, (void __user *)section_head, - sizeof(*section_head))) { - dev_err(&GET_DEV(accel_dev), - "failed to copy section info\n"); - goto out_err; - } - - if (adf_cfg_section_add(accel_dev, section.name)) { - dev_err(&GET_DEV(accel_dev), - "failed to add section.\n"); - goto out_err; - } - - params_head = section.params; - - for (j = 0; params_head && j < ADF_CFG_MAX_KEY_VAL; j++) { - if (copy_from_user(&key_val, (void __user *)params_head, - sizeof(key_val))) { - dev_err(&GET_DEV(accel_dev), - "Failed to copy keyvalue.\n"); - goto out_err; - } - if (adf_add_key_value_data(accel_dev, section.name, - &key_val)) { - goto out_err; - } - params_head = key_val.next; - } - section_head = section.next; - } - return 0; -out_err: - adf_cfg_del_all(accel_dev); - return -EFAULT; -} - -static int adf_ctl_ioctl_dev_config(struct file *fp, unsigned int cmd, - unsigned long arg) -{ - int ret; - struct adf_user_cfg_ctl_data *ctl_data; - struct adf_accel_dev *accel_dev; - - ret = adf_ctl_alloc_resources(&ctl_data, arg); - if (ret) - return ret; - - accel_dev = adf_devmgr_get_dev_by_id(ctl_data->device_id); - if (!accel_dev) { - ret = -EFAULT; - goto out; - } - - if (adf_dev_started(accel_dev)) { - ret = -EFAULT; - goto out; - } - - if (adf_copy_key_value_data(accel_dev, ctl_data)) { - ret = -EFAULT; - goto out; - } - set_bit(ADF_STATUS_CONFIGURED, &accel_dev->status); -out: - kfree(ctl_data); - return ret; -} - -static int adf_ctl_is_device_in_use(int id) -{ - struct adf_accel_dev *dev; - - list_for_each_entry(dev, adf_devmgr_get_head(), list) { - if (id == dev->accel_id || id == ADF_CFG_ALL_DEVICES) { - if (adf_devmgr_in_reset(dev) || adf_dev_in_use(dev)) { - dev_info(&GET_DEV(dev), - "device qat_dev%d is busy\n", - dev->accel_id); - return -EBUSY; - } - } - } - return 0; -} - -static void adf_ctl_stop_devices(u32 id) -{ - struct adf_accel_dev *accel_dev; - - list_for_each_entry(accel_dev, adf_devmgr_get_head(), list) { - if (id == accel_dev->accel_id || id == ADF_CFG_ALL_DEVICES) { - if (!adf_dev_started(accel_dev)) - continue; - - /* First stop all VFs */ - if (!accel_dev->is_vf) - continue; - - adf_dev_down(accel_dev); - } - } - - list_for_each_entry(accel_dev, adf_devmgr_get_head(), list) { - if (id == accel_dev->accel_id || id == ADF_CFG_ALL_DEVICES) { - if (!adf_dev_started(accel_dev)) - continue; - - adf_dev_down(accel_dev); - } - } -} - -static int adf_ctl_ioctl_dev_stop(struct file *fp, unsigned int cmd, - unsigned long arg) -{ - int ret; - struct adf_user_cfg_ctl_data *ctl_data; - - ret = adf_ctl_alloc_resources(&ctl_data, arg); - if (ret) - return ret; - - if (adf_devmgr_verify_id(ctl_data->device_id)) { - pr_err("QAT: Device %d not found\n", ctl_data->device_id); - ret = -ENODEV; - goto out; - } - - ret = adf_ctl_is_device_in_use(ctl_data->device_id); - if (ret) - goto out; - - if (ctl_data->device_id == ADF_CFG_ALL_DEVICES) - pr_info("QAT: Stopping all acceleration devices.\n"); - else - pr_info("QAT: Stopping acceleration device qat_dev%d.\n", - ctl_data->device_id); - - adf_ctl_stop_devices(ctl_data->device_id); - -out: - kfree(ctl_data); - return ret; -} - -static int adf_ctl_ioctl_dev_start(struct file *fp, unsigned int cmd, - unsigned long arg) -{ - int ret; - struct adf_user_cfg_ctl_data *ctl_data; - struct adf_accel_dev *accel_dev; - - ret = adf_ctl_alloc_resources(&ctl_data, arg); - if (ret) - return ret; - - ret = -ENODEV; - accel_dev = adf_devmgr_get_dev_by_id(ctl_data->device_id); - if (!accel_dev) - goto out; - - dev_info(&GET_DEV(accel_dev), - "Starting acceleration device qat_dev%d.\n", - ctl_data->device_id); - - ret = adf_dev_up(accel_dev, false); - - if (ret) { - dev_err(&GET_DEV(accel_dev), "Failed to start qat_dev%d\n", - ctl_data->device_id); - adf_dev_down(accel_dev); - } -out: - kfree(ctl_data); - return ret; -} - -static int adf_ctl_ioctl_get_num_devices(struct file *fp, unsigned int cmd, - unsigned long arg) -{ - u32 num_devices = 0; - - adf_devmgr_get_num_dev(&num_devices); - if (copy_to_user((void __user *)arg, &num_devices, sizeof(num_devices))) - return -EFAULT; - - return 0; -} - -static int adf_ctl_ioctl_get_status(struct file *fp, unsigned int cmd, - unsigned long arg) -{ - struct adf_hw_device_data *hw_data; - struct adf_dev_status_info dev_info; - struct adf_accel_dev *accel_dev; - - if (copy_from_user(&dev_info, (void __user *)arg, - sizeof(struct adf_dev_status_info))) { - pr_err("QAT: failed to copy from user.\n"); - return -EFAULT; - } - - accel_dev = adf_devmgr_get_dev_by_id(dev_info.accel_id); - if (!accel_dev) - return -ENODEV; - - hw_data = accel_dev->hw_device; - dev_info.state = adf_dev_started(accel_dev) ? DEV_UP : DEV_DOWN; - dev_info.num_ae = hw_data->get_num_aes(hw_data); - dev_info.num_accel = hw_data->get_num_accels(hw_data); - dev_info.num_logical_accel = hw_data->num_logical_accel; - dev_info.banks_per_accel = hw_data->num_banks - / hw_data->num_logical_accel; - strscpy(dev_info.name, hw_data->dev_class->name, sizeof(dev_info.name)); - dev_info.instance_id = hw_data->instance_id; - dev_info.type = hw_data->dev_class->type; - dev_info.bus = accel_to_pci_dev(accel_dev)->bus->number; - dev_info.dev = PCI_SLOT(accel_to_pci_dev(accel_dev)->devfn); - dev_info.fun = PCI_FUNC(accel_to_pci_dev(accel_dev)->devfn); - - if (copy_to_user((void __user *)arg, &dev_info, - sizeof(struct adf_dev_status_info))) { - dev_err(&GET_DEV(accel_dev), "failed to copy status.\n"); - return -EFAULT; - } - return 0; -} - -static long adf_ctl_ioctl(struct file *fp, unsigned int cmd, unsigned long arg) -{ - int ret; - - if (mutex_lock_interruptible(&adf_ctl_lock)) - return -EFAULT; - - switch (cmd) { - case IOCTL_CONFIG_SYS_RESOURCE_PARAMETERS: - ret = adf_ctl_ioctl_dev_config(fp, cmd, arg); - break; - - case IOCTL_STOP_ACCEL_DEV: - ret = adf_ctl_ioctl_dev_stop(fp, cmd, arg); - break; - - case IOCTL_START_ACCEL_DEV: - ret = adf_ctl_ioctl_dev_start(fp, cmd, arg); - break; - - case IOCTL_GET_NUM_DEVICES: - ret = adf_ctl_ioctl_get_num_devices(fp, cmd, arg); - break; - - case IOCTL_STATUS_ACCEL_DEV: - ret = adf_ctl_ioctl_get_status(fp, cmd, arg); - break; - default: - pr_err_ratelimited("QAT: Invalid ioctl %d\n", cmd); - ret = -EFAULT; - break; - } - mutex_unlock(&adf_ctl_lock); - return ret; -} static int __init adf_register_ctl_device_driver(void) { - if (adf_chr_drv_create()) - goto err_chr_dev; - if (adf_init_misc_wq()) goto err_misc_wq; @@ -449,15 +40,11 @@ static int __init adf_register_ctl_device_driver(void) err_aer: adf_exit_misc_wq(); err_misc_wq: - adf_chr_drv_destroy(); -err_chr_dev: - mutex_destroy(&adf_ctl_lock); return -EFAULT; } static void __exit adf_unregister_ctl_device_driver(void) { - adf_chr_drv_destroy(); adf_exit_misc_wq(); adf_exit_aer(); adf_exit_vf_wq(); @@ -465,7 +52,6 @@ static void __exit adf_unregister_ctl_device_driver(void) qat_crypto_unregister(); qat_compression_unregister(); adf_clean_vf_map(false); - mutex_destroy(&adf_ctl_lock); } module_init(adf_register_ctl_device_driver); diff --git a/drivers/crypto/intel/qat/qat_common/adf_dev_mgr.c b/drivers/crypto/intel/qat/qat_common/adf_dev_mgr.c index 96ddd1c419c4..9013a04f425b 100644 --- a/drivers/crypto/intel/qat/qat_common/adf_dev_mgr.c +++ b/drivers/crypto/intel/qat/qat_common/adf_dev_mgr.c @@ -45,19 +45,6 @@ static struct vf_id_map *adf_find_vf(u32 bdf) return NULL; } -static int adf_get_vf_real_id(u32 fake) -{ - struct list_head *itr; - - list_for_each(itr, &vfs_table) { - struct vf_id_map *ptr = - list_entry(itr, struct vf_id_map, list); - if (ptr->fake_id == fake) - return ptr->id; - } - return -1; -} - /** * adf_clean_vf_map() - Cleans VF id mappings * @vf: flag indicating whether mappings is cleaned @@ -314,63 +301,6 @@ struct adf_accel_dev *adf_devmgr_pci_to_accel_dev(struct pci_dev *pci_dev) } EXPORT_SYMBOL_GPL(adf_devmgr_pci_to_accel_dev); -struct adf_accel_dev *adf_devmgr_get_dev_by_id(u32 id) -{ - struct list_head *itr; - int real_id; - - mutex_lock(&table_lock); - real_id = adf_get_vf_real_id(id); - if (real_id < 0) - goto unlock; - - id = real_id; - - list_for_each(itr, &accel_table) { - struct adf_accel_dev *ptr = - list_entry(itr, struct adf_accel_dev, list); - if (ptr->accel_id == id) { - mutex_unlock(&table_lock); - return ptr; - } - } -unlock: - mutex_unlock(&table_lock); - return NULL; -} - -int adf_devmgr_verify_id(u32 id) -{ - if (id == ADF_CFG_ALL_DEVICES) - return 0; - - if (adf_devmgr_get_dev_by_id(id)) - return 0; - - return -ENODEV; -} - -static int adf_get_num_dettached_vfs(void) -{ - struct list_head *itr; - int vfs = 0; - - mutex_lock(&table_lock); - list_for_each(itr, &vfs_table) { - struct vf_id_map *ptr = - list_entry(itr, struct vf_id_map, list); - if (ptr->bdf != ~0 && !ptr->attached) - vfs++; - } - mutex_unlock(&table_lock); - return vfs; -} - -void adf_devmgr_get_num_dev(u32 *num) -{ - *num = num_devices - adf_get_num_dettached_vfs(); -} - /** * adf_dev_in_use() - Check whether accel_dev is currently in use * @accel_dev: Pointer to acceleration device. diff --git a/drivers/fpga/of-fpga-region.c b/drivers/fpga/of-fpga-region.c index efc35dad6aaa..cbf6cece2f80 100644 --- a/drivers/fpga/of-fpga-region.c +++ b/drivers/fpga/of-fpga-region.c @@ -168,11 +168,10 @@ static int child_regions_with_firmware(struct device_node *overlay) fpga_region_of_match); } - of_node_put(child_region); - if (ret) pr_err("firmware-name not allowed in child FPGA region: %pOF", child_region); + of_node_put(child_region); return ret; } diff --git a/drivers/gpio/gpiolib.c b/drivers/gpio/gpiolib.c index 5c8cd8165696..878f9ab4a098 100644 --- a/drivers/gpio/gpiolib.c +++ b/drivers/gpio/gpiolib.c @@ -785,13 +785,15 @@ static const struct device_type gpio_dev_type = { #define gcdev_unregister(gdev) device_del(&(gdev)->dev) #endif +/* + * An initial reference count has been held in gpiochip_add_data_with_key(). + * The caller should drop the reference via gpio_device_put() on errors. + */ static int gpiochip_setup_dev(struct gpio_device *gdev) { struct fwnode_handle *fwnode = dev_fwnode(&gdev->dev); int ret; - device_initialize(&gdev->dev); - /* * If fwnode doesn't belong to another device, it's safe to clear its * initialized flag. @@ -859,9 +861,11 @@ static void gpiochip_setup_devs(void) list_for_each_entry_srcu(gdev, &gpio_devices, list, srcu_read_lock_held(&gpio_devices_srcu)) { ret = gpiochip_setup_dev(gdev); - if (ret) + if (ret) { + gpio_device_put(gdev); dev_err(&gdev->dev, "Failed to initialize gpio device (%d)\n", ret); + } } } @@ -883,6 +887,21 @@ void *gpiochip_get_data(struct gpio_chip *gc) } EXPORT_SYMBOL_GPL(gpiochip_get_data); +/* + * If the calling driver provides the specific firmware node, + * use it. Otherwise use the one from the parent device, if any. + */ +static struct fwnode_handle *gpiochip_choose_fwnode(struct gpio_chip *gc) +{ + if (gc->fwnode) + return gc->fwnode; + + if (gc->parent) + return dev_fwnode(gc->parent); + + return NULL; +} + int gpiochip_get_ngpios(struct gpio_chip *gc, struct device *dev) { u32 ngpios = gc->ngpio; @@ -924,80 +943,73 @@ int gpiochip_add_data_with_key(struct gpio_chip *gc, void *data, struct gpio_device *gdev; unsigned int desc_index; int base = 0; - int ret = 0; + int ret; - /* - * First: allocate and populate the internal stat container, and - * set up the struct device. - */ gdev = kzalloc(sizeof(*gdev), GFP_KERNEL); if (!gdev) return -ENOMEM; - - gdev->dev.type = &gpio_dev_type; - gdev->dev.bus = &gpio_bus_type; - gdev->dev.parent = gc->parent; - rcu_assign_pointer(gdev->chip, gc); - gc->gpiodev = gdev; gpiochip_set_data(gc, data); - /* - * If the calling driver did not initialize firmware node, - * do it here using the parent device, if any. - */ - if (gc->fwnode) - device_set_node(&gdev->dev, gc->fwnode); - else if (gc->parent) - device_set_node(&gdev->dev, dev_fwnode(gc->parent)); - - gdev->id = ida_alloc(&gpio_ida, GFP_KERNEL); - if (gdev->id < 0) { - ret = gdev->id; + ret = ida_alloc(&gpio_ida, GFP_KERNEL); + if (ret < 0) goto err_free_gdev; - } + gdev->id = ret; - ret = dev_set_name(&gdev->dev, GPIOCHIP_NAME "%d", gdev->id); + ret = init_srcu_struct(&gdev->srcu); if (ret) goto err_free_ida; + rcu_assign_pointer(gdev->chip, gc); - if (gc->parent && gc->parent->driver) - gdev->owner = gc->parent->driver->owner; - else if (gc->owner) - /* TODO: remove chip->owner */ - gdev->owner = gc->owner; - else - gdev->owner = THIS_MODULE; + ret = init_srcu_struct(&gdev->desc_srcu); + if (ret) + goto err_cleanup_gdev_srcu; + + ret = dev_set_name(&gdev->dev, GPIOCHIP_NAME "%d", gdev->id); + if (ret) + goto err_cleanup_desc_srcu; + + device_initialize(&gdev->dev); + /* + * After this point any allocated resources to `gdev` will be + * free():ed by gpiodev_release(). If you add new resources + * then make sure they get free():ed there. + */ + gdev->dev.type = &gpio_dev_type; + gdev->dev.bus = &gpio_bus_type; + gdev->dev.parent = gc->parent; + device_set_node(&gdev->dev, gpiochip_choose_fwnode(gc)); ret = gpiochip_get_ngpios(gc, &gdev->dev); if (ret) - goto err_free_dev_name; + goto err_put_device; + gdev->ngpio = gc->ngpio; gdev->descs = kcalloc(gc->ngpio, sizeof(*gdev->descs), GFP_KERNEL); if (!gdev->descs) { ret = -ENOMEM; - goto err_free_dev_name; + goto err_put_device; } gdev->label = kstrdup_const(gc->label ?: "unknown", GFP_KERNEL); if (!gdev->label) { ret = -ENOMEM; - goto err_free_descs; + goto err_put_device; } - gdev->ngpio = gc->ngpio; gdev->can_sleep = gc->can_sleep; - BLOCKING_INIT_NOTIFIER_HEAD(&gdev->line_state_notifier); BLOCKING_INIT_NOTIFIER_HEAD(&gdev->device_notifier); - - ret = init_srcu_struct(&gdev->srcu); - if (ret) - goto err_free_label; - - ret = init_srcu_struct(&gdev->desc_srcu); - if (ret) - goto err_cleanup_gdev_srcu; +#ifdef CONFIG_PINCTRL + INIT_LIST_HEAD(&gdev->pin_ranges); +#endif + if (gc->parent && gc->parent->driver) + gdev->owner = gc->parent->driver->owner; + else if (gc->owner) + /* TODO: remove chip->owner */ + gdev->owner = gc->owner; + else + gdev->owner = THIS_MODULE; scoped_guard(mutex, &gpio_devices_lock) { /* @@ -1013,7 +1025,7 @@ int gpiochip_add_data_with_key(struct gpio_chip *gc, void *data, if (base < 0) { ret = base; base = 0; - goto err_cleanup_desc_srcu; + goto err_put_device; } /* @@ -1033,14 +1045,10 @@ int gpiochip_add_data_with_key(struct gpio_chip *gc, void *data, ret = gpiodev_add_to_list_unlocked(gdev); if (ret) { chip_err(gc, "GPIO integer space overlap, cannot add chip\n"); - goto err_cleanup_desc_srcu; + goto err_put_device; } } -#ifdef CONFIG_PINCTRL - INIT_LIST_HEAD(&gdev->pin_ranges); -#endif - if (gc->names) gpiochip_set_desc_names(gc); @@ -1121,25 +1129,19 @@ int gpiochip_add_data_with_key(struct gpio_chip *gc, void *data, scoped_guard(mutex, &gpio_devices_lock) list_del_rcu(&gdev->list); synchronize_srcu(&gpio_devices_srcu); - if (gdev->dev.release) { - /* release() has been registered by gpiochip_setup_dev() */ - gpio_device_put(gdev); - goto err_print_message; - } +err_put_device: + gpio_device_put(gdev); + goto err_print_message; + err_cleanup_desc_srcu: cleanup_srcu_struct(&gdev->desc_srcu); err_cleanup_gdev_srcu: cleanup_srcu_struct(&gdev->srcu); -err_free_label: - kfree_const(gdev->label); -err_free_descs: - kfree(gdev->descs); -err_free_dev_name: - kfree(dev_name(&gdev->dev)); err_free_ida: ida_free(&gpio_ida, gdev->id); err_free_gdev: kfree(gdev); + err_print_message: /* failures here can mean systems won't boot... */ if (ret != -EPROBE_DEFER) { @@ -2874,7 +2876,7 @@ EXPORT_SYMBOL_GPL(gpiod_direction_output); */ int gpiod_enable_hw_timestamp_ns(struct gpio_desc *desc, unsigned long flags) { - int ret = 0; + int ret; VALIDATE_DESC(desc); @@ -2907,7 +2909,7 @@ EXPORT_SYMBOL_GPL(gpiod_enable_hw_timestamp_ns); */ int gpiod_disable_hw_timestamp_ns(struct gpio_desc *desc, unsigned long flags) { - int ret = 0; + int ret; VALIDATE_DESC(desc); diff --git a/drivers/gpu/drm/v3d/v3d_drv.h b/drivers/gpu/drm/v3d/v3d_drv.h index d4b0549205c2..b6e11968fba4 100644 --- a/drivers/gpu/drm/v3d/v3d_drv.h +++ b/drivers/gpu/drm/v3d/v3d_drv.h @@ -59,6 +59,9 @@ struct v3d_queue_state { /* Stores the GPU stats for this queue in the global context. */ struct v3d_stats stats; + + /* Currently active job for this queue */ + struct v3d_job *active_job; }; /* Performance monitor object. The perform lifetime is controlled by userspace @@ -147,10 +150,6 @@ struct v3d_dev { struct work_struct overflow_mem_work; - struct v3d_bin_job *bin_job; - struct v3d_render_job *render_job; - struct v3d_tfu_job *tfu_job; - struct v3d_csd_job *csd_job; struct v3d_cpu_job *cpu_job; struct v3d_queue_state queue[V3D_MAX_QUEUES]; diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c index 6b6ba7a68fcb..cf3b93101429 100644 --- a/drivers/gpu/drm/v3d/v3d_gem.c +++ b/drivers/gpu/drm/v3d/v3d_gem.c @@ -304,16 +304,15 @@ void v3d_gem_destroy(struct drm_device *dev) { struct v3d_dev *v3d = to_v3d_dev(dev); + enum v3d_queue q; v3d_sched_fini(v3d); /* Waiting for jobs to finish would need to be done before * unregistering V3D. */ - WARN_ON(v3d->bin_job); - WARN_ON(v3d->render_job); - WARN_ON(v3d->tfu_job); - WARN_ON(v3d->csd_job); + for (q = 0; q < V3D_MAX_QUEUES; q++) + WARN_ON(v3d->queue[q].active_job); drm_mm_takedown(&v3d->mm); diff --git a/drivers/gpu/drm/v3d/v3d_irq.c b/drivers/gpu/drm/v3d/v3d_irq.c index b98e1a4b33c7..2464ea4d935d 100644 --- a/drivers/gpu/drm/v3d/v3d_irq.c +++ b/drivers/gpu/drm/v3d/v3d_irq.c @@ -42,6 +42,8 @@ v3d_overflow_mem_work(struct work_struct *work) container_of(work, struct v3d_dev, overflow_mem_work); struct drm_device *dev = &v3d->drm; struct v3d_bo *bo = v3d_bo_create(dev, NULL /* XXX: GMP */, 256 * 1024); + struct v3d_queue_state *queue = &v3d->queue[V3D_BIN]; + struct v3d_bin_job *bin_job; struct drm_gem_object *obj; unsigned long irqflags; @@ -61,13 +63,15 @@ v3d_overflow_mem_work(struct work_struct *work) * some binner pool anyway. */ spin_lock_irqsave(&v3d->job_lock, irqflags); - if (!v3d->bin_job) { + bin_job = (struct v3d_bin_job *)queue->active_job; + + if (!bin_job) { spin_unlock_irqrestore(&v3d->job_lock, irqflags); goto out; } drm_gem_object_get(obj); - list_add_tail(&bo->unref_head, &v3d->bin_job->render->unref_list); + list_add_tail(&bo->unref_head, &bin_job->render->unref_list); spin_unlock_irqrestore(&v3d->job_lock, irqflags); v3d_mmu_flush_all(v3d); @@ -79,6 +83,20 @@ v3d_overflow_mem_work(struct work_struct *work) drm_gem_object_put(obj); } +static void +v3d_irq_signal_fence(struct v3d_dev *v3d, enum v3d_queue q, + void (*trace_irq)(struct drm_device *, uint64_t)) +{ + struct v3d_queue_state *queue = &v3d->queue[q]; + struct v3d_fence *fence = to_v3d_fence(queue->active_job->irq_fence); + + v3d_job_update_stats(queue->active_job, q); + trace_irq(&v3d->drm, fence->seqno); + + queue->active_job = NULL; + dma_fence_signal(&fence->base); +} + static irqreturn_t v3d_irq(int irq, void *arg) { @@ -102,41 +120,17 @@ v3d_irq(int irq, void *arg) } if (intsts & V3D_INT_FLDONE) { - struct v3d_fence *fence = - to_v3d_fence(v3d->bin_job->base.irq_fence); - - v3d_job_update_stats(&v3d->bin_job->base, V3D_BIN); - trace_v3d_bcl_irq(&v3d->drm, fence->seqno); - - v3d->bin_job = NULL; - dma_fence_signal(&fence->base); - + v3d_irq_signal_fence(v3d, V3D_BIN, trace_v3d_bcl_irq); status = IRQ_HANDLED; } if (intsts & V3D_INT_FRDONE) { - struct v3d_fence *fence = - to_v3d_fence(v3d->render_job->base.irq_fence); - - v3d_job_update_stats(&v3d->render_job->base, V3D_RENDER); - trace_v3d_rcl_irq(&v3d->drm, fence->seqno); - - v3d->render_job = NULL; - dma_fence_signal(&fence->base); - + v3d_irq_signal_fence(v3d, V3D_RENDER, trace_v3d_rcl_irq); status = IRQ_HANDLED; } if (intsts & V3D_INT_CSDDONE(v3d->ver)) { - struct v3d_fence *fence = - to_v3d_fence(v3d->csd_job->base.irq_fence); - - v3d_job_update_stats(&v3d->csd_job->base, V3D_CSD); - trace_v3d_csd_irq(&v3d->drm, fence->seqno); - - v3d->csd_job = NULL; - dma_fence_signal(&fence->base); - + v3d_irq_signal_fence(v3d, V3D_CSD, trace_v3d_csd_irq); status = IRQ_HANDLED; } @@ -168,15 +162,7 @@ v3d_hub_irq(int irq, void *arg) V3D_WRITE(V3D_HUB_INT_CLR, intsts); if (intsts & V3D_HUB_INT_TFUC) { - struct v3d_fence *fence = - to_v3d_fence(v3d->tfu_job->base.irq_fence); - - v3d_job_update_stats(&v3d->tfu_job->base, V3D_TFU); - trace_v3d_tfu_irq(&v3d->drm, fence->seqno); - - v3d->tfu_job = NULL; - dma_fence_signal(&fence->base); - + v3d_irq_signal_fence(v3d, V3D_TFU, trace_v3d_tfu_irq); status = IRQ_HANDLED; } diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c index 8e3c8ffc2a42..f2bac920af89 100644 --- a/drivers/gpu/drm/v3d/v3d_sched.c +++ b/drivers/gpu/drm/v3d/v3d_sched.c @@ -208,14 +208,18 @@ static struct dma_fence *v3d_bin_job_run(struct drm_sched_job *sched_job) struct dma_fence *fence; unsigned long irqflags; - if (unlikely(job->base.base.s_fence->finished.error)) + if (unlikely(job->base.base.s_fence->finished.error)) { + spin_lock_irqsave(&v3d->job_lock, irqflags); + v3d->queue[V3D_BIN].active_job = NULL; + spin_unlock_irqrestore(&v3d->job_lock, irqflags); return NULL; + } /* Lock required around bin_job update vs * v3d_overflow_mem_work(). */ spin_lock_irqsave(&v3d->job_lock, irqflags); - v3d->bin_job = job; + v3d->queue[V3D_BIN].active_job = &job->base; /* Clear out the overflow allocation, so we don't * reuse the overflow attached to a previous job. */ @@ -263,10 +267,12 @@ static struct dma_fence *v3d_render_job_run(struct drm_sched_job *sched_job) struct drm_device *dev = &v3d->drm; struct dma_fence *fence; - if (unlikely(job->base.base.s_fence->finished.error)) + if (unlikely(job->base.base.s_fence->finished.error)) { + v3d->queue[V3D_RENDER].active_job = NULL; return NULL; + } - v3d->render_job = job; + v3d->queue[V3D_RENDER].active_job = &job->base; /* Can we avoid this flush? We need to be careful of * scheduling, though -- imagine job0 rendering to texture and @@ -309,10 +315,12 @@ v3d_tfu_job_run(struct drm_sched_job *sched_job) struct drm_device *dev = &v3d->drm; struct dma_fence *fence; - if (unlikely(job->base.base.s_fence->finished.error)) + if (unlikely(job->base.base.s_fence->finished.error)) { + v3d->queue[V3D_TFU].active_job = NULL; return NULL; + } - v3d->tfu_job = job; + v3d->queue[V3D_TFU].active_job = &job->base; fence = v3d_fence_create(v3d, V3D_TFU); if (IS_ERR(fence)) @@ -355,10 +363,22 @@ v3d_csd_job_run(struct drm_sched_job *sched_job) struct dma_fence *fence; int i, csd_cfg0_reg; - if (unlikely(job->base.base.s_fence->finished.error)) + if (unlikely(job->base.base.s_fence->finished.error)) { + v3d->queue[V3D_CSD].active_job = NULL; + return NULL; + } + + /* The HW interprets a workgroup size of 0 as 65536; however, the + * user-space driver exposes a maximum of 65535. Therefore, a 0 in + * any dimension means that we have no workgroups and the compute + * shader should not be dispatched. + */ + if (!V3D_GET_FIELD(job->args.cfg[0], V3D_CSD_QUEUED_CFG0_NUM_WGS_X) || + !V3D_GET_FIELD(job->args.cfg[1], V3D_CSD_QUEUED_CFG1_NUM_WGS_Y) || + !V3D_GET_FIELD(job->args.cfg[2], V3D_CSD_QUEUED_CFG2_NUM_WGS_Z)) return NULL; - v3d->csd_job = job; + v3d->queue[V3D_CSD].active_job = &job->base; v3d_invalidate_caches(v3d); @@ -408,13 +428,13 @@ v3d_rewrite_csd_job_wg_counts_from_indirect(struct v3d_cpu_job *job) wg_counts = (uint32_t *)(bo->vaddr + indirect_csd->offset); - if (wg_counts[0] == 0 || wg_counts[1] == 0 || wg_counts[2] == 0) - goto unmap_bo; - args->cfg[0] = wg_counts[0] << V3D_CSD_CFG012_WG_COUNT_SHIFT; args->cfg[1] = wg_counts[1] << V3D_CSD_CFG012_WG_COUNT_SHIFT; args->cfg[2] = wg_counts[2] << V3D_CSD_CFG012_WG_COUNT_SHIFT; + if (wg_counts[0] == 0 || wg_counts[1] == 0 || wg_counts[2] == 0) + goto unmap_bo; + num_batches = DIV_ROUND_UP(indirect_csd->wg_size, 16) * (wg_counts[0] * wg_counts[1] * wg_counts[2]); diff --git a/drivers/gpu/drm/xe/display/xe_display.c b/drivers/gpu/drm/xe/display/xe_display.c index e164e2d71e11..de1fd4dff0e8 100644 --- a/drivers/gpu/drm/xe/display/xe_display.c +++ b/drivers/gpu/drm/xe/display/xe_display.c @@ -148,6 +148,15 @@ int xe_display_init_noirq(struct xe_device *xe) intel_display_driver_early_probe(xe); + intel_display_device_info_runtime_init(xe); + + /* Display may have been disabled at runtime init */ + if (!has_display(xe)) { + xe->info.probe_display = false; + unset_display_features(xe); + return 0; + } + /* Early display init.. */ intel_opregion_setup(display); @@ -159,8 +168,6 @@ int xe_display_init_noirq(struct xe_device *xe) intel_bw_init_hw(xe); - intel_display_device_info_runtime_init(xe); - err = intel_display_driver_probe_noirq(xe); if (err) { intel_opregion_cleanup(display); diff --git a/drivers/hv/hv_kvp.c b/drivers/hv/hv_kvp.c index 77017d951826..8019ef2cdcf2 100644 --- a/drivers/hv/hv_kvp.c +++ b/drivers/hv/hv_kvp.c @@ -93,7 +93,7 @@ static void kvp_send_key(struct work_struct *dummy); static void kvp_respond_to_host(struct hv_kvp_msg *msg, int error); static void kvp_timeout_func(struct work_struct *dummy); static void kvp_host_handshake_func(struct work_struct *dummy); -static void kvp_register(int); +static int kvp_register(int); static DECLARE_DELAYED_WORK(kvp_timeout_work, kvp_timeout_func); static DECLARE_DELAYED_WORK(kvp_host_handshake_work, kvp_host_handshake_func); @@ -127,24 +127,26 @@ static void kvp_register_done(void) hv_poll_channel(kvp_transaction.recv_channel, kvp_poll_wrapper); } -static void +static int kvp_register(int reg_value) { struct hv_kvp_msg *kvp_msg; char *version; + int ret; kvp_msg = kzalloc(sizeof(*kvp_msg), GFP_KERNEL); + if (!kvp_msg) + return -ENOMEM; - if (kvp_msg) { - version = kvp_msg->body.kvp_register.version; - kvp_msg->kvp_hdr.operation = reg_value; - strcpy(version, HV_DRV_VERSION); + version = kvp_msg->body.kvp_register.version; + kvp_msg->kvp_hdr.operation = reg_value; + strcpy(version, HV_DRV_VERSION); - hvutil_transport_send(hvt, kvp_msg, sizeof(*kvp_msg), - kvp_register_done); - kfree(kvp_msg); - } + ret = hvutil_transport_send(hvt, kvp_msg, sizeof(*kvp_msg), + kvp_register_done); + kfree(kvp_msg); + return ret; } static void kvp_timeout_func(struct work_struct *dummy) @@ -186,9 +188,8 @@ static int kvp_handle_handshake(struct hv_kvp_msg *msg) */ pr_debug("KVP: userspace daemon ver. %d connected\n", msg->kvp_hdr.operation); - kvp_register(dm_reg_value); - return 0; + return kvp_register(dm_reg_value); } diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c index 128d732665bf..0191562e606d 100644 --- a/drivers/hv/vmbus_drv.c +++ b/drivers/hv/vmbus_drv.c @@ -2205,8 +2205,8 @@ static acpi_status vmbus_walk_resources(struct acpi_resource *res, void *ctx) return AE_NO_MEMORY; /* If this range overlaps the virtual TPM, truncate it. */ - if (end > VTPM_BASE_ADDRESS && start < VTPM_BASE_ADDRESS) - end = VTPM_BASE_ADDRESS; + if (end >= VTPM_BASE_ADDRESS && start < VTPM_BASE_ADDRESS) + end = VTPM_BASE_ADDRESS - 1; new_res->name = "hyperv mmio"; new_res->flags = IORESOURCE_MEM; @@ -2273,6 +2273,7 @@ static void vmbus_mmio_remove(void) static void __maybe_unused vmbus_reserve_fb(void) { resource_size_t start = 0, size; + resource_size_t low_mmio_base; struct pci_dev *pdev; if (efi_enabled(EFI_BOOT)) { @@ -2280,6 +2281,24 @@ static void __maybe_unused vmbus_reserve_fb(void) if (IS_ENABLED(CONFIG_SYSFB)) { start = screen_info.lfb_base; size = max_t(__u32, screen_info.lfb_size, 0x800000); + + low_mmio_base = hyperv_mmio->start; + if (!low_mmio_base || upper_32_bits(low_mmio_base) || + (start && start < low_mmio_base)) { + pr_warn("Unexpected low mmio base %pa\n", &low_mmio_base); + } else { + /* + * If the kdump/kexec or CVM kernel's lfb_base + * is 0, fall back to the low mmio base. + */ + if (!start) + start = low_mmio_base; + /* + * Reserve half of the space below 4GB for high + * resolutions, but cap the reservation to 128MB. + */ + size = min((SZ_4G - start) / 2, SZ_128M); + } } } else { /* Gen1 VM: get FB base from PCI */ @@ -2300,8 +2319,10 @@ static void __maybe_unused vmbus_reserve_fb(void) pci_dev_put(pdev); } - if (!start) + if (!start) { + pr_warn("Unexpected framebuffer mmio base of zero\n"); return; + } /* * Make a claim for the frame buffer in the resource tree under the @@ -2311,6 +2332,8 @@ static void __maybe_unused vmbus_reserve_fb(void) */ for (; !fb_mmio && (size >= 0x100000); size >>= 1) fb_mmio = __request_region(hyperv_mmio, start, size, fb_mmio_name, 0); + + pr_info("hv_mmio=%pR,%pR fb=%pR\n", hyperv_mmio, hyperv_mmio->sibling, fb_mmio); } /** diff --git a/drivers/i2c/i2c-core-base.c b/drivers/i2c/i2c-core-base.c index 75d30861ffe2..461758b9293c 100644 --- a/drivers/i2c/i2c-core-base.c +++ b/drivers/i2c/i2c-core-base.c @@ -1559,6 +1559,10 @@ static int i2c_register_adapter(struct i2c_adapter *adap) pm_suspend_ignore_children(&adap->dev, true); pm_runtime_enable(&adap->dev); + mutex_lock(&core_lock); + idr_replace(&i2c_adapter_idr, adap, adap->nr); + mutex_unlock(&core_lock); + res = device_add(&adap->dev); if (res) { pr_err("adapter '%s': can't register device (%d)\n", adap->name, res); @@ -1617,7 +1621,7 @@ static int __i2c_add_numbered_adapter(struct i2c_adapter *adap) int id; mutex_lock(&core_lock); - id = idr_alloc(&i2c_adapter_idr, adap, adap->nr, adap->nr + 1, GFP_KERNEL); + id = idr_alloc(&i2c_adapter_idr, NULL, adap->nr, adap->nr + 1, GFP_KERNEL); mutex_unlock(&core_lock); if (WARN(id < 0, "couldn't get idr")) return id == -ENOSPC ? -EBUSY : id; @@ -1653,7 +1657,7 @@ int i2c_add_adapter(struct i2c_adapter *adapter) } mutex_lock(&core_lock); - id = idr_alloc(&i2c_adapter_idr, adapter, + id = idr_alloc(&i2c_adapter_idr, NULL, __i2c_first_dynamic_bus_num, 0, GFP_KERNEL); mutex_unlock(&core_lock); if (WARN(id < 0, "couldn't get idr")) diff --git a/drivers/i2c/i2c-stub.c b/drivers/i2c/i2c-stub.c index 09e7b7bf4c5f..a6e2ce4360ca 100644 --- a/drivers/i2c/i2c-stub.c +++ b/drivers/i2c/i2c-stub.c @@ -214,6 +214,11 @@ static s32 stub_xfer(struct i2c_adapter *adap, u16 addr, unsigned short flags, * We ignore banks here, because banked chips don't use I2C * block transfers */ + if (data->block[0] == 0 || + data->block[0] > I2C_SMBUS_BLOCK_MAX) { + ret = -EINVAL; + break; + } if (data->block[0] > 256 - command) /* Avoid overrun */ data->block[0] = 256 - command; len = data->block[0]; diff --git a/drivers/iio/adc/ti-ads1298.c b/drivers/iio/adc/ti-ads1298.c index d00cd169e8df..00d93fc043e2 100644 --- a/drivers/iio/adc/ti-ads1298.c +++ b/drivers/iio/adc/ti-ads1298.c @@ -279,6 +279,7 @@ static const u8 ads1298_pga_settings[] = { 6, 1, 2, 3, 4, 8, 12 }; static int ads1298_get_scale(struct ads1298_private *priv, int channel, int *val, int *val2) { + unsigned int pga_idx; int ret; unsigned int regval; u8 gain; @@ -302,7 +303,11 @@ static int ads1298_get_scale(struct ads1298_private *priv, if (ret) return ret; - gain = ads1298_pga_settings[FIELD_GET(ADS1298_MASK_CH_PGA, regval)]; + pga_idx = FIELD_GET(ADS1298_MASK_CH_PGA, regval); + if (pga_idx >= ARRAY_SIZE(ads1298_pga_settings)) + return -EINVAL; + + gain = ads1298_pga_settings[pga_idx]; *val /= gain; /* Full scale is VREF / gain */ *val2 = ADS1298_BITS_PER_SAMPLE - 1; /* Signed, hence the -1 */ diff --git a/drivers/iio/light/bh1780.c b/drivers/iio/light/bh1780.c index 475f44954f61..f478f12640d5 100644 --- a/drivers/iio/light/bh1780.c +++ b/drivers/iio/light/bh1780.c @@ -109,10 +109,10 @@ static int bh1780_read_raw(struct iio_dev *indio_dev, case IIO_LIGHT: pm_runtime_get_sync(&bh1780->client->dev); value = bh1780_read_word(bh1780, BH1780_REG_DLOW); - if (value < 0) - return value; pm_runtime_mark_last_busy(&bh1780->client->dev); pm_runtime_put_autosuspend(&bh1780->client->dev); + if (value < 0) + return value; *val = value; return IIO_VAL_INT; diff --git a/drivers/iio/light/veml6075.c b/drivers/iio/light/veml6075.c index 859891e8f115..876abac9acde 100644 --- a/drivers/iio/light/veml6075.c +++ b/drivers/iio/light/veml6075.c @@ -100,7 +100,7 @@ static const struct iio_chan_spec veml6075_channels[] = { static int veml6075_request_measurement(struct veml6075_data *data) { - int ret, conf, int_time; + int ret, conf, int_time, int_index; ret = regmap_read(data->regmap, VEML6075_CMD_CONF, &conf); if (ret < 0) @@ -117,7 +117,11 @@ static int veml6075_request_measurement(struct veml6075_data *data) * time for all possible configurations. Using a 1.50 factor simplifies * operations and ensures reliability under all circumstances. */ - int_time = veml6075_it_ms[FIELD_GET(VEML6075_CONF_IT, conf)]; + int_index = FIELD_GET(VEML6075_CONF_IT, conf); + if (int_index >= ARRAY_SIZE(veml6075_it_ms)) + return -EINVAL; + + int_time = veml6075_it_ms[int_index]; msleep(int_time + (int_time / 2)); /* shutdown again, data registers are still accessible */ diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index c1587845f280..7ad5d88358b0 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -4231,7 +4231,7 @@ int bnxt_re_alloc_ucontext(struct ib_ucontext *ctx, struct ib_udata *udata) uctx->rdev = rdev; - uctx->shpg = (void *)__get_free_page(GFP_KERNEL); + uctx->shpg = (void *)get_zeroed_page(GFP_KERNEL); if (!uctx->shpg) { rc = -ENOMEM; goto fail; diff --git a/drivers/irqchip/irq-imgpdc.c b/drivers/irqchip/irq-imgpdc.c index b42ed68acfa6..602cea5e52d2 100644 --- a/drivers/irqchip/irq-imgpdc.c +++ b/drivers/irqchip/irq-imgpdc.c @@ -378,6 +378,7 @@ static int pdc_intc_probe(struct platform_device *pdev) dev_err(&pdev->dev, "cannot add IRQ domain\n"); return -ENOMEM; } + priv->domain->flags |= IRQ_DOMAIN_FLAG_DESTROY_GC; /* * Set up 2 generic irq chips with 2 chip types. @@ -465,6 +466,11 @@ static void pdc_intc_remove(struct platform_device *pdev) { struct pdc_intc_priv *priv = platform_get_drvdata(pdev); + for (unsigned int i = 0; i < priv->nr_perips; ++i) + irq_set_chained_handler_and_data(priv->perip_irqs[i], NULL, NULL); + + irq_set_chained_handler_and_data(priv->syswake_irq, NULL, NULL); + irq_domain_remove(priv->domain); } diff --git a/drivers/media/test-drivers/vidtv/vidtv_mux.c b/drivers/media/test-drivers/vidtv/vidtv_mux.c index 7dad97881fdb..00ae52dbc96e 100644 --- a/drivers/media/test-drivers/vidtv/vidtv_mux.c +++ b/drivers/media/test-drivers/vidtv/vidtv_mux.c @@ -101,7 +101,8 @@ static int vidtv_mux_pid_ctx_init(struct vidtv_mux *m) /* add a ctx for all PMT sections */ while (p) { pid = vidtv_psi_get_pat_program_pid(p); - vidtv_mux_create_pid_ctx_once(m, pid); + if (!vidtv_mux_create_pid_ctx_once(m, pid)) + goto free; p = p->next; } @@ -170,6 +171,9 @@ static u32 vidtv_mux_push_si(struct vidtv_mux *m) nit_ctx = vidtv_mux_get_pid_ctx(m, VIDTV_NIT_PID); eit_ctx = vidtv_mux_get_pid_ctx(m, VIDTV_EIT_PID); + if (!pat_ctx || !sdt_ctx || !nit_ctx || !eit_ctx) + return 0; + pat_args.offset = m->mux_buf_offset; pat_args.continuity_counter = &pat_ctx->cc; @@ -186,6 +190,8 @@ static u32 vidtv_mux_push_si(struct vidtv_mux *m) } pmt_ctx = vidtv_mux_get_pid_ctx(m, pmt_pid); + if (!pmt_ctx) + continue; pmt_args.offset = m->mux_buf_offset; pmt_args.pmt = m->si.pmt_secs[i]; diff --git a/drivers/mtd/spi-nor/macronix.c b/drivers/mtd/spi-nor/macronix.c index ea6be95e75a5..6127565372c5 100644 --- a/drivers/mtd/spi-nor/macronix.c +++ b/drivers/mtd/spi-nor/macronix.c @@ -28,8 +28,26 @@ mx25l25635_post_bfpt_fixups(struct spi_nor *nor, return 0; } +static int +macronix_qpp4b_post_sfdp_fixups(struct spi_nor *nor) +{ + /* PP_1_1_4_4B is supported but missing in 4BAIT. */ + struct spi_nor_flash_parameter *params = nor->params; + + params->hwcaps.mask |= SNOR_HWCAPS_PP_1_1_4; + spi_nor_set_pp_settings(¶ms->page_programs[SNOR_CMD_PP_1_1_4], + SPINOR_OP_PP_1_1_4_4B, SNOR_PROTO_1_1_4); + + return 0; +} + static const struct spi_nor_fixups mx25l25635_fixups = { .post_bfpt = mx25l25635_post_bfpt_fixups, + .post_sfdp = macronix_qpp4b_post_sfdp_fixups, +}; + +static const struct spi_nor_fixups macronix_qpp4b_fixups = { + .post_sfdp = macronix_qpp4b_post_sfdp_fixups, }; static const struct flash_info macronix_nor_parts[] = { @@ -85,11 +103,17 @@ static const struct flash_info macronix_nor_parts[] = { .size = SZ_64M, .no_sfdp_flags = SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ, .fixup_flags = SPI_NOR_4B_OPCODES, + .fixups = ¯onix_qpp4b_fixups, }, { .id = SNOR_ID(0xc2, 0x20, 0x1b), .name = "mx66l1g45g", .size = SZ_128M, .no_sfdp_flags = SECT_4K | SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ, + .fixups = ¯onix_qpp4b_fixups, + }, { + /* MX66L2G45G */ + .id = SNOR_ID(0xc2, 0x20, 0x1c), + .fixups = ¯onix_qpp4b_fixups, }, { .id = SNOR_ID(0xc2, 0x23, 0x14), .name = "mx25v8035f", @@ -137,18 +161,25 @@ static const struct flash_info macronix_nor_parts[] = { .size = SZ_64M, .no_sfdp_flags = SECT_4K | SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ, .fixup_flags = SPI_NOR_4B_OPCODES, + .fixups = ¯onix_qpp4b_fixups, }, { .id = SNOR_ID(0xc2, 0x25, 0x3a), .name = "mx66u51235f", .size = SZ_64M, .no_sfdp_flags = SECT_4K | SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ, .fixup_flags = SPI_NOR_4B_OPCODES, + .fixups = ¯onix_qpp4b_fixups, + }, { + /* MX66U1G45G */ + .id = SNOR_ID(0xc2, 0x25, 0x3b), + .fixups = ¯onix_qpp4b_fixups, }, { .id = SNOR_ID(0xc2, 0x25, 0x3c), .name = "mx66u2g45g", .size = SZ_256M, .no_sfdp_flags = SECT_4K | SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ, .fixup_flags = SPI_NOR_4B_OPCODES, + .fixups = ¯onix_qpp4b_fixups, }, { .id = SNOR_ID(0xc2, 0x26, 0x18), .name = "mx25l12855e", diff --git a/drivers/net/bonding/bond_3ad.c b/drivers/net/bonding/bond_3ad.c index 4c2560ae8866..b777902ed850 100644 --- a/drivers/net/bonding/bond_3ad.c +++ b/drivers/net/bonding/bond_3ad.c @@ -436,6 +436,7 @@ static void __ad_actor_update_port(struct port *port) port->actor_system = BOND_AD_INFO(bond).system.sys_mac_addr; port->actor_system_priority = BOND_AD_INFO(bond).system.sys_priority; + port->actor_port_priority = SLAVE_AD_INFO(port->slave)->port_priority; } /* Conversions */ @@ -990,6 +991,7 @@ static int ad_marker_send(struct port *port, struct bond_marker *marker) static void ad_mux_machine(struct port *port, bool *update_slave_arr) { struct bonding *bond = __get_bond_by_port(port); + struct aggregator *aggregator; mux_states_t last_state; /* keep current State Machine state to compare later if it was @@ -997,6 +999,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr) */ last_state = port->sm_mux_state; + aggregator = rcu_dereference(port->aggregator); if (port->sm_vars & AD_PORT_BEGIN) { port->sm_mux_state = AD_MUX_DETACHED; } else { @@ -1016,7 +1019,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr) * cycle to update ready variable, we check * READY_N and update READY here */ - __set_agg_ports_ready(port->aggregator, __agg_ports_are_ready(port->aggregator)); + __set_agg_ports_ready(aggregator, __agg_ports_are_ready(aggregator)); port->sm_mux_state = AD_MUX_DETACHED; break; } @@ -1031,7 +1034,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr) * update ready variable, we check READY_N and update * READY here */ - __set_agg_ports_ready(port->aggregator, __agg_ports_are_ready(port->aggregator)); + __set_agg_ports_ready(aggregator, __agg_ports_are_ready(aggregator)); /* if the wait_while_timer expired, and the port is * in READY state, move to ATTACHED state @@ -1047,7 +1050,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr) if ((port->sm_vars & AD_PORT_SELECTED) && (port->partner_oper.port_state & LACP_STATE_SYNCHRONIZATION) && !__check_agg_selection_timer(port)) { - if (port->aggregator->is_active) { + if (aggregator->is_active) { int state = AD_MUX_COLLECTING_DISTRIBUTING; if (!bond->params.coupled_control) @@ -1063,9 +1066,9 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr) * cycle to update ready variable, we check * READY_N and update READY here */ - __set_agg_ports_ready(port->aggregator, __agg_ports_are_ready(port->aggregator)); + __set_agg_ports_ready(aggregator, __agg_ports_are_ready(aggregator)); port->sm_mux_state = AD_MUX_DETACHED; - } else if (port->aggregator->is_active) { + } else if (aggregator->is_active) { port->actor_oper_port_state |= LACP_STATE_SYNCHRONIZATION; } @@ -1076,7 +1079,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr) * sure that a collecting distributing * port in an active aggregator is enabled */ - if (port->aggregator->is_active && + if (aggregator->is_active && !__port_is_collecting_distributing(port)) { __enable_port(port); *update_slave_arr = true; @@ -1095,7 +1098,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr) */ struct slave *slave = port->slave; - if (port->aggregator->is_active && + if (aggregator->is_active && bond_is_slave_rx_disabled(slave)) { ad_enable_collecting(port); *update_slave_arr = true; @@ -1115,8 +1118,8 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr) * sure that a collecting distributing * port in an active aggregator is enabled */ - if (port->aggregator && - port->aggregator->is_active && + if (aggregator && + aggregator->is_active && !__port_is_collecting_distributing(port)) { __enable_port(port); *update_slave_arr = true; @@ -1148,7 +1151,7 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr) port->sm_mux_timer_counter = __ad_timer_to_ticks(AD_WAIT_WHILE_TIMER, 0); break; case AD_MUX_ATTACHED: - if (port->aggregator->is_active) + if (aggregator->is_active) port->actor_oper_port_state |= LACP_STATE_SYNCHRONIZATION; else @@ -1345,8 +1348,8 @@ static void ad_churn_machine(struct port *port) { if (port->sm_vars & AD_PORT_CHURNED) { port->sm_vars &= ~AD_PORT_CHURNED; - port->sm_churn_actor_state = AD_CHURN_MONITOR; - port->sm_churn_partner_state = AD_CHURN_MONITOR; + WRITE_ONCE(port->sm_churn_actor_state, AD_CHURN_MONITOR); + WRITE_ONCE(port->sm_churn_partner_state, AD_CHURN_MONITOR); port->sm_churn_actor_timer_counter = __ad_timer_to_ticks(AD_ACTOR_CHURN_TIMER, 0); port->sm_churn_partner_timer_counter = @@ -1357,20 +1360,22 @@ static void ad_churn_machine(struct port *port) !(--port->sm_churn_actor_timer_counter) && port->sm_churn_actor_state == AD_CHURN_MONITOR) { if (port->actor_oper_port_state & LACP_STATE_SYNCHRONIZATION) { - port->sm_churn_actor_state = AD_NO_CHURN; + WRITE_ONCE(port->sm_churn_actor_state, AD_NO_CHURN); } else { - port->churn_actor_count++; - port->sm_churn_actor_state = AD_CHURN; + WRITE_ONCE(port->churn_actor_count, + port->churn_actor_count + 1); + WRITE_ONCE(port->sm_churn_actor_state, AD_CHURN); } } if (port->sm_churn_partner_timer_counter && !(--port->sm_churn_partner_timer_counter) && port->sm_churn_partner_state == AD_CHURN_MONITOR) { if (port->partner_oper.port_state & LACP_STATE_SYNCHRONIZATION) { - port->sm_churn_partner_state = AD_NO_CHURN; + WRITE_ONCE(port->sm_churn_partner_state, AD_NO_CHURN); } else { - port->churn_partner_count++; - port->sm_churn_partner_state = AD_CHURN; + WRITE_ONCE(port->churn_partner_count, + port->churn_partner_count + 1); + WRITE_ONCE(port->sm_churn_partner_state, AD_CHURN); } } } @@ -1521,9 +1526,9 @@ static void ad_port_selection_logic(struct port *port, bool *update_slave_arr) bond = __get_bond_by_port(port); /* if the port is connected to other aggregator, detach it */ - if (port->aggregator) { + temp_aggregator = rcu_dereference(port->aggregator); + if (temp_aggregator) { /* detach the port from its former aggregator */ - temp_aggregator = port->aggregator; for (curr_port = temp_aggregator->lag_ports; curr_port; last_port = curr_port, curr_port = curr_port->next_port_in_aggregator) { @@ -1546,7 +1551,7 @@ static void ad_port_selection_logic(struct port *port, bool *update_slave_arr) /* clear the port's relations to this * aggregator */ - port->aggregator = NULL; + RCU_INIT_POINTER(port->aggregator, NULL); port->next_port_in_aggregator = NULL; port->actor_port_aggregator_identifier = 0; @@ -1569,7 +1574,7 @@ static void ad_port_selection_logic(struct port *port, bool *update_slave_arr) port->slave->bond->dev->name, port->slave->dev->name, port->actor_port_number, - port->aggregator->aggregator_identifier); + temp_aggregator->aggregator_identifier); } } /* search on all aggregators for a suitable aggregator for this port */ @@ -1593,15 +1598,15 @@ static void ad_port_selection_logic(struct port *port, bool *update_slave_arr) ) ) { /* attach to the founded aggregator */ - port->aggregator = aggregator; + rcu_assign_pointer(port->aggregator, aggregator); port->actor_port_aggregator_identifier = - port->aggregator->aggregator_identifier; + aggregator->aggregator_identifier; port->next_port_in_aggregator = aggregator->lag_ports; - port->aggregator->num_of_ports++; + aggregator->num_of_ports++; aggregator->lag_ports = port; slave_dbg(bond->dev, slave->dev, "Port %d joined LAG %d (existing LAG)\n", port->actor_port_number, - port->aggregator->aggregator_identifier); + aggregator->aggregator_identifier); /* mark this port as selected */ port->sm_vars |= AD_PORT_SELECTED; @@ -1616,39 +1621,40 @@ static void ad_port_selection_logic(struct port *port, bool *update_slave_arr) if (!found) { if (free_aggregator) { /* assign port a new aggregator */ - port->aggregator = free_aggregator; port->actor_port_aggregator_identifier = - port->aggregator->aggregator_identifier; + free_aggregator->aggregator_identifier; /* update the new aggregator's parameters * if port was responsed from the end-user */ if (port->actor_oper_port_key & AD_DUPLEX_KEY_MASKS) /* if port is full duplex */ - port->aggregator->is_individual = false; + free_aggregator->is_individual = false; else - port->aggregator->is_individual = true; + free_aggregator->is_individual = true; - port->aggregator->actor_admin_aggregator_key = + free_aggregator->actor_admin_aggregator_key = port->actor_admin_port_key; - port->aggregator->actor_oper_aggregator_key = + free_aggregator->actor_oper_aggregator_key = port->actor_oper_port_key; - port->aggregator->partner_system = + free_aggregator->partner_system = port->partner_oper.system; - port->aggregator->partner_system_priority = + free_aggregator->partner_system_priority = port->partner_oper.system_priority; - port->aggregator->partner_oper_aggregator_key = port->partner_oper.key; - port->aggregator->receive_state = 1; - port->aggregator->transmit_state = 1; - port->aggregator->lag_ports = port; - port->aggregator->num_of_ports++; + free_aggregator->partner_oper_aggregator_key = port->partner_oper.key; + free_aggregator->receive_state = 1; + free_aggregator->transmit_state = 1; + free_aggregator->lag_ports = port; + free_aggregator->num_of_ports++; + + rcu_assign_pointer(port->aggregator, free_aggregator); /* mark this port as selected */ port->sm_vars |= AD_PORT_SELECTED; slave_dbg(bond->dev, port->slave->dev, "Port %d joined LAG %d (new LAG)\n", port->actor_port_number, - port->aggregator->aggregator_identifier); + free_aggregator->aggregator_identifier); } else { slave_err(bond->dev, port->slave->dev, "Port %d did not find a suitable aggregator\n", @@ -1660,13 +1666,12 @@ static void ad_port_selection_logic(struct port *port, bool *update_slave_arr) * in all aggregator's ports, else set ready=FALSE in all * aggregator's ports */ - __set_agg_ports_ready(port->aggregator, - __agg_ports_are_ready(port->aggregator)); + aggregator = rcu_dereference(port->aggregator); + __set_agg_ports_ready(aggregator, __agg_ports_are_ready(aggregator)); - aggregator = __get_first_agg(port); - ad_agg_selection_logic(aggregator, update_slave_arr); + ad_agg_selection_logic(__get_first_agg(port), update_slave_arr); - if (!port->aggregator->is_active) + if (!aggregator->is_active) port->actor_oper_port_state &= ~LACP_STATE_SYNCHRONIZATION; } @@ -2020,13 +2025,15 @@ static void ad_initialize_port(struct port *port, const struct bond_params *bond */ static void ad_enable_collecting(struct port *port) { - if (port->aggregator->is_active) { + struct aggregator *aggregator = rcu_dereference(port->aggregator); + + if (aggregator->is_active) { struct slave *slave = port->slave; slave_dbg(slave->bond->dev, slave->dev, "Enabling collecting on port %d (LAG %d)\n", port->actor_port_number, - port->aggregator->aggregator_identifier); + aggregator->aggregator_identifier); __enable_collecting_port(port); } } @@ -2038,11 +2045,13 @@ static void ad_enable_collecting(struct port *port) */ static void ad_disable_distributing(struct port *port, bool *update_slave_arr) { - if (port->aggregator && __agg_has_partner(port->aggregator)) { + struct aggregator *aggregator = rcu_dereference(port->aggregator); + + if (aggregator && __agg_has_partner(aggregator)) { slave_dbg(port->slave->bond->dev, port->slave->dev, "Disabling distributing on port %d (LAG %d)\n", port->actor_port_number, - port->aggregator->aggregator_identifier); + aggregator->aggregator_identifier); __disable_distributing_port(port); /* Slave array needs an update */ *update_slave_arr = true; @@ -2059,11 +2068,13 @@ static void ad_disable_distributing(struct port *port, bool *update_slave_arr) static void ad_enable_collecting_distributing(struct port *port, bool *update_slave_arr) { - if (port->aggregator->is_active) { + struct aggregator *aggregator = rcu_dereference(port->aggregator); + + if (aggregator->is_active) { slave_dbg(port->slave->bond->dev, port->slave->dev, "Enabling port %d (LAG %d)\n", port->actor_port_number, - port->aggregator->aggregator_identifier); + aggregator->aggregator_identifier); __enable_port(port); /* Slave array needs update */ *update_slave_arr = true; @@ -2078,11 +2089,13 @@ static void ad_enable_collecting_distributing(struct port *port, static void ad_disable_collecting_distributing(struct port *port, bool *update_slave_arr) { - if (port->aggregator && __agg_has_partner(port->aggregator)) { + struct aggregator *aggregator = rcu_dereference(port->aggregator); + + if (aggregator && __agg_has_partner(aggregator)) { slave_dbg(port->slave->bond->dev, port->slave->dev, "Disabling port %d (LAG %d)\n", port->actor_port_number, - port->aggregator->aggregator_identifier); + aggregator->aggregator_identifier); __disable_port(port); /* Slave array needs an update */ *update_slave_arr = true; @@ -2195,6 +2208,9 @@ void bond_3ad_bind_slave(struct slave *slave) ad_initialize_port(port, &bond->params); + /* Port priority is initialized. Update it to slave's ad info */ + SLAVE_AD_INFO(slave)->port_priority = port->actor_port_priority; + port->slave = slave; port->actor_port_number = SLAVE_AD_INFO(slave)->id; /* key is determined according to the link speed, duplex and @@ -2319,7 +2335,7 @@ void bond_3ad_unbind_slave(struct slave *slave) */ for (temp_port = aggregator->lag_ports; temp_port; temp_port = temp_port->next_port_in_aggregator) { - temp_port->aggregator = new_aggregator; + rcu_assign_pointer(temp_port->aggregator, new_aggregator); temp_port->actor_port_aggregator_identifier = new_aggregator->aggregator_identifier; } @@ -2788,15 +2804,16 @@ int bond_3ad_set_carrier(struct bonding *bond) int __bond_3ad_get_active_agg_info(struct bonding *bond, struct ad_info *ad_info) { - struct aggregator *aggregator = NULL; + struct aggregator *aggregator = NULL, *tmp; struct list_head *iter; struct slave *slave; struct port *port; bond_for_each_slave_rcu(bond, slave, iter) { port = &(SLAVE_AD_INFO(slave)->port); - if (port->aggregator && port->aggregator->is_active) { - aggregator = port->aggregator; + tmp = rcu_dereference(port->aggregator); + if (tmp && tmp->is_active) { + aggregator = tmp; break; } } diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index 7b8555f6102a..69e79b22e319 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -211,6 +211,8 @@ atomic_t netpoll_block_tx = ATOMIC_INIT(0); unsigned int bond_net_id __read_mostly; +DEFINE_STATIC_KEY_FALSE(bond_bcast_neigh_enabled); + static const struct flow_dissector_key flow_keys_bonding_keys[] = { { .key_id = FLOW_DISSECTOR_KEY_CONTROL, @@ -1468,7 +1470,7 @@ static void bond_poll_controller(struct net_device *bond_dev) if (BOND_MODE(bond) == BOND_MODE_8023AD) { struct aggregator *agg = - SLAVE_AD_INFO(slave)->port.aggregator; + rcu_dereference(SLAVE_AD_INFO(slave)->port.aggregator); if (agg && agg->aggregator_identifier != ad_info.aggregator_id) @@ -2388,7 +2390,9 @@ int bond_enslave(struct net_device *bond_dev, struct net_device *slave_dev, bpf_prog_inc(bond->xdp_prog); } - if (bond_mode_can_use_xmit_hash(bond)) + /* broadcast mode uses the all_slaves to loop through slaves. */ + if (bond_mode_can_use_xmit_hash(bond) || + BOND_MODE(bond) == BOND_MODE_BROADCAST) bond_update_slave_arr(bond, NULL); bond_xdp_set_features(bond_dev); @@ -2531,7 +2535,8 @@ static int __bond_release_one(struct net_device *bond_dev, bond_upper_dev_unlink(bond, slave); - if (bond_mode_can_use_xmit_hash(bond)) + if (bond_mode_can_use_xmit_hash(bond) || + BOND_MODE(bond) == BOND_MODE_BROADCAST) bond_update_slave_arr(bond, slave); slave_info(bond_dev, slave_dev, "Releasing %s interface\n", @@ -4445,6 +4450,9 @@ static int bond_open(struct net_device *bond_dev) bond_for_each_slave(bond, slave, iter) dev_mc_add(slave->dev, lacpdu_mcast_addr); + + if (bond->params.broadcast_neighbor) + static_branch_inc(&bond_bcast_neigh_enabled); } if (bond_mode_can_use_xmit_hash(bond)) @@ -4468,6 +4476,10 @@ static int bond_close(struct net_device *bond_dev) if (bond_is_lb(bond)) bond_alb_deinitialize(bond); + if (BOND_MODE(bond) == BOND_MODE_8023AD && + bond->params.broadcast_neighbor) + static_branch_dec(&bond_bcast_neigh_enabled); + if (bond_uses_primary(bond)) { rcu_read_lock(); slave = rcu_dereference(bond->curr_active_slave); @@ -5179,13 +5191,18 @@ static void bond_set_slave_arr(struct bonding *bond, { struct bond_up_slave *usable, *all; - usable = rtnl_dereference(bond->usable_slaves); - rcu_assign_pointer(bond->usable_slaves, usable_slaves); - kfree_rcu(usable, rcu); - all = rtnl_dereference(bond->all_slaves); rcu_assign_pointer(bond->all_slaves, all_slaves); kfree_rcu(all, rcu); + + if (BOND_MODE(bond) == BOND_MODE_BROADCAST) { + kfree_rcu(usable_slaves, rcu); + return; + } + + usable = rtnl_dereference(bond->usable_slaves); + rcu_assign_pointer(bond->usable_slaves, usable_slaves); + kfree_rcu(usable, rcu); } static void bond_reset_slave_arr(struct bonding *bond) @@ -5235,15 +5252,16 @@ int bond_update_slave_arr(struct bonding *bond, struct slave *skipslave) spin_unlock_bh(&bond->mode_lock); agg_id = ad_info.aggregator_id; } + rcu_read_lock(); bond_for_each_slave(bond, slave, iter) { if (skipslave == slave) continue; all_slaves->arr[all_slaves->count++] = slave; if (BOND_MODE(bond) == BOND_MODE_8023AD) { - struct aggregator *agg; + const struct aggregator *agg; - agg = SLAVE_AD_INFO(slave)->port.aggregator; + agg = rcu_dereference(SLAVE_AD_INFO(slave)->port.aggregator); if (!agg || agg->aggregator_identifier != agg_id) continue; } @@ -5255,6 +5273,7 @@ int bond_update_slave_arr(struct bonding *bond, struct slave *skipslave) usable_slaves->arr[usable_slaves->count++] = slave; } + rcu_read_unlock(); bond_set_slave_arr(bond, usable_slaves, all_slaves); return ret; @@ -5304,6 +5323,37 @@ static struct slave *bond_xdp_xmit_3ad_xor_slave_get(struct bonding *bond, return slaves->arr[hash % count]; } +static bool bond_should_broadcast_neighbor(struct sk_buff *skb, + struct net_device *dev) +{ + struct bonding *bond = netdev_priv(dev); + struct { + struct ipv6hdr ip6; + struct icmp6hdr icmp6; + } *combined, _combined; + + if (!static_branch_unlikely(&bond_bcast_neigh_enabled)) + return false; + + if (!bond->params.broadcast_neighbor) + return false; + + if (skb->protocol == htons(ETH_P_ARP)) + return true; + + if (skb->protocol == htons(ETH_P_IPV6)) { + combined = skb_header_pointer(skb, skb_mac_header_len(skb), + sizeof(_combined), + &_combined); + if (combined && combined->ip6.nexthdr == NEXTHDR_ICMP && + (combined->icmp6.icmp6_type == NDISC_NEIGHBOUR_SOLICITATION || + combined->icmp6.icmp6_type == NDISC_NEIGHBOUR_ADVERTISEMENT)) + return true; + } + + return false; +} + /* Use this Xmit function for 3AD as well as XOR modes. The current * usable slave array is formed in the control path. The xmit function * just calculates hash and sends the packet out. @@ -5323,9 +5373,12 @@ static netdev_tx_t bond_3ad_xor_xmit(struct sk_buff *skb, return bond_tx_drop(dev, skb); } -/* in broadcast mode, we send everything to all usable interfaces. */ +/* in broadcast mode, we send everything to all or usable slave interfaces. + * under rcu_read_lock when this function is called. + */ static netdev_tx_t bond_xmit_broadcast(struct sk_buff *skb, - struct net_device *bond_dev) + struct net_device *bond_dev, + bool all_slaves) { struct bonding *bond = netdev_priv(bond_dev); struct bond_up_slave *slaves; @@ -5333,7 +5386,10 @@ static netdev_tx_t bond_xmit_broadcast(struct sk_buff *skb, bool skb_used = false; int slaves_count, i; - slaves = rcu_dereference(bond->all_slaves); + if (all_slaves) + slaves = rcu_dereference(bond->all_slaves); + else + slaves = rcu_dereference(bond->usable_slaves); slaves_count = slaves ? READ_ONCE(slaves->count) : 0; for (i = 0; i < slaves_count; i++) { @@ -5575,10 +5631,13 @@ static netdev_tx_t __bond_start_xmit(struct sk_buff *skb, struct net_device *dev case BOND_MODE_ACTIVEBACKUP: return bond_xmit_activebackup(skb, dev); case BOND_MODE_8023AD: + if (bond_should_broadcast_neighbor(skb, dev)) + return bond_xmit_broadcast(skb, dev, false); + fallthrough; case BOND_MODE_XOR: return bond_3ad_xor_xmit(skb, dev); case BOND_MODE_BROADCAST: - return bond_xmit_broadcast(skb, dev); + return bond_xmit_broadcast(skb, dev, true); case BOND_MODE_ALB: return bond_alb_xmit(skb, dev); case BOND_MODE_TLB: @@ -6454,6 +6513,7 @@ static int __init bond_check_params(struct bond_params *params) eth_zero_addr(params->ad_actor_system); params->ad_user_port_key = ad_user_port_key; params->coupled_control = 1; + params->broadcast_neighbor = 0; if (packets_per_slave > 0) { params->reciprocal_packets_per_slave = reciprocal_value(packets_per_slave); diff --git a/drivers/net/bonding/bond_netlink.c b/drivers/net/bonding/bond_netlink.c index 2a6a424806aa..d7fc781b642c 100644 --- a/drivers/net/bonding/bond_netlink.c +++ b/drivers/net/bonding/bond_netlink.c @@ -28,6 +28,9 @@ static size_t bond_get_slave_size(const struct net_device *bond_dev, nla_total_size(sizeof(u8)) + /* IFLA_BOND_SLAVE_AD_ACTOR_OPER_PORT_STATE */ nla_total_size(sizeof(u16)) + /* IFLA_BOND_SLAVE_AD_PARTNER_OPER_PORT_STATE */ nla_total_size(sizeof(s32)) + /* IFLA_BOND_SLAVE_PRIO */ + nla_total_size(sizeof(u16)) + /* IFLA_BOND_SLAVE_ACTOR_PORT_PRIO */ + nla_total_size(sizeof(u8)) + /* IFLA_BOND_SLAVE_AD_CHURN_ACTOR_STATE */ + nla_total_size(sizeof(u8)) + /* IFLA_BOND_SLAVE_AD_CHURN_PARTNER_STATE */ 0; } @@ -63,24 +66,39 @@ static int bond_fill_slave_info(struct sk_buff *skb, const struct port *ad_port; ad_port = &SLAVE_AD_INFO(slave)->port; - agg = SLAVE_AD_INFO(slave)->port.aggregator; + rcu_read_lock(); + agg = rcu_dereference(SLAVE_AD_INFO(slave)->port.aggregator); if (agg) { if (nla_put_u16(skb, IFLA_BOND_SLAVE_AD_AGGREGATOR_ID, agg->aggregator_identifier)) - goto nla_put_failure; + goto nla_put_failure_rcu; if (nla_put_u8(skb, IFLA_BOND_SLAVE_AD_ACTOR_OPER_PORT_STATE, ad_port->actor_oper_port_state)) - goto nla_put_failure; + goto nla_put_failure_rcu; if (nla_put_u16(skb, IFLA_BOND_SLAVE_AD_PARTNER_OPER_PORT_STATE, ad_port->partner_oper.port_state)) - goto nla_put_failure; + goto nla_put_failure_rcu; + + if (nla_put_u8(skb, IFLA_BOND_SLAVE_AD_CHURN_ACTOR_STATE, + READ_ONCE(ad_port->sm_churn_actor_state))) + goto nla_put_failure_rcu; + if (nla_put_u8(skb, IFLA_BOND_SLAVE_AD_CHURN_PARTNER_STATE, + READ_ONCE(ad_port->sm_churn_partner_state))) + goto nla_put_failure_rcu; } + rcu_read_unlock(); + + if (nla_put_u16(skb, IFLA_BOND_SLAVE_ACTOR_PORT_PRIO, + SLAVE_AD_INFO(slave)->port_priority)) + goto nla_put_failure; } return 0; +nla_put_failure_rcu: + rcu_read_unlock(); nla_put_failure: return -EMSGSIZE; } @@ -129,6 +147,7 @@ static const struct nla_policy bond_policy[IFLA_BOND_MAX + 1] = { static const struct nla_policy bond_slave_policy[IFLA_BOND_SLAVE_MAX + 1] = { [IFLA_BOND_SLAVE_QUEUE_ID] = { .type = NLA_U16 }, [IFLA_BOND_SLAVE_PRIO] = { .type = NLA_S32 }, + [IFLA_BOND_SLAVE_ACTOR_PORT_PRIO] = { .type = NLA_U16 }, }; static int bond_validate(struct nlattr *tb[], struct nlattr *data[], @@ -179,6 +198,16 @@ static int bond_slave_changelink(struct net_device *bond_dev, return err; } + if (data[IFLA_BOND_SLAVE_ACTOR_PORT_PRIO]) { + u16 ad_prio = nla_get_u16(data[IFLA_BOND_SLAVE_ACTOR_PORT_PRIO]); + + bond_opt_slave_initval(&newval, &slave_dev, ad_prio); + err = __bond_opt_set(bond, BOND_OPT_ACTOR_PORT_PRIO, &newval, + data[IFLA_BOND_SLAVE_ACTOR_PORT_PRIO], extack); + if (err) + return err; + } + return 0; } diff --git a/drivers/net/bonding/bond_options.c b/drivers/net/bonding/bond_options.c index 33af81a55a45..4d01f1081956 100644 --- a/drivers/net/bonding/bond_options.c +++ b/drivers/net/bonding/bond_options.c @@ -79,6 +79,8 @@ static int bond_option_tlb_dynamic_lb_set(struct bonding *bond, const struct bond_opt_value *newval); static int bond_option_ad_actor_sys_prio_set(struct bonding *bond, const struct bond_opt_value *newval); +static int bond_option_actor_port_prio_set(struct bonding *bond, + const struct bond_opt_value *newval); static int bond_option_ad_actor_system_set(struct bonding *bond, const struct bond_opt_value *newval); static int bond_option_ad_user_port_key_set(struct bonding *bond, @@ -87,6 +89,8 @@ static int bond_option_missed_max_set(struct bonding *bond, const struct bond_opt_value *newval); static int bond_option_coupled_control_set(struct bonding *bond, const struct bond_opt_value *newval); +static int bond_option_broadcast_neigh_set(struct bonding *bond, + const struct bond_opt_value *newval); static const struct bond_opt_value bond_mode_tbl[] = { { "balance-rr", BOND_MODE_ROUNDROBIN, BOND_VALFLAG_DEFAULT}, @@ -240,6 +244,12 @@ static const struct bond_opt_value bond_coupled_control_tbl[] = { { NULL, -1, 0}, }; +static const struct bond_opt_value bond_broadcast_neigh_tbl[] = { + { "off", 0, BOND_VALFLAG_DEFAULT}, + { "on", 1, 0}, + { NULL, -1, 0} +}; + static const struct bond_option bond_opts[BOND_OPT_LAST] = { [BOND_OPT_MODE] = { .id = BOND_OPT_MODE, @@ -476,6 +486,13 @@ static const struct bond_option bond_opts[BOND_OPT_LAST] = { .values = bond_ad_actor_sys_prio_tbl, .set = bond_option_ad_actor_sys_prio_set, }, + [BOND_OPT_ACTOR_PORT_PRIO] = { + .id = BOND_OPT_ACTOR_PORT_PRIO, + .name = "actor_port_prio", + .unsuppmodes = BOND_MODE_ALL_EX(BIT(BOND_MODE_8023AD)), + .flags = BOND_OPTFLAG_RAWVAL, + .set = bond_option_actor_port_prio_set, + }, [BOND_OPT_AD_ACTOR_SYSTEM] = { .id = BOND_OPT_AD_ACTOR_SYSTEM, .name = "ad_actor_system", @@ -513,6 +530,14 @@ static const struct bond_option bond_opts[BOND_OPT_LAST] = { .flags = BOND_OPTFLAG_IFDOWN, .values = bond_coupled_control_tbl, .set = bond_option_coupled_control_set, + }, + [BOND_OPT_BROADCAST_NEIGH] = { + .id = BOND_OPT_BROADCAST_NEIGH, + .name = "broadcast_neighbor", + .desc = "Broadcast neighbor packets to all active slaves", + .unsuppmodes = BOND_MODE_ALL_EX(BIT(BOND_MODE_8023AD)), + .values = bond_broadcast_neigh_tbl, + .set = bond_option_broadcast_neigh_set, } }; @@ -894,6 +919,13 @@ static int bond_option_mode_set(struct bonding *bond, bond->params.arp_validate = BOND_ARP_VALIDATE_NONE; bond->params.mode = newval->value; + /* When changing mode, the bond device is down, we may reduce + * the bond_bcast_neigh_enabled in bond_close() if broadcast_neighbor + * enabled in 8023ad mode. Therefore, only clear broadcast_neighbor + * to 0. + */ + bond->params.broadcast_neighbor = 0; + if (bond->dev->reg_state == NETREG_REGISTERED) { bool update = false; @@ -1796,6 +1828,26 @@ static int bond_option_ad_actor_sys_prio_set(struct bonding *bond, return 0; } +static int bond_option_actor_port_prio_set(struct bonding *bond, + const struct bond_opt_value *newval) +{ + struct slave *slave; + + slave = bond_slave_get_rtnl(newval->slave_dev); + if (!slave) { + netdev_dbg(bond->dev, "%s called on NULL slave\n", __func__); + return -ENODEV; + } + + netdev_dbg(newval->slave_dev, "Setting actor_port_prio to %llu\n", + newval->value); + + SLAVE_AD_INFO(slave)->port_priority = newval->value; + bond_3ad_update_ad_actor_settings(bond); + + return 0; +} + static int bond_option_ad_actor_system_set(struct bonding *bond, const struct bond_opt_value *newval) { @@ -1843,3 +1895,22 @@ static int bond_option_coupled_control_set(struct bonding *bond, bond->params.coupled_control = newval->value; return 0; } + +static int bond_option_broadcast_neigh_set(struct bonding *bond, + const struct bond_opt_value *newval) +{ + if (bond->params.broadcast_neighbor == newval->value) + return 0; + + bond->params.broadcast_neighbor = newval->value; + if (bond->dev->flags & IFF_UP) { + if (bond->params.broadcast_neighbor) + static_branch_inc(&bond_bcast_neigh_enabled); + else + static_branch_dec(&bond_bcast_neigh_enabled); + } + + netdev_dbg(bond->dev, "Setting broadcast_neighbor to %s (%llu)\n", + newval->string, newval->value); + return 0; +} diff --git a/drivers/net/bonding/bond_procfs.c b/drivers/net/bonding/bond_procfs.c index 7edf72ec816a..62a5b40b43a2 100644 --- a/drivers/net/bonding/bond_procfs.c +++ b/drivers/net/bonding/bond_procfs.c @@ -187,6 +187,7 @@ static void bond_info_show_master(struct seq_file *seq) } } +/* Note: runs under rcu_read_lock() */ static void bond_info_show_slave(struct seq_file *seq, const struct slave *slave) { @@ -213,19 +214,19 @@ static void bond_info_show_slave(struct seq_file *seq, if (BOND_MODE(bond) == BOND_MODE_8023AD) { const struct port *port = &SLAVE_AD_INFO(slave)->port; - const struct aggregator *agg = port->aggregator; + const struct aggregator *agg = rcu_dereference(port->aggregator); if (agg) { seq_printf(seq, "Aggregator ID: %d\n", agg->aggregator_identifier); seq_printf(seq, "Actor Churn State: %s\n", - bond_3ad_churn_desc(port->sm_churn_actor_state)); + bond_3ad_churn_desc(READ_ONCE(port->sm_churn_actor_state))); seq_printf(seq, "Partner Churn State: %s\n", - bond_3ad_churn_desc(port->sm_churn_partner_state)); + bond_3ad_churn_desc(READ_ONCE(port->sm_churn_partner_state))); seq_printf(seq, "Actor Churned Count: %d\n", - port->churn_actor_count); + READ_ONCE(port->churn_actor_count)); seq_printf(seq, "Partner Churned Count: %d\n", - port->churn_partner_count); + READ_ONCE(port->churn_partner_count)); if (capable(CAP_NET_ADMIN)) { seq_puts(seq, "details actor lacp pdu:\n"); diff --git a/drivers/net/bonding/bond_sysfs_slave.c b/drivers/net/bonding/bond_sysfs_slave.c index 36d0e8440b5b..fc6fe7181789 100644 --- a/drivers/net/bonding/bond_sysfs_slave.c +++ b/drivers/net/bonding/bond_sysfs_slave.c @@ -62,10 +62,15 @@ static ssize_t ad_aggregator_id_show(struct slave *slave, char *buf) const struct aggregator *agg; if (BOND_MODE(slave->bond) == BOND_MODE_8023AD) { - agg = SLAVE_AD_INFO(slave)->port.aggregator; - if (agg) - return sysfs_emit(buf, "%d\n", - agg->aggregator_identifier); + rcu_read_lock(); + agg = rcu_dereference(SLAVE_AD_INFO(slave)->port.aggregator); + if (agg) { + ssize_t res = sysfs_emit(buf, "%d\n", + agg->aggregator_identifier); + rcu_read_unlock(); + return res; + } + rcu_read_unlock(); } return sysfs_emit(buf, "N/A\n"); @@ -78,7 +83,7 @@ static ssize_t ad_actor_oper_port_state_show(struct slave *slave, char *buf) if (BOND_MODE(slave->bond) == BOND_MODE_8023AD) { ad_port = &SLAVE_AD_INFO(slave)->port; - if (ad_port->aggregator) + if (rcu_access_pointer(ad_port->aggregator)) return sysfs_emit(buf, "%u\n", ad_port->actor_oper_port_state); } @@ -93,7 +98,7 @@ static ssize_t ad_partner_oper_port_state_show(struct slave *slave, char *buf) if (BOND_MODE(slave->bond) == BOND_MODE_8023AD) { ad_port = &SLAVE_AD_INFO(slave)->port; - if (ad_port->aggregator) + if (rcu_access_pointer(ad_port->aggregator)) return sysfs_emit(buf, "%u\n", ad_port->partner_oper.port_state); } diff --git a/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.c b/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.c index f3bea196a8f9..1c118381a0bf 100644 --- a/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.c +++ b/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.c @@ -210,8 +210,8 @@ static void rmnet_dellink(struct net_device *dev, struct list_head *head) ep = rmnet_get_endpoint(real_port, mux_id); if (ep) { hlist_del_init_rcu(&ep->hlnode); - rmnet_vnd_dellink(mux_id, real_port, ep); - kfree(ep); + real_port->nr_rmnet_devs--; + kfree_rcu(ep, rcu); } netdev_upper_dev_unlink(real_dev, dev); @@ -235,9 +235,9 @@ static void rmnet_force_unassociate_device(struct net_device *real_dev) hash_for_each_safe(port->muxed_ep, bkt_ep, tmp_ep, ep, hlnode) { unregister_netdevice_queue(ep->egress_dev, &list); netdev_upper_dev_unlink(real_dev, ep->egress_dev); - rmnet_vnd_dellink(ep->mux_id, port, ep); hlist_del_init_rcu(&ep->hlnode); - kfree(ep); + port->nr_rmnet_devs--; + kfree_rcu(ep, rcu); } rmnet_unregister_real_device(real_dev); unregister_netdevice_many(&list); diff --git a/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.h b/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.h index ed112d51ac5a..f50fae1c6bdd 100644 --- a/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.h +++ b/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.h @@ -18,6 +18,7 @@ struct rmnet_endpoint { u8 mux_id; struct net_device *egress_dev; struct hlist_node hlnode; + struct rcu_head rcu; }; struct rmnet_egress_agg_params { diff --git a/drivers/net/wan/hdlc_ppp.c b/drivers/net/wan/hdlc_ppp.c index 37a3c989cba1..27b771146656 100644 --- a/drivers/net/wan/hdlc_ppp.c +++ b/drivers/net/wan/hdlc_ppp.c @@ -621,7 +621,6 @@ static void ppp_start(struct net_device *dev) struct proto *proto = &ppp->protos[i]; proto->dev = dev; - timer_setup(&proto->timer, ppp_timer, 0); proto->state = CLOSED; } ppp->protos[IDX_LCP].pid = PID_LCP; @@ -641,6 +640,15 @@ static void ppp_close(struct net_device *dev) ppp_tx_flush(); } +static void ppp_timer_release(struct net_device *dev) +{ + struct ppp *ppp = get_ppp(dev); + int i; + + for (i = 0; i < IDX_COUNT; i++) + timer_shutdown_sync(&ppp->protos[i].timer); +} + static struct hdlc_proto proto = { .start = ppp_start, .stop = ppp_stop, @@ -649,6 +657,7 @@ static struct hdlc_proto proto = { .ioctl = ppp_ioctl, .netif_rx = ppp_rx, .module = THIS_MODULE, + .detach = ppp_timer_release, }; static const struct header_ops ppp_header_ops = { @@ -659,7 +668,7 @@ static int ppp_ioctl(struct net_device *dev, struct if_settings *ifs) { hdlc_device *hdlc = dev_to_hdlc(dev); struct ppp *ppp; - int result; + int i, result; switch (ifs->type) { case IF_GET_PROTO: @@ -687,6 +696,8 @@ static int ppp_ioctl(struct net_device *dev, struct if_settings *ifs) return result; ppp = get_ppp(dev); + for (i = 0; i < IDX_COUNT; i++) + timer_setup(&ppp->protos[i].timer, ppp_timer, 0); spin_lock_init(&ppp->lock); ppp->req_timeout = 2; ppp->cr_retries = 10; diff --git a/drivers/net/wireless/ath/ath11k/dp.c b/drivers/net/wireless/ath/ath11k/dp.c index f124b7329e1a..06b574ff950e 100644 --- a/drivers/net/wireless/ath/ath11k/dp.c +++ b/drivers/net/wireless/ath/ath11k/dp.c @@ -1040,6 +1040,7 @@ void ath11k_dp_free(struct ath11k_base *ab) idr_destroy(&dp->tx_ring[i].txbuf_idr); spin_unlock_bh(&dp->tx_ring[i].tx_idr_lock); kfree(dp->tx_ring[i].tx_status); + dp->tx_ring[i].tx_status = NULL; } /* Deinit any SOC level resource */ diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ptp.c b/drivers/net/wireless/intel/iwlwifi/mvm/ptp.c index e89259de6f4c..273a6718c907 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/ptp.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/ptp.c @@ -316,11 +316,11 @@ void iwl_mvm_ptp_remove(struct iwl_mvm *mvm) mvm->ptp_data.ptp_clock_info.name, ptp_clock_index(mvm->ptp_data.ptp_clock)); + cancel_delayed_work_sync(&mvm->ptp_data.dwork); ptp_clock_unregister(mvm->ptp_data.ptp_clock); mvm->ptp_data.ptp_clock = NULL; memset(&mvm->ptp_data.ptp_clock_info, 0, sizeof(mvm->ptp_data.ptp_clock_info)); mvm->ptp_data.last_gp2 = 0; - cancel_delayed_work_sync(&mvm->ptp_data.dwork); } } diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/usb.c b/drivers/net/wireless/mediatek/mt76/mt76x2/usb.c index 96cecc576a98..427b294423d3 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76x2/usb.c +++ b/drivers/net/wireless/mediatek/mt76/mt76x2/usb.c @@ -16,6 +16,7 @@ static const struct usb_device_id mt76x2u_device_table[] = { { USB_DEVICE(0x0e8d, 0x7612) }, /* Aukey USBAC1200 - Alfa AWUS036ACM */ { USB_DEVICE(0x057c, 0x8503) }, /* Avm FRITZ!WLAN AC860 */ { USB_DEVICE(0x7392, 0xb711) }, /* Edimax EW 7722 UAC */ + { USB_DEVICE(0x056e, 0x400a) }, /* ELECOM WDC-867SU3S */ { USB_DEVICE(0x0e8d, 0x7632) }, /* HC-M7662BU1 */ { USB_DEVICE(0x0471, 0x2126) }, /* LiteOn WN4516R module, nonstandard USB connector */ { USB_DEVICE(0x0471, 0x7600) }, /* LiteOn WN4519R module, nonstandard USB connector */ diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/init.c b/drivers/net/wireless/mediatek/mt76/mt7921/init.c index 4bd533c4ba9a..276dfb9c26e0 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/init.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/init.c @@ -137,6 +137,13 @@ mt7921_regd_notifier(struct wiphy *wiphy, dev->mt76.region = request->dfs_region; dev->country_ie_env = request->country_ie_env; + if (request->initiator == NL80211_REGDOM_SET_BY_USER) { + if (dev->mt76.alpha2[0] == '0' && dev->mt76.alpha2[1] == '0') + wiphy->regulatory_flags &= ~REGULATORY_COUNTRY_IE_IGNORE; + else + wiphy->regulatory_flags |= REGULATORY_COUNTRY_IE_IGNORE; + } + if (pm->suspended) return; diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/main.c b/drivers/net/wireless/mediatek/mt76/mt7921/main.c index a93ae4e44f16..99561094640f 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/main.c @@ -365,12 +365,15 @@ void mt7921_roc_abort_sync(struct mt792x_dev *dev) { struct mt792x_phy *phy = &dev->phy; + if (!test_and_clear_bit(MT76_STATE_ROC, &phy->mt76->state)) + return; + del_timer_sync(&phy->roc_timer); - cancel_work_sync(&phy->roc_work); - if (test_and_clear_bit(MT76_STATE_ROC, &phy->mt76->state)) - ieee80211_iterate_active_interfaces(mt76_hw(dev), - IEEE80211_IFACE_ITER_RESUME_ALL, - mt7921_roc_iter, (void *)phy); + cancel_work(&phy->roc_work); + + ieee80211_iterate_interfaces(mt76_hw(dev), + IEEE80211_IFACE_ITER_RESUME_ALL, + mt7921_roc_iter, (void *)phy); } EXPORT_SYMBOL_GPL(mt7921_roc_abort_sync); @@ -881,6 +884,7 @@ void mt7921_mac_sta_remove(struct mt76_dev *mdev, struct ieee80211_vif *vif, struct mt792x_dev *dev = container_of(mdev, struct mt792x_dev, mt76); struct mt792x_sta *msta = (struct mt792x_sta *)sta->drv_priv; + mt7921_roc_abort_sync(dev); mt76_connac_free_pending_tx_skbs(&dev->pm, &msta->deflink.wcid); mt76_connac_pm_wake(&dev->mphy, &dev->pm); diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/main.c b/drivers/net/wireless/mediatek/mt76/mt7925/main.c index 5fc95c862364..b1f54cb54e17 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7925/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7925/main.c @@ -1190,6 +1190,9 @@ mt7925_mac_sta_remove_links(struct mt792x_dev *dev, struct ieee80211_vif *vif, if (vif->type == NL80211_IFTYPE_AP) break; + if (vif->type == NL80211_IFTYPE_STATION && sta->tdls) + continue; + link_sta = mt792x_sta_to_link_sta(vif, sta, link_id); if (!link_sta) continue; diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/trx.h b/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/trx.h index 1155365348f3..d5de09d75f45 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/trx.h +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/trx.h @@ -291,7 +291,7 @@ static inline int get_rx_desc_paggr(__le32 *__pdesc) static inline int get_rx_status_desc_rpt_sel(__le32 *__pdesc) { - return le32_get_bits(*(__pdesc + 1), BIT(28)); + return le32_get_bits(*(__pdesc + 2), BIT(28)); } static inline int get_rx_desc_rxmcs(__le32 *__pdesc) diff --git a/drivers/net/wireless/realtek/rtw88/tx.c b/drivers/net/wireless/realtek/rtw88/tx.c index dae7ca148865..2e66140c6482 100644 --- a/drivers/net/wireless/realtek/rtw88/tx.c +++ b/drivers/net/wireless/realtek/rtw88/tx.c @@ -191,6 +191,7 @@ void rtw_tx_report_purge_timer(struct timer_list *t) void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn) { struct rtw_tx_report *tx_report = &rtwdev->tx_report; + unsigned long timeout = RTW_TX_PROBE_TIMEOUT; unsigned long flags; u8 *drv_data; @@ -202,7 +203,11 @@ void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn) __skb_queue_tail(&tx_report->queue, skb); spin_unlock_irqrestore(&tx_report->q_lock, flags); - mod_timer(&tx_report->purge_timer, jiffies + RTW_TX_PROBE_TIMEOUT); + if (rtwdev->chip->id == RTW_CHIP_TYPE_8723D && + rtwdev->hci.type == RTW_HCI_TYPE_USB) + timeout = msecs_to_jiffies(2500); + + mod_timer(&tx_report->purge_timer, jiffies + timeout); } EXPORT_SYMBOL(rtw_tx_report_enqueue); diff --git a/drivers/net/wireless/realtek/rtw88/usb.c b/drivers/net/wireless/realtek/rtw88/usb.c index 52b83191f325..b3e1e9a0fda6 100644 --- a/drivers/net/wireless/realtek/rtw88/usb.c +++ b/drivers/net/wireless/realtek/rtw88/usb.c @@ -398,6 +398,7 @@ static bool rtw_usb_tx_agg_skb(struct rtw_usb *rtwusb, struct sk_buff_head *list int agg_num = 0; unsigned int align_next = 0; u8 qsel; + int ret; if (skb_queue_empty(list)) return false; @@ -455,7 +456,13 @@ static bool rtw_usb_tx_agg_skb(struct rtw_usb *rtwusb, struct sk_buff_head *list tx_desc = (struct rtw_tx_desc *)skb_head->data; qsel = le32_get_bits(tx_desc->w1, RTW_TX_DESC_W1_QSEL); - rtw_usb_write_port(rtwdev, qsel, skb_head, rtw_usb_write_port_tx_complete, txcb); + ret = rtw_usb_write_port(rtwdev, qsel, skb_head, + rtw_usb_write_port_tx_complete, txcb); + if (ret) { + ieee80211_purge_tx_queue(rtwdev->hw, &txcb->tx_ack_queue); + kfree(txcb); + return false; + } return true; } @@ -517,8 +524,10 @@ static int rtw_usb_write_data(struct rtw_dev *rtwdev, ret = rtw_usb_write_port(rtwdev, qsel, skb, rtw_usb_write_port_complete, skb); - if (unlikely(ret)) + if (unlikely(ret)) { rtw_err(rtwdev, "failed to do USB write, ret=%d\n", ret); + dev_kfree_skb_any(skb); + } return ret; } diff --git a/drivers/ntb/hw/epf/ntb_hw_epf.c b/drivers/ntb/hw/epf/ntb_hw_epf.c index 2b51156e01b0..f248c03134a2 100644 --- a/drivers/ntb/hw/epf/ntb_hw_epf.c +++ b/drivers/ntb/hw/epf/ntb_hw_epf.c @@ -646,7 +646,8 @@ static void ntb_epf_deinit_pci(struct ntb_epf_dev *ndev) struct pci_dev *pdev = ndev->ntb.pdev; pci_iounmap(pdev, ndev->ctrl_reg); - pci_iounmap(pdev, ndev->peer_spad_reg); + if (ndev->barno_map[BAR_PEER_SPAD] != ndev->barno_map[BAR_CONFIG]) + pci_iounmap(pdev, ndev->peer_spad_reg); pci_iounmap(pdev, ndev->db_reg); pci_release_regions(pdev); diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c index ae0f36e270ba..5d27cd149f51 100644 --- a/drivers/pci/controller/dwc/pcie-qcom.c +++ b/drivers/pci/controller/dwc/pcie-qcom.c @@ -329,20 +329,15 @@ static void qcom_pcie_clear_aspm_l0s(struct dw_pcie *pci) dw_pcie_dbi_ro_wr_dis(pci); } -static void qcom_pcie_set_slot_nccs(struct dw_pcie *pci) +static void qcom_pcie_clear_hpc(struct dw_pcie *pci) { u16 offset = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP); u32 val; dw_pcie_dbi_ro_wr_en(pci); - /* - * Qcom PCIe Root Ports do not support generating command completion - * notifications for the Hot-Plug commands. So set the NCCS field to - * avoid waiting for the completions. - */ val = readl(pci->dbi_base + offset + PCI_EXP_SLTCAP); - val |= PCI_EXP_SLTCAP_NCCS; + val &= ~PCI_EXP_SLTCAP_HPC; writel(val, pci->dbi_base + offset + PCI_EXP_SLTCAP); dw_pcie_dbi_ro_wr_dis(pci); @@ -537,7 +532,7 @@ static int qcom_pcie_post_init_2_1_0(struct qcom_pcie *pcie) writel(CFG_BRIDGE_SB_INIT, pci->dbi_base + AXI_MSTR_RESP_COMP_CTRL1); - qcom_pcie_set_slot_nccs(pcie->pci); + qcom_pcie_clear_hpc(pcie->pci); return 0; } @@ -617,7 +612,7 @@ static int qcom_pcie_post_init_1_0_0(struct qcom_pcie *pcie) writel(val, pcie->parf + PARF_AXI_MSTR_WR_ADDR_HALT); } - qcom_pcie_set_slot_nccs(pcie->pci); + qcom_pcie_clear_hpc(pcie->pci); return 0; } @@ -710,7 +705,7 @@ static int qcom_pcie_post_init_2_3_2(struct qcom_pcie *pcie) val |= EN; writel(val, pcie->parf + PARF_AXI_MSTR_WR_ADDR_HALT_V2); - qcom_pcie_set_slot_nccs(pcie->pci); + qcom_pcie_clear_hpc(pcie->pci); return 0; } @@ -1014,7 +1009,7 @@ static int qcom_pcie_post_init_2_7_0(struct qcom_pcie *pcie) writel(WR_NO_SNOOP_OVERIDE_EN | RD_NO_SNOOP_OVERIDE_EN, pcie->parf + PARF_NO_SNOOP_OVERIDE); - qcom_pcie_set_slot_nccs(pcie->pci); + qcom_pcie_clear_hpc(pcie->pci); return 0; } diff --git a/drivers/power/reset/linkstation-poweroff.c b/drivers/power/reset/linkstation-poweroff.c index 02f5fdb8ffc4..e56d75bfcc43 100644 --- a/drivers/power/reset/linkstation-poweroff.c +++ b/drivers/power/reset/linkstation-poweroff.c @@ -163,10 +163,10 @@ static int __init linkstation_poweroff_init(void) dn = of_find_matching_node(NULL, ls_poweroff_of_match); if (!dn) return -ENODEV; - of_node_put(dn); match = of_match_node(ls_poweroff_of_match, dn); cfg = match->data; + of_node_put(dn); dn = of_find_node_by_name(NULL, cfg->mdio_node_name); if (!dn) diff --git a/drivers/power/sequencing/core.c b/drivers/power/sequencing/core.c index 70b92e1c3ac5..50aed0a5e8b2 100644 --- a/drivers/power/sequencing/core.c +++ b/drivers/power/sequencing/core.c @@ -990,8 +990,9 @@ static void *pwrseq_debugfs_seq_start(struct seq_file *seq, loff_t *pos) ctx.index = *pos; /* - * We're holding the lock for the entire printout so no need to fiddle - * with device reference count. + * Hold the lock for the entire printout to prevent device removal. + * Reference counts are managed by start()/next()/stop() as required + * by the seq_file contract. */ down_read(&pwrseq_sem); @@ -999,7 +1000,7 @@ static void *pwrseq_debugfs_seq_start(struct seq_file *seq, loff_t *pos) if (!ctx.index) return NULL; - return ctx.dev; + return get_device(ctx.dev); } static void *pwrseq_debugfs_seq_next(struct seq_file *seq, void *data, @@ -1009,8 +1010,9 @@ static void *pwrseq_debugfs_seq_next(struct seq_file *seq, void *data, ++*pos; - struct device *next __free(put_device) = - bus_find_next_device(&pwrseq_bus, curr); + struct device *next = bus_find_next_device(&pwrseq_bus, curr); + + put_device(curr); return next; } @@ -1059,6 +1061,8 @@ static int pwrseq_debugfs_seq_show(struct seq_file *seq, void *data) static void pwrseq_debugfs_seq_stop(struct seq_file *seq, void *data) { + if (data) + put_device(data); up_read(&pwrseq_sem); } diff --git a/drivers/regulator/core.c b/drivers/regulator/core.c index 078d3dc50aa3..54fd06891bdc 100644 --- a/drivers/regulator/core.c +++ b/drivers/regulator/core.c @@ -2116,8 +2116,16 @@ static int regulator_resolve_supply(struct regulator_dev *rdev) if (rdev->use_count) { ret = regulator_enable(rdev->supply); if (ret < 0) { - _regulator_put(rdev->supply); + struct regulator *supply; + + regulator_lock_two(rdev, rdev->supply->rdev, &ww_ctx); + + supply = rdev->supply; rdev->supply = NULL; + + regulator_unlock_two(rdev, supply->rdev, &ww_ctx); + + regulator_put(supply); goto out; } } diff --git a/drivers/rpmsg/rpmsg_char.c b/drivers/rpmsg/rpmsg_char.c index 96fcdd2d7093..d918cb30db9b 100644 --- a/drivers/rpmsg/rpmsg_char.c +++ b/drivers/rpmsg/rpmsg_char.c @@ -104,6 +104,9 @@ static int rpmsg_ept_cb(struct rpmsg_device *rpdev, void *buf, int len, struct rpmsg_eptdev *eptdev = priv; struct sk_buff *skb; + if (!eptdev) + return 0; + skb = alloc_skb(len, GFP_ATOMIC); if (!skb) return -ENOMEM; @@ -124,6 +127,9 @@ static int rpmsg_ept_flow_cb(struct rpmsg_device *rpdev, void *priv, bool enable { struct rpmsg_eptdev *eptdev = priv; + if (!eptdev) + return 0; + eptdev->remote_flow_restricted = enable; eptdev->remote_flow_updated = true; @@ -490,6 +496,7 @@ static int rpmsg_chrdev_probe(struct rpmsg_device *rpdev) struct rpmsg_channel_info chinfo; struct rpmsg_eptdev *eptdev; struct device *dev = &rpdev->dev; + int ret; memcpy(chinfo.name, rpdev->id.name, RPMSG_NAME_SIZE); chinfo.src = rpdev->src; @@ -502,13 +509,17 @@ static int rpmsg_chrdev_probe(struct rpmsg_device *rpdev) /* Set the default_ept to the rpmsg device endpoint */ eptdev->default_ept = rpdev->ept; + ret = rpmsg_chrdev_eptdev_add(eptdev, chinfo); + + if (ret) + return ret; /* * The rpmsg_ept_cb uses *priv parameter to get its rpmsg_eptdev context. - * Storedit in default_ept *priv field. + * Stored it in default_ept *priv field. */ eptdev->default_ept->priv = eptdev; - return rpmsg_chrdev_eptdev_add(eptdev, chinfo); + return 0; } static void rpmsg_chrdev_remove(struct rpmsg_device *rpdev) diff --git a/drivers/staging/rtl8723bs/core/rtw_mlme.c b/drivers/staging/rtl8723bs/core/rtw_mlme.c index 3563017acc31..d4b6e748f11f 100644 --- a/drivers/staging/rtl8723bs/core/rtw_mlme.c +++ b/drivers/staging/rtl8723bs/core/rtw_mlme.c @@ -449,8 +449,11 @@ static void update_current_network(struct adapter *adapter, struct wlan_bssid_ex if ((check_fwstate(pmlmepriv, _FW_LINKED) == true) && (is_same_network(&(pmlmepriv->cur_network.network), pnetwork, 0))) { update_network(&(pmlmepriv->cur_network.network), pnetwork, adapter, true); + if (pmlmepriv->cur_network.network.ie_length < sizeof(struct ndis_802_11_fix_ie)) + return; + rtw_update_protection(adapter, (pmlmepriv->cur_network.network.ies) + sizeof(struct ndis_802_11_fix_ie), - pmlmepriv->cur_network.network.ie_length); + pmlmepriv->cur_network.network.ie_length - sizeof(struct ndis_802_11_fix_ie)); } } @@ -1070,8 +1073,11 @@ static void rtw_joinbss_update_network(struct adapter *padapter, struct wlan_net break; } + if (cur_network->network.ie_length < sizeof(struct ndis_802_11_fix_ie)) + return; + rtw_update_protection(padapter, (cur_network->network.ies) + sizeof(struct ndis_802_11_fix_ie), - (cur_network->network.ie_length)); + (cur_network->network.ie_length - sizeof(struct ndis_802_11_fix_ie))); rtw_update_ht_cap(padapter, cur_network->network.ies, cur_network->network.ie_length, (u8) cur_network->network.configuration.ds_config); } diff --git a/drivers/tty/serial/8250/8250_dw.c b/drivers/tty/serial/8250/8250_dw.c index f17dc3de020c..7b2e72bfeb75 100644 --- a/drivers/tty/serial/8250/8250_dw.c +++ b/drivers/tty/serial/8250/8250_dw.c @@ -655,8 +655,10 @@ static int dw8250_probe(struct platform_device *pdev) */ if (data->clk) { err = clk_notifier_register(data->clk, &data->clk_notifier); - if (err) + if (err) { + serial8250_unregister_port(data->data.line); return dev_err_probe(dev, err, "Failed to set the clock notifier\n"); + } queue_work(system_unbound_wq, &data->clk_work); } diff --git a/drivers/tty/serial/qcom_geni_serial.c b/drivers/tty/serial/qcom_geni_serial.c index c09526e2d423..cf705b01e8bd 100644 --- a/drivers/tty/serial/qcom_geni_serial.c +++ b/drivers/tty/serial/qcom_geni_serial.c @@ -867,12 +867,9 @@ static void qcom_geni_serial_handle_rx_dma(struct uart_port *uport, bool drop) port->rx_dma_addr = 0; rx_in = readl(uport->membase + SE_DMA_RX_LEN_IN); - if (!rx_in) { - dev_warn(uport->dev, "serial engine reports 0 RX bytes in!\n"); - return; - } - - if (!drop) + if (!rx_in) + dev_warn_ratelimited(uport->dev, "serial engine reports 0 RX bytes in!\n"); + else if (!drop) handle_rx_uart(uport, rx_in); ret = geni_se_rx_dma_prep(&port->se, port->rx_buf, diff --git a/drivers/tty/vt/vc_screen.c b/drivers/tty/vt/vc_screen.c index 79b33d998d43..253663515355 100644 --- a/drivers/tty/vt/vc_screen.c +++ b/drivers/tty/vt/vc_screen.c @@ -699,7 +699,7 @@ vcs_write(struct file *file, const char __user *buf, size_t count, loff_t *ppos) } *ppos += written; ret = written; - if (written) + if (written && vc) vcs_scr_updated(vc); unlock_out: diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c index 86b9a371645b..a689b390fce3 100644 --- a/drivers/video/fbdev/core/fbmem.c +++ b/drivers/video/fbdev/core/fbmem.c @@ -680,6 +680,18 @@ int fb_new_modelist(struct fb_info *info) if (list_empty(&info->modelist)) return 1; + /* + * The new modelist may not contain the current mode (info->var), and + * fbcon_new_modelist() below only re-points consoles mapped to this + * framebuffer. Add the current mode here so info->var keeps a match + * even when fbcon is unbound. + */ + if (!fb_match_mode(&info->var, &info->modelist)) { + fb_var_to_videomode(&mode, &info->var); + if (fb_add_videomode(&mode, &info->modelist)) + return 1; + } + fbcon_new_modelist(info); return 0; diff --git a/drivers/video/fbdev/core/fbsysfs.c b/drivers/video/fbdev/core/fbsysfs.c index 1b3c9958ef5c..10c3cfb959f4 100644 --- a/drivers/video/fbdev/core/fbsysfs.c +++ b/drivers/video/fbdev/core/fbsysfs.c @@ -11,6 +11,7 @@ #include #include "fb_internal.h" +#include "fbcon.h" #define FB_SYSFS_FLAG_ATTR 1 @@ -113,8 +114,15 @@ static ssize_t store_modes(struct device *device, if (fb_new_modelist(fb_info)) { fb_destroy_modelist(&fb_info->modelist); list_splice(&old_list, &fb_info->modelist); - } else + } else { + /* + * fb_display[i].mode and fb_info->mode both point into the old + * list. Clear them before it is freed. + */ + fbcon_delete_modelist(&old_list); + fb_info->mode = NULL; fb_destroy_modelist(&old_list); + } unlock_fb_info(fb_info); console_unlock(); diff --git a/drivers/video/fbdev/core/modedb.c b/drivers/video/fbdev/core/modedb.c index 7196b055f2bd..7428d58e6ca8 100644 --- a/drivers/video/fbdev/core/modedb.c +++ b/drivers/video/fbdev/core/modedb.c @@ -258,7 +258,7 @@ static const struct fb_videomode modedb[] = { FB_VMODE_DOUBLE }, /* 1920x1080 @ 60 Hz, 67.3 kHz hsync */ - { NULL, 60, 1920, 1080, 6734, 148, 88, 36, 4, 44, 5, 0, + { NULL, 60, 1920, 1080, 6734, 148, 88, 36, 4, 44, 5, FB_SYNC_HOR_HIGH_ACT | FB_SYNC_VERT_HIGH_ACT, FB_VMODE_NONINTERLACED }, @@ -625,7 +625,7 @@ int fb_find_mode(struct fb_var_screeninfo *var, const struct fb_videomode *default_mode, unsigned int default_bpp) { - char *mode_option_buf = NULL; + char *mode_option_buf __free(kfree) = NULL; int i; /* Set up defaults */ @@ -723,7 +723,6 @@ int fb_find_mode(struct fb_var_screeninfo *var, res_specified = 1; } done: - kfree(mode_option_buf); if (cvt) { struct fb_videomode cvt_mode; int ret; diff --git a/fs/backing-file.c b/fs/backing-file.c index 09a9be945d45..53690754810f 100644 --- a/fs/backing-file.c +++ b/fs/backing-file.c @@ -12,6 +12,7 @@ #include #include #include +#include #include "internal.h" @@ -29,19 +30,20 @@ * returned file into a container structure that also stores the stacked * file's path, which can be retrieved using backing_file_user_path(). */ -struct file *backing_file_open(const struct path *user_path, int flags, +struct file *backing_file_open(const struct file *user_file, int flags, const struct path *real_path, const struct cred *cred) { + const struct path *user_path = &user_file->f_path; struct file *f; int error; - f = alloc_empty_backing_file(flags, cred); + f = alloc_empty_backing_file(flags, cred, user_file); if (IS_ERR(f)) return f; path_get(user_path); - *backing_file_user_path(f) = *user_path; + backing_file_set_user_path(f, user_path); error = vfs_open(real_path, f); if (error) { fput(f); @@ -52,20 +54,21 @@ struct file *backing_file_open(const struct path *user_path, int flags, } EXPORT_SYMBOL_GPL(backing_file_open); -struct file *backing_tmpfile_open(const struct path *user_path, int flags, +struct file *backing_tmpfile_open(const struct file *user_file, int flags, const struct path *real_parentpath, umode_t mode, const struct cred *cred) { struct mnt_idmap *real_idmap = mnt_idmap(real_parentpath->mnt); + const struct path *user_path = &user_file->f_path; struct file *f; int error; - f = alloc_empty_backing_file(flags, cred); + f = alloc_empty_backing_file(flags, cred, user_file); if (IS_ERR(f)) return f; path_get(user_path); - *backing_file_user_path(f) = *user_path; + backing_file_set_user_path(f, user_path); error = vfs_tmpfile(real_idmap, real_parentpath, f, mode); if (error) { fput(f); @@ -326,6 +329,7 @@ EXPORT_SYMBOL_GPL(backing_file_splice_write); int backing_file_mmap(struct file *file, struct vm_area_struct *vma, struct backing_file_ctx *ctx) { + struct file *user_file = vma->vm_file; const struct cred *old_cred; int ret; @@ -339,6 +343,12 @@ int backing_file_mmap(struct file *file, struct vm_area_struct *vma, vma_set_file(vma, file); old_cred = override_creds(ctx->cred); + ret = security_mmap_backing_file(vma, file, user_file); + if (ret) { + revert_creds(old_cred); + return ret; + } + ret = call_mmap(vma->vm_file, vma); revert_creds(old_cred); diff --git a/fs/dlm/lockspace.c b/fs/dlm/lockspace.c index 7b4b6977dcd6..ee11a70def92 100644 --- a/fs/dlm/lockspace.c +++ b/fs/dlm/lockspace.c @@ -576,7 +576,7 @@ static int new_lockspace(const char *name, const char *cluster, lockspace to start running (via sysfs) in dlm_ls_start(). */ error = do_uevent(ls, 1); - if (error) + if (error < 0) goto out_recoverd; /* wait until recovery is successful or failed */ diff --git a/fs/eventpoll.c b/fs/eventpoll.c index a860cb54658a..22605fbc12de 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -798,36 +798,47 @@ static void ep_free(struct eventpoll *ep) } /* - * Removes a "struct epitem" from the eventpoll RB tree and deallocates - * all the associated resources. Must be called with "mtx" held. - * If the dying flag is set, do the removal only if force is true. - * This prevents ep_clear_and_put() from dropping all the ep references - * while running concurrently with eventpoll_release_file(). - * Returns true if the eventpoll can be disposed. + * The ffd.file pointer may be in the process of being torn down due to + * being closed, but we may not have finished eventpoll_release() yet. + * + * Normally, even with the atomic_long_inc_not_zero, the file may have + * been free'd and then gotten re-allocated to something else (since + * files are not RCU-delayed, they are SLAB_TYPESAFE_BY_RCU). + * + * But for epoll, users hold the ep->mtx mutex, and as such any file in + * the process of being free'd will block in eventpoll_release_file() + * and thus the underlying file allocation will not be free'd, and the + * file re-use cannot happen. + * + * For the same reason we can avoid a rcu_read_lock() around the + * operation - 'ffd.file' cannot go away even if the refcount has + * reached zero (but we must still not call out to ->poll() functions + * etc). */ -static bool __ep_remove(struct eventpoll *ep, struct epitem *epi, bool force) +static struct file *epi_fget(const struct epitem *epi) { - struct file *file = epi->ffd.file; - struct epitems_head *to_free; - struct hlist_head *head; + struct file *file; - lockdep_assert_irqs_enabled(); + file = epi->ffd.file; + if (!atomic_long_inc_not_zero(&file->f_count)) + file = NULL; + return file; +} - /* - * Removes poll wait queue hooks. - */ - ep_unregister_pollwait(ep, epi); +/* + * Called with &file->f_lock held, + * returns with it released + */ +static void ep_remove_file(struct eventpoll *ep, struct epitem *epi, + struct file *file) +{ + struct epitems_head *to_free = NULL; + struct hlist_head *head = file->f_ep; - /* Remove the current item from the list of epoll hooks */ - spin_lock(&file->f_lock); - if (epi->dying && !force) { - spin_unlock(&file->f_lock); - return false; - } + lockdep_assert_held(&ep->mtx); + lockdep_assert_held(&file->f_lock); - to_free = NULL; - head = file->f_ep; - if (head->first == &epi->fllink && !epi->fllink.next) { + if (hlist_is_singular_node(&epi->fllink, head)) { /* See eventpoll_release() for details. */ WRITE_ONCE(file->f_ep, NULL); if (!is_file_epoll(file)) { @@ -840,6 +851,11 @@ static bool __ep_remove(struct eventpoll *ep, struct epitem *epi, bool force) hlist_del_rcu(&epi->fllink); spin_unlock(&file->f_lock); free_ephead(to_free); +} + +static bool ep_remove_epi(struct eventpoll *ep, struct epitem *epi) +{ + lockdep_assert_held(&ep->mtx); rb_erase_cached(&epi->rbn, &ep->rbr); @@ -865,9 +881,31 @@ static bool __ep_remove(struct eventpoll *ep, struct epitem *epi, bool force) /* * ep_remove variant for callers owing an additional reference to the ep */ -static void ep_remove_safe(struct eventpoll *ep, struct epitem *epi) +static void ep_remove(struct eventpoll *ep, struct epitem *epi) { - if (__ep_remove(ep, epi, false)) + struct file *file __free(fput) = NULL; + + lockdep_assert_irqs_enabled(); + lockdep_assert_held(&ep->mtx); + + ep_unregister_pollwait(ep, epi); + + /* cheap sync with eventpoll_release_file() */ + if (unlikely(READ_ONCE(epi->dying))) + return; + + /* + * If we manage to grab a reference it means we're not in + * eventpoll_release_file() and aren't going to be. + */ + file = epi_fget(epi); + if (!file) + return; + + spin_lock(&file->f_lock); + ep_remove_file(ep, epi, file); + + if (ep_remove_epi(ep, epi)) WARN_ON_ONCE(ep_refcount_dec_and_test(ep)); } @@ -894,7 +932,7 @@ static void ep_clear_and_put(struct eventpoll *ep) /* * Walks through the whole tree and try to free each "struct epitem". - * Note that ep_remove_safe() will not remove the epitem in case of a + * Note that ep_remove() will not remove the epitem in case of a * racing eventpoll_release_file(); the latter will do the removal. * At this point we are sure no poll callbacks will be lingering around. * Since we still own a reference to the eventpoll struct, the loop can't @@ -903,7 +941,7 @@ static void ep_clear_and_put(struct eventpoll *ep) for (rbp = rb_first_cached(&ep->rbr); rbp; rbp = next) { next = rb_next(rbp); epi = rb_entry(rbp, struct epitem, rbn); - ep_remove_safe(ep, epi); + ep_remove(ep, epi); cond_resched(); } @@ -983,34 +1021,6 @@ static __poll_t __ep_eventpoll_poll(struct file *file, poll_table *wait, int dep return res; } -/* - * The ffd.file pointer may be in the process of being torn down due to - * being closed, but we may not have finished eventpoll_release() yet. - * - * Normally, even with the atomic_long_inc_not_zero, the file may have - * been free'd and then gotten re-allocated to something else (since - * files are not RCU-delayed, they are SLAB_TYPESAFE_BY_RCU). - * - * But for epoll, users hold the ep->mtx mutex, and as such any file in - * the process of being free'd will block in eventpoll_release_file() - * and thus the underlying file allocation will not be free'd, and the - * file re-use cannot happen. - * - * For the same reason we can avoid a rcu_read_lock() around the - * operation - 'ffd.file' cannot go away even if the refcount has - * reached zero (but we must still not call out to ->poll() functions - * etc). - */ -static struct file *epi_fget(const struct epitem *epi) -{ - struct file *file; - - file = epi->ffd.file; - if (!atomic_long_inc_not_zero(&file->f_count)) - file = NULL; - return file; -} - /* * Differs from ep_eventpoll_poll() in that internal callers already have * the ep->mtx so we need to start from depth=1, such that mutex_lock_nested() @@ -1099,7 +1109,7 @@ void eventpoll_release_file(struct file *file) spin_lock(&file->f_lock); if (file->f_ep && file->f_ep->first) { epi = hlist_entry(file->f_ep->first, struct epitem, fllink); - epi->dying = true; + WRITE_ONCE(epi->dying, true); spin_unlock(&file->f_lock); /* @@ -1108,7 +1118,13 @@ void eventpoll_release_file(struct file *file) */ ep = epi->ep; mutex_lock(&ep->mtx); - dispose = __ep_remove(ep, epi, true); + + ep_unregister_pollwait(ep, epi); + + spin_lock(&file->f_lock); + ep_remove_file(ep, epi, file); + dispose = ep_remove_epi(ep, epi); + mutex_unlock(&ep->mtx); if (dispose && ep_refcount_dec_and_test(ep)) @@ -1590,21 +1606,21 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event, mutex_unlock(&tep->mtx); /* - * ep_remove_safe() calls in the later error paths can't lead to + * ep_remove() calls in the later error paths can't lead to * ep_free() as the ep file itself still holds an ep reference. */ ep_get(ep); /* now check if we've created too many backpaths */ if (unlikely(full_check && reverse_path_check())) { - ep_remove_safe(ep, epi); + ep_remove(ep, epi); return -EINVAL; } if (epi->event.events & EPOLLWAKEUP) { error = ep_create_wakeup_source(epi); if (error) { - ep_remove_safe(ep, epi); + ep_remove(ep, epi); return error; } } @@ -1628,7 +1644,7 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event, * high memory pressure. */ if (unlikely(!epq.epi)) { - ep_remove_safe(ep, epi); + ep_remove(ep, epi); return -ENOMEM; } @@ -2317,7 +2333,7 @@ int do_epoll_ctl(int epfd, int op, int fd, struct epoll_event *epds, * The eventpoll itself is still alive: the refcount * can't go to zero here. */ - ep_remove_safe(ep, epi); + ep_remove(ep, epi); error = 0; } else { error = -ENOENT; diff --git a/fs/exfat/dir.c b/fs/exfat/dir.c index 1c428f7f83f5..a3bdee0461e0 100644 --- a/fs/exfat/dir.c +++ b/fs/exfat/dir.c @@ -1100,12 +1100,12 @@ int exfat_find_dir_entry(struct super_block *sb, struct exfat_inode_info *ei, continue; } - brelse(bh); if (entry_type == TYPE_EXTEND) { unsigned short entry_uniname[16], unichar; if (step != DIRENT_STEP_NAME || name_len >= MAX_NAME_LENGTH) { + brelse(bh); step = DIRENT_STEP_FILE; continue; } @@ -1116,6 +1116,7 @@ int exfat_find_dir_entry(struct super_block *sb, struct exfat_inode_info *ei, uniname += EXFAT_FILE_NAME_LEN; len = exfat_extract_uni_name(ep, entry_uniname); + brelse(bh); name_len += len; unichar = *(uniname+len); @@ -1134,6 +1135,7 @@ int exfat_find_dir_entry(struct super_block *sb, struct exfat_inode_info *ei, continue; } + brelse(bh); if (entry_type & (TYPE_CRITICAL_SEC | TYPE_BENIGN_SEC)) { if (step == DIRENT_STEP_SECD) { diff --git a/fs/f2fs/acl.c b/fs/f2fs/acl.c index 8bffdeccdbc3..e45f68254075 100644 --- a/fs/f2fs/acl.c +++ b/fs/f2fs/acl.c @@ -46,6 +46,7 @@ static inline int f2fs_acl_count(size_t size) static struct posix_acl *f2fs_acl_from_disk(const char *value, size_t size) { int i, count; + int err = -EINVAL; struct posix_acl *acl; struct f2fs_acl_header *hdr = (struct f2fs_acl_header *)value; struct f2fs_acl_entry *entry = (struct f2fs_acl_entry *)(hdr + 1); @@ -69,8 +70,11 @@ static struct posix_acl *f2fs_acl_from_disk(const char *value, size_t size) for (i = 0; i < count; i++) { - if ((char *)entry > end) + if (unlikely((char *)entry + + sizeof(struct f2fs_acl_entry_short) > end)) { + err = -EFSCORRUPTED; goto fail; + } acl->a_entries[i].e_tag = le16_to_cpu(entry->e_tag); acl->a_entries[i].e_perm = le16_to_cpu(entry->e_perm); @@ -85,6 +89,11 @@ static struct posix_acl *f2fs_acl_from_disk(const char *value, size_t size) break; case ACL_USER: + if (unlikely((char *)entry + + sizeof(struct f2fs_acl_entry) > end)) { + err = -EFSCORRUPTED; + goto fail; + } acl->a_entries[i].e_uid = make_kuid(&init_user_ns, le32_to_cpu(entry->e_id)); @@ -92,6 +101,11 @@ static struct posix_acl *f2fs_acl_from_disk(const char *value, size_t size) sizeof(struct f2fs_acl_entry)); break; case ACL_GROUP: + if (unlikely((char *)entry + + sizeof(struct f2fs_acl_entry) > end)) { + err = -EFSCORRUPTED; + goto fail; + } acl->a_entries[i].e_gid = make_kgid(&init_user_ns, le32_to_cpu(entry->e_id)); @@ -107,7 +121,7 @@ static struct posix_acl *f2fs_acl_from_disk(const char *value, size_t size) return acl; fail: posix_acl_release(acl); - return ERR_PTR(-EINVAL); + return ERR_PTR(err); } static void *f2fs_acl_to_disk(struct f2fs_sb_info *sbi, diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index 234444794b55..faa350c0da93 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -3566,6 +3566,7 @@ static int prepare_atomic_write_begin(struct f2fs_sb_info *sbi, pgoff_t index = folio->index; int err = 0; block_t ori_blk_addr = NULL_ADDR; + bool cow_has_reserved_block = false; /* If pos is beyond the end of file, reserve a new block in COW inode */ if ((pos & PAGE_MASK) >= i_size_read(inode)) @@ -3575,9 +3576,11 @@ static int prepare_atomic_write_begin(struct f2fs_sb_info *sbi, err = __find_data_block(cow_inode, index, blk_addr); if (err) { return err; - } else if (*blk_addr != NULL_ADDR) { + } else if (__is_valid_data_blkaddr(*blk_addr)) { *use_cow = true; return 0; + } else if (*blk_addr == NEW_ADDR) { + cow_has_reserved_block = true; } if (is_inode_flag_set(inode, FI_ATOMIC_REPLACE)) @@ -3590,10 +3593,13 @@ static int prepare_atomic_write_begin(struct f2fs_sb_info *sbi, reserve_block: /* Finally, we should reserve a new block in COW inode for the update */ - err = __reserve_data_block(cow_inode, index, blk_addr, node_changed); - if (err) - return err; - inc_atomic_write_cnt(inode); + if (!cow_has_reserved_block) { + err = __reserve_data_block(cow_inode, index, blk_addr, + node_changed); + if (err) + return err; + inc_atomic_write_cnt(inode); + } if (ori_blk_addr != NULL_ADDR) *blk_addr = ori_blk_addr; diff --git a/fs/f2fs/extent_cache.c b/fs/f2fs/extent_cache.c index e8119d51fb4b..d0acbd6e2123 100644 --- a/fs/f2fs/extent_cache.c +++ b/fs/f2fs/extent_cache.c @@ -86,10 +86,9 @@ static bool __may_extent_tree(struct inode *inode, enum extent_type type) if (!__init_may_extent_tree(inode, type)) return false; - if (is_inode_flag_set(inode, FI_NO_EXTENT)) - return false; - if (type == EX_READ) { + if (is_inode_flag_set(inode, FI_NO_EXTENT)) + return false; if (is_inode_flag_set(inode, FI_COMPRESSED_FILE) && !f2fs_sb_has_readonly(F2FS_I_SB(inode))) return false; @@ -602,14 +601,10 @@ static unsigned int __destroy_extent_node(struct inode *inode, while (atomic_read(&et->node_cnt)) { write_lock(&et->lock); - if (!is_inode_flag_set(inode, FI_NO_EXTENT)) - set_inode_flag(inode, FI_NO_EXTENT); node_cnt += __free_extent_tree(sbi, et, nr_shrink); write_unlock(&et->lock); } - f2fs_bug_on(sbi, atomic_read(&et->node_cnt)); - return node_cnt; } @@ -639,12 +634,12 @@ static void __update_extent_tree_range(struct inode *inode, write_lock(&et->lock); - if (is_inode_flag_set(inode, FI_NO_EXTENT)) { - write_unlock(&et->lock); - return; - } - if (type == EX_READ) { + if (is_inode_flag_set(inode, FI_NO_EXTENT)) { + write_unlock(&et->lock); + return; + } + prev = et->largest; dei.len = 0; diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c index 3a56ab6a22c0..5f69bdca95c2 100644 --- a/fs/f2fs/file.c +++ b/fs/f2fs/file.c @@ -1838,8 +1838,15 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset, if (f2fs_is_pinned_file(inode)) { block_t sec_blks = CAP_BLKS_PER_SEC(sbi); - block_t sec_len = roundup(map.m_len, sec_blks); + block_t sec_len; + if (map.m_lblk % sec_blks) { + map.m_lblk = rounddown(map.m_lblk, sec_blks); + map.m_len = pg_end - map.m_lblk; + if (off_end) + map.m_len++; + } + sec_len = roundup(map.m_len, sec_blks); map.m_len = sec_blks; next_alloc: f2fs_down_write(&sbi->pin_sem); diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c index 9180693b933f..67c800802141 100644 --- a/fs/f2fs/inode.c +++ b/fs/f2fs/inode.c @@ -541,8 +541,13 @@ static int do_read_inode(struct inode *inode) static bool is_meta_ino(struct f2fs_sb_info *sbi, unsigned int ino) { - return ino == F2FS_NODE_INO(sbi) || ino == F2FS_META_INO(sbi) || - ino == F2FS_COMPRESS_INO(sbi); + if (ino == F2FS_NODE_INO(sbi) || ino == F2FS_META_INO(sbi)) + return true; +#ifdef CONFIG_F2FS_FS_COMPRESSION + if (test_opt(sbi, COMPRESS_CACHE) && ino == F2FS_COMPRESS_INO(sbi)) + return true; +#endif + return false; } struct inode *f2fs_iget(struct super_block *sb, unsigned long ino) diff --git a/fs/fhandle.c b/fs/fhandle.c index 38d803a28ab9..bbb5857c77b9 100644 --- a/fs/fhandle.c +++ b/fs/fhandle.c @@ -242,6 +242,19 @@ static int do_handle_to_path(struct file_handle *handle, struct path *path, return 0; } +static bool capable_wrt_mount(struct mount *mount) +{ + struct mnt_namespace *mnt_ns; + + /* + * For ->mnt_ns access. + * The following READ_ONCE() is semantically rcu_dereference(). + */ + guard(rcu)(); + mnt_ns = READ_ONCE(mount->mnt_ns); + return ns_capable(mnt_ns->user_ns, CAP_SYS_ADMIN); +} + /* * Allow relaxed permissions of file handles if the caller has the * ability to mount the filesystem or create a bind-mount of the @@ -273,8 +286,7 @@ static inline bool may_decode_fh(struct handle_to_path_ctx *ctx, if (ns_capable(root->mnt->mnt_sb->s_user_ns, CAP_SYS_ADMIN)) ctx->flags = HANDLE_CHECK_PERMS; else if (is_mounted(root->mnt) && - ns_capable(real_mount(root->mnt)->mnt_ns->user_ns, - CAP_SYS_ADMIN) && + capable_wrt_mount(real_mount(root->mnt)) && !has_locked_children(real_mount(root->mnt), root->dentry)) ctx->flags = HANDLE_CHECK_PERMS | HANDLE_CHECK_SUBTREE; else diff --git a/fs/file_table.c b/fs/file_table.c index 2a08bc93b0b9..f78bcff79f1a 100644 --- a/fs/file_table.c +++ b/fs/file_table.c @@ -46,21 +46,40 @@ static struct percpu_counter nr_files __cacheline_aligned_in_smp; struct backing_file { struct file file; struct path user_path; +#ifdef CONFIG_SECURITY + void *security; +#endif }; -static inline struct backing_file *backing_file(struct file *f) -{ - return container_of(f, struct backing_file, file); -} +#define backing_file(f) container_of(f, struct backing_file, file) -struct path *backing_file_user_path(struct file *f) +struct path *backing_file_user_path(const struct file *f) { return &backing_file(f)->user_path; } EXPORT_SYMBOL_GPL(backing_file_user_path); +void backing_file_set_user_path(struct file *f, const struct path *path) +{ + backing_file(f)->user_path = *path; +} +EXPORT_SYMBOL_GPL(backing_file_set_user_path); + +#ifdef CONFIG_SECURITY +void *backing_file_security(const struct file *f) +{ + return backing_file(f)->security; +} + +void backing_file_set_security(struct file *f, void *security) +{ + backing_file(f)->security = security; +} +#endif /* CONFIG_SECURITY */ + static inline void backing_file_free(struct backing_file *ff) { + security_backing_file_free(&ff->file); path_put(&ff->user_path); kfree(ff); } @@ -259,10 +278,12 @@ struct file *alloc_empty_file_noaccount(int flags, const struct cred *cred) return f; } -static int init_backing_file(struct backing_file *ff) +static int init_backing_file(struct backing_file *ff, + const struct file *user_file) { memset(&ff->user_path, 0, sizeof(ff->user_path)); - return 0; + backing_file_set_security(&ff->file, NULL); + return security_backing_file_alloc(&ff->file, user_file); } /* @@ -272,7 +293,8 @@ static int init_backing_file(struct backing_file *ff) * This is only for kernel internal use, and the allocate file must not be * installed into file tables or such. */ -struct file *alloc_empty_backing_file(int flags, const struct cred *cred) +struct file *alloc_empty_backing_file(int flags, const struct cred *cred, + const struct file *user_file) { struct backing_file *ff; int error; @@ -289,7 +311,7 @@ struct file *alloc_empty_backing_file(int flags, const struct cred *cred) /* The f_mode flags must be set before fput(). */ ff->file.f_mode |= FMODE_BACKING | FMODE_NOACCOUNT; - error = init_backing_file(ff); + error = init_backing_file(ff, user_file); if (unlikely(error)) { fput(&ff->file); return ERR_PTR(error); diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c index 8f4a2ff56cc3..f7e305c9ad4e 100644 --- a/fs/fuse/dev.c +++ b/fs/fuse/dev.c @@ -891,6 +891,10 @@ static int fuse_try_move_page(struct fuse_copy_state *cs, struct page **pagep) if (WARN_ON(folio_test_mlocked(oldfolio))) goto out_fallback_unlock; + err = lock_request(cs->req); + if (err) + goto out_fallback_unlock; + replace_page_cache_folio(oldfolio, newfolio); folio_get(newfolio); @@ -904,20 +908,7 @@ static int fuse_try_move_page(struct fuse_copy_state *cs, struct page **pagep) */ pipe_buf_release(cs->pipe, buf); - err = 0; - spin_lock(&cs->req->waitq.lock); - if (test_bit(FR_ABORTED, &cs->req->flags)) - err = -ENOENT; - else - *pagep = &newfolio->page; - spin_unlock(&cs->req->waitq.lock); - - if (err) { - folio_unlock(newfolio); - folio_put(newfolio); - goto out_put_old; - } - + *pagep = &newfolio->page; folio_unlock(oldfolio); /* Drop ref for ap->pages[] array */ folio_put(oldfolio); @@ -1769,6 +1760,10 @@ static int fuse_retrieve(struct fuse_mount *fm, struct inode *inode, page = find_get_page(mapping, index); if (!page) break; + if (!PageUptodate(page)) { + put_page(page); + break; + } this_num = min_t(unsigned, num, PAGE_SIZE - offset); ap->pages[ap->num_pages] = page; diff --git a/fs/fuse/file.c b/fs/fuse/file.c index 23afaadd5f3e..1f71ef3ead7a 100644 --- a/fs/fuse/file.c +++ b/fs/fuse/file.c @@ -373,8 +373,14 @@ void fuse_file_release(struct inode *inode, struct fuse_file *ff, * aio and closes the fd before the aio completes. Since aio takes its * own ref to the file, the IO completion has to drop the ref, which is * how the fuse server can end up closing its clients' files. + * + * Exception is virtio-fs, which is not affected by the above (server is + * on host, cannot close open files in guest). Virtio-fs needs sync + * release, because the num_waiting mechanism to wait for all requests + * before commencing with fs shutdown doesn't work if submounts are + * used. */ - fuse_file_put(ff, false); + fuse_file_put(ff, ff->fm->fc->auto_submounts); } void fuse_release_common(struct file *file, bool isdir) diff --git a/fs/fuse/passthrough.c b/fs/fuse/passthrough.c index 6bfd09dda9e3..140e150be0de 100644 --- a/fs/fuse/passthrough.c +++ b/fs/fuse/passthrough.c @@ -326,7 +326,7 @@ struct fuse_backing *fuse_passthrough_open(struct file *file, goto out; /* Allocate backing file per fuse file to store fuse path */ - backing_file = backing_file_open(&file->f_path, file->f_flags, + backing_file = backing_file_open(file, file->f_flags, &fb->file->f_path, fb->cred); err = PTR_ERR(backing_file); if (IS_ERR(backing_file)) { diff --git a/fs/gfs2/super.c b/fs/gfs2/super.c index e6f8be03190c..3d5d4b844621 100644 --- a/fs/gfs2/super.c +++ b/fs/gfs2/super.c @@ -652,6 +652,7 @@ static void gfs2_put_super(struct super_block *sb) gfs2_delete_debugfs_file(sdp); gfs2_sys_fs_del(sdp); + rcu_barrier(); free_sbd(sdp); } diff --git a/fs/internal.h b/fs/internal.h index 8c1b7acbbe8f..997acc721f61 100644 --- a/fs/internal.h +++ b/fs/internal.h @@ -99,7 +99,9 @@ extern void chroot_fs_refs(const struct path *, const struct path *); */ struct file *alloc_empty_file(int flags, const struct cred *cred); struct file *alloc_empty_file_noaccount(int flags, const struct cred *cred); -struct file *alloc_empty_backing_file(int flags, const struct cred *cred); +struct file *alloc_empty_backing_file(int flags, const struct cred *cred, + const struct file *user_file); +void backing_file_set_user_path(struct file *f, const struct path *path); static inline void file_put_write_access(struct file *file) { diff --git a/fs/mount.h b/fs/mount.h index 179f690a0c72..c3987f1d1749 100644 --- a/fs/mount.h +++ b/fs/mount.h @@ -58,7 +58,15 @@ struct mount { struct list_head mnt_slave_list;/* list of slave mounts */ struct list_head mnt_slave; /* slave list entry */ struct mount *mnt_master; /* slave is on master->mnt_slave_list */ - struct mnt_namespace *mnt_ns; /* containing namespace */ + /* + * Containing namespace (active or deactivating, non-refcounted). + * Normally protected by namespace_sem. + * Can also be accessed locklessly under RCU. RCU readers can't rely on + * the namespace still being active, but implicitly hold a passive + * reference (because an RCU delay happens between a namespace being + * deactivated and the corresponding passive refcount drop). + */ + struct mnt_namespace *mnt_ns; struct mountpoint *mnt_mp; /* where is it mounted */ union { struct hlist_node mnt_mp_list; /* list mounts with the same mountpoint */ diff --git a/fs/namespace.c b/fs/namespace.c index df6d6a2c6b89..94c06c842902 100644 --- a/fs/namespace.c +++ b/fs/namespace.c @@ -1132,7 +1132,7 @@ static void mnt_add_to_ns(struct mnt_namespace *ns, struct mount *mnt) struct rb_node *parent = NULL; WARN_ON(mnt_ns_attached(mnt)); - mnt->mnt_ns = ns; + WRITE_ONCE(mnt->mnt_ns, ns); while (*link) { parent = *link; if (mnt->mnt_id_unique < node_to_mount(parent)->mnt_id_unique) @@ -1493,7 +1493,7 @@ EXPORT_SYMBOL(mntget); void mnt_make_shortterm(struct vfsmount *mnt) { if (mnt) - real_mount(mnt)->mnt_ns = NULL; + WRITE_ONCE(real_mount(mnt)->mnt_ns, NULL); } /** @@ -1805,7 +1805,7 @@ static void umount_tree(struct mount *mnt, enum umount_tree_flags how) ns->nr_mounts--; __touch_mnt_namespace(ns); } - p->mnt_ns = NULL; + WRITE_ONCE(p->mnt_ns, NULL); if (how & UMOUNT_SYNC) p->mnt.mnt_flags |= MNT_SYNC_UMOUNT; diff --git a/fs/nfs/client.c b/fs/nfs/client.c index 5fe20936e145..8bdde6cc873f 100644 --- a/fs/nfs/client.c +++ b/fs/nfs/client.c @@ -1054,6 +1054,7 @@ struct nfs_server *nfs_alloc_server(void) server->io_stats = nfs_alloc_iostats(); if (!server->io_stats) { + ida_free(&s_sysfs_ids, server->s_sysfs_id); kfree(server); return NULL; } diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index 116499e0f5ce..5928d1cb1de2 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -2217,11 +2217,11 @@ pnfs_update_layout(struct inode *ino, dprintk("%s wait for layoutreturn\n", __func__); lseg = ERR_PTR(pnfs_prepare_to_retry_layoutget(lo)); if (!IS_ERR(lseg)) { - pnfs_put_layout_hdr(lo); dprintk("%s retrying\n", __func__); trace_pnfs_update_layout(ino, pos, count, iomode, lo, lseg, PNFS_UPDATE_LAYOUT_RETRY); + pnfs_put_layout_hdr(lo); goto lookup_again; } trace_pnfs_update_layout(ino, pos, count, iomode, lo, lseg, diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index dd688d17b5b9..39507b2e259b 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -1055,14 +1055,14 @@ nfs4_decode_mp_ds_addr(struct net *net, struct xdr_stream *xdr, gfp_t gfp_flags) /* r_netid */ nlen = xdr_stream_decode_string_dup(xdr, &netid, XDR_MAX_NETOBJ, gfp_flags); - if (unlikely(nlen < 0)) + if (unlikely(nlen <= 0)) goto out_err; /* r_addr: ip/ip6addr with port in dec octets - see RFC 5665 */ /* port is ".ABC.DEF", 8 chars max */ rlen = xdr_stream_decode_string_dup(xdr, &buf, INET6_ADDRSTRLEN + IPV6_SCOPE_ID_LEN + 8, gfp_flags); - if (unlikely(rlen < 0)) + if (unlikely(rlen <= 0)) goto out_free_netid; /* replace port '.' with '-' */ diff --git a/fs/nfsd/nfs2acl.c b/fs/nfsd/nfs2acl.c index 0ac538c76180..76305b86c1a9 100644 --- a/fs/nfsd/nfs2acl.c +++ b/fs/nfsd/nfs2acl.c @@ -131,10 +131,7 @@ static __be32 nfsacld_proc_setacl(struct svc_rqst *rqstp) resp->status = fh_getattr(fh, &resp->stat); out: - /* argp->acl_{access,default} may have been allocated in - nfssvc_decode_setaclargs. */ - posix_acl_release(argp->acl_access); - posix_acl_release(argp->acl_default); + /* argp->acl_{access,default} are released in nfsaclsvc_release_setacl. */ return rpc_success; out_drop_lock: @@ -310,6 +307,16 @@ static void nfsaclsvc_release_access(struct svc_rqst *rqstp) fh_put(&resp->fh); } +static void nfsaclsvc_release_setacl(struct svc_rqst *rqstp) +{ + struct nfsd3_setaclargs *argp = rqstp->rq_argp; + struct nfsd_attrstat *resp = rqstp->rq_resp; + + fh_put(&resp->fh); + posix_acl_release(argp->acl_access); + posix_acl_release(argp->acl_default); +} + #define ST 1 /* status*/ #define AT 21 /* attributes */ #define pAT (1+AT) /* post attributes - conditional */ @@ -343,7 +350,7 @@ static const struct svc_procedure nfsd_acl_procedures2[5] = { .pc_func = nfsacld_proc_setacl, .pc_decode = nfsaclsvc_decode_setaclargs, .pc_encode = nfssvc_encode_attrstatres, - .pc_release = nfssvc_release_attrstat, + .pc_release = nfsaclsvc_release_setacl, .pc_argsize = sizeof(struct nfsd3_setaclargs), .pc_argzero = sizeof(struct nfsd3_setaclargs), .pc_ressize = sizeof(struct nfsd_attrstat), diff --git a/fs/nfsd/nfs3acl.c b/fs/nfsd/nfs3acl.c index 7b5433bd3019..e87731380be8 100644 --- a/fs/nfsd/nfs3acl.c +++ b/fs/nfsd/nfs3acl.c @@ -118,10 +118,7 @@ static __be32 nfsd3_proc_setacl(struct svc_rqst *rqstp) out_errno: resp->status = nfserrno(error); out: - /* argp->acl_{access,default} may have been allocated in - nfs3svc_decode_setaclargs. */ - posix_acl_release(argp->acl_access); - posix_acl_release(argp->acl_default); + /* argp->acl_{access,default} are released in nfs3svc_release_setacl. */ return rpc_success; } @@ -223,6 +220,16 @@ static void nfs3svc_release_getacl(struct svc_rqst *rqstp) posix_acl_release(resp->acl_default); } +static void nfs3svc_release_setacl(struct svc_rqst *rqstp) +{ + struct nfsd3_setaclargs *argp = rqstp->rq_argp; + struct nfsd3_attrstat *resp = rqstp->rq_resp; + + fh_put(&resp->fh); + posix_acl_release(argp->acl_access); + posix_acl_release(argp->acl_default); +} + #define ST 1 /* status*/ #define AT 21 /* attributes */ #define pAT (1+AT) /* post attributes - conditional */ @@ -256,7 +263,7 @@ static const struct svc_procedure nfsd_acl_procedures3[3] = { .pc_func = nfsd3_proc_setacl, .pc_decode = nfs3svc_decode_setaclargs, .pc_encode = nfs3svc_encode_setaclres, - .pc_release = nfs3svc_release_fhandle, + .pc_release = nfs3svc_release_setacl, .pc_argsize = sizeof(struct nfsd3_setaclargs), .pc_argzero = sizeof(struct nfsd3_setaclargs), .pc_ressize = sizeof(struct nfsd3_attrstat), diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c index 1c8fcb04b3cd..075715998648 100644 --- a/fs/nfsd/nfs4recover.c +++ b/fs/nfsd/nfs4recover.c @@ -818,7 +818,8 @@ __cld_pipe_inprogress_downcall(const struct cld_msg_v2 __user *cmsg, if (IS_ERR(name.data)) return PTR_ERR(name.data); name.len = namelen; - get_user(princhashlen, &ci->cc_princhash.cp_len); + if (get_user(princhashlen, &ci->cc_princhash.cp_len)) + return -EFAULT; if (princhashlen > 0) { princhash.data = memdup_user( &ci->cc_princhash.cp_data, diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 2d9174729782..1ce66acc9b6b 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -4971,6 +4971,7 @@ find_or_alloc_open_stateowner(unsigned int strhashval, struct nfsd4_open *open, /* Replace unconfirmed owners without checking for replay. */ release_openowner(oo); oo = NULL; + goto retry; } if (oo) { if (new) diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c index 8471371c6888..9c8767cab51d 100644 --- a/fs/nfsd/nfs4xdr.c +++ b/fs/nfsd/nfs4xdr.c @@ -1849,10 +1849,11 @@ static __be32 nfsd4_decode_secinfo_no_name(struct nfsd4_compoundargs *argp, union nfsd4_op_u *u) { struct nfsd4_secinfo_no_name *sin = &u->secinfo_no_name; + + sin->sin_exp = NULL; if (xdr_stream_decode_u32(argp->xdr, &sin->sin_style) < 0) return nfserr_bad_xdr; - sin->sin_exp = NULL; return nfs_ok; } diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index 08c8babfdd75..67d09fcee669 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -1203,8 +1203,10 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, nfsd_stats_io_write_add(nn, exp, *cnt); fsnotify_modify(file); host_err = filemap_check_wb_err(file->f_mapping, since); - if (host_err < 0) + if (host_err < 0) { + commit_reset_write_verifier(nn, rqstp, host_err); goto out_nfserr; + } if (stable && fhp->fh_use_wgather) { host_err = wait_for_concurrent_writes(file); @@ -1370,6 +1372,8 @@ nfsd_commit(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, nfsd_copy_write_verifier(verf, nn); err2 = filemap_check_wb_err(nf->nf_file->f_mapping, since); + if (err2 < 0) + commit_reset_write_verifier(nn, rqstp, err2); err = nfserrno(err2); break; case -EINVAL: diff --git a/fs/ntfs3/xattr.c b/fs/ntfs3/xattr.c index 6861c09d66d7..287534823346 100644 --- a/fs/ntfs3/xattr.c +++ b/fs/ntfs3/xattr.c @@ -829,6 +829,12 @@ static int ntfs_getxattr(const struct xattr_handler *handler, struct dentry *de, return err; } +static bool ntfs_is_reserved_lxattr(const char *name) +{ + return !strcmp(name, "$LXUID") || !strcmp(name, "$LXGID") || + !strcmp(name, "$LXMOD") || !strcmp(name, "$LXDEV"); +} + /* * ntfs_setxattr - inode_operations::setxattr */ @@ -933,6 +939,12 @@ static noinline int ntfs_setxattr(const struct xattr_handler *handler, goto out; } + /* Do not allow non privileged users to change $LXUID/$LXGID... */ + if (ntfs_is_reserved_lxattr(name) && !capable(CAP_SYS_ADMIN)) { + err = -EPERM; + goto out; + } + /* Deal with NTFS extended attribute. */ err = ntfs_set_ea(inode, name, strlen(name), value, size, flags, 0, NULL); diff --git a/fs/ocfs2/suballoc.c b/fs/ocfs2/suballoc.c index e93fc842bb20..a752ea59a167 100644 --- a/fs/ocfs2/suballoc.c +++ b/fs/ocfs2/suballoc.c @@ -205,8 +205,16 @@ static int ocfs2_validate_gd_parent(struct super_block *sb, int resize) { unsigned int max_bits; + unsigned int max_bitmap_bits; + unsigned int max_bitmap_size; + int suballocator; struct ocfs2_group_desc *gd = (struct ocfs2_group_desc *)bh->b_data; + suballocator = le64_to_cpu(di->i_blkno) != OCFS2_SB(sb)->bitmap_blkno; + max_bitmap_size = ocfs2_group_bitmap_size(sb, suballocator, + OCFS2_SB(sb)->s_feature_incompat); + max_bitmap_bits = max_bitmap_size * 8; + if (di->i_blkno != gd->bg_parent_dinode) { do_error("Group descriptor #%llu has bad parent pointer (%llu, expected %llu)\n", (unsigned long long)bh->b_blocknr, @@ -214,6 +222,20 @@ static int ocfs2_validate_gd_parent(struct super_block *sb, (unsigned long long)le64_to_cpu(di->i_blkno)); } + if (le16_to_cpu(gd->bg_size) > max_bitmap_size) { + do_error("Group descriptor #%llu has bitmap size %u but physical max of %u\n", + (unsigned long long)bh->b_blocknr, + le16_to_cpu(gd->bg_size), + max_bitmap_size); + } + + if (le16_to_cpu(gd->bg_bits) > max_bitmap_bits) { + do_error("Group descriptor #%llu has bit count %u but physical max of %u\n", + (unsigned long long)bh->b_blocknr, + le16_to_cpu(gd->bg_bits), + max_bitmap_bits); + } + max_bits = le16_to_cpu(di->id2.i_chain.cl_cpg) * le16_to_cpu(di->id2.i_chain.cl_bpc); if (le16_to_cpu(gd->bg_bits) > max_bits) { do_error("Group descriptor #%llu has bit count of %u\n", diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c index ab65e98a1def..1c8009bf194b 100644 --- a/fs/overlayfs/dir.c +++ b/fs/overlayfs/dir.c @@ -1320,7 +1320,7 @@ static int ovl_create_tmpfile(struct file *file, struct dentry *dentry, goto out_revert_creds; ovl_path_upper(dentry->d_parent, &realparentpath); - realfile = backing_tmpfile_open(&file->f_path, flags, &realparentpath, + realfile = backing_tmpfile_open(file, flags, &realparentpath, mode, current_cred()); err = PTR_ERR_OR_ZERO(realfile); pr_debug("tmpfile/open(%pd2, 0%o) = %i\n", realparentpath.dentry, mode, err); diff --git a/fs/overlayfs/file.c b/fs/overlayfs/file.c index 94095058da34..3765e1defa19 100644 --- a/fs/overlayfs/file.c +++ b/fs/overlayfs/file.c @@ -47,8 +47,7 @@ static struct file *ovl_open_realfile(const struct file *file, } else { if (!inode_owner_or_capable(real_idmap, realinode)) flags &= ~O_NOATIME; - - realfile = backing_file_open(file_user_path((struct file *) file), + realfile = backing_file_open(file, flags, realpath, current_cred()); } revert_creds(old_cred); diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c index eb8ab4c8c7f4..162bcd46100e 100644 --- a/fs/smb/server/smb2pdu.c +++ b/fs/smb/server/smb2pdu.c @@ -612,6 +612,11 @@ int smb2_check_user_session(struct ksmbd_work *work) sess_id, work->sess->id); return -EINVAL; } + if (work->sess->state != SMB2_SESSION_VALID) { + pr_err("compound request on a non-valid session (state %d)\n", + work->sess->state); + return -EINVAL; + } return 1; } diff --git a/fs/smb/server/smbacl.c b/fs/smb/server/smbacl.c index e3c512675c63..f5864dd1dd5f 100644 --- a/fs/smb/server/smbacl.c +++ b/fs/smb/server/smbacl.c @@ -1480,7 +1480,9 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path, break; aces_size -= ace_size; - if (ace->sid.num_subauth > SID_MAX_SUB_AUTHORITIES) + if (ace->sid.num_subauth > SID_MAX_SUB_AUTHORITIES || + ace_size < offsetof(struct smb_ace, sid) + CIFS_SID_BASE_SIZE + + sizeof(__le32) * ace->sid.num_subauth) break; if (!compare_sids(&sid, &ace->sid) || diff --git a/fs/xfs/libxfs/xfs_ag_resv.c b/fs/xfs/libxfs/xfs_ag_resv.c index 216423df939e..4886fad98155 100644 --- a/fs/xfs/libxfs/xfs_ag_resv.c +++ b/fs/xfs/libxfs/xfs_ag_resv.c @@ -91,9 +91,9 @@ xfs_ag_resv_critical( trace_xfs_ag_resv_critical(pag, type, avail); /* Critically low if less than 10% or max btree height remains. */ - return XFS_TEST_ERROR(avail < orig / 10 || - avail < pag->pag_mount->m_agbtree_maxlevels, - pag->pag_mount, XFS_ERRTAG_AG_RESV_CRITICAL); + return avail < orig / 10 || + avail < pag->pag_mount->m_agbtree_maxlevels || + XFS_TEST_ERROR(pag->pag_mount, XFS_ERRTAG_AG_RESV_CRITICAL); } /* @@ -201,7 +201,7 @@ __xfs_ag_resv_init( return -EINVAL; } - if (XFS_TEST_ERROR(false, mp, XFS_ERRTAG_AG_RESV_FAIL)) + if (XFS_TEST_ERROR(mp, XFS_ERRTAG_AG_RESV_FAIL)) error = -ENOSPC; else error = xfs_dec_fdblocks(mp, hidden_space, true); diff --git a/fs/xfs/libxfs/xfs_alloc.c b/fs/xfs/libxfs/xfs_alloc.c index 22bdbb3e9980..3633f4d28c3b 100644 --- a/fs/xfs/libxfs/xfs_alloc.c +++ b/fs/xfs/libxfs/xfs_alloc.c @@ -3312,7 +3312,7 @@ xfs_agf_read_verify( xfs_verifier_error(bp, -EFSBADCRC, __this_address); else { fa = xfs_agf_verify(bp); - if (XFS_TEST_ERROR(fa, mp, XFS_ERRTAG_ALLOC_READ_AGF)) + if (fa || XFS_TEST_ERROR(mp, XFS_ERRTAG_ALLOC_READ_AGF)) xfs_verifier_error(bp, -EFSCORRUPTED, fa); } } @@ -3986,8 +3986,7 @@ __xfs_free_extent( ASSERT(len != 0); ASSERT(type != XFS_AG_RESV_AGFL); - if (XFS_TEST_ERROR(false, mp, - XFS_ERRTAG_FREE_EXTENT)) + if (XFS_TEST_ERROR(mp, XFS_ERRTAG_FREE_EXTENT)) return -EIO; error = xfs_free_extent_fix_freelist(tp, pag, &agbp); diff --git a/fs/xfs/libxfs/xfs_attr_leaf.c b/fs/xfs/libxfs/xfs_attr_leaf.c index bfcac9036c11..afdd3fc0a8be 100644 --- a/fs/xfs/libxfs/xfs_attr_leaf.c +++ b/fs/xfs/libxfs/xfs_attr_leaf.c @@ -1225,7 +1225,7 @@ xfs_attr3_leaf_to_node( trace_xfs_attr_leaf_to_node(args); - if (XFS_TEST_ERROR(false, mp, XFS_ERRTAG_ATTR_LEAF_TO_NODE)) { + if (XFS_TEST_ERROR(mp, XFS_ERRTAG_ATTR_LEAF_TO_NODE)) { error = -EIO; goto out; } diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c index 36dd08d13293..b2b5a36ead97 100644 --- a/fs/xfs/libxfs/xfs_bmap.c +++ b/fs/xfs/libxfs/xfs_bmap.c @@ -3766,8 +3766,7 @@ xfs_bmap_btalloc( /* Trim the allocation back to the maximum an AG can fit. */ args.maxlen = min(ap->length, mp->m_ag_max_usable); - if (unlikely(XFS_TEST_ERROR(false, mp, - XFS_ERRTAG_BMAP_ALLOC_MINLEN_EXTENT))) + if (unlikely(XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAP_ALLOC_MINLEN_EXTENT))) error = xfs_bmap_exact_minlen_extent_alloc(ap, &args); else if ((ap->datatype & XFS_ALLOC_USERDATA) && xfs_inode_is_filestream(ap->ip)) @@ -3953,7 +3952,7 @@ xfs_bmapi_read( } if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) || - XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) { + XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) { xfs_bmap_mark_sick(ip, whichfork); return -EFSCORRUPTED; } @@ -4442,7 +4441,7 @@ xfs_bmapi_write( (XFS_BMAPI_PREALLOC | XFS_BMAPI_ZERO)); if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) || - XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) { + XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) { xfs_bmap_mark_sick(ip, whichfork); return -EFSCORRUPTED; } @@ -4785,7 +4784,7 @@ xfs_bmapi_remap( (XFS_BMAPI_ATTRFORK | XFS_BMAPI_PREALLOC)); if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) || - XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) { + XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) { xfs_bmap_mark_sick(ip, whichfork); return -EFSCORRUPTED; } @@ -5873,7 +5872,7 @@ xfs_bmap_collapse_extents( int logflags = 0; if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) || - XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) { + XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) { xfs_bmap_mark_sick(ip, whichfork); return -EFSCORRUPTED; } @@ -5988,7 +5987,7 @@ xfs_bmap_insert_extents( int logflags = 0; if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) || - XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) { + XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) { xfs_bmap_mark_sick(ip, whichfork); return -EFSCORRUPTED; } @@ -6092,7 +6091,7 @@ xfs_bmap_split_extent( int i = 0; if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) || - XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) { + XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) { xfs_bmap_mark_sick(ip, whichfork); return -EFSCORRUPTED; } @@ -6257,7 +6256,7 @@ xfs_bmap_finish_one( trace_xfs_bmap_deferred(bi); - if (XFS_TEST_ERROR(false, tp->t_mountp, XFS_ERRTAG_BMAP_FINISH_ONE)) + if (XFS_TEST_ERROR(tp->t_mountp, XFS_ERRTAG_BMAP_FINISH_ONE)) return -EIO; switch (bi->bi_type) { diff --git a/fs/xfs/libxfs/xfs_btree.c b/fs/xfs/libxfs/xfs_btree.c index 134d87b3489a..f0ddaa136a0b 100644 --- a/fs/xfs/libxfs/xfs_btree.c +++ b/fs/xfs/libxfs/xfs_btree.c @@ -300,7 +300,7 @@ xfs_btree_check_block( fa = __xfs_btree_check_block(cur, block, level, bp); if (XFS_IS_CORRUPT(mp, fa != NULL) || - XFS_TEST_ERROR(false, mp, xfs_btree_block_errtag(cur))) { + XFS_TEST_ERROR(mp, xfs_btree_block_errtag(cur))) { if (bp) trace_xfs_btree_corrupt(bp, _RET_IP_); xfs_btree_mark_sick(cur); diff --git a/fs/xfs/libxfs/xfs_da_btree.c b/fs/xfs/libxfs/xfs_da_btree.c index 723a0643b838..90f7fc219fcc 100644 --- a/fs/xfs/libxfs/xfs_da_btree.c +++ b/fs/xfs/libxfs/xfs_da_btree.c @@ -565,7 +565,7 @@ xfs_da3_split( trace_xfs_da_split(state->args); - if (XFS_TEST_ERROR(false, state->mp, XFS_ERRTAG_DA_LEAF_SPLIT)) + if (XFS_TEST_ERROR(state->mp, XFS_ERRTAG_DA_LEAF_SPLIT)) return -EIO; /* diff --git a/fs/xfs/libxfs/xfs_dir2.c b/fs/xfs/libxfs/xfs_dir2.c index 202468223bf9..945285060e65 100644 --- a/fs/xfs/libxfs/xfs_dir2.c +++ b/fs/xfs/libxfs/xfs_dir2.c @@ -223,7 +223,7 @@ xfs_dir_ino_validate( bool ino_ok = xfs_verify_dir_ino(mp, ino); if (XFS_IS_CORRUPT(mp, !ino_ok) || - XFS_TEST_ERROR(false, mp, XFS_ERRTAG_DIR_INO_VALIDATE)) { + XFS_TEST_ERROR(mp, XFS_ERRTAG_DIR_INO_VALIDATE)) { xfs_warn(mp, "Invalid inode number 0x%Lx", (unsigned long long) ino); return -EFSCORRUPTED; diff --git a/fs/xfs/libxfs/xfs_exchmaps.c b/fs/xfs/libxfs/xfs_exchmaps.c index 2021396651de..f9df18a84f65 100644 --- a/fs/xfs/libxfs/xfs_exchmaps.c +++ b/fs/xfs/libxfs/xfs_exchmaps.c @@ -616,7 +616,7 @@ xfs_exchmaps_finish_one( return error; } - if (XFS_TEST_ERROR(false, tp->t_mountp, XFS_ERRTAG_EXCHMAPS_FINISH_ONE)) + if (XFS_TEST_ERROR(tp->t_mountp, XFS_ERRTAG_EXCHMAPS_FINISH_ONE)) return -EIO; /* If we still have work to do, ask for a new transaction. */ @@ -880,7 +880,7 @@ xmi_ensure_delta_nextents( &new_nextents)) return -EFBIG; - if (XFS_TEST_ERROR(false, mp, XFS_ERRTAG_REDUCE_MAX_IEXTENTS) && + if (XFS_TEST_ERROR(mp, XFS_ERRTAG_REDUCE_MAX_IEXTENTS) && new_nextents > 10) return -EFBIG; diff --git a/fs/xfs/libxfs/xfs_ialloc.c b/fs/xfs/libxfs/xfs_ialloc.c index 8223464e23e7..0fdb2a30e50d 100644 --- a/fs/xfs/libxfs/xfs_ialloc.c +++ b/fs/xfs/libxfs/xfs_ialloc.c @@ -2690,7 +2690,7 @@ xfs_agi_read_verify( xfs_verifier_error(bp, -EFSBADCRC, __this_address); else { fa = xfs_agi_verify(bp); - if (XFS_TEST_ERROR(fa, mp, XFS_ERRTAG_IALLOC_READ_AGI)) + if (fa || XFS_TEST_ERROR(mp, XFS_ERRTAG_IALLOC_READ_AGI)) xfs_verifier_error(bp, -EFSCORRUPTED, fa); } } diff --git a/fs/xfs/libxfs/xfs_inode_buf.c b/fs/xfs/libxfs/xfs_inode_buf.c index 79babeac9d75..63f4abd87076 100644 --- a/fs/xfs/libxfs/xfs_inode_buf.c +++ b/fs/xfs/libxfs/xfs_inode_buf.c @@ -60,8 +60,8 @@ xfs_inode_buf_verify( di_ok = xfs_verify_magic16(bp, dip->di_magic) && xfs_dinode_good_version(mp, dip->di_version) && xfs_verify_agino_or_null(bp->b_pag, unlinked_ino); - if (unlikely(XFS_TEST_ERROR(!di_ok, mp, - XFS_ERRTAG_ITOBP_INOTOBP))) { + if (unlikely(!di_ok || + XFS_TEST_ERROR(mp, XFS_ERRTAG_ITOBP_INOTOBP))) { if (readahead) { bp->b_flags &= ~XBF_DONE; xfs_buf_ioerror(bp, -EIO); diff --git a/fs/xfs/libxfs/xfs_inode_fork.c b/fs/xfs/libxfs/xfs_inode_fork.c index 1158ca48626b..78af39caa0f1 100644 --- a/fs/xfs/libxfs/xfs_inode_fork.c +++ b/fs/xfs/libxfs/xfs_inode_fork.c @@ -795,8 +795,7 @@ xfs_iext_count_extend( if (nr_exts < ifp->if_nextents) return -EFBIG; - if (XFS_TEST_ERROR(false, mp, XFS_ERRTAG_REDUCE_MAX_IEXTENTS) && - nr_exts > 10) + if (XFS_TEST_ERROR(mp, XFS_ERRTAG_REDUCE_MAX_IEXTENTS) && nr_exts > 10) return -EFBIG; if (nr_exts > xfs_iext_max_nextents(has_large, whichfork)) { diff --git a/fs/xfs/libxfs/xfs_refcount.c b/fs/xfs/libxfs/xfs_refcount.c index 198b84117df1..e26a2472c25d 100644 --- a/fs/xfs/libxfs/xfs_refcount.c +++ b/fs/xfs/libxfs/xfs_refcount.c @@ -1073,8 +1073,7 @@ xfs_refcount_still_have_space( * refcount continue update "error" has been injected. */ if (cur->bc_refc.nr_ops > 2 && - XFS_TEST_ERROR(false, cur->bc_mp, - XFS_ERRTAG_REFCOUNT_CONTINUE_UPDATE)) + XFS_TEST_ERROR(cur->bc_mp, XFS_ERRTAG_REFCOUNT_CONTINUE_UPDATE)) return false; if (cur->bc_refc.nr_ops == 0) @@ -1353,7 +1352,7 @@ xfs_refcount_finish_one( trace_xfs_refcount_deferred(mp, ri); - if (XFS_TEST_ERROR(false, mp, XFS_ERRTAG_REFCOUNT_FINISH_ONE)) + if (XFS_TEST_ERROR(mp, XFS_ERRTAG_REFCOUNT_FINISH_ONE)) return -EIO; /* diff --git a/fs/xfs/libxfs/xfs_rmap.c b/fs/xfs/libxfs/xfs_rmap.c index 6ef4687b3aba..0fbec9134cb5 100644 --- a/fs/xfs/libxfs/xfs_rmap.c +++ b/fs/xfs/libxfs/xfs_rmap.c @@ -2579,7 +2579,7 @@ xfs_rmap_finish_one( trace_xfs_rmap_deferred(mp, ri); - if (XFS_TEST_ERROR(false, mp, XFS_ERRTAG_RMAP_FINISH_ONE)) + if (XFS_TEST_ERROR(mp, XFS_ERRTAG_RMAP_FINISH_ONE)) return -EIO; /* diff --git a/fs/xfs/scrub/cow_repair.c b/fs/xfs/scrub/cow_repair.c index 4de3f0f40f48..337bdd524b01 100644 --- a/fs/xfs/scrub/cow_repair.c +++ b/fs/xfs/scrub/cow_repair.c @@ -297,18 +297,15 @@ xrep_cow_find_bad( * on the debugging knob, replace everything in the CoW fork. */ if ((sc->sm->sm_flags & XFS_SCRUB_IFLAG_FORCE_REBUILD) || - XFS_TEST_ERROR(false, sc->mp, XFS_ERRTAG_FORCE_SCRUB_REPAIR)) { + XFS_TEST_ERROR(sc->mp, XFS_ERRTAG_FORCE_SCRUB_REPAIR)) error = xrep_cow_mark_file_range(xc, xc->irec.br_startblock, xc->irec.br_blockcount); - if (error) - return error; - } out_sa: xchk_ag_free(sc, &sc->sa); out_pag: xfs_perag_put(pag); - return 0; + return error; } /* diff --git a/fs/xfs/scrub/repair.c b/fs/xfs/scrub/repair.c index c19abc687072..4d763e7b6446 100644 --- a/fs/xfs/scrub/repair.c +++ b/fs/xfs/scrub/repair.c @@ -990,7 +990,7 @@ xrep_will_attempt( return true; /* Let debug users force us into the repair routines. */ - if (XFS_TEST_ERROR(false, sc->mp, XFS_ERRTAG_FORCE_SCRUB_REPAIR)) + if (XFS_TEST_ERROR(sc->mp, XFS_ERRTAG_FORCE_SCRUB_REPAIR)) return true; /* Metadata is corrupt or failed cross-referencing. */ diff --git a/fs/xfs/xfs_attr_item.c b/fs/xfs/xfs_attr_item.c index 6a2ee93d5fff..b81eafc9543b 100644 --- a/fs/xfs/xfs_attr_item.c +++ b/fs/xfs/xfs_attr_item.c @@ -490,7 +490,7 @@ xfs_attr_finish_item( /* Reset trans after EAGAIN cycle since the transaction is new */ args->trans = tp; - if (XFS_TEST_ERROR(false, args->dp->i_mount, XFS_ERRTAG_LARP)) { + if (XFS_TEST_ERROR(args->dp->i_mount, XFS_ERRTAG_LARP)) { error = -EIO; goto out; } diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c index afcdfe317b7e..1f6b2888bd41 100644 --- a/fs/xfs/xfs_buf.c +++ b/fs/xfs/xfs_buf.c @@ -1498,7 +1498,7 @@ xfs_buf_bio_end_io( if (!bio->bi_status && (bp->b_flags & XBF_WRITE) && (bp->b_flags & XBF_ASYNC) && - XFS_TEST_ERROR(false, bp->b_mount, XFS_ERRTAG_BUF_IOERROR)) + XFS_TEST_ERROR(bp->b_mount, XFS_ERRTAG_BUF_IOERROR)) bio->bi_status = BLK_STS_IOERR; /* @@ -2451,7 +2451,7 @@ void xfs_buf_set_ref(struct xfs_buf *bp, int lru_ref) * This allows userspace to disrupt buffer caching for debug/testing * purposes. */ - if (XFS_TEST_ERROR(false, bp->b_mount, XFS_ERRTAG_BUF_LRU_REF)) + if (XFS_TEST_ERROR(bp->b_mount, XFS_ERRTAG_BUF_LRU_REF)) lru_ref = 0; atomic_set(&bp->b_lru_ref, lru_ref); diff --git a/fs/xfs/xfs_error.c b/fs/xfs/xfs_error.c index 78cdc5064a8c..3b41cc591e92 100644 --- a/fs/xfs/xfs_error.c +++ b/fs/xfs/xfs_error.c @@ -292,7 +292,6 @@ xfs_errortag_enabled( bool xfs_errortag_test( struct xfs_mount *mp, - const char *expression, const char *file, int line, unsigned int error_tag) @@ -318,8 +317,8 @@ xfs_errortag_test( return false; xfs_warn_ratelimited(mp, -"Injecting error (%s) at file %s, line %d, on filesystem \"%s\"", - expression, file, line, mp->m_super->s_id); +"Injecting error at file %s, line %d, on filesystem \"%s\"", + file, line, mp->m_super->s_id); return true; } diff --git a/fs/xfs/xfs_error.h b/fs/xfs/xfs_error.h index 0b9c5ba8a598..ff7792988b8a 100644 --- a/fs/xfs/xfs_error.h +++ b/fs/xfs/xfs_error.h @@ -41,10 +41,10 @@ extern void xfs_inode_verifier_error(struct xfs_inode *ip, int error, #ifdef DEBUG extern int xfs_errortag_init(struct xfs_mount *mp); extern void xfs_errortag_del(struct xfs_mount *mp); -extern bool xfs_errortag_test(struct xfs_mount *mp, const char *expression, - const char *file, int line, unsigned int error_tag); -#define XFS_TEST_ERROR(expr, mp, tag) \ - ((expr) || xfs_errortag_test((mp), #expr, __FILE__, __LINE__, (tag))) +bool xfs_errortag_test(struct xfs_mount *mp, const char *file, int line, + unsigned int error_tag); +#define XFS_TEST_ERROR(mp, tag) \ + xfs_errortag_test((mp), __FILE__, __LINE__, (tag)) bool xfs_errortag_enabled(struct xfs_mount *mp, unsigned int tag); #define XFS_ERRORTAG_DELAY(mp, tag) \ do { \ @@ -66,7 +66,7 @@ extern int xfs_errortag_clearall(struct xfs_mount *mp); #else #define xfs_errortag_init(mp) (0) #define xfs_errortag_del(mp) -#define XFS_TEST_ERROR(expr, mp, tag) (expr) +#define XFS_TEST_ERROR(mp, tag) (false) #define XFS_ERRORTAG_DELAY(mp, tag) ((void)0) #define xfs_errortag_set(mp, tag, val) (ENOSYS) #define xfs_errortag_add(mp, tag) (ENOSYS) diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index ed09b4a3084e..266fcc789098 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -2367,37 +2367,35 @@ xfs_iflush( * error handling as the caller will shutdown and fail the buffer. */ error = -EFSCORRUPTED; - if (XFS_TEST_ERROR(dip->di_magic != cpu_to_be16(XFS_DINODE_MAGIC), - mp, XFS_ERRTAG_IFLUSH_1)) { + if (dip->di_magic != cpu_to_be16(XFS_DINODE_MAGIC) || + XFS_TEST_ERROR(mp, XFS_ERRTAG_IFLUSH_1)) { xfs_alert_tag(mp, XFS_PTAG_IFLUSH, "%s: Bad inode %llu magic number 0x%x, ptr "PTR_FMT, __func__, ip->i_ino, be16_to_cpu(dip->di_magic), dip); goto flush_out; } if (S_ISREG(VFS_I(ip)->i_mode)) { - if (XFS_TEST_ERROR( - ip->i_df.if_format != XFS_DINODE_FMT_EXTENTS && - ip->i_df.if_format != XFS_DINODE_FMT_BTREE, - mp, XFS_ERRTAG_IFLUSH_3)) { + if ((ip->i_df.if_format != XFS_DINODE_FMT_EXTENTS && + ip->i_df.if_format != XFS_DINODE_FMT_BTREE) || + XFS_TEST_ERROR(mp, XFS_ERRTAG_IFLUSH_3)) { xfs_alert_tag(mp, XFS_PTAG_IFLUSH, "%s: Bad regular inode %llu, ptr "PTR_FMT, __func__, ip->i_ino, ip); goto flush_out; } } else if (S_ISDIR(VFS_I(ip)->i_mode)) { - if (XFS_TEST_ERROR( - ip->i_df.if_format != XFS_DINODE_FMT_EXTENTS && - ip->i_df.if_format != XFS_DINODE_FMT_BTREE && - ip->i_df.if_format != XFS_DINODE_FMT_LOCAL, - mp, XFS_ERRTAG_IFLUSH_4)) { + if ((ip->i_df.if_format != XFS_DINODE_FMT_EXTENTS && + ip->i_df.if_format != XFS_DINODE_FMT_BTREE && + ip->i_df.if_format != XFS_DINODE_FMT_LOCAL) || + XFS_TEST_ERROR(mp, XFS_ERRTAG_IFLUSH_4)) { xfs_alert_tag(mp, XFS_PTAG_IFLUSH, "%s: Bad directory inode %llu, ptr "PTR_FMT, __func__, ip->i_ino, ip); goto flush_out; } } - if (XFS_TEST_ERROR(ip->i_df.if_nextents + xfs_ifork_nextents(&ip->i_af) > - ip->i_nblocks, mp, XFS_ERRTAG_IFLUSH_5)) { + if (ip->i_df.if_nextents + xfs_ifork_nextents(&ip->i_af) > + ip->i_nblocks || XFS_TEST_ERROR(mp, XFS_ERRTAG_IFLUSH_5)) { xfs_alert_tag(mp, XFS_PTAG_IFLUSH, "%s: detected corrupt incore inode %llu, " "total extents = %llu nblocks = %lld, ptr "PTR_FMT, @@ -2406,8 +2404,8 @@ xfs_iflush( ip->i_nblocks, ip); goto flush_out; } - if (XFS_TEST_ERROR(ip->i_forkoff > mp->m_sb.sb_inodesize, - mp, XFS_ERRTAG_IFLUSH_6)) { + if (ip->i_forkoff > mp->m_sb.sb_inodesize || + XFS_TEST_ERROR(mp, XFS_ERRTAG_IFLUSH_6)) { xfs_alert_tag(mp, XFS_PTAG_IFLUSH, "%s: bad inode %llu, forkoff 0x%x, ptr "PTR_FMT, __func__, ip->i_ino, ip->i_forkoff, ip); diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c index 6335b122486f..3483b3728b7b 100644 --- a/fs/xfs/xfs_iomap.c +++ b/fs/xfs/xfs_iomap.c @@ -993,7 +993,7 @@ xfs_buffered_write_iomap_begin( return error; if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(&ip->i_df)) || - XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) { + XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) { xfs_bmap_mark_sick(ip, XFS_DATA_FORK); error = -EFSCORRUPTED; goto out_unlock; diff --git a/fs/xfs/xfs_log.c b/fs/xfs/xfs_log.c index bf9245b1b4c6..e275dd265faf 100644 --- a/fs/xfs/xfs_log.c +++ b/fs/xfs/xfs_log.c @@ -968,8 +968,8 @@ xfs_log_unmount_write( * counters will be recalculated. Refer to xlog_check_unmount_rec for * more details. */ - if (XFS_TEST_ERROR(xfs_fs_has_sickness(mp, XFS_SICK_FS_COUNTERS), mp, - XFS_ERRTAG_FORCE_SUMMARY_RECALC)) { + if (xfs_fs_has_sickness(mp, XFS_SICK_FS_COUNTERS) || + XFS_TEST_ERROR(mp, XFS_ERRTAG_FORCE_SUMMARY_RECALC)) { xfs_alert(mp, "%s: will fix summary counters at next mount", __func__); return; @@ -1239,7 +1239,7 @@ xlog_ioend_work( /* * Race to shutdown the filesystem if we see an error. */ - if (XFS_TEST_ERROR(error, log->l_mp, XFS_ERRTAG_IODONE_IOERR)) { + if (error || XFS_TEST_ERROR(log->l_mp, XFS_ERRTAG_IODONE_IOERR)) { xfs_alert(log->l_mp, "log I/O error %d", error); xlog_force_shutdown(log, SHUTDOWN_LOG_IO_ERROR); } @@ -1848,7 +1848,7 @@ xlog_sync( * detects the bad CRC and attempts to recover. */ #ifdef DEBUG - if (XFS_TEST_ERROR(false, log->l_mp, XFS_ERRTAG_LOG_BAD_CRC)) { + if (XFS_TEST_ERROR(log->l_mp, XFS_ERRTAG_LOG_BAD_CRC)) { iclog->ic_header.h_crc &= cpu_to_le32(0xAAAAAAAA); iclog->ic_fail_crc = true; xfs_warn(log->l_mp, diff --git a/fs/xfs/xfs_trans_ail.c b/fs/xfs/xfs_trans_ail.c index 85641af916a0..62d764205206 100644 --- a/fs/xfs/xfs_trans_ail.c +++ b/fs/xfs/xfs_trans_ail.c @@ -385,7 +385,7 @@ xfsaild_push_item( * If log item pinning is enabled, skip the push and track the item as * pinned. This can help induce head-behind-tail conditions. */ - if (XFS_TEST_ERROR(false, ailp->ail_log->l_mp, XFS_ERRTAG_LOG_ITEM_PIN)) + if (XFS_TEST_ERROR(ailp->ail_log->l_mp, XFS_ERRTAG_LOG_ITEM_PIN)) return XFS_ITEM_PINNED; /* diff --git a/include/keys/request_key_auth-type.h b/include/keys/request_key_auth-type.h index 36b89a933310..01e42ee5f409 100644 --- a/include/keys/request_key_auth-type.h +++ b/include/keys/request_key_auth-type.h @@ -9,12 +9,14 @@ #define _KEYS_REQUEST_KEY_AUTH_TYPE_H #include +#include /* * Authorisation record for request_key(). */ struct request_key_auth { struct rcu_head rcu; + refcount_t usage; struct key *target_key; struct key *dest_keyring; const struct cred *cred; diff --git a/include/linux/backing-file.h b/include/linux/backing-file.h index 2eed0ffb5e8f..cd18acd7ac5b 100644 --- a/include/linux/backing-file.h +++ b/include/linux/backing-file.h @@ -19,10 +19,10 @@ struct backing_file_ctx { void (*end_write)(struct file *, loff_t, ssize_t); }; -struct file *backing_file_open(const struct path *user_path, int flags, +struct file *backing_file_open(const struct file *user_file, int flags, const struct path *real_path, const struct cred *cred); -struct file *backing_tmpfile_open(const struct path *user_path, int flags, +struct file *backing_tmpfile_open(const struct file *user_file, int flags, const struct path *real_parentpath, umode_t mode, const struct cred *cred); ssize_t backing_file_read_iter(struct file *file, struct iov_iter *iter, diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 11d0a1b8daa2..33f5b16c0fd9 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1105,16 +1105,12 @@ static inline void blk_flush_plug(struct blk_plug *plug, bool async) __blk_flush_plug(plug, async); } -/* - * tsk == current here - */ -static inline void blk_plug_invalidate_ts(struct task_struct *tsk) +static __always_inline void blk_plug_invalidate_ts(void) { - struct blk_plug *plug = tsk->plug; - - if (plug) - plug->cur_ktime = 0; - current->flags &= ~PF_BLOCK_TS; + if (unlikely(current->flags & PF_BLOCK_TS)) { + current->plug->cur_ktime = 0; + current->flags &= ~PF_BLOCK_TS; + } } int blkdev_issue_flush(struct block_device *bdev); @@ -1140,7 +1136,7 @@ static inline void blk_flush_plug(struct blk_plug *plug, bool async) { } -static inline void blk_plug_invalidate_ts(struct task_struct *tsk) +static inline void blk_plug_invalidate_ts(void) { } diff --git a/include/linux/err.h b/include/linux/err.h index 1d60aa86db53..281115ff0425 100644 --- a/include/linux/err.h +++ b/include/linux/err.h @@ -36,7 +36,7 @@ * * Return: A pointer with @error encoded within its value. */ -static inline void * __must_check ERR_PTR(long error) +static __always_inline void * __must_check ERR_PTR(long error) { return (void *) error; } @@ -52,7 +52,7 @@ static inline void * __must_check ERR_PTR(long error) * @ptr: An error pointer. * Return: The error code within @ptr. */ -static inline long __must_check PTR_ERR(__force const void *ptr) +static __always_inline long __must_check PTR_ERR(__force const void *ptr) { return (long) ptr; } @@ -65,7 +65,7 @@ static inline long __must_check PTR_ERR(__force const void *ptr) * @ptr: The pointer to check. * Return: true if @ptr is an error pointer, false otherwise. */ -static inline bool __must_check IS_ERR(__force const void *ptr) +static __always_inline bool __must_check IS_ERR(__force const void *ptr) { return IS_ERR_VALUE((unsigned long)ptr); } @@ -79,7 +79,7 @@ static inline bool __must_check IS_ERR(__force const void *ptr) * * Like IS_ERR(), but also returns true for a null pointer. */ -static inline bool __must_check IS_ERR_OR_NULL(__force const void *ptr) +static __always_inline bool __must_check IS_ERR_OR_NULL(__force const void *ptr) { return unlikely(!ptr) || IS_ERR_VALUE((unsigned long)ptr); } @@ -91,7 +91,7 @@ static inline bool __must_check IS_ERR_OR_NULL(__force const void *ptr) * Explicitly cast an error-valued pointer to another pointer type in such a * way as to make it clear that's what's going on. */ -static inline void * __must_check ERR_CAST(__force const void *ptr) +static __always_inline void * __must_check ERR_CAST(__force const void *ptr) { /* cast away the const */ return (void *) ptr; @@ -114,7 +114,7 @@ static inline void * __must_check ERR_CAST(__force const void *ptr) * * Return: The error code within @ptr if it is an error pointer; 0 otherwise. */ -static inline int __must_check PTR_ERR_OR_ZERO(__force const void *ptr) +static __always_inline int __must_check PTR_ERR_OR_ZERO(__force const void *ptr) { if (IS_ERR(ptr)) return PTR_ERR(ptr); diff --git a/include/linux/fs.h b/include/linux/fs.h index 87720e1b5419..0eb43147dc87 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2738,7 +2738,20 @@ struct file *dentry_open(const struct path *path, int flags, const struct cred *creds); struct file *dentry_create(const struct path *path, int flags, umode_t mode, const struct cred *cred); -struct path *backing_file_user_path(struct file *f); +struct path *backing_file_user_path(const struct file *f); + +#ifdef CONFIG_SECURITY +void *backing_file_security(const struct file *f); +void backing_file_set_security(struct file *f, void *security); +#else +static inline void *backing_file_security(const struct file *f) +{ + return NULL; +} +static inline void backing_file_set_security(struct file *f, void *security) +{ +} +#endif /* CONFIG_SECURITY */ /* * When mmapping a file on a stackable filesystem (e.g., overlayfs), the file @@ -2750,14 +2763,14 @@ struct path *backing_file_user_path(struct file *f); * by fstat() on that same fd. */ /* Get the path to display in /proc//maps */ -static inline const struct path *file_user_path(struct file *f) +static inline const struct path *file_user_path(const struct file *f) { if (unlikely(f->f_mode & FMODE_BACKING)) return backing_file_user_path(f); return &f->f_path; } /* Get the inode whose inode number to display in /proc//maps */ -static inline const struct inode *file_user_inode(struct file *f) +static inline const struct inode *file_user_inode(const struct file *f) { if (unlikely(f->f_mode & FMODE_BACKING)) return d_inode(backing_file_user_path(f)->dentry); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 04b81e2166d5..b4235e99f0a9 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1745,6 +1745,11 @@ int kvm_request_irq_source_id(struct kvm *kvm); void kvm_free_irq_source_id(struct kvm *kvm, int irq_source_id); bool kvm_arch_irqfd_allowed(struct kvm *kvm, struct kvm_irqfd *args); +static inline bool is_gfn_in_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) +{ + return gfn >= slot->base_gfn && gfn < slot->base_gfn + slot->npages; +} + /* * Returns a pointer to the memslot if it contains gfn. * Otherwise returns NULL. @@ -1755,7 +1760,7 @@ try_get_memslot(struct kvm_memory_slot *slot, gfn_t gfn) if (!slot) return NULL; - if (gfn >= slot->base_gfn && gfn < slot->base_gfn + slot->npages) + if (is_gfn_in_memslot(slot, gfn)) return slot; else return NULL; diff --git a/include/linux/lsm_audit.h b/include/linux/lsm_audit.h index 97a8b21eb033..c0a2839253fa 100644 --- a/include/linux/lsm_audit.h +++ b/include/linux/lsm_audit.h @@ -93,7 +93,7 @@ struct common_audit_data { #endif char *kmod_name; struct lsm_ioctlop_audit *op; - struct file *file; + const struct file *file; struct lsm_ibpkey_audit *ibpkey; struct lsm_ibendport_audit *ibendport; int reason; diff --git a/include/linux/lsm_hook_defs.h b/include/linux/lsm_hook_defs.h index 9eca013aa5e1..addb34abffa1 100644 --- a/include/linux/lsm_hook_defs.h +++ b/include/linux/lsm_hook_defs.h @@ -188,6 +188,9 @@ LSM_HOOK(int, 0, file_permission, struct file *file, int mask) LSM_HOOK(int, 0, file_alloc_security, struct file *file) LSM_HOOK(void, LSM_RET_VOID, file_release, struct file *file) LSM_HOOK(void, LSM_RET_VOID, file_free_security, struct file *file) +LSM_HOOK(int, 0, backing_file_alloc, struct file *backing_file, + const struct file *user_file) +LSM_HOOK(void, LSM_RET_VOID, backing_file_free, struct file *backing_file) LSM_HOOK(int, 0, file_ioctl, struct file *file, unsigned int cmd, unsigned long arg) LSM_HOOK(int, 0, file_ioctl_compat, struct file *file, unsigned int cmd, @@ -195,6 +198,8 @@ LSM_HOOK(int, 0, file_ioctl_compat, struct file *file, unsigned int cmd, LSM_HOOK(int, 0, mmap_addr, unsigned long addr) LSM_HOOK(int, 0, mmap_file, struct file *file, unsigned long reqprot, unsigned long prot, unsigned long flags) +LSM_HOOK(int, 0, mmap_backing_file, struct vm_area_struct *vma, + struct file *backing_file, struct file *user_file) LSM_HOOK(int, 0, file_mprotect, struct vm_area_struct *vma, unsigned long reqprot, unsigned long prot) LSM_HOOK(int, 0, file_lock, struct file *file, unsigned int cmd) diff --git a/include/linux/lsm_hooks.h b/include/linux/lsm_hooks.h index 090d1d3e19fe..0876cf11e200 100644 --- a/include/linux/lsm_hooks.h +++ b/include/linux/lsm_hooks.h @@ -104,6 +104,7 @@ struct security_hook_list { struct lsm_blob_sizes { int lbs_cred; int lbs_file; + int lbs_backing_file; int lbs_ib; int lbs_inode; int lbs_sock; diff --git a/include/linux/security.h b/include/linux/security.h index 2c6db949ad1a..e4300f2ff11b 100644 --- a/include/linux/security.h +++ b/include/linux/security.h @@ -421,11 +421,17 @@ int security_file_permission(struct file *file, int mask); int security_file_alloc(struct file *file); void security_file_release(struct file *file); void security_file_free(struct file *file); +int security_backing_file_alloc(struct file *backing_file, + const struct file *user_file); +void security_backing_file_free(struct file *backing_file); int security_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg); int security_file_ioctl_compat(struct file *file, unsigned int cmd, unsigned long arg); int security_mmap_file(struct file *file, unsigned long prot, unsigned long flags); +int security_mmap_backing_file(struct vm_area_struct *vma, + struct file *backing_file, + struct file *user_file); int security_mmap_addr(unsigned long addr); int security_file_mprotect(struct vm_area_struct *vma, unsigned long reqprot, unsigned long prot); @@ -1065,6 +1071,15 @@ static inline void security_file_release(struct file *file) static inline void security_file_free(struct file *file) { } +static inline int security_backing_file_alloc(struct file *backing_file, + const struct file *user_file) +{ + return 0; +} + +static inline void security_backing_file_free(struct file *backing_file) +{ } + static inline int security_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -1084,6 +1099,13 @@ static inline int security_mmap_file(struct file *file, unsigned long prot, return 0; } +static inline int security_mmap_backing_file(struct vm_area_struct *vma, + struct file *backing_file, + struct file *user_file) +{ + return 0; +} + static inline int security_mmap_addr(unsigned long addr) { return cap_mmap_addr(addr); diff --git a/include/linux/skmsg.h b/include/linux/skmsg.h index 5581e7263c50..ce8464324d42 100644 --- a/include/linux/skmsg.h +++ b/include/linux/skmsg.h @@ -4,6 +4,7 @@ #ifndef _LINUX_SKMSG_H #define _LINUX_SKMSG_H +#include #include #include #include @@ -199,11 +200,14 @@ static inline void sk_msg_xfer(struct sk_msg *dst, struct sk_msg *src, int which, u32 size) { dst->sg.data[which] = src->sg.data[which]; + __assign_bit(which, dst->sg.copy, test_bit(which, src->sg.copy)); dst->sg.data[which].length = size; dst->sg.size += size; src->sg.size -= size; src->sg.data[which].length -= size; src->sg.data[which].offset += size; + if (!src->sg.data[which].length) + __clear_bit(which, src->sg.copy); } static inline void sk_msg_xfer_full(struct sk_msg *dst, struct sk_msg *src) @@ -273,16 +277,19 @@ static inline void sk_msg_page_add(struct sk_msg *msg, struct page *page, static inline void sk_msg_sg_copy(struct sk_msg *msg, u32 i, bool copy_state) { do { - if (copy_state) - __set_bit(i, msg->sg.copy); - else - __clear_bit(i, msg->sg.copy); + __assign_bit(i, msg->sg.copy, copy_state); sk_msg_iter_var_next(i); if (i == msg->sg.end) break; } while (1); } +static inline void sk_msg_sg_copy_assign(struct sk_msg *dst, u32 dst_i, + const struct sk_msg *src, u32 src_i) +{ + __assign_bit(dst_i, dst->sg.copy, test_bit(src_i, src->sg.copy)); +} + static inline void sk_msg_sg_copy_set(struct sk_msg *msg, u32 start) { sk_msg_sg_copy(msg, start, true); diff --git a/include/net/bond_3ad.h b/include/net/bond_3ad.h index dba369a2cf27..4ee2ccf46279 100644 --- a/include/net/bond_3ad.h +++ b/include/net/bond_3ad.h @@ -242,7 +242,7 @@ typedef struct port { churn_state_t sm_churn_actor_state; churn_state_t sm_churn_partner_state; struct slave *slave; /* pointer to the bond slave that this port belongs to */ - struct aggregator *aggregator; /* pointer to an aggregator that this port related to */ + struct aggregator __rcu *aggregator; /* pointer to an aggregator that this port related to */ struct port *next_port_in_aggregator; /* Next port on the linked list of the parent aggregator */ u32 transaction_id; /* continuous number for identification of Marker PDU's; */ struct lacpdu lacpdu; /* the lacpdu that will be sent for this port */ @@ -274,6 +274,7 @@ struct ad_slave_info { struct port port; /* 802.3ad port structure */ struct bond_3ad_stats stats; u16 id; + u16 port_priority; }; static inline const char *bond_3ad_churn_desc(churn_state_t state) diff --git a/include/net/bond_options.h b/include/net/bond_options.h index 18687ccf0638..e6eedf23aea1 100644 --- a/include/net/bond_options.h +++ b/include/net/bond_options.h @@ -77,6 +77,8 @@ enum { BOND_OPT_NS_TARGETS, BOND_OPT_PRIO, BOND_OPT_COUPLED_CONTROL, + BOND_OPT_BROADCAST_NEIGH, + BOND_OPT_ACTOR_PORT_PRIO, BOND_OPT_LAST }; diff --git a/include/net/bonding.h b/include/net/bonding.h index 66940d41d485..0d9c1eb40d12 100644 --- a/include/net/bonding.h +++ b/include/net/bonding.h @@ -115,6 +115,8 @@ static inline int is_netpoll_tx_blocked(struct net_device *dev) #define is_netpoll_tx_blocked(dev) (0) #endif +DECLARE_STATIC_KEY_FALSE(bond_bcast_neigh_enabled); + struct bond_params { int mode; int xmit_policy; @@ -149,6 +151,7 @@ struct bond_params { struct in6_addr ns_targets[BOND_MAX_NS_TARGETS]; #endif int coupled_control; + int broadcast_neighbor; /* 2 bytes of padding : see ether_addr_equal_64bits() */ u8 ad_actor_system[ETH_ALEN + 2]; diff --git a/include/net/phonet/pn_dev.h b/include/net/phonet/pn_dev.h index e9dc8dca5817..6b2102b4ece3 100644 --- a/include/net/phonet/pn_dev.h +++ b/include/net/phonet/pn_dev.h @@ -38,7 +38,7 @@ int phonet_address_add(struct net_device *dev, u8 addr); int phonet_address_del(struct net_device *dev, u8 addr); u8 phonet_address_get(struct net_device *dev, u8 addr); int phonet_address_lookup(struct net *net, u8 addr); -void phonet_address_notify(int event, struct net_device *dev, u8 addr); +void phonet_address_notify(struct net *net, int event, u32 ifindex, u8 addr); int phonet_route_add(struct net_device *dev, u8 daddr); int phonet_route_del(struct net_device *dev, u8 daddr); diff --git a/include/net/rtnetlink.h b/include/net/rtnetlink.h index 2d3eb7cb4dff..7c057611f4ab 100644 --- a/include/net/rtnetlink.h +++ b/include/net/rtnetlink.h @@ -212,6 +212,8 @@ int rtnl_configure_link(struct net_device *dev, const struct ifinfomsg *ifm, int rtnl_nla_parse_ifinfomsg(struct nlattr **tb, const struct nlattr *nla_peer, struct netlink_ext_ack *exterr); struct net *rtnl_get_net_ns_capable(struct sock *sk, int netnsid); +bool rtnl_dev_link_net_capable(const struct net_device *dev, + const struct net *link_net); #define MODULE_ALIAS_RTNL_LINK(kind) MODULE_ALIAS("rtnl-link-" kind) diff --git a/include/net/sock.h b/include/net/sock.h index 0d77a87929f9..dffbaaa7fe49 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -2040,7 +2040,7 @@ static inline int sk_rx_queue_get(const struct sock *sk) static inline void sk_set_socket(struct sock *sk, struct socket *sock) { - sk->sk_socket = sock; + WRITE_ONCE(sk->sk_socket, sock); } static inline wait_queue_head_t *sk_sleep(struct sock *sk) diff --git a/include/net/udp_tunnel.h b/include/net/udp_tunnel.h index 6e2c5c77031f..8ed36ec520d7 100644 --- a/include/net/udp_tunnel.h +++ b/include/net/udp_tunnel.h @@ -152,13 +152,13 @@ void udp_tunnel_xmit_skb(struct rtable *rt, struct sock *sk, struct sk_buff *skb __be16 df, __be16 src_port, __be16 dst_port, bool xnet, bool nocheck); -int udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk, - struct sk_buff *skb, - struct net_device *dev, - const struct in6_addr *saddr, - const struct in6_addr *daddr, - __u8 prio, __u8 ttl, __be32 label, - __be16 src_port, __be16 dst_port, bool nocheck); +void udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk, + struct sk_buff *skb, + struct net_device *dev, + const struct in6_addr *saddr, + const struct in6_addr *daddr, + __u8 prio, __u8 ttl, __be32 label, + __be16 src_port, __be16 dst_port, bool nocheck); void udp_tunnel_sock_release(struct socket *sock); diff --git a/include/uapi/linux/if_link.h b/include/uapi/linux/if_link.h index 2acc7687e017..087048d620f3 100644 --- a/include/uapi/linux/if_link.h +++ b/include/uapi/linux/if_link.h @@ -1551,6 +1551,9 @@ enum { IFLA_BOND_SLAVE_AD_ACTOR_OPER_PORT_STATE, IFLA_BOND_SLAVE_AD_PARTNER_OPER_PORT_STATE, IFLA_BOND_SLAVE_PRIO, + IFLA_BOND_SLAVE_ACTOR_PORT_PRIO, + IFLA_BOND_SLAVE_AD_CHURN_ACTOR_STATE, + IFLA_BOND_SLAVE_AD_CHURN_PARTNER_STATE, __IFLA_BOND_SLAVE_MAX, }; diff --git a/io_uring/net.c b/io_uring/net.c index 8eb0ebdc6a72..f2f2ae1037e9 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -1719,7 +1719,7 @@ int io_socket(struct io_kiocb *req, unsigned int issue_flags) int io_connect_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) { struct io_connect *conn = io_kiocb_to_cmd(req, struct io_connect); - struct io_async_msghdr *io; + struct sockaddr_storage *addr; if (sqe->len || sqe->buf_index || sqe->rw_flags || sqe->splice_fd_in) return -EINVAL; @@ -1728,17 +1728,17 @@ int io_connect_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) conn->addr_len = READ_ONCE(sqe->addr2); conn->in_progress = conn->seen_econnaborted = false; - io = io_msg_alloc_async(req); - if (unlikely(!io)) + if (io_alloc_async_data(req)) return -ENOMEM; + addr = req->async_data; - return move_addr_to_kernel(conn->addr, conn->addr_len, &io->addr); + return move_addr_to_kernel(conn->addr, conn->addr_len, addr); } int io_connect(struct io_kiocb *req, unsigned int issue_flags) { struct io_connect *connect = io_kiocb_to_cmd(req, struct io_connect); - struct io_async_msghdr *io = req->async_data; + struct sockaddr_storage *addr = req->async_data; unsigned file_flags; int ret; bool force_nonblock = issue_flags & IO_URING_F_NONBLOCK; @@ -1752,8 +1752,7 @@ int io_connect(struct io_kiocb *req, unsigned int issue_flags) file_flags = force_nonblock ? O_NONBLOCK : 0; - ret = __sys_connect_file(req->file, &io->addr, connect->addr_len, - file_flags); + ret = __sys_connect_file(req->file, addr, connect->addr_len, file_flags); if ((ret == -EAGAIN || ret == -EINPROGRESS || ret == -ECONNABORTED) && force_nonblock) { if (ret == -EINPROGRESS) { @@ -1782,7 +1781,6 @@ int io_connect(struct io_kiocb *req, unsigned int issue_flags) out: if (ret < 0) req_set_fail(req); - io_req_msg_cleanup(req, issue_flags); io_req_set_res(req, ret, 0); return IOU_OK; } @@ -1792,15 +1790,15 @@ int io_connect(struct io_kiocb *req, unsigned int issue_flags) * which in turn end up in mnt_want_write() which will grab the fs * percpu start write sem. This can trigger a lockdep warning. */ -static int io_bind_file_create(const struct io_async_msghdr *io, int addr_len) +static int io_bind_file_create(const struct sockaddr_storage *addr, int addr_len) { const struct sockaddr_un *sun; - if (io->addr.ss_family != AF_UNIX) + if (addr->ss_family != AF_UNIX) return 0; if (addr_len <= offsetof(struct sockaddr_un, sun_path)) return 0; - sun = (const struct sockaddr_un *) &io->addr; + sun = (const struct sockaddr_un *) addr; return sun->sun_path[0] != '\0'; } @@ -1808,7 +1806,7 @@ int io_bind_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) { struct io_bind *bind = io_kiocb_to_cmd(req, struct io_bind); struct sockaddr __user *uaddr; - struct io_async_msghdr *io; + struct sockaddr_storage *addr; int ret; if (sqe->len || sqe->buf_index || sqe->rw_flags || sqe->splice_fd_in) @@ -1817,21 +1815,23 @@ int io_bind_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) uaddr = u64_to_user_ptr(READ_ONCE(sqe->addr)); bind->addr_len = READ_ONCE(sqe->addr2); - io = io_msg_alloc_async(req); - if (unlikely(!io)) + if (io_alloc_async_data(req)) return -ENOMEM; - ret = move_addr_to_kernel(uaddr, bind->addr_len, &io->addr); + addr = req->async_data; + + ret = move_addr_to_kernel(uaddr, bind->addr_len, addr); if (unlikely(ret)) return ret; - if (io_bind_file_create(io, bind->addr_len)) + if (io_bind_file_create(addr, bind->addr_len)) req->flags |= REQ_F_FORCE_ASYNC; return 0; } + int io_bind(struct io_kiocb *req, unsigned int issue_flags) { struct io_bind *bind = io_kiocb_to_cmd(req, struct io_bind); - struct io_async_msghdr *io = req->async_data; + struct sockaddr_storage *addr = req->async_data; struct socket *sock; int ret; @@ -1839,7 +1839,7 @@ int io_bind(struct io_kiocb *req, unsigned int issue_flags) if (unlikely(!sock)) return -ENOTSOCK; - ret = __sys_bind_socket(sock, &io->addr, bind->addr_len); + ret = __sys_bind_socket(sock, addr, bind->addr_len); if (ret < 0) req_set_fail(req); io_req_set_res(req, ret, 0); diff --git a/io_uring/opdef.c b/io_uring/opdef.c index 5dc1cba158a0..bbb62d2ab2a3 100644 --- a/io_uring/opdef.c +++ b/io_uring/opdef.c @@ -205,7 +205,7 @@ const struct io_issue_def io_issue_defs[] = { .unbound_nonreg_file = 1, .pollout = 1, #if defined(CONFIG_NET) - .async_size = sizeof(struct io_async_msghdr), + .async_size = sizeof(struct sockaddr_storage), .prep = io_connect_prep, .issue = io_connect, #else @@ -501,7 +501,7 @@ const struct io_issue_def io_issue_defs[] = { .needs_file = 1, .prep = io_bind_prep, .issue = io_bind, - .async_size = sizeof(struct io_async_msghdr), + .async_size = sizeof(struct sockaddr_storage), #else .prep = io_eopnotsupp_prep, #endif diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c index 1ebf40badbf6..9b89732f2cb4 100644 --- a/kernel/bpf/cgroup.c +++ b/kernel/bpf/cgroup.c @@ -1774,7 +1774,7 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head, kfree(ctx.cur_val); if (ret == 1 && ctx.new_updated) { - kfree(*buf); + kvfree(*buf); *buf = ctx.new_val; *pcount = ctx.new_len; } else { diff --git a/kernel/fork.c b/kernel/fork.c index c4955cffcb6f..c81b45509e8a 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -2450,6 +2450,7 @@ __latent_entropy struct task_struct *copy_process( #ifdef CONFIG_BLOCK p->plug = NULL; + p->flags &= ~PF_BLOCK_TS; #endif futex_init_task(p); diff --git a/kernel/futex/pi.c b/kernel/futex/pi.c index b4cdda1ea130..ccd5d19dea68 100644 --- a/kernel/futex/pi.c +++ b/kernel/futex/pi.c @@ -922,6 +922,7 @@ int futex_lock_pi(u32 __user *uaddr, unsigned int flags, ktime_t *time, int tryl struct rt_mutex_waiter rt_waiter; struct futex_hash_bucket *hb; struct futex_q q = futex_q_init; + DEFINE_WAKE_Q(wake_q); int res, ret; if (!IS_ENABLED(CONFIG_FUTEX_PI)) @@ -1019,8 +1020,11 @@ int futex_lock_pi(u32 __user *uaddr, unsigned int flags, ktime_t *time, int tryl * such that futex_unlock_pi() is guaranteed to observe the waiter when * it sees the futex_q::pi_state. */ - ret = __rt_mutex_start_proxy_lock(&q.pi_state->pi_mutex, &rt_waiter, current); + ret = __rt_mutex_start_proxy_lock(&q.pi_state->pi_mutex, &rt_waiter, current, &wake_q); + preempt_disable(); raw_spin_unlock_irq(&q.pi_state->pi_mutex.wait_lock); + wake_up_q(&wake_q); + preempt_enable(); if (ret) { if (ret == 1) diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c index cbae8c0b89ab..6c94da061ec2 100644 --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -575,6 +575,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas struct lockdep_map *nest_lock, unsigned long ip, struct ww_acquire_ctx *ww_ctx, const bool use_ww_ctx) { + DEFINE_WAKE_Q(wake_q); struct mutex_waiter waiter; struct ww_mutex *ww; int ret; @@ -625,7 +626,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas */ if (__mutex_trylock(lock)) { if (ww_ctx) - __ww_mutex_check_waiters(lock, ww_ctx); + __ww_mutex_check_waiters(lock, ww_ctx, &wake_q); goto skip_wait; } @@ -645,7 +646,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas * Add in stamp order, waking up waiters that must kill * themselves. */ - ret = __ww_mutex_add_waiter(&waiter, lock, ww_ctx); + ret = __ww_mutex_add_waiter(&waiter, lock, ww_ctx, &wake_q); if (ret) goto err_early_kill; } @@ -681,6 +682,10 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas } raw_spin_unlock(&lock->wait_lock); + /* Make sure we do wakeups before calling schedule */ + wake_up_q(&wake_q); + wake_q_init(&wake_q); + schedule_preempt_disabled(); first = __mutex_waiter_is_first(lock, &waiter); @@ -714,7 +719,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas */ if (!ww_ctx->is_wait_die && !__mutex_waiter_is_first(lock, &waiter)) - __ww_mutex_check_waiters(lock, ww_ctx); + __ww_mutex_check_waiters(lock, ww_ctx, &wake_q); } __mutex_remove_waiter(lock, &waiter); @@ -730,6 +735,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas ww_mutex_lock_acquired(ww, ww_ctx); raw_spin_unlock(&lock->wait_lock); + wake_up_q(&wake_q); preempt_enable(); return 0; @@ -741,6 +747,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int subclas raw_spin_unlock(&lock->wait_lock); debug_mutex_free_waiter(&waiter); mutex_release(&lock->dep_map, ip); + wake_up_q(&wake_q); preempt_enable(); return ret; } @@ -951,9 +958,10 @@ static noinline void __sched __mutex_unlock_slowpath(struct mutex *lock, unsigne if (owner & MUTEX_FLAG_HANDOFF) __mutex_handoff(lock, next); + preempt_disable(); raw_spin_unlock(&lock->wait_lock); - wake_up_q(&wake_q); + preempt_enable(); } #ifndef CONFIG_DEBUG_LOCK_ALLOC diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index 66f9fb25d4be..9b4bd06dd9ac 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -34,13 +34,15 @@ static inline int __ww_mutex_add_waiter(struct rt_mutex_waiter *waiter, struct rt_mutex *lock, - struct ww_acquire_ctx *ww_ctx) + struct ww_acquire_ctx *ww_ctx, + struct wake_q_head *wake_q) { return 0; } static inline void __ww_mutex_check_waiters(struct rt_mutex *lock, - struct ww_acquire_ctx *ww_ctx) + struct ww_acquire_ctx *ww_ctx, + struct wake_q_head *wake_q) { } @@ -1201,7 +1203,8 @@ static int __sched task_blocks_on_rt_mutex(struct rt_mutex_base *lock, struct rt_mutex_waiter *waiter, struct task_struct *task, struct ww_acquire_ctx *ww_ctx, - enum rtmutex_chainwalk chwalk) + enum rtmutex_chainwalk chwalk, + struct wake_q_head *wake_q) { struct task_struct *owner = rt_mutex_owner(lock); struct rt_mutex_waiter *top_waiter = waiter; @@ -1245,7 +1248,7 @@ static int __sched task_blocks_on_rt_mutex(struct rt_mutex_base *lock, /* Check whether the waiter should back out immediately */ rtm = container_of(lock, struct rt_mutex, rtmutex); - res = __ww_mutex_add_waiter(waiter, rtm, ww_ctx); + res = __ww_mutex_add_waiter(waiter, rtm, ww_ctx, wake_q); if (res) { raw_spin_lock(&task->pi_lock); rt_mutex_dequeue(lock, waiter); @@ -1544,6 +1547,9 @@ static void __sched remove_waiter(struct rt_mutex_base *lock, lockdep_assert_held(&lock->wait_lock); + if (!waiter_task) /* never enqueued */ + return; + scoped_guard(raw_spinlock, &waiter_task->pi_lock) { rt_mutex_dequeue(lock, waiter); waiter_task->pi_blocked_on = NULL; @@ -1677,12 +1683,14 @@ static void __sched rt_mutex_handle_deadlock(int res, int detect_deadlock, * @state: The task state for sleeping * @chwalk: Indicator whether full or partial chainwalk is requested * @waiter: Initializer waiter for blocking + * @wake_q: The wake_q to wake tasks after we release the wait_lock */ static int __sched __rt_mutex_slowlock(struct rt_mutex_base *lock, struct ww_acquire_ctx *ww_ctx, unsigned int state, enum rtmutex_chainwalk chwalk, - struct rt_mutex_waiter *waiter) + struct rt_mutex_waiter *waiter, + struct wake_q_head *wake_q) { struct rt_mutex *rtm = container_of(lock, struct rt_mutex, rtmutex); struct ww_mutex *ww = ww_container_of(rtm); @@ -1693,7 +1701,7 @@ static int __sched __rt_mutex_slowlock(struct rt_mutex_base *lock, /* Try to acquire the lock again: */ if (try_to_take_rt_mutex(lock, current, NULL)) { if (build_ww_mutex() && ww_ctx) { - __ww_mutex_check_waiters(rtm, ww_ctx); + __ww_mutex_check_waiters(rtm, ww_ctx, wake_q); ww_mutex_lock_acquired(ww, ww_ctx); } return 0; @@ -1703,7 +1711,7 @@ static int __sched __rt_mutex_slowlock(struct rt_mutex_base *lock, trace_contention_begin(lock, LCB_F_RT); - ret = task_blocks_on_rt_mutex(lock, waiter, current, ww_ctx, chwalk); + ret = task_blocks_on_rt_mutex(lock, waiter, current, ww_ctx, chwalk, wake_q); if (likely(!ret)) ret = rt_mutex_slowlock_block(lock, ww_ctx, state, NULL, waiter); @@ -1711,7 +1719,7 @@ static int __sched __rt_mutex_slowlock(struct rt_mutex_base *lock, /* acquired the lock */ if (build_ww_mutex() && ww_ctx) { if (!ww_ctx->is_wait_die) - __ww_mutex_check_waiters(rtm, ww_ctx); + __ww_mutex_check_waiters(rtm, ww_ctx, wake_q); ww_mutex_lock_acquired(ww, ww_ctx); } } else { @@ -1733,7 +1741,8 @@ static int __sched __rt_mutex_slowlock(struct rt_mutex_base *lock, static inline int __rt_mutex_slowlock_locked(struct rt_mutex_base *lock, struct ww_acquire_ctx *ww_ctx, - unsigned int state) + unsigned int state, + struct wake_q_head *wake_q) { struct rt_mutex_waiter waiter; int ret; @@ -1742,7 +1751,7 @@ static inline int __rt_mutex_slowlock_locked(struct rt_mutex_base *lock, waiter.ww_ctx = ww_ctx; ret = __rt_mutex_slowlock(lock, ww_ctx, state, RT_MUTEX_MIN_CHAINWALK, - &waiter); + &waiter, wake_q); debug_rt_mutex_free_waiter(&waiter); return ret; @@ -1758,6 +1767,7 @@ static int __sched rt_mutex_slowlock(struct rt_mutex_base *lock, struct ww_acquire_ctx *ww_ctx, unsigned int state) { + DEFINE_WAKE_Q(wake_q); unsigned long flags; int ret; @@ -1779,8 +1789,11 @@ static int __sched rt_mutex_slowlock(struct rt_mutex_base *lock, * irqsave/restore variants. */ raw_spin_lock_irqsave(&lock->wait_lock, flags); - ret = __rt_mutex_slowlock_locked(lock, ww_ctx, state); + ret = __rt_mutex_slowlock_locked(lock, ww_ctx, state, &wake_q); + preempt_disable(); raw_spin_unlock_irqrestore(&lock->wait_lock, flags); + wake_up_q(&wake_q); + preempt_enable(); rt_mutex_post_schedule(); return ret; @@ -1806,8 +1819,10 @@ static __always_inline int __rt_mutex_lock(struct rt_mutex_base *lock, /** * rtlock_slowlock_locked - Slow path lock acquisition for RT locks * @lock: The underlying RT mutex + * @wake_q: The wake_q to wake tasks after we release the wait_lock */ -static void __sched rtlock_slowlock_locked(struct rt_mutex_base *lock) +static void __sched rtlock_slowlock_locked(struct rt_mutex_base *lock, + struct wake_q_head *wake_q) { struct rt_mutex_waiter waiter; struct task_struct *owner; @@ -1824,7 +1839,7 @@ static void __sched rtlock_slowlock_locked(struct rt_mutex_base *lock) trace_contention_begin(lock, LCB_F_RT); - task_blocks_on_rt_mutex(lock, &waiter, current, NULL, RT_MUTEX_MIN_CHAINWALK); + task_blocks_on_rt_mutex(lock, &waiter, current, NULL, RT_MUTEX_MIN_CHAINWALK, wake_q); for (;;) { /* Try to acquire the lock again */ @@ -1835,7 +1850,11 @@ static void __sched rtlock_slowlock_locked(struct rt_mutex_base *lock) owner = rt_mutex_owner(lock); else owner = NULL; + preempt_disable(); raw_spin_unlock_irq(&lock->wait_lock); + wake_up_q(wake_q); + wake_q_init(wake_q); + preempt_enable(); if (!owner || !rtmutex_spin_on_owner(lock, &waiter, owner)) schedule_rtlock(); @@ -1860,10 +1879,14 @@ static void __sched rtlock_slowlock_locked(struct rt_mutex_base *lock) static __always_inline void __sched rtlock_slowlock(struct rt_mutex_base *lock) { unsigned long flags; + DEFINE_WAKE_Q(wake_q); raw_spin_lock_irqsave(&lock->wait_lock, flags); - rtlock_slowlock_locked(lock); + rtlock_slowlock_locked(lock, &wake_q); + preempt_disable(); raw_spin_unlock_irqrestore(&lock->wait_lock, flags); + wake_up_q(&wake_q); + preempt_enable(); } #endif /* RT_MUTEX_BUILD_SPINLOCKS */ diff --git a/kernel/locking/rtmutex_api.c b/kernel/locking/rtmutex_api.c index a6974d044593..091b109acdc5 100644 --- a/kernel/locking/rtmutex_api.c +++ b/kernel/locking/rtmutex_api.c @@ -275,6 +275,7 @@ void __sched rt_mutex_proxy_unlock(struct rt_mutex_base *lock) * @lock: the rt_mutex to take * @waiter: the pre-initialized rt_mutex_waiter * @task: the task to prepare + * @wake_q: the wake_q to wake tasks after we release the wait_lock * * Starts the rt_mutex acquire; it enqueues the @waiter and does deadlock * detection. It does not wait, see rt_mutex_wait_proxy_lock() for that. @@ -291,7 +292,8 @@ void __sched rt_mutex_proxy_unlock(struct rt_mutex_base *lock) */ int __sched __rt_mutex_start_proxy_lock(struct rt_mutex_base *lock, struct rt_mutex_waiter *waiter, - struct task_struct *task) + struct task_struct *task, + struct wake_q_head *wake_q) { int ret; @@ -302,7 +304,7 @@ int __sched __rt_mutex_start_proxy_lock(struct rt_mutex_base *lock, /* We enforce deadlock detection for futexes */ ret = task_blocks_on_rt_mutex(lock, waiter, task, NULL, - RT_MUTEX_FULL_CHAINWALK); + RT_MUTEX_FULL_CHAINWALK, wake_q); if (ret && !rt_mutex_owner(lock)) { /* @@ -341,12 +343,16 @@ int __sched rt_mutex_start_proxy_lock(struct rt_mutex_base *lock, struct task_struct *task) { int ret; + DEFINE_WAKE_Q(wake_q); raw_spin_lock_irq(&lock->wait_lock); - ret = __rt_mutex_start_proxy_lock(lock, waiter, task); - if (unlikely(ret)) + ret = __rt_mutex_start_proxy_lock(lock, waiter, task, &wake_q); + if (unlikely(ret < 0)) remove_waiter(lock, waiter); + preempt_disable(); raw_spin_unlock_irq(&lock->wait_lock); + wake_up_q(&wake_q); + preempt_enable(); return ret; } diff --git a/kernel/locking/rtmutex_common.h b/kernel/locking/rtmutex_common.h index 1162e07cdaea..c38a2d2d4a7e 100644 --- a/kernel/locking/rtmutex_common.h +++ b/kernel/locking/rtmutex_common.h @@ -83,7 +83,8 @@ extern void rt_mutex_init_proxy_locked(struct rt_mutex_base *lock, extern void rt_mutex_proxy_unlock(struct rt_mutex_base *lock); extern int __rt_mutex_start_proxy_lock(struct rt_mutex_base *lock, struct rt_mutex_waiter *waiter, - struct task_struct *task); + struct task_struct *task, + struct wake_q_head *); extern int rt_mutex_start_proxy_lock(struct rt_mutex_base *lock, struct rt_mutex_waiter *waiter, struct task_struct *task); diff --git a/kernel/locking/rwbase_rt.c b/kernel/locking/rwbase_rt.c index 34a59569db6b..9f4322c07486 100644 --- a/kernel/locking/rwbase_rt.c +++ b/kernel/locking/rwbase_rt.c @@ -69,6 +69,7 @@ static int __sched __rwbase_read_lock(struct rwbase_rt *rwb, unsigned int state) { struct rt_mutex_base *rtm = &rwb->rtmutex; + DEFINE_WAKE_Q(wake_q); int ret; rwbase_pre_schedule(); @@ -110,7 +111,7 @@ static int __sched __rwbase_read_lock(struct rwbase_rt *rwb, * For rwlocks this returns 0 unconditionally, so the below * !ret conditionals are optimized out. */ - ret = rwbase_rtmutex_slowlock_locked(rtm, state); + ret = rwbase_rtmutex_slowlock_locked(rtm, state, &wake_q); /* * On success the rtmutex is held, so there can't be a writer @@ -121,7 +122,12 @@ static int __sched __rwbase_read_lock(struct rwbase_rt *rwb, */ if (!ret) atomic_inc(&rwb->readers); + + preempt_disable(); raw_spin_unlock_irq(&rtm->wait_lock); + wake_up_q(&wake_q); + preempt_enable(); + if (!ret) rwbase_rtmutex_unlock(rtm); diff --git a/kernel/locking/rwsem.c b/kernel/locking/rwsem.c index 2bbb6eca5144..2ddb827e3bea 100644 --- a/kernel/locking/rwsem.c +++ b/kernel/locking/rwsem.c @@ -1413,8 +1413,8 @@ static inline void __downgrade_write(struct rw_semaphore *sem) #define rwbase_rtmutex_lock_state(rtm, state) \ __rt_mutex_lock(rtm, state) -#define rwbase_rtmutex_slowlock_locked(rtm, state) \ - __rt_mutex_slowlock_locked(rtm, NULL, state) +#define rwbase_rtmutex_slowlock_locked(rtm, state, wq) \ + __rt_mutex_slowlock_locked(rtm, NULL, state, wq) #define rwbase_rtmutex_unlock(rtm) \ __rt_mutex_unlock(rtm) diff --git a/kernel/locking/spinlock_rt.c b/kernel/locking/spinlock_rt.c index 38e292454fcc..014143934e00 100644 --- a/kernel/locking/spinlock_rt.c +++ b/kernel/locking/spinlock_rt.c @@ -162,9 +162,10 @@ rwbase_rtmutex_lock_state(struct rt_mutex_base *rtm, unsigned int state) } static __always_inline int -rwbase_rtmutex_slowlock_locked(struct rt_mutex_base *rtm, unsigned int state) +rwbase_rtmutex_slowlock_locked(struct rt_mutex_base *rtm, unsigned int state, + struct wake_q_head *wake_q) { - rtlock_slowlock_locked(rtm); + rtlock_slowlock_locked(rtm, wake_q); return 0; } diff --git a/kernel/locking/ww_mutex.h b/kernel/locking/ww_mutex.h index 76d204b7d29c..a54bd16d0f17 100644 --- a/kernel/locking/ww_mutex.h +++ b/kernel/locking/ww_mutex.h @@ -275,7 +275,7 @@ __ww_ctx_less(struct ww_acquire_ctx *a, struct ww_acquire_ctx *b) */ static bool __ww_mutex_die(struct MUTEX *lock, struct MUTEX_WAITER *waiter, - struct ww_acquire_ctx *ww_ctx) + struct ww_acquire_ctx *ww_ctx, struct wake_q_head *wake_q) { if (!ww_ctx->is_wait_die) return false; @@ -284,7 +284,7 @@ __ww_mutex_die(struct MUTEX *lock, struct MUTEX_WAITER *waiter, #ifndef WW_RT debug_mutex_wake_waiter(lock, waiter); #endif - wake_up_process(waiter->task); + wake_q_add(wake_q, waiter->task); } return true; @@ -299,7 +299,8 @@ __ww_mutex_die(struct MUTEX *lock, struct MUTEX_WAITER *waiter, */ static bool __ww_mutex_wound(struct MUTEX *lock, struct ww_acquire_ctx *ww_ctx, - struct ww_acquire_ctx *hold_ctx) + struct ww_acquire_ctx *hold_ctx, + struct wake_q_head *wake_q) { struct task_struct *owner = __ww_mutex_owner(lock); @@ -331,7 +332,7 @@ static bool __ww_mutex_wound(struct MUTEX *lock, * wakeup pending to re-read the wounded state. */ if (owner != current) - wake_up_process(owner); + wake_q_add(wake_q, owner); return true; } @@ -352,7 +353,8 @@ static bool __ww_mutex_wound(struct MUTEX *lock, * The current task must not be on the wait list. */ static void -__ww_mutex_check_waiters(struct MUTEX *lock, struct ww_acquire_ctx *ww_ctx) +__ww_mutex_check_waiters(struct MUTEX *lock, struct ww_acquire_ctx *ww_ctx, + struct wake_q_head *wake_q) { struct MUTEX_WAITER *cur; @@ -364,8 +366,8 @@ __ww_mutex_check_waiters(struct MUTEX *lock, struct ww_acquire_ctx *ww_ctx) if (!cur->ww_ctx) continue; - if (__ww_mutex_die(lock, cur, ww_ctx) || - __ww_mutex_wound(lock, cur->ww_ctx, ww_ctx)) + if (__ww_mutex_die(lock, cur, ww_ctx, wake_q) || + __ww_mutex_wound(lock, cur->ww_ctx, ww_ctx, wake_q)) break; } } @@ -377,6 +379,8 @@ __ww_mutex_check_waiters(struct MUTEX *lock, struct ww_acquire_ctx *ww_ctx) static __always_inline void ww_mutex_set_context_fastpath(struct ww_mutex *lock, struct ww_acquire_ctx *ctx) { + DEFINE_WAKE_Q(wake_q); + ww_mutex_lock_acquired(lock, ctx); /* @@ -405,8 +409,11 @@ ww_mutex_set_context_fastpath(struct ww_mutex *lock, struct ww_acquire_ctx *ctx) * die or wound us. */ lock_wait_lock(&lock->base); - __ww_mutex_check_waiters(&lock->base, ctx); + __ww_mutex_check_waiters(&lock->base, ctx, &wake_q); + preempt_disable(); unlock_wait_lock(&lock->base); + wake_up_q(&wake_q); + preempt_enable(); } static __always_inline int @@ -488,7 +495,8 @@ __ww_mutex_check_kill(struct MUTEX *lock, struct MUTEX_WAITER *waiter, static inline int __ww_mutex_add_waiter(struct MUTEX_WAITER *waiter, struct MUTEX *lock, - struct ww_acquire_ctx *ww_ctx) + struct ww_acquire_ctx *ww_ctx, + struct wake_q_head *wake_q) { struct MUTEX_WAITER *cur, *pos = NULL; bool is_wait_die; @@ -532,7 +540,7 @@ __ww_mutex_add_waiter(struct MUTEX_WAITER *waiter, pos = cur; /* Wait-Die: ensure younger waiters die. */ - __ww_mutex_die(lock, cur, ww_ctx); + __ww_mutex_die(lock, cur, ww_ctx, wake_q); } __ww_waiter_add(lock, waiter, pos); @@ -550,7 +558,7 @@ __ww_mutex_add_waiter(struct MUTEX_WAITER *waiter, * such that either we or the fastpath will wound @ww->ctx. */ smp_mb(); - __ww_mutex_wound(lock, ww_ctx, ww->ctx); + __ww_mutex_wound(lock, ww_ctx, ww->ctx, wake_q); } return 0; diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 1b1ddd24cb22..49dbb52842c9 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5226,6 +5226,12 @@ static struct rq *finish_task_switch(struct task_struct *prev) */ kmap_local_sched_in(); + /* + * Any cached block-layer timestamp (plug->cur_ktime) is stale now, + * invalidate it. + */ + blk_plug_invalidate_ts(); + fire_sched_in_preempt_notifiers(current); /* * When switching through a kernel thread, the loop in @@ -6785,12 +6791,10 @@ static inline void sched_submit_work(struct task_struct *tsk) static void sched_update_worker(struct task_struct *tsk) { - if (tsk->flags & (PF_WQ_WORKER | PF_IO_WORKER | PF_BLOCK_TS)) { - if (tsk->flags & PF_BLOCK_TS) - blk_plug_invalidate_ts(tsk); + if (tsk->flags & (PF_WQ_WORKER | PF_IO_WORKER)) { if (tsk->flags & PF_WQ_WORKER) wq_worker_running(tsk); - else if (tsk->flags & PF_IO_WORKER) + else io_wq_worker_running(tsk); } } diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 4a44451efbcc..41c874fbd6fa 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -2943,6 +2943,10 @@ int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr if (!is_kprobe_multi(prog)) return -EINVAL; + /* kprobe_multi is not allowed to be sleepable. */ + if (prog->sleepable) + return -EINVAL; + flags = attr->link_create.kprobe_multi.flags; if (flags & ~BPF_F_KPROBE_MULTI_RETURN) return -EINVAL; diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index c8a6c0bb907e..4a486f7457e2 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -1122,7 +1122,6 @@ struct ftrace_page { }; #define ENTRY_SIZE sizeof(struct dyn_ftrace) -#define ENTRIES_PER_PAGE (PAGE_SIZE / ENTRY_SIZE) static struct ftrace_page *ftrace_pages_start; static struct ftrace_page *ftrace_pages; @@ -3754,7 +3753,8 @@ static int ftrace_update_code(struct module *mod, struct ftrace_page *new_pgs) return 0; } -static int ftrace_allocate_records(struct ftrace_page *pg, int count) +static int ftrace_allocate_records(struct ftrace_page *pg, int count, + unsigned long *num_pages) { int order; int pages; @@ -3764,7 +3764,7 @@ static int ftrace_allocate_records(struct ftrace_page *pg, int count) return -EINVAL; /* We want to fill as much as possible, with no empty pages */ - pages = DIV_ROUND_UP(count, ENTRIES_PER_PAGE); + pages = DIV_ROUND_UP(count * ENTRY_SIZE, PAGE_SIZE); order = fls(pages) - 1; again: @@ -3779,6 +3779,7 @@ static int ftrace_allocate_records(struct ftrace_page *pg, int count) } ftrace_number_of_pages += 1 << order; + *num_pages += 1 << order; ftrace_number_of_groups++; cnt = (PAGE_SIZE << order) / ENTRY_SIZE; @@ -3807,12 +3808,14 @@ static void ftrace_free_pages(struct ftrace_page *pages) } static struct ftrace_page * -ftrace_allocate_pages(unsigned long num_to_init) +ftrace_allocate_pages(unsigned long num_to_init, unsigned long *num_pages) { struct ftrace_page *start_pg; struct ftrace_page *pg; int cnt; + *num_pages = 0; + if (!num_to_init) return NULL; @@ -3826,7 +3829,7 @@ ftrace_allocate_pages(unsigned long num_to_init) * waste as little space as possible. */ for (;;) { - cnt = ftrace_allocate_records(pg, num_to_init); + cnt = ftrace_allocate_records(pg, num_to_init, num_pages); if (cnt < 0) goto free_pages; @@ -7049,6 +7052,7 @@ static int ftrace_process_locs(struct module *mod, unsigned long *p; unsigned long addr; unsigned long flags = 0; /* Shut up gcc */ + unsigned long pages; int ret = -ENOMEM; count = end - start; @@ -7068,7 +7072,7 @@ static int ftrace_process_locs(struct module *mod, test_is_sorted(start, count); } - start_pg = ftrace_allocate_pages(count); + start_pg = ftrace_allocate_pages(count, &pages); if (!start_pg) return -ENOMEM; @@ -7100,7 +7104,9 @@ static int ftrace_process_locs(struct module *mod, pg = start_pg; while (p < end) { unsigned long end_offset; - addr = ftrace_call_adjust(*p++); + + addr = *p++; + /* * Some architecture linkers will pad between * the different mcount_loc sections of different @@ -7112,6 +7118,19 @@ static int ftrace_process_locs(struct module *mod, continue; } + /* + * If this is core kernel, make sure the address is in core + * or inittext, as weak functions get zeroed and KASLR can + * move them to something other than zero. It just will not + * move it to an area where kernel text is. + */ + if (!mod && !(is_kernel_text(addr) || is_kernel_inittext(addr))) { + skipped++; + continue; + } + + addr = ftrace_call_adjust(addr); + end_offset = (pg->index+1) * sizeof(pg->records[0]); if (end_offset > PAGE_SIZE << pg->order) { /* We should have allocated enough */ @@ -7151,11 +7170,41 @@ static int ftrace_process_locs(struct module *mod, /* We should have used all pages unless we skipped some */ if (pg_unuse) { - WARN_ON(!skipped); + unsigned long pg_remaining, remaining = 0; + long skip; + + /* Count the number of entries unused and compare it to skipped. */ + pg_remaining = (PAGE_SIZE << pg->order) / ENTRY_SIZE - pg->index; + + if (!WARN(skipped < pg_remaining, "Extra allocated pages for ftrace")) { + + skip = skipped - pg_remaining; + + for (pg = pg_unuse; pg && skip > 0; pg = pg->next) { + remaining += 1 << pg->order; + skip -= (PAGE_SIZE << pg->order) / ENTRY_SIZE; + } + + pages -= remaining; + + /* + * Check to see if the number of pages remaining would + * just fit the number of entries skipped. + */ + WARN(pg || skip > 0, "Extra allocated pages for ftrace: %lu with %lu skipped", + remaining, skipped); + } /* Need to synchronize with ftrace_location_range() */ synchronize_rcu(); ftrace_free_pages(pg_unuse); } + + if (!mod) { + count -= skipped; + pr_info("ftrace: allocating %ld entries in %ld pages\n", + count, pages); + } + return ret; } @@ -7810,9 +7859,6 @@ void __init ftrace_init(void) goto failed; } - pr_info("ftrace: allocating %ld entries in %ld pages\n", - count, DIV_ROUND_UP(count, ENTRIES_PER_PAGE)); - ret = ftrace_process_locs(NULL, __start_mcount_loc, __stop_mcount_loc); diff --git a/lib/debugobjects.c b/lib/debugobjects.c index 932e2d8dbd9b..7abd909c8076 100644 --- a/lib/debugobjects.c +++ b/lib/debugobjects.c @@ -596,21 +596,59 @@ static struct debug_obj *lookup_object_or_alloc(void *addr, struct debug_bucket return NULL; } -static void debug_objects_fill_pool(void) +static inline bool debug_objects_is_pi_blocked_on(void) +{ +#ifdef CONFIG_RT_MUTEXES + return current->pi_blocked_on != NULL; +#else + return false; +#endif +} + +static inline bool can_fill_pool(void) { /* - * On RT enabled kernels the pool refill must happen in preemptible - * context -- for !RT kernels we rely on the fact that spinlock_t and - * raw_spinlock_t are basically the same type and this lock-type - * inversion works just fine. + * On !RT enabled kernels there are no restrictions and spinlock_t and + * raw_spinlock_t are the same types. + */ + if (!IS_ENABLED(CONFIG_PREEMPT_RT)) + return true; + + /* + * On RT enabled kernels, the task must not be blocked on a lock as + * that could corrupt the PI state when blocking on a lock in the + * allocation path. */ - if (!IS_ENABLED(CONFIG_PREEMPT_RT) || preemptible()) { + if (debug_objects_is_pi_blocked_on()) + return false; + + /* + * On RT enabled kernels the pool refill should happen in preemptible + * context. + */ + if (preemptible()) + return true; + + /* + * Though during system boot before scheduling is set up, preemption is + * disabled and the pool can get exhausted. Before scheduling is active + * a task cannot be blocked on a sleeping lock, but it might hold a lock + * and if interrupted then hard interrupt context might run into a lock + * inversion. So exclude hard interrupt context from allocations before + * scheduling is active. + */ + return system_state < SYSTEM_SCHEDULING && !in_hardirq(); +} + +static void debug_objects_fill_pool(void) +{ + if (can_fill_pool()) { /* * Annotate away the spinlock_t inside raw_spinlock_t warning - * by temporarily raising the wait-type to WAIT_SLEEP, matching - * the preemptible() condition above. + * by temporarily raising the wait-type to LD_WAIT_CONFIG, matching + * the preemptible() condition in can_fill_pool(). */ - static DEFINE_WAIT_OVERRIDE_MAP(fill_pool_map, LD_WAIT_SLEEP); + static DEFINE_WAIT_OVERRIDE_MAP(fill_pool_map, LD_WAIT_CONFIG); lock_map_acquire_try(&fill_pool_map); fill_pool(); lock_map_release(&fill_pool_map); diff --git a/net/9p/client.c b/net/9p/client.c index 52a5497cfca7..2f6baeb47500 100644 --- a/net/9p/client.c +++ b/net/9p/client.c @@ -1215,7 +1215,8 @@ struct p9_fid *p9_client_walk(struct p9_fid *oldfid, uint16_t nwname, clunk_fid: kfree(wqids); - p9_fid_put(fid); + if (fid != oldfid) + p9_fid_put(fid); fid = NULL; error: diff --git a/net/batman-adv/bat_iv_ogm.c b/net/batman-adv/bat_iv_ogm.c index b37c9fb178ae..5dd3e1f281ba 100644 --- a/net/batman-adv/bat_iv_ogm.c +++ b/net/batman-adv/bat_iv_ogm.c @@ -310,14 +310,23 @@ batadv_iv_ogm_aggr_packet(int buff_pos, int packet_len, const struct batadv_ogm_packet *ogm_packet) { int next_buff_pos = 0; + u16 tvlv_len; /* check if there is enough space for the header */ next_buff_pos += buff_pos + sizeof(*ogm_packet); if (next_buff_pos > packet_len) return false; + tvlv_len = ntohs(ogm_packet->tvlv_len); + + /* the fields of an aggregated OGM are accessed assuming (at least) + * 2-byte alignment, so a following OGM must start at an even offset. + */ + if (tvlv_len & 1) + return false; + /* check if there is enough space for the optional TVLV */ - next_buff_pos += ntohs(ogm_packet->tvlv_len); + next_buff_pos += tvlv_len; return next_buff_pos <= packet_len; } diff --git a/net/batman-adv/bat_v.c b/net/batman-adv/bat_v.c index d35479c465e2..e13b39121f2d 100644 --- a/net/batman-adv/bat_v.c +++ b/net/batman-adv/bat_v.c @@ -819,6 +819,7 @@ void batadv_v_hardif_init(struct batadv_hard_iface *hard_iface) hard_iface->bat_v.aggr_len = 0; skb_queue_head_init(&hard_iface->bat_v.aggr_list); + hard_iface->bat_v.aggr_list_enabled = false; INIT_DELAYED_WORK(&hard_iface->bat_v.aggr_wq, batadv_v_ogm_aggr_work); } diff --git a/net/batman-adv/bat_v_ogm.c b/net/batman-adv/bat_v_ogm.c index 8cfc3944dcfd..c5c4d33cb198 100644 --- a/net/batman-adv/bat_v_ogm.c +++ b/net/batman-adv/bat_v_ogm.c @@ -254,11 +254,18 @@ static void batadv_v_ogm_queue_on_if(struct batadv_priv *bat_priv, } spin_lock_bh(&hard_iface->bat_v.aggr_list.lock); + if (!hard_iface->bat_v.aggr_list_enabled) { + kfree_skb(skb); + goto unlock; + } + if (!batadv_v_ogm_queue_left(skb, hard_iface)) batadv_v_ogm_aggr_send(bat_priv, hard_iface); hard_iface->bat_v.aggr_len += batadv_v_ogm_len(skb); __skb_queue_tail(&hard_iface->bat_v.aggr_list, skb); + +unlock: spin_unlock_bh(&hard_iface->bat_v.aggr_list.lock); } @@ -421,6 +428,10 @@ int batadv_v_ogm_iface_enable(struct batadv_hard_iface *hard_iface) { struct batadv_priv *bat_priv = netdev_priv(hard_iface->soft_iface); + spin_lock_bh(&hard_iface->bat_v.aggr_list.lock); + hard_iface->bat_v.aggr_list_enabled = true; + spin_unlock_bh(&hard_iface->bat_v.aggr_list.lock); + batadv_v_ogm_start_queue_timer(hard_iface); batadv_v_ogm_start_timer(bat_priv); @@ -436,6 +447,7 @@ void batadv_v_ogm_iface_disable(struct batadv_hard_iface *hard_iface) cancel_delayed_work_sync(&hard_iface->bat_v.aggr_wq); spin_lock_bh(&hard_iface->bat_v.aggr_list.lock); + hard_iface->bat_v.aggr_list_enabled = false; batadv_v_ogm_aggr_list_free(hard_iface); spin_unlock_bh(&hard_iface->bat_v.aggr_list.lock); } @@ -841,14 +853,23 @@ batadv_v_ogm_aggr_packet(int buff_pos, int packet_len, const struct batadv_ogm2_packet *ogm2_packet) { int next_buff_pos = 0; + u16 tvlv_len; /* check if there is enough space for the header */ next_buff_pos += buff_pos + sizeof(*ogm2_packet); if (next_buff_pos > packet_len) return false; + tvlv_len = ntohs(ogm2_packet->tvlv_len); + + /* the fields of an aggregated OGMv2 are accessed assuming (at least) + * 2-byte alignment, so a following OGMv2 must start at an even offset. + */ + if (tvlv_len & 1) + return false; + /* check if there is enough space for the optional TVLV */ - next_buff_pos += ntohs(ogm2_packet->tvlv_len); + next_buff_pos += tvlv_len; return next_buff_pos <= packet_len; } diff --git a/net/batman-adv/bridge_loop_avoidance.c b/net/batman-adv/bridge_loop_avoidance.c index 15aeb07285e6..cc6c14cac062 100644 --- a/net/batman-adv/bridge_loop_avoidance.c +++ b/net/batman-adv/bridge_loop_avoidance.c @@ -512,7 +512,7 @@ batadv_bla_get_backbone_gw(struct batadv_priv *bat_priv, const u8 *orig, return NULL; entry->vid = vid; - entry->lasttime = jiffies; + WRITE_ONCE(entry->lasttime, jiffies); entry->crc = BATADV_BLA_CRC_INIT; entry->bat_priv = bat_priv; spin_lock_init(&entry->crc_lock); @@ -580,7 +580,7 @@ batadv_bla_update_own_backbone_gw(struct batadv_priv *bat_priv, if (unlikely(!backbone_gw)) return; - backbone_gw->lasttime = jiffies; + WRITE_ONCE(backbone_gw->lasttime, jiffies); batadv_backbone_gw_put(backbone_gw); } @@ -714,7 +714,7 @@ static void batadv_bla_add_claim(struct batadv_priv *bat_priv, ether_addr_copy(claim->addr, mac); spin_lock_init(&claim->backbone_lock); claim->vid = vid; - claim->lasttime = jiffies; + WRITE_ONCE(claim->lasttime, jiffies); kref_get(&backbone_gw->refcount); claim->backbone_gw = backbone_gw; kref_init(&claim->refcount); @@ -736,7 +736,7 @@ static void batadv_bla_add_claim(struct batadv_priv *bat_priv, return; } } else { - claim->lasttime = jiffies; + WRITE_ONCE(claim->lasttime, jiffies); if (claim->backbone_gw == backbone_gw) /* no need to register a new backbone */ goto claim_free_ref; @@ -769,7 +769,7 @@ static void batadv_bla_add_claim(struct batadv_priv *bat_priv, spin_lock_bh(&backbone_gw->crc_lock); backbone_gw->crc ^= crc16(0, claim->addr, ETH_ALEN); spin_unlock_bh(&backbone_gw->crc_lock); - backbone_gw->lasttime = jiffies; + WRITE_ONCE(backbone_gw->lasttime, jiffies); claim_free_ref: batadv_claim_put(claim); @@ -858,7 +858,7 @@ static bool batadv_handle_announce(struct batadv_priv *bat_priv, u8 *an_addr, return true; /* handle as ANNOUNCE frame */ - backbone_gw->lasttime = jiffies; + WRITE_ONCE(backbone_gw->lasttime, jiffies); crc = ntohs(*((__force __be16 *)(&an_addr[4]))); batadv_dbg(BATADV_DBG_BLA, bat_priv, @@ -1253,7 +1253,7 @@ static void batadv_bla_purge_backbone_gw(struct batadv_priv *bat_priv, int now) head, hash_entry) { if (now) goto purge_now; - if (!batadv_has_timed_out(backbone_gw->lasttime, + if (!batadv_has_timed_out(READ_ONCE(backbone_gw->lasttime), BATADV_BLA_BACKBONE_TIMEOUT)) continue; @@ -1334,7 +1334,7 @@ static void batadv_bla_purge_claims(struct batadv_priv *bat_priv, primary_if->net_dev->dev_addr)) goto skip; - if (!batadv_has_timed_out(claim->lasttime, + if (!batadv_has_timed_out(READ_ONCE(claim->lasttime), BATADV_BLA_CLAIM_TIMEOUT)) goto skip; @@ -1494,7 +1494,7 @@ static void batadv_bla_periodic_work(struct work_struct *work) eth_random_addr(bat_priv->bla.loopdetect_addr); bat_priv->bla.loopdetect_addr[0] = 0xba; bat_priv->bla.loopdetect_addr[1] = 0xbe; - bat_priv->bla.loopdetect_lasttime = jiffies; + WRITE_ONCE(bat_priv->bla.loopdetect_lasttime, jiffies); atomic_set(&bat_priv->bla.loopdetect_next, BATADV_BLA_LOOPDETECT_PERIODS); @@ -1515,7 +1515,7 @@ static void batadv_bla_periodic_work(struct work_struct *work) primary_if->net_dev->dev_addr)) continue; - backbone_gw->lasttime = jiffies; + WRITE_ONCE(backbone_gw->lasttime, jiffies); batadv_bla_send_announce(bat_priv, backbone_gw); if (send_loopdetect) @@ -1900,7 +1900,7 @@ batadv_bla_loopdetect_check(struct batadv_priv *bat_priv, struct sk_buff *skb, /* If the packet came too late, don't forward it on the mesh * but don't consider that as loop. It might be a coincidence. */ - if (batadv_has_timed_out(bat_priv->bla.loopdetect_lasttime, + if (batadv_has_timed_out(READ_ONCE(bat_priv->bla.loopdetect_lasttime), BATADV_BLA_LOOPDETECT_TIMEOUT)) return true; @@ -2016,7 +2016,7 @@ bool batadv_bla_rx(struct batadv_priv *bat_priv, struct sk_buff *skb, if (own_claim) { /* ... allow it in any case */ - claim->lasttime = jiffies; + WRITE_ONCE(claim->lasttime, jiffies); goto allow; } @@ -2118,7 +2118,7 @@ bool batadv_bla_tx(struct batadv_priv *bat_priv, struct sk_buff *skb, /* if yes, the client has roamed and we have * to unclaim it. */ - if (batadv_has_timed_out(claim->lasttime, 100)) { + if (batadv_has_timed_out(READ_ONCE(claim->lasttime), 100)) { /* only unclaim if the last claim entry is * older than 100 ms to make sure we really * have a roaming client here. @@ -2372,7 +2372,7 @@ batadv_bla_backbone_dump_entry(struct sk_buff *msg, u32 portid, backbone_crc = backbone_gw->crc; spin_unlock_bh(&backbone_gw->crc_lock); - msecs = jiffies_to_msecs(jiffies - backbone_gw->lasttime); + msecs = jiffies_to_msecs(jiffies - READ_ONCE(backbone_gw->lasttime)); if (is_own) if (nla_put_flag(msg, BATADV_ATTR_BLA_OWN)) { diff --git a/net/batman-adv/distributed-arp-table.c b/net/batman-adv/distributed-arp-table.c index 4d53e7c3ea54..bd5477263ecd 100644 --- a/net/batman-adv/distributed-arp-table.c +++ b/net/batman-adv/distributed-arp-table.c @@ -215,10 +215,13 @@ static void batadv_dat_purge(struct work_struct *work) */ static bool batadv_compare_dat(const struct hlist_node *node, const void *data2) { - const void *data1 = container_of(node, struct batadv_dat_entry, - hash_entry); + const struct batadv_dat_entry *entry1; + const struct batadv_dat_entry *entry2; - return memcmp(data1, data2, sizeof(__be32)) == 0; + entry1 = container_of(node, struct batadv_dat_entry, hash_entry); + entry2 = data2; + + return entry1->ip == entry2->ip && entry1->vid == entry2->vid; } /** @@ -345,6 +348,9 @@ batadv_dat_entry_hash_find(struct batadv_priv *bat_priv, __be32 ip, if (dat_entry->ip != ip) continue; + if (dat_entry->vid != vid) + continue; + if (!kref_get_unless_zero(&dat_entry->refcount)) continue; diff --git a/net/batman-adv/fragmentation.c b/net/batman-adv/fragmentation.c index fd7cb789ae9a..375c94991213 100644 --- a/net/batman-adv/fragmentation.c +++ b/net/batman-adv/fragmentation.c @@ -384,6 +384,8 @@ bool batadv_frag_skb_buffer(struct sk_buff **skb, * @skb: skb to forward * @recv_if: interface that the skb is received on * @orig_node_src: originator that the skb is received from + * @rx_result: set to NET_RX_SUCCESS when the fragment was forwarded and + * NET_RX_DROP when it was dropped; only valid when true is returned * * Look up the next-hop of the fragments payload and check if the merged packet * will exceed the MTU towards the next-hop. If so, the fragment is forwarded @@ -393,7 +395,8 @@ bool batadv_frag_skb_buffer(struct sk_buff **skb, */ bool batadv_frag_skb_fwd(struct sk_buff *skb, struct batadv_hard_iface *recv_if, - struct batadv_orig_node *orig_node_src) + struct batadv_orig_node *orig_node_src, + int *rx_result) { struct batadv_priv *bat_priv = netdev_priv(recv_if->soft_iface); struct batadv_neigh_node *neigh_node = NULL; @@ -412,12 +415,29 @@ bool batadv_frag_skb_fwd(struct sk_buff *skb, */ total_size = ntohs(packet->total_size); if (total_size > neigh_node->if_incoming->net_dev->mtu) { + if (packet->ttl < 2) { + kfree_skb(skb); + *rx_result = NET_RX_DROP; + ret = true; + goto out; + } + + if (skb_cow(skb, ETH_HLEN) < 0) { + kfree_skb(skb); + *rx_result = NET_RX_DROP; + ret = true; + goto out; + } + + packet = (struct batadv_frag_packet *)skb->data; + batadv_inc_counter(bat_priv, BATADV_CNT_FRAG_FWD); batadv_add_counter(bat_priv, BATADV_CNT_FRAG_FWD_BYTES, skb->len + ETH_HLEN); packet->ttl--; batadv_send_unicast_skb(skb, neigh_node); + *rx_result = NET_RX_SUCCESS; ret = true; } diff --git a/net/batman-adv/fragmentation.h b/net/batman-adv/fragmentation.h index dbf0871f8703..51e281027ab6 100644 --- a/net/batman-adv/fragmentation.h +++ b/net/batman-adv/fragmentation.h @@ -19,7 +19,8 @@ void batadv_frag_purge_orig(struct batadv_orig_node *orig, bool (*check_cb)(struct batadv_frag_table_entry *)); bool batadv_frag_skb_fwd(struct sk_buff *skb, struct batadv_hard_iface *recv_if, - struct batadv_orig_node *orig_node_src); + struct batadv_orig_node *orig_node_src, + int *rx_result); bool batadv_frag_skb_buffer(struct sk_buff **skb, struct batadv_orig_node *orig_node); int batadv_frag_send_packet(struct sk_buff *skb, diff --git a/net/batman-adv/netlink.c b/net/batman-adv/netlink.c index 9362cd9d6f3d..a7522015224a 100644 --- a/net/batman-adv/netlink.c +++ b/net/batman-adv/netlink.c @@ -936,9 +936,15 @@ static int batadv_netlink_set_hardif(struct sk_buff *skb, #ifdef CONFIG_BATMAN_ADV_BATMAN_V if (info->attrs[BATADV_ATTR_ELP_INTERVAL]) { + u32 elp_interval; + attr = info->attrs[BATADV_ATTR_ELP_INTERVAL]; + elp_interval = nla_get_u32(attr); + + elp_interval = min_t(u32, elp_interval, INT_MAX); + elp_interval = max_t(u32, elp_interval, BATADV_JITTER); - atomic_set(&hard_iface->bat_v.elp_interval, nla_get_u32(attr)); + atomic_set(&hard_iface->bat_v.elp_interval, elp_interval); } if (info->attrs[BATADV_ATTR_THROUGHPUT_OVERRIDE]) { diff --git a/net/batman-adv/routing.c b/net/batman-adv/routing.c index f1061985149f..f828d8143f75 100644 --- a/net/batman-adv/routing.c +++ b/net/batman-adv/routing.c @@ -8,6 +8,7 @@ #include "main.h" #include +#include #include #include #include @@ -205,6 +206,59 @@ bool batadv_check_management_packet(struct sk_buff *skb, return true; } +/** + * batadv_skb_decrement_ttl() - decrement ttl in a batman-adv header, csum-safe + * @skb: the received packet with @skb->data pointing to the batman-adv header + * + * Supports the following packet types, all of which carry the TTL at offset 2: + * + * - batadv_ogm_packet + * - batadv_ogm2_packet + * - batadv_icmp_header + * - batadv_icmp_packet + * - batadv_icmp_tp_packet + * - batadv_icmp_packet_rr + * - batadv_unicast_packet + * - batadv_frag_packet + * - batadv_bcast_packet + * - batadv_mcast_packet + * - batadv_coded_packet + * - batadv_unicast_tvlv_packet + * + * Return: true if the packet may be forwarded (ttl decremented), + * false if it must be dropped (ttl would expire) + */ +static bool batadv_skb_decrement_ttl(struct sk_buff *skb) +{ + static const size_t ttl_offset = 2; + u8 *ttl_pos; + + BUILD_BUG_ON(offsetof(struct batadv_ogm_packet, ttl) != ttl_offset); + BUILD_BUG_ON(offsetof(struct batadv_ogm2_packet, ttl) != ttl_offset); + BUILD_BUG_ON(offsetof(struct batadv_icmp_header, ttl) != ttl_offset); + BUILD_BUG_ON(offsetof(struct batadv_icmp_packet, ttl) != ttl_offset); + BUILD_BUG_ON(offsetof(struct batadv_icmp_tp_packet, ttl) != ttl_offset); + BUILD_BUG_ON(offsetof(struct batadv_icmp_packet_rr, ttl) != ttl_offset); + BUILD_BUG_ON(offsetof(struct batadv_unicast_packet, ttl) != ttl_offset); + BUILD_BUG_ON(offsetof(struct batadv_frag_packet, ttl) != ttl_offset); + BUILD_BUG_ON(offsetof(struct batadv_bcast_packet, ttl) != ttl_offset); + BUILD_BUG_ON(offsetof(struct batadv_mcast_packet, ttl) != ttl_offset); + BUILD_BUG_ON(offsetof(struct batadv_coded_packet, ttl) != ttl_offset); + BUILD_BUG_ON(offsetof(struct batadv_unicast_tvlv_packet, ttl) != ttl_offset); + + ttl_pos = skb->data + ttl_offset; + + /* would expire on this hop -> drop, leave header + csum untouched */ + if (*ttl_pos < 2) + return false; + + skb_postpull_rcsum(skb, ttl_pos, 1); + (*ttl_pos)--; + skb_postpush_rcsum(skb, ttl_pos, 1); + + return true; +} + /** * batadv_recv_my_icmp_packet() - receive an icmp packet locally * @bat_priv: the bat priv with all the soft interface information @@ -1121,10 +1175,9 @@ int batadv_recv_frag_packet(struct sk_buff *skb, /* Route the fragment if it is not for us and too big to be merged. */ if (!batadv_is_my_mac(bat_priv, frag_packet->dest) && - batadv_frag_skb_fwd(skb, recv_if, orig_node_src)) { + batadv_frag_skb_fwd(skb, recv_if, orig_node_src, &ret)) { /* skb was consumed */ skb = NULL; - ret = NET_RX_SUCCESS; goto put_orig_node; } @@ -1198,7 +1251,13 @@ int batadv_recv_bcast_packet(struct sk_buff *skb, if (batadv_is_my_mac(bat_priv, bcast_packet->orig)) goto free_skb; - if (bcast_packet->ttl-- < 2) + /* create a copy of the skb, if needed, to modify it. */ + if (skb_cow(skb, ETH_HLEN) < 0) + goto free_skb; + + bcast_packet = (struct batadv_bcast_packet *)skb->data; + + if (!batadv_skb_decrement_ttl(skb)) goto free_skb; orig_node = batadv_orig_hash_find(bat_priv, bcast_packet->orig); @@ -1305,7 +1364,7 @@ int batadv_recv_mcast_packet(struct sk_buff *skb, goto free_skb; mcast_packet = (struct batadv_mcast_packet *)skb->data; - if (mcast_packet->ttl-- < 2) + if (!batadv_skb_decrement_ttl(skb)) goto free_skb; tvlv_buff = (unsigned char *)(skb->data + hdr_size); @@ -1314,6 +1373,12 @@ int batadv_recv_mcast_packet(struct sk_buff *skb, if (tvlv_buff_len > skb->len - hdr_size) goto free_skb; + /* the fields of an multicast payload are accessed assuming (at least) + * 2-byte alignment, so a following packet must start at an even offset. + */ + if (tvlv_buff_len & 1) + goto free_skb; + ret = batadv_tvlv_containers_process(bat_priv, BATADV_MCAST, NULL, skb, tvlv_buff, tvlv_buff_len); if (ret >= 0) { diff --git a/net/batman-adv/tp_meter.c b/net/batman-adv/tp_meter.c index dfc337454992..50b83e0438a0 100644 --- a/net/batman-adv/tp_meter.c +++ b/net/batman-adv/tp_meter.c @@ -87,6 +87,11 @@ #define BATADV_TP_PLEN (BATADV_TP_PACKET_LEN - ETH_HLEN - \ sizeof(struct batadv_unicast_packet)) +/** + * BATADV_TP_MAX_UNACKED - maximum number of packets a receiver didn't yet ack + */ +#define BATADV_TP_MAX_UNACKED 100 + static u8 batadv_tp_prerandom[4096] __read_mostly; /** @@ -154,9 +159,12 @@ static void batadv_tp_update_cwnd(struct batadv_tp_vars *tp_vars, u32 mss) return; } + /* prevent overflow in (mss * mss) << 3 */ + mss = min_t(u32, mss, (1U << 14) - 1); + /* increment CWND at least of 1 (section 3.1 of RFC5681) */ tp_vars->dec_cwnd += max_t(u32, 1U << 3, - ((mss * mss) << 6) / (tp_vars->cwnd << 3)); + ((mss * mss) << 3) / tp_vars->cwnd); if (tp_vars->dec_cwnd < (mss << 3)) { spin_unlock_bh(&tp_vars->cwnd_lock); return; @@ -730,7 +738,7 @@ static void batadv_tp_recv_ack(struct batadv_priv *bat_priv, if (atomic_read(&tp_vars->dup_acks) != 3) goto out; - if (recv_ack >= tp_vars->recover) + if (!batadv_seq_before(tp_vars->recover, recv_ack)) goto out; /* if this is the third duplicate ACK do Fast Retransmit */ @@ -817,10 +825,15 @@ static void batadv_tp_recv_ack(struct batadv_priv *bat_priv, static bool batadv_tp_avail(struct batadv_tp_vars *tp_vars, size_t payload_len) { + u32 last_sent = READ_ONCE(tp_vars->last_sent); u32 win_left, win_limit; win_limit = atomic_read(&tp_vars->last_acked) + tp_vars->cwnd; - win_left = win_limit - tp_vars->last_sent; + + if (batadv_seq_before(last_sent, win_limit)) + win_left = win_limit - last_sent; + else + win_left = 0; return win_left >= payload_len; } @@ -1045,6 +1058,7 @@ void batadv_tp_start(struct batadv_priv *bat_priv, const u8 *dst, tp_vars->icmp_uid = icmp_uid; tp_vars->last_sent = BATADV_TP_FIRST_SEQ; + atomic_set(&tp_vars->dup_acks, 0); atomic_set(&tp_vars->last_acked, BATADV_TP_FIRST_SEQ); tp_vars->fast_recovery = false; tp_vars->recover = BATADV_TP_FIRST_SEQ; @@ -1054,6 +1068,8 @@ void batadv_tp_start(struct batadv_priv *bat_priv, const u8 *dst, * soft_interface, hence its MTU */ tp_vars->cwnd = BATADV_TP_PLEN * 3; + tp_vars->dec_cwnd = 0; + /* at the beginning initialise the SS threshold to the biggest possible * window size, hence the AWND size */ @@ -1085,21 +1101,21 @@ void batadv_tp_start(struct batadv_priv *bat_priv, const u8 *dst, tp_vars->prerandom_offset = 0; spin_lock_init(&tp_vars->prerandom_lock); - kref_get(&tp_vars->refcount); - hlist_add_head_rcu(&tp_vars->list, &bat_priv->tp_list); - spin_unlock_bh(&bat_priv->tp_list_lock); - tp_vars->test_length = test_length; if (!tp_vars->test_length) tp_vars->test_length = BATADV_TP_DEF_TEST_LENGTH; + /* init work item for finished tp tests */ + INIT_DELAYED_WORK(&tp_vars->finish_work, batadv_tp_sender_finish); + + kref_get(&tp_vars->refcount); + hlist_add_head_rcu(&tp_vars->list, &bat_priv->tp_list); + spin_unlock_bh(&bat_priv->tp_list_lock); + batadv_dbg(BATADV_DBG_TP_METER, bat_priv, "Meter: starting throughput meter towards %pM (length=%ums)\n", dst, test_length); - /* init work item for finished tp tests */ - INIT_DELAYED_WORK(&tp_vars->finish_work, batadv_tp_sender_finish); - /* start tp kthread. This way the write() call issued from userspace can * happily return and avoid to block */ @@ -1167,7 +1183,7 @@ static void batadv_tp_receiver_shutdown(struct timer_list *t) bat_priv = tp_vars->bat_priv; /* if there is recent activity rearm the timer */ - if (!batadv_has_timed_out(tp_vars->last_recv_time, + if (!batadv_has_timed_out(READ_ONCE(tp_vars->last_recv_time), BATADV_TP_RECV_TIMEOUT)) { /* reset the receiver shutdown timer */ batadv_tp_reset_receiver_timer(tp_vars); @@ -1184,6 +1200,7 @@ static void batadv_tp_receiver_shutdown(struct timer_list *t) list_for_each_entry_safe(un, safe, &tp_vars->unacked_list, list) { list_del(&un->list); kfree(un); + tp_vars->unacked_count--; } spin_unlock_bh(&tp_vars->unacked_lock); @@ -1267,7 +1284,8 @@ static int batadv_tp_send_ack(struct batadv_priv *bat_priv, const u8 *dst, /** * batadv_tp_handle_out_of_order() - store an out of order packet * @tp_vars: the private data of the current TP meter session - * @skb: the buffer containing the received packet + * @seqno: sequence number of new received packet + * @payload_len: length of the received packet * * Store the out of order packet in the unacked list for late processing. This * packets are kept in this list so that they can be ACKed at once as soon as @@ -1276,28 +1294,24 @@ static int batadv_tp_send_ack(struct batadv_priv *bat_priv, const u8 *dst, * Return: true if the packed has been successfully processed, false otherwise */ static bool batadv_tp_handle_out_of_order(struct batadv_tp_vars *tp_vars, - const struct sk_buff *skb) + u32 seqno, u32 payload_len) + __must_hold(&tp_vars->unacked_lock) { - const struct batadv_icmp_tp_packet *icmp; struct batadv_tp_unacked *un, *new; - u32 payload_len; bool added = false; new = kmalloc(sizeof(*new), GFP_ATOMIC); if (unlikely(!new)) return false; - icmp = (struct batadv_icmp_tp_packet *)skb->data; - - new->seqno = ntohl(icmp->seqno); - payload_len = skb->len - sizeof(struct batadv_unicast_packet); + new->seqno = seqno; new->len = payload_len; - spin_lock_bh(&tp_vars->unacked_lock); /* if the list is empty immediately attach this new object */ if (list_empty(&tp_vars->unacked_list)) { list_add(&new->list, &tp_vars->unacked_list); - goto out; + tp_vars->unacked_count++; + return true; } /* otherwise loop over the list and either drop the packet because this @@ -1325,17 +1339,26 @@ static bool batadv_tp_handle_out_of_order(struct batadv_tp_vars *tp_vars, * one is attached _after_ it. In this way the list is kept in * ascending order */ - list_add_tail(&new->list, &un->list); + list_add(&new->list, &un->list); added = true; + tp_vars->unacked_count++; break; } /* received packet with smallest seqno out of order; add it to front */ - if (!added) + if (!added) { list_add(&new->list, &tp_vars->unacked_list); + tp_vars->unacked_count++; + } -out: - spin_unlock_bh(&tp_vars->unacked_lock); + /* remove the last (biggest) unacked seqno when list is too large */ + if (tp_vars->unacked_count > BATADV_TP_MAX_UNACKED) { + un = list_last_entry(&tp_vars->unacked_list, + struct batadv_tp_unacked, list); + list_del(&un->list); + kfree(un); + tp_vars->unacked_count--; + } return true; } @@ -1346,6 +1369,7 @@ static bool batadv_tp_handle_out_of_order(struct batadv_tp_vars *tp_vars, * @tp_vars: the private data of the current TP meter session */ static void batadv_tp_ack_unordered(struct batadv_tp_vars *tp_vars) + __must_hold(&tp_vars->unacked_lock) { struct batadv_tp_unacked *un, *safe; u32 to_ack; @@ -1353,7 +1377,6 @@ static void batadv_tp_ack_unordered(struct batadv_tp_vars *tp_vars) /* go through the unacked packet list and possibly ACK them as * well */ - spin_lock_bh(&tp_vars->unacked_lock); list_for_each_entry_safe(un, safe, &tp_vars->unacked_list, list) { /* the list is ordered, therefore it is possible to stop as soon * there is a gap between the last acked seqno and the seqno of @@ -1369,8 +1392,8 @@ static void batadv_tp_ack_unordered(struct batadv_tp_vars *tp_vars) list_del(&un->list); kfree(un); + tp_vars->unacked_count--; } - spin_unlock_bh(&tp_vars->unacked_lock); } /** @@ -1392,8 +1415,10 @@ batadv_tp_init_recv(struct batadv_priv *bat_priv, tp_vars = batadv_tp_list_find_session(bat_priv, icmp->orig, icmp->session, BATADV_TP_RECEIVER); - if (tp_vars) + if (tp_vars) { + WRITE_ONCE(tp_vars->last_recv_time, jiffies); goto out_unlock; + } if (!atomic_add_unless(&bat_priv->tp_num, 1, BATADV_TP_MAX_NUM)) { batadv_dbg(BATADV_DBG_TP_METER, bat_priv, @@ -1417,12 +1442,15 @@ batadv_tp_init_recv(struct batadv_priv *bat_priv, spin_lock_init(&tp_vars->unacked_lock); INIT_LIST_HEAD(&tp_vars->unacked_list); + tp_vars->unacked_count = 0; kref_get(&tp_vars->refcount); - hlist_add_head_rcu(&tp_vars->list, &bat_priv->tp_list); + timer_setup(&tp_vars->timer, batadv_tp_receiver_shutdown, 0); + + WRITE_ONCE(tp_vars->last_recv_time, jiffies); kref_get(&tp_vars->refcount); - timer_setup(&tp_vars->timer, batadv_tp_receiver_shutdown, 0); + hlist_add_head_rcu(&tp_vars->list, &bat_priv->tp_list); batadv_tp_reset_receiver_timer(tp_vars); @@ -1444,7 +1472,8 @@ static void batadv_tp_recv_msg(struct batadv_priv *bat_priv, { const struct batadv_icmp_tp_packet *icmp; struct batadv_tp_vars *tp_vars; - size_t packet_size; + u32 payload_len; + u32 to_ack; u32 seqno; icmp = (struct batadv_icmp_tp_packet *)skb->data; @@ -1469,41 +1498,49 @@ static void batadv_tp_recv_msg(struct batadv_priv *bat_priv, icmp->orig); goto out; } + + WRITE_ONCE(tp_vars->last_recv_time, jiffies); } - tp_vars->last_recv_time = jiffies; + spin_lock_bh(&tp_vars->unacked_lock); /* if the packet is a duplicate, it may be the case that an ACK has been * lost. Resend the ACK */ - if (batadv_seq_before(seqno, tp_vars->last_recv)) + payload_len = skb->len - sizeof(struct batadv_unicast_packet); + to_ack = seqno + payload_len; + if (batadv_seq_before(to_ack, tp_vars->last_recv)) goto send_ack; /* if the packet is out of order enqueue it */ - if (ntohl(icmp->seqno) != tp_vars->last_recv) { + if (batadv_seq_before(tp_vars->last_recv, seqno)) { /* exit immediately (and do not send any ACK) if the packet has * not been enqueued correctly */ - if (!batadv_tp_handle_out_of_order(tp_vars, skb)) + if (!batadv_tp_handle_out_of_order(tp_vars, seqno, payload_len)) { + spin_unlock_bh(&tp_vars->unacked_lock); goto out; + } /* send a duplicate ACK */ goto send_ack; } /* if everything was fine count the ACKed bytes */ - packet_size = skb->len - sizeof(struct batadv_unicast_packet); - tp_vars->last_recv += packet_size; + tp_vars->last_recv = to_ack; /* check if this ordered message filled a gap.... */ batadv_tp_ack_unordered(tp_vars); send_ack: + to_ack = tp_vars->last_recv; + spin_unlock_bh(&tp_vars->unacked_lock); + /* send the ACK. If the received packet was out of order, the ACK that * is going to be sent is a duplicate (the sender will count them and * possibly enter Fast Retransmit as soon as it has reached 3) */ - batadv_tp_send_ack(bat_priv, icmp->orig, tp_vars->last_recv, + batadv_tp_send_ack(bat_priv, icmp->orig, to_ack, icmp->timestamp, icmp->session, icmp->uid); out: batadv_tp_vars_put(tp_vars); diff --git a/net/batman-adv/translation-table.c b/net/batman-adv/translation-table.c index 7041cd69e200..2a7caab5c197 100644 --- a/net/batman-adv/translation-table.c +++ b/net/batman-adv/translation-table.c @@ -485,6 +485,9 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv, if (!batadv_compare_eth(entry->change.addr, common->addr)) continue; + if (entry->change.vid != tt_change_node->change.vid) + continue; + /* DEL+ADD in the same orig interval have no effect and can be * removed to avoid silly behaviour on the receiver side. The * other way around (ADD+DEL) can happen in case of roaming of @@ -3514,6 +3517,7 @@ static void batadv_tt_roam_purge(struct batadv_priv *bat_priv) * batadv_tt_check_roam_count() - check if a client has roamed too frequently * @bat_priv: the bat priv with all the soft interface information * @client: mac address of the roaming client + * @vid: VLAN identifier * * This function checks whether the client already reached the * maximum number of possible roaming phases. In this case the ROAMING_ADV @@ -3521,7 +3525,7 @@ static void batadv_tt_roam_purge(struct batadv_priv *bat_priv) * * Return: true if the ROAMING_ADV can be sent, false otherwise */ -static bool batadv_tt_check_roam_count(struct batadv_priv *bat_priv, u8 *client) +static bool batadv_tt_check_roam_count(struct batadv_priv *bat_priv, u8 *client, u16 vid) { struct batadv_tt_roam_node *tt_roam_node; bool ret = false; @@ -3534,6 +3538,9 @@ static bool batadv_tt_check_roam_count(struct batadv_priv *bat_priv, u8 *client) if (!batadv_compare_eth(tt_roam_node->addr, client)) continue; + if (tt_roam_node->vid != vid) + continue; + if (batadv_has_timed_out(tt_roam_node->first_time, BATADV_ROAMING_MAX_TIME)) continue; @@ -3555,6 +3562,7 @@ static bool batadv_tt_check_roam_count(struct batadv_priv *bat_priv, u8 *client) atomic_set(&tt_roam_node->counter, BATADV_ROAMING_MAX_COUNT - 1); ether_addr_copy(tt_roam_node->addr, client); + tt_roam_node->vid = vid; list_add(&tt_roam_node->list, &bat_priv->tt.roam_list); ret = true; @@ -3591,7 +3599,7 @@ static void batadv_send_roam_adv(struct batadv_priv *bat_priv, u8 *client, /* before going on we have to check whether the client has * already roamed to us too many times */ - if (!batadv_tt_check_roam_count(bat_priv, client)) + if (!batadv_tt_check_roam_count(bat_priv, client, vid)) goto out; batadv_dbg(BATADV_DBG_TT, bat_priv, diff --git a/net/batman-adv/tvlv.c b/net/batman-adv/tvlv.c index 8d6b017c433c..cc66e231ccb8 100644 --- a/net/batman-adv/tvlv.c +++ b/net/batman-adv/tvlv.c @@ -398,7 +398,6 @@ static int batadv_tvlv_call_handler(struct batadv_priv *bat_priv, tvlv_handler->ogm_handler(bat_priv, orig_node, BATADV_NO_FLAGS, tvlv_value, tvlv_value_len); - tvlv_handler->flags |= BATADV_TVLV_HANDLER_OGM_CALLED; break; case BATADV_UNICAST_TVLV: if (!skb) @@ -430,6 +429,48 @@ static int batadv_tvlv_call_handler(struct batadv_priv *bat_priv, return NET_RX_SUCCESS; } +/** + * batadv_tvlv_containers_contain() - check if a tvlv buffer holds a container + * @tvlv_value: tvlv content + * @tvlv_value_len: tvlv content length + * @type: tvlv container type to look for + * @version: tvlv container version to look for + * + * Return: true if a container of the given type and version is present in the + * tvlv buffer, false otherwise. + */ +static bool batadv_tvlv_containers_contain(void *tvlv_value, + u16 tvlv_value_len, u8 type, + u8 version) +{ + struct batadv_tvlv_hdr *tvlv_hdr; + u16 tvlv_value_cont_len; + + while (tvlv_value_len >= sizeof(*tvlv_hdr)) { + tvlv_hdr = tvlv_value; + tvlv_value_cont_len = ntohs(tvlv_hdr->len); + tvlv_value = tvlv_hdr + 1; + tvlv_value_len -= sizeof(*tvlv_hdr); + + if (tvlv_value_cont_len > tvlv_value_len) + break; + + /* the next tvlv header is accessed assuming (at least) 2-byte + * alignment, so it must start at an even offset. + */ + if (tvlv_value_cont_len & 1) + break; + + if (tvlv_hdr->type == type && tvlv_hdr->version == version) + return true; + + tvlv_value = (u8 *)tvlv_value + tvlv_value_cont_len; + tvlv_value_len -= tvlv_value_cont_len; + } + + return false; +} + /** * batadv_tvlv_containers_process() - parse the given tvlv buffer to call the * appropriate handlers @@ -449,7 +490,9 @@ int batadv_tvlv_containers_process(struct batadv_priv *bat_priv, struct sk_buff *skb, void *tvlv_value, u16 tvlv_value_len) { + u16 tvlv_value_start_len = tvlv_value_len; struct batadv_tvlv_handler *tvlv_handler; + void *tvlv_value_start = tvlv_value; struct batadv_tvlv_hdr *tvlv_hdr; u16 tvlv_value_cont_len; u8 cifnotfound = BATADV_TVLV_HANDLER_OGM_CIFNOTFND; @@ -464,6 +507,12 @@ int batadv_tvlv_containers_process(struct batadv_priv *bat_priv, if (tvlv_value_cont_len > tvlv_value_len) break; + /* the next tvlv header is accessed assuming (at least) 2-byte + * alignment, so it must start at an even offset. + */ + if (tvlv_value_cont_len & 1) + break; + tvlv_handler = batadv_tvlv_handler_get(bat_priv, tvlv_hdr->type, tvlv_hdr->version); @@ -487,12 +536,20 @@ int batadv_tvlv_containers_process(struct batadv_priv *bat_priv, if (!tvlv_handler->ogm_handler) continue; - if ((tvlv_handler->flags & BATADV_TVLV_HANDLER_OGM_CIFNOTFND) && - !(tvlv_handler->flags & BATADV_TVLV_HANDLER_OGM_CALLED)) - tvlv_handler->ogm_handler(bat_priv, orig_node, - cifnotfound, NULL, 0); + if (!(tvlv_handler->flags & BATADV_TVLV_HANDLER_OGM_CIFNOTFND)) + continue; + + /* if the corresponding container was present then the handler + * was already called from the loop above + */ + if (batadv_tvlv_containers_contain(tvlv_value_start, + tvlv_value_start_len, + tvlv_handler->type, + tvlv_handler->version)) + continue; - tvlv_handler->flags &= ~BATADV_TVLV_HANDLER_OGM_CALLED; + tvlv_handler->ogm_handler(bat_priv, orig_node, + cifnotfound, NULL, 0); } rcu_read_unlock(); diff --git a/net/batman-adv/types.h b/net/batman-adv/types.h index f703d266780d..0022bee14574 100644 --- a/net/batman-adv/types.h +++ b/net/batman-adv/types.h @@ -130,6 +130,12 @@ struct batadv_hard_iface_bat_v { /** @aggr_list: queue for to be aggregated OGM packets */ struct sk_buff_head aggr_list; + /** + * @aggr_list_enabled: aggr_list is active and new skbs can be + * enqueued. Protected by aggr_list.lock after initialization + */ + bool aggr_list_enabled:1; + /** @aggr_len: size of the OGM aggregate (excluding ethernet header) */ unsigned int aggr_len; @@ -1555,9 +1561,12 @@ struct batadv_tp_vars { /** @unacked_list: list of unacked packets (meta-info only) */ struct list_head unacked_list; - /** @unacked_lock: protect unacked_list */ + /** @unacked_lock: protect unacked_list + &batadv_tp_receiver.last_recv */ spinlock_t unacked_lock; + /** @unacked_count: number of unacked entries */ + size_t unacked_count; + /** @last_recv_time: time (jiffies) a msg was received */ unsigned long last_recv_time; @@ -2056,6 +2065,9 @@ struct batadv_tt_roam_node { /** @addr: mac address of the client in the roaming phase */ u8 addr[ETH_ALEN]; + /** @vid: VLAN identifier */ + u16 vid; + /** * @counter: number of allowed roaming events per client within a single * OGM interval (changes are committed with each OGM) @@ -2471,13 +2483,6 @@ enum batadv_tvlv_handler_flags { * will call this handler even if its type was not found (with no data) */ BATADV_TVLV_HANDLER_OGM_CIFNOTFND = BIT(1), - - /** - * @BATADV_TVLV_HANDLER_OGM_CALLED: interval tvlv handling flag - the - * API marks a handler as being called, so it won't be called if the - * BATADV_TVLV_HANDLER_OGM_CIFNOTFND flag was set - */ - BATADV_TVLV_HANDLER_OGM_CALLED = BIT(2), }; #endif /* _NET_BATMAN_ADV_TYPES_H_ */ diff --git a/net/core/filter.c b/net/core/filter.c index 3d71a5907253..b03621405d06 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -2740,11 +2740,13 @@ BPF_CALL_4(bpf_msg_pull_data, struct sk_msg *, msg, u32, start, poffset += len; sge->length = 0; put_page(sg_page(sge)); + __clear_bit(i, msg->sg.copy); sk_msg_iter_var_next(i); } while (i != last_sge); sg_set_page(&msg->sg.data[first_sge], page, copy, 0); + __clear_bit(first_sge, msg->sg.copy); /* To repair sg ring we need to shift entries. If we only * had a single entry though we can just replace it and @@ -2770,9 +2772,11 @@ BPF_CALL_4(bpf_msg_pull_data, struct sk_msg *, msg, u32, start, break; msg->sg.data[i] = msg->sg.data[move_from]; + sk_msg_sg_copy_assign(msg, i, msg, move_from); msg->sg.data[move_from].length = 0; msg->sg.data[move_from].page_link = 0; msg->sg.data[move_from].offset = 0; + __clear_bit(move_from, msg->sg.copy); sk_msg_iter_var_next(i); } while (1); @@ -2801,6 +2805,7 @@ BPF_CALL_4(bpf_msg_push_data, struct sk_msg *, msg, u32, start, { struct scatterlist sge, nsge, nnsge, rsge = {0}, *psge; u32 new, i = 0, l = 0, space, copy = 0, offset = 0; + bool sge_copy, nsge_copy, nnsge_copy, rsge_copy = false; u8 *raw, *to, *from; struct page *page; @@ -2873,6 +2878,7 @@ BPF_CALL_4(bpf_msg_push_data, struct sk_msg *, msg, u32, start, sk_msg_iter_var_prev(i); psge = sk_msg_elem(msg, i); rsge = sk_msg_elem_cpy(msg, i); + rsge_copy = test_bit(i, msg->sg.copy); psge->length = start - offset; rsge.length -= psge->length; @@ -2897,24 +2903,32 @@ BPF_CALL_4(bpf_msg_push_data, struct sk_msg *, msg, u32, start, /* Shift one or two slots as needed */ sge = sk_msg_elem_cpy(msg, new); + sge_copy = test_bit(new, msg->sg.copy); sg_unmark_end(&sge); nsge = sk_msg_elem_cpy(msg, i); + nsge_copy = test_bit(i, msg->sg.copy); if (rsge.length) { sk_msg_iter_var_next(i); nnsge = sk_msg_elem_cpy(msg, i); + nnsge_copy = test_bit(i, msg->sg.copy); sk_msg_iter_next(msg, end); } while (i != msg->sg.end) { msg->sg.data[i] = sge; + __assign_bit(i, msg->sg.copy, sge_copy); sge = nsge; + sge_copy = nsge_copy; sk_msg_iter_var_next(i); if (rsge.length) { nsge = nnsge; + nsge_copy = nnsge_copy; nnsge = sk_msg_elem_cpy(msg, i); + nnsge_copy = test_bit(i, msg->sg.copy); } else { nsge = sk_msg_elem_cpy(msg, i); + nsge_copy = test_bit(i, msg->sg.copy); } } @@ -2928,6 +2942,7 @@ BPF_CALL_4(bpf_msg_push_data, struct sk_msg *, msg, u32, start, get_page(sg_page(&rsge)); sk_msg_iter_var_next(new); msg->sg.data[new] = rsge; + __assign_bit(new, msg->sg.copy, rsge_copy); } sk_msg_reset_curr(msg); @@ -2955,25 +2970,33 @@ static void sk_msg_shift_left(struct sk_msg *msg, int i) prev = i; sk_msg_iter_var_next(i); msg->sg.data[prev] = msg->sg.data[i]; + sk_msg_sg_copy_assign(msg, prev, msg, i); } while (i != msg->sg.end); sk_msg_iter_prev(msg, end); + __clear_bit(msg->sg.end, msg->sg.copy); } static void sk_msg_shift_right(struct sk_msg *msg, int i) { struct scatterlist tmp, sge; + bool tmp_copy, sge_copy; sk_msg_iter_next(msg, end); sge = sk_msg_elem_cpy(msg, i); + sge_copy = test_bit(i, msg->sg.copy); sk_msg_iter_var_next(i); tmp = sk_msg_elem_cpy(msg, i); + tmp_copy = test_bit(i, msg->sg.copy); while (i != msg->sg.end) { msg->sg.data[i] = sge; + __assign_bit(i, msg->sg.copy, sge_copy); sk_msg_iter_var_next(i); sge = tmp; + sge_copy = tmp_copy; tmp = sk_msg_elem_cpy(msg, i); + tmp_copy = test_bit(i, msg->sg.copy); } } @@ -3033,6 +3056,8 @@ BPF_CALL_4(bpf_msg_pop_data, struct sk_msg *, msg, u32, start, struct scatterlist *nsge, *sge = sk_msg_elem(msg, i); int a = start - offset; int b = sge->length - pop - a; + u32 sge_i = i; + bool sge_copy = test_bit(i, msg->sg.copy); sk_msg_iter_var_next(i); @@ -3045,6 +3070,7 @@ BPF_CALL_4(bpf_msg_pop_data, struct sk_msg *, msg, u32, start, sg_set_page(nsge, sg_page(sge), b, sge->offset + pop + a); + __assign_bit(i, msg->sg.copy, sge_copy); } else { struct page *page, *orig; u8 *to, *from; @@ -3061,6 +3087,7 @@ BPF_CALL_4(bpf_msg_pop_data, struct sk_msg *, msg, u32, start, memcpy(to, from, a); memcpy(to + a, from + a + pop, b); sg_set_page(sge, page, a + b, 0); + __clear_bit(sge_i, msg->sg.copy); put_page(orig); } pop = 0; diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c index 2176cd1bc765..118612325ce9 100644 --- a/net/core/rtnetlink.c +++ b/net/core/rtnetlink.c @@ -2205,6 +2205,14 @@ struct net *rtnl_get_net_ns_capable(struct sock *sk, int netnsid) } EXPORT_SYMBOL_GPL(rtnl_get_net_ns_capable); +bool rtnl_dev_link_net_capable(const struct net_device *dev, + const struct net *link_net) +{ + return net_eq(link_net, dev_net(dev)) || + ns_capable(link_net->user_ns, CAP_NET_ADMIN); +} +EXPORT_SYMBOL_GPL(rtnl_dev_link_net_capable); + static int rtnl_valid_dump_ifinfo_req(const struct nlmsghdr *nlh, bool strict_check, struct nlattr **tb, struct netlink_ext_ack *extack) diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 4be699bd3a17..fede3aa3ddbc 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -5525,15 +5525,28 @@ static void __skb_complete_tx_timestamp(struct sk_buff *skb, static bool skb_may_tx_timestamp(struct sock *sk, bool tsonly) { - bool ret; + struct socket *sock; + struct file *file; + bool ret = false; if (likely(READ_ONCE(sysctl_tstamp_allow_data) || tsonly)) return true; - read_lock_bh(&sk->sk_callback_lock); - ret = sk->sk_socket && sk->sk_socket->file && - file_ns_capable(sk->sk_socket->file, &init_user_ns, CAP_NET_RAW); - read_unlock_bh(&sk->sk_callback_lock); + /* The sk pointer remains valid as long as the skb is. The sk_socket and + * file pointer may become NULL if the socket is closed. Both structures + * (including file->cred) are RCU freed which means they can be accessed + * within a RCU read section. + */ + rcu_read_lock(); + sock = READ_ONCE(sk->sk_socket); + if (!sock) + goto out; + file = READ_ONCE(sock->file); + if (!file) + goto out; + ret = file_ns_capable(file, &init_user_ns, CAP_NET_RAW); +out: + rcu_read_unlock(); return ret; } diff --git a/net/core/skmsg.c b/net/core/skmsg.c index fbaee3d09990..7e68b4fdbfce 100644 --- a/net/core/skmsg.c +++ b/net/core/skmsg.c @@ -66,6 +66,7 @@ int sk_msg_alloc(struct sock *sk, struct sk_msg *msg, int len, sge = &msg->sg.data[msg->sg.end]; sg_unmark_end(sge); sg_set_page(sge, pfrag->page, use, orig_offset); + __clear_bit(msg->sg.end, msg->sg.copy); get_page(pfrag->page); sk_msg_iter_next(msg, end); } @@ -186,6 +187,7 @@ static int sk_msg_free_elem(struct sock *sk, struct sk_msg *msg, u32 i, sk_mem_uncharge(sk, len); put_page(sg_page(sge)); } + __clear_bit(i, msg->sg.copy); memset(sge, 0, sizeof(*sge)); return len; } diff --git a/net/ipv4/ip_gre.c b/net/ipv4/ip_gre.c index 084556b03a2e..ffad16b023ad 100644 --- a/net/ipv4/ip_gre.c +++ b/net/ipv4/ip_gre.c @@ -1446,6 +1446,9 @@ static int ipgre_changelink(struct net_device *dev, struct nlattr *tb[], __u32 fwmark = t->fwmark; int err; + if (!rtnl_dev_link_net_capable(dev, t->net)) + return -EPERM; + err = ipgre_newlink_encap_setup(dev, data); if (err) return err; @@ -1475,6 +1478,9 @@ static int erspan_changelink(struct net_device *dev, struct nlattr *tb[], __u32 fwmark = t->fwmark; int err; + if (!rtnl_dev_link_net_capable(dev, t->net)) + return -EPERM; + err = ipgre_newlink_encap_setup(dev, data); if (err) return err; diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index ba51fc42531c..41031c5c4303 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -1118,8 +1118,8 @@ static int __ip_append_data(struct sock *sk, !(rt->dst.dev->features & NETIF_F_SG))) alloclen = fraglen; else { - alloclen = fragheaderlen + transhdrlen; - pagedlen = datalen - transhdrlen; + alloclen = fragheaderlen + transhdrlen + fraggap; + pagedlen = datalen - transhdrlen - fraggap; } alloclen += alloc_extra; @@ -1166,10 +1166,10 @@ static int __ip_append_data(struct sock *sk, } copy = datalen - transhdrlen - fraggap - pagedlen; - /* [!] NOTE: copy will be negative if pagedlen>0 - * because then the equation reduces to -fraggap. - */ - if (copy > 0 && getfrag(from, data + transhdrlen, offset, copy, fraggap, skb) < 0) { + if (copy > 0 && + INDIRECT_CALL_1(getfrag, ip_generic_getfrag, + from, data + transhdrlen, offset, + copy, fraggap, skb) < 0) { err = -EFAULT; kfree_skb(skb); goto error; @@ -1213,8 +1213,9 @@ static int __ip_append_data(struct sock *sk, unsigned int off; off = skb->len; - if (getfrag(from, skb_put(skb, copy), - offset, copy, off, skb) < 0) { + if (INDIRECT_CALL_1(getfrag, ip_generic_getfrag, + from, skb_put(skb, copy), + offset, copy, off, skb) < 0) { __skb_trim(skb, off); err = -EFAULT; goto error; @@ -1254,7 +1255,8 @@ static int __ip_append_data(struct sock *sk, get_page(pfrag->page); } copy = min_t(int, copy, pfrag->size - pfrag->offset); - if (getfrag(from, + if (INDIRECT_CALL_1(getfrag, ip_generic_getfrag, + from, page_address(pfrag->page) + pfrag->offset, offset, copy, skb->len, skb) < 0) goto error_efault; diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c index be38712265c3..a7b1fe194fb1 100644 --- a/net/ipv4/tcp_ao.c +++ b/net/ipv4/tcp_ao.c @@ -1777,6 +1777,10 @@ static int tcp_ao_delete_key(struct sock *sk, struct tcp_ao_info *ao_info, * them and we can just free all resources in RCU fashion. */ if (del_async) { + if (ao_info->current_key == key) + WRITE_ONCE(ao_info->current_key, NULL); + if (ao_info->rnext_key == key) + WRITE_ONCE(ao_info->rnext_key, NULL); atomic_sub(tcp_ao_sizeof_key(key), &sk->sk_omem_alloc); call_rcu(&key->rcu, tcp_ao_key_free_rcu); return 0; diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 31021dc40e52..50b41be5d38d 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1633,8 +1633,8 @@ static int __ip6_append_data(struct sock *sk, !(rt->dst.dev->features & NETIF_F_SG))) alloclen = fraglen; else { - alloclen = fragheaderlen + transhdrlen; - pagedlen = datalen - transhdrlen; + alloclen = fragheaderlen + transhdrlen + fraggap; + pagedlen = datalen - transhdrlen - fraggap; } alloclen += alloc_extra; @@ -1649,10 +1649,7 @@ static int __ip6_append_data(struct sock *sk, fraglen = datalen + fragheaderlen; copy = datalen - transhdrlen - fraggap - pagedlen; - /* [!] NOTE: copy may be negative if pagedlen>0 - * because then the equation may reduces to -fraggap. - */ - if (copy < 0 && !(flags & MSG_SPLICE_PAGES)) { + if (copy < 0) { err = -EINVAL; goto error; } @@ -1698,8 +1695,9 @@ static int __ip6_append_data(struct sock *sk, pskb_trim_unique(skb_prev, maxfraglen); } if (copy > 0 && - getfrag(from, data + transhdrlen, offset, - copy, fraggap, skb) < 0) { + INDIRECT_CALL_1(getfrag, ip_generic_getfrag, + from, data + transhdrlen, offset, + copy, fraggap, skb) < 0) { err = -EFAULT; kfree_skb(skb); goto error; @@ -1743,8 +1741,9 @@ static int __ip6_append_data(struct sock *sk, unsigned int off; off = skb->len; - if (getfrag(from, skb_put(skb, copy), - offset, copy, off, skb) < 0) { + if (INDIRECT_CALL_1(getfrag, ip_generic_getfrag, + from, skb_put(skb, copy), + offset, copy, off, skb) < 0) { __skb_trim(skb, off); err = -EFAULT; goto error; @@ -1784,7 +1783,8 @@ static int __ip6_append_data(struct sock *sk, get_page(pfrag->page); } copy = min_t(int, copy, pfrag->size - pfrag->offset); - if (getfrag(from, + if (INDIRECT_CALL_1(getfrag, ip_generic_getfrag, + from, page_address(pfrag->page) + pfrag->offset, offset, copy, skb->len, skb) < 0) goto error_efault; diff --git a/net/ipv6/ip6_udp_tunnel.c b/net/ipv6/ip6_udp_tunnel.c index 2acf1bb93fc0..f22eff2ba77c 100644 --- a/net/ipv6/ip6_udp_tunnel.c +++ b/net/ipv6/ip6_udp_tunnel.c @@ -74,13 +74,13 @@ int udp_sock_create6(struct net *net, struct udp_port_cfg *cfg, } EXPORT_SYMBOL_GPL(udp_sock_create6); -int udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk, - struct sk_buff *skb, - struct net_device *dev, - const struct in6_addr *saddr, - const struct in6_addr *daddr, - __u8 prio, __u8 ttl, __be32 label, - __be16 src_port, __be16 dst_port, bool nocheck) +void udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk, + struct sk_buff *skb, + struct net_device *dev, + const struct in6_addr *saddr, + const struct in6_addr *daddr, + __u8 prio, __u8 ttl, __be32 label, + __be16 src_port, __be16 dst_port, bool nocheck) { struct udphdr *uh; struct ipv6hdr *ip6h; @@ -109,7 +109,6 @@ int udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk, ip6h->saddr = *saddr; ip6tunnel_xmit(sk, skb, dev); - return 0; } EXPORT_SYMBOL_GPL(udp_tunnel6_xmit_skb); diff --git a/net/ipv6/ip6_vti.c b/net/ipv6/ip6_vti.c index 6fe696939d04..e0e6e67a25e0 100644 --- a/net/ipv6/ip6_vti.c +++ b/net/ipv6/ip6_vti.c @@ -1159,6 +1159,7 @@ static int __net_init vti6_init_net(struct net *net) goto err_alloc_dev; dev_net_set(ip6n->fb_tnl_dev, net); ip6n->fb_tnl_dev->rtnl_link_ops = &vti6_link_ops; + ip6n->fb_tnl_dev->netns_local = true; err = vti6_fb_tnl_dev_init(ip6n->fb_tnl_dev); if (err < 0) diff --git a/net/mac802154/llsec.c b/net/mac802154/llsec.c index f13b07ebfb98..09a47104b577 100644 --- a/net/mac802154/llsec.c +++ b/net/mac802154/llsec.c @@ -710,6 +710,7 @@ int mac802154_llsec_encrypt(struct mac802154_llsec *sec, struct sk_buff *skb) { struct ieee802154_hdr hdr; int rc, authlen, hlen; + struct sk_buff *trailer; struct mac802154_llsec_key *key; u32 frame_ctr; @@ -769,6 +770,12 @@ int mac802154_llsec_encrypt(struct mac802154_llsec *sec, struct sk_buff *skb) skb->mac_len = ieee802154_hdr_push(skb, &hdr); skb_reset_mac_header(skb); + rc = skb_cow_data(skb, 0, &trailer); + if (rc < 0) { + llsec_key_put(key); + return rc; + } + rc = llsec_do_encrypt(skb, sec, &hdr, key); llsec_key_put(key); @@ -908,6 +915,13 @@ llsec_do_decrypt(struct sk_buff *skb, const struct mac802154_llsec *sec, const struct ieee802154_hdr *hdr, struct mac802154_llsec_key *key, __le64 dev_addr) { + struct sk_buff *trailer; + int err; + + err = skb_cow_data(skb, 0, &trailer); + if (err < 0) + return err; + if (hdr->sec.level == IEEE802154_SCF_SECLEVEL_ENC) return llsec_do_decrypt_unauth(skb, sec, hdr, key, dev_addr); else diff --git a/net/phonet/pn_dev.c b/net/phonet/pn_dev.c index cde671d29d5d..3f020c362d6c 100644 --- a/net/phonet/pn_dev.c +++ b/net/phonet/pn_dev.c @@ -98,11 +98,14 @@ static void phonet_device_destroy(struct net_device *dev) mutex_unlock(&pndevs->lock); if (pnd) { + struct net *net = dev_net(dev); + u32 ifindex = dev->ifindex; u8 addr; for_each_set_bit(addr, pnd->addrs, 64) - phonet_address_notify(RTM_DELADDR, dev, addr); - kfree(pnd); + phonet_address_notify(net, RTM_DELADDR, ifindex, addr); + + kfree_rcu(pnd, rcu); } } @@ -244,8 +247,9 @@ static int phonet_device_autoconf(struct net_device *dev) ret = phonet_address_add(dev, req.ifr_phonet_autoconf.device); if (ret) return ret; - phonet_address_notify(RTM_NEWADDR, dev, - req.ifr_phonet_autoconf.device); + + phonet_address_notify(dev_net(dev), RTM_NEWADDR, dev->ifindex, + req.ifr_phonet_autoconf.device); return 0; } diff --git a/net/phonet/pn_netlink.c b/net/phonet/pn_netlink.c index 894e5c72d6bf..23097085ad38 100644 --- a/net/phonet/pn_netlink.c +++ b/net/phonet/pn_netlink.c @@ -19,10 +19,10 @@ /* Device address handling */ -static int fill_addr(struct sk_buff *skb, struct net_device *dev, u8 addr, +static int fill_addr(struct sk_buff *skb, u32 ifindex, u8 addr, u32 portid, u32 seq, int event); -void phonet_address_notify(int event, struct net_device *dev, u8 addr) +void phonet_address_notify(struct net *net, int event, u32 ifindex, u8 addr) { struct sk_buff *skb; int err = -ENOBUFS; @@ -31,17 +31,18 @@ void phonet_address_notify(int event, struct net_device *dev, u8 addr) nla_total_size(1), GFP_KERNEL); if (skb == NULL) goto errout; - err = fill_addr(skb, dev, addr, 0, 0, event); + + err = fill_addr(skb, ifindex, addr, 0, 0, event); if (err < 0) { WARN_ON(err == -EMSGSIZE); kfree_skb(skb); goto errout; } - rtnl_notify(skb, dev_net(dev), 0, - RTNLGRP_PHONET_IFADDR, NULL, GFP_KERNEL); + + rtnl_notify(skb, net, 0, RTNLGRP_PHONET_IFADDR, NULL, GFP_KERNEL); return; errout: - rtnl_set_sk_err(dev_net(dev), RTNLGRP_PHONET_IFADDR, err); + rtnl_set_sk_err(net, RTNLGRP_PHONET_IFADDR, err); } static const struct nla_policy ifa_phonet_policy[IFA_MAX+1] = { @@ -88,12 +89,12 @@ static int addr_doit(struct sk_buff *skb, struct nlmsghdr *nlh, else err = phonet_address_del(dev, pnaddr); if (!err) - phonet_address_notify(nlh->nlmsg_type, dev, pnaddr); + phonet_address_notify(net, nlh->nlmsg_type, ifm->ifa_index, pnaddr); return err; } -static int fill_addr(struct sk_buff *skb, struct net_device *dev, u8 addr, - u32 portid, u32 seq, int event) +static int fill_addr(struct sk_buff *skb, u32 ifindex, u8 addr, + u32 portid, u32 seq, int event) { struct ifaddrmsg *ifm; struct nlmsghdr *nlh; @@ -107,7 +108,7 @@ static int fill_addr(struct sk_buff *skb, struct net_device *dev, u8 addr, ifm->ifa_prefixlen = 0; ifm->ifa_flags = IFA_F_PERMANENT; ifm->ifa_scope = RT_SCOPE_LINK; - ifm->ifa_index = dev->ifindex; + ifm->ifa_index = ifindex; if (nla_put_u8(skb, IFA_LOCAL, addr)) goto nla_put_failure; nlmsg_end(skb, nlh); @@ -140,7 +141,7 @@ static int getaddr_dumpit(struct sk_buff *skb, struct netlink_callback *cb) if (addr_idx++ < addr_start_idx) continue; - if (fill_addr(skb, pnd->netdev, addr << 2, + if (fill_addr(skb, pnd->netdev->ifindex, addr << 2, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq, RTM_NEWADDR) < 0) goto out; diff --git a/net/rxrpc/input.c b/net/rxrpc/input.c index 6a075a7c190d..ca2d40ba7098 100644 --- a/net/rxrpc/input.c +++ b/net/rxrpc/input.c @@ -775,9 +775,23 @@ static void rxrpc_input_soft_acks(struct rxrpc_call *call, rxrpc_seq_t since) { struct rxrpc_skb_priv *sp = rxrpc_skb(skb); - unsigned int i, old_nacks = 0; + unsigned int i, old_nacks = 0, nsack; rxrpc_seq_t lowest_nak = seq + sp->ack.nr_acks; - u8 *acks = skb->data + sizeof(struct rxrpc_wire_header) + sizeof(struct rxrpc_ackpacket); + u8 sack[256] __aligned(sizeof(unsigned long)); + u8 *acks = sack; + + /* AF_RXRPC assumes that it can access the SACK table directly from + * skb->data as a flat buffer, but the skb may be non-linear (e.g. a + * fragmented UDP packet) and skb_condense() can silently fail to + * linearise it. Copy the SACK table out into a local buffer before + * parsing it. + */ + memset(sack, 0, sizeof(sack)); + nsack = umin(sp->ack.nr_acks, 256); + if (skb_copy_bits(skb, + sizeof(struct rxrpc_wire_header) + sizeof(struct rxrpc_ackpacket), + sack, nsack) < 0) + return; for (i = 0; i < sp->ack.nr_acks; i++) { if (acks[i] == RXRPC_ACK_TYPE_ACK) { @@ -934,9 +948,6 @@ static void rxrpc_input_ack(struct rxrpc_call *call, struct sk_buff *skb) skb_copy_bits(skb, ioffset, &trailer, sizeof(trailer)) < 0) return rxrpc_proto_abort(call, 0, rxrpc_badmsg_short_ack_trailer); - if (nr_acks > 0) - skb_condense(skb); - if (call->cong_last_nack) { since = rxrpc_input_check_prev_ack(call, &summary, first_soft_ack); rxrpc_free_skb(call->cong_last_nack, rxrpc_skb_put_last_nack); diff --git a/net/sctp/ipv6.c b/net/sctp/ipv6.c index 9f835e674c59..b45cc51dfc35 100644 --- a/net/sctp/ipv6.c +++ b/net/sctp/ipv6.c @@ -261,9 +261,12 @@ static int sctp_v6_xmit(struct sk_buff *skb, struct sctp_transport *t) skb_set_inner_ipproto(skb, IPPROTO_SCTP); label = ip6_make_flowlabel(sock_net(sk), skb, fl6->flowlabel, true, fl6); - return udp_tunnel6_xmit_skb(dst, sk, skb, NULL, &fl6->saddr, - &fl6->daddr, tclass, ip6_dst_hoplimit(dst), - label, sctp_sk(sk)->udp_port, t->encap_port, false); + local_bh_disable(); + udp_tunnel6_xmit_skb(dst, sk, skb, NULL, &fl6->saddr, &fl6->daddr, + tclass, ip6_dst_hoplimit(dst), label, + sctp_sk(sk)->udp_port, t->encap_port, false); + local_bh_enable(); + return 0; } /* Returns the dst cache entry for the given source and destination ip diff --git a/net/sctp/protocol.c b/net/sctp/protocol.c index 39ca5403d4d7..6ea15361088b 100644 --- a/net/sctp/protocol.c +++ b/net/sctp/protocol.c @@ -1086,9 +1086,11 @@ static inline int sctp_v4_xmit(struct sk_buff *skb, struct sctp_transport *t) skb_reset_inner_mac_header(skb); skb_reset_inner_transport_header(skb); skb_set_inner_ipproto(skb, IPPROTO_SCTP); + local_bh_disable(); udp_tunnel_xmit_skb(dst_rtable(dst), sk, skb, fl4->saddr, fl4->daddr, dscp, ip4_dst_hoplimit(dst), df, sctp_sk(sk)->udp_port, t->encap_port, false, false); + local_bh_enable(); return 0; } diff --git a/net/socket.c b/net/socket.c index 5c5dd9f6605a..723bc3a1ba5c 100644 --- a/net/socket.c +++ b/net/socket.c @@ -652,7 +652,7 @@ static void __sock_release(struct socket *sock, struct inode *inode) iput(SOCK_INODE(sock)); return; } - sock->file = NULL; + WRITE_ONCE(sock->file, NULL); } /** diff --git a/net/tipc/crypto.c b/net/tipc/crypto.c index d7736d900271..d58db40d6c03 100644 --- a/net/tipc/crypto.c +++ b/net/tipc/crypto.c @@ -950,12 +950,20 @@ static int tipc_aead_decrypt(struct net *net, struct tipc_aead *aead, goto exit; } + /* Get net to avoid freed tipc_crypto when delete namespace */ + if (!maybe_get_net(net)) { + tipc_bearer_put(b); + rc = -ENODEV; + goto exit; + } + /* Now, do decrypt */ rc = crypto_aead_decrypt(req); if (rc == -EINPROGRESS || rc == -EBUSY) return rc; tipc_bearer_put(b); + put_net(net); exit: kfree(ctx); @@ -993,6 +1001,7 @@ static void tipc_aead_decrypt_done(void *data, int err) } tipc_bearer_put(b); + put_net(net); } static inline int tipc_ehdr_size(struct tipc_ehdr *ehdr) diff --git a/net/tipc/udp_media.c b/net/tipc/udp_media.c index 258d6aa4f21a..1b8d6bbf8a8e 100644 --- a/net/tipc/udp_media.c +++ b/net/tipc/udp_media.c @@ -172,7 +172,7 @@ static int tipc_udp_xmit(struct net *net, struct sk_buff *skb, struct udp_media_addr *dst, struct dst_cache *cache) { struct dst_entry *ndst; - int ttl, err = 0; + int ttl, err; local_bh_disable(); ndst = dst_cache_get(cache); @@ -217,13 +217,13 @@ static int tipc_udp_xmit(struct net *net, struct sk_buff *skb, dst_cache_set_ip6(cache, ndst, &fl6.saddr); } ttl = ip6_dst_hoplimit(ndst); - err = udp_tunnel6_xmit_skb(ndst, ub->ubsock->sk, skb, NULL, - &src->ipv6, &dst->ipv6, 0, ttl, 0, - src->port, dst->port, false); + udp_tunnel6_xmit_skb(ndst, ub->ubsock->sk, skb, NULL, + &src->ipv6, &dst->ipv6, 0, ttl, 0, + src->port, dst->port, false); #endif } local_bh_enable(); - return err; + return 0; tx_error: local_bh_enable(); diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index 129a8c778d32..d3eab47b47ea 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -623,6 +623,7 @@ static int tls_split_open_record(struct sock *sk, struct tls_rec *from, struct scatterlist *sge, *osge, *nsge; u32 orig_size = msg_opl->sg.size; struct scatterlist tmp = { }; + u32 tmp_i = 0; struct sk_msg *msg_npl; struct tls_rec *new; int ret; @@ -644,6 +645,7 @@ static int tls_split_open_record(struct sock *sk, struct tls_rec *from, if (sge->length > apply) { u32 len = sge->length - apply; + tmp_i = i; get_page(sg_page(sge)); sg_set_page(&tmp, sg_page(sge), len, sge->offset + apply); @@ -675,6 +677,7 @@ static int tls_split_open_record(struct sock *sk, struct tls_rec *from, nsge = sk_msg_elem(msg_npl, j); if (tmp.length) { memcpy(nsge, &tmp, sizeof(*nsge)); + sk_msg_sg_copy_assign(msg_npl, j, msg_opl, tmp_i); sk_msg_iter_var_next(j); nsge = sk_msg_elem(msg_npl, j); } @@ -682,6 +685,7 @@ static int tls_split_open_record(struct sock *sk, struct tls_rec *from, osge = sk_msg_elem(msg_opl, i); while (osge->length) { memcpy(nsge, osge, sizeof(*nsge)); + sk_msg_sg_copy_assign(msg_npl, j, msg_opl, i); sg_unmark_end(nsge); sk_msg_iter_var_next(i); sk_msg_iter_var_next(j); diff --git a/net/unix/garbage.c b/net/unix/garbage.c index 1cdb54c61619..fa6983dc3181 100644 --- a/net/unix/garbage.c +++ b/net/unix/garbage.c @@ -583,6 +583,8 @@ static void __unix_gc(struct work_struct *work) struct sk_buff_head hitlist; struct sk_buff *skb; + WRITE_ONCE(gc_in_progress, true); + spin_lock(&unix_gc_lock); if (!unix_graph_maybe_cyclic) { diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index a9b3f34a78d2..94f794ca11ca 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -173,12 +173,14 @@ mksysmap() sorttable() { - ${objtree}/scripts/sorttable ${1} + ${NM} -S ${1} > .tmp_vmlinux.nm-sort + ${objtree}/scripts/sorttable -s .tmp_vmlinux.nm-sort ${1} } cleanup() { rm -f .btf.* + rm -f .tmp_vmlinux.nm-sort rm -f System.map rm -f vmlinux rm -f vmlinux.map diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 83cdb843d92f..deed676bfe38 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -64,14 +65,246 @@ #define EM_LOONGARCH 258 #endif +typedef union { + Elf32_Ehdr e32; + Elf64_Ehdr e64; +} Elf_Ehdr; + +typedef union { + Elf32_Shdr e32; + Elf64_Shdr e64; +} Elf_Shdr; + +typedef union { + Elf32_Sym e32; + Elf64_Sym e64; +} Elf_Sym; + +typedef union { + Elf32_Rela e32; + Elf64_Rela e64; +} Elf_Rela; + static uint32_t (*r)(const uint32_t *); static uint16_t (*r2)(const uint16_t *); static uint64_t (*r8)(const uint64_t *); static void (*w)(uint32_t, uint32_t *); -static void (*w2)(uint16_t, uint16_t *); static void (*w8)(uint64_t, uint64_t *); typedef void (*table_sort_t)(char *, int); +static struct elf_funcs { + int (*compare_extable)(const void *a, const void *b); + uint64_t (*ehdr_shoff)(Elf_Ehdr *ehdr); + uint16_t (*ehdr_shstrndx)(Elf_Ehdr *ehdr); + uint16_t (*ehdr_shentsize)(Elf_Ehdr *ehdr); + uint16_t (*ehdr_shnum)(Elf_Ehdr *ehdr); + uint64_t (*shdr_addr)(Elf_Shdr *shdr); + uint64_t (*shdr_offset)(Elf_Shdr *shdr); + uint64_t (*shdr_size)(Elf_Shdr *shdr); + uint64_t (*shdr_entsize)(Elf_Shdr *shdr); + uint32_t (*shdr_link)(Elf_Shdr *shdr); + uint32_t (*shdr_name)(Elf_Shdr *shdr); + uint32_t (*shdr_type)(Elf_Shdr *shdr); + uint8_t (*sym_type)(Elf_Sym *sym); + uint32_t (*sym_name)(Elf_Sym *sym); + uint64_t (*sym_value)(Elf_Sym *sym); + uint16_t (*sym_shndx)(Elf_Sym *sym); + uint64_t (*rela_offset)(Elf_Rela *rela); + uint64_t (*rela_info)(Elf_Rela *rela); + uint64_t (*rela_addend)(Elf_Rela *rela); + void (*rela_write_addend)(Elf_Rela *rela, uint64_t val); +} e; + +static uint64_t ehdr64_shoff(Elf_Ehdr *ehdr) +{ + return r8(&ehdr->e64.e_shoff); +} + +static uint64_t ehdr32_shoff(Elf_Ehdr *ehdr) +{ + return r(&ehdr->e32.e_shoff); +} + +static uint64_t ehdr_shoff(Elf_Ehdr *ehdr) +{ + return e.ehdr_shoff(ehdr); +} + +#define EHDR_HALF(fn_name) \ +static uint16_t ehdr64_##fn_name(Elf_Ehdr *ehdr) \ +{ \ + return r2(&ehdr->e64.e_##fn_name); \ +} \ + \ +static uint16_t ehdr32_##fn_name(Elf_Ehdr *ehdr) \ +{ \ + return r2(&ehdr->e32.e_##fn_name); \ +} \ + \ +static uint16_t ehdr_##fn_name(Elf_Ehdr *ehdr) \ +{ \ + return e.ehdr_##fn_name(ehdr); \ +} + +EHDR_HALF(shentsize) +EHDR_HALF(shstrndx) +EHDR_HALF(shnum) + +#define SHDR_WORD(fn_name) \ +static uint32_t shdr64_##fn_name(Elf_Shdr *shdr) \ +{ \ + return r(&shdr->e64.sh_##fn_name); \ +} \ + \ +static uint32_t shdr32_##fn_name(Elf_Shdr *shdr) \ +{ \ + return r(&shdr->e32.sh_##fn_name); \ +} \ + \ +static uint32_t shdr_##fn_name(Elf_Shdr *shdr) \ +{ \ + return e.shdr_##fn_name(shdr); \ +} + +#define SHDR_ADDR(fn_name) \ +static uint64_t shdr64_##fn_name(Elf_Shdr *shdr) \ +{ \ + return r8(&shdr->e64.sh_##fn_name); \ +} \ + \ +static uint64_t shdr32_##fn_name(Elf_Shdr *shdr) \ +{ \ + return r(&shdr->e32.sh_##fn_name); \ +} \ + \ +static uint64_t shdr_##fn_name(Elf_Shdr *shdr) \ +{ \ + return e.shdr_##fn_name(shdr); \ +} + +#define SHDR_WORD(fn_name) \ +static uint32_t shdr64_##fn_name(Elf_Shdr *shdr) \ +{ \ + return r(&shdr->e64.sh_##fn_name); \ +} \ + \ +static uint32_t shdr32_##fn_name(Elf_Shdr *shdr) \ +{ \ + return r(&shdr->e32.sh_##fn_name); \ +} \ +static uint32_t shdr_##fn_name(Elf_Shdr *shdr) \ +{ \ + return e.shdr_##fn_name(shdr); \ +} + +SHDR_ADDR(addr) +SHDR_ADDR(offset) +SHDR_ADDR(size) +SHDR_ADDR(entsize) + +SHDR_WORD(link) +SHDR_WORD(name) +SHDR_WORD(type) + +#define SYM_ADDR(fn_name) \ +static uint64_t sym64_##fn_name(Elf_Sym *sym) \ +{ \ + return r8(&sym->e64.st_##fn_name); \ +} \ + \ +static uint64_t sym32_##fn_name(Elf_Sym *sym) \ +{ \ + return r(&sym->e32.st_##fn_name); \ +} \ + \ +static uint64_t sym_##fn_name(Elf_Sym *sym) \ +{ \ + return e.sym_##fn_name(sym); \ +} + +#define SYM_WORD(fn_name) \ +static uint32_t sym64_##fn_name(Elf_Sym *sym) \ +{ \ + return r(&sym->e64.st_##fn_name); \ +} \ + \ +static uint32_t sym32_##fn_name(Elf_Sym *sym) \ +{ \ + return r(&sym->e32.st_##fn_name); \ +} \ + \ +static uint32_t sym_##fn_name(Elf_Sym *sym) \ +{ \ + return e.sym_##fn_name(sym); \ +} + +#define SYM_HALF(fn_name) \ +static uint16_t sym64_##fn_name(Elf_Sym *sym) \ +{ \ + return r2(&sym->e64.st_##fn_name); \ +} \ + \ +static uint16_t sym32_##fn_name(Elf_Sym *sym) \ +{ \ + return r2(&sym->e32.st_##fn_name); \ +} \ + \ +static uint16_t sym_##fn_name(Elf_Sym *sym) \ +{ \ + return e.sym_##fn_name(sym); \ +} + +static uint8_t sym64_type(Elf_Sym *sym) +{ + return ELF64_ST_TYPE(sym->e64.st_info); +} + +static uint8_t sym32_type(Elf_Sym *sym) +{ + return ELF32_ST_TYPE(sym->e32.st_info); +} + +static uint8_t sym_type(Elf_Sym *sym) +{ + return e.sym_type(sym); +} + +SYM_ADDR(value) +SYM_WORD(name) +SYM_HALF(shndx) + +#define __maybe_unused __attribute__((__unused__)) + +#define RELA_ADDR(fn_name) \ +static uint64_t rela64_##fn_name(Elf_Rela *rela) \ +{ \ + return r8((uint64_t *)&rela->e64.r_##fn_name); \ +} \ + \ +static uint64_t rela32_##fn_name(Elf_Rela *rela) \ +{ \ + return r((uint32_t *)&rela->e32.r_##fn_name); \ +} \ + \ +static uint64_t __maybe_unused rela_##fn_name(Elf_Rela *rela) \ +{ \ + return e.rela_##fn_name(rela); \ +} + +RELA_ADDR(offset) +RELA_ADDR(info) +RELA_ADDR(addend) + +static void rela64_write_addend(Elf_Rela *rela, uint64_t val) +{ + w8(val, (uint64_t *)&rela->e64.r_addend); +} + +static void rela32_write_addend(Elf_Rela *rela, uint64_t val) +{ + w(val, (uint32_t *)&rela->e32.r_addend); +} + /* * Get the whole file as a programming convenience in order to avoid * malloc+lseek+read+free of many pieces. If successful, then mmap @@ -146,24 +379,14 @@ static void wbe(uint32_t val, uint32_t *x) put_unaligned_be32(val, x); } -static void w2be(uint16_t val, uint16_t *x) -{ - put_unaligned_be16(val, x); -} - -static void w8be(uint64_t val, uint64_t *x) -{ - put_unaligned_be64(val, x); -} - static void wle(uint32_t val, uint32_t *x) { put_unaligned_le32(val, x); } -static void w2le(uint16_t val, uint16_t *x) +static void w8be(uint64_t val, uint64_t *x) { - put_unaligned_le16(val, x); + put_unaligned_be64(val, x); } static void w8le(uint64_t val, uint64_t *x) @@ -195,10 +418,740 @@ static inline unsigned int get_secindex(unsigned int shndx, return r(&symtab_shndx_start[sym_offs]); } -/* 32 bit and 64 bit are very similar */ -#include "sorttable.h" -#define SORTTABLE_64 -#include "sorttable.h" +static int compare_extable_32(const void *a, const void *b) +{ + Elf32_Addr av = r(a); + Elf32_Addr bv = r(b); + + if (av < bv) + return -1; + return av > bv; +} + +static int compare_extable_64(const void *a, const void *b) +{ + Elf64_Addr av = r8(a); + Elf64_Addr bv = r8(b); + + if (av < bv) + return -1; + return av > bv; +} + +static int compare_extable(const void *a, const void *b) +{ + return e.compare_extable(a, b); +} + +static inline void *get_index(void *start, int entsize, int index) +{ + return start + (entsize * index); +} + +static int extable_ent_size; +static int long_size; + +#define ERRSTR_MAXSZ 256 + +#ifdef UNWINDER_ORC_ENABLED +/* ORC unwinder only support X86_64 */ +#include + +static char g_err[ERRSTR_MAXSZ]; +static int *g_orc_ip_table; +static struct orc_entry *g_orc_table; + +static pthread_t orc_sort_thread; + +static inline unsigned long orc_ip(const int *ip) +{ + return (unsigned long)ip + *ip; +} + +static int orc_sort_cmp(const void *_a, const void *_b) +{ + struct orc_entry *orc_a, *orc_b; + const int *a = g_orc_ip_table + *(int *)_a; + const int *b = g_orc_ip_table + *(int *)_b; + unsigned long a_val = orc_ip(a); + unsigned long b_val = orc_ip(b); + + if (a_val > b_val) + return 1; + if (a_val < b_val) + return -1; + + /* + * The "weak" section terminator entries need to always be on the left + * to ensure the lookup code skips them in favor of real entries. + * These terminator entries exist to handle any gaps created by + * whitelisted .o files which didn't get objtool generation. + */ + orc_a = g_orc_table + (a - g_orc_ip_table); + orc_b = g_orc_table + (b - g_orc_ip_table); + if (orc_a->type == ORC_TYPE_UNDEFINED && orc_b->type == ORC_TYPE_UNDEFINED) + return 0; + return orc_a->type == ORC_TYPE_UNDEFINED ? -1 : 1; +} + +static void *sort_orctable(void *arg) +{ + int i; + int *idxs = NULL; + int *tmp_orc_ip_table = NULL; + struct orc_entry *tmp_orc_table = NULL; + unsigned int *orc_ip_size = (unsigned int *)arg; + unsigned int num_entries = *orc_ip_size / sizeof(int); + unsigned int orc_size = num_entries * sizeof(struct orc_entry); + + idxs = (int *)malloc(*orc_ip_size); + if (!idxs) { + snprintf(g_err, ERRSTR_MAXSZ, "malloc idxs: %s", + strerror(errno)); + pthread_exit(g_err); + } + + tmp_orc_ip_table = (int *)malloc(*orc_ip_size); + if (!tmp_orc_ip_table) { + snprintf(g_err, ERRSTR_MAXSZ, "malloc tmp_orc_ip_table: %s", + strerror(errno)); + pthread_exit(g_err); + } + + tmp_orc_table = (struct orc_entry *)malloc(orc_size); + if (!tmp_orc_table) { + snprintf(g_err, ERRSTR_MAXSZ, "malloc tmp_orc_table: %s", + strerror(errno)); + pthread_exit(g_err); + } + + /* initialize indices array, convert ip_table to absolute address */ + for (i = 0; i < num_entries; i++) { + idxs[i] = i; + tmp_orc_ip_table[i] = g_orc_ip_table[i] + i * sizeof(int); + } + memcpy(tmp_orc_table, g_orc_table, orc_size); + + qsort(idxs, num_entries, sizeof(int), orc_sort_cmp); + + for (i = 0; i < num_entries; i++) { + if (idxs[i] == i) + continue; + + /* convert back to relative address */ + g_orc_ip_table[i] = tmp_orc_ip_table[idxs[i]] - i * sizeof(int); + g_orc_table[i] = tmp_orc_table[idxs[i]]; + } + + free(idxs); + free(tmp_orc_ip_table); + free(tmp_orc_table); + pthread_exit(NULL); +} +#endif + +#ifdef MCOUNT_SORT_ENABLED + +static int compare_values_64(const void *a, const void *b) +{ + uint64_t av = *(uint64_t *)a; + uint64_t bv = *(uint64_t *)b; + + if (av < bv) + return -1; + return av > bv; +} + +static int compare_values_32(const void *a, const void *b) +{ + uint32_t av = *(uint32_t *)a; + uint32_t bv = *(uint32_t *)b; + + if (av < bv) + return -1; + return av > bv; +} + +static int (*compare_values)(const void *a, const void *b); + +/* Only used for sorting mcount table */ +static void rela_write_addend(Elf_Rela *rela, uint64_t val) +{ + e.rela_write_addend(rela, val); +} + +struct func_info { + uint64_t addr; + uint64_t size; +}; + +/* List of functions created by: nm -S vmlinux */ +static struct func_info *function_list; +static int function_list_size; + +/* Allocate functions in 1k blocks */ +#define FUNC_BLK_SIZE 1024 +#define FUNC_BLK_MASK (FUNC_BLK_SIZE - 1) + +static int add_field(uint64_t addr, uint64_t size) +{ + struct func_info *fi; + int fsize = function_list_size; + + if (!(fsize & FUNC_BLK_MASK)) { + fsize += FUNC_BLK_SIZE; + fi = realloc(function_list, fsize * sizeof(struct func_info)); + if (!fi) + return -1; + function_list = fi; + } + fi = &function_list[function_list_size++]; + fi->addr = addr; + fi->size = size; + return 0; +} + +/* Used for when mcount/fentry is before the function entry */ +static int before_func; + +/* Only return match if the address lies inside the function size */ +static int cmp_func_addr(const void *K, const void *A) +{ + uint64_t key = *(const uint64_t *)K; + const struct func_info *a = A; + + if (key + before_func < a->addr) + return -1; + return key >= a->addr + a->size; +} + +/* Find the function in function list that is bounded by the function size */ +static int find_func(uint64_t key) +{ + return bsearch(&key, function_list, function_list_size, + sizeof(struct func_info), cmp_func_addr) != NULL; +} + +static int cmp_funcs(const void *A, const void *B) +{ + const struct func_info *a = A; + const struct func_info *b = B; + + if (a->addr < b->addr) + return -1; + return a->addr > b->addr; +} + +static int parse_symbols(const char *fname) +{ + FILE *fp; + char addr_str[20]; /* Only need 17, but round up to next int size */ + char size_str[20]; + char type; + + fp = fopen(fname, "r"); + if (!fp) { + perror(fname); + return -1; + } + + while (fscanf(fp, "%16s %16s %c %*s\n", addr_str, size_str, &type) == 3) { + uint64_t addr; + uint64_t size; + + /* Only care about functions */ + if (type != 't' && type != 'T' && type != 'W') + continue; + + addr = strtoull(addr_str, NULL, 16); + size = strtoull(size_str, NULL, 16); + if (add_field(addr, size) < 0) + return -1; + } + fclose(fp); + + qsort(function_list, function_list_size, sizeof(struct func_info), cmp_funcs); + + return 0; +} + +static pthread_t mcount_sort_thread; +static bool sort_reloc; + +static long rela_type; + +static char m_err[ERRSTR_MAXSZ]; + +struct elf_mcount_loc { + Elf_Ehdr *ehdr; + Elf_Shdr *init_data_sec; + uint64_t start_mcount_loc; + uint64_t stop_mcount_loc; +}; + +/* Fill the array with the content of the relocs */ +static int fill_relocs(void *ptr, uint64_t size, Elf_Ehdr *ehdr, uint64_t start_loc) +{ + Elf_Shdr *shdr_start; + Elf_Rela *rel; + unsigned int shnum; + unsigned int count = 0; + int shentsize; + void *array_end = ptr + size; + + shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); + shentsize = ehdr_shentsize(ehdr); + + shnum = ehdr_shnum(ehdr); + if (shnum == SHN_UNDEF) + shnum = shdr_size(shdr_start); + + for (int i = 0; i < shnum; i++) { + Elf_Shdr *shdr = get_index(shdr_start, shentsize, i); + void *end; + + if (shdr_type(shdr) != SHT_RELA) + continue; + + rel = (void *)ehdr + shdr_offset(shdr); + end = (void *)rel + shdr_size(shdr); + + for (; (void *)rel < end; rel = (void *)rel + shdr_entsize(shdr)) { + uint64_t offset = rela_offset(rel); + + if (offset >= start_loc && offset < start_loc + size) { + if (ptr + long_size > array_end) { + snprintf(m_err, ERRSTR_MAXSZ, + "Too many relocations"); + return -1; + } + + /* Make sure this has the correct type */ + if (rela_info(rel) != rela_type) { + snprintf(m_err, ERRSTR_MAXSZ, + "rela has type %lx but expected %lx\n", + (long)rela_info(rel), rela_type); + return -1; + } + + if (long_size == 4) + *(uint32_t *)ptr = rela_addend(rel); + else + *(uint64_t *)ptr = rela_addend(rel); + ptr += long_size; + count++; + } + } + } + return count; +} + +/* Put the sorted vals back into the relocation elements */ +static void replace_relocs(void *ptr, uint64_t size, Elf_Ehdr *ehdr, uint64_t start_loc) +{ + Elf_Shdr *shdr_start; + Elf_Rela *rel; + unsigned int shnum; + int shentsize; + + shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); + shentsize = ehdr_shentsize(ehdr); + + shnum = ehdr_shnum(ehdr); + if (shnum == SHN_UNDEF) + shnum = shdr_size(shdr_start); + + for (int i = 0; i < shnum; i++) { + Elf_Shdr *shdr = get_index(shdr_start, shentsize, i); + void *end; + + if (shdr_type(shdr) != SHT_RELA) + continue; + + rel = (void *)ehdr + shdr_offset(shdr); + end = (void *)rel + shdr_size(shdr); + + for (; (void *)rel < end; rel = (void *)rel + shdr_entsize(shdr)) { + uint64_t offset = rela_offset(rel); + + if (offset >= start_loc && offset < start_loc + size) { + if (long_size == 4) + rela_write_addend(rel, *(uint32_t *)ptr); + else + rela_write_addend(rel, *(uint64_t *)ptr); + ptr += long_size; + } + } + } +} + +static int fill_addrs(void *ptr, uint64_t size, void *addrs) +{ + void *end = ptr + size; + int count = 0; + + for (; ptr < end; ptr += long_size, addrs += long_size, count++) { + if (long_size == 4) + *(uint32_t *)ptr = r(addrs); + else + *(uint64_t *)ptr = r8(addrs); + } + return count; +} + +static void replace_addrs(void *ptr, uint64_t size, void *addrs) +{ + void *end = ptr + size; + + for (; ptr < end; ptr += long_size, addrs += long_size) { + if (long_size == 4) + w(*(uint32_t *)ptr, addrs); + else + w8(*(uint64_t *)ptr, addrs); + } +} + +/* Sort the addresses stored between __start_mcount_loc to __stop_mcount_loc in vmlinux */ +static void *sort_mcount_loc(void *arg) +{ + struct elf_mcount_loc *emloc = (struct elf_mcount_loc *)arg; + uint64_t offset = emloc->start_mcount_loc - shdr_addr(emloc->init_data_sec) + + shdr_offset(emloc->init_data_sec); + uint64_t size = emloc->stop_mcount_loc - emloc->start_mcount_loc; + unsigned char *start_loc = (void *)emloc->ehdr + offset; + Elf_Ehdr *ehdr = emloc->ehdr; + void *e_msg = NULL; + void *vals; + int count; + + vals = malloc(long_size * size); + if (!vals) { + snprintf(m_err, ERRSTR_MAXSZ, "Failed to allocate sort array"); + pthread_exit(m_err); + } + + if (sort_reloc) { + count = fill_relocs(vals, size, ehdr, emloc->start_mcount_loc); + /* gcc may use relocs to save the addresses, but clang does not. */ + if (!count) { + count = fill_addrs(vals, size, start_loc); + sort_reloc = 0; + } + } else + count = fill_addrs(vals, size, start_loc); + + if (count < 0) { + e_msg = m_err; + goto out; + } + + if (count != size / long_size) { + snprintf(m_err, ERRSTR_MAXSZ, "Expected %u mcount elements but found %u\n", + (int)(size / long_size), count); + e_msg = m_err; + goto out; + } + + /* zero out any locations not found by function list */ + if (function_list_size) { + for (void *ptr = vals; ptr < vals + size; ptr += long_size) { + uint64_t key; + + key = long_size == 4 ? *(uint32_t *)ptr : *(uint64_t *)ptr; + if (!find_func(key)) { + if (long_size == 4) + *(uint32_t *)ptr = 0; + else + *(uint64_t *)ptr = 0; + } + } + } + + compare_values = long_size == 4 ? compare_values_32 : compare_values_64; + + qsort(vals, count, long_size, compare_values); + + if (sort_reloc) + replace_relocs(vals, size, ehdr, emloc->start_mcount_loc); + else + replace_addrs(vals, size, start_loc); + +out: + free(vals); + + pthread_exit(e_msg); +} + +/* Get the address of __start_mcount_loc and __stop_mcount_loc in System.map */ +static void get_mcount_loc(struct elf_mcount_loc *emloc, Elf_Shdr *symtab_sec, + const char *strtab) +{ + Elf_Sym *sym, *end_sym; + int symentsize = shdr_entsize(symtab_sec); + int found = 0; + + sym = (void *)emloc->ehdr + shdr_offset(symtab_sec); + end_sym = (void *)sym + shdr_size(symtab_sec); + + while (sym < end_sym) { + if (!strcmp(strtab + sym_name(sym), "__start_mcount_loc")) { + emloc->start_mcount_loc = sym_value(sym); + if (++found == 2) + break; + } else if (!strcmp(strtab + sym_name(sym), "__stop_mcount_loc")) { + emloc->stop_mcount_loc = sym_value(sym); + if (++found == 2) + break; + } + sym = (void *)sym + symentsize; + } + + if (!emloc->start_mcount_loc) { + fprintf(stderr, "get start_mcount_loc error!"); + return; + } + + if (!emloc->stop_mcount_loc) { + fprintf(stderr, "get stop_mcount_loc error!"); + return; + } +} +#else /* MCOUNT_SORT_ENABLED */ +static inline int parse_symbols(const char *fname) { return 0; } +#endif + +static int do_sort(Elf_Ehdr *ehdr, + char const *const fname, + table_sort_t custom_sort) +{ + int rc = -1; + Elf_Shdr *shdr_start; + Elf_Shdr *strtab_sec = NULL; + Elf_Shdr *symtab_sec = NULL; + Elf_Shdr *extab_sec = NULL; + Elf_Shdr *string_sec; + Elf_Sym *sym; + const Elf_Sym *symtab; + Elf32_Word *symtab_shndx = NULL; + Elf_Sym *sort_needed_sym = NULL; + Elf_Shdr *sort_needed_sec; + uint32_t *sort_needed_loc; + void *sym_start; + void *sym_end; + const char *secstrings; + const char *strtab; + char *extab_image; + int sort_need_index; + int symentsize; + int shentsize; + int idx; + int i; + unsigned int shnum; + unsigned int shstrndx; +#ifdef MCOUNT_SORT_ENABLED + struct elf_mcount_loc mstruct = {0}; +#endif +#ifdef UNWINDER_ORC_ENABLED + unsigned int orc_ip_size = 0; + unsigned int orc_size = 0; + unsigned int orc_num_entries = 0; +#endif + + shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); + shentsize = ehdr_shentsize(ehdr); + + shstrndx = ehdr_shstrndx(ehdr); + if (shstrndx == SHN_XINDEX) + shstrndx = shdr_link(shdr_start); + string_sec = get_index(shdr_start, shentsize, shstrndx); + secstrings = (const char *)ehdr + shdr_offset(string_sec); + + shnum = ehdr_shnum(ehdr); + if (shnum == SHN_UNDEF) + shnum = shdr_size(shdr_start); + + for (i = 0; i < shnum; i++) { + Elf_Shdr *shdr = get_index(shdr_start, shentsize, i); + + idx = shdr_name(shdr); + if (!strcmp(secstrings + idx, "__ex_table")) + extab_sec = shdr; + if (!strcmp(secstrings + idx, ".symtab")) + symtab_sec = shdr; + if (!strcmp(secstrings + idx, ".strtab")) + strtab_sec = shdr; + + if (shdr_type(shdr) == SHT_SYMTAB_SHNDX) + symtab_shndx = (Elf32_Word *)((const char *)ehdr + + shdr_offset(shdr)); + +#ifdef MCOUNT_SORT_ENABLED + /* locate the .init.data section in vmlinux */ + if (!strcmp(secstrings + idx, ".init.data")) + mstruct.init_data_sec = shdr; +#endif + +#ifdef UNWINDER_ORC_ENABLED + /* locate the ORC unwind tables */ + if (!strcmp(secstrings + idx, ".orc_unwind_ip")) { + orc_ip_size = shdr_size(shdr); + g_orc_ip_table = (int *)((void *)ehdr + + shdr_offset(shdr)); + } + if (!strcmp(secstrings + idx, ".orc_unwind")) { + orc_size = shdr_size(shdr); + g_orc_table = (struct orc_entry *)((void *)ehdr + + shdr_offset(shdr)); + } +#endif + } /* for loop */ + +#ifdef UNWINDER_ORC_ENABLED + if (!g_orc_ip_table || !g_orc_table) { + fprintf(stderr, + "incomplete ORC unwind tables in file: %s\n", fname); + goto out; + } + + orc_num_entries = orc_ip_size / sizeof(int); + if (orc_ip_size % sizeof(int) != 0 || + orc_size % sizeof(struct orc_entry) != 0 || + orc_num_entries != orc_size / sizeof(struct orc_entry)) { + fprintf(stderr, + "inconsistent ORC unwind table entries in file: %s\n", + fname); + goto out; + } + + /* create thread to sort ORC unwind tables concurrently */ + if (pthread_create(&orc_sort_thread, NULL, + sort_orctable, &orc_ip_size)) { + fprintf(stderr, + "pthread_create orc_sort_thread failed '%s': %s\n", + strerror(errno), fname); + goto out; + } +#endif + if (!extab_sec) { + fprintf(stderr, "no __ex_table in file: %s\n", fname); + goto out; + } + + if (!symtab_sec) { + fprintf(stderr, "no .symtab in file: %s\n", fname); + goto out; + } + + if (!strtab_sec) { + fprintf(stderr, "no .strtab in file: %s\n", fname); + goto out; + } + + extab_image = (void *)ehdr + shdr_offset(extab_sec); + strtab = (const char *)ehdr + shdr_offset(strtab_sec); + symtab = (const Elf_Sym *)((const char *)ehdr + shdr_offset(symtab_sec)); + +#ifdef MCOUNT_SORT_ENABLED + mstruct.ehdr = ehdr; + get_mcount_loc(&mstruct, symtab_sec, strtab); + + if (!mstruct.init_data_sec || !mstruct.start_mcount_loc || !mstruct.stop_mcount_loc) { + fprintf(stderr, + "incomplete mcount's sort in file: %s\n", + fname); + goto out; + } + + /* create thread to sort mcount_loc concurrently */ + if (pthread_create(&mcount_sort_thread, NULL, &sort_mcount_loc, &mstruct)) { + fprintf(stderr, + "pthread_create mcount_sort_thread failed '%s': %s\n", + strerror(errno), fname); + goto out; + } +#endif + + if (custom_sort) { + custom_sort(extab_image, shdr_size(extab_sec)); + } else { + int num_entries = shdr_size(extab_sec) / extable_ent_size; + qsort(extab_image, num_entries, + extable_ent_size, compare_extable); + } + + /* find the flag main_extable_sort_needed */ + sym_start = (void *)ehdr + shdr_offset(symtab_sec); + sym_end = sym_start + shdr_size(symtab_sec); + symentsize = shdr_entsize(symtab_sec); + + for (sym = sym_start; (void *)sym + symentsize < sym_end; + sym = (void *)sym + symentsize) { + if (sym_type(sym) != STT_OBJECT) + continue; + if (!strcmp(strtab + sym_name(sym), + "main_extable_sort_needed")) { + sort_needed_sym = sym; + break; + } + } + + if (!sort_needed_sym) { + fprintf(stderr, + "no main_extable_sort_needed symbol in file: %s\n", + fname); + goto out; + } + + sort_need_index = get_secindex(sym_shndx(sym), + ((void *)sort_needed_sym - (void *)symtab) / symentsize, + symtab_shndx); + sort_needed_sec = get_index(shdr_start, shentsize, sort_need_index); + sort_needed_loc = (void *)ehdr + + shdr_offset(sort_needed_sec) + + sym_value(sort_needed_sym) - shdr_addr(sort_needed_sec); + + /* extable has been sorted, clear the flag */ + w(0, sort_needed_loc); + rc = 0; + +out: +#ifdef UNWINDER_ORC_ENABLED + if (orc_sort_thread) { + void *retval = NULL; + /* wait for ORC tables sort done */ + rc = pthread_join(orc_sort_thread, &retval); + if (rc) { + fprintf(stderr, + "pthread_join failed '%s': %s\n", + strerror(errno), fname); + } else if (retval) { + rc = -1; + fprintf(stderr, + "failed to sort ORC tables '%s': %s\n", + (char *)retval, fname); + } + } +#endif + +#ifdef MCOUNT_SORT_ENABLED + if (mcount_sort_thread) { + void *retval = NULL; + /* wait for mcount sort done */ + rc = pthread_join(mcount_sort_thread, &retval); + if (rc) { + fprintf(stderr, + "pthread_join failed '%s': %s\n", + strerror(errno), fname); + } else if (retval) { + rc = -1; + fprintf(stderr, + "failed to sort mcount '%s': %s\n", + (char *)retval, fname); + } + } +#endif + return rc; +} static int compare_relative_table(const void *a, const void *b) { @@ -267,17 +1220,15 @@ static void sort_relative_table_with_data(char *extab_image, int image_size) static int do_file(char const *const fname, void *addr) { - int rc = -1; - Elf32_Ehdr *ehdr = addr; + Elf_Ehdr *ehdr = addr; table_sort_t custom_sort = NULL; - switch (ehdr->e_ident[EI_DATA]) { + switch (ehdr->e32.e_ident[EI_DATA]) { case ELFDATA2LSB: r = rle; r2 = r2le; r8 = r8le; w = wle; - w2 = w2le; w8 = w8le; break; case ELFDATA2MSB: @@ -285,25 +1236,31 @@ static int do_file(char const *const fname, void *addr) r2 = r2be; r8 = r8be; w = wbe; - w2 = w2be; w8 = w8be; break; default: fprintf(stderr, "unrecognized ELF data encoding %d: %s\n", - ehdr->e_ident[EI_DATA], fname); + ehdr->e32.e_ident[EI_DATA], fname); return -1; } - if (memcmp(ELFMAG, ehdr->e_ident, SELFMAG) != 0 || - (r2(&ehdr->e_type) != ET_EXEC && r2(&ehdr->e_type) != ET_DYN) || - ehdr->e_ident[EI_VERSION] != EV_CURRENT) { + if (memcmp(ELFMAG, ehdr->e32.e_ident, SELFMAG) != 0 || + (r2(&ehdr->e32.e_type) != ET_EXEC && r2(&ehdr->e32.e_type) != ET_DYN) || + ehdr->e32.e_ident[EI_VERSION] != EV_CURRENT) { fprintf(stderr, "unrecognized ET_EXEC/ET_DYN file %s\n", fname); return -1; } - switch (r2(&ehdr->e_machine)) { - case EM_386: + switch (r2(&ehdr->e32.e_machine)) { case EM_AARCH64: +#ifdef MCOUNT_SORT_ENABLED + sort_reloc = true; + rela_type = 0x403; + /* arm64 uses patchable function entry placing before function */ + before_func = 8; +#endif + /* fallthrough */ + case EM_386: case EM_LOONGARCH: case EM_RISCV: case EM_S390: @@ -324,40 +1281,93 @@ static int do_file(char const *const fname, void *addr) break; default: fprintf(stderr, "unrecognized e_machine %d %s\n", - r2(&ehdr->e_machine), fname); + r2(&ehdr->e32.e_machine), fname); return -1; } - switch (ehdr->e_ident[EI_CLASS]) { - case ELFCLASS32: - if (r2(&ehdr->e_ehsize) != sizeof(Elf32_Ehdr) || - r2(&ehdr->e_shentsize) != sizeof(Elf32_Shdr)) { + switch (ehdr->e32.e_ident[EI_CLASS]) { + case ELFCLASS32: { + struct elf_funcs efuncs = { + .compare_extable = compare_extable_32, + .ehdr_shoff = ehdr32_shoff, + .ehdr_shentsize = ehdr32_shentsize, + .ehdr_shstrndx = ehdr32_shstrndx, + .ehdr_shnum = ehdr32_shnum, + .shdr_addr = shdr32_addr, + .shdr_offset = shdr32_offset, + .shdr_link = shdr32_link, + .shdr_size = shdr32_size, + .shdr_name = shdr32_name, + .shdr_type = shdr32_type, + .shdr_entsize = shdr32_entsize, + .sym_type = sym32_type, + .sym_name = sym32_name, + .sym_value = sym32_value, + .sym_shndx = sym32_shndx, + .rela_offset = rela32_offset, + .rela_info = rela32_info, + .rela_addend = rela32_addend, + .rela_write_addend = rela32_write_addend, + }; + + e = efuncs; + long_size = 4; + extable_ent_size = 8; + + if (r2(&ehdr->e32.e_ehsize) != sizeof(Elf32_Ehdr) || + r2(&ehdr->e32.e_shentsize) != sizeof(Elf32_Shdr)) { fprintf(stderr, "unrecognized ET_EXEC/ET_DYN file: %s\n", fname); - break; + return -1; + } + } - rc = do_sort_32(ehdr, fname, custom_sort); break; - case ELFCLASS64: - { - Elf64_Ehdr *const ghdr = (Elf64_Ehdr *)ehdr; - if (r2(&ghdr->e_ehsize) != sizeof(Elf64_Ehdr) || - r2(&ghdr->e_shentsize) != sizeof(Elf64_Shdr)) { + case ELFCLASS64: { + struct elf_funcs efuncs = { + .compare_extable = compare_extable_64, + .ehdr_shoff = ehdr64_shoff, + .ehdr_shentsize = ehdr64_shentsize, + .ehdr_shstrndx = ehdr64_shstrndx, + .ehdr_shnum = ehdr64_shnum, + .shdr_addr = shdr64_addr, + .shdr_offset = shdr64_offset, + .shdr_link = shdr64_link, + .shdr_size = shdr64_size, + .shdr_name = shdr64_name, + .shdr_type = shdr64_type, + .shdr_entsize = shdr64_entsize, + .sym_type = sym64_type, + .sym_name = sym64_name, + .sym_value = sym64_value, + .sym_shndx = sym64_shndx, + .rela_offset = rela64_offset, + .rela_info = rela64_info, + .rela_addend = rela64_addend, + .rela_write_addend = rela64_write_addend, + }; + + e = efuncs; + long_size = 8; + extable_ent_size = 16; + + if (r2(&ehdr->e64.e_ehsize) != sizeof(Elf64_Ehdr) || + r2(&ehdr->e64.e_shentsize) != sizeof(Elf64_Shdr)) { fprintf(stderr, "unrecognized ET_EXEC/ET_DYN file: %s\n", fname); - break; + return -1; } - rc = do_sort_64(ghdr, fname, custom_sort); + } break; default: fprintf(stderr, "unrecognized ELF class %d %s\n", - ehdr->e_ident[EI_CLASS], fname); - break; + ehdr->e32.e_ident[EI_CLASS], fname); + return -1; } - return rc; + return do_sort(ehdr, fname, custom_sort); } int main(int argc, char *argv[]) @@ -365,14 +1375,29 @@ int main(int argc, char *argv[]) int i, n_error = 0; /* gcc-4.3.0 false positive complaint */ size_t size = 0; void *addr = NULL; + int c; + + while ((c = getopt(argc, argv, "s:")) >= 0) { + switch (c) { + case 's': + if (parse_symbols(optarg) < 0) { + fprintf(stderr, "Could not parse %s\n", optarg); + return -1; + } + break; + default: + fprintf(stderr, "usage: sorttable [-s nm-file] vmlinux...\n"); + return 0; + } + } - if (argc < 2) { + if ((argc - optind) < 1) { fprintf(stderr, "usage: sorttable vmlinux...\n"); return 0; } /* Process each file in turn, allowing deep failure. */ - for (i = 1; i < argc; i++) { + for (i = optind; i < argc; i++) { addr = mmap_file(argv[i], &size); if (!addr) { ++n_error; diff --git a/scripts/sorttable.h b/scripts/sorttable.h deleted file mode 100644 index a7c5445baf00..000000000000 --- a/scripts/sorttable.h +++ /dev/null @@ -1,500 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * sorttable.h - * - * Added ORC unwind tables sort support and other updates: - * Copyright (C) 1999-2019 Alibaba Group Holding Limited. by: - * Shile Zhang - * - * Copyright 2011 - 2012 Cavium, Inc. - * - * Some of code was taken out of arch/x86/kernel/unwind_orc.c, written by: - * Copyright (C) 2017 Josh Poimboeuf - * - * Some of this code was taken out of recordmcount.h written by: - * - * Copyright 2009 John F. Reiser . All rights reserved. - * Copyright 2010 Steven Rostedt , Red Hat Inc. - */ - -#undef extable_ent_size -#undef compare_extable -#undef get_mcount_loc -#undef sort_mcount_loc -#undef elf_mcount_loc -#undef do_sort -#undef Elf_Addr -#undef Elf_Ehdr -#undef Elf_Shdr -#undef Elf_Rel -#undef Elf_Rela -#undef Elf_Sym -#undef ELF_R_SYM -#undef Elf_r_sym -#undef ELF_R_INFO -#undef Elf_r_info -#undef ELF_ST_BIND -#undef ELF_ST_TYPE -#undef fn_ELF_R_SYM -#undef fn_ELF_R_INFO -#undef uint_t -#undef _r -#undef _w - -#ifdef SORTTABLE_64 -# define extable_ent_size 16 -# define compare_extable compare_extable_64 -# define get_mcount_loc get_mcount_loc_64 -# define sort_mcount_loc sort_mcount_loc_64 -# define elf_mcount_loc elf_mcount_loc_64 -# define do_sort do_sort_64 -# define Elf_Addr Elf64_Addr -# define Elf_Ehdr Elf64_Ehdr -# define Elf_Shdr Elf64_Shdr -# define Elf_Rel Elf64_Rel -# define Elf_Rela Elf64_Rela -# define Elf_Sym Elf64_Sym -# define ELF_R_SYM ELF64_R_SYM -# define Elf_r_sym Elf64_r_sym -# define ELF_R_INFO ELF64_R_INFO -# define Elf_r_info Elf64_r_info -# define ELF_ST_BIND ELF64_ST_BIND -# define ELF_ST_TYPE ELF64_ST_TYPE -# define fn_ELF_R_SYM fn_ELF64_R_SYM -# define fn_ELF_R_INFO fn_ELF64_R_INFO -# define uint_t uint64_t -# define _r r8 -# define _w w8 -#else -# define extable_ent_size 8 -# define compare_extable compare_extable_32 -# define get_mcount_loc get_mcount_loc_32 -# define sort_mcount_loc sort_mcount_loc_32 -# define elf_mcount_loc elf_mcount_loc_32 -# define do_sort do_sort_32 -# define Elf_Addr Elf32_Addr -# define Elf_Ehdr Elf32_Ehdr -# define Elf_Shdr Elf32_Shdr -# define Elf_Rel Elf32_Rel -# define Elf_Rela Elf32_Rela -# define Elf_Sym Elf32_Sym -# define ELF_R_SYM ELF32_R_SYM -# define Elf_r_sym Elf32_r_sym -# define ELF_R_INFO ELF32_R_INFO -# define Elf_r_info Elf32_r_info -# define ELF_ST_BIND ELF32_ST_BIND -# define ELF_ST_TYPE ELF32_ST_TYPE -# define fn_ELF_R_SYM fn_ELF32_R_SYM -# define fn_ELF_R_INFO fn_ELF32_R_INFO -# define uint_t uint32_t -# define _r r -# define _w w -#endif - -#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) -/* ORC unwinder only support X86_64 */ -#include - -#define ERRSTR_MAXSZ 256 - -char g_err[ERRSTR_MAXSZ]; -int *g_orc_ip_table; -struct orc_entry *g_orc_table; - -pthread_t orc_sort_thread; - -static inline unsigned long orc_ip(const int *ip) -{ - return (unsigned long)ip + *ip; -} - -static int orc_sort_cmp(const void *_a, const void *_b) -{ - struct orc_entry *orc_a, *orc_b; - const int *a = g_orc_ip_table + *(int *)_a; - const int *b = g_orc_ip_table + *(int *)_b; - unsigned long a_val = orc_ip(a); - unsigned long b_val = orc_ip(b); - - if (a_val > b_val) - return 1; - if (a_val < b_val) - return -1; - - /* - * The "weak" section terminator entries need to always be on the left - * to ensure the lookup code skips them in favor of real entries. - * These terminator entries exist to handle any gaps created by - * whitelisted .o files which didn't get objtool generation. - */ - orc_a = g_orc_table + (a - g_orc_ip_table); - orc_b = g_orc_table + (b - g_orc_ip_table); - if (orc_a->type == ORC_TYPE_UNDEFINED && orc_b->type == ORC_TYPE_UNDEFINED) - return 0; - return orc_a->type == ORC_TYPE_UNDEFINED ? -1 : 1; -} - -static void *sort_orctable(void *arg) -{ - int i; - int *idxs = NULL; - int *tmp_orc_ip_table = NULL; - struct orc_entry *tmp_orc_table = NULL; - unsigned int *orc_ip_size = (unsigned int *)arg; - unsigned int num_entries = *orc_ip_size / sizeof(int); - unsigned int orc_size = num_entries * sizeof(struct orc_entry); - - idxs = (int *)malloc(*orc_ip_size); - if (!idxs) { - snprintf(g_err, ERRSTR_MAXSZ, "malloc idxs: %s", - strerror(errno)); - pthread_exit(g_err); - } - - tmp_orc_ip_table = (int *)malloc(*orc_ip_size); - if (!tmp_orc_ip_table) { - snprintf(g_err, ERRSTR_MAXSZ, "malloc tmp_orc_ip_table: %s", - strerror(errno)); - pthread_exit(g_err); - } - - tmp_orc_table = (struct orc_entry *)malloc(orc_size); - if (!tmp_orc_table) { - snprintf(g_err, ERRSTR_MAXSZ, "malloc tmp_orc_table: %s", - strerror(errno)); - pthread_exit(g_err); - } - - /* initialize indices array, convert ip_table to absolute address */ - for (i = 0; i < num_entries; i++) { - idxs[i] = i; - tmp_orc_ip_table[i] = g_orc_ip_table[i] + i * sizeof(int); - } - memcpy(tmp_orc_table, g_orc_table, orc_size); - - qsort(idxs, num_entries, sizeof(int), orc_sort_cmp); - - for (i = 0; i < num_entries; i++) { - if (idxs[i] == i) - continue; - - /* convert back to relative address */ - g_orc_ip_table[i] = tmp_orc_ip_table[idxs[i]] - i * sizeof(int); - g_orc_table[i] = tmp_orc_table[idxs[i]]; - } - - free(idxs); - free(tmp_orc_ip_table); - free(tmp_orc_table); - pthread_exit(NULL); -} -#endif - -static int compare_extable(const void *a, const void *b) -{ - Elf_Addr av = _r(a); - Elf_Addr bv = _r(b); - - if (av < bv) - return -1; - if (av > bv) - return 1; - return 0; -} -#ifdef MCOUNT_SORT_ENABLED -pthread_t mcount_sort_thread; - -struct elf_mcount_loc { - Elf_Ehdr *ehdr; - Elf_Shdr *init_data_sec; - uint_t start_mcount_loc; - uint_t stop_mcount_loc; -}; - -/* Sort the addresses stored between __start_mcount_loc to __stop_mcount_loc in vmlinux */ -static void *sort_mcount_loc(void *arg) -{ - struct elf_mcount_loc *emloc = (struct elf_mcount_loc *)arg; - uint_t offset = emloc->start_mcount_loc - _r(&(emloc->init_data_sec)->sh_addr) - + _r(&(emloc->init_data_sec)->sh_offset); - uint_t count = emloc->stop_mcount_loc - emloc->start_mcount_loc; - unsigned char *start_loc = (void *)emloc->ehdr + offset; - - qsort(start_loc, count/sizeof(uint_t), sizeof(uint_t), compare_extable); - return NULL; -} - -/* Get the address of __start_mcount_loc and __stop_mcount_loc in System.map */ -static void get_mcount_loc(uint_t *_start, uint_t *_stop) -{ - FILE *file_start, *file_stop; - char start_buff[20]; - char stop_buff[20]; - int len = 0; - - file_start = popen(" grep start_mcount System.map | awk '{print $1}' ", "r"); - if (!file_start) { - fprintf(stderr, "get start_mcount_loc error!"); - return; - } - - file_stop = popen(" grep stop_mcount System.map | awk '{print $1}' ", "r"); - if (!file_stop) { - fprintf(stderr, "get stop_mcount_loc error!"); - pclose(file_start); - return; - } - - while (fgets(start_buff, sizeof(start_buff), file_start) != NULL) { - len = strlen(start_buff); - start_buff[len - 1] = '\0'; - } - *_start = strtoul(start_buff, NULL, 16); - - while (fgets(stop_buff, sizeof(stop_buff), file_stop) != NULL) { - len = strlen(stop_buff); - stop_buff[len - 1] = '\0'; - } - *_stop = strtoul(stop_buff, NULL, 16); - - pclose(file_start); - pclose(file_stop); -} -#endif -static int do_sort(Elf_Ehdr *ehdr, - char const *const fname, - table_sort_t custom_sort) -{ - int rc = -1; - Elf_Shdr *s, *shdr = (Elf_Shdr *)((char *)ehdr + _r(&ehdr->e_shoff)); - Elf_Shdr *strtab_sec = NULL; - Elf_Shdr *symtab_sec = NULL; - Elf_Shdr *extab_sec = NULL; - Elf_Sym *sym; - const Elf_Sym *symtab; - Elf32_Word *symtab_shndx = NULL; - Elf_Sym *sort_needed_sym = NULL; - Elf_Shdr *sort_needed_sec; - Elf_Rel *relocs = NULL; - int relocs_size = 0; - uint32_t *sort_needed_loc; - const char *secstrings; - const char *strtab; - char *extab_image; - int extab_index = 0; - int i; - int idx; - unsigned int shnum; - unsigned int shstrndx; -#ifdef MCOUNT_SORT_ENABLED - struct elf_mcount_loc mstruct = {0}; - uint_t _start_mcount_loc = 0; - uint_t _stop_mcount_loc = 0; -#endif -#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) - unsigned int orc_ip_size = 0; - unsigned int orc_size = 0; - unsigned int orc_num_entries = 0; -#endif - - shstrndx = r2(&ehdr->e_shstrndx); - if (shstrndx == SHN_XINDEX) - shstrndx = r(&shdr[0].sh_link); - secstrings = (const char *)ehdr + _r(&shdr[shstrndx].sh_offset); - - shnum = r2(&ehdr->e_shnum); - if (shnum == SHN_UNDEF) - shnum = _r(&shdr[0].sh_size); - - for (i = 0, s = shdr; s < shdr + shnum; i++, s++) { - idx = r(&s->sh_name); - if (!strcmp(secstrings + idx, "__ex_table")) { - extab_sec = s; - extab_index = i; - } - if (!strcmp(secstrings + idx, ".symtab")) - symtab_sec = s; - if (!strcmp(secstrings + idx, ".strtab")) - strtab_sec = s; - - if ((r(&s->sh_type) == SHT_REL || - r(&s->sh_type) == SHT_RELA) && - r(&s->sh_info) == extab_index) { - relocs = (void *)ehdr + _r(&s->sh_offset); - relocs_size = _r(&s->sh_size); - } - if (r(&s->sh_type) == SHT_SYMTAB_SHNDX) - symtab_shndx = (Elf32_Word *)((const char *)ehdr + - _r(&s->sh_offset)); - -#ifdef MCOUNT_SORT_ENABLED - /* locate the .init.data section in vmlinux */ - if (!strcmp(secstrings + idx, ".init.data")) { - get_mcount_loc(&_start_mcount_loc, &_stop_mcount_loc); - mstruct.ehdr = ehdr; - mstruct.init_data_sec = s; - mstruct.start_mcount_loc = _start_mcount_loc; - mstruct.stop_mcount_loc = _stop_mcount_loc; - } -#endif - -#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) - /* locate the ORC unwind tables */ - if (!strcmp(secstrings + idx, ".orc_unwind_ip")) { - orc_ip_size = s->sh_size; - g_orc_ip_table = (int *)((void *)ehdr + - s->sh_offset); - } - if (!strcmp(secstrings + idx, ".orc_unwind")) { - orc_size = s->sh_size; - g_orc_table = (struct orc_entry *)((void *)ehdr + - s->sh_offset); - } -#endif - } /* for loop */ - -#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) - if (!g_orc_ip_table || !g_orc_table) { - fprintf(stderr, - "incomplete ORC unwind tables in file: %s\n", fname); - goto out; - } - - orc_num_entries = orc_ip_size / sizeof(int); - if (orc_ip_size % sizeof(int) != 0 || - orc_size % sizeof(struct orc_entry) != 0 || - orc_num_entries != orc_size / sizeof(struct orc_entry)) { - fprintf(stderr, - "inconsistent ORC unwind table entries in file: %s\n", - fname); - goto out; - } - - /* create thread to sort ORC unwind tables concurrently */ - if (pthread_create(&orc_sort_thread, NULL, - sort_orctable, &orc_ip_size)) { - fprintf(stderr, - "pthread_create orc_sort_thread failed '%s': %s\n", - strerror(errno), fname); - goto out; - } -#endif - -#ifdef MCOUNT_SORT_ENABLED - if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) { - fprintf(stderr, - "incomplete mcount's sort in file: %s\n", - fname); - goto out; - } - - /* create thread to sort mcount_loc concurrently */ - if (pthread_create(&mcount_sort_thread, NULL, &sort_mcount_loc, &mstruct)) { - fprintf(stderr, - "pthread_create mcount_sort_thread failed '%s': %s\n", - strerror(errno), fname); - goto out; - } -#endif - if (!extab_sec) { - fprintf(stderr, "no __ex_table in file: %s\n", fname); - goto out; - } - - if (!symtab_sec) { - fprintf(stderr, "no .symtab in file: %s\n", fname); - goto out; - } - - if (!strtab_sec) { - fprintf(stderr, "no .strtab in file: %s\n", fname); - goto out; - } - - extab_image = (void *)ehdr + _r(&extab_sec->sh_offset); - strtab = (const char *)ehdr + _r(&strtab_sec->sh_offset); - symtab = (const Elf_Sym *)((const char *)ehdr + - _r(&symtab_sec->sh_offset)); - - if (custom_sort) { - custom_sort(extab_image, _r(&extab_sec->sh_size)); - } else { - int num_entries = _r(&extab_sec->sh_size) / extable_ent_size; - qsort(extab_image, num_entries, - extable_ent_size, compare_extable); - } - - /* If there were relocations, we no longer need them. */ - if (relocs) - memset(relocs, 0, relocs_size); - - /* find the flag main_extable_sort_needed */ - for (sym = (void *)ehdr + _r(&symtab_sec->sh_offset); - sym < sym + _r(&symtab_sec->sh_size) / sizeof(Elf_Sym); - sym++) { - if (ELF_ST_TYPE(sym->st_info) != STT_OBJECT) - continue; - if (!strcmp(strtab + r(&sym->st_name), - "main_extable_sort_needed")) { - sort_needed_sym = sym; - break; - } - } - - if (!sort_needed_sym) { - fprintf(stderr, - "no main_extable_sort_needed symbol in file: %s\n", - fname); - goto out; - } - - sort_needed_sec = &shdr[get_secindex(r2(&sym->st_shndx), - sort_needed_sym - symtab, - symtab_shndx)]; - sort_needed_loc = (void *)ehdr + - _r(&sort_needed_sec->sh_offset) + - _r(&sort_needed_sym->st_value) - - _r(&sort_needed_sec->sh_addr); - - /* extable has been sorted, clear the flag */ - w(0, sort_needed_loc); - rc = 0; - -out: -#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) - if (orc_sort_thread) { - void *retval = NULL; - /* wait for ORC tables sort done */ - rc = pthread_join(orc_sort_thread, &retval); - if (rc) { - fprintf(stderr, - "pthread_join failed '%s': %s\n", - strerror(errno), fname); - } else if (retval) { - rc = -1; - fprintf(stderr, - "failed to sort ORC tables '%s': %s\n", - (char *)retval, fname); - } - } -#endif - -#ifdef MCOUNT_SORT_ENABLED - if (mcount_sort_thread) { - void *retval = NULL; - /* wait for mcount sort done */ - rc = pthread_join(mcount_sort_thread, &retval); - if (rc) { - fprintf(stderr, - "pthread_join failed '%s': %s\n", - strerror(errno), fname); - } else if (retval) { - rc = -1; - fprintf(stderr, - "failed to sort mcount '%s': %s\n", - (char *)retval, fname); - } - } -#endif - return rc; -} diff --git a/security/apparmor/include/policy_unpack.h b/security/apparmor/include/policy_unpack.h index e5a95dc4da1f..b9de0fdf9ee5 100644 --- a/security/apparmor/include/policy_unpack.h +++ b/security/apparmor/include/policy_unpack.h @@ -163,6 +163,25 @@ aa_get_profile_loaddata(struct aa_loaddata *data) return data; } +/** + * aa_get_profile_loaddata_not0 - get a profile reference count if not zero + * @data: reference to get a count on + * + * Like aa_get_profile_loaddata(), but safe to call on an entry that may + * be on a list (e.g. ns->rawdata_list) where the last pcount has already + * dropped and the deferred cleanup has not yet run. + * + * Returns: pointer to reference, or %NULL if @data is NULL or its + * profile refcount has already reached zero. + */ +static inline struct aa_loaddata * +aa_get_profile_loaddata_not0(struct aa_loaddata *data) +{ + if (data && kref_get_unless_zero(&data->pcount)) + return data; + return NULL; +} + void __aa_loaddata_update(struct aa_loaddata *data, long revision); bool aa_rawdata_eq(struct aa_loaddata *l, struct aa_loaddata *r); void aa_loaddata_kref(struct kref *kref); diff --git a/security/apparmor/lsm.c b/security/apparmor/lsm.c index 0182b09cc8a0..89c79fd241ba 100644 --- a/security/apparmor/lsm.c +++ b/security/apparmor/lsm.c @@ -1207,7 +1207,21 @@ static int aa_sock_msg_perm(const char *op, u32 request, struct socket *sock, static int apparmor_socket_sendmsg(struct socket *sock, struct msghdr *msg, int size) { - return aa_sock_msg_perm(OP_SENDMSG, AA_MAY_SEND, sock, msg, size); + int error = aa_sock_msg_perm(OP_SENDMSG, AA_MAY_SEND, sock, msg, size); + + if (error) + return error; + + /* TCP fast open carries connect() semantics in sendmsg(); mediate + * the implicit connect so it cannot bypass the connect permission. + */ + if ((msg->msg_flags & MSG_FASTOPEN) && msg->msg_name && + (sk_is_tcp(sock->sk) || + (sk_is_inet(sock->sk) && sock->sk->sk_type == SOCK_STREAM && + sock->sk->sk_protocol == IPPROTO_MPTCP))) + error = aa_sk_perm(OP_CONNECT, AA_MAY_CONNECT, sock->sk); + + return error; } static int apparmor_socket_recvmsg(struct socket *sock, diff --git a/security/apparmor/policy.c b/security/apparmor/policy.c index b5ae0314b384..9140b613c0cf 100644 --- a/security/apparmor/policy.c +++ b/security/apparmor/policy.c @@ -1175,8 +1175,12 @@ ssize_t aa_replace_profiles(struct aa_ns *policy_ns, struct aa_label *label, if (aa_rawdata_eq(rawdata_ent, udata)) { struct aa_loaddata *tmp; - tmp = aa_get_profile_loaddata(rawdata_ent); - /* check we didn't fail the race */ + /* + * Entries remain on rawdata_list with + * pcount == 0 until do_ploaddata_rmfs() + * runs; only take a live profile ref. + */ + tmp = aa_get_profile_loaddata_not0(rawdata_ent); if (tmp) { aa_put_profile_loaddata(udata); udata = tmp; diff --git a/security/keys/internal.h b/security/keys/internal.h index 2cffa6dc8255..b7b622bc36a1 100644 --- a/security/keys/internal.h +++ b/security/keys/internal.h @@ -208,6 +208,8 @@ extern struct key *request_key_auth_new(struct key *target, const void *callout_info, size_t callout_len, struct key *dest_keyring); +struct request_key_auth *request_key_auth_get(struct key *authkey); +void request_key_auth_put(struct request_key_auth *rka); extern struct key *key_get_instantiation_authkey(key_serial_t target_id); diff --git a/security/keys/keyctl.c b/security/keys/keyctl.c index ab927a142f51..11bedca4e1c6 100644 --- a/security/keys/keyctl.c +++ b/security/keys/keyctl.c @@ -1197,9 +1197,13 @@ static long keyctl_instantiate_key_common(key_serial_t id, if (!instkey) goto error; - rka = instkey->payload.data[0]; - if (rka->target_key->serial != id) + rka = request_key_auth_get(instkey); + if (!rka) { + ret = -EKEYREVOKED; goto error; + } + if (rka->target_key->serial != id) + goto error_put_rka; /* pull the payload in if one was supplied */ payload = NULL; @@ -1208,7 +1212,7 @@ static long keyctl_instantiate_key_common(key_serial_t id, ret = -ENOMEM; payload = kvmalloc(plen, GFP_KERNEL); if (!payload) - goto error; + goto error_put_rka; ret = -EFAULT; if (!copy_from_iter_full(payload, plen, from)) @@ -1234,6 +1238,8 @@ static long keyctl_instantiate_key_common(key_serial_t id, error2: kvfree_sensitive(payload, plen); +error_put_rka: + request_key_auth_put(rka); error: return ret; } @@ -1358,15 +1364,19 @@ long keyctl_reject_key(key_serial_t id, unsigned timeout, unsigned error, if (!instkey) goto error; - rka = instkey->payload.data[0]; - if (rka->target_key->serial != id) + rka = request_key_auth_get(instkey); + if (!rka) { + ret = -EKEYREVOKED; goto error; + } + if (rka->target_key->serial != id) + goto error_put_rka; /* find the destination keyring if present (which must also be * writable) */ ret = get_instantiation_keyring(ringid, rka, &dest_keyring); if (ret < 0) - goto error; + goto error_put_rka; /* instantiate the key and link it into a keyring */ ret = key_reject_and_link(rka->target_key, timeout, error, @@ -1379,6 +1389,8 @@ long keyctl_reject_key(key_serial_t id, unsigned timeout, unsigned error, if (ret == 0) keyctl_change_reqkey_auth(NULL); +error_put_rka: + request_key_auth_put(rka); error: return ret; } diff --git a/security/keys/keyctl_pkey.c b/security/keys/keyctl_pkey.c index 97bc27bbf079..ba150ee2d4a3 100644 --- a/security/keys/keyctl_pkey.c +++ b/security/keys/keyctl_pkey.c @@ -138,28 +138,35 @@ static int keyctl_pkey_params_get_2(const struct keyctl_pkey_params __user *_par if (uparams.in_len > info.max_dec_size || uparams.out_len > info.max_enc_size) return -EINVAL; + + params->out_len = info.max_enc_size; break; case KEYCTL_PKEY_DECRYPT: if (uparams.in_len > info.max_enc_size || uparams.out_len > info.max_dec_size) return -EINVAL; + + params->out_len = info.max_dec_size; break; case KEYCTL_PKEY_SIGN: if (uparams.in_len > info.max_data_size || uparams.out_len > info.max_sig_size) return -EINVAL; + + params->out_len = info.max_sig_size; break; case KEYCTL_PKEY_VERIFY: if (uparams.in_len > info.max_data_size || uparams.in2_len > info.max_sig_size) return -EINVAL; + + params->out_len = info.max_sig_size; break; default: BUG(); } params->in_len = uparams.in_len; - params->out_len = uparams.out_len; /* Note: same as in2_len */ return 0; } diff --git a/security/keys/request_key_auth.c b/security/keys/request_key_auth.c index 8f33cd170e42..bd34317b58cc 100644 --- a/security/keys/request_key_auth.c +++ b/security/keys/request_key_auth.c @@ -23,6 +23,7 @@ static void request_key_auth_describe(const struct key *, struct seq_file *); static void request_key_auth_revoke(struct key *); static void request_key_auth_destroy(struct key *); static long request_key_auth_read(const struct key *, char *, size_t); +static void request_key_auth_rcu_disposal(struct rcu_head *); /* * The request-key authorisation key type definition. @@ -115,6 +116,31 @@ static void free_request_key_auth(struct request_key_auth *rka) kfree(rka); } +/* + * Take a reference to the request-key authorisation payload so callers can + * drop authkey->sem before doing operations that may sleep. + */ +struct request_key_auth *request_key_auth_get(struct key *authkey) +{ + struct request_key_auth *rka; + + down_read(&authkey->sem); + rka = dereference_key_locked(authkey); + if (rka && !test_bit(KEY_FLAG_REVOKED, &authkey->flags)) + refcount_inc(&rka->usage); + else + rka = NULL; + up_read(&authkey->sem); + + return rka; +} + +void request_key_auth_put(struct request_key_auth *rka) +{ + if (rka && refcount_dec_and_test(&rka->usage)) + call_rcu(&rka->rcu, request_key_auth_rcu_disposal); +} + /* * Dispose of the request_key_auth record under RCU conditions */ @@ -136,8 +162,10 @@ static void request_key_auth_revoke(struct key *key) struct request_key_auth *rka = dereference_key_locked(key); kenter("{%d}", key->serial); + if (!rka) + return; rcu_assign_keypointer(key, NULL); - call_rcu(&rka->rcu, request_key_auth_rcu_disposal); + request_key_auth_put(rka); } /* @@ -150,7 +178,7 @@ static void request_key_auth_destroy(struct key *key) kenter("{%d}", key->serial); if (rka) { rcu_assign_keypointer(key, NULL); - call_rcu(&rka->rcu, request_key_auth_rcu_disposal); + request_key_auth_put(rka); } } @@ -174,6 +202,7 @@ struct key *request_key_auth_new(struct key *target, const char *op, rka = kzalloc(sizeof(*rka), GFP_KERNEL); if (!rka) goto error; + refcount_set(&rka->usage, 1); rka->callout_info = kmemdup(callout_info, callout_len, GFP_KERNEL); if (!rka->callout_info) goto error_free_rka; diff --git a/security/security.c b/security/security.c index 6e4deac6ec07..dd6b922c12de 100644 --- a/security/security.c +++ b/security/security.c @@ -95,6 +95,7 @@ const char *const lockdown_reasons[LOCKDOWN_CONFIDENTIALITY_MAX + 1] = { static BLOCKING_NOTIFIER_HEAD(blocking_lsm_notifier_chain); static struct kmem_cache *lsm_file_cache; +static struct kmem_cache *lsm_backing_file_cache; static struct kmem_cache *lsm_inode_cache; char *lsm_names; @@ -266,6 +267,7 @@ static void __init lsm_set_blob_sizes(struct lsm_blob_sizes *needed) lsm_set_blob_size(&needed->lbs_cred, &blob_sizes.lbs_cred); lsm_set_blob_size(&needed->lbs_file, &blob_sizes.lbs_file); + lsm_set_blob_size(&needed->lbs_backing_file, &blob_sizes.lbs_backing_file); lsm_set_blob_size(&needed->lbs_ib, &blob_sizes.lbs_ib); /* * The inode blob gets an rcu_head in addition to @@ -468,6 +470,7 @@ static void __init ordered_lsm_init(void) init_debug("cred blob size = %d\n", blob_sizes.lbs_cred); init_debug("file blob size = %d\n", blob_sizes.lbs_file); + init_debug("lsm_backing_file_cache = %d\n", blob_sizes.lbs_backing_file); init_debug("ib blob size = %d\n", blob_sizes.lbs_ib); init_debug("inode blob size = %d\n", blob_sizes.lbs_inode); init_debug("ipc blob size = %d\n", blob_sizes.lbs_ipc); @@ -490,6 +493,11 @@ static void __init ordered_lsm_init(void) lsm_file_cache = kmem_cache_create("lsm_file_cache", blob_sizes.lbs_file, 0, SLAB_PANIC, NULL); + if (blob_sizes.lbs_backing_file) + lsm_backing_file_cache = kmem_cache_create( + "lsm_backing_file_cache", + blob_sizes.lbs_backing_file, + 0, SLAB_PANIC, NULL); if (blob_sizes.lbs_inode) lsm_inode_cache = kmem_cache_create("lsm_inode_cache", blob_sizes.lbs_inode, 0, @@ -666,6 +674,30 @@ int unregister_blocking_lsm_notifier(struct notifier_block *nb) } EXPORT_SYMBOL(unregister_blocking_lsm_notifier); +/** + * lsm_backing_file_alloc - allocate a composite backing file blob + * @backing_file: the backing file + * + * Allocate the backing file blob for all the modules. + * + * Returns 0, or -ENOMEM if memory can't be allocated. + */ +static int lsm_backing_file_alloc(struct file *backing_file) +{ + void *blob; + + if (!lsm_backing_file_cache) { + backing_file_set_security(backing_file, NULL); + return 0; + } + + blob = kmem_cache_zalloc(lsm_backing_file_cache, GFP_KERNEL); + backing_file_set_security(backing_file, blob); + if (!blob) + return -ENOMEM; + return 0; +} + /** * lsm_blob_alloc - allocate a composite blob * @dest: the destination for the blob @@ -2893,6 +2925,57 @@ void security_file_free(struct file *file) } } +/** + * security_backing_file_alloc() - Allocate and setup a backing file blob + * @backing_file: the backing file + * @user_file: the associated user visible file + * + * Allocate a backing file LSM blob and perform any necessary initialization of + * the LSM blob. There will be some operations where the LSM will not have + * access to @user_file after this point, so any important state associated + * with @user_file that is important to the LSM should be captured in the + * backing file's LSM blob. + * + * LSM's should avoid taking a reference to @user_file in this hook as it will + * result in problems later when the system attempts to drop/put the file + * references due to a circular dependency. + * + * Return: Return 0 if the hook is successful, negative values otherwise. + */ +int security_backing_file_alloc(struct file *backing_file, + const struct file *user_file) +{ + int rc; + + rc = lsm_backing_file_alloc(backing_file); + if (rc) + return rc; + rc = call_int_hook(backing_file_alloc, backing_file, user_file); + if (unlikely(rc)) + security_backing_file_free(backing_file); + + return rc; +} + +/** + * security_backing_file_free() - Free a backing file blob + * @backing_file: the backing file + * + * Free any LSM state associate with a backing file's LSM blob, including the + * blob itself. + */ +void security_backing_file_free(struct file *backing_file) +{ + void *blob = backing_file_security(backing_file); + + call_void_hook(backing_file_free, backing_file); + + if (blob) { + backing_file_set_security(backing_file, NULL); + kmem_cache_free(lsm_backing_file_cache, blob); + } +} + /** * security_file_ioctl() - Check if an ioctl is allowed * @file: associated file @@ -2981,6 +3064,32 @@ int security_mmap_file(struct file *file, unsigned long prot, flags); } +/** + * security_mmap_backing_file - Check if mmap'ing a backing file is allowed + * @vma: the vm_area_struct for the mmap'd region + * @backing_file: the backing file being mmap'd + * @user_file: the user file being mmap'd + * + * Check permissions for a mmap operation on a stacked filesystem. This hook + * is called after the security_mmap_file() and is responsible for authorizing + * the mmap on @backing_file. It is important to note that the mmap operation + * on @user_file has already been authorized and the @vma->vm_file has been + * set to @backing_file. + * + * Return: Returns 0 if permission is granted. + */ +int security_mmap_backing_file(struct vm_area_struct *vma, + struct file *backing_file, + struct file *user_file) +{ + /* recommended by the stackable filesystem devs */ + if (WARN_ON_ONCE(!(backing_file->f_mode & FMODE_BACKING))) + return -EIO; + + return call_int_hook(mmap_backing_file, vma, backing_file, user_file); +} +EXPORT_SYMBOL_GPL(security_mmap_backing_file); + /** * security_mmap_addr() - Check if mmap'ing an address is allowed * @addr: address diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index 8e31d3b60fc6..1b89c8d5fa2f 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -1724,49 +1724,72 @@ static inline int file_path_has_perm(const struct cred *cred, static int bpf_fd_pass(const struct file *file, u32 sid); #endif -/* Check whether a task can use an open file descriptor to - access an inode in a given way. Check access to the - descriptor itself, and then use dentry_has_perm to - check a particular permission to the file. - Access to the descriptor is implicitly granted if it - has the same SID as the process. If av is zero, then - access to the file is not checked, e.g. for cases - where only the descriptor is affected like seek. */ -static int file_has_perm(const struct cred *cred, - struct file *file, - u32 av) +static int __file_has_perm(const struct cred *cred, const struct file *file, + u32 av, bool bf_user_file) + { - struct file_security_struct *fsec = selinux_file(file); - struct inode *inode = file_inode(file); struct common_audit_data ad; - u32 sid = cred_sid(cred); + struct inode *inode; + u32 ssid = cred_sid(cred); + u32 tsid_fd; int rc; - ad.type = LSM_AUDIT_DATA_FILE; - ad.u.file = file; + if (bf_user_file) { + struct backing_file_security_struct *bfsec; + const struct path *path; - if (sid != fsec->sid) { - rc = avc_has_perm(sid, fsec->sid, - SECCLASS_FD, - FD__USE, - &ad); + if (WARN_ON(!(file->f_mode & FMODE_BACKING))) + return -EIO; + + bfsec = selinux_backing_file(file); + path = backing_file_user_path(file); + tsid_fd = bfsec->uf_sid; + inode = d_inode(path->dentry); + + ad.type = LSM_AUDIT_DATA_PATH; + ad.u.path = *path; + } else { + struct file_security_struct *fsec = selinux_file(file); + + tsid_fd = fsec->sid; + inode = file_inode(file); + + ad.type = LSM_AUDIT_DATA_FILE; + ad.u.file = file; + } + + if (ssid != tsid_fd) { + rc = avc_has_perm(ssid, tsid_fd, SECCLASS_FD, FD__USE, &ad); if (rc) - goto out; + return rc; } #ifdef CONFIG_BPF_SYSCALL - rc = bpf_fd_pass(file, cred_sid(cred)); + /* regardless of backing vs user file, use the underlying file here */ + rc = bpf_fd_pass(file, ssid); if (rc) return rc; #endif /* av is zero if only checking access to the descriptor. */ - rc = 0; if (av) - rc = inode_has_perm(cred, inode, av, &ad); + return inode_has_perm(cred, inode, av, &ad); -out: - return rc; + return 0; +} + +/* Check whether a task can use an open file descriptor to + access an inode in a given way. Check access to the + descriptor itself, and then use dentry_has_perm to + check a particular permission to the file. + Access to the descriptor is implicitly granted if it + has the same SID as the process. If av is zero, then + access to the file is not checked, e.g. for cases + where only the descriptor is affected like seek. */ +static inline int file_has_perm(const struct cred *cred, + const struct file *file, u32 av) +{ + return __file_has_perm(cred, file, av, false); } /* @@ -3653,6 +3676,17 @@ static int selinux_file_alloc_security(struct file *file) return 0; } +static int selinux_backing_file_alloc(struct file *backing_file, + const struct file *user_file) +{ + struct backing_file_security_struct *bfsec; + + bfsec = selinux_backing_file(backing_file); + bfsec->uf_sid = selinux_file(user_file)->sid; + + return 0; +} + /* * Check whether a task has the ioctl permission and cmd * operation to an inode. @@ -3770,42 +3804,55 @@ static int selinux_file_ioctl_compat(struct file *file, unsigned int cmd, static int default_noexec __ro_after_init; -static int file_map_prot_check(struct file *file, unsigned long prot, int shared) +static int __file_map_prot_check(const struct cred *cred, + const struct file *file, unsigned long prot, + bool shared, bool bf_user_file) { - const struct cred *cred = current_cred(); - u32 sid = cred_sid(cred); - int rc = 0; + struct inode *inode = NULL; + bool prot_exec = prot & PROT_EXEC; + bool prot_write = prot & PROT_WRITE; + + if (file) { + if (bf_user_file) + inode = d_inode(backing_file_user_path(file)->dentry); + else + inode = file_inode(file); + } + + if (default_noexec && prot_exec && + (!file || IS_PRIVATE(inode) || (!shared && prot_write))) { + int rc; + u32 sid = cred_sid(cred); - if (default_noexec && - (prot & PROT_EXEC) && (!file || IS_PRIVATE(file_inode(file)) || - (!shared && (prot & PROT_WRITE)))) { /* - * We are making executable an anonymous mapping or a - * private file mapping that will also be writable. - * This has an additional check. + * We are making executable an anonymous mapping or a private + * file mapping that will also be writable. */ - rc = avc_has_perm(sid, sid, SECCLASS_PROCESS, - PROCESS__EXECMEM, NULL); + rc = avc_has_perm(sid, sid, SECCLASS_PROCESS, PROCESS__EXECMEM, + NULL); if (rc) - goto error; + return rc; } if (file) { - /* read access is always possible with a mapping */ + /* "read" always possible, "write" only if shared */ u32 av = FILE__READ; - - /* write access only matters if the mapping is shared */ - if (shared && (prot & PROT_WRITE)) + if (shared && prot_write) av |= FILE__WRITE; - - if (prot & PROT_EXEC) + if (prot_exec) av |= FILE__EXECUTE; - return file_has_perm(cred, file, av); + return __file_has_perm(cred, file, av, bf_user_file); } -error: - return rc; + return 0; +} + +static inline int file_map_prot_check(const struct cred *cred, + const struct file *file, + unsigned long prot, bool shared) +{ + return __file_map_prot_check(cred, file, prot, shared, false); } static int selinux_mmap_addr(unsigned long addr) @@ -3821,36 +3868,80 @@ static int selinux_mmap_addr(unsigned long addr) return rc; } -static int selinux_mmap_file(struct file *file, - unsigned long reqprot __always_unused, - unsigned long prot, unsigned long flags) +static int selinux_mmap_file_common(const struct cred *cred, struct file *file, + unsigned long prot, bool shared) { - struct common_audit_data ad; - int rc; - if (file) { + int rc; + struct common_audit_data ad; + ad.type = LSM_AUDIT_DATA_FILE; ad.u.file = file; - rc = inode_has_perm(current_cred(), file_inode(file), - FILE__MAP, &ad); + rc = inode_has_perm(cred, file_inode(file), FILE__MAP, &ad); if (rc) return rc; } - return file_map_prot_check(file, prot, - (flags & MAP_TYPE) == MAP_SHARED); + return file_map_prot_check(cred, file, prot, shared); +} + +static int selinux_mmap_file(struct file *file, + unsigned long reqprot __always_unused, + unsigned long prot, unsigned long flags) +{ + return selinux_mmap_file_common(current_cred(), file, prot, + (flags & MAP_TYPE) == MAP_SHARED); +} + +/** + * selinux_mmap_backing_file - Check mmap permissions on a backing file + * @vma: memory region + * @backing_file: stacked filesystem backing file + * @user_file: user visible file + * + * This is called after selinux_mmap_file() on stacked filesystems, and it + * is this function's responsibility to verify access to @backing_file and + * setup the SELinux state for possible later use in the mprotect() code path. + * + * By the time this function is called, mmap() access to @user_file has already + * been authorized and @vma->vm_file has been set to point to @backing_file. + * + * Return zero on success, negative values otherwise. + */ +static int selinux_mmap_backing_file(struct vm_area_struct *vma, + struct file *backing_file, + struct file *user_file __always_unused) +{ + unsigned long prot = 0; + + /* translate vma->vm_flags perms into PROT perms */ + if (vma->vm_flags & VM_READ) + prot |= PROT_READ; + if (vma->vm_flags & VM_WRITE) + prot |= PROT_WRITE; + if (vma->vm_flags & VM_EXEC) + prot |= PROT_EXEC; + + return selinux_mmap_file_common(backing_file->f_cred, backing_file, + prot, vma->vm_flags & VM_SHARED); } static int selinux_file_mprotect(struct vm_area_struct *vma, unsigned long reqprot __always_unused, unsigned long prot) { + int rc; const struct cred *cred = current_cred(); u32 sid = cred_sid(cred); + const struct file *file = vma->vm_file; + bool backing_file; + bool shared = vma->vm_flags & VM_SHARED; + + /* check if we need to trigger the "backing files are awful" mode */ + backing_file = file && (file->f_mode & FMODE_BACKING); if (default_noexec && (prot & PROT_EXEC) && !(vma->vm_flags & VM_EXEC)) { - int rc = 0; /* * We don't use the vma_is_initial_heap() helper as it has * a history of problems and is currently broken on systems @@ -3864,11 +3955,15 @@ static int selinux_file_mprotect(struct vm_area_struct *vma, vma->vm_end <= vma->vm_mm->brk) { rc = avc_has_perm(sid, sid, SECCLASS_PROCESS, PROCESS__EXECHEAP, NULL); - } else if (!vma->vm_file && (vma_is_initial_stack(vma) || + if (rc) + return rc; + } else if (!file && (vma_is_initial_stack(vma) || vma_is_stack_for_current(vma))) { rc = avc_has_perm(sid, sid, SECCLASS_PROCESS, PROCESS__EXECSTACK, NULL); - } else if (vma->vm_file && vma->anon_vma) { + if (rc) + return rc; + } else if (file && vma->anon_vma) { /* * We are making executable a file mapping that has * had some COW done. Since pages might have been @@ -3876,13 +3971,29 @@ static int selinux_file_mprotect(struct vm_area_struct *vma, * modified content. This typically should only * occur for text relocations. */ - rc = file_has_perm(cred, vma->vm_file, FILE__EXECMOD); + rc = __file_has_perm(cred, file, FILE__EXECMOD, + backing_file); + if (rc) + return rc; + if (backing_file) { + rc = file_has_perm(file->f_cred, file, + FILE__EXECMOD); + if (rc) + return rc; + } } + } + + rc = __file_map_prot_check(cred, file, prot, shared, backing_file); + if (rc) + return rc; + if (backing_file) { + rc = file_map_prot_check(file->f_cred, file, prot, shared); if (rc) return rc; } - return file_map_prot_check(vma->vm_file, prot, vma->vm_flags&VM_SHARED); + return 0; } static int selinux_file_lock(struct file *file, unsigned int cmd) @@ -6960,6 +7071,7 @@ static void selinux_bpf_token_free(struct bpf_token *token) struct lsm_blob_sizes selinux_blob_sizes __ro_after_init = { .lbs_cred = sizeof(struct task_security_struct), .lbs_file = sizeof(struct file_security_struct), + .lbs_backing_file = sizeof(struct backing_file_security_struct), .lbs_inode = sizeof(struct inode_security_struct), .lbs_ipc = sizeof(struct ipc_security_struct), .lbs_key = sizeof(struct key_security_struct), @@ -7165,9 +7277,11 @@ static struct security_hook_list selinux_hooks[] __ro_after_init = { LSM_HOOK_INIT(file_permission, selinux_file_permission), LSM_HOOK_INIT(file_alloc_security, selinux_file_alloc_security), + LSM_HOOK_INIT(backing_file_alloc, selinux_backing_file_alloc), LSM_HOOK_INIT(file_ioctl, selinux_file_ioctl), LSM_HOOK_INIT(file_ioctl_compat, selinux_file_ioctl_compat), LSM_HOOK_INIT(mmap_file, selinux_mmap_file), + LSM_HOOK_INIT(mmap_backing_file, selinux_mmap_backing_file), LSM_HOOK_INIT(mmap_addr, selinux_mmap_addr), LSM_HOOK_INIT(file_mprotect, selinux_file_mprotect), LSM_HOOK_INIT(file_lock, selinux_file_lock), diff --git a/security/selinux/include/objsec.h b/security/selinux/include/objsec.h index c88cae81ee4c..dc42282a2c05 100644 --- a/security/selinux/include/objsec.h +++ b/security/selinux/include/objsec.h @@ -61,6 +61,10 @@ struct file_security_struct { u32 pseqno; /* Policy seqno at the time of file open */ }; +struct backing_file_security_struct { + u32 uf_sid; /* associated user file fsec->sid */ +}; + struct superblock_security_struct { u32 sid; /* SID of file system superblock */ u32 def_sid; /* default SID for labeling */ @@ -159,6 +163,13 @@ static inline struct file_security_struct *selinux_file(const struct file *file) return file->f_security + selinux_blob_sizes.lbs_file; } +static inline struct backing_file_security_struct * +selinux_backing_file(const struct file *backing_file) +{ + void *blob = backing_file_security(backing_file); + return blob + selinux_blob_sizes.lbs_backing_file; +} + static inline struct inode_security_struct * selinux_inode(const struct inode *inode) { diff --git a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c index 960c9323d1e0..4183c2c057a3 100644 --- a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c @@ -6,6 +6,7 @@ #include "kprobe_multi_override.skel.h" #include "kprobe_multi_session.skel.h" #include "kprobe_multi_session_cookie.skel.h" +#include "kprobe_multi_sleepable.skel.h" #include "bpf/libbpf_internal.h" #include "bpf/hashmap.h" @@ -216,7 +217,9 @@ static void test_attach_api_syms(void) static void test_attach_api_fails(void) { LIBBPF_OPTS(bpf_kprobe_multi_opts, opts); + LIBBPF_OPTS(bpf_test_run_opts, topts); struct kprobe_multi *skel = NULL; + struct kprobe_multi_sleepable *sl_skel = NULL; struct bpf_link *link = NULL; unsigned long long addrs[2]; const char *syms[2] = { @@ -224,7 +227,7 @@ static void test_attach_api_fails(void) "bpf_fentry_test2", }; __u64 cookies[2]; - int saved_error; + int saved_error, err; addrs[0] = ksym_get_addr("bpf_fentry_test1"); addrs[1] = ksym_get_addr("bpf_fentry_test2"); @@ -323,9 +326,39 @@ static void test_attach_api_fails(void) if (!ASSERT_EQ(saved_error, -E2BIG, "fail_6_error")) goto cleanup; + /* fail_9 - sleepable kprobe multi should not attach */ + sl_skel = kprobe_multi_sleepable__open(); + if (!ASSERT_OK_PTR(sl_skel, "sleep_skel_open")) + goto cleanup; + + sl_skel->bss->user_ptr = sl_skel; + + err = bpf_program__set_flags(sl_skel->progs.handle_kprobe_multi_sleepable, + BPF_F_SLEEPABLE); + if (!ASSERT_OK(err, "sleep_skel_set_flags")) + goto cleanup; + + err = kprobe_multi_sleepable__load(sl_skel); + if (!ASSERT_OK(err, "sleep_skel_load")) + goto cleanup; + + link = bpf_program__attach_kprobe_multi_opts(sl_skel->progs.handle_kprobe_multi_sleepable, + "bpf_fentry_test1", NULL); + saved_error = -errno; + + if (!ASSERT_ERR_PTR(link, "fail_9")) + goto cleanup; + + if (!ASSERT_EQ(saved_error, -EINVAL, "fail_9_error")) + goto cleanup; + + err = bpf_prog_test_run_opts(bpf_program__fd(sl_skel->progs.fentry), &topts); + ASSERT_OK(err, "bpf_prog_test_run_opts"); + cleanup: bpf_link__destroy(link); kprobe_multi__destroy(skel); + kprobe_multi_sleepable__destroy(sl_skel); } static void test_session_skel_api(void) diff --git a/tools/testing/selftests/bpf/progs/kprobe_multi_sleepable.c b/tools/testing/selftests/bpf/progs/kprobe_multi_sleepable.c new file mode 100644 index 000000000000..932e1d9c72e2 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/kprobe_multi_sleepable.c @@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "vmlinux.h" +#include +#include + +void *user_ptr = 0; + +SEC("kprobe.multi") +int handle_kprobe_multi_sleepable(struct pt_regs *ctx) +{ + int a, err; + + err = bpf_copy_from_user(&a, sizeof(a), user_ptr); + barrier_var(a); + return err; +} + +SEC("fentry/bpf_fentry_test1") +int BPF_PROG(fentry) +{ + return 0; +} + +char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/ptp/testptp.c b/tools/testing/selftests/ptp/testptp.c index e0aed424fe42..edc08a4433fd 100644 --- a/tools/testing/selftests/ptp/testptp.c +++ b/tools/testing/selftests/ptp/testptp.c @@ -147,6 +147,7 @@ static void usage(char *progname) " -T val set the ptp clock time to 'val' seconds\n" " -x val get an extended ptp clock time with the desired number of samples (up to %d)\n" " -X get a ptp clock cross timestamp\n" + " -y val pre/post tstamp timebase to use {realtime|monotonic|monotonic-raw}\n" " -z test combinations of rising/falling external time stamp flags\n", progname, PTP_MAX_SAMPLES); } @@ -191,6 +192,7 @@ int main(int argc, char *argv[]) int readonly = 0; int settime = 0; int channel = -1; + clockid_t ext_clockid = CLOCK_REALTIME; int64_t t1, t2, tp; int64_t interval, offset; @@ -200,7 +202,7 @@ int main(int argc, char *argv[]) progname = strrchr(argv[0], '/'); progname = progname ? 1+progname : argv[0]; - while (EOF != (c = getopt(argc, argv, "cd:e:f:F:ghH:i:k:lL:n:o:p:P:rsSt:T:w:x:Xz"))) { + while (EOF != (c = getopt(argc, argv, "cd:e:f:F:ghH:i:k:lL:n:o:p:P:rsSt:T:w:x:Xy:z"))) { switch (c) { case 'c': capabilities = 1; @@ -283,6 +285,21 @@ int main(int argc, char *argv[]) case 'X': getcross = 1; break; + case 'y': + if (!strcasecmp(optarg, "realtime")) + ext_clockid = CLOCK_REALTIME; + else if (!strcasecmp(optarg, "monotonic")) + ext_clockid = CLOCK_MONOTONIC; + else if (!strcasecmp(optarg, "monotonic-raw")) + ext_clockid = CLOCK_MONOTONIC_RAW; + else { + fprintf(stderr, + "type needs to be realtime, monotonic or monotonic-raw; was given %s\n", + optarg); + return -1; + } + break; + case 'z': flagtest = 1; break; @@ -575,6 +592,7 @@ int main(int argc, char *argv[]) } soe->n_samples = getextended; + soe->clockid = ext_clockid; if (ioctl(fd, PTP_SYS_OFFSET_EXTENDED, soe)) { perror("PTP_SYS_OFFSET_EXTENDED"); @@ -583,12 +601,46 @@ int main(int argc, char *argv[]) getextended); for (i = 0; i < getextended; i++) { - printf("sample #%2d: system time before: %lld.%09u\n", - i, soe->ts[i][0].sec, soe->ts[i][0].nsec); + switch (ext_clockid) { + case CLOCK_REALTIME: + printf("sample #%2d: real time before: %lld.%09u\n", + i, soe->ts[i][0].sec, + soe->ts[i][0].nsec); + break; + case CLOCK_MONOTONIC: + printf("sample #%2d: monotonic time before: %lld.%09u\n", + i, soe->ts[i][0].sec, + soe->ts[i][0].nsec); + break; + case CLOCK_MONOTONIC_RAW: + printf("sample #%2d: monotonic-raw time before: %lld.%09u\n", + i, soe->ts[i][0].sec, + soe->ts[i][0].nsec); + break; + default: + break; + } printf(" phc time: %lld.%09u\n", soe->ts[i][1].sec, soe->ts[i][1].nsec); - printf(" system time after: %lld.%09u\n", - soe->ts[i][2].sec, soe->ts[i][2].nsec); + switch (ext_clockid) { + case CLOCK_REALTIME: + printf(" real time after: %lld.%09u\n", + soe->ts[i][2].sec, + soe->ts[i][2].nsec); + break; + case CLOCK_MONOTONIC: + printf(" monotonic time after: %lld.%09u\n", + soe->ts[i][2].sec, + soe->ts[i][2].nsec); + break; + case CLOCK_MONOTONIC_RAW: + printf(" monotonic-raw time after: %lld.%09u\n", + soe->ts[i][2].sec, + soe->ts[i][2].nsec); + break; + default: + break; + } } } diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c index ee4d5e4e68d7..c9c9ccdb0e7e 100644 --- a/virt/kvm/eventfd.c +++ b/virt/kvm/eventfd.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include @@ -744,21 +745,18 @@ ioeventfd_in_range(struct _ioeventfd *p, gpa_t addr, int len, const void *val) return true; /* otherwise, we have to actually compare the data */ - - BUG_ON(!IS_ALIGNED((unsigned long)val, len)); - switch (len) { case 1: - _val = *(u8 *)val; + _val = get_unaligned((u8 *)val); break; case 2: - _val = *(u16 *)val; + _val = get_unaligned((u16 *)val); break; case 4: - _val = *(u32 *)val; + _val = get_unaligned((u32 *)val); break; case 8: - _val = *(u64 *)val; + _val = get_unaligned((u64 *)val); break; default: return false;