From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49DBA3793BD; Sun, 9 Aug 2026 18:36:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786300618; cv=none; b=qfwm0oKt/BprX/pwtMqhq+2akAykwsfiOwdSSbDgxavBy7ep4iOa0pCFCFOFMkEj9iBKlNqCXub8fM4uaLRyGs6tw+ldaX0jJbfbibI6FvYp1hi3Scj2t5+SjamYpzT7qzisJQKoqihOJ0APtIH7BVd8QL0YSj0swqzVFJAD3m8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786300618; c=relaxed/simple; bh=e10bdWECtFkxz0gCalUtGNxg58aSJSVC1gxx7tKXCHg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Wq5vaipS4jCVIE/pbsqEAYA6NBuh6wSvOHNFOHLCF1XT3wp5P/yL+axQWT3k3SR4WoOQ1mb3PDYqb5VeVsZtWdXT2fwmoxWciiHFfKW0hwZS30vU3dNvIMkzrWo5fEMaJ8ER8Pyce/ed4eGNXGu9GNkJPggAueqQF/OG91a+EoQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=CKvmUwIY; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="CKvmUwIY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D2EE91F00A3A; Sun, 9 Aug 2026 18:36:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1786300591; bh=qILMqDKcUnClJd8rnWjmUKTIcxNgsHVJt9kkHTWYfDM=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=CKvmUwIYDo35lhAelvd9QSvnxim253XY2j74LDxwMzVVsN4kLUDvJ3XeA+R1ERgbi 5k2tZF5vW1iHqtJ/99PrwOB+yfcy5d1KrUDQMvZTuMRmQb2UqIbLhNFGPhZmdGL41s yd+EJ5mYnQOQFNWUBQPTsceRzl0ZcnEP7FZUt1Bo= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org, akpm@linux-foundation.org, torvalds@linux-foundation.org, stable@vger.kernel.org Cc: lwn@lwn.net, jslaby@suse.cz, Greg Kroah-Hartman Subject: Re: Linux 6.12.103 Date: Sun, 9 Aug 2026 20:34:45 +0200 Message-ID: <2026080945-cytoplast-stubborn-0bb3@gregkh> X-Mailer: git-send-email 2.55.0 In-Reply-To: <2026080944-sediment-diocese-e9c3@gregkh> References: <2026080944-sediment-diocese-e9c3@gregkh> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit diff --git a/Documentation/dev-tools/kmemleak.rst b/Documentation/dev-tools/kmemleak.rst index 2cb00b53339f..7d784e03f3f9 100644 --- a/Documentation/dev-tools/kmemleak.rst +++ b/Documentation/dev-tools/kmemleak.rst @@ -161,6 +161,7 @@ See the include/linux/kmemleak.h header for the functions prototype. - ``kmemleak_free_percpu`` - notify of a percpu memory block freeing - ``kmemleak_update_trace`` - update object allocation stack trace - ``kmemleak_not_leak`` - mark an object as not a leak +- ``kmemleak_transient_leak`` - mark an object as a transient leak - ``kmemleak_ignore`` - do not scan or report an object as leak - ``kmemleak_scan_area`` - add scan areas inside a memory block - ``kmemleak_no_scan`` - do not scan a memory block diff --git a/Documentation/hwmon/ina2xx.rst b/Documentation/hwmon/ina2xx.rst index 7f1939b40f74..d64e7af46a12 100644 --- a/Documentation/hwmon/ina2xx.rst +++ b/Documentation/hwmon/ina2xx.rst @@ -53,6 +53,37 @@ Supported chips: https://www.ti.com/ + * Texas Instruments INA260 + + Prefix: 'ina260' + + Addresses: I2C 0x40 - 0x4f + + Datasheet: Publicly available at the Texas Instruments website + + https://www.ti.com/ + + * Silergy SY24655 + + Prefix: 'sy24655' + + Addresses: I2C 0x40 - 0x4f + + Datasheet: Publicly available at the Silergy website + + https://us1.silergy.com/ + + + * Texas Instruments INA234 + + Prefix: 'ina234' + + Addresses: I2C 0x40 - 0x43 + + Datasheet: Publicly available at the Texas Instruments website + + https://www.ti.com/ + Author: Lothar Felten Description @@ -68,10 +99,18 @@ interface. The INA220 monitors both shunt drop and supply voltage. The INA226 is a current shunt and power monitor with an I2C interface. The INA226 monitors both a shunt voltage drop and bus supply voltage. -INA230 and INA231 are high or low side current shunt and power monitors +INA230, INA231, and INA234 are high or low side current shunt and power monitors with an I2C interface. The chips monitor both a shunt voltage drop and bus supply voltage. +INA260 is a high or low side current and power monitor with integrated shunt +resistor. + +The SY24655 is a high- and low-side current shunt and power monitor with an I2C +interface. The SY24655 supports both shunt drop and supply voltage, with +programmable calibration value and conversion times. The SY24655 can also +calculate average power for use in energy conversion. + The shunt value in micro-ohms can be set via platform data or device tree at compile-time or via the shunt_resistor attribute in sysfs at run-time. Please refer to the Documentation/devicetree/bindings/hwmon/ti,ina2xx.yaml for bindings @@ -87,16 +126,25 @@ The actual programmed interval may vary from the desired value. General sysfs entries --------------------- -======================= =============================== +======================= =============================================== in0_input Shunt voltage(mV) channel in1_input Bus voltage(mV) channel curr1_input Current(mA) measurement channel power1_input Power(uW) measurement channel -shunt_resistor Shunt resistance(uOhm) channel -======================= =============================== +shunt_resistor Shunt resistance(uOhm) channel (not for ina260) +======================= =============================================== + +Additional sysfs entries +------------------------ -Sysfs entries for ina226, ina230 and ina231 only ------------------------------------------------- +Additional entries are available for the following chips: + + * ina226 + * ina230 + * ina231 + * ina234 + * ina260 + * sy24655 ======================= ==================================================== curr1_lcrit Critical low current @@ -117,6 +165,13 @@ update_interval data conversion time; affects number of samples used to average results for shunt and bus voltages. ======================= ==================================================== +Sysfs entries for sy24655 only +------------------------------ + +======================= ==================================================== +power1_average average power from last reading to the present. +======================= ==================================================== + .. note:: - Configure `shunt_resistor` before configure `power1_crit`, because power diff --git a/Makefile b/Makefile index 115a476fefe8..f3bacdc021a4 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 VERSION = 6 PATCHLEVEL = 12 -SUBLEVEL = 102 +SUBLEVEL = 103 EXTRAVERSION = NAME = Baby Opossum Posse diff --git a/arch/powerpc/boot/simpleboot.c b/arch/powerpc/boot/simpleboot.c index c80691d83880..27591df41e9e 100644 --- a/arch/powerpc/boot/simpleboot.c +++ b/arch/powerpc/boot/simpleboot.c @@ -68,7 +68,7 @@ void platform_init(unsigned long r3, unsigned long r4, unsigned long r5, /* finally, setup the timebase */ node = fdt_node_offset_by_prop_value(_dtb_start, -1, "device_type", "cpu", sizeof("cpu")); - if (!node) + if (node < 0) fatal("Cannot find cpu node\n"); timebase = fdt_getprop(_dtb_start, node, "timebase-frequency", &size); if (timebase && (size == 4)) diff --git a/arch/powerpc/boot/treeboot-akebono.c b/arch/powerpc/boot/treeboot-akebono.c index e3cc2599869c..1b529037480f 100644 --- a/arch/powerpc/boot/treeboot-akebono.c +++ b/arch/powerpc/boot/treeboot-akebono.c @@ -146,7 +146,7 @@ void platform_init(char *userdata) node = fdt_node_offset_by_prop_value(_dtb_start, -1, "device_type", "cpu", sizeof("cpu")); - if (!node) + if (node < 0) fatal("Cannot find cpu node\n"); timebase = fdt_getprop(_dtb_start, node, "timebase-frequency", &size); if (timebase && (size == 4)) diff --git a/arch/powerpc/boot/treeboot-currituck.c b/arch/powerpc/boot/treeboot-currituck.c index d53e8a592f81..5b5363b74f9f 100644 --- a/arch/powerpc/boot/treeboot-currituck.c +++ b/arch/powerpc/boot/treeboot-currituck.c @@ -102,7 +102,7 @@ void platform_init(void) node = fdt_node_offset_by_prop_value(_dtb_start, -1, "device_type", "cpu", sizeof("cpu")); - if (!node) + if (node < 0) fatal("Cannot find cpu node\n"); timebase = fdt_getprop(_dtb_start, node, "timebase-frequency", &size); if (timebase && (size == 4)) diff --git a/arch/powerpc/platforms/ps3/mm.c b/arch/powerpc/platforms/ps3/mm.c index 1326de55fda6..e04952bb4360 100644 --- a/arch/powerpc/platforms/ps3/mm.c +++ b/arch/powerpc/platforms/ps3/mm.c @@ -615,6 +615,7 @@ static int dma_ioc0_map_pages(struct ps3_dma_region *r, unsigned long phys_addr, fail_map: for (iopage--; 0 <= iopage; iopage--) { + offset = (1 << r->page_size) * iopage; lv1_put_iopte(0, c->bus_addr + offset, c->lpar_addr + offset, diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 4594f77a06ee..e52f7dabb370 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -1726,7 +1726,7 @@ static void __meminit remove_pud_mapping(pud_t *pud_base, unsigned long addr, un for (; addr < end; addr = next) { next = pud_addr_end(addr, end); - pudp = pud_base + pud_index(addr); + pudp = pgtable_l4_enabled ? pud_base + pud_index(addr) : pud_base; pud = pudp_get(pudp); if (!pud_present(pud)) continue; @@ -1757,7 +1757,7 @@ static void __meminit remove_p4d_mapping(p4d_t *p4d_base, unsigned long addr, un for (; addr < end; addr = next) { next = p4d_addr_end(addr, end); - p4dp = p4d_base + p4d_index(addr); + p4dp = pgtable_l5_enabled ? p4d_base + p4d_index(addr) : p4d_base; p4d = p4dp_get(p4dp); if (!p4d_present(p4d)) continue; diff --git a/arch/s390/kvm/pci.c b/arch/s390/kvm/pci.c index 6e6726b2a441..17fedfc7bb99 100644 --- a/arch/s390/kvm/pci.c +++ b/arch/s390/kvm/pci.c @@ -225,7 +225,7 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, bool assist) { struct page *pages[1], *aibv_page, *aisb_page = NULL; - unsigned int msi_vecs, idx; + unsigned int msi_vecs, idx, size; struct zpci_gaite *gaite; unsigned long hva, bit; struct kvm *kvm; @@ -239,6 +239,10 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, if (zdev->gisa == 0) return -EINVAL; + /* AIF already enabled for the device */ + if (zdev->kzdev->fib.fmt0.aibv != 0) + return -EINVAL; + kvm = zdev->kzdev->kvm; msi_vecs = min_t(unsigned int, fib->fmt0.noi, zdev->max_msi); @@ -248,6 +252,14 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, return gisc; /* Replace AIBV address */ + size = BITS_TO_LONGS(msi_vecs + fib->fmt0.aibvo) * sizeof(unsigned long); + npages = DIV_ROUND_UP((fib->fmt0.aibv & ~PAGE_MASK) + size, PAGE_SIZE); + /* AIBV cannot span more than 1 page */ + if (npages > 1) { + rc = -EINVAL; + goto out; + } + idx = srcu_read_lock(&kvm->srcu); hva = gfn_to_hva(kvm, gpa_to_gfn((gpa_t)fib->fmt0.aibv)); npages = pin_user_pages_fast(hva, 1, FOLL_WRITE | FOLL_LONGTERM, pages); @@ -263,6 +275,12 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, /* Pin the guest AISB if one was specified */ if (fib->fmt0.sum == 1) { + /* AISB must be dword aligned */ + if (fib->fmt0.aisb & 0x7) { + rc = -EINVAL; + goto unpin1; + } + idx = srcu_read_lock(&kvm->srcu); hva = gfn_to_hva(kvm, gpa_to_gfn((gpa_t)fib->fmt0.aisb)); npages = pin_user_pages_fast(hva, 1, FOLL_WRITE | FOLL_LONGTERM, @@ -291,6 +309,11 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, AIRQ_IV_GUESTVEC, phys_to_virt(fib->fmt0.aibv)); + if (!zdev->aibv) { + rc = -ENOMEM; + goto free_aisb; + } + spin_lock_irq(&aift->gait_lock); gaite = aift->gait + zdev->aisb; @@ -327,6 +350,9 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, rc = kvm_zpci_set_airq(zdev); return rc; +free_aisb: + airq_iv_free_bit(aift->sbv, zdev->aisb); + zdev->aisb = 0; unlock: mutex_unlock(&aift->aift_lock); unpin2: diff --git a/arch/s390/pci/pci_mmio.c b/arch/s390/pci/pci_mmio.c index 0fa34c501296..969e2a23eb21 100644 --- a/arch/s390/pci/pci_mmio.c +++ b/arch/s390/pci/pci_mmio.c @@ -178,6 +178,7 @@ SYSCALL_DEFINE3(s390_pci_mmio_write, unsigned long, mmio_addr, goto out_unlock_mmap; } + ret = -EFAULT; io_addr = (void __iomem *)((args.pfn << PAGE_SHIFT) | (mmio_addr & ~PAGE_MASK)); diff --git a/arch/um/drivers/chan_user.c b/arch/um/drivers/chan_user.c index a66e556012c4..35f9beeb19b3 100644 --- a/arch/um/drivers/chan_user.c +++ b/arch/um/drivers/chan_user.c @@ -161,6 +161,8 @@ static __noreturn int winch_thread(void *arg) int count; char c = 1; + os_set_pdeathsig(); + pty_fd = data->pty_fd; pipe_fd = data->pipe_fd; count = write(pipe_fd, &c, sizeof(c)); diff --git a/arch/um/drivers/ubd_kern.c b/arch/um/drivers/ubd_kern.c index 2bfb17373244..66c1a8835e36 100644 --- a/arch/um/drivers/ubd_kern.c +++ b/arch/um/drivers/ubd_kern.c @@ -1501,6 +1501,7 @@ int io_thread(void *arg) { int n, count, written, res; + os_set_pdeathsig(); os_fix_helper_signals(); while(1){ diff --git a/arch/um/drivers/vector_kern.c b/arch/um/drivers/vector_kern.c index 7a88b13d289f..3f11f665ff2c 100644 --- a/arch/um/drivers/vector_kern.c +++ b/arch/um/drivers/vector_kern.c @@ -999,6 +999,9 @@ static int vector_mmsg_rx(struct vector_private *vp, int budget) */ dev_kfree_skb_irq(skb); vp->estats.rx_encaps_errors++; + (*skbuff_vector) = NULL; + mmsg_vector++; + skbuff_vector++; continue; } if (header_check > 0) { diff --git a/arch/um/include/shared/os.h b/arch/um/include/shared/os.h index 77a8593f219a..2a0843cc2208 100644 --- a/arch/um/include/shared/os.h +++ b/arch/um/include/shared/os.h @@ -219,6 +219,8 @@ extern int os_unmap_memory(void *addr, int len); extern int os_drop_memory(void *addr, int length); extern int can_drop_memory(void); +void os_set_pdeathsig(void); + /* execvp.c */ extern int execvp_noalloc(char *buf, const char *file, char *const argv[]); /* helper.c */ diff --git a/arch/um/os-Linux/process.c b/arch/um/os-Linux/process.c index 2686120ab232..4015d3dc3958 100644 --- a/arch/um/os-Linux/process.c +++ b/arch/um/os-Linux/process.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -234,3 +235,8 @@ void init_new_thread_signals(void) set_handler(SIGIO); signal(SIGWINCH, SIG_IGN); } + +void os_set_pdeathsig(void) +{ + prctl(PR_SET_PDEATHSIG, SIGKILL); +} diff --git a/arch/um/os-Linux/sigio.c b/arch/um/os-Linux/sigio.c index 9e71794839e8..9aac8def4d63 100644 --- a/arch/um/os-Linux/sigio.c +++ b/arch/um/os-Linux/sigio.c @@ -55,6 +55,7 @@ static int write_sigio_thread(void *unused) int i, n, respond_fd; char c; + os_set_pdeathsig(); os_fix_helper_signals(); fds = ¤t_poll; while (1) { diff --git a/arch/um/os-Linux/signal.c b/arch/um/os-Linux/signal.c index b11ed66c8bb0..94abc9e3ec7e 100644 --- a/arch/um/os-Linux/signal.c +++ b/arch/um/os-Linux/signal.c @@ -190,6 +190,7 @@ static void hard_handler(int sig, siginfo_t *si, void *p) { ucontext_t *uc = p; mcontext_t *mc = &uc->uc_mcontext; + int save_errno = errno; unsigned long pending = 1UL << sig; do { @@ -227,6 +228,8 @@ static void hard_handler(int sig, siginfo_t *si, void *p) if (!nested) pending = from_irq_stack(nested); } while (pending); + + errno = save_errno; } void set_handler(int sig) diff --git a/arch/um/os-Linux/skas/process.c b/arch/um/os-Linux/skas/process.c index b6f656bcffb1..cca016b0b29b 100644 --- a/arch/um/os-Linux/skas/process.c +++ b/arch/um/os-Linux/skas/process.c @@ -214,6 +214,9 @@ static int userspace_tramp(void *stack) (unsigned long) stub_segv_handler - (unsigned long) __syscall_stub_start; + /* Make sure this process dies if the kernel dies */ + os_set_pdeathsig(); + ptrace(PTRACE_TRACEME, 0, 0, 0); signal(SIGTERM, SIG_DFL); diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c index 9e1fccb39eab..744a72d11ed9 100644 --- a/arch/x86/kvm/svm/avic.c +++ b/arch/x86/kvm/svm/avic.c @@ -167,14 +167,6 @@ static void avic_deactivate_vmcb(struct vcpu_svm *svm) if (!sev_es_guest(svm->vcpu.kvm)) svm_set_intercept(svm, INTERCEPT_CR8_WRITE); - /* - * If running nested and the guest uses its own MSR bitmap, there - * is no need to update L0's msr bitmap - */ - if (is_guest_mode(&svm->vcpu) && - vmcb12_is_intercept(&svm->nested.ctl, INTERCEPT_MSR_PROT)) - return; - /* Enabling MSR intercept for x2APIC registers */ svm_set_x2apic_msr_interception(svm, true); } diff --git a/drivers/accel/qaic/qaic_control.c b/drivers/accel/qaic/qaic_control.c index 8eae30fe14f9..8759b53ba38e 100644 --- a/drivers/accel/qaic/qaic_control.c +++ b/drivers/accel/qaic/qaic_control.c @@ -782,7 +782,7 @@ static int encode_message(struct qaic_device *qdev, struct manage_msg *user_msg, break; } trans_hdr = (struct qaic_manage_trans_hdr *)(user_msg->data + user_len); - if (trans_hdr->len < sizeof(trans_hdr) || + if (trans_hdr->len < sizeof(*trans_hdr) || size_add(user_len, trans_hdr->len) > user_msg->len) { ret = -EINVAL; break; diff --git a/drivers/ata/ahci.h b/drivers/ata/ahci.h index 10a5fe02f0a4..61158eb1f840 100644 --- a/drivers/ata/ahci.h +++ b/drivers/ata/ahci.h @@ -329,7 +329,7 @@ struct ahci_port_priv { struct ahci_host_priv { /* Input fields */ unsigned int flags; /* AHCI_HFLAG_* */ - u32 mask_port_map; /* mask out particular bits */ + u32 mask_port_map; /* Mask of valid ports */ void __iomem * mmio; /* bus-independent mem map */ u32 cap; /* cap to use */ @@ -380,6 +380,21 @@ struct ahci_host_priv { int port); }; +/* + * Return true if a port should be ignored because it is excluded from + * the host port map. + */ +static inline bool ahci_ignore_port(struct ahci_host_priv *hpriv, + unsigned int portid) +{ + if (portid >= hpriv->nports) + return true; + /* mask_port_map not set means that all ports are available */ + if (!hpriv->mask_port_map) + return false; + return !(hpriv->mask_port_map & (1 << portid)); +} + extern int ahci_ignore_sss; extern const struct attribute_group *ahci_shost_groups[]; diff --git a/drivers/ata/ahci_brcm.c b/drivers/ata/ahci_brcm.c index 2f16524c2526..12bb0c2f914e 100644 --- a/drivers/ata/ahci_brcm.c +++ b/drivers/ata/ahci_brcm.c @@ -288,6 +288,9 @@ static unsigned int brcm_ahci_read_id(struct ata_device *dev, /* Re-initialize and calibrate the PHY */ for (i = 0; i < hpriv->nports; i++) { + if (ahci_ignore_port(hpriv, i)) + continue; + rc = phy_init(hpriv->phys[i]); if (rc) goto disable_phys; diff --git a/drivers/ata/ahci_ceva.c b/drivers/ata/ahci_ceva.c index 11a2c199a7c2..b2918ad8e7e0 100644 --- a/drivers/ata/ahci_ceva.c +++ b/drivers/ata/ahci_ceva.c @@ -206,15 +206,21 @@ static int ceva_ahci_platform_enable_resources(struct ahci_host_priv *hpriv) goto disable_clks; for (i = 0; i < hpriv->nports; i++) { + if (ahci_ignore_port(hpriv, i)) + continue; + rc = phy_init(hpriv->phys[i]); if (rc) - goto disable_rsts; + goto exit_phys; } /* De-assert the controller reset */ ahci_platform_deassert_rsts(hpriv); for (i = 0; i < hpriv->nports; i++) { + if (ahci_ignore_port(hpriv, i)) + continue; + rc = phy_power_on(hpriv->phys[i]); if (rc) { phy_exit(hpriv->phys[i]); @@ -224,14 +230,24 @@ static int ceva_ahci_platform_enable_resources(struct ahci_host_priv *hpriv) return 0; -disable_rsts: - ahci_platform_deassert_rsts(hpriv); - disable_phys: while (--i >= 0) { + if (ahci_ignore_port(hpriv, i)) + continue; + phy_power_off(hpriv->phys[i]); phy_exit(hpriv->phys[i]); } + ahci_platform_assert_rsts(hpriv); + goto disable_clks; + +exit_phys: + while (--i >= 0) { + if (ahci_ignore_port(hpriv, i)) + continue; + + phy_exit(hpriv->phys[i]); + } disable_clks: ahci_platform_disable_clks(hpriv); diff --git a/drivers/ata/libahci.c b/drivers/ata/libahci.c index 7824e8836a54..9b5b1e4c7148 100644 --- a/drivers/ata/libahci.c +++ b/drivers/ata/libahci.c @@ -542,6 +542,7 @@ void ahci_save_initial_config(struct device *dev, struct ahci_host_priv *hpriv) hpriv->saved_port_map = port_map; } + /* mask_port_map not set means that all ports are available */ if (hpriv->mask_port_map) { dev_warn(dev, "masking port_map 0x%lx -> 0x%lx\n", port_map, diff --git a/drivers/ata/libahci_platform.c b/drivers/ata/libahci_platform.c index 7a8064520a35..91d44302eac9 100644 --- a/drivers/ata/libahci_platform.c +++ b/drivers/ata/libahci_platform.c @@ -49,6 +49,9 @@ int ahci_platform_enable_phys(struct ahci_host_priv *hpriv) int rc, i; for (i = 0; i < hpriv->nports; i++) { + if (ahci_ignore_port(hpriv, i)) + continue; + rc = phy_init(hpriv->phys[i]); if (rc) goto disable_phys; @@ -70,6 +73,9 @@ int ahci_platform_enable_phys(struct ahci_host_priv *hpriv) disable_phys: while (--i >= 0) { + if (ahci_ignore_port(hpriv, i)) + continue; + phy_power_off(hpriv->phys[i]); phy_exit(hpriv->phys[i]); } @@ -88,6 +94,9 @@ void ahci_platform_disable_phys(struct ahci_host_priv *hpriv) int i; for (i = 0; i < hpriv->nports; i++) { + if (ahci_ignore_port(hpriv, i)) + continue; + phy_power_off(hpriv->phys[i]); phy_exit(hpriv->phys[i]); } @@ -432,6 +441,20 @@ static int ahci_platform_get_firmware(struct ahci_host_priv *hpriv, return 0; } +static u32 ahci_platform_find_max_port_id(struct device *dev) +{ + u32 max_port = 0; + + for_each_child_of_node_scoped(dev->of_node, child) { + u32 port; + + if (!of_property_read_u32(child, "reg", &port)) + max_port = max(max_port, port); + } + + return max_port; +} + /** * ahci_platform_get_resources - Get platform resources * @pdev: platform device to get resources for @@ -458,6 +481,7 @@ struct ahci_host_priv *ahci_platform_get_resources(struct platform_device *pdev, struct device *dev = &pdev->dev; struct ahci_host_priv *hpriv; u32 mask_port_map = 0; + u32 max_port; if (!devres_open_group(dev, NULL, GFP_KERNEL)) return ERR_PTR(-ENOMEM); @@ -549,15 +573,17 @@ struct ahci_host_priv *ahci_platform_get_resources(struct platform_device *pdev, goto err_out; } + /* find maximum port id for allocating structures */ + max_port = ahci_platform_find_max_port_id(dev); /* - * If no sub-node was found, we still need to set nports to - * one in order to be able to use the + * Set nports according to maximum port id. Clamp at + * AHCI_MAX_PORTS, warning message for invalid port id + * is generated later. + * When DT has no sub-nodes max_port is 0, nports is 1, + * in order to be able to use the * ahci_platform_[en|dis]able_[phys|regulators] functions. */ - if (child_nodes) - hpriv->nports = child_nodes; - else - hpriv->nports = 1; + hpriv->nports = min(AHCI_MAX_PORTS, max_port + 1); hpriv->phys = devm_kcalloc(dev, hpriv->nports, sizeof(*hpriv->phys), GFP_KERNEL); if (!hpriv->phys) { diff --git a/drivers/ata/libata-eh.c b/drivers/ata/libata-eh.c index e9d2d84034fd..a1331274ffcb 100644 --- a/drivers/ata/libata-eh.c +++ b/drivers/ata/libata-eh.c @@ -106,6 +106,12 @@ static const unsigned int ata_eh_flush_timeouts[] = { UINT_MAX, }; +static const unsigned int ata_eh_standby_timeouts[] = { + 15000, /* Some drives may be slow to standby */ + /* but don't hold up a suspend too long waiting for them */ + UINT_MAX, +}; + static const unsigned int ata_eh_other_timeouts[] = { 5000, /* same rationale as identify timeout */ 10000, /* ditto */ @@ -147,6 +153,8 @@ ata_eh_cmd_timeout_table[ATA_EH_CMD_TIMEOUT_TABLE_SIZE] = { .timeouts = ata_eh_other_timeouts, }, { .commands = CMDS(ATA_CMD_FLUSH, ATA_CMD_FLUSH_EXT), .timeouts = ata_eh_flush_timeouts }, + { .commands = CMDS(ATA_CMD_STANDBYNOW1), + .timeouts = ata_eh_standby_timeouts }, { .commands = CMDS(ATA_CMD_VERIFY), .timeouts = ata_eh_reset_timeouts }, }; diff --git a/drivers/ata/libata-sata.c b/drivers/ata/libata-sata.c index 5fbbdf6f87e3..fc64fae9e23f 100644 --- a/drivers/ata/libata-sata.c +++ b/drivers/ata/libata-sata.c @@ -913,7 +913,7 @@ static bool ata_scsi_lpm_supported(struct ata_port *ap) return false; ata_for_each_link(link, ap, EDGE) { - ata_for_each_dev(dev, &ap->link, ENABLED) { + ata_for_each_dev(dev, link, ENABLED) { if (dev->quirks & ATA_QUIRK_NOLPM) return false; } diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c index 05c905827dc5..40fdc422940d 100644 --- a/drivers/ata/sata_mv.c +++ b/drivers/ata/sata_mv.c @@ -4026,7 +4026,7 @@ static int mv_platform_probe(struct platform_device *pdev) /* * Simple resource validation .. */ - if (unlikely(pdev->num_resources != 1)) { + if (unlikely(pdev->num_resources != 1 && pdev->num_resources != 2)) { dev_err(&pdev->dev, "invalid number of resources\n"); return -EINVAL; } diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index 6854b847bcce..404cb44dd708 100644 --- a/drivers/block/ublk_drv.c +++ b/drivers/block/ublk_drv.c @@ -2558,6 +2558,15 @@ static int ublk_ctrl_add_dev(struct io_uring_cmd *cmd) /* update device id */ ub->dev_info.dev_id = ub->ub_number; + /* + * ->state and ->ublksrv_pid are owned by the driver and only read back + * by userspace, but they come from the copied-in dev_info, so reset + * them. Otherwise a device added with ->state != DEAD looks live while + * ->ub_disk is still NULL. + */ + ub->dev_info.state = UBLK_S_DEV_DEAD; + ub->dev_info.ublksrv_pid = -1; + /* * 64bit flags will be copied back to userspace as feature * negotiation result, so have to clear flags which driver diff --git a/drivers/bluetooth/btintel.c b/drivers/bluetooth/btintel.c index 6e3e4a817e72..cb948423a9bd 100644 --- a/drivers/bluetooth/btintel.c +++ b/drivers/bluetooth/btintel.c @@ -3335,6 +3335,9 @@ int btintel_diagnostics(struct hci_dev *hdev, struct sk_buff *skb) { struct intel_tlv *tlv = (void *)&skb->data[5]; + if (skb->len < 5 + sizeof(*tlv) + sizeof(tlv->val[0])) + goto recv_frame; + /* The first event is always an event type TLV */ if (tlv->type != INTEL_TLV_TYPE_ID) goto recv_frame; diff --git a/drivers/bluetooth/btmtk.c b/drivers/bluetooth/btmtk.c index 4d3f440565c1..22b05628e52e 100644 --- a/drivers/bluetooth/btmtk.c +++ b/drivers/bluetooth/btmtk.c @@ -763,30 +763,24 @@ static int btmtk_usb_uhw_reg_write(struct hci_dev *hdev, u32 reg, u32 val) static int btmtk_usb_uhw_reg_read(struct hci_dev *hdev, u32 reg, u32 *val) { struct btmtk_data *data = hci_get_priv(hdev); - int pipe, err; - void *buf; - - buf = kzalloc(4, GFP_KERNEL); - if (!buf) - return -ENOMEM; + u8 buf[sizeof(u32)]; + int err; - pipe = usb_rcvctrlpipe(data->udev, 0); - err = usb_control_msg(data->udev, pipe, 0x01, - 0xDE, - reg >> 16, reg & 0xffff, - buf, 4, USB_CTRL_GET_TIMEOUT); - if (err < 0) { + *val = 0; + err = usb_control_msg_recv(data->udev, 0, 0x01, + 0xDE, + reg >> 16, reg & 0xffff, + buf, sizeof(buf), USB_CTRL_GET_TIMEOUT, + GFP_KERNEL); + if (err) { bt_dev_err(hdev, "Failed to read uhw reg(%d)", err); - goto err_free_buf; + return err; } *val = get_unaligned_le32(buf); bt_dev_dbg(hdev, "reg=%x, value=0x%08x", reg, *val); -err_free_buf: - kfree(buf); - - return err; + return 0; } static int btmtk_usb_reg_read(struct hci_dev *hdev, u32 reg, u32 *val) @@ -836,7 +830,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) if (dev_id == 0x7922) { err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); - if (err < 0) + if (err) return err; val |= 0x00002020; err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, val); @@ -846,7 +840,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) if (err < 0) return err; err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); - if (err < 0) + if (err) return err; val |= BIT(0); err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, val); @@ -855,14 +849,14 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) msleep(100); } else if (dev_id == 0x7925) { err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); - if (err < 0) + if (err) return err; val |= (1 << 5); err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_RESET_REG_CONNV3, val); if (err < 0) return err; err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); - if (err < 0) + if (err) return err; val &= 0xFFFF00FF; val |= (1 << 13); @@ -873,7 +867,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) if (err < 0) return err; err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); - if (err < 0) + if (err) return err; val |= (1 << 0); err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_RESET_REG_CONNV3, val); @@ -883,13 +877,13 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) if (err < 0) return err; err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT, &val); - if (err < 0) + if (err) return err; err = btmtk_usb_uhw_reg_write(hdev, MTK_UDMA_INT_STA_BT1, 0x000000FF); if (err < 0) return err; err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT1, &val); - if (err < 0) + if (err) return err; msleep(100); } else { @@ -899,7 +893,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) if (err < 0) return err; err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_WDT_STATUS, &val); - if (err < 0) + if (err) return err; /* Reset the bluetooth chip via USB interface. */ err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, 1); @@ -909,13 +903,13 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) if (err < 0) return err; err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT, &val); - if (err < 0) + if (err) return err; err = btmtk_usb_uhw_reg_write(hdev, MTK_UDMA_INT_STA_BT1, 0x000000FF); if (err < 0) return err; err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT1, &val); - if (err < 0) + if (err) return err; /* MT7921 need to delay 20ms between toggle reset bit */ msleep(20); @@ -923,7 +917,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) if (err < 0) return err; err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); - if (err < 0) + if (err) return err; } diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c index aadbda2fd5a2..17680e172921 100644 --- a/drivers/bluetooth/btusb.c +++ b/drivers/bluetooth/btusb.c @@ -3304,28 +3304,16 @@ static u16 qca_extract_board_id(const struct qca_version *ver) static int btusb_qca_send_vendor_req(struct usb_device *udev, u8 request, void *data, u16 size) { - int pipe, err; - u8 *buf; - - buf = kmalloc(size, GFP_KERNEL); - if (!buf) - return -ENOMEM; + int err; /* Found some of USB hosts have IOT issues with ours so that we should * not wait until HCI layer is ready. */ - pipe = usb_rcvctrlpipe(udev, 0); - err = usb_control_msg(udev, pipe, request, USB_TYPE_VENDOR | USB_DIR_IN, - 0, 0, buf, size, USB_CTRL_GET_TIMEOUT); - if (err < 0) { + err = usb_control_msg_recv(udev, 0, request, USB_TYPE_VENDOR | USB_DIR_IN, + 0, 0, data, size, USB_CTRL_GET_TIMEOUT, + GFP_KERNEL); + if (err) dev_err(&udev->dev, "Failed to access otp area (%d)", err); - goto done; - } - - memcpy(data, buf, size); - -done: - kfree(buf); return err; } @@ -3519,7 +3507,7 @@ static bool btusb_qca_need_patch(struct usb_device *udev) struct qca_version ver; if (btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, - sizeof(ver)) < 0) + sizeof(ver))) return false; /* only low ROM versions need patches */ return !(le32_to_cpu(ver.rom_version) & ~0xffffU); @@ -3537,7 +3525,7 @@ static int btusb_setup_qca(struct hci_dev *hdev) err = btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, sizeof(ver)); - if (err < 0) + if (err) return err; ver_rom = le32_to_cpu(ver.rom_version); @@ -3560,7 +3548,7 @@ static int btusb_setup_qca(struct hci_dev *hdev) err = btusb_qca_send_vendor_req(udev, QCA_CHECK_STATUS, &status, sizeof(status)); - if (err < 0) + if (err) return err; if (!(status & QCA_PATCH_UPDATED)) { @@ -3571,7 +3559,7 @@ static int btusb_setup_qca(struct hci_dev *hdev) err = btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, sizeof(ver)); - if (err < 0) + if (err) return err; btdata->qca_dump.fw_version = le32_to_cpu(ver.patch_version); diff --git a/drivers/cpufreq/powernow-k8.c b/drivers/cpufreq/powernow-k8.c index a01170f7d01c..1260d4023a59 100644 --- a/drivers/cpufreq/powernow-k8.c +++ b/drivers/cpufreq/powernow-k8.c @@ -1083,6 +1083,7 @@ static int powernowk8_cpu_init(struct cpufreq_policy *pol) err_out_exit_acpi: powernow_k8_cpu_exit_acpi(data); + kfree(data->powernow_table); err_out: kfree(data); diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index 8dcd2331bb1a..14b322ac6e6c 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -293,6 +293,7 @@ static int idxd_cdev_open(struct inode *inode, struct file *filp) fdev->parent = cdev_dev(idxd_cdev); fdev->bus = &dsa_bus_type; fdev->type = &idxd_cdev_file_type; + idxd_wq_get(wq); rc = dev_set_name(fdev, "file%d", ctx->id); if (rc < 0) { @@ -306,13 +307,14 @@ static int idxd_cdev_open(struct inode *inode, struct file *filp) goto failed_dev_add; } - idxd_wq_get(wq); mutex_unlock(&wq->wq_lock); return 0; failed_dev_add: failed_dev_name: + mutex_unlock(&wq->wq_lock); put_device(fdev); + return rc; failed_ida: failed_set_pasid: if (device_user_pasid_enabled(idxd)) diff --git a/drivers/dma/sun6i-dma.c b/drivers/dma/sun6i-dma.c index c97027379263..1f1e21d48ad7 100644 --- a/drivers/dma/sun6i-dma.c +++ b/drivers/dma/sun6i-dma.c @@ -934,16 +934,13 @@ static int sun6i_dma_terminate_all(struct dma_chan *chan) spin_lock_irqsave(&vchan->vc.lock, flags); - if (vchan->cyclic) { - vchan->cyclic = false; - if (pchan && pchan->desc) { - struct virt_dma_desc *vd = &pchan->desc->vd; - struct virt_dma_chan *vc = &vchan->vc; + if (pchan && pchan->desc && pchan->desc != pchan->done) { + struct virt_dma_desc *vd = &pchan->desc->vd; - list_add_tail(&vd->node, &vc->desc_completed); - } + vchan_terminate_vdesc(vd); } + vchan->cyclic = false; vchan_get_all_descriptors(&vchan->vc, &head); if (pchan) { diff --git a/drivers/gpio/gpio-pca953x.c b/drivers/gpio/gpio-pca953x.c index 489c7ebe87c8..ad9280472bdd 100644 --- a/drivers/gpio/gpio-pca953x.c +++ b/drivers/gpio/gpio-pca953x.c @@ -1256,9 +1256,20 @@ static int pca953x_restore_context(struct pca953x_chip *chip) regcache_mark_dirty(chip->regmap); ret = pca953x_regcache_sync(chip); if (ret) - return ret; + goto err; + + ret = regcache_sync(chip->regmap); + if (ret) + goto err; + + return 0; - return regcache_sync(chip->regmap); +err: + if (chip->client->irq > 0) + disable_irq(chip->client->irq); + regcache_cache_only(chip->regmap, true); + + return ret; } static void pca953x_save_context(struct pca953x_chip *chip) diff --git a/drivers/gpio/gpio-pch.c b/drivers/gpio/gpio-pch.c index 63f25c72eac2..75dd65957e4a 100644 --- a/drivers/gpio/gpio-pch.c +++ b/drivers/gpio/gpio-pch.c @@ -96,7 +96,7 @@ struct pch_gpio { struct pch_gpio_reg_data pch_gpio_reg; int irq_base; enum pch_type_t ioh; - spinlock_t spinlock; + raw_spinlock_t spinlock; }; static void pch_gpio_set(struct gpio_chip *gpio, unsigned int nr, int val) @@ -105,7 +105,7 @@ static void pch_gpio_set(struct gpio_chip *gpio, unsigned int nr, int val) struct pch_gpio *chip = gpiochip_get_data(gpio); unsigned long flags; - spin_lock_irqsave(&chip->spinlock, flags); + raw_spin_lock_irqsave(&chip->spinlock, flags); reg_val = ioread32(&chip->reg->po); if (val) reg_val |= BIT(nr); @@ -113,7 +113,7 @@ static void pch_gpio_set(struct gpio_chip *gpio, unsigned int nr, int val) reg_val &= ~BIT(nr); iowrite32(reg_val, &chip->reg->po); - spin_unlock_irqrestore(&chip->spinlock, flags); + raw_spin_unlock_irqrestore(&chip->spinlock, flags); } static int pch_gpio_get(struct gpio_chip *gpio, unsigned int nr) @@ -131,7 +131,7 @@ static int pch_gpio_direction_output(struct gpio_chip *gpio, unsigned int nr, u32 reg_val; unsigned long flags; - spin_lock_irqsave(&chip->spinlock, flags); + raw_spin_lock_irqsave(&chip->spinlock, flags); reg_val = ioread32(&chip->reg->po); if (val) @@ -145,7 +145,7 @@ static int pch_gpio_direction_output(struct gpio_chip *gpio, unsigned int nr, pm |= BIT(nr); iowrite32(pm, &chip->reg->pm); - spin_unlock_irqrestore(&chip->spinlock, flags); + raw_spin_unlock_irqrestore(&chip->spinlock, flags); return 0; } @@ -156,12 +156,12 @@ static int pch_gpio_direction_input(struct gpio_chip *gpio, unsigned int nr) u32 pm; unsigned long flags; - spin_lock_irqsave(&chip->spinlock, flags); + raw_spin_lock_irqsave(&chip->spinlock, flags); pm = ioread32(&chip->reg->pm); pm &= BIT(gpio_pins[chip->ioh]) - 1; pm &= ~BIT(nr); iowrite32(pm, &chip->reg->pm); - spin_unlock_irqrestore(&chip->spinlock, flags); + raw_spin_unlock_irqrestore(&chip->spinlock, flags); return 0; } @@ -263,7 +263,7 @@ static int pch_irq_type(struct irq_data *d, unsigned int type) return 0; } - spin_lock_irqsave(&chip->spinlock, flags); + raw_spin_lock_irqsave(&chip->spinlock, flags); /* Set interrupt mode */ im = ioread32(im_reg) & ~(PCH_IM_MASK << (im_pos * 4)); @@ -275,7 +275,7 @@ static int pch_irq_type(struct irq_data *d, unsigned int type) else if (type & IRQ_TYPE_EDGE_BOTH) irq_set_handler_locked(d, handle_edge_irq); - spin_unlock_irqrestore(&chip->spinlock, flags); + raw_spin_unlock_irqrestore(&chip->spinlock, flags); return 0; } @@ -372,7 +372,7 @@ static int pch_gpio_probe(struct pci_dev *pdev, chip->ioh = id->driver_data; chip->reg = chip->base; pci_set_drvdata(pdev, chip); - spin_lock_init(&chip->spinlock); + raw_spin_lock_init(&chip->spinlock); pch_gpio_setup(chip); ret = devm_gpiochip_add_data(dev, &chip->gpio, chip); @@ -405,9 +405,9 @@ static int __maybe_unused pch_gpio_suspend(struct device *dev) struct pch_gpio *chip = dev_get_drvdata(dev); unsigned long flags; - spin_lock_irqsave(&chip->spinlock, flags); + raw_spin_lock_irqsave(&chip->spinlock, flags); pch_gpio_save_reg_conf(chip); - spin_unlock_irqrestore(&chip->spinlock, flags); + raw_spin_unlock_irqrestore(&chip->spinlock, flags); return 0; } @@ -417,11 +417,11 @@ static int __maybe_unused pch_gpio_resume(struct device *dev) struct pch_gpio *chip = dev_get_drvdata(dev); unsigned long flags; - spin_lock_irqsave(&chip->spinlock, flags); + raw_spin_lock_irqsave(&chip->spinlock, flags); iowrite32(0x01, &chip->reg->reset); iowrite32(0x00, &chip->reg->reset); pch_gpio_restore_reg_conf(chip); - spin_unlock_irqrestore(&chip->spinlock, flags); + raw_spin_unlock_irqrestore(&chip->spinlock, flags); return 0; } diff --git a/drivers/gpio/gpio-sloppy-logic-analyzer.c b/drivers/gpio/gpio-sloppy-logic-analyzer.c index 59a8f3a5c4e4..9c526aeadeba 100644 --- a/drivers/gpio/gpio-sloppy-logic-analyzer.c +++ b/drivers/gpio/gpio-sloppy-logic-analyzer.c @@ -161,6 +161,13 @@ static int fops_buf_size_get(void *data, u64 *val) return 0; } +static void fops_buf_release(void *data) +{ + struct gpio_la_poll_priv *priv = data; + + vfree(priv->blob.data); +} + static int fops_buf_size_set(void *data, u64 val) { struct gpio_la_poll_priv *priv = data; @@ -239,6 +246,9 @@ static int gpio_la_poll_probe(struct platform_device *pdev) return ret; fops_buf_size_set(priv, GPIO_LA_DEFAULT_BUF_SIZE); + ret = devm_add_action_or_reset(dev, fops_buf_release, priv); + if (ret) + return ret; priv->descs = devm_gpiod_get_array(dev, "probe", GPIOD_IN); if (IS_ERR(priv->descs)) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index a872da632486..0863550cd82e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -681,7 +681,7 @@ int amdgpu_amdkfd_submit_ib(struct amdgpu_device *adev, goto err; } - ret = amdgpu_job_alloc(adev, NULL, NULL, NULL, 1, &job); + ret = amdgpu_job_alloc(adev, NULL, NULL, NULL, 1, &job, 0); if (ret) goto err; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index 028af05dec6c..43308c631037 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -297,7 +297,8 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p, for (i = 0; i < p->gang_size; ++i) { ret = amdgpu_job_alloc(p->adev, vm, p->entities[i], vm, - num_ibs[i], &p->jobs[i]); + num_ibs[i], &p->jobs[i], + p->filp->client_id); if (ret) goto free_all_kdata; p->jobs[i]->enforce_isolation = p->adev->enforce_isolation[fpriv->xcp_id]; @@ -847,7 +848,6 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, struct amdgpu_vm *vm = &fpriv->vm; struct amdgpu_bo_list_entry *e; struct drm_gem_object *obj; - unsigned long index; unsigned int i; int r; @@ -968,7 +968,7 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, goto out_free_user_pages; } - drm_exec_for_each_locked_object(&p->exec, index, obj) { + drm_exec_for_each_locked_object(&p->exec, obj) { r = amdgpu_cs_bo_validate(p, gem_to_amdgpu_bo(obj)); if (unlikely(r)) goto out_free_user_pages; @@ -1206,7 +1206,6 @@ static int amdgpu_cs_sync_rings(struct amdgpu_cs_parser *p) struct drm_gpu_scheduler *sched; struct drm_gem_object *obj; struct dma_fence *fence; - unsigned long index; unsigned int i; int r; @@ -1217,7 +1216,7 @@ static int amdgpu_cs_sync_rings(struct amdgpu_cs_parser *p) return r; } - drm_exec_for_each_locked_object(&p->exec, index, obj) { + drm_exec_for_each_locked_object(&p->exec, obj) { struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj); struct dma_resv *resv = bo->tbo.base.resv; @@ -1285,7 +1284,6 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p, struct amdgpu_vm *vm = &fpriv->vm; struct amdgpu_bo_list_entry *e; struct drm_gem_object *gobj; - unsigned long index; unsigned int i; uint64_t seq; int r; @@ -1336,7 +1334,7 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p, } p->fence = dma_fence_get(&leader->base.s_fence->finished); - drm_exec_for_each_locked_object(&p->exec, index, gobj) { + drm_exec_for_each_locked_object(&p->exec, gobj) { ttm_bo_move_to_lru_tail_unlocked(&gem_to_amdgpu_bo(gobj)->tbo); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c index c43d1b6e5d66..6e59a4c56e4f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c @@ -321,7 +321,6 @@ static int amdgpu_ctx_init(struct amdgpu_ctx_mgr *mgr, int32_t priority, struct drm_file *filp, struct amdgpu_ctx *ctx) { struct amdgpu_fpriv *fpriv = filp->driver_priv; - u32 current_stable_pstate; int r; r = amdgpu_ctx_priority_permit(filp, priority); @@ -339,37 +338,22 @@ static int amdgpu_ctx_init(struct amdgpu_ctx_mgr *mgr, int32_t priority, ctx->generation = amdgpu_vm_generation(mgr->adev, &fpriv->vm); ctx->init_priority = priority; ctx->override_priority = AMDGPU_CTX_PRIORITY_UNSET; - - r = amdgpu_ctx_get_stable_pstate(ctx, ¤t_stable_pstate); - if (r) - return r; - - if (mgr->adev->pm.stable_pstate_ctx) - ctx->stable_pstate = mgr->adev->pm.stable_pstate_ctx->stable_pstate; - else - ctx->stable_pstate = current_stable_pstate; + ctx->stable_pstate = AMDGPU_CTX_STABLE_PSTATE_NONE; ctx->ctx_mgr = &(fpriv->ctx_mgr); return 0; } -static int amdgpu_ctx_set_stable_pstate(struct amdgpu_ctx *ctx, - u32 stable_pstate) +static int __amdgpu_ctx_set_stable_pstate(struct amdgpu_ctx *ctx, + u32 stable_pstate) { struct amdgpu_device *adev = ctx->mgr->adev; enum amd_dpm_forced_level level; + struct amdgpu_ctx *current_ctx; u32 current_stable_pstate; - int r; + int r = 0; - mutex_lock(&adev->pm.stable_pstate_ctx_lock); - if (adev->pm.stable_pstate_ctx && adev->pm.stable_pstate_ctx != ctx) { - r = -EBUSY; - goto done; - } - - r = amdgpu_ctx_get_stable_pstate(ctx, ¤t_stable_pstate); - if (r || (stable_pstate == current_stable_pstate)) - goto done; + lockdep_assert_held(&adev->pm.stable_pstate_ctx_lock); switch (stable_pstate) { case AMDGPU_CTX_STABLE_PSTATE_NONE: @@ -388,17 +372,41 @@ static int amdgpu_ctx_set_stable_pstate(struct amdgpu_ctx *ctx, level = AMD_DPM_FORCED_LEVEL_PROFILE_PEAK; break; default: - r = -EINVAL; - goto done; + return -EINVAL; } + current_ctx = adev->pm.stable_pstate_ctx; + if (current_ctx && current_ctx != ctx) + return -EBUSY; + + r = amdgpu_ctx_get_stable_pstate(ctx, ¤t_stable_pstate); + if (r || current_stable_pstate == stable_pstate) + return r; + r = amdgpu_dpm_force_performance_level(adev, level); + if (r) + return r; - if (level == AMD_DPM_FORCED_LEVEL_AUTO) - adev->pm.stable_pstate_ctx = NULL; - else + if (!current_ctx) { adev->pm.stable_pstate_ctx = ctx; -done: + /* + * Serialized by context taking ownership for the first time + * while holding adev->pm.stable_pstate_ctx_lock). + */ + WRITE_ONCE(ctx->stable_pstate, current_stable_pstate); + } + + return 0; +} + +static int amdgpu_ctx_set_stable_pstate(struct amdgpu_ctx *ctx, + u32 stable_pstate) +{ + struct amdgpu_device *adev = ctx->mgr->adev; + int r; + + mutex_lock(&adev->pm.stable_pstate_ctx_lock); + r = __amdgpu_ctx_set_stable_pstate(ctx, stable_pstate); mutex_unlock(&adev->pm.stable_pstate_ctx_lock); return r; @@ -424,7 +432,12 @@ static void amdgpu_ctx_fini(struct kref *ref) } if (drm_dev_enter(adev_to_drm(adev), &idx)) { - amdgpu_ctx_set_stable_pstate(ctx, ctx->stable_pstate); + mutex_lock(&adev->pm.stable_pstate_ctx_lock); + if (adev->pm.stable_pstate_ctx == ctx) { + __amdgpu_ctx_set_stable_pstate(ctx, ctx->stable_pstate); + adev->pm.stable_pstate_ctx = NULL; + } + mutex_unlock(&adev->pm.stable_pstate_ctx_lock); drm_dev_exit(idx); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index bf706ee2e0ed..fb26a4c1089c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -2824,6 +2824,19 @@ static int amdgpu_pmops_runtime_suspend(struct device *dev) return 0; } +static void amdgpu_restore_umd_profile_pstate_after_runpm(struct amdgpu_device *adev) +{ + enum amd_dpm_forced_level level; + uint32_t profile_mode_mask = AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD | + AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK | + AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK | + AMD_DPM_FORCED_LEVEL_PROFILE_PEAK; + + level = amdgpu_dpm_get_performance_level(adev); + if (level & profile_mode_mask) + amdgpu_asic_update_umd_stable_pstate(adev, true); +} + static int amdgpu_pmops_runtime_resume(struct device *dev) { struct pci_dev *pdev = to_pci_dev(dev); @@ -2868,6 +2881,8 @@ static int amdgpu_pmops_runtime_resume(struct device *dev) if (adev->pm.rpm_mode == AMDGPU_RUNPM_PX) drm_dev->switch_power_state = DRM_SWITCH_POWER_ON; + + amdgpu_restore_umd_profile_pstate_after_runpm(adev); adev->in_runpm = false; return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c index 3c2ac5f4e814..171603c4ee6c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c @@ -1426,12 +1426,13 @@ static int amdgpu_gfx_run_cleaner_shader_job(struct amdgpu_ring *ring) struct amdgpu_device *adev = ring->adev; struct drm_gpu_scheduler *sched = &ring->sched; struct drm_sched_entity entity; + unsigned int ib_size_dw = 16; static atomic_t counter; struct dma_fence *f; struct amdgpu_job *job; struct amdgpu_ib *ib; void *owner; - int i, r; + int r; /* Initialize the scheduler entity */ r = drm_sched_entity_init(&entity, DRM_SCHED_PRIORITY_NORMAL, @@ -1449,16 +1450,16 @@ static int amdgpu_gfx_run_cleaner_shader_job(struct amdgpu_ring *ring) owner = (void *)(unsigned long)atomic_inc_return(&counter); r = amdgpu_job_alloc_with_ib(ring->adev, &entity, owner, - 64, 0, &job); + ib_size_dw * sizeof(uint32_t), 0, &job, + AMDGPU_KERNEL_JOB_ID_CLEANER_SHADER); if (r) goto err; job->enforce_isolation = true; ib = &job->ibs[0]; - for (i = 0; i <= ring->funcs->align_mask; ++i) - ib->ptr[i] = ring->funcs->nop; - ib->length_dw = ring->funcs->align_mask + 1; + memset32(ib->ptr, ring->funcs->nop, ib_size_dw); + ib->length_dw = ib_size_dw; f = amdgpu_job_submit(job); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c index f63b5a429b10..9a0239da80e8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c @@ -664,7 +664,7 @@ void amdgpu_gmc_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, r = amdgpu_job_alloc_with_ib(ring->adev, &adev->mman.high_pr, AMDGPU_FENCE_OWNER_UNDEFINED, 16 * 4, AMDGPU_IB_POOL_IMMEDIATE, - &job); + &job, AMDGPU_KERNEL_JOB_ID_FLUSH_GPU_TLB); if (r) goto error_alloc; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c index 0760e70402ec..52ed0fa000d7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c @@ -217,7 +217,20 @@ static bool amdgpu_gtt_mgr_intersects(struct ttm_resource_manager *man, const struct ttm_place *place, size_t size) { - return !place->lpfn || amdgpu_gtt_mgr_has_gart_addr(res); + const struct drm_mm_node *const node = &to_ttm_range_mgr_node(res)->mm_nodes[0]; + const u32 num_pages = PFN_UP(size); + + if (!place->lpfn) + return true; + + if (!amdgpu_gtt_mgr_has_gart_addr(res)) + return false; + + if (place->fpfn >= (node->start + num_pages) || + (place->lpfn && place->lpfn <= node->start)) + return false; + + return true; } /** @@ -235,7 +248,20 @@ static bool amdgpu_gtt_mgr_compatible(struct ttm_resource_manager *man, const struct ttm_place *place, size_t size) { - return !place->lpfn || amdgpu_gtt_mgr_has_gart_addr(res); + const struct drm_mm_node *const node = &to_ttm_range_mgr_node(res)->mm_nodes[0]; + const u32 num_pages = PFN_UP(size); + + if (!place->lpfn) + return true; + + if (!amdgpu_gtt_mgr_has_gart_addr(res)) + return false; + + if (node->start < place->fpfn || + (place->lpfn && (node->start + num_pages) > place->lpfn)) + return false; + + return true; } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index 70762c7fcfe4..99e9e96f5b7f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c @@ -187,7 +187,8 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) int amdgpu_job_alloc(struct amdgpu_device *adev, struct amdgpu_vm *vm, struct drm_sched_entity *entity, void *owner, - unsigned int num_ibs, struct amdgpu_job **job) + unsigned int num_ibs, struct amdgpu_job **job, + u64 drm_client_id) { if (num_ibs == 0) return -EINVAL; @@ -210,17 +211,19 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, struct amdgpu_vm *vm, if (!entity) return 0; - return drm_sched_job_init(&(*job)->base, entity, 1, owner); + return drm_sched_job_init(&(*job)->base, entity, 1, owner, + drm_client_id); } int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev, struct drm_sched_entity *entity, void *owner, size_t size, enum amdgpu_ib_pool_type pool_type, - struct amdgpu_job **job) + struct amdgpu_job **job, u64 k_job_id) { int r; - r = amdgpu_job_alloc(adev, NULL, entity, owner, 1, job); + r = amdgpu_job_alloc(adev, NULL, entity, owner, 1, job, + k_job_id); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.h index 4fe033d8f356..4fb05c737840 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.h @@ -44,6 +44,22 @@ struct amdgpu_fence; enum amdgpu_ib_pool_type; +/* Internal kernel job ids. (decreasing values, starting from U64_MAX). */ +#define AMDGPU_KERNEL_JOB_ID_VM_UPDATE (18446744073709551615ULL) +#define AMDGPU_KERNEL_JOB_ID_VM_UPDATE_PDES (18446744073709551614ULL) +#define AMDGPU_KERNEL_JOB_ID_VM_UPDATE_RANGE (18446744073709551613ULL) +#define AMDGPU_KERNEL_JOB_ID_VM_PT_CLEAR (18446744073709551612ULL) +#define AMDGPU_KERNEL_JOB_ID_TTM_MAP_BUFFER (18446744073709551611ULL) +#define AMDGPU_KERNEL_JOB_ID_TTM_ACCESS_MEMORY_SDMA (18446744073709551610ULL) +#define AMDGPU_KERNEL_JOB_ID_TTM_COPY_BUFFER (18446744073709551609ULL) +#define AMDGPU_KERNEL_JOB_ID_CLEAR_ON_RELEASE (18446744073709551608ULL) +#define AMDGPU_KERNEL_JOB_ID_MOVE_BLIT (18446744073709551607ULL) +#define AMDGPU_KERNEL_JOB_ID_TTM_CLEAR_BUFFER (18446744073709551606ULL) +#define AMDGPU_KERNEL_JOB_ID_CLEANER_SHADER (18446744073709551605ULL) +#define AMDGPU_KERNEL_JOB_ID_FLUSH_GPU_TLB (18446744073709551604ULL) +#define AMDGPU_KERNEL_JOB_ID_KFD_GART_MAP (18446744073709551603ULL) +#define AMDGPU_KERNEL_JOB_ID_VCN_RING_TEST (18446744073709551602ULL) + struct amdgpu_job { struct drm_sched_job base; struct amdgpu_vm *vm; @@ -90,11 +106,13 @@ static inline struct amdgpu_ring *amdgpu_job_ring(struct amdgpu_job *job) int amdgpu_job_alloc(struct amdgpu_device *adev, struct amdgpu_vm *vm, struct drm_sched_entity *entity, void *owner, - unsigned int num_ibs, struct amdgpu_job **job); + unsigned int num_ibs, struct amdgpu_job **job, + u64 drm_client_id); int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev, struct drm_sched_entity *entity, void *owner, size_t size, enum amdgpu_ib_pool_type pool_type, - struct amdgpu_job **job); + struct amdgpu_job **job, + u64 k_job_id); void amdgpu_job_set_resources(struct amdgpu_job *job, struct amdgpu_bo *gds, struct amdgpu_bo *gws, struct amdgpu_bo *oa); void amdgpu_job_free_resources(struct amdgpu_job *job); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c index f96a79a4d539..4b0e34f9249a 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c @@ -192,7 +192,8 @@ static int amdgpu_jpeg_dec_set_reg(struct amdgpu_ring *ring, uint32_t handle, int i, r; r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, ib_size_dw * 4, - AMDGPU_IB_POOL_DIRECT, &job); + AMDGPU_IB_POOL_DIRECT, &job, + AMDGPU_KERNEL_JOB_ID_VCN_RING_TEST); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 3d7c6d0dd394..1db8ca26f1e1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -1269,7 +1269,8 @@ void amdgpu_bo_release_notify(struct ttm_buffer_object *bo) if (r) goto out; - r = amdgpu_fill_buffer(abo, 0, &bo->base._resv, &fence, true); + r = amdgpu_fill_buffer(abo, 0, &bo->base._resv, &fence, true, + AMDGPU_KERNEL_JOB_ID_CLEAR_ON_RELEASE); if (WARN_ON(r)) goto out; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 13d97d5a094b..31f9fb3cfeaa 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -229,7 +229,8 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, r = amdgpu_job_alloc_with_ib(adev, &adev->mman.high_pr, AMDGPU_FENCE_OWNER_UNDEFINED, num_dw * 4 + num_bytes, - AMDGPU_IB_POOL_DELAYED, &job); + AMDGPU_IB_POOL_DELAYED, &job, + AMDGPU_KERNEL_JOB_ID_TTM_MAP_BUFFER); if (r) return r; @@ -408,7 +409,7 @@ static int amdgpu_move_blit(struct ttm_buffer_object *bo, struct dma_fence *wipe_fence = NULL; r = amdgpu_fill_buffer(abo, 0, NULL, &wipe_fence, - false); + false, AMDGPU_KERNEL_JOB_ID_MOVE_BLIT); if (r) { goto error; } else if (wipe_fence) { @@ -1530,7 +1531,8 @@ static int amdgpu_ttm_access_memory_sdma(struct ttm_buffer_object *bo, r = amdgpu_job_alloc_with_ib(adev, &adev->mman.high_pr, AMDGPU_FENCE_OWNER_UNDEFINED, num_dw * 4, AMDGPU_IB_POOL_DELAYED, - &job); + &job, + AMDGPU_KERNEL_JOB_ID_TTM_ACCESS_MEMORY_SDMA); if (r) goto out; @@ -1996,6 +1998,18 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) else gtt_size = (uint64_t)amdgpu_gtt_size << 20; + /* Cap GTT so that it does not exceed total physical RAM. */ + if (adev->flags & AMD_IS_APU) { + u64 phys_ram = (u64)totalram_pages() << PAGE_SHIFT; + + if (gtt_size > phys_ram) { + gtt_size = phys_ram; + dev_info(adev->dev, + "Capping GTT to %uM to not exceed available system memory\n", + (unsigned int)(gtt_size / (1024 * 1024))); + } + } + /* Initialize GTT memory pool */ r = amdgpu_gtt_mgr_init(adev, gtt_size); if (r) { @@ -2163,7 +2177,7 @@ static int amdgpu_ttm_prepare_job(struct amdgpu_device *adev, struct dma_resv *resv, bool vm_needs_flush, struct amdgpu_job **job, - bool delayed) + bool delayed, u64 k_job_id) { enum amdgpu_ib_pool_type pool = direct_submit ? AMDGPU_IB_POOL_DIRECT : @@ -2173,7 +2187,7 @@ static int amdgpu_ttm_prepare_job(struct amdgpu_device *adev, &adev->mman.high_pr; r = amdgpu_job_alloc_with_ib(adev, entity, AMDGPU_FENCE_OWNER_UNDEFINED, - num_dw * 4, pool, job); + num_dw * 4, pool, job, k_job_id); if (r) return r; @@ -2212,7 +2226,8 @@ int amdgpu_copy_buffer(struct amdgpu_ring *ring, uint64_t src_offset, num_loops = DIV_ROUND_UP(byte_count, max_bytes); num_dw = ALIGN(num_loops * adev->mman.buffer_funcs->copy_num_dw, 8); r = amdgpu_ttm_prepare_job(adev, direct_submit, num_dw, - resv, vm_needs_flush, &job, false); + resv, vm_needs_flush, &job, false, + AMDGPU_KERNEL_JOB_ID_TTM_COPY_BUFFER); if (r) return r; @@ -2247,7 +2262,8 @@ static int amdgpu_ttm_fill_mem(struct amdgpu_ring *ring, uint32_t src_data, uint64_t dst_addr, uint32_t byte_count, struct dma_resv *resv, struct dma_fence **fence, - bool vm_needs_flush, bool delayed) + bool vm_needs_flush, bool delayed, + u64 k_job_id) { struct amdgpu_device *adev = ring->adev; unsigned int num_loops, num_dw; @@ -2260,7 +2276,7 @@ static int amdgpu_ttm_fill_mem(struct amdgpu_ring *ring, uint32_t src_data, num_loops = DIV_ROUND_UP_ULL(byte_count, max_bytes); num_dw = ALIGN(num_loops * adev->mman.buffer_funcs->fill_num_dw, 8); r = amdgpu_ttm_prepare_job(adev, false, num_dw, resv, vm_needs_flush, - &job, delayed); + &job, delayed, k_job_id); if (r) return r; @@ -2330,7 +2346,8 @@ int amdgpu_ttm_clear_buffer(struct amdgpu_bo *bo, goto err; r = amdgpu_ttm_fill_mem(ring, 0, addr, size, resv, - &next, true, true); + &next, true, true, + AMDGPU_KERNEL_JOB_ID_TTM_CLEAR_BUFFER); if (r) goto err; @@ -2349,7 +2366,8 @@ int amdgpu_fill_buffer(struct amdgpu_bo *bo, uint32_t src_data, struct dma_resv *resv, struct dma_fence **f, - bool delayed) + bool delayed, + u64 k_job_id) { struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; @@ -2378,7 +2396,7 @@ int amdgpu_fill_buffer(struct amdgpu_bo *bo, goto error; r = amdgpu_ttm_fill_mem(ring, src_data, to, cur_size, resv, - &next, true, delayed); + &next, true, delayed, k_job_id); if (r) goto error; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h index 3c883f1cf068..7c020e3fbfb4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h @@ -180,7 +180,8 @@ int amdgpu_fill_buffer(struct amdgpu_bo *bo, uint32_t src_data, struct dma_resv *resv, struct dma_fence **fence, - bool delayed); + bool delayed, + u64 k_job_id); int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo); void amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c index 779dcd820d4a..549a56b7a876 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c @@ -1164,7 +1164,8 @@ static int amdgpu_uvd_send_msg(struct amdgpu_ring *ring, struct amdgpu_bo *bo, r = amdgpu_job_alloc_with_ib(ring->adev, &adev->uvd.entity, AMDGPU_FENCE_OWNER_UNDEFINED, 64, direct ? AMDGPU_IB_POOL_DIRECT : - AMDGPU_IB_POOL_DELAYED, &job); + AMDGPU_IB_POOL_DELAYED, &job, + AMDGPU_KERNEL_JOB_ID_VCN_RING_TEST); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c index 33b4305bd305..0f2d75749ae0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c @@ -449,7 +449,7 @@ static int amdgpu_vce_get_create_msg(struct amdgpu_ring *ring, uint32_t handle, r = amdgpu_job_alloc_with_ib(ring->adev, &ring->adev->vce.entity, AMDGPU_FENCE_OWNER_UNDEFINED, ib_size_dw * 4, AMDGPU_IB_POOL_DIRECT, - &job); + &job, AMDGPU_KERNEL_JOB_ID_VCN_RING_TEST); if (r) return r; @@ -540,7 +540,8 @@ static int amdgpu_vce_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handle, AMDGPU_FENCE_OWNER_UNDEFINED, ib_size_dw * 4, direct ? AMDGPU_IB_POOL_DIRECT : - AMDGPU_IB_POOL_DELAYED, &job); + AMDGPU_IB_POOL_DELAYED, &job, + AMDGPU_KERNEL_JOB_ID_VCN_RING_TEST); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c index 43f44cc201cb..0e2fef666dd5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c @@ -553,7 +553,7 @@ static int amdgpu_vcn_dec_send_msg(struct amdgpu_ring *ring, r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, 64, AMDGPU_IB_POOL_DIRECT, - &job); + &job, AMDGPU_KERNEL_JOB_ID_VCN_RING_TEST); if (r) goto err; @@ -733,7 +733,7 @@ static int amdgpu_vcn_dec_sw_send_msg(struct amdgpu_ring *ring, r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, ib_size_dw * 4, AMDGPU_IB_POOL_DIRECT, - &job); + &job, AMDGPU_KERNEL_JOB_ID_VCN_RING_TEST); if (r) goto err; @@ -863,7 +863,7 @@ static int amdgpu_vcn_enc_get_create_msg(struct amdgpu_ring *ring, uint32_t hand r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, ib_size_dw * 4, AMDGPU_IB_POOL_DIRECT, - &job); + &job, AMDGPU_KERNEL_JOB_ID_VCN_RING_TEST); if (r) return r; @@ -930,7 +930,7 @@ static int amdgpu_vcn_enc_get_destroy_msg(struct amdgpu_ring *ring, uint32_t han r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, ib_size_dw * 4, AMDGPU_IB_POOL_DIRECT, - &job); + &job, AMDGPU_KERNEL_JOB_ID_VCN_RING_TEST); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index a6b34e5ba96c..8864e43a165e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -838,7 +838,8 @@ int amdgpu_vm_update_pdes(struct amdgpu_device *adev, params.vm = vm; params.immediate = immediate; - r = vm->update_funcs->prepare(¶ms, NULL); + r = vm->update_funcs->prepare(¶ms, NULL, + AMDGPU_KERNEL_JOB_ID_VM_UPDATE_PDES); if (r) goto error; @@ -1007,7 +1008,8 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm, dma_fence_put(tmp); } - r = vm->update_funcs->prepare(¶ms, sync); + r = vm->update_funcs->prepare(¶ms, sync, + AMDGPU_KERNEL_JOB_ID_VM_UPDATE_RANGE); if (r) goto error_free; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h index a451f405afe3..095209a2cb33 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h @@ -305,7 +305,7 @@ struct amdgpu_vm_update_params { struct amdgpu_vm_update_funcs { int (*map_table)(struct amdgpu_bo_vm *bo); int (*prepare)(struct amdgpu_vm_update_params *p, - struct amdgpu_sync *sync); + struct amdgpu_sync *sync, u64 k_job_id); int (*update)(struct amdgpu_vm_update_params *p, struct amdgpu_bo_vm *bo, uint64_t pe, uint64_t addr, unsigned count, uint32_t incr, uint64_t flags); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c index f80b611ce190..f078db3fef79 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c @@ -42,12 +42,14 @@ static int amdgpu_vm_cpu_map_table(struct amdgpu_bo_vm *table) * * @p: see amdgpu_vm_update_params definition * @sync: sync obj with fences to wait on + * @k_job_id: the id for tracing/debug purposes * * Returns: * Negativ errno, 0 for success. */ static int amdgpu_vm_cpu_prepare(struct amdgpu_vm_update_params *p, - struct amdgpu_sync *sync) + struct amdgpu_sync *sync, + u64 k_job_id) { if (!sync) return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c index 54ae0e9bc6d7..ba7bd65f5b7d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c @@ -26,6 +26,7 @@ #include "amdgpu.h" #include "amdgpu_trace.h" #include "amdgpu_vm.h" +#include "amdgpu_job.h" /* * amdgpu_vm_pt_cursor - state for for_each_amdgpu_vm_pt @@ -395,7 +396,8 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm, params.vm = vm; params.immediate = immediate; - r = vm->update_funcs->prepare(¶ms, NULL); + r = vm->update_funcs->prepare(¶ms, NULL, + AMDGPU_KERNEL_JOB_ID_VM_PT_CLEAR); if (r) goto exit; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c index 46d9fb433ab2..36805dcfa159 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c @@ -40,7 +40,7 @@ static int amdgpu_vm_sdma_map_table(struct amdgpu_bo_vm *table) /* Allocate a new job for @count PTE updates */ static int amdgpu_vm_sdma_alloc_job(struct amdgpu_vm_update_params *p, - unsigned int count) + unsigned int count, u64 k_job_id) { enum amdgpu_ib_pool_type pool = p->immediate ? AMDGPU_IB_POOL_IMMEDIATE : AMDGPU_IB_POOL_DELAYED; @@ -56,7 +56,7 @@ static int amdgpu_vm_sdma_alloc_job(struct amdgpu_vm_update_params *p, ndw = min(ndw, AMDGPU_VM_SDMA_MAX_NUM_DW); r = amdgpu_job_alloc_with_ib(p->adev, entity, AMDGPU_FENCE_OWNER_VM, - ndw * 4, pool, &p->job); + ndw * 4, pool, &p->job, k_job_id); if (r) return r; @@ -69,16 +69,17 @@ static int amdgpu_vm_sdma_alloc_job(struct amdgpu_vm_update_params *p, * * @p: see amdgpu_vm_update_params definition * @sync: amdgpu_sync object with fences to wait for + * @k_job_id: identifier of the job, for tracing purpose * * Returns: * Negativ errno, 0 for success. */ static int amdgpu_vm_sdma_prepare(struct amdgpu_vm_update_params *p, - struct amdgpu_sync *sync) + struct amdgpu_sync *sync, u64 k_job_id) { int r; - r = amdgpu_vm_sdma_alloc_job(p, 0); + r = amdgpu_vm_sdma_alloc_job(p, 0, k_job_id); if (r) return r; @@ -249,7 +250,8 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p, if (r) return r; - r = amdgpu_vm_sdma_alloc_job(p, count); + r = amdgpu_vm_sdma_alloc_job(p, count, + AMDGPU_KERNEL_JOB_ID_VM_UPDATE); if (r) return r; } diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c index e05e81d6fbd4..71a16073c43b 100644 --- a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c @@ -217,7 +217,8 @@ static int uvd_v6_0_enc_get_create_msg(struct amdgpu_ring *ring, uint32_t handle int i, r; r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, ib_size_dw * 4, - AMDGPU_IB_POOL_DIRECT, &job); + AMDGPU_IB_POOL_DIRECT, &job, + AMDGPU_KERNEL_JOB_ID_VCN_RING_TEST); if (r) return r; @@ -281,7 +282,8 @@ static int uvd_v6_0_enc_get_destroy_msg(struct amdgpu_ring *ring, int i, r; r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, ib_size_dw * 4, - AMDGPU_IB_POOL_DIRECT, &job); + AMDGPU_IB_POOL_DIRECT, &job, + AMDGPU_KERNEL_JOB_ID_VCN_RING_TEST); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c index e3c20cbc7a00..bc0fca75417c 100644 --- a/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c +++ b/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c @@ -225,7 +225,8 @@ static int uvd_v7_0_enc_get_create_msg(struct amdgpu_ring *ring, u32 handle, int i, r; r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, ib_size_dw * 4, - AMDGPU_IB_POOL_DIRECT, &job); + AMDGPU_IB_POOL_DIRECT, &job, + AMDGPU_KERNEL_JOB_ID_VCN_RING_TEST); if (r) return r; @@ -288,7 +289,8 @@ static int uvd_v7_0_enc_get_destroy_msg(struct amdgpu_ring *ring, u32 handle, int i, r; r = amdgpu_job_alloc_with_ib(ring->adev, NULL, NULL, ib_size_dw * 4, - AMDGPU_IB_POOL_DIRECT, &job); + AMDGPU_IB_POOL_DIRECT, &job, + AMDGPU_KERNEL_JOB_ID_VCN_RING_TEST); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c index 8366709a7211..697325a8f1c6 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c @@ -2978,10 +2978,14 @@ static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, v goto out; } - /* Check if target is still PTRACED. */ + /* + * Verify debugger has permission to debug target process. + * For cross-process debugging, require active ptrace relationship. + * This applies to ALL operations to prevent unauthorized interference. + */ rcu_read_lock(); - if (target != p && args->op != KFD_IOC_DBG_TRAP_DISABLE - && ptrace_parent(target->lead_thread) != current) { + if (target != p && ptrace_parent(target->lead_thread) != current + && target->debugger_process != p) { pr_err("PID %i is not PTRACED and cannot be debugged\n", args->pid); r = -EPERM; } diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_events.c b/drivers/gpu/drm/amd/amdkfd/kfd_events.c index 05c049c6d3b5..a8c91c0e2f8e 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_events.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_events.c @@ -521,6 +521,9 @@ int kfd_criu_restore_event(struct file *devkfd, ret = create_other_event(p, ev, &ev_priv->event_id); break; + default: + ret = -EINVAL; + break; } mutex_unlock(&p->event_mutex); @@ -542,15 +545,27 @@ int kfd_criu_checkpoint_events(struct kfd_process *p, int ret = 0; struct kfd_event *ev; uint32_t ev_id; + uint32_t num_events; - uint32_t num_events = kfd_get_num_events(p); + /* Serialize the count and the walk below against concurrent event + * create/destroy. Those paths take only p->event_mutex, not the + * p->mutex held by the CRIU checkpoint caller, so without this the + * event_idr can grow between kfd_get_num_events() and the loop and the + * walk writes past the ev_privs allocation. + */ + mutex_lock(&p->event_mutex); - if (!num_events) + num_events = kfd_get_num_events(p); + if (!num_events) { + mutex_unlock(&p->event_mutex); return 0; + } ev_privs = kvzalloc(num_events * sizeof(*ev_privs), GFP_KERNEL); - if (!ev_privs) + if (!ev_privs) { + mutex_unlock(&p->event_mutex); return -ENOMEM; + } idr_for_each_entry(&p->event_idr, ev, ev_id) { @@ -591,6 +606,8 @@ int kfd_criu_checkpoint_events(struct kfd_process *p, i++; } + mutex_unlock(&p->event_mutex); + ret = copy_to_user(user_priv_data + *priv_data_offset, ev_privs, num_events * sizeof(*ev_privs)); if (ret) { diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c index 52246f16cc24..ca6c48e3703c 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c @@ -68,7 +68,8 @@ svm_migrate_gart_map(struct amdgpu_ring *ring, uint64_t npages, AMDGPU_FENCE_OWNER_UNDEFINED, num_dw * 4 + num_bytes, AMDGPU_IB_POOL_DELAYED, - &job); + &job, + AMDGPU_KERNEL_JOB_ID_KFD_GART_MAP); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c index e3749dae5e59..4ba100236abb 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c @@ -376,7 +376,7 @@ int pqm_create_queue(struct process_queue_manager *pqm, false); if (retval) { dev_err(dev->adev->dev, "failed to allocate process context bo\n"); - return retval; + goto err_allocate_pqn; } memset(pdd->proc_ctx_cpu_ptr, 0, AMDGPU_MES_PROC_CTX_SIZE); } diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_queue.c index de2af34865fb..3efbe6773487 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_queue.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_queue.c @@ -285,7 +285,7 @@ int kfd_queue_acquire_buffers(struct kfd_process_device *pdd, struct queue_prope } err = kfd_queue_buffer_get(vm, (void *)properties->eop_ring_buffer_address, &properties->eop_buf_bo, - ALIGN(properties->eop_ring_buffer_size, PAGE_SIZE)); + ALIGN((u64)properties->eop_ring_buffer_size, PAGE_SIZE)); if (err) goto out_err_unreserve; } diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_clock_source.c b/drivers/gpu/drm/amd/display/dc/dce/dce_clock_source.c index b700608e4240..cf09c1d3d1b2 100644 --- a/drivers/gpu/drm/amd/display/dc/dce/dce_clock_source.c +++ b/drivers/gpu/drm/amd/display/dc/dce/dce_clock_source.c @@ -45,9 +45,7 @@ clk_src->base.ctx #define DC_LOGGER \ - calc_pll_cs->ctx->logger -#define DC_LOGGER_INIT() \ - struct calc_pll_clock_source *calc_pll_cs = &clk_src->calc_pll + CTX->logger #undef FN #define FN(reg_name, field_name) \ @@ -291,6 +289,7 @@ static bool calc_pll_dividers_in_range( } static uint32_t calculate_pixel_clock_pll_dividers( + struct dce110_clk_src *clk_src, struct calc_pll_clock_source *calc_pll_cs, struct pll_settings *pll_settings) { @@ -479,7 +478,7 @@ static uint32_t dce110_get_pix_clk_dividers_helper ( { uint32_t field = 0; uint32_t pll_calc_error = MAX_PLL_CALC_ERROR; - DC_LOGGER_INIT(); + /* Check if reference clock is external (not pcie/xtalin) * HW Dce80 spec: * 00 - PCIE_REFCLK, 01 - XTALIN, 02 - GENERICA, 03 - GENERICB @@ -522,12 +521,14 @@ static uint32_t dce110_get_pix_clk_dividers_helper ( /*Calculate Dividers by HDMI object, no SS case or SS case */ pll_calc_error = calculate_pixel_clock_pll_dividers( + clk_src, &clk_src->calc_pll_hdmi, pll_settings); else /*Calculate Dividers by default object, no SS case or SS case */ pll_calc_error = calculate_pixel_clock_pll_dividers( + clk_src, &clk_src->calc_pll, pll_settings); @@ -572,7 +573,6 @@ static uint32_t dce110_get_pix_clk_dividers( { struct dce110_clk_src *clk_src = TO_DCE110_CLK_SRC(cs); uint32_t pll_calc_error = MAX_PLL_CALC_ERROR; - DC_LOGGER_INIT(); if (pix_clk_params == NULL || pll_settings == NULL || pix_clk_params->requested_pix_clk_100hz == 0) { @@ -604,7 +604,6 @@ static uint32_t dce112_get_pix_clk_dividers( struct pll_settings *pll_settings) { struct dce110_clk_src *clk_src = TO_DCE110_CLK_SRC(cs); - DC_LOGGER_INIT(); if (pix_clk_params == NULL || pll_settings == NULL || pix_clk_params->requested_pix_clk_100hz == 0) { @@ -1366,8 +1365,6 @@ static uint32_t dcn3_get_pix_clk_dividers( unsigned long long actual_pix_clk_100Hz = pix_clk_params ? pix_clk_params->requested_pix_clk_100hz : 0; struct dce110_clk_src *clk_src = TO_DCE110_CLK_SRC(cs); - DC_LOGGER_INIT(); - if (pix_clk_params == NULL || pll_settings == NULL || pix_clk_params->requested_pix_clk_100hz == 0) { DC_LOG_ERROR( @@ -1437,7 +1434,6 @@ static const struct clock_source_funcs dce110_clk_src_funcs = { .get_pixel_clk_frequency_100hz = get_pixel_clk_frequency_100hz }; - static void get_ss_info_from_atombios( struct dce110_clk_src *clk_src, enum as_signal_type as_signal, @@ -1450,7 +1446,7 @@ static void get_ss_info_from_atombios( struct spread_spectrum_info *ss_info_cur; struct spread_spectrum_data *ss_data_cur; uint32_t i; - DC_LOGGER_INIT(); + if (ss_entries_num == NULL) { DC_LOG_SYNC( "Invalid entry !!!\n"); @@ -1585,6 +1581,7 @@ static void ss_info_from_atombios_create( } static bool calc_pll_max_vco_construct( + struct dce110_clk_src *clk_src, struct calc_pll_clock_source *calc_pll_cs, struct calc_pll_clock_source_init_data *init_data) { @@ -1736,6 +1733,7 @@ bool dce110_clk_src_construct( ss_info_from_atombios_create(clk_src); if (!calc_pll_max_vco_construct( + clk_src, &clk_src->calc_pll, &calc_pll_cs_init_data)) { ASSERT_CRITICAL(false); @@ -1750,7 +1748,7 @@ bool dce110_clk_src_construct( if (!calc_pll_max_vco_construct( - &clk_src->calc_pll_hdmi, &calc_pll_cs_init_data_hdmi)) { + clk_src, &clk_src->calc_pll_hdmi, &calc_pll_cs_init_data_hdmi)) { ASSERT_CRITICAL(false); goto unexpected_failure; } diff --git a/drivers/gpu/drm/amd/display/dc/hwss/dcn30/dcn30_hwseq.c b/drivers/gpu/drm/amd/display/dc/hwss/dcn30/dcn30_hwseq.c index de4282a98626..e77fd63dfc0f 100644 --- a/drivers/gpu/drm/amd/display/dc/hwss/dcn30/dcn30_hwseq.c +++ b/drivers/gpu/drm/amd/display/dc/hwss/dcn30/dcn30_hwseq.c @@ -821,13 +821,19 @@ void dcn30_set_avmute(struct pipe_ctx *pipe_ctx, bool enable) pipe_ctx->stream_res.stream_enc, enable); - /* Wait for two frame to make sure AV mute is sent out */ + /* Wait for three frames to make sure AV mute is sent out. + * Some HDMI sinks need additional GCP packets to properly + * process the mute state, especially after link re-establishment + * with HDMI 2.0 scrambling enabled. + */ if (enable && pipe_ctx->stream_res.tg->funcs->is_tg_enabled(pipe_ctx->stream_res.tg)) { + int i; + pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VACTIVE); - pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VBLANK); - pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VACTIVE); - pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VBLANK); - pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VACTIVE); + for (i = 0; i < 3; i++) { + pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VBLANK); + pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VACTIVE); + } } } } diff --git a/drivers/gpu/drm/armada/armada_fbdev.c b/drivers/gpu/drm/armada/armada_fbdev.c index d223176912b6..e409029638b0 100644 --- a/drivers/gpu/drm/armada/armada_fbdev.c +++ b/drivers/gpu/drm/armada/armada_fbdev.c @@ -43,10 +43,10 @@ static int armada_fbdev_create(struct drm_fb_helper *fbh, struct drm_fb_helper_surface_size *sizes) { struct drm_device *dev = fbh->dev; + struct fb_info *info = fbh->info; struct drm_mode_fb_cmd2 mode; struct armada_framebuffer *dfb; struct armada_gem_object *obj; - struct fb_info *info; int size, ret; void *ptr; @@ -87,12 +87,6 @@ static int armada_fbdev_create(struct drm_fb_helper *fbh, if (IS_ERR(dfb)) return PTR_ERR(dfb); - info = drm_fb_helper_alloc_info(fbh); - if (IS_ERR(info)) { - ret = PTR_ERR(info); - goto err_fballoc; - } - info->fbops = &armada_fb_ops; info->fix.smem_start = obj->phys_addr; info->fix.smem_len = obj->obj.size; @@ -107,10 +101,6 @@ static int armada_fbdev_create(struct drm_fb_helper *fbh, (unsigned long long)obj->phys_addr); return 0; - - err_fballoc: - dfb->fb.funcs->destroy(&dfb->fb); - return ret; } static int armada_fb_probe(struct drm_fb_helper *fbh, diff --git a/drivers/gpu/drm/display/drm_dp_helper.c b/drivers/gpu/drm/display/drm_dp_helper.c index 9fa13da513d2..abcc2538b9be 100644 --- a/drivers/gpu/drm/display/drm_dp_helper.c +++ b/drivers/gpu/drm/display/drm_dp_helper.c @@ -3145,6 +3145,18 @@ int drm_dp_get_pcon_max_frl_bw(const u8 dpcd[DP_RECEIVER_CAP_SIZE], int bw; u8 buf; + if (!drm_dp_is_branch(dpcd)) + return 0; + + if (dpcd[DP_DPCD_REV] < 0x11) + return 0; + + if ((dpcd[DP_DOWNSTREAMPORT_PRESENT] & DP_DETAILED_CAP_INFO_AVAILABLE) == 0) + return 0; + + if ((port_cap[0] & DP_DS_PORT_TYPE_MASK) != DP_DS_PORT_TYPE_HDMI) + return 0; + buf = port_cap[2]; bw = buf & DP_PCON_MAX_FRL_BW; diff --git a/drivers/gpu/drm/drm_exec.c b/drivers/gpu/drm/drm_exec.c index 18e366cc4993..6c1351cd7f15 100644 --- a/drivers/gpu/drm/drm_exec.c +++ b/drivers/gpu/drm/drm_exec.c @@ -22,7 +22,6 @@ * * struct drm_gem_object *obj; * struct drm_exec exec; - * unsigned long index; * int ret; * * drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT); @@ -38,7 +37,7 @@ * goto error; * } * - * drm_exec_for_each_locked_object(&exec, index, obj) { + * drm_exec_for_each_locked_object(&exec, obj) { * dma_resv_add_fence(obj->resv, fence, DMA_RESV_USAGE_READ); * ... * } @@ -54,9 +53,8 @@ static void drm_exec_unlock_all(struct drm_exec *exec) { struct drm_gem_object *obj; - unsigned long index; - drm_exec_for_each_locked_object_reverse(exec, index, obj) { + drm_exec_for_each_locked_object_reverse(exec, obj) { dma_resv_unlock(obj->resv); drm_gem_object_put(obj); } diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c index a8971c4eb9f0..91d76e5e2785 100644 --- a/drivers/gpu/drm/drm_fb_helper.c +++ b/drivers/gpu/drm/drm_fb_helper.c @@ -485,20 +485,7 @@ int drm_fb_helper_init(struct drm_device *dev, } EXPORT_SYMBOL(drm_fb_helper_init); -/** - * drm_fb_helper_alloc_info - allocate fb_info and some of its members - * @fb_helper: driver-allocated fbdev helper - * - * A helper to alloc fb_info and the member cmap. Called by the driver - * within the struct &drm_driver.fbdev_probe callback function. Drivers do - * not need to release the allocated fb_info structure themselves, this is - * automatically done when calling drm_fb_helper_fini(). - * - * RETURNS: - * fb_info pointer if things went okay, pointer containing error code - * otherwise - */ -struct fb_info *drm_fb_helper_alloc_info(struct drm_fb_helper *fb_helper) +static struct fb_info *drm_fb_helper_alloc_info(struct drm_fb_helper *fb_helper) { struct device *dev = fb_helper->dev->dev; struct fb_info *info; @@ -525,17 +512,8 @@ struct fb_info *drm_fb_helper_alloc_info(struct drm_fb_helper *fb_helper) framebuffer_release(info); return ERR_PTR(ret); } -EXPORT_SYMBOL(drm_fb_helper_alloc_info); -/** - * drm_fb_helper_release_info - release fb_info and its members - * @fb_helper: driver-allocated fbdev helper - * - * A helper to release fb_info and the member cmap. Drivers do not - * need to release the allocated fb_info structure themselves, this is - * automatically done when calling drm_fb_helper_fini(). - */ -void drm_fb_helper_release_info(struct drm_fb_helper *fb_helper) +static void drm_fb_helper_release_info(struct drm_fb_helper *fb_helper) { struct fb_info *info = fb_helper->info; @@ -548,7 +526,6 @@ void drm_fb_helper_release_info(struct drm_fb_helper *fb_helper) fb_dealloc_cmap(&info->cmap); framebuffer_release(info); } -EXPORT_SYMBOL(drm_fb_helper_release_info); /** * drm_fb_helper_unregister_info - unregister fb_info framebuffer device @@ -1803,6 +1780,13 @@ __drm_fb_helper_initial_config_and_unlock(struct drm_fb_helper *fb_helper) height = dev->mode_config.max_height; drm_client_modeset_probe(&fb_helper->client, width, height); + + info = drm_fb_helper_alloc_info(fb_helper); + if (IS_ERR(info)) { + mutex_unlock(&fb_helper->lock); + return PTR_ERR(info); + } + ret = drm_fb_helper_single_fb_probe(fb_helper); if (ret < 0) { if (ret == -EAGAIN) { @@ -1811,13 +1795,12 @@ __drm_fb_helper_initial_config_and_unlock(struct drm_fb_helper *fb_helper) } mutex_unlock(&fb_helper->lock); - return ret; + goto err_drm_fb_helper_release_info; } drm_setup_crtcs_fb(fb_helper); fb_helper->deferred_setup = false; - info = fb_helper->info; info->var.pixclock = 0; /* Need to drop locks to avoid recursive deadlock in @@ -1840,6 +1823,10 @@ __drm_fb_helper_initial_config_and_unlock(struct drm_fb_helper *fb_helper) mutex_unlock(&kernel_fb_helper_lock); return 0; + +err_drm_fb_helper_release_info: + drm_fb_helper_release_info(fb_helper); + return ret; } /** diff --git a/drivers/gpu/drm/drm_fbdev_dma.c b/drivers/gpu/drm/drm_fbdev_dma.c index 6fcf2a8bf676..b7ec75770f08 100644 --- a/drivers/gpu/drm/drm_fbdev_dma.c +++ b/drivers/gpu/drm/drm_fbdev_dma.c @@ -280,9 +280,9 @@ int drm_fbdev_dma_driver_fbdev_probe(struct drm_fb_helper *fb_helper, { struct drm_client_dev *client = &fb_helper->client; struct drm_device *dev = fb_helper->dev; + struct fb_info *info = fb_helper->info; struct drm_client_buffer *buffer; struct drm_framebuffer *fb; - struct fb_info *info; u32 format; struct iosys_map map; int ret; @@ -312,12 +312,6 @@ int drm_fbdev_dma_driver_fbdev_probe(struct drm_fb_helper *fb_helper, fb_helper->buffer = buffer; fb_helper->fb = fb; - info = drm_fb_helper_alloc_info(fb_helper); - if (IS_ERR(info)) { - ret = PTR_ERR(info); - goto err_drm_client_buffer_vunmap; - } - drm_fb_helper_fill_info(info, fb_helper, sizes); if (fb->funcs->dirty) @@ -325,12 +319,10 @@ int drm_fbdev_dma_driver_fbdev_probe(struct drm_fb_helper *fb_helper, else ret = drm_fbdev_dma_driver_fbdev_probe_tail(fb_helper, sizes); if (ret) - goto err_drm_fb_helper_release_info; + goto err_drm_client_buffer_vunmap; return 0; -err_drm_fb_helper_release_info: - drm_fb_helper_release_info(fb_helper); err_drm_client_buffer_vunmap: fb_helper->fb = NULL; fb_helper->buffer = NULL; diff --git a/drivers/gpu/drm/drm_fbdev_shmem.c b/drivers/gpu/drm/drm_fbdev_shmem.c index 0c785007f11b..e038377449d4 100644 --- a/drivers/gpu/drm/drm_fbdev_shmem.c +++ b/drivers/gpu/drm/drm_fbdev_shmem.c @@ -110,10 +110,10 @@ static int drm_fbdev_shmem_helper_fb_probe(struct drm_fb_helper *fb_helper, { struct drm_client_dev *client = &fb_helper->client; struct drm_device *dev = fb_helper->dev; + struct fb_info *info = fb_helper->info; struct drm_client_buffer *buffer; struct drm_gem_shmem_object *shmem; struct drm_framebuffer *fb; - struct fb_info *info; u32 format; struct iosys_map map; int ret; @@ -142,12 +142,6 @@ static int drm_fbdev_shmem_helper_fb_probe(struct drm_fb_helper *fb_helper, fb_helper->buffer = buffer; fb_helper->fb = fb; - info = drm_fb_helper_alloc_info(fb_helper); - if (IS_ERR(info)) { - ret = PTR_ERR(info); - goto err_drm_client_buffer_vunmap; - } - drm_fb_helper_fill_info(info, fb_helper, sizes); info->fbops = &drm_fbdev_shmem_fb_ops; @@ -168,12 +162,10 @@ static int drm_fbdev_shmem_helper_fb_probe(struct drm_fb_helper *fb_helper, info->fbdefio = &fb_helper->fbdefio; ret = fb_deferred_io_init(info); if (ret) - goto err_drm_fb_helper_release_info; + goto err_drm_client_buffer_vunmap; return 0; -err_drm_fb_helper_release_info: - drm_fb_helper_release_info(fb_helper); err_drm_client_buffer_vunmap: fb_helper->fb = NULL; fb_helper->buffer = NULL; diff --git a/drivers/gpu/drm/drm_fbdev_ttm.c b/drivers/gpu/drm/drm_fbdev_ttm.c index d799cbe944cd..e3f6ae3d7bad 100644 --- a/drivers/gpu/drm/drm_fbdev_ttm.c +++ b/drivers/gpu/drm/drm_fbdev_ttm.c @@ -185,8 +185,8 @@ int drm_fbdev_ttm_driver_fbdev_probe(struct drm_fb_helper *fb_helper, { struct drm_client_dev *client = &fb_helper->client; struct drm_device *dev = fb_helper->dev; + struct fb_info *info = fb_helper->info; struct drm_client_buffer *buffer; - struct fb_info *info; size_t screen_size; void *screen_buffer; u32 format; @@ -214,12 +214,6 @@ int drm_fbdev_ttm_driver_fbdev_probe(struct drm_fb_helper *fb_helper, goto err_drm_client_framebuffer_delete; } - info = drm_fb_helper_alloc_info(fb_helper); - if (IS_ERR(info)) { - ret = PTR_ERR(info); - goto err_vfree; - } - drm_fb_helper_fill_info(info, fb_helper, sizes); info->fbops = &drm_fbdev_ttm_fb_ops; @@ -236,12 +230,10 @@ int drm_fbdev_ttm_driver_fbdev_probe(struct drm_fb_helper *fb_helper, info->fbdefio = &fb_helper->fbdefio; ret = fb_deferred_io_init(info); if (ret) - goto err_drm_fb_helper_release_info; + goto err_vfree; return 0; -err_drm_fb_helper_release_info: - drm_fb_helper_release_info(fb_helper); err_vfree: vfree(screen_buffer); err_drm_client_framebuffer_delete: diff --git a/drivers/gpu/drm/drm_gpuvm.c b/drivers/gpu/drm/drm_gpuvm.c index 2515cff33485..338a21a58ec5 100644 --- a/drivers/gpu/drm/drm_gpuvm.c +++ b/drivers/gpu/drm/drm_gpuvm.c @@ -1451,9 +1451,8 @@ drm_gpuvm_resv_add_fence(struct drm_gpuvm *gpuvm, enum dma_resv_usage extobj_usage) { struct drm_gem_object *obj; - unsigned long index; - drm_exec_for_each_locked_object(exec, index, obj) { + drm_exec_for_each_locked_object(exec, obj) { dma_resv_assert_held(obj->resv); dma_resv_add_fence(obj->resv, fence, drm_gpuvm_is_extobj(gpuvm, obj) ? diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c index 3d0f8d182506..70294ca6202f 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c @@ -535,7 +535,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data, ret = drm_sched_job_init(&submit->sched_job, &ctx->sched_entity[args->pipe], - 1, submit->ctx); + 1, submit->ctx, file->client_id); if (ret) goto err_submit_put; diff --git a/drivers/gpu/drm/exynos/exynos_drm_fbdev.c b/drivers/gpu/drm/exynos/exynos_drm_fbdev.c index a379c8ca435a..b24516998755 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_fbdev.c +++ b/drivers/gpu/drm/exynos/exynos_drm_fbdev.c @@ -60,18 +60,11 @@ static int exynos_drm_fbdev_update(struct drm_fb_helper *helper, struct drm_fb_helper_surface_size *sizes, struct exynos_drm_gem *exynos_gem) { - struct fb_info *fbi; + struct fb_info *fbi = helper->info; struct drm_framebuffer *fb = helper->fb; unsigned int size = fb->width * fb->height * fb->format->cpp[0]; unsigned long offset; - fbi = drm_fb_helper_alloc_info(helper); - if (IS_ERR(fbi)) { - DRM_DEV_ERROR(to_dma_dev(helper->dev), - "failed to allocate fb info.\n"); - return PTR_ERR(fbi); - } - fbi->fbops = &exynos_drm_fb_ops; drm_fb_helper_fill_info(fbi, helper, sizes); diff --git a/drivers/gpu/drm/gma500/fbdev.c b/drivers/gpu/drm/gma500/fbdev.c index 8e3128298728..d80a18fde1aa 100644 --- a/drivers/gpu/drm/gma500/fbdev.c +++ b/drivers/gpu/drm/gma500/fbdev.c @@ -110,7 +110,7 @@ static int psb_fbdev_fb_probe(struct drm_fb_helper *fb_helper, struct drm_device *dev = fb_helper->dev; struct drm_psb_private *dev_priv = to_drm_psb_private(dev); struct pci_dev *pdev = to_pci_dev(dev->dev); - struct fb_info *info; + struct fb_info *info = fb_helper->info; struct drm_framebuffer *fb; struct drm_mode_fb_cmd2 mode_cmd = { }; int size; @@ -165,12 +165,6 @@ static int psb_fbdev_fb_probe(struct drm_fb_helper *fb_helper, fb_helper->fb = fb; - info = drm_fb_helper_alloc_info(fb_helper); - if (IS_ERR(info)) { - ret = PTR_ERR(info); - goto err_drm_framebuffer_unregister_private; - } - info->fbops = &psb_fbdev_fb_ops; /* Accessed stolen memory directly */ @@ -194,11 +188,6 @@ static int psb_fbdev_fb_probe(struct drm_fb_helper *fb_helper, return 0; -err_drm_framebuffer_unregister_private: - drm_framebuffer_unregister_private(fb); - fb->obj[0] = NULL; - drm_framebuffer_cleanup(fb); - kfree(fb); err_drm_gem_object_put: drm_gem_object_put(obj); return ret; diff --git a/drivers/gpu/drm/i915/display/intel_fbdev.c b/drivers/gpu/drm/i915/display/intel_fbdev.c index 337cc9fc31b1..2420be4c3127 100644 --- a/drivers/gpu/drm/i915/display/intel_fbdev.c +++ b/drivers/gpu/drm/i915/display/intel_fbdev.c @@ -183,7 +183,7 @@ static int intelfb_create(struct drm_fb_helper *helper, .type = I915_GTT_VIEW_NORMAL, }; intel_wakeref_t wakeref; - struct fb_info *info; + struct fb_info *info = helper->info; struct i915_vma *vma; unsigned long flags = 0; bool prealloc = false; @@ -236,13 +236,6 @@ static int intelfb_create(struct drm_fb_helper *helper, goto out_unlock; } - info = drm_fb_helper_alloc_info(helper); - if (IS_ERR(info)) { - drm_err(&dev_priv->drm, "Failed to allocate fb_info (%pe)\n", info); - ret = PTR_ERR(info); - goto out_unpin; - } - ifbdev->helper.fb = &fb->base; info->fbops = &intelfb_ops; diff --git a/drivers/gpu/drm/i915/display/intel_hdcp.c b/drivers/gpu/drm/i915/display/intel_hdcp.c index 377939de0ff4..ccc48dc14689 100644 --- a/drivers/gpu/drm/i915/display/intel_hdcp.c +++ b/drivers/gpu/drm/i915/display/intel_hdcp.c @@ -35,20 +35,20 @@ static void intel_hdcp_disable_hdcp_line_rekeying(struct intel_encoder *encoder, struct intel_hdcp *hdcp) { - struct drm_i915_private *dev_priv = to_i915(encoder->base.dev); + struct intel_display *display = to_intel_display(encoder); /* Here we assume HDMI is in TMDS mode of operation */ if (encoder->type != INTEL_OUTPUT_HDMI) return; - if (DISPLAY_VER(dev_priv) >= 14) { - if (IS_DISPLAY_VER_STEP(dev_priv, IP_VER(14, 0), STEP_D0, STEP_FOREVER)) - intel_de_rmw(dev_priv, MTL_CHICKEN_TRANS(hdcp->cpu_transcoder), + if (DISPLAY_VER(display) >= 14) { + if (IS_DISPLAY_VER_STEP(display, IP_VER(14, 0), STEP_D0, STEP_FOREVER)) + intel_de_rmw(display, MTL_CHICKEN_TRANS(hdcp->cpu_transcoder), 0, HDCP_LINE_REKEY_DISABLE); - else if (IS_DISPLAY_VER_STEP(dev_priv, IP_VER(14, 1), STEP_B0, STEP_FOREVER) || - IS_DISPLAY_VER_STEP(dev_priv, IP_VER(20, 0), STEP_B0, STEP_FOREVER)) - intel_de_rmw(dev_priv, - TRANS_DDI_FUNC_CTL(dev_priv, hdcp->cpu_transcoder), + else if (IS_DISPLAY_VER_STEP(display, IP_VER(14, 1), STEP_B0, STEP_FOREVER) || + IS_DISPLAY_VER_STEP(display, IP_VER(20, 0), STEP_B0, STEP_FOREVER)) + intel_de_rmw(display, + TRANS_DDI_FUNC_CTL(display, hdcp->cpu_transcoder), 0, TRANS_DDI_HDCP_LINE_REKEY_DISABLE); } } @@ -95,10 +95,11 @@ static int intel_hdcp_required_content_stream(struct intel_atomic_state *state, struct intel_digital_port *dig_port) { + struct intel_display *display = to_intel_display(state); struct drm_connector_list_iter conn_iter; + struct drm_connector_state *new_conn_state; struct intel_digital_port *conn_dig_port; struct intel_connector *connector; - struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev); struct hdcp_port_data *data = &dig_port->hdcp_port_data; bool enforce_type0 = false; int k; @@ -111,7 +112,7 @@ intel_hdcp_required_content_stream(struct intel_atomic_state *state, if (!dig_port->hdcp_mst_type1_capable) enforce_type0 = true; - drm_connector_list_iter_begin(&i915->drm, &conn_iter); + drm_connector_list_iter_begin(display->drm, &conn_iter); for_each_intel_connector_iter(connector, &conn_iter) { if (connector->base.status == connector_status_disconnected) continue; @@ -123,6 +124,14 @@ intel_hdcp_required_content_stream(struct intel_atomic_state *state, if (conn_dig_port != dig_port) continue; + new_conn_state = drm_atomic_get_new_connector_state(&state->base, + &connector->base); + if (!new_conn_state || !new_conn_state->crtc) + continue; + + if (drm_WARN_ON(display->drm, data->k >= INTEL_NUM_PIPES(display))) + return -EINVAL; + data->streams[data->k].stream_id = intel_conn_to_vcpi(state, connector); data->k++; @@ -133,7 +142,7 @@ intel_hdcp_required_content_stream(struct intel_atomic_state *state, } drm_connector_list_iter_end(&conn_iter); - if (drm_WARN_ON(&i915->drm, data->k > INTEL_NUM_PIPES(i915) || data->k == 0)) + if (drm_WARN_ON(display->drm, !data->k)) return -EINVAL; /* @@ -181,7 +190,7 @@ static int intel_hdcp_read_valid_bksv(struct intel_digital_port *dig_port, const struct intel_hdcp_shim *shim, u8 *bksv) { - struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev); + struct intel_display *display = to_intel_display(dig_port); int ret, i, tries = 2; /* HDCP spec states that we must retry the bksv if it is invalid */ @@ -193,7 +202,7 @@ int intel_hdcp_read_valid_bksv(struct intel_digital_port *dig_port, break; } if (i == tries) { - drm_dbg_kms(&i915->drm, "Bksv is invalid\n"); + drm_dbg_kms(display->drm, "Bksv is invalid\n"); return -ENODEV; } @@ -232,7 +241,7 @@ bool intel_hdcp_get_capability(struct intel_connector *connector) */ static bool intel_hdcp2_prerequisite(struct intel_connector *connector) { - struct drm_i915_private *i915 = to_i915(connector->base.dev); + struct intel_display *display = to_intel_display(connector); struct intel_hdcp *hdcp = &connector->hdcp; /* I915 support for HDCP2.2 */ @@ -240,18 +249,18 @@ static bool intel_hdcp2_prerequisite(struct intel_connector *connector) return false; /* If MTL+ make sure gsc is loaded and proxy is setup */ - if (intel_hdcp_gsc_cs_required(i915)) { - if (!intel_hdcp_gsc_check_status(i915)) + if (intel_hdcp_gsc_cs_required(display)) { + if (!intel_hdcp_gsc_check_status(display)) return false; } /* MEI/GSC interface is solid depending on which is used */ - mutex_lock(&i915->display.hdcp.hdcp_mutex); - if (!i915->display.hdcp.comp_added || !i915->display.hdcp.arbiter) { - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_lock(&display->hdcp.hdcp_mutex); + if (!display->hdcp.comp_added || !display->hdcp.arbiter) { + mutex_unlock(&display->hdcp.hdcp_mutex); return false; } - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return true; } @@ -287,19 +296,19 @@ void intel_hdcp_get_remote_capability(struct intel_connector *connector, *hdcp2_capable = false; } -static bool intel_hdcp_in_use(struct drm_i915_private *i915, +static bool intel_hdcp_in_use(struct intel_display *display, enum transcoder cpu_transcoder, enum port port) { - return intel_de_read(i915, - HDCP_STATUS(i915, cpu_transcoder, port)) & + return intel_de_read(display, + HDCP_STATUS(display, cpu_transcoder, port)) & HDCP_STATUS_ENC; } -static bool intel_hdcp2_in_use(struct drm_i915_private *i915, +static bool intel_hdcp2_in_use(struct intel_display *display, enum transcoder cpu_transcoder, enum port port) { - return intel_de_read(i915, - HDCP2_STATUS(i915, cpu_transcoder, port)) & + return intel_de_read(display, + HDCP2_STATUS(display, cpu_transcoder, port)) & LINK_ENCRYPTION_STATUS; } @@ -324,8 +333,9 @@ static int intel_hdcp_poll_ksv_fifo(struct intel_digital_port *dig_port, return 0; } -static bool hdcp_key_loadable(struct drm_i915_private *i915) +static bool hdcp_key_loadable(struct intel_display *display) { + struct drm_i915_private *i915 = to_i915(display->drm); enum i915_power_well_id id; intel_wakeref_t wakeref; bool enabled = false; @@ -352,19 +362,20 @@ static bool hdcp_key_loadable(struct drm_i915_private *i915) return enabled; } -static void intel_hdcp_clear_keys(struct drm_i915_private *i915) +static void intel_hdcp_clear_keys(struct intel_display *display) { - intel_de_write(i915, HDCP_KEY_CONF, HDCP_CLEAR_KEYS_TRIGGER); - intel_de_write(i915, HDCP_KEY_STATUS, + intel_de_write(display, HDCP_KEY_CONF, HDCP_CLEAR_KEYS_TRIGGER); + intel_de_write(display, HDCP_KEY_STATUS, HDCP_KEY_LOAD_DONE | HDCP_KEY_LOAD_STATUS | HDCP_FUSE_IN_PROGRESS | HDCP_FUSE_ERROR | HDCP_FUSE_DONE); } -static int intel_hdcp_load_keys(struct drm_i915_private *i915) +static int intel_hdcp_load_keys(struct intel_display *display) { + struct drm_i915_private *i915 = to_i915(display->drm); int ret; u32 val; - val = intel_de_read(i915, HDCP_KEY_STATUS); + val = intel_de_read(display, HDCP_KEY_STATUS); if ((val & HDCP_KEY_LOAD_DONE) && (val & HDCP_KEY_LOAD_STATUS)) return 0; @@ -373,7 +384,7 @@ static int intel_hdcp_load_keys(struct drm_i915_private *i915) * out of reset. So if Key is not already loaded, its an error state. */ if (IS_HASWELL(i915) || IS_BROADWELL(i915)) - if (!(intel_de_read(i915, HDCP_KEY_STATUS) & HDCP_KEY_LOAD_DONE)) + if (!(intel_de_read(display, HDCP_KEY_STATUS) & HDCP_KEY_LOAD_DONE)) return -ENXIO; /* @@ -384,20 +395,20 @@ static int intel_hdcp_load_keys(struct drm_i915_private *i915) * process from other platforms. These platforms use the GT Driver * Mailbox interface. */ - if (DISPLAY_VER(i915) == 9 && !IS_BROXTON(i915)) { + if (DISPLAY_VER(display) == 9 && !IS_BROXTON(i915)) { ret = snb_pcode_write(&i915->uncore, SKL_PCODE_LOAD_HDCP_KEYS, 1); if (ret) { - drm_err(&i915->drm, + drm_err(display->drm, "Failed to initiate HDCP key load (%d)\n", ret); return ret; } } else { - intel_de_write(i915, HDCP_KEY_CONF, HDCP_KEY_LOAD_TRIGGER); + intel_de_write(display, HDCP_KEY_CONF, HDCP_KEY_LOAD_TRIGGER); } /* Wait for the keys to load (500us) */ - ret = intel_de_wait_custom(i915, HDCP_KEY_STATUS, + ret = intel_de_wait_custom(display, HDCP_KEY_STATUS, HDCP_KEY_LOAD_DONE, HDCP_KEY_LOAD_DONE, 10, 1, &val); if (ret) @@ -406,27 +417,27 @@ static int intel_hdcp_load_keys(struct drm_i915_private *i915) return -ENXIO; /* Send Aksv over to PCH display for use in authentication */ - intel_de_write(i915, HDCP_KEY_CONF, HDCP_AKSV_SEND_TRIGGER); + intel_de_write(display, HDCP_KEY_CONF, HDCP_AKSV_SEND_TRIGGER); return 0; } /* Returns updated SHA-1 index */ -static int intel_write_sha_text(struct drm_i915_private *i915, u32 sha_text) +static int intel_write_sha_text(struct intel_display *display, u32 sha_text) { - intel_de_write(i915, HDCP_SHA_TEXT, sha_text); - if (intel_de_wait_for_set(i915, HDCP_REP_CTL, HDCP_SHA1_READY, 1)) { - drm_err(&i915->drm, "Timed out waiting for SHA1 ready\n"); + intel_de_write(display, HDCP_SHA_TEXT, sha_text); + if (intel_de_wait_for_set(display, HDCP_REP_CTL, HDCP_SHA1_READY, 1)) { + drm_err(display->drm, "Timed out waiting for SHA1 ready\n"); return -ETIMEDOUT; } return 0; } static -u32 intel_hdcp_get_repeater_ctl(struct drm_i915_private *i915, +u32 intel_hdcp_get_repeater_ctl(struct intel_display *display, enum transcoder cpu_transcoder, enum port port) { - if (DISPLAY_VER(i915) >= 12) { + if (DISPLAY_VER(display) >= 12) { switch (cpu_transcoder) { case TRANSCODER_A: return HDCP_TRANSA_REP_PRESENT | @@ -441,7 +452,7 @@ u32 intel_hdcp_get_repeater_ctl(struct drm_i915_private *i915, return HDCP_TRANSD_REP_PRESENT | HDCP_TRANSD_SHA1_M0; default: - drm_err(&i915->drm, "Unknown transcoder %d\n", + drm_err(display->drm, "Unknown transcoder %d\n", cpu_transcoder); return 0; } @@ -459,7 +470,7 @@ u32 intel_hdcp_get_repeater_ctl(struct drm_i915_private *i915, case PORT_E: return HDCP_DDIE_REP_PRESENT | HDCP_DDIE_SHA1_M0; default: - drm_err(&i915->drm, "Unknown port %d\n", port); + drm_err(display->drm, "Unknown port %d\n", port); return 0; } } @@ -469,8 +480,8 @@ int intel_hdcp_validate_v_prime(struct intel_connector *connector, const struct intel_hdcp_shim *shim, u8 *ksv_fifo, u8 num_downstream, u8 *bstatus) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); - struct drm_i915_private *i915 = to_i915(connector->base.dev); enum transcoder cpu_transcoder = connector->hdcp.cpu_transcoder; enum port port = dig_port->base.port; u32 vprime, sha_text, sha_leftovers, rep_ctl; @@ -481,7 +492,7 @@ int intel_hdcp_validate_v_prime(struct intel_connector *connector, ret = shim->read_v_prime_part(dig_port, i, &vprime); if (ret) return ret; - intel_de_write(i915, HDCP_SHA_V_PRIME(i), vprime); + intel_de_write(display, HDCP_SHA_V_PRIME(i), vprime); } /* @@ -497,8 +508,8 @@ int intel_hdcp_validate_v_prime(struct intel_connector *connector, sha_idx = 0; sha_text = 0; sha_leftovers = 0; - rep_ctl = intel_hdcp_get_repeater_ctl(i915, cpu_transcoder, port); - intel_de_write(i915, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_32); + rep_ctl = intel_hdcp_get_repeater_ctl(display, cpu_transcoder, port); + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_32); for (i = 0; i < num_downstream; i++) { unsigned int sha_empty; u8 *ksv = &ksv_fifo[i * DRM_HDCP_KSV_LEN]; @@ -510,14 +521,14 @@ int intel_hdcp_validate_v_prime(struct intel_connector *connector, sha_text |= ksv[j] << off; } - ret = intel_write_sha_text(i915, sha_text); + ret = intel_write_sha_text(display, sha_text); if (ret < 0) return ret; /* Programming guide writes this every 64 bytes */ sha_idx += sizeof(sha_text); if (!(sha_idx % 64)) - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_32); /* Store the leftover bytes from the ksv in sha_text */ @@ -534,7 +545,7 @@ int intel_hdcp_validate_v_prime(struct intel_connector *connector, if (sizeof(sha_text) > sha_leftovers) continue; - ret = intel_write_sha_text(i915, sha_text); + ret = intel_write_sha_text(display, sha_text); if (ret < 0) return ret; sha_leftovers = 0; @@ -550,73 +561,73 @@ int intel_hdcp_validate_v_prime(struct intel_connector *connector, */ if (sha_leftovers == 0) { /* Write 16 bits of text, 16 bits of M0 */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_16); - ret = intel_write_sha_text(i915, + ret = intel_write_sha_text(display, bstatus[0] << 8 | bstatus[1]); if (ret < 0) return ret; sha_idx += sizeof(sha_text); /* Write 32 bits of M0 */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_0); - ret = intel_write_sha_text(i915, 0); + ret = intel_write_sha_text(display, 0); if (ret < 0) return ret; sha_idx += sizeof(sha_text); /* Write 16 bits of M0 */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_16); - ret = intel_write_sha_text(i915, 0); + ret = intel_write_sha_text(display, 0); if (ret < 0) return ret; sha_idx += sizeof(sha_text); } else if (sha_leftovers == 1) { /* Write 24 bits of text, 8 bits of M0 */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_24); sha_text |= bstatus[0] << 16 | bstatus[1] << 8; /* Only 24-bits of data, must be in the LSB */ sha_text = (sha_text & 0xffffff00) >> 8; - ret = intel_write_sha_text(i915, sha_text); + ret = intel_write_sha_text(display, sha_text); if (ret < 0) return ret; sha_idx += sizeof(sha_text); /* Write 32 bits of M0 */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_0); - ret = intel_write_sha_text(i915, 0); + ret = intel_write_sha_text(display, 0); if (ret < 0) return ret; sha_idx += sizeof(sha_text); /* Write 24 bits of M0 */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_8); - ret = intel_write_sha_text(i915, 0); + ret = intel_write_sha_text(display, 0); if (ret < 0) return ret; sha_idx += sizeof(sha_text); } else if (sha_leftovers == 2) { /* Write 32 bits of text */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_32); sha_text |= bstatus[0] << 8 | bstatus[1]; - ret = intel_write_sha_text(i915, sha_text); + ret = intel_write_sha_text(display, sha_text); if (ret < 0) return ret; sha_idx += sizeof(sha_text); /* Write 64 bits of M0 */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_0); for (i = 0; i < 2; i++) { - ret = intel_write_sha_text(i915, 0); + ret = intel_write_sha_text(display, 0); if (ret < 0) return ret; sha_idx += sizeof(sha_text); @@ -626,56 +637,56 @@ int intel_hdcp_validate_v_prime(struct intel_connector *connector, * Terminate the SHA-1 stream by hand. For the other leftover * cases this is appended by the hardware. */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_32); sha_text = DRM_HDCP_SHA1_TERMINATOR << 24; - ret = intel_write_sha_text(i915, sha_text); + ret = intel_write_sha_text(display, sha_text); if (ret < 0) return ret; sha_idx += sizeof(sha_text); } else if (sha_leftovers == 3) { /* Write 32 bits of text (filled from LSB) */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_32); sha_text |= bstatus[0]; - ret = intel_write_sha_text(i915, sha_text); + ret = intel_write_sha_text(display, sha_text); if (ret < 0) return ret; sha_idx += sizeof(sha_text); /* Write 8 bits of text (filled from LSB), 24 bits of M0 */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_8); - ret = intel_write_sha_text(i915, bstatus[1]); + ret = intel_write_sha_text(display, bstatus[1]); if (ret < 0) return ret; sha_idx += sizeof(sha_text); /* Write 32 bits of M0 */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_0); - ret = intel_write_sha_text(i915, 0); + ret = intel_write_sha_text(display, 0); if (ret < 0) return ret; sha_idx += sizeof(sha_text); /* Write 8 bits of M0 */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_24); - ret = intel_write_sha_text(i915, 0); + ret = intel_write_sha_text(display, 0); if (ret < 0) return ret; sha_idx += sizeof(sha_text); } else { - drm_dbg_kms(&i915->drm, "Invalid number of leftovers %d\n", + drm_dbg_kms(display->drm, "Invalid number of leftovers %d\n", sha_leftovers); return -EINVAL; } - intel_de_write(i915, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_32); + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_TEXT_32); /* Fill up to 64-4 bytes with zeros (leave the last write for length) */ while ((sha_idx % 64) < (64 - sizeof(sha_text))) { - ret = intel_write_sha_text(i915, 0); + ret = intel_write_sha_text(display, 0); if (ret < 0) return ret; sha_idx += sizeof(sha_text); @@ -687,20 +698,20 @@ int intel_hdcp_validate_v_prime(struct intel_connector *connector, * - 10 bytes for BINFO/BSTATUS(2), M0(8) */ sha_text = (num_downstream * 5 + 10) * 8; - ret = intel_write_sha_text(i915, sha_text); + ret = intel_write_sha_text(display, sha_text); if (ret < 0) return ret; /* Tell the HW we're done with the hash and wait for it to ACK */ - intel_de_write(i915, HDCP_REP_CTL, + intel_de_write(display, HDCP_REP_CTL, rep_ctl | HDCP_SHA1_COMPLETE_HASH); - if (intel_de_wait_for_set(i915, HDCP_REP_CTL, + if (intel_de_wait_for_set(display, HDCP_REP_CTL, HDCP_SHA1_COMPLETE, 1)) { - drm_err(&i915->drm, "Timed out waiting for SHA1 complete\n"); + drm_err(display->drm, "Timed out waiting for SHA1 complete\n"); return -ETIMEDOUT; } - if (!(intel_de_read(i915, HDCP_REP_CTL) & HDCP_SHA1_V_MATCH)) { - drm_dbg_kms(&i915->drm, "SHA-1 mismatch, HDCP failed\n"); + if (!(intel_de_read(display, HDCP_REP_CTL) & HDCP_SHA1_V_MATCH)) { + drm_dbg_kms(display->drm, "SHA-1 mismatch, HDCP failed\n"); return -ENXIO; } @@ -711,15 +722,15 @@ int intel_hdcp_validate_v_prime(struct intel_connector *connector, static int intel_hdcp_auth_downstream(struct intel_connector *connector) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); - struct drm_i915_private *i915 = to_i915(connector->base.dev); const struct intel_hdcp_shim *shim = connector->hdcp.shim; u8 bstatus[2], num_downstream, *ksv_fifo; int ret, i, tries = 3; ret = intel_hdcp_poll_ksv_fifo(dig_port, shim); if (ret) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "KSV list failed to become ready (%d)\n", ret); return ret; } @@ -730,7 +741,7 @@ int intel_hdcp_auth_downstream(struct intel_connector *connector) if (DRM_HDCP_MAX_DEVICE_EXCEEDED(bstatus[0]) || DRM_HDCP_MAX_CASCADE_EXCEEDED(bstatus[1])) { - drm_dbg_kms(&i915->drm, "Max Topology Limit Exceeded\n"); + drm_dbg_kms(display->drm, "Max Topology Limit Exceeded\n"); return -EPERM; } @@ -743,14 +754,14 @@ int intel_hdcp_auth_downstream(struct intel_connector *connector) */ num_downstream = DRM_HDCP_NUM_DOWNSTREAM(bstatus[0]); if (num_downstream == 0) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "Repeater with zero downstream devices\n"); return -EINVAL; } ksv_fifo = kcalloc(DRM_HDCP_KSV_LEN, num_downstream, GFP_KERNEL); if (!ksv_fifo) { - drm_dbg_kms(&i915->drm, "Out of mem: ksv_fifo\n"); + drm_dbg_kms(display->drm, "Out of mem: ksv_fifo\n"); return -ENOMEM; } @@ -758,9 +769,9 @@ int intel_hdcp_auth_downstream(struct intel_connector *connector) if (ret) goto err; - if (drm_hdcp_check_ksvs_revoked(&i915->drm, ksv_fifo, + if (drm_hdcp_check_ksvs_revoked(display->drm, ksv_fifo, num_downstream) > 0) { - drm_err(&i915->drm, "Revoked Ksv(s) in ksv_fifo\n"); + drm_err(display->drm, "Revoked Ksv(s) in ksv_fifo\n"); ret = -EPERM; goto err; } @@ -778,12 +789,12 @@ int intel_hdcp_auth_downstream(struct intel_connector *connector) } if (i == tries) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "V Prime validation failed.(%d)\n", ret); goto err; } - drm_dbg_kms(&i915->drm, "HDCP is enabled (%d downstream devices)\n", + drm_dbg_kms(display->drm, "HDCP is enabled (%d downstream devices)\n", num_downstream); ret = 0; err: @@ -794,8 +805,8 @@ int intel_hdcp_auth_downstream(struct intel_connector *connector) /* Implements Part 1 of the HDCP authorization procedure */ static int intel_hdcp_auth(struct intel_connector *connector) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct intel_hdcp *hdcp = &connector->hdcp; const struct intel_hdcp_shim *shim = hdcp->shim; enum transcoder cpu_transcoder = connector->hdcp.cpu_transcoder; @@ -827,7 +838,7 @@ static int intel_hdcp_auth(struct intel_connector *connector) if (ret) return ret; if (!hdcp_capable) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "Panel is not HDCP capable\n"); return -EINVAL; } @@ -835,24 +846,24 @@ static int intel_hdcp_auth(struct intel_connector *connector) /* Initialize An with 2 random values and acquire it */ for (i = 0; i < 2; i++) - intel_de_write(i915, - HDCP_ANINIT(i915, cpu_transcoder, port), + intel_de_write(display, + HDCP_ANINIT(display, cpu_transcoder, port), get_random_u32()); - intel_de_write(i915, HDCP_CONF(i915, cpu_transcoder, port), + intel_de_write(display, HDCP_CONF(display, cpu_transcoder, port), HDCP_CONF_CAPTURE_AN); /* Wait for An to be acquired */ - if (intel_de_wait_for_set(i915, - HDCP_STATUS(i915, cpu_transcoder, port), + if (intel_de_wait_for_set(display, + HDCP_STATUS(display, cpu_transcoder, port), HDCP_STATUS_AN_READY, 1)) { - drm_err(&i915->drm, "Timed out waiting for An\n"); + drm_err(display->drm, "Timed out waiting for An\n"); return -ETIMEDOUT; } - an.reg[0] = intel_de_read(i915, - HDCP_ANLO(i915, cpu_transcoder, port)); - an.reg[1] = intel_de_read(i915, - HDCP_ANHI(i915, cpu_transcoder, port)); + an.reg[0] = intel_de_read(display, + HDCP_ANLO(display, cpu_transcoder, port)); + an.reg[1] = intel_de_read(display, + HDCP_ANHI(display, cpu_transcoder, port)); ret = shim->write_an_aksv(dig_port, an.shim); if (ret) return ret; @@ -865,34 +876,34 @@ static int intel_hdcp_auth(struct intel_connector *connector) if (ret < 0) return ret; - if (drm_hdcp_check_ksvs_revoked(&i915->drm, bksv.shim, 1) > 0) { - drm_err(&i915->drm, "BKSV is revoked\n"); + if (drm_hdcp_check_ksvs_revoked(display->drm, bksv.shim, 1) > 0) { + drm_err(display->drm, "BKSV is revoked\n"); return -EPERM; } - intel_de_write(i915, HDCP_BKSVLO(i915, cpu_transcoder, port), + intel_de_write(display, HDCP_BKSVLO(display, cpu_transcoder, port), bksv.reg[0]); - intel_de_write(i915, HDCP_BKSVHI(i915, cpu_transcoder, port), + intel_de_write(display, HDCP_BKSVHI(display, cpu_transcoder, port), bksv.reg[1]); ret = shim->repeater_present(dig_port, &repeater_present); if (ret) return ret; if (repeater_present) - intel_de_write(i915, HDCP_REP_CTL, - intel_hdcp_get_repeater_ctl(i915, cpu_transcoder, port)); + intel_de_write(display, HDCP_REP_CTL, + intel_hdcp_get_repeater_ctl(display, cpu_transcoder, port)); ret = shim->toggle_signalling(dig_port, cpu_transcoder, true); if (ret) return ret; - intel_de_write(i915, HDCP_CONF(i915, cpu_transcoder, port), + intel_de_write(display, HDCP_CONF(display, cpu_transcoder, port), HDCP_CONF_AUTH_AND_ENC); /* Wait for R0 ready */ - if (wait_for(intel_de_read(i915, HDCP_STATUS(i915, cpu_transcoder, port)) & + if (wait_for(intel_de_read(display, HDCP_STATUS(display, cpu_transcoder, port)) & (HDCP_STATUS_R0_READY | HDCP_STATUS_ENC), 1)) { - drm_err(&i915->drm, "Timed out waiting for R0 ready\n"); + drm_err(display->drm, "Timed out waiting for R0 ready\n"); return -ETIMEDOUT; } @@ -918,30 +929,30 @@ static int intel_hdcp_auth(struct intel_connector *connector) ret = shim->read_ri_prime(dig_port, ri.shim); if (ret) return ret; - intel_de_write(i915, - HDCP_RPRIME(i915, cpu_transcoder, port), + intel_de_write(display, + HDCP_RPRIME(display, cpu_transcoder, port), ri.reg); /* Wait for Ri prime match */ - if (!wait_for(intel_de_read(i915, HDCP_STATUS(i915, cpu_transcoder, port)) & + if (!wait_for(intel_de_read(display, HDCP_STATUS(display, cpu_transcoder, port)) & (HDCP_STATUS_RI_MATCH | HDCP_STATUS_ENC), 1)) break; } if (i == tries) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "Timed out waiting for Ri prime match (%x)\n", - intel_de_read(i915, - HDCP_STATUS(i915, cpu_transcoder, port))); + intel_de_read(display, + HDCP_STATUS(display, cpu_transcoder, port))); return -ETIMEDOUT; } /* Wait for encryption confirmation */ - if (intel_de_wait_for_set(i915, - HDCP_STATUS(i915, cpu_transcoder, port), + if (intel_de_wait_for_set(display, + HDCP_STATUS(display, cpu_transcoder, port), HDCP_STATUS_ENC, HDCP_ENCRYPT_STATUS_CHANGE_TIMEOUT_MS)) { - drm_err(&i915->drm, "Timed out waiting for encryption\n"); + drm_err(display->drm, "Timed out waiting for encryption\n"); return -ETIMEDOUT; } @@ -949,42 +960,42 @@ static int intel_hdcp_auth(struct intel_connector *connector) if (shim->stream_encryption) { ret = shim->stream_encryption(connector, true); if (ret) { - drm_err(&i915->drm, "[CONNECTOR:%d:%s] Failed to enable HDCP 1.4 stream enc\n", + drm_err(display->drm, "[CONNECTOR:%d:%s] Failed to enable HDCP 1.4 stream enc\n", connector->base.base.id, connector->base.name); return ret; } - drm_dbg_kms(&i915->drm, "HDCP 1.4 transcoder: %s stream encrypted\n", + drm_dbg_kms(display->drm, "HDCP 1.4 transcoder: %s stream encrypted\n", transcoder_name(hdcp->stream_transcoder)); } if (repeater_present) return intel_hdcp_auth_downstream(connector); - drm_dbg_kms(&i915->drm, "HDCP is enabled (no repeater present)\n"); + drm_dbg_kms(display->drm, "HDCP is enabled (no repeater present)\n"); return 0; } static int _intel_hdcp_disable(struct intel_connector *connector) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct intel_hdcp *hdcp = &connector->hdcp; enum port port = dig_port->base.port; enum transcoder cpu_transcoder = hdcp->cpu_transcoder; u32 repeater_ctl; int ret; - drm_dbg_kms(&i915->drm, "[CONNECTOR:%d:%s] HDCP is being disabled...\n", + drm_dbg_kms(display->drm, "[CONNECTOR:%d:%s] HDCP is being disabled...\n", connector->base.base.id, connector->base.name); if (hdcp->shim->stream_encryption) { ret = hdcp->shim->stream_encryption(connector, false); if (ret) { - drm_err(&i915->drm, "[CONNECTOR:%d:%s] Failed to disable HDCP 1.4 stream enc\n", + drm_err(display->drm, "[CONNECTOR:%d:%s] Failed to disable HDCP 1.4 stream enc\n", connector->base.base.id, connector->base.name); return ret; } - drm_dbg_kms(&i915->drm, "HDCP 1.4 transcoder: %s stream encryption disabled\n", + drm_dbg_kms(display->drm, "HDCP 1.4 transcoder: %s stream encryption disabled\n", transcoder_name(hdcp->stream_transcoder)); /* * If there are other connectors on this port using HDCP, @@ -996,51 +1007,51 @@ static int _intel_hdcp_disable(struct intel_connector *connector) } hdcp->hdcp_encrypted = false; - intel_de_write(i915, HDCP_CONF(i915, cpu_transcoder, port), 0); - if (intel_de_wait_for_clear(i915, - HDCP_STATUS(i915, cpu_transcoder, port), + intel_de_write(display, HDCP_CONF(display, cpu_transcoder, port), 0); + if (intel_de_wait_for_clear(display, + HDCP_STATUS(display, cpu_transcoder, port), ~0, HDCP_ENCRYPT_STATUS_CHANGE_TIMEOUT_MS)) { - drm_err(&i915->drm, + drm_err(display->drm, "Failed to disable HDCP, timeout clearing status\n"); return -ETIMEDOUT; } - repeater_ctl = intel_hdcp_get_repeater_ctl(i915, cpu_transcoder, + repeater_ctl = intel_hdcp_get_repeater_ctl(display, cpu_transcoder, port); - intel_de_rmw(i915, HDCP_REP_CTL, repeater_ctl, 0); + intel_de_rmw(display, HDCP_REP_CTL, repeater_ctl, 0); ret = hdcp->shim->toggle_signalling(dig_port, cpu_transcoder, false); if (ret) { - drm_err(&i915->drm, "Failed to disable HDCP signalling\n"); + drm_err(display->drm, "Failed to disable HDCP signalling\n"); return ret; } - drm_dbg_kms(&i915->drm, "HDCP is disabled\n"); + drm_dbg_kms(display->drm, "HDCP is disabled\n"); return 0; } static int intel_hdcp1_enable(struct intel_connector *connector) { - struct drm_i915_private *i915 = to_i915(connector->base.dev); + struct intel_display *display = to_intel_display(connector); struct intel_hdcp *hdcp = &connector->hdcp; int i, ret, tries = 3; - drm_dbg_kms(&i915->drm, "[CONNECTOR:%d:%s] HDCP is being enabled...\n", + drm_dbg_kms(display->drm, "[CONNECTOR:%d:%s] HDCP is being enabled...\n", connector->base.base.id, connector->base.name); - if (!hdcp_key_loadable(i915)) { - drm_err(&i915->drm, "HDCP key Load is not possible\n"); + if (!hdcp_key_loadable(display)) { + drm_err(display->drm, "HDCP key Load is not possible\n"); return -ENXIO; } for (i = 0; i < KEY_LOAD_TRIES; i++) { - ret = intel_hdcp_load_keys(i915); + ret = intel_hdcp_load_keys(display); if (!ret) break; - intel_hdcp_clear_keys(i915); + intel_hdcp_clear_keys(display); } if (ret) { - drm_err(&i915->drm, "Could not load HDCP keys, (%d)\n", + drm_err(display->drm, "Could not load HDCP keys, (%d)\n", ret); return ret; } @@ -1053,13 +1064,13 @@ static int intel_hdcp1_enable(struct intel_connector *connector) return 0; } - drm_dbg_kms(&i915->drm, "HDCP Auth failure (%d)\n", ret); + drm_dbg_kms(display->drm, "HDCP Auth failure (%d)\n", ret); /* Ensuring HDCP encryption and signalling are stopped. */ _intel_hdcp_disable(connector); } - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "HDCP authentication failed (%d tries/%d)\n", tries, ret); return ret; } @@ -1072,20 +1083,20 @@ static struct intel_connector *intel_hdcp_to_connector(struct intel_hdcp *hdcp) static void intel_hdcp_update_value(struct intel_connector *connector, u64 value, bool update_property) { - struct drm_device *dev = connector->base.dev; + struct intel_display *display = to_intel_display(connector); + struct drm_i915_private *i915 = to_i915(display->drm); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); struct intel_hdcp *hdcp = &connector->hdcp; - struct drm_i915_private *i915 = to_i915(connector->base.dev); - drm_WARN_ON(connector->base.dev, !mutex_is_locked(&hdcp->mutex)); + drm_WARN_ON(display->drm, !mutex_is_locked(&hdcp->mutex)); if (hdcp->value == value) return; - drm_WARN_ON(dev, !mutex_is_locked(&dig_port->hdcp_mutex)); + drm_WARN_ON(display->drm, !mutex_is_locked(&dig_port->hdcp_mutex)); if (hdcp->value == DRM_MODE_CONTENT_PROTECTION_ENABLED) { - if (!drm_WARN_ON(dev, dig_port->num_hdcp_streams == 0)) + if (!drm_WARN_ON(display->drm, dig_port->num_hdcp_streams == 0)) dig_port->num_hdcp_streams--; } else if (value == DRM_MODE_CONTENT_PROTECTION_ENABLED) { dig_port->num_hdcp_streams++; @@ -1102,8 +1113,8 @@ static void intel_hdcp_update_value(struct intel_connector *connector, /* Implements Part 3 of the HDCP authorization procedure */ static int intel_hdcp_check_link(struct intel_connector *connector) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct intel_hdcp *hdcp = &connector->hdcp; enum port port = dig_port->base.port; enum transcoder cpu_transcoder; @@ -1121,12 +1132,12 @@ static int intel_hdcp_check_link(struct intel_connector *connector) goto out; } - if (drm_WARN_ON(&i915->drm, - !intel_hdcp_in_use(i915, cpu_transcoder, port))) { - drm_err(&i915->drm, + if (drm_WARN_ON(display->drm, + !intel_hdcp_in_use(display, cpu_transcoder, port))) { + drm_err(display->drm, "[CONNECTOR:%d:%s] HDCP link stopped encryption,%x\n", connector->base.base.id, connector->base.name, - intel_de_read(i915, HDCP_STATUS(i915, cpu_transcoder, port))); + intel_de_read(display, HDCP_STATUS(display, cpu_transcoder, port))); ret = -ENXIO; intel_hdcp_update_value(connector, DRM_MODE_CONTENT_PROTECTION_DESIRED, @@ -1142,13 +1153,13 @@ static int intel_hdcp_check_link(struct intel_connector *connector) goto out; } - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "[CONNECTOR:%d:%s] HDCP link failed, retrying authentication\n", connector->base.base.id, connector->base.name); ret = _intel_hdcp_disable(connector); if (ret) { - drm_err(&i915->drm, "Failed to disable hdcp (%d)\n", ret); + drm_err(display->drm, "Failed to disable hdcp (%d)\n", ret); intel_hdcp_update_value(connector, DRM_MODE_CONTENT_PROTECTION_DESIRED, true); @@ -1169,9 +1180,9 @@ static void intel_hdcp_prop_work(struct work_struct *work) struct intel_hdcp *hdcp = container_of(work, struct intel_hdcp, prop_work); struct intel_connector *connector = intel_hdcp_to_connector(hdcp); - struct drm_i915_private *i915 = to_i915(connector->base.dev); + struct intel_display *display = to_intel_display(connector); - drm_modeset_lock(&i915->drm.mode_config.connection_mutex, NULL); + drm_modeset_lock(&display->drm->mode_config.connection_mutex, NULL); mutex_lock(&hdcp->mutex); /* @@ -1184,7 +1195,7 @@ static void intel_hdcp_prop_work(struct work_struct *work) hdcp->value); mutex_unlock(&hdcp->mutex); - drm_modeset_unlock(&i915->drm.mode_config.connection_mutex); + drm_modeset_unlock(&display->drm->mode_config.connection_mutex); drm_connector_put(&connector->base); } @@ -1199,25 +1210,25 @@ static int hdcp2_prepare_ake_init(struct intel_connector *connector, struct hdcp2_ake_init *ake_data) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); struct hdcp_port_data *data = &dig_port->hdcp_port_data; - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct i915_hdcp_arbiter *arbiter; int ret; - mutex_lock(&i915->display.hdcp.hdcp_mutex); - arbiter = i915->display.hdcp.arbiter; + mutex_lock(&display->hdcp.hdcp_mutex); + arbiter = display->hdcp.arbiter; if (!arbiter || !arbiter->ops) { - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return -EINVAL; } ret = arbiter->ops->initiate_hdcp2_session(arbiter->hdcp_dev, data, ake_data); if (ret) - drm_dbg_kms(&i915->drm, "Prepare_ake_init failed. %d\n", + drm_dbg_kms(display->drm, "Prepare_ake_init failed. %d\n", ret); - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return ret; } @@ -1229,17 +1240,17 @@ hdcp2_verify_rx_cert_prepare_km(struct intel_connector *connector, struct hdcp2_ake_no_stored_km *ek_pub_km, size_t *msg_sz) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); struct hdcp_port_data *data = &dig_port->hdcp_port_data; - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct i915_hdcp_arbiter *arbiter; int ret; - mutex_lock(&i915->display.hdcp.hdcp_mutex); - arbiter = i915->display.hdcp.arbiter; + mutex_lock(&display->hdcp.hdcp_mutex); + arbiter = display->hdcp.arbiter; if (!arbiter || !arbiter->ops) { - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return -EINVAL; } @@ -1247,9 +1258,9 @@ hdcp2_verify_rx_cert_prepare_km(struct intel_connector *connector, rx_cert, paired, ek_pub_km, msg_sz); if (ret < 0) - drm_dbg_kms(&i915->drm, "Verify rx_cert failed. %d\n", + drm_dbg_kms(display->drm, "Verify rx_cert failed. %d\n", ret); - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return ret; } @@ -1257,24 +1268,24 @@ hdcp2_verify_rx_cert_prepare_km(struct intel_connector *connector, static int hdcp2_verify_hprime(struct intel_connector *connector, struct hdcp2_ake_send_hprime *rx_hprime) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); struct hdcp_port_data *data = &dig_port->hdcp_port_data; - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct i915_hdcp_arbiter *arbiter; int ret; - mutex_lock(&i915->display.hdcp.hdcp_mutex); - arbiter = i915->display.hdcp.arbiter; + mutex_lock(&display->hdcp.hdcp_mutex); + arbiter = display->hdcp.arbiter; if (!arbiter || !arbiter->ops) { - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return -EINVAL; } ret = arbiter->ops->verify_hprime(arbiter->hdcp_dev, data, rx_hprime); if (ret < 0) - drm_dbg_kms(&i915->drm, "Verify hprime failed. %d\n", ret); - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + drm_dbg_kms(display->drm, "Verify hprime failed. %d\n", ret); + mutex_unlock(&display->hdcp.hdcp_mutex); return ret; } @@ -1283,25 +1294,25 @@ static int hdcp2_store_pairing_info(struct intel_connector *connector, struct hdcp2_ake_send_pairing_info *pairing_info) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); struct hdcp_port_data *data = &dig_port->hdcp_port_data; - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct i915_hdcp_arbiter *arbiter; int ret; - mutex_lock(&i915->display.hdcp.hdcp_mutex); - arbiter = i915->display.hdcp.arbiter; + mutex_lock(&display->hdcp.hdcp_mutex); + arbiter = display->hdcp.arbiter; if (!arbiter || !arbiter->ops) { - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return -EINVAL; } ret = arbiter->ops->store_pairing_info(arbiter->hdcp_dev, data, pairing_info); if (ret < 0) - drm_dbg_kms(&i915->drm, "Store pairing info failed. %d\n", + drm_dbg_kms(display->drm, "Store pairing info failed. %d\n", ret); - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return ret; } @@ -1310,25 +1321,25 @@ static int hdcp2_prepare_lc_init(struct intel_connector *connector, struct hdcp2_lc_init *lc_init) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); struct hdcp_port_data *data = &dig_port->hdcp_port_data; - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct i915_hdcp_arbiter *arbiter; int ret; - mutex_lock(&i915->display.hdcp.hdcp_mutex); - arbiter = i915->display.hdcp.arbiter; + mutex_lock(&display->hdcp.hdcp_mutex); + arbiter = display->hdcp.arbiter; if (!arbiter || !arbiter->ops) { - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return -EINVAL; } ret = arbiter->ops->initiate_locality_check(arbiter->hdcp_dev, data, lc_init); if (ret < 0) - drm_dbg_kms(&i915->drm, "Prepare lc_init failed. %d\n", + drm_dbg_kms(display->drm, "Prepare lc_init failed. %d\n", ret); - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return ret; } @@ -1337,25 +1348,25 @@ static int hdcp2_verify_lprime(struct intel_connector *connector, struct hdcp2_lc_send_lprime *rx_lprime) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); struct hdcp_port_data *data = &dig_port->hdcp_port_data; - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct i915_hdcp_arbiter *arbiter; int ret; - mutex_lock(&i915->display.hdcp.hdcp_mutex); - arbiter = i915->display.hdcp.arbiter; + mutex_lock(&display->hdcp.hdcp_mutex); + arbiter = display->hdcp.arbiter; if (!arbiter || !arbiter->ops) { - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return -EINVAL; } ret = arbiter->ops->verify_lprime(arbiter->hdcp_dev, data, rx_lprime); if (ret < 0) - drm_dbg_kms(&i915->drm, "Verify L_Prime failed. %d\n", + drm_dbg_kms(display->drm, "Verify L_Prime failed. %d\n", ret); - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return ret; } @@ -1363,25 +1374,25 @@ hdcp2_verify_lprime(struct intel_connector *connector, static int hdcp2_prepare_skey(struct intel_connector *connector, struct hdcp2_ske_send_eks *ske_data) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); struct hdcp_port_data *data = &dig_port->hdcp_port_data; - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct i915_hdcp_arbiter *arbiter; int ret; - mutex_lock(&i915->display.hdcp.hdcp_mutex); - arbiter = i915->display.hdcp.arbiter; + mutex_lock(&display->hdcp.hdcp_mutex); + arbiter = display->hdcp.arbiter; if (!arbiter || !arbiter->ops) { - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return -EINVAL; } ret = arbiter->ops->get_session_key(arbiter->hdcp_dev, data, ske_data); if (ret < 0) - drm_dbg_kms(&i915->drm, "Get session key failed. %d\n", + drm_dbg_kms(display->drm, "Get session key failed. %d\n", ret); - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return ret; } @@ -1392,17 +1403,17 @@ hdcp2_verify_rep_topology_prepare_ack(struct intel_connector *connector, *rep_topology, struct hdcp2_rep_send_ack *rep_send_ack) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); struct hdcp_port_data *data = &dig_port->hdcp_port_data; - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct i915_hdcp_arbiter *arbiter; int ret; - mutex_lock(&i915->display.hdcp.hdcp_mutex); - arbiter = i915->display.hdcp.arbiter; + mutex_lock(&display->hdcp.hdcp_mutex); + arbiter = display->hdcp.arbiter; if (!arbiter || !arbiter->ops) { - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return -EINVAL; } @@ -1411,9 +1422,9 @@ hdcp2_verify_rep_topology_prepare_ack(struct intel_connector *connector, rep_topology, rep_send_ack); if (ret < 0) - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "Verify rep topology failed. %d\n", ret); - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return ret; } @@ -1422,71 +1433,71 @@ static int hdcp2_verify_mprime(struct intel_connector *connector, struct hdcp2_rep_stream_ready *stream_ready) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); struct hdcp_port_data *data = &dig_port->hdcp_port_data; - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct i915_hdcp_arbiter *arbiter; int ret; - mutex_lock(&i915->display.hdcp.hdcp_mutex); - arbiter = i915->display.hdcp.arbiter; + mutex_lock(&display->hdcp.hdcp_mutex); + arbiter = display->hdcp.arbiter; if (!arbiter || !arbiter->ops) { - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return -EINVAL; } ret = arbiter->ops->verify_mprime(arbiter->hdcp_dev, data, stream_ready); if (ret < 0) - drm_dbg_kms(&i915->drm, "Verify mprime failed. %d\n", ret); - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + drm_dbg_kms(display->drm, "Verify mprime failed. %d\n", ret); + mutex_unlock(&display->hdcp.hdcp_mutex); return ret; } static int hdcp2_authenticate_port(struct intel_connector *connector) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); struct hdcp_port_data *data = &dig_port->hdcp_port_data; - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct i915_hdcp_arbiter *arbiter; int ret; - mutex_lock(&i915->display.hdcp.hdcp_mutex); - arbiter = i915->display.hdcp.arbiter; + mutex_lock(&display->hdcp.hdcp_mutex); + arbiter = display->hdcp.arbiter; if (!arbiter || !arbiter->ops) { - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return -EINVAL; } ret = arbiter->ops->enable_hdcp_authentication(arbiter->hdcp_dev, data); if (ret < 0) - drm_dbg_kms(&i915->drm, "Enable hdcp auth failed. %d\n", + drm_dbg_kms(display->drm, "Enable hdcp auth failed. %d\n", ret); - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return ret; } static int hdcp2_close_session(struct intel_connector *connector) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct i915_hdcp_arbiter *arbiter; int ret; - mutex_lock(&i915->display.hdcp.hdcp_mutex); - arbiter = i915->display.hdcp.arbiter; + mutex_lock(&display->hdcp.hdcp_mutex); + arbiter = display->hdcp.arbiter; if (!arbiter || !arbiter->ops) { - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return -EINVAL; } ret = arbiter->ops->close_hdcp_session(arbiter->hdcp_dev, &dig_port->hdcp_port_data); - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_unlock(&display->hdcp.hdcp_mutex); return ret; } @@ -1499,7 +1510,7 @@ static int hdcp2_deauthenticate_port(struct intel_connector *connector) /* Authentication flow starts from here */ static int hdcp2_authentication_key_exchange(struct intel_connector *connector) { - struct drm_i915_private *i915 = to_i915(connector->base.dev); + struct intel_display *display = to_intel_display(connector); struct intel_hdcp *hdcp = &connector->hdcp; union { struct hdcp2_ake_init ake_init; @@ -1531,16 +1542,16 @@ static int hdcp2_authentication_key_exchange(struct intel_connector *connector) return ret; if (msgs.send_cert.rx_caps[0] != HDCP_2_2_RX_CAPS_VERSION_VAL) { - drm_dbg_kms(&i915->drm, "cert.rx_caps dont claim HDCP2.2\n"); + drm_dbg_kms(display->drm, "cert.rx_caps dont claim HDCP2.2\n"); return -EINVAL; } hdcp->is_repeater = HDCP_2_2_RX_REPEATER(msgs.send_cert.rx_caps[2]); - if (drm_hdcp_check_ksvs_revoked(&i915->drm, + if (drm_hdcp_check_ksvs_revoked(display->drm, msgs.send_cert.cert_rx.receiver_id, 1) > 0) { - drm_err(&i915->drm, "Receiver ID is revoked\n"); + drm_err(display->drm, "Receiver ID is revoked\n"); return -EPERM; } @@ -1691,8 +1702,8 @@ int _hdcp2_propagate_stream_management_info(struct intel_connector *connector) static int hdcp2_authenticate_repeater_topology(struct intel_connector *connector) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct intel_hdcp *hdcp = &connector->hdcp; union { struct hdcp2_rep_send_receiverid_list recvid_list; @@ -1712,7 +1723,7 @@ int hdcp2_authenticate_repeater_topology(struct intel_connector *connector) if (HDCP_2_2_MAX_CASCADE_EXCEEDED(rx_info[1]) || HDCP_2_2_MAX_DEVS_EXCEEDED(rx_info[1])) { - drm_dbg_kms(&i915->drm, "Topology Max Size Exceeded\n"); + drm_dbg_kms(display->drm, "Topology Max Size Exceeded\n"); return -EINVAL; } @@ -1725,7 +1736,7 @@ int hdcp2_authenticate_repeater_topology(struct intel_connector *connector) !HDCP_2_2_HDCP_2_0_REP_CONNECTED(rx_info[1]); if (!dig_port->hdcp_mst_type1_capable && hdcp->content_type) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "HDCP1.x or 2.0 Legacy Device Downstream\n"); return -EINVAL; } @@ -1735,23 +1746,24 @@ int hdcp2_authenticate_repeater_topology(struct intel_connector *connector) drm_hdcp_be24_to_cpu((const u8 *)msgs.recvid_list.seq_num_v); if (!hdcp->hdcp2_encrypted && seq_num_v) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "Non zero Seq_num_v at first RecvId_List msg\n"); return -EINVAL; } - if (seq_num_v < hdcp->seq_num_v) { - /* Roll over of the seq_num_v from repeater. Reauthenticate. */ - drm_dbg_kms(&i915->drm, "Seq_num_v roll over.\n"); + if (hdcp->hdcp2_encrypted && seq_num_v <= hdcp->seq_num_v) { + /* Reauthenticate on Seq_num_v repeat or rollover */ + drm_dbg_kms(display->drm, "Seq_num_v %s\n", + seq_num_v == hdcp->seq_num_v ? "repeat" : "rollover"); return -EINVAL; } device_cnt = (HDCP_2_2_DEV_COUNT_HI(rx_info[0]) << 4 | HDCP_2_2_DEV_COUNT_LO(rx_info[1])); - if (drm_hdcp_check_ksvs_revoked(&i915->drm, + if (drm_hdcp_check_ksvs_revoked(display->drm, msgs.recvid_list.receiver_ids, device_cnt) > 0) { - drm_err(&i915->drm, "Revoked receiver ID(s) is in list\n"); + drm_err(display->drm, "Revoked receiver ID(s) is in list\n"); return -EPERM; } @@ -1772,27 +1784,27 @@ int hdcp2_authenticate_repeater_topology(struct intel_connector *connector) static int hdcp2_authenticate_sink(struct intel_connector *connector) { - struct drm_i915_private *i915 = to_i915(connector->base.dev); + struct intel_display *display = to_intel_display(connector); struct intel_hdcp *hdcp = &connector->hdcp; const struct intel_hdcp_shim *shim = hdcp->shim; int ret; ret = hdcp2_authentication_key_exchange(connector); if (ret < 0) { - drm_dbg_kms(&i915->drm, "AKE Failed. Err : %d\n", ret); + drm_dbg_kms(display->drm, "AKE Failed. Err : %d\n", ret); return ret; } ret = hdcp2_locality_check(connector); if (ret < 0) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "Locality Check failed. Err : %d\n", ret); return ret; } ret = hdcp2_session_key_exchange(connector); if (ret < 0) { - drm_dbg_kms(&i915->drm, "SKE Failed. Err : %d\n", ret); + drm_dbg_kms(display->drm, "SKE Failed. Err : %d\n", ret); return ret; } @@ -1807,7 +1819,7 @@ static int hdcp2_authenticate_sink(struct intel_connector *connector) if (hdcp->is_repeater) { ret = hdcp2_authenticate_repeater_topology(connector); if (ret < 0) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "Repeater Auth Failed. Err: %d\n", ret); return ret; } @@ -1818,17 +1830,17 @@ static int hdcp2_authenticate_sink(struct intel_connector *connector) static int hdcp2_enable_stream_encryption(struct intel_connector *connector) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct hdcp_port_data *data = &dig_port->hdcp_port_data; struct intel_hdcp *hdcp = &connector->hdcp; enum transcoder cpu_transcoder = hdcp->cpu_transcoder; enum port port = dig_port->base.port; int ret = 0; - if (!(intel_de_read(i915, HDCP2_STATUS(i915, cpu_transcoder, port)) & + if (!(intel_de_read(display, HDCP2_STATUS(display, cpu_transcoder, port)) & LINK_ENCRYPTION_STATUS)) { - drm_err(&i915->drm, "[CONNECTOR:%d:%s] HDCP 2.2 Link is not encrypted\n", + drm_err(display->drm, "[CONNECTOR:%d:%s] HDCP 2.2 Link is not encrypted\n", connector->base.base.id, connector->base.name); ret = -EPERM; goto link_recover; @@ -1837,11 +1849,11 @@ static int hdcp2_enable_stream_encryption(struct intel_connector *connector) if (hdcp->shim->stream_2_2_encryption) { ret = hdcp->shim->stream_2_2_encryption(connector, true); if (ret) { - drm_err(&i915->drm, "[CONNECTOR:%d:%s] Failed to enable HDCP 2.2 stream enc\n", + drm_err(display->drm, "[CONNECTOR:%d:%s] Failed to enable HDCP 2.2 stream enc\n", connector->base.base.id, connector->base.name); return ret; } - drm_dbg_kms(&i915->drm, "HDCP 2.2 transcoder: %s stream encrypted\n", + drm_dbg_kms(display->drm, "HDCP 2.2 transcoder: %s stream encrypted\n", transcoder_name(hdcp->stream_transcoder)); } @@ -1849,7 +1861,7 @@ static int hdcp2_enable_stream_encryption(struct intel_connector *connector) link_recover: if (hdcp2_deauthenticate_port(connector) < 0) - drm_dbg_kms(&i915->drm, "Port deauth failed.\n"); + drm_dbg_kms(display->drm, "Port deauth failed.\n"); dig_port->hdcp_auth_status = false; data->k = 0; @@ -1859,35 +1871,35 @@ static int hdcp2_enable_stream_encryption(struct intel_connector *connector) static int hdcp2_enable_encryption(struct intel_connector *connector) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct intel_hdcp *hdcp = &connector->hdcp; enum port port = dig_port->base.port; enum transcoder cpu_transcoder = hdcp->cpu_transcoder; int ret; - drm_WARN_ON(&i915->drm, - intel_de_read(i915, HDCP2_STATUS(i915, cpu_transcoder, port)) & + drm_WARN_ON(display->drm, + intel_de_read(display, HDCP2_STATUS(display, cpu_transcoder, port)) & LINK_ENCRYPTION_STATUS); if (hdcp->shim->toggle_signalling) { ret = hdcp->shim->toggle_signalling(dig_port, cpu_transcoder, true); if (ret) { - drm_err(&i915->drm, + drm_err(display->drm, "Failed to enable HDCP signalling. %d\n", ret); return ret; } } - if (intel_de_read(i915, HDCP2_STATUS(i915, cpu_transcoder, port)) & + if (intel_de_read(display, HDCP2_STATUS(display, cpu_transcoder, port)) & LINK_AUTH_STATUS) /* Link is Authenticated. Now set for Encryption */ - intel_de_rmw(i915, HDCP2_CTL(i915, cpu_transcoder, port), + intel_de_rmw(display, HDCP2_CTL(display, cpu_transcoder, port), 0, CTL_LINK_ENCRYPTION_REQ); - ret = intel_de_wait_for_set(i915, - HDCP2_STATUS(i915, cpu_transcoder, + ret = intel_de_wait_for_set(display, + HDCP2_STATUS(display, cpu_transcoder, port), LINK_ENCRYPTION_STATUS, HDCP_ENCRYPT_STATUS_CHANGE_TIMEOUT_MS); @@ -1898,32 +1910,33 @@ static int hdcp2_enable_encryption(struct intel_connector *connector) static int hdcp2_disable_encryption(struct intel_connector *connector) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct intel_hdcp *hdcp = &connector->hdcp; enum port port = dig_port->base.port; enum transcoder cpu_transcoder = hdcp->cpu_transcoder; int ret; - drm_WARN_ON(&i915->drm, !(intel_de_read(i915, HDCP2_STATUS(i915, cpu_transcoder, port)) & - LINK_ENCRYPTION_STATUS)); + drm_WARN_ON(display->drm, + !(intel_de_read(display, HDCP2_STATUS(display, cpu_transcoder, port)) & + LINK_ENCRYPTION_STATUS)); - intel_de_rmw(i915, HDCP2_CTL(i915, cpu_transcoder, port), + intel_de_rmw(display, HDCP2_CTL(display, cpu_transcoder, port), CTL_LINK_ENCRYPTION_REQ, 0); - ret = intel_de_wait_for_clear(i915, - HDCP2_STATUS(i915, cpu_transcoder, + ret = intel_de_wait_for_clear(display, + HDCP2_STATUS(display, cpu_transcoder, port), LINK_ENCRYPTION_STATUS, HDCP_ENCRYPT_STATUS_CHANGE_TIMEOUT_MS); if (ret == -ETIMEDOUT) - drm_dbg_kms(&i915->drm, "Disable Encryption Timedout"); + drm_dbg_kms(display->drm, "Disable Encryption Timedout"); if (hdcp->shim->toggle_signalling) { ret = hdcp->shim->toggle_signalling(dig_port, cpu_transcoder, false); if (ret) { - drm_err(&i915->drm, + drm_err(display->drm, "Failed to disable HDCP signalling. %d\n", ret); return ret; @@ -1936,7 +1949,7 @@ static int hdcp2_disable_encryption(struct intel_connector *connector) static int hdcp2_propagate_stream_management_info(struct intel_connector *connector) { - struct drm_i915_private *i915 = to_i915(connector->base.dev); + struct intel_display *display = to_intel_display(connector); int i, tries = 3, ret; if (!connector->hdcp.is_repeater) @@ -1949,12 +1962,12 @@ hdcp2_propagate_stream_management_info(struct intel_connector *connector) /* Lets restart the auth incase of seq_num_m roll over */ if (connector->hdcp.seq_num_m > HDCP_2_2_SEQ_NUM_MAX) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "seq_num_m roll over.(%d)\n", ret); break; } - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "HDCP2 stream management %d of %d Failed.(%d)\n", i + 1, tries, ret); } @@ -1965,8 +1978,8 @@ hdcp2_propagate_stream_management_info(struct intel_connector *connector) static int hdcp2_authenticate_and_encrypt(struct intel_atomic_state *state, struct intel_connector *connector) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); - struct drm_i915_private *i915 = to_i915(connector->base.dev); int ret = 0, i, tries = 3; for (i = 0; i < tries && !dig_port->hdcp_auth_status; i++) { @@ -1974,7 +1987,7 @@ static int hdcp2_authenticate_and_encrypt(struct intel_atomic_state *state, if (!ret) { ret = intel_hdcp_prepare_streams(state, connector); if (ret) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "Prepare stream failed.(%d)\n", ret); break; @@ -1982,7 +1995,7 @@ static int hdcp2_authenticate_and_encrypt(struct intel_atomic_state *state, ret = hdcp2_propagate_stream_management_info(connector); if (ret) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "Stream management failed.(%d)\n", ret); break; @@ -1991,15 +2004,15 @@ static int hdcp2_authenticate_and_encrypt(struct intel_atomic_state *state, ret = hdcp2_authenticate_port(connector); if (!ret) break; - drm_dbg_kms(&i915->drm, "HDCP2 port auth failed.(%d)\n", + drm_dbg_kms(display->drm, "HDCP2 port auth failed.(%d)\n", ret); } /* Clearing the mei hdcp session */ - drm_dbg_kms(&i915->drm, "HDCP2.2 Auth %d of %d Failed.(%d)\n", + drm_dbg_kms(display->drm, "HDCP2.2 Auth %d of %d Failed.(%d)\n", i + 1, tries, ret); if (hdcp2_deauthenticate_port(connector) < 0) - drm_dbg_kms(&i915->drm, "Port deauth failed.\n"); + drm_dbg_kms(display->drm, "Port deauth failed.\n"); } if (!ret && !dig_port->hdcp_auth_status) { @@ -2010,10 +2023,10 @@ static int hdcp2_authenticate_and_encrypt(struct intel_atomic_state *state, msleep(HDCP_2_2_DELAY_BEFORE_ENCRYPTION_EN); ret = hdcp2_enable_encryption(connector); if (ret < 0) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "Encryption Enable Failed.(%d)\n", ret); if (hdcp2_deauthenticate_port(connector) < 0) - drm_dbg_kms(&i915->drm, "Port deauth failed.\n"); + drm_dbg_kms(display->drm, "Port deauth failed.\n"); } } @@ -2026,11 +2039,11 @@ static int hdcp2_authenticate_and_encrypt(struct intel_atomic_state *state, static int _intel_hdcp2_enable(struct intel_atomic_state *state, struct intel_connector *connector) { - struct drm_i915_private *i915 = to_i915(connector->base.dev); + struct intel_display *display = to_intel_display(connector); struct intel_hdcp *hdcp = &connector->hdcp; int ret; - drm_dbg_kms(&i915->drm, "[CONNECTOR:%d:%s] HDCP2.2 is being enabled. Type: %d\n", + drm_dbg_kms(display->drm, "[CONNECTOR:%d:%s] HDCP2.2 is being enabled. Type: %d\n", connector->base.base.id, connector->base.name, hdcp->content_type); @@ -2038,12 +2051,12 @@ static int _intel_hdcp2_enable(struct intel_atomic_state *state, ret = hdcp2_authenticate_and_encrypt(state, connector); if (ret) { - drm_dbg_kms(&i915->drm, "HDCP2 Type%d Enabling Failed. (%d)\n", + drm_dbg_kms(display->drm, "HDCP2 Type%d Enabling Failed. (%d)\n", hdcp->content_type, ret); return ret; } - drm_dbg_kms(&i915->drm, "[CONNECTOR:%d:%s] HDCP2.2 is enabled. Type %d\n", + drm_dbg_kms(display->drm, "[CONNECTOR:%d:%s] HDCP2.2 is enabled. Type %d\n", connector->base.base.id, connector->base.name, hdcp->content_type); @@ -2054,23 +2067,23 @@ static int _intel_hdcp2_enable(struct intel_atomic_state *state, static int _intel_hdcp2_disable(struct intel_connector *connector, bool hdcp2_link_recovery) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct hdcp_port_data *data = &dig_port->hdcp_port_data; struct intel_hdcp *hdcp = &connector->hdcp; int ret; - drm_dbg_kms(&i915->drm, "[CONNECTOR:%d:%s] HDCP2.2 is being Disabled\n", + drm_dbg_kms(display->drm, "[CONNECTOR:%d:%s] HDCP2.2 is being Disabled\n", connector->base.base.id, connector->base.name); if (hdcp->shim->stream_2_2_encryption) { ret = hdcp->shim->stream_2_2_encryption(connector, false); if (ret) { - drm_err(&i915->drm, "[CONNECTOR:%d:%s] Failed to disable HDCP 2.2 stream enc\n", + drm_err(display->drm, "[CONNECTOR:%d:%s] Failed to disable HDCP 2.2 stream enc\n", connector->base.base.id, connector->base.name); return ret; } - drm_dbg_kms(&i915->drm, "HDCP 2.2 transcoder: %s stream encryption disabled\n", + drm_dbg_kms(display->drm, "HDCP 2.2 transcoder: %s stream encryption disabled\n", transcoder_name(hdcp->stream_transcoder)); if (dig_port->num_hdcp_streams > 0 && !hdcp2_link_recovery) @@ -2080,7 +2093,7 @@ _intel_hdcp2_disable(struct intel_connector *connector, bool hdcp2_link_recovery ret = hdcp2_disable_encryption(connector); if (hdcp2_deauthenticate_port(connector) < 0) - drm_dbg_kms(&i915->drm, "Port deauth failed.\n"); + drm_dbg_kms(display->drm, "Port deauth failed.\n"); connector->hdcp.hdcp2_encrypted = false; dig_port->hdcp_auth_status = false; @@ -2092,8 +2105,8 @@ _intel_hdcp2_disable(struct intel_connector *connector, bool hdcp2_link_recovery /* Implements the Link Integrity Check for HDCP2.2 */ static int intel_hdcp2_check_link(struct intel_connector *connector) { + struct intel_display *display = to_intel_display(connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); - struct drm_i915_private *i915 = to_i915(connector->base.dev); struct intel_hdcp *hdcp = &connector->hdcp; enum port port = dig_port->base.port; enum transcoder cpu_transcoder; @@ -2110,11 +2123,11 @@ static int intel_hdcp2_check_link(struct intel_connector *connector) goto out; } - if (drm_WARN_ON(&i915->drm, - !intel_hdcp2_in_use(i915, cpu_transcoder, port))) { - drm_err(&i915->drm, + if (drm_WARN_ON(display->drm, + !intel_hdcp2_in_use(display, cpu_transcoder, port))) { + drm_err(display->drm, "HDCP2.2 link stopped the encryption, %x\n", - intel_de_read(i915, HDCP2_STATUS(i915, cpu_transcoder, port))); + intel_de_read(display, HDCP2_STATUS(display, cpu_transcoder, port))); ret = -ENXIO; _intel_hdcp2_disable(connector, true); intel_hdcp_update_value(connector, @@ -2137,17 +2150,17 @@ static int intel_hdcp2_check_link(struct intel_connector *connector) if (hdcp->value == DRM_MODE_CONTENT_PROTECTION_UNDESIRED) goto out; - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "HDCP2.2 Downstream topology change\n"); } else { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "[CONNECTOR:%d:%s] HDCP2.2 link failed, retrying auth\n", connector->base.base.id, connector->base.name); } ret = _intel_hdcp2_disable(connector, true); if (ret) { - drm_err(&i915->drm, + drm_err(display->drm, "[CONNECTOR:%d:%s] Failed to disable hdcp2.2 (%d)\n", connector->base.base.id, connector->base.name, ret); intel_hdcp_update_value(connector, @@ -2169,7 +2182,8 @@ static void intel_hdcp_check_work(struct work_struct *work) struct intel_hdcp, check_work); struct intel_connector *connector = intel_hdcp_to_connector(hdcp); - struct drm_i915_private *i915 = to_i915(connector->base.dev); + struct intel_display *display = to_intel_display(connector); + struct drm_i915_private *i915 = to_i915(display->drm); if (drm_connector_is_unregistered(&connector->base)) return; @@ -2186,13 +2200,12 @@ static int i915_hdcp_component_bind(struct device *drv_kdev, struct device *mei_kdev, void *data) { struct intel_display *display = to_intel_display(drv_kdev); - struct drm_i915_private *i915 = to_i915(display->drm); - drm_dbg(&i915->drm, "I915 HDCP comp bind\n"); - mutex_lock(&i915->display.hdcp.hdcp_mutex); - i915->display.hdcp.arbiter = (struct i915_hdcp_arbiter *)data; - i915->display.hdcp.arbiter->hdcp_dev = mei_kdev; - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + drm_dbg(display->drm, "I915 HDCP comp bind\n"); + mutex_lock(&display->hdcp.hdcp_mutex); + display->hdcp.arbiter = (struct i915_hdcp_arbiter *)data; + display->hdcp.arbiter->hdcp_dev = mei_kdev; + mutex_unlock(&display->hdcp.hdcp_mutex); return 0; } @@ -2201,12 +2214,11 @@ static void i915_hdcp_component_unbind(struct device *drv_kdev, struct device *mei_kdev, void *data) { struct intel_display *display = to_intel_display(drv_kdev); - struct drm_i915_private *i915 = to_i915(display->drm); - drm_dbg(&i915->drm, "I915 HDCP comp unbind\n"); - mutex_lock(&i915->display.hdcp.hdcp_mutex); - i915->display.hdcp.arbiter = NULL; - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + drm_dbg(display->drm, "I915 HDCP comp unbind\n"); + mutex_lock(&display->hdcp.hdcp_mutex); + display->hdcp.arbiter = NULL; + mutex_unlock(&display->hdcp.hdcp_mutex); } static const struct component_ops i915_hdcp_ops = { @@ -2240,11 +2252,11 @@ static int initialize_hdcp_port_data(struct intel_connector *connector, struct intel_digital_port *dig_port, const struct intel_hdcp_shim *shim) { - struct drm_i915_private *i915 = to_i915(connector->base.dev); + struct intel_display *display = to_intel_display(connector); struct hdcp_port_data *data = &dig_port->hdcp_port_data; enum port port = dig_port->base.port; - if (DISPLAY_VER(i915) < 12) + if (DISPLAY_VER(display) < 12) data->hdcp_ddi = intel_get_hdcp_ddi_index(port); else /* @@ -2264,11 +2276,11 @@ static int initialize_hdcp_port_data(struct intel_connector *connector, data->protocol = (u8)shim->protocol; if (!data->streams) - data->streams = kcalloc(INTEL_NUM_PIPES(i915), + data->streams = kcalloc(INTEL_NUM_PIPES(display), sizeof(struct hdcp2_streamid_type), GFP_KERNEL); if (!data->streams) { - drm_err(&i915->drm, "Out of Memory\n"); + drm_err(display->drm, "Out of Memory\n"); return -ENOMEM; } @@ -2277,13 +2289,15 @@ static int initialize_hdcp_port_data(struct intel_connector *connector, static bool is_hdcp2_supported(struct drm_i915_private *i915) { - if (intel_hdcp_gsc_cs_required(i915)) + struct intel_display *display = to_intel_display(&i915->drm); + + if (intel_hdcp_gsc_cs_required(display)) return true; if (!IS_ENABLED(CONFIG_INTEL_MEI_HDCP)) return false; - return (DISPLAY_VER(i915) >= 10 || + return (DISPLAY_VER(display) >= 10 || IS_KABYLAKE(i915) || IS_COFFEELAKE(i915) || IS_COMETLAKE(i915)); @@ -2291,28 +2305,29 @@ static bool is_hdcp2_supported(struct drm_i915_private *i915) void intel_hdcp_component_init(struct drm_i915_private *i915) { + struct intel_display *display = to_intel_display(&i915->drm); int ret; if (!is_hdcp2_supported(i915)) return; - mutex_lock(&i915->display.hdcp.hdcp_mutex); - drm_WARN_ON(&i915->drm, i915->display.hdcp.comp_added); + mutex_lock(&display->hdcp.hdcp_mutex); + drm_WARN_ON(display->drm, display->hdcp.comp_added); - i915->display.hdcp.comp_added = true; - mutex_unlock(&i915->display.hdcp.hdcp_mutex); - if (intel_hdcp_gsc_cs_required(i915)) + display->hdcp.comp_added = true; + mutex_unlock(&display->hdcp.hdcp_mutex); + if (intel_hdcp_gsc_cs_required(display)) ret = intel_hdcp_gsc_init(i915); else - ret = component_add_typed(i915->drm.dev, &i915_hdcp_ops, + ret = component_add_typed(display->drm->dev, &i915_hdcp_ops, I915_COMPONENT_HDCP); if (ret < 0) { - drm_dbg_kms(&i915->drm, "Failed at fw component add(%d)\n", + drm_dbg_kms(display->drm, "Failed at fw component add(%d)\n", ret); - mutex_lock(&i915->display.hdcp.hdcp_mutex); - i915->display.hdcp.comp_added = false; - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + mutex_lock(&display->hdcp.hdcp_mutex); + display->hdcp.comp_added = false; + mutex_unlock(&display->hdcp.hdcp_mutex); return; } } @@ -2321,13 +2336,13 @@ static void intel_hdcp2_init(struct intel_connector *connector, struct intel_digital_port *dig_port, const struct intel_hdcp_shim *shim) { - struct drm_i915_private *i915 = to_i915(connector->base.dev); + struct intel_display *display = to_intel_display(connector); struct intel_hdcp *hdcp = &connector->hdcp; int ret; ret = initialize_hdcp_port_data(connector, dig_port, shim); if (ret) { - drm_dbg_kms(&i915->drm, "Mei hdcp data init failed\n"); + drm_dbg_kms(display->drm, "Mei hdcp data init failed\n"); return; } @@ -2371,7 +2386,8 @@ static int _intel_hdcp_enable(struct intel_atomic_state *state, const struct intel_crtc_state *pipe_config, const struct drm_connector_state *conn_state) { - struct drm_i915_private *i915 = to_i915(encoder->base.dev); + struct intel_display *display = to_intel_display(encoder); + struct drm_i915_private *i915 = to_i915(display->drm); struct intel_connector *connector = to_intel_connector(conn_state->connector); struct intel_digital_port *dig_port = intel_attached_dig_port(connector); @@ -2383,14 +2399,14 @@ static int _intel_hdcp_enable(struct intel_atomic_state *state, return -ENOENT; if (!connector->encoder) { - drm_err(&i915->drm, "[CONNECTOR:%d:%s] encoder is not initialized\n", + drm_err(display->drm, "[CONNECTOR:%d:%s] encoder is not initialized\n", connector->base.base.id, connector->base.name); return -ENODEV; } mutex_lock(&hdcp->mutex); mutex_lock(&dig_port->hdcp_mutex); - drm_WARN_ON(&i915->drm, + drm_WARN_ON(display->drm, hdcp->value == DRM_MODE_CONTENT_PROTECTION_ENABLED); hdcp->content_type = (u8)conn_state->hdcp_content_type; @@ -2555,19 +2571,21 @@ void intel_hdcp_update_pipe(struct intel_atomic_state *state, void intel_hdcp_component_fini(struct drm_i915_private *i915) { - mutex_lock(&i915->display.hdcp.hdcp_mutex); - if (!i915->display.hdcp.comp_added) { - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + struct intel_display *display = to_intel_display(&i915->drm); + + mutex_lock(&display->hdcp.hdcp_mutex); + if (!display->hdcp.comp_added) { + mutex_unlock(&display->hdcp.hdcp_mutex); return; } - i915->display.hdcp.comp_added = false; - mutex_unlock(&i915->display.hdcp.hdcp_mutex); + display->hdcp.comp_added = false; + mutex_unlock(&display->hdcp.hdcp_mutex); - if (intel_hdcp_gsc_cs_required(i915)) + if (intel_hdcp_gsc_cs_required(display)) intel_hdcp_gsc_fini(i915); else - component_del(i915->drm.dev, &i915_hdcp_ops); + component_del(display->drm->dev, &i915_hdcp_ops); } void intel_hdcp_cleanup(struct intel_connector *connector) @@ -2657,7 +2675,8 @@ void intel_hdcp_atomic_check(struct drm_connector *connector, void intel_hdcp_handle_cp_irq(struct intel_connector *connector) { struct intel_hdcp *hdcp = &connector->hdcp; - struct drm_i915_private *i915 = to_i915(connector->base.dev); + struct intel_display *display = to_intel_display(connector); + struct drm_i915_private *i915 = to_i915(display->drm); if (!hdcp->shim) return; diff --git a/drivers/gpu/drm/i915/display/intel_hdcp_gsc.c b/drivers/gpu/drm/i915/display/intel_hdcp_gsc.c index 16afeb8a3a8d..dc5cc1d54c85 100644 --- a/drivers/gpu/drm/i915/display/intel_hdcp_gsc.c +++ b/drivers/gpu/drm/i915/display/intel_hdcp_gsc.c @@ -19,18 +19,19 @@ struct intel_hdcp_gsc_message { void *hdcp_cmd_out; }; -bool intel_hdcp_gsc_cs_required(struct drm_i915_private *i915) +bool intel_hdcp_gsc_cs_required(struct intel_display *display) { - return DISPLAY_VER(i915) >= 14; + return DISPLAY_VER(display) >= 14; } -bool intel_hdcp_gsc_check_status(struct drm_i915_private *i915) +bool intel_hdcp_gsc_check_status(struct intel_display *display) { + struct drm_i915_private *i915 = to_i915(display->drm); struct intel_gt *gt = i915->media_gt; struct intel_gsc_uc *gsc = gt ? >->uc.gsc : NULL; if (!gsc || !intel_uc_fw_is_running(&gsc->fw)) { - drm_dbg_kms(&i915->drm, + drm_dbg_kms(display->drm, "GSC components required for HDCP2.2 are not ready\n"); return false; } diff --git a/drivers/gpu/drm/i915/display/intel_hdcp_gsc.h b/drivers/gpu/drm/i915/display/intel_hdcp_gsc.h index 5f610df61cc9..b6aabd855478 100644 --- a/drivers/gpu/drm/i915/display/intel_hdcp_gsc.h +++ b/drivers/gpu/drm/i915/display/intel_hdcp_gsc.h @@ -10,14 +10,15 @@ #include struct drm_i915_private; +struct intel_display; struct intel_hdcp_gsc_message; -bool intel_hdcp_gsc_cs_required(struct drm_i915_private *i915); +bool intel_hdcp_gsc_cs_required(struct intel_display *display); ssize_t intel_hdcp_gsc_msg_send(struct drm_i915_private *i915, u8 *msg_in, size_t msg_in_len, u8 *msg_out, size_t msg_out_len); int intel_hdcp_gsc_init(struct drm_i915_private *i915); void intel_hdcp_gsc_fini(struct drm_i915_private *i915); -bool intel_hdcp_gsc_check_status(struct drm_i915_private *i915); +bool intel_hdcp_gsc_check_status(struct intel_display *display); #endif /* __INTEL_HDCP_GCS_H__ */ diff --git a/drivers/gpu/drm/i915/display/intel_hdcp_gsc_message.h b/drivers/gpu/drm/i915/display/intel_hdcp_gsc_message.h index ce199d6f6232..2d597f27e931 100644 --- a/drivers/gpu/drm/i915/display/intel_hdcp_gsc_message.h +++ b/drivers/gpu/drm/i915/display/intel_hdcp_gsc_message.h @@ -22,11 +22,12 @@ struct hdcp2_ske_send_eks; struct hdcp2_rep_send_receiverid_list; struct hdcp2_rep_send_ack; struct hdcp2_rep_stream_ready; +struct intel_display; ssize_t intel_hdcp_gsc_msg_send(struct drm_i915_private *i915, u8 *msg_in, size_t msg_in_len, u8 *msg_out, size_t msg_out_len); -bool intel_hdcp_gsc_check_status(struct drm_i915_private *i915); +bool intel_hdcp_gsc_check_status(struct intel_display *display); int intel_hdcp_gsc_initiate_session(struct device *dev, struct hdcp_port_data *data, struct hdcp2_ake_init *ake_data); diff --git a/drivers/gpu/drm/i915/display/intel_vrr.c b/drivers/gpu/drm/i915/display/intel_vrr.c index 9a51f5bac307..96de5e82af38 100644 --- a/drivers/gpu/drm/i915/display/intel_vrr.c +++ b/drivers/gpu/drm/i915/display/intel_vrr.c @@ -21,6 +21,9 @@ bool intel_vrr_is_capable(struct intel_connector *connector) const struct drm_display_info *info = &connector->base.display_info; struct intel_dp *intel_dp; + if (!HAS_VRR(display)) + return false; + /* * DP Sink is capable of VRR video timings if * Ignore MSA bit is set in DPCD. @@ -43,8 +46,11 @@ bool intel_vrr_is_capable(struct intel_connector *connector) return false; } - return HAS_VRR(display) && - info->monitor_range.max_vfreq - info->monitor_range.min_vfreq > 10; + if (!info->monitor_range.min_vfreq || !info->monitor_range.max_vfreq || + info->monitor_range.min_vfreq > info->monitor_range.max_vfreq) + return false; + + return info->monitor_range.max_vfreq - info->monitor_range.min_vfreq > 10; } bool intel_vrr_is_in_range(struct intel_connector *connector, int vrefresh) diff --git a/drivers/gpu/drm/imagination/pvr_job.c b/drivers/gpu/drm/imagination/pvr_job.c index 6a15c1d2d871..ab2f43b99b4b 100644 --- a/drivers/gpu/drm/imagination/pvr_job.c +++ b/drivers/gpu/drm/imagination/pvr_job.c @@ -453,7 +453,7 @@ create_job(struct pvr_device *pvr_dev, if (err) goto err_put_job; - err = pvr_queue_job_init(job); + err = pvr_queue_job_init(job, pvr_file->file->client_id); if (err) goto err_put_job; diff --git a/drivers/gpu/drm/imagination/pvr_queue.c b/drivers/gpu/drm/imagination/pvr_queue.c index 083cb6181985..367d5915130a 100644 --- a/drivers/gpu/drm/imagination/pvr_queue.c +++ b/drivers/gpu/drm/imagination/pvr_queue.c @@ -1088,6 +1088,7 @@ static int pvr_queue_cleanup_fw_context(struct pvr_queue *queue) /** * pvr_queue_job_init() - Initialize queue related fields in a pvr_job object. * @job: The job to initialize. + * @drm_client_id: drm_file.client_id submitting the job * * Bind the job to a queue and allocate memory to guarantee pvr_queue_job_arm() * and pvr_queue_job_push() can't fail. We also make sure the context type is @@ -1097,7 +1098,7 @@ static int pvr_queue_cleanup_fw_context(struct pvr_queue *queue) * * 0 on success, or * * An error code if something failed. */ -int pvr_queue_job_init(struct pvr_job *job) +int pvr_queue_job_init(struct pvr_job *job, u64 drm_client_id) { /* Fragment jobs need at least one native fence wait on the geometry job fence. */ u32 min_native_dep_count = job->type == DRM_PVR_JOB_TYPE_FRAGMENT ? 1 : 0; @@ -1114,7 +1115,7 @@ int pvr_queue_job_init(struct pvr_job *job) if (!pvr_cccb_cmdseq_can_fit(&queue->cccb, job_cmds_size(job, min_native_dep_count))) return -E2BIG; - err = drm_sched_job_init(&job->base, &queue->entity, 1, THIS_MODULE); + err = drm_sched_job_init(&job->base, &queue->entity, 1, THIS_MODULE, drm_client_id); if (err) return err; diff --git a/drivers/gpu/drm/imagination/pvr_queue.h b/drivers/gpu/drm/imagination/pvr_queue.h index 887f60b5daa6..a205e29437f4 100644 --- a/drivers/gpu/drm/imagination/pvr_queue.h +++ b/drivers/gpu/drm/imagination/pvr_queue.h @@ -143,7 +143,7 @@ struct pvr_queue { bool pvr_queue_fence_is_ufo_backed(struct dma_fence *f); -int pvr_queue_job_init(struct pvr_job *job); +int pvr_queue_job_init(struct pvr_job *job, u64 drm_client_id); void pvr_queue_job_cleanup(struct pvr_job *job); diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c index 9bb997dbb4b9..f46f961afc56 100644 --- a/drivers/gpu/drm/lima/lima_gem.c +++ b/drivers/gpu/drm/lima/lima_gem.c @@ -341,7 +341,7 @@ int lima_gem_submit(struct drm_file *file, struct lima_submit *submit) err = lima_sched_task_init( submit->task, submit->ctx->context + submit->pipe, - bos, submit->nr_bos, vm); + bos, submit->nr_bos, vm, file->client_id); if (err) goto err_out1; diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c index 1a944edb6ddc..00f0b2907cb6 100644 --- a/drivers/gpu/drm/lima/lima_sched.c +++ b/drivers/gpu/drm/lima/lima_sched.c @@ -113,7 +113,8 @@ static inline struct lima_sched_pipe *to_lima_pipe(struct drm_gpu_scheduler *sch int lima_sched_task_init(struct lima_sched_task *task, struct lima_sched_context *context, struct lima_bo **bos, int num_bos, - struct lima_vm *vm) + struct lima_vm *vm, + u64 drm_client_id) { int err, i; @@ -124,7 +125,8 @@ int lima_sched_task_init(struct lima_sched_task *task, for (i = 0; i < num_bos; i++) drm_gem_object_get(&bos[i]->base.base); - err = drm_sched_job_init(&task->base, &context->base, 1, vm); + err = drm_sched_job_init(&task->base, &context->base, 1, vm, + drm_client_id); if (err) { kfree(task->bos); return err; diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h index 85b23ba901d5..1a08faf8a529 100644 --- a/drivers/gpu/drm/lima/lima_sched.h +++ b/drivers/gpu/drm/lima/lima_sched.h @@ -88,7 +88,8 @@ struct lima_sched_pipe { int lima_sched_task_init(struct lima_sched_task *task, struct lima_sched_context *context, struct lima_bo **bos, int num_bos, - struct lima_vm *vm); + struct lima_vm *vm, + u64 drm_client_id); void lima_sched_task_fini(struct lima_sched_task *task); int lima_sched_context_init(struct lima_sched_pipe *pipe, diff --git a/drivers/gpu/drm/mediatek/mtk_crtc.c b/drivers/gpu/drm/mediatek/mtk_crtc.c index c4c6d0249df5..aa8e6f6ddcd3 100644 --- a/drivers/gpu/drm/mediatek/mtk_crtc.c +++ b/drivers/gpu/drm/mediatek/mtk_crtc.c @@ -153,10 +153,10 @@ static void mtk_crtc_reset(struct drm_crtc *crtc) { struct mtk_crtc_state *state; - if (crtc->state) + if (crtc->state) { __drm_atomic_helper_crtc_destroy_state(crtc->state); - - kfree(to_mtk_crtc_state(crtc->state)); + kfree(to_mtk_crtc_state(crtc->state)); + } crtc->state = NULL; state = kzalloc(sizeof(*state), GFP_KERNEL); diff --git a/drivers/gpu/drm/mediatek/mtk_disp_ovl_adaptor.c b/drivers/gpu/drm/mediatek/mtk_disp_ovl_adaptor.c index bf2546c4681a..8f11d1f42c4c 100644 --- a/drivers/gpu/drm/mediatek/mtk_disp_ovl_adaptor.c +++ b/drivers/gpu/drm/mediatek/mtk_disp_ovl_adaptor.c @@ -585,6 +585,7 @@ static void mtk_disp_ovl_adaptor_master_unbind(struct device *dev) struct mtk_disp_ovl_adaptor *priv = dev_get_drvdata(dev); priv->children_bound = false; + component_unbind_all(dev, priv->mmsys_dev); } static const struct component_master_ops mtk_disp_ovl_adaptor_master_ops = { @@ -611,12 +612,15 @@ static int mtk_disp_ovl_adaptor_probe(struct platform_device *pdev) priv->mmsys_dev = pdev->dev.platform_data; - component_master_add_with_match(dev, &mtk_disp_ovl_adaptor_master_ops, match); + ret = component_master_add_with_match(dev, &mtk_disp_ovl_adaptor_master_ops, match); + if (ret) + return dev_err_probe(dev, ret, "Failed to add component master\n"); pm_runtime_enable(dev); ret = component_add(dev, &mtk_disp_ovl_adaptor_comp_ops); if (ret != 0) { + component_master_del(dev, &mtk_disp_ovl_adaptor_master_ops); pm_runtime_disable(dev); return dev_err_probe(dev, ret, "Failed to add component\n"); } @@ -626,6 +630,7 @@ static int mtk_disp_ovl_adaptor_probe(struct platform_device *pdev) static void mtk_disp_ovl_adaptor_remove(struct platform_device *pdev) { + component_del(&pdev->dev, &mtk_disp_ovl_adaptor_comp_ops); component_master_del(&pdev->dev, &mtk_disp_ovl_adaptor_master_ops); pm_runtime_disable(&pdev->dev); } diff --git a/drivers/gpu/drm/msm/msm_fbdev.c b/drivers/gpu/drm/msm/msm_fbdev.c index 030bedac632d..174a116fba2b 100644 --- a/drivers/gpu/drm/msm/msm_fbdev.c +++ b/drivers/gpu/drm/msm/msm_fbdev.c @@ -70,9 +70,9 @@ static int msm_fbdev_create(struct drm_fb_helper *helper, { struct drm_device *dev = helper->dev; struct msm_drm_private *priv = dev->dev_private; + struct fb_info *fbi = helper->info; struct drm_framebuffer *fb = NULL; struct drm_gem_object *bo; - struct fb_info *fbi = NULL; uint64_t paddr; uint32_t format; int ret, pitch; @@ -105,13 +105,6 @@ static int msm_fbdev_create(struct drm_fb_helper *helper, goto fail; } - fbi = drm_fb_helper_alloc_info(helper); - if (IS_ERR(fbi)) { - DRM_DEV_ERROR(dev->dev, "failed to allocate fb info\n"); - ret = PTR_ERR(fbi); - goto fail; - } - DBG("fbi=%p, dev=%p", fbi, dev); helper->fb = fb; diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index 3eee6517541e..c0b4a346eb1f 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -30,7 +30,7 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev, struct msm_gpu *gpu, struct msm_gpu_submitqueue *queue, uint32_t nr_bos, - uint32_t nr_cmds) + uint32_t nr_cmds, u64 drm_client_id) { static atomic_t ident = ATOMIC_INIT(0); struct msm_gem_submit *submit; @@ -54,7 +54,8 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev, return ERR_PTR(ret); } - ret = drm_sched_job_init(&submit->base, queue->entity, 1, queue); + ret = drm_sched_job_init(&submit->base, queue->entity, 1, queue, + drm_client_id); if (ret) { kfree(submit->hw_fence); kfree(submit); @@ -712,7 +713,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, } } - submit = submit_create(dev, gpu, queue, args->nr_bos, args->nr_cmds); + submit = submit_create(dev, gpu, queue, args->nr_bos, args->nr_cmds, + file->client_id); if (IS_ERR(submit)) { ret = PTR_ERR(submit); goto out_post_unlock; diff --git a/drivers/gpu/drm/nouveau/nouveau_sched.c b/drivers/gpu/drm/nouveau/nouveau_sched.c index 9e6244347ed7..390c2cc21c2f 100644 --- a/drivers/gpu/drm/nouveau/nouveau_sched.c +++ b/drivers/gpu/drm/nouveau/nouveau_sched.c @@ -87,7 +87,8 @@ nouveau_job_init(struct nouveau_job *job, } ret = drm_sched_job_init(&job->base, &sched->entity, - args->credits, NULL); + args->credits, NULL, + job->file_priv->client_id); if (ret) goto err_free_chains; diff --git a/drivers/gpu/drm/omapdrm/omap_fbdev.c b/drivers/gpu/drm/omapdrm/omap_fbdev.c index 523be34682ca..483dc30d7840 100644 --- a/drivers/gpu/drm/omapdrm/omap_fbdev.c +++ b/drivers/gpu/drm/omapdrm/omap_fbdev.c @@ -131,9 +131,9 @@ static int omap_fbdev_create(struct drm_fb_helper *helper, struct omap_fbdev *fbdev = to_omap_fbdev(helper); struct drm_device *dev = helper->dev; struct omap_drm_private *priv = dev->dev_private; + struct fb_info *fbi = helper->info; struct drm_framebuffer *fb = NULL; union omap_gem_size gsize; - struct fb_info *fbi = NULL; struct drm_mode_fb_cmd2 mode_cmd = {0}; struct drm_gem_object *bo; dma_addr_t dma_addr; @@ -199,13 +199,6 @@ static int omap_fbdev_create(struct drm_fb_helper *helper, goto fail; } - fbi = drm_fb_helper_alloc_info(helper); - if (IS_ERR(fbi)) { - dev_err(dev->dev, "failed to allocate fb info\n"); - ret = PTR_ERR(fbi); - goto fail; - } - DBG("fbi=%p, dev=%p", fbi, dev); helper->fb = fb; diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c index b6e5442f8728..c20a817dcd1f 100644 --- a/drivers/gpu/drm/panfrost/panfrost_drv.c +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c @@ -273,7 +273,7 @@ static int panfrost_ioctl_submit(struct drm_device *dev, void *data, ret = drm_sched_job_init(&job->base, &file_priv->sched_entity[slot], - 1, NULL); + 1, NULL, file->client_id); if (ret) goto out_put_job; diff --git a/drivers/gpu/drm/panthor/panthor_drv.c b/drivers/gpu/drm/panthor/panthor_drv.c index 9360d486e52e..9d4bf7406e24 100644 --- a/drivers/gpu/drm/panthor/panthor_drv.c +++ b/drivers/gpu/drm/panthor/panthor_drv.c @@ -908,7 +908,8 @@ static int panthor_ioctl_group_submit(struct drm_device *ddev, void *data, const struct drm_panthor_queue_submit *qsubmit = &jobs_args[i]; struct drm_sched_job *job; - job = panthor_job_create(pfile, args->group_handle, qsubmit); + job = panthor_job_create(pfile, args->group_handle, qsubmit, + file->client_id); if (IS_ERR(job)) { ret = PTR_ERR(job); goto out_cleanup_submit_ctx; diff --git a/drivers/gpu/drm/panthor/panthor_fw.c b/drivers/gpu/drm/panthor/panthor_fw.c index eb4aface8a4a..f6a7a095fb0e 100644 --- a/drivers/gpu/drm/panthor/panthor_fw.c +++ b/drivers/gpu/drm/panthor/panthor_fw.c @@ -491,6 +491,7 @@ static int panthor_fw_load_section_entry(struct panthor_device *ptdev, struct panthor_fw_binary_section_entry_hdr hdr; struct panthor_fw_section *section; u32 section_size; + u32 data_size; u32 name_len; int ret; @@ -541,6 +542,13 @@ static int panthor_fw_load_section_entry(struct panthor_device *ptdev, return -EINVAL; } + section_size = hdr.va.end - hdr.va.start; + data_size = hdr.data.end - hdr.data.start; + if (data_size > section_size) { + drm_err(&ptdev->base, "Firmware corrupted, section data exceeds section size\n"); + return -EINVAL; + } + name_len = iter->size - iter->offset; section = drmm_kzalloc(&ptdev->base, sizeof(*section), GFP_KERNEL); @@ -549,7 +557,7 @@ static int panthor_fw_load_section_entry(struct panthor_device *ptdev, list_add_tail(§ion->node, &ptdev->fw->sections); section->flags = hdr.flags; - section->data.size = hdr.data.end - hdr.data.start; + section->data.size = data_size; if (section->data.size > 0) { void *data = drmm_kmalloc(&ptdev->base, section->data.size, GFP_KERNEL); @@ -572,7 +580,6 @@ static int panthor_fw_load_section_entry(struct panthor_device *ptdev, section->name = name; } - section_size = hdr.va.end - hdr.va.start; if (section_size) { u32 cache_mode = hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_RD_CACHE_MODE_MASK; struct panthor_gem_object *bo; @@ -760,18 +767,24 @@ static int panthor_fw_load(struct panthor_device *ptdev) * iface_fw_to_cpu_addr() - Turn an MCU address into a CPU address * @ptdev: Device. * @mcu_va: MCU address. + * @size: Size of the object pointed to by @mcu_va. * - * Return: NULL if the address is not part of the shared section, non-NULL otherwise. + * Return: NULL if the object is not part of the shared section, non-NULL otherwise. */ -static void *iface_fw_to_cpu_addr(struct panthor_device *ptdev, u32 mcu_va) +static void *iface_fw_to_cpu_addr(struct panthor_device *ptdev, u32 mcu_va, size_t size) { u64 shared_mem_start = panthor_kernel_bo_gpuva(ptdev->fw->shared_section->mem); - u64 shared_mem_end = shared_mem_start + - panthor_kernel_bo_size(ptdev->fw->shared_section->mem); - if (mcu_va < shared_mem_start || mcu_va >= shared_mem_end) + size_t shared_mem_size = panthor_kernel_bo_size(ptdev->fw->shared_section->mem); + u64 offset; + + if (mcu_va < shared_mem_start) + return NULL; + + offset = mcu_va - shared_mem_start; + if (offset > shared_mem_size || size > shared_mem_size - offset) return NULL; - return ptdev->fw->shared_section->mem->kmap + (mcu_va - shared_mem_start); + return ptdev->fw->shared_section->mem->kmap + offset; } static int panthor_init_cs_iface(struct panthor_device *ptdev, @@ -793,8 +806,10 @@ static int panthor_init_cs_iface(struct panthor_device *ptdev, spin_lock_init(&cs_iface->lock); cs_iface->control = ptdev->fw->shared_section->mem->kmap + iface_offset; - cs_iface->input = iface_fw_to_cpu_addr(ptdev, cs_iface->control->input_va); - cs_iface->output = iface_fw_to_cpu_addr(ptdev, cs_iface->control->output_va); + cs_iface->input = iface_fw_to_cpu_addr(ptdev, cs_iface->control->input_va, + sizeof(*cs_iface->input)); + cs_iface->output = iface_fw_to_cpu_addr(ptdev, cs_iface->control->output_va, + sizeof(*cs_iface->output)); if (!cs_iface->input || !cs_iface->output) { drm_err(&ptdev->base, "Invalid stream control interface input/output VA"); @@ -844,8 +859,10 @@ static int panthor_init_csg_iface(struct panthor_device *ptdev, spin_lock_init(&csg_iface->lock); csg_iface->control = ptdev->fw->shared_section->mem->kmap + iface_offset; - csg_iface->input = iface_fw_to_cpu_addr(ptdev, csg_iface->control->input_va); - csg_iface->output = iface_fw_to_cpu_addr(ptdev, csg_iface->control->output_va); + csg_iface->input = iface_fw_to_cpu_addr(ptdev, csg_iface->control->input_va, + sizeof(*csg_iface->input)); + csg_iface->output = iface_fw_to_cpu_addr(ptdev, csg_iface->control->output_va, + sizeof(*csg_iface->output)); if (csg_iface->control->stream_num < MIN_CS_PER_CSG || csg_iface->control->stream_num > MAX_CS_PER_CSG) @@ -902,8 +919,10 @@ static int panthor_fw_init_ifaces(struct panthor_device *ptdev) return -EINVAL; } - glb_iface->input = iface_fw_to_cpu_addr(ptdev, glb_iface->control->input_va); - glb_iface->output = iface_fw_to_cpu_addr(ptdev, glb_iface->control->output_va); + glb_iface->input = iface_fw_to_cpu_addr(ptdev, glb_iface->control->input_va, + sizeof(*glb_iface->input)); + glb_iface->output = iface_fw_to_cpu_addr(ptdev, glb_iface->control->output_va, + sizeof(*glb_iface->output)); if (!glb_iface->input || !glb_iface->output) { drm_err(&ptdev->base, "Invalid global control interface input/output VA"); return -EINVAL; diff --git a/drivers/gpu/drm/panthor/panthor_mmu.c b/drivers/gpu/drm/panthor/panthor_mmu.c index e221708cf1aa..9b6afc9c1da1 100644 --- a/drivers/gpu/drm/panthor/panthor_mmu.c +++ b/drivers/gpu/drm/panthor/panthor_mmu.c @@ -2498,7 +2498,7 @@ panthor_vm_bind_job_create(struct drm_file *file, kref_init(&job->refcount); job->vm = panthor_vm_get(vm); - ret = drm_sched_job_init(&job->base, &vm->entity, 1, vm); + ret = drm_sched_job_init(&job->base, &vm->entity, 1, vm, file->client_id); if (ret) goto err_put_job; diff --git a/drivers/gpu/drm/panthor/panthor_sched.c b/drivers/gpu/drm/panthor/panthor_sched.c index cedcde0f10bd..2f80c47e1f54 100644 --- a/drivers/gpu/drm/panthor/panthor_sched.c +++ b/drivers/gpu/drm/panthor/panthor_sched.c @@ -3664,7 +3664,8 @@ struct panthor_vm *panthor_job_vm(struct drm_sched_job *sched_job) struct drm_sched_job * panthor_job_create(struct panthor_file *pfile, u16 group_handle, - const struct drm_panthor_queue_submit *qsubmit) + const struct drm_panthor_queue_submit *qsubmit, + u64 drm_client_id) { struct panthor_group_pool *gpool = pfile->groups; struct panthor_job *job; @@ -3736,7 +3737,7 @@ panthor_job_create(struct panthor_file *pfile, ret = drm_sched_job_init(&job->base, &job->group->queues[job->queue_idx]->entity, - credits, job->group); + credits, job->group, drm_client_id); if (ret) goto err_put_job; diff --git a/drivers/gpu/drm/panthor/panthor_sched.h b/drivers/gpu/drm/panthor/panthor_sched.h index 666d1655ee18..281b2f22a16d 100644 --- a/drivers/gpu/drm/panthor/panthor_sched.h +++ b/drivers/gpu/drm/panthor/panthor_sched.h @@ -28,7 +28,8 @@ int panthor_group_get_state(struct panthor_file *pfile, struct drm_sched_job * panthor_job_create(struct panthor_file *pfile, u16 group_handle, - const struct drm_panthor_queue_submit *qsubmit); + const struct drm_panthor_queue_submit *qsubmit, + u64 drm_client_id); struct drm_sched_job *panthor_job_get(struct drm_sched_job *job); struct panthor_vm *panthor_job_vm(struct drm_sched_job *sched_job); void panthor_job_put(struct drm_sched_job *job); diff --git a/drivers/gpu/drm/radeon/radeon_fbdev.c b/drivers/gpu/drm/radeon/radeon_fbdev.c index a197ba2f2717..0a05b373bd9e 100644 --- a/drivers/gpu/drm/radeon/radeon_fbdev.c +++ b/drivers/gpu/drm/radeon/radeon_fbdev.c @@ -207,7 +207,7 @@ static int radeon_fbdev_fb_helper_fb_probe(struct drm_fb_helper *fb_helper, { struct radeon_device *rdev = fb_helper->dev->dev_private; struct drm_mode_fb_cmd2 mode_cmd = { }; - struct fb_info *info; + struct fb_info *info = fb_helper->info; struct drm_gem_object *gobj; struct radeon_bo *rbo; struct drm_framebuffer *fb; @@ -245,13 +245,6 @@ static int radeon_fbdev_fb_helper_fb_probe(struct drm_fb_helper *fb_helper, /* setup helper */ fb_helper->fb = fb; - /* okay we have an object now allocate the framebuffer */ - info = drm_fb_helper_alloc_info(fb_helper); - if (IS_ERR(info)) { - ret = PTR_ERR(info); - goto err_drm_framebuffer_unregister_private; - } - info->fbops = &radeon_fbdev_fb_ops; /* radeon resume is fragile and needs a vt switch to help it along */ @@ -277,10 +270,6 @@ static int radeon_fbdev_fb_helper_fb_probe(struct drm_fb_helper *fb_helper, return 0; -err_drm_framebuffer_unregister_private: - fb_helper->fb = NULL; - drm_framebuffer_unregister_private(fb); - drm_framebuffer_cleanup(fb); err_kfree: kfree(fb); err_radeon_fbdev_destroy_pinned_object: diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c index 0f35f009b9d3..41c3d596603c 100644 --- a/drivers/gpu/drm/scheduler/sched_fence.c +++ b/drivers/gpu/drm/scheduler/sched_fence.c @@ -204,7 +204,8 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f) EXPORT_SYMBOL(to_drm_sched_fence); struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity, - void *owner) + void *owner, + u64 drm_client_id) { struct drm_sched_fence *fence = NULL; @@ -213,6 +214,7 @@ struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity, return NULL; fence->owner = owner; + fence->drm_client_id = drm_client_id; spin_lock_init(&fence->lock); return fence; diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 4f43c0fa4019..34ebc2fa3a5a 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -772,6 +772,8 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs); * @credits: the number of credits this job contributes to the schedulers * credit limit * @owner: job owner for debugging + * @drm_client_id: &struct drm_file.client_id of the owner (used by trace + * events) * * Refer to drm_sched_entity_push_job() documentation * for locking considerations. @@ -788,7 +790,8 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs); */ int drm_sched_job_init(struct drm_sched_job *job, struct drm_sched_entity *entity, - u32 credits, void *owner) + u32 credits, void *owner, + uint64_t drm_client_id) { if (!entity->rq) { /* This will most likely be followed by missing frames @@ -814,7 +817,7 @@ int drm_sched_job_init(struct drm_sched_job *job, job->entity = entity; job->credits = credits; - job->s_fence = drm_sched_fence_alloc(entity, owner); + job->s_fence = drm_sched_fence_alloc(entity, owner, drm_client_id); if (!job->s_fence) return -ENOMEM; diff --git a/drivers/gpu/drm/tegra/fbdev.c b/drivers/gpu/drm/tegra/fbdev.c index db6eaac3d30e..08cd471e5aa2 100644 --- a/drivers/gpu/drm/tegra/fbdev.c +++ b/drivers/gpu/drm/tegra/fbdev.c @@ -72,10 +72,9 @@ static int tegra_fbdev_probe(struct drm_fb_helper *helper, struct tegra_drm *tegra = helper->dev->dev_private; struct drm_device *drm = helper->dev; struct drm_mode_fb_cmd2 cmd = { 0 }; + struct fb_info *info = helper->info; unsigned int bytes_per_pixel; struct drm_framebuffer *fb; - unsigned long offset; - struct fb_info *info; struct tegra_bo *bo; size_t size; int err; @@ -96,13 +95,6 @@ static int tegra_fbdev_probe(struct drm_fb_helper *helper, if (IS_ERR(bo)) return PTR_ERR(bo); - info = drm_fb_helper_alloc_info(helper); - if (IS_ERR(info)) { - dev_err(drm->dev, "failed to allocate framebuffer info\n"); - drm_gem_object_put(&bo->gem); - return PTR_ERR(info); - } - fb = tegra_fb_alloc(drm, &cmd, &bo, 1); if (IS_ERR(fb)) { err = PTR_ERR(fb); @@ -113,15 +105,11 @@ static int tegra_fbdev_probe(struct drm_fb_helper *helper, } helper->fb = fb; - helper->info = info; info->fbops = &tegra_fb_ops; drm_fb_helper_fill_info(info, helper, sizes); - offset = info->var.xoffset * bytes_per_pixel + - info->var.yoffset * fb->pitches[0]; - if (bo->pages) { bo->vaddr = vmap(bo->pages, bo->num_pages, VM_MAP, pgprot_writecombine(PAGE_KERNEL)); @@ -133,9 +121,9 @@ static int tegra_fbdev_probe(struct drm_fb_helper *helper, } info->flags |= FBINFO_VIRTFB; - info->screen_buffer = bo->vaddr + offset; + info->screen_buffer = bo->vaddr; info->screen_size = size; - info->fix.smem_start = (unsigned long)(bo->iova + offset); + info->fix.smem_start = (unsigned long)(bo->iova); info->fix.smem_len = size; return 0; diff --git a/drivers/gpu/drm/v3d/v3d_submit.c b/drivers/gpu/drm/v3d/v3d_submit.c index 5ffa4c0f9e8a..41615f41ac0d 100644 --- a/drivers/gpu/drm/v3d/v3d_submit.c +++ b/drivers/gpu/drm/v3d/v3d_submit.c @@ -186,7 +186,7 @@ v3d_job_init(struct v3d_dev *v3d, struct drm_file *file_priv, job->file = file_priv; ret = drm_sched_job_init(&job->base, &v3d_priv->sched_entity[queue], - 1, v3d_priv); + 1, v3d_priv, file_priv->client_id); if (ret) return ret; diff --git a/drivers/gpu/drm/vc4/vc4_irq.c b/drivers/gpu/drm/vc4/vc4_irq.c index 968356d1b91d..c201a8aaccb4 100644 --- a/drivers/gpu/drm/vc4/vc4_irq.c +++ b/drivers/gpu/drm/vc4/vc4_irq.c @@ -104,7 +104,7 @@ vc4_overflow_mem_work(struct work_struct *work) vc4->bin_alloc_overflow = BIT(bin_bo_slot); V3D_WRITE(V3D_BPOA, bo->base.dma_addr + bin_bo_slot * vc4->bin_alloc_size); - V3D_WRITE(V3D_BPOS, bo->base.base.size); + V3D_WRITE(V3D_BPOS, vc4->bin_alloc_size); V3D_WRITE(V3D_INTCTL, V3D_INT_OUTOMEM); V3D_WRITE(V3D_INTENA, V3D_INT_OUTOMEM); spin_unlock_irqrestore(&vc4->job_lock, irqflags); diff --git a/drivers/gpu/drm/vc4/vc4_validate.c b/drivers/gpu/drm/vc4/vc4_validate.c index f3d7fdbe9083..8434037c509a 100644 --- a/drivers/gpu/drm/vc4/vc4_validate.c +++ b/drivers/gpu/drm/vc4/vc4_validate.c @@ -387,6 +387,23 @@ validate_tile_binning_config(VALIDATE_ARGS) return -EINVAL; } + /* The tile state data array is 48 bytes per tile, and we put it at + * the start of a BO containing both it and the tile alloc. + */ + tile_state_size = 48 * tile_count; + + /* Since the tile alloc array will follow us, align. */ + tile_state_size = roundup(tile_state_size, 4096); + + /* Reject configurations whose tile state would leave no room for + * the tile alloc pool that follows it in the slot. + */ + if (tile_state_size >= vc4->bin_alloc_size) { + DRM_DEBUG("Tile binning config of %dx%d too large\n", + exec->bin_tiles_x, exec->bin_tiles_y); + return -EINVAL; + } + bin_slot = vc4_v3d_get_bin_slot(vc4); if (bin_slot < 0) { if (bin_slot != -EINTR && bin_slot != -ERESTARTSYS) { @@ -402,13 +419,13 @@ validate_tile_binning_config(VALIDATE_ARGS) exec->bin_slots |= BIT(bin_slot); bin_addr = vc4->bin_bo->base.dma_addr + bin_slot * vc4->bin_alloc_size; - /* The tile state data array is 48 bytes per tile, and we put it at - * the start of a BO containing both it and the tile alloc. - */ - tile_state_size = 48 * tile_count; + exec->tile_alloc_offset = bin_addr + tile_state_size; - /* Since the tile alloc array will follow us, align. */ - exec->tile_alloc_offset = bin_addr + roundup(tile_state_size, 4096); + /* The TSDA area must be zeroed out before use, otherwise the PTB might + * consume a stale tile state. + */ + memset(vc4->bin_bo->base.vaddr + bin_slot * vc4->bin_alloc_size, 0, + tile_state_size); *(uint8_t *)(validated + 14) = ((flags & ~(VC4_BIN_CONFIG_ALLOC_INIT_BLOCK_SIZE_MASK | diff --git a/drivers/gpu/drm/vmwgfx/ttm_object.c b/drivers/gpu/drm/vmwgfx/ttm_object.c index 3353e97687d1..5a73f5719afd 100644 --- a/drivers/gpu/drm/vmwgfx/ttm_object.c +++ b/drivers/gpu/drm/vmwgfx/ttm_object.c @@ -547,14 +547,17 @@ int ttm_prime_fd_to_handle(struct ttm_object_file *tfile, if (IS_ERR(dma_buf)) return PTR_ERR(dma_buf); - if (dma_buf->ops != &tdev->ops) - return -ENOSYS; + if (dma_buf->ops != &tdev->ops) { + ret = -ENOSYS; + goto out; + } prime = (struct ttm_prime_object *) dma_buf->priv; base = &prime->base; *handle = base->handle; ret = ttm_ref_object_add(tfile, base, NULL, false); +out: dma_buf_put(dma_buf); return ret; diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_blit.c b/drivers/gpu/drm/vmwgfx/vmwgfx_blit.c index 64bd7d74854e..a1da91991093 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_blit.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_blit.c @@ -30,6 +30,7 @@ #include "vmwgfx_bo.h" #include +#include /* * Template that implements find_first_diff() for a generic @@ -463,19 +464,42 @@ static int vmw_external_bo_copy(struct vmw_bo *dst, u32 dst_offset, container_of(dst->tbo.bdev, struct vmw_private, bdev); size_t dst_size = dst->tbo.resource->size; size_t src_size = src->tbo.resource->size; + size_t dst_end, src_end; struct iosys_map dst_map = {0}; struct iosys_map src_map = {0}; + bool dst_mapped = false; + bool src_mapped = false; int ret, i; int x_in_bytes; u8 *vsrc; u8 *vdst; + if (!height || !width_in_bytes) + return 0; + + if (!dst_stride || !src_stride) + return -EINVAL; + if (dst_stride < width_in_bytes || src_stride < width_in_bytes) + return -EINVAL; + if (check_mul_overflow((size_t)dst_stride, (size_t)height - 1, &dst_end) || + check_add_overflow(dst_end, (size_t)width_in_bytes, &dst_end) || + check_add_overflow((size_t)dst_offset, dst_end, &dst_end) || + dst_end > dst_size || + check_mul_overflow((size_t)src_stride, (size_t)height - 1, &src_end) || + check_add_overflow(src_end, (size_t)width_in_bytes, &src_end) || + check_add_overflow((size_t)src_offset, src_end, &src_end) || + src_end > src_size) { + drm_dbg_driver(&vmw->drm, "Out-of-bounds external BO copy\n"); + return -EINVAL; + } + vsrc = map_external(src, &src_map); if (!vsrc) { drm_dbg_driver(&vmw->drm, "Wasn't able to map src\n"); ret = -ENOMEM; goto out; } + src_mapped = true; vdst = map_external(dst, &dst_map); if (!vdst) { @@ -483,16 +507,13 @@ static int vmw_external_bo_copy(struct vmw_bo *dst, u32 dst_offset, ret = -ENOMEM; goto out; } + dst_mapped = true; vsrc += src_offset; vdst += dst_offset; - if (src_stride == dst_stride) { - dst_size -= dst_offset; - src_size -= src_offset; - memcpy(vdst, vsrc, - min(dst_stride * height, min(dst_size, src_size))); + if (src_stride == dst_stride && width_in_bytes == dst_stride) { + memcpy(vdst, vsrc, dst_stride * (size_t)height); } else { - WARN_ON(dst_stride < width_in_bytes); for (i = 0; i < height; ++i) { memcpy(vdst, vsrc, width_in_bytes); vsrc += src_stride; @@ -508,8 +529,10 @@ static int vmw_external_bo_copy(struct vmw_bo *dst, u32 dst_offset, ret = 0; out: - unmap_external(src, &src_map); - unmap_external(dst, &dst_map); + if (src_mapped) + unmap_external(src, &src_map); + if (dst_mapped) + unmap_external(dst, &dst_map); return ret; } diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c b/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c index 6b921db2dcd2..8c53d46cd38c 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c @@ -1290,9 +1290,13 @@ static int vmw_cmd_dx_bind_query(struct vmw_private *dev_priv, SVGA3dCmdHeader *header) { VMW_DECLARE_CMD_VAR(*cmd, SVGA3dCmdDXBindQuery); + struct vmw_ctx_validation_info *ctx_node = VMW_GET_CTX_NODE(sw_context); struct vmw_bo *vmw_bo; int ret; + if (!ctx_node) + return -EINVAL; + cmd = container_of(header, typeof(*cmd), header); /* @@ -1306,7 +1310,7 @@ static int vmw_cmd_dx_bind_query(struct vmw_private *dev_priv, return ret; sw_context->dx_query_mob = vmw_bo; - sw_context->dx_query_ctx = sw_context->dx_ctx_node->ctx; + sw_context->dx_query_ctx = ctx_node->ctx; return 0; } @@ -1524,6 +1528,12 @@ static int vmw_cmd_dma(struct vmw_private *dev_priv, bool dirty; cmd = container_of(header, typeof(*cmd), header); + + if (unlikely(header->size < sizeof(cmd->body) + sizeof(*suffix))) { + VMW_DEBUG_USER("Illegal SVGA_3D_CMD_SURFACE_DMA size.\n"); + return -EINVAL; + } + suffix = (SVGA3dCmdSurfaceDMASuffix *)((unsigned long) &cmd->body + header->size - sizeof(*suffix)); @@ -1585,11 +1595,17 @@ static int vmw_cmd_draw(struct vmw_private *dev_priv, uint32_t maxnum; int ret; + cmd = container_of(header, typeof(*cmd), header); + + if (unlikely(header->size < sizeof(cmd->body))) { + VMW_DEBUG_USER("Illegal DRAW_PRIMITIVES header size.\n"); + return -EINVAL; + } + ret = vmw_cmd_cid_check(dev_priv, sw_context, header); if (unlikely(ret != 0)) return ret; - cmd = container_of(header, typeof(*cmd), header); maxnum = (header->size - sizeof(cmd->body)) / sizeof(*decl); if (unlikely(cmd->body.numVertexDecls > maxnum)) { diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c index c4d5fe5f330f..6c6a7bd76fcf 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c @@ -136,7 +136,7 @@ static void vmw_resource_release(struct kref *kref) val_buf.num_shared = 0; res->func->unbind(res, false, &val_buf); } - res->guest_memory_size = false; + res->guest_memory_dirty = false; vmw_resource_mob_detach(res); if (res->dirty) res->func->dirty_free(res); @@ -773,7 +773,7 @@ void vmw_resource_unbind_list(struct vmw_bo *vbo) if (!WARN_ON_ONCE(!res->func->unbind)) (void) res->func->unbind(res, res->res_dirty, &val_buf); - res->guest_memory_size = true; + res->guest_memory_dirty = true; res->res_dirty = false; vmw_resource_mob_detach(res); } diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c b/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c index a8c8c9375d29..bc89a137ab6d 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c @@ -25,6 +25,8 @@ * **************************************************************************/ +#include + #include #include "vmwgfx_binding.h" @@ -685,7 +687,7 @@ int vmw_shader_destroy_ioctl(struct drm_device *dev, void *data, static int vmw_user_shader_alloc(struct vmw_private *dev_priv, struct vmw_bo *buffer, size_t shader_size, - size_t offset, + u64 offset, SVGA3dShaderType shader_type, uint8_t num_input_sig, uint8_t num_output_sig, @@ -739,7 +741,7 @@ static int vmw_user_shader_alloc(struct vmw_private *dev_priv, static struct vmw_resource *vmw_shader_alloc(struct vmw_private *dev_priv, struct vmw_bo *buffer, size_t shader_size, - size_t offset, + u64 offset, SVGA3dShaderType shader_type) { struct vmw_shader *shader; @@ -768,7 +770,7 @@ static struct vmw_resource *vmw_shader_alloc(struct vmw_private *dev_priv, static int vmw_shader_define(struct drm_device *dev, struct drm_file *file_priv, enum drm_vmw_shader_type shader_type_drm, - u32 buffer_handle, size_t size, size_t offset, + u32 buffer_handle, size_t size, u64 offset, uint8_t num_input_sig, uint8_t num_output_sig, uint32_t *shader_handle) { @@ -779,13 +781,16 @@ static int vmw_shader_define(struct drm_device *dev, struct drm_file *file_priv, int ret; if (buffer_handle != SVGA3D_INVALID_ID) { + u64 end; + ret = vmw_user_bo_lookup(file_priv, buffer_handle, &buffer); if (unlikely(ret != 0)) { VMW_DEBUG_USER("Couldn't find buffer for shader creation.\n"); return ret; } - if ((u64)buffer->tbo.base.size < (u64)size + (u64)offset) { + if (check_add_overflow((u64)size, (u64)offset, &end) || + end > buffer->tbo.base.size) { VMW_DEBUG_USER("Illegal buffer- or shader size.\n"); ret = -EINVAL; goto out_bad_arg; diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c b/drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c index 8651b788e98b..c8a614869756 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c @@ -214,14 +214,14 @@ vmw_vkms_init(struct vmw_private *vmw) vmw->vkms_enabled = false; ret = vmw_host_get_guestinfo(GUESTINFO_VBLANK, buffer, &buf_len); - if (ret || buf_len > max_buf_len) - return; - buffer[buf_len] = '\0'; + if (!ret && buf_len <= max_buf_len) { + buffer[buf_len] = '\0'; - ret = kstrtobool(buffer, &vmw->vkms_enabled); - if (!ret && vmw->vkms_enabled) { - ret = drm_vblank_init(&vmw->drm, VMWGFX_NUM_DISPLAY_UNITS); - vmw->vkms_enabled = (ret == 0); + ret = kstrtobool(buffer, &vmw->vkms_enabled); + if (!ret && vmw->vkms_enabled) { + ret = drm_vblank_init(&vmw->drm, VMWGFX_NUM_DISPLAY_UNITS); + vmw->vkms_enabled = (ret == 0); + } } vmw->crc_workq = alloc_ordered_workqueue("vmwgfx_crc_generator", 0); @@ -236,7 +236,8 @@ vmw_vkms_init(struct vmw_private *vmw) void vmw_vkms_cleanup(struct vmw_private *vmw) { - destroy_workqueue(vmw->crc_workq); + if (vmw->crc_workq) + destroy_workqueue(vmw->crc_workq); } bool diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile index edfd812e0f41..4464d56b4540 100644 --- a/drivers/gpu/drm/xe/Makefile +++ b/drivers/gpu/drm/xe/Makefile @@ -78,6 +78,7 @@ xe-y += xe_bb.o \ xe_module.o \ xe_oa.o \ xe_observation.o \ + xe_pagefault.o \ xe_pat.o \ xe_pci.o \ xe_pcode.o \ diff --git a/drivers/gpu/drm/xe/display/xe_hdcp_gsc.c b/drivers/gpu/drm/xe/display/xe_hdcp_gsc.c index 695d625c83ee..8dd2cfbf7e64 100644 --- a/drivers/gpu/drm/xe/display/xe_hdcp_gsc.c +++ b/drivers/gpu/drm/xe/display/xe_hdcp_gsc.c @@ -30,13 +30,14 @@ struct intel_hdcp_gsc_message { #define HDCP_GSC_HEADER_SIZE sizeof(struct intel_gsc_mtl_header) -bool intel_hdcp_gsc_cs_required(struct xe_device *xe) +bool intel_hdcp_gsc_cs_required(struct intel_display *display) { - return DISPLAY_VER(xe) >= 14; + return DISPLAY_VER(display) >= 14; } -bool intel_hdcp_gsc_check_status(struct xe_device *xe) +bool intel_hdcp_gsc_check_status(struct intel_display *display) { + struct xe_device *xe = to_xe_device(display->drm); struct xe_tile *tile = xe_device_get_root_tile(xe); struct xe_gt *gt = tile->media_gt; struct xe_gsc *gsc; diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index d5b63d90947f..b5f5f85ac3c5 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -1131,6 +1131,8 @@ static void xe_ttm_bo_destroy(struct ttm_buffer_object *ttm_bo) if (bo->ttm.base.import_attach) drm_prime_gem_destroy(&bo->ttm.base, NULL); + if (bo->dma_buf) + dma_buf_put(bo->dma_buf); drm_gem_object_release(&bo->ttm.base); xe_assert(xe, list_empty(&ttm_bo->base.gpuva.list)); @@ -1288,11 +1290,11 @@ void xe_bo_free(struct xe_bo *bo) kfree(bo); } -struct xe_bo *___xe_bo_create_locked(struct xe_device *xe, struct xe_bo *bo, - struct xe_tile *tile, struct dma_resv *resv, - struct ttm_lru_bulk_move *bulk, size_t size, - u16 cpu_caching, enum ttm_bo_type type, - u32 flags) +struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo, + struct xe_tile *tile, struct dma_resv *resv, + struct ttm_lru_bulk_move *bulk, size_t size, + u16 cpu_caching, enum ttm_bo_type type, + u32 flags, struct dma_buf *dma_buf) { struct ttm_operation_ctx ctx = { .interruptible = true, @@ -1377,6 +1379,17 @@ struct xe_bo *___xe_bo_create_locked(struct xe_device *xe, struct xe_bo *bo, placement = (type == ttm_bo_type_sg || bo->flags & XE_BO_FLAG_DEFER_BACKING) ? &sys_placement : &bo->placement; + + /* + * For imported BOs, keep the exporter dma-buf alive for the BO + * lifetime. Taken before ttm_bo_init_reserved() to also cover a + * creation failure there. Released in xe_ttm_bo_destroy(). + */ + if (dma_buf) { + get_dma_buf(dma_buf); + bo->dma_buf = dma_buf; + } + err = ttm_bo_init_reserved(&xe->ttm, &bo->ttm, type, placement, alignment, &ctx, NULL, resv, xe_ttm_bo_destroy); @@ -1487,11 +1500,11 @@ __xe_bo_create_locked(struct xe_device *xe, } } - bo = ___xe_bo_create_locked(xe, bo, tile, vm ? xe_vm_resv(vm) : NULL, - vm && !xe_vm_in_fault_mode(vm) && - flags & XE_BO_FLAG_USER ? - &vm->lru_bulk_move : NULL, size, - cpu_caching, type, flags); + bo = xe_bo_init_locked(xe, bo, tile, vm ? xe_vm_resv(vm) : NULL, + vm && !xe_vm_in_fault_mode(vm) && + flags & XE_BO_FLAG_USER ? + &vm->lru_bulk_move : NULL, size, + cpu_caching, type, flags, NULL); if (IS_ERR(bo)) return bo; diff --git a/drivers/gpu/drm/xe/xe_bo.h b/drivers/gpu/drm/xe/xe_bo.h index 5e982be66c74..556ce5390009 100644 --- a/drivers/gpu/drm/xe/xe_bo.h +++ b/drivers/gpu/drm/xe/xe_bo.h @@ -80,11 +80,11 @@ struct sg_table; struct xe_bo *xe_bo_alloc(void); void xe_bo_free(struct xe_bo *bo); -struct xe_bo *___xe_bo_create_locked(struct xe_device *xe, struct xe_bo *bo, - struct xe_tile *tile, struct dma_resv *resv, - struct ttm_lru_bulk_move *bulk, size_t size, - u16 cpu_caching, enum ttm_bo_type type, - u32 flags); +struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo, + struct xe_tile *tile, struct dma_resv *resv, + struct ttm_lru_bulk_move *bulk, size_t size, + u16 cpu_caching, enum ttm_bo_type type, + u32 flags, struct dma_buf *dma_buf); struct xe_bo * xe_bo_create_locked_range(struct xe_device *xe, struct xe_tile *tile, struct xe_vm *vm, diff --git a/drivers/gpu/drm/xe/xe_bo_types.h b/drivers/gpu/drm/xe/xe_bo_types.h index aa298d33c250..cecb893ba128 100644 --- a/drivers/gpu/drm/xe/xe_bo_types.h +++ b/drivers/gpu/drm/xe/xe_bo_types.h @@ -30,6 +30,8 @@ struct xe_bo { struct ttm_buffer_object ttm; /** @size: Size of this buffer object */ size_t size; + /** @dma_buf: Imported dma-buf ref to keep its resv alive. */ + struct dma_buf *dma_buf; /** @flags: flags for this buffer object */ u32 flags; /** @vm: VM this BO is attached to, for extobj this will be NULL */ diff --git a/drivers/gpu/drm/xe/xe_dma_buf.c b/drivers/gpu/drm/xe/xe_dma_buf.c index c422f7d60115..23ffd1d89565 100644 --- a/drivers/gpu/drm/xe/xe_dma_buf.c +++ b/drivers/gpu/drm/xe/xe_dma_buf.c @@ -212,9 +212,9 @@ xe_dma_buf_init_obj(struct drm_device *dev, struct xe_bo *storage, int ret; dma_resv_lock(resv, NULL); - bo = ___xe_bo_create_locked(xe, storage, NULL, resv, NULL, dma_buf->size, - 0, /* Will require 1way or 2way for vm_bind */ - ttm_bo_type_sg, XE_BO_FLAG_SYSTEM); + bo = xe_bo_init_locked(xe, storage, NULL, resv, NULL, dma_buf->size, + 0, /* Will require 1way or 2way for vm_bind */ + ttm_bo_type_sg, XE_BO_FLAG_SYSTEM, dma_buf); if (IS_ERR(bo)) { ret = PTR_ERR(bo); goto error; diff --git a/drivers/gpu/drm/xe/xe_exec.c b/drivers/gpu/drm/xe/xe_exec.c index 886d03ccf744..04a88c16d835 100644 --- a/drivers/gpu/drm/xe/xe_exec.c +++ b/drivers/gpu/drm/xe/xe_exec.c @@ -270,13 +270,23 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto err_exec; } - /* Wait behind rebinds */ + /* + * Wait behind rebinds and any kernel operations (evictions, defrag + * moves, ...) on the VM and all external BOs. The VM's private BOs + * carry their kernel ops in the VM dma-resv KERNEL slot, while each + * external BO carries them in its own dma-resv KERNEL slot; both are + * covered by iterating every object locked by the exec, mirroring the + * drm_gpuvm_resv_add_fence() below. + */ if (!xe_vm_in_lr_mode(vm)) { - err = xe_sched_job_add_deps(job, - xe_vm_resv(vm), - DMA_RESV_USAGE_KERNEL); - if (err) - goto err_put_job; + struct drm_gem_object *obj; + + drm_exec_for_each_locked_object(exec, obj) { + err = xe_sched_job_add_deps(job, obj->resv, + DMA_RESV_USAGE_KERNEL); + if (err) + goto err_put_job; + } } for (i = 0; i < num_syncs && !err; i++) diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c index 292947e44a8a..e2ac1589dfbf 100644 --- a/drivers/gpu/drm/xe/xe_gt.c +++ b/drivers/gpu/drm/xe/xe_gt.c @@ -722,10 +722,8 @@ static int do_gt_restart(struct xe_gt *gt) xe_mocs_init(gt); - for_each_hw_engine(hwe, gt, id) { + for_each_hw_engine(hwe, gt, id) xe_reg_sr_apply_mmio(&hwe->reg_sr, gt); - xe_reg_sr_apply_whitelist(hwe); - } /* Get CCS mode in sync between sw/hw */ xe_gt_apply_ccs_mode(gt); diff --git a/drivers/gpu/drm/xe/xe_gt_debugfs.c b/drivers/gpu/drm/xe/xe_gt_debugfs.c index 8f95d3a5949b..15ff52dfe461 100644 --- a/drivers/gpu/drm/xe/xe_gt_debugfs.c +++ b/drivers/gpu/drm/xe/xe_gt_debugfs.c @@ -191,8 +191,10 @@ static int register_save_restore(struct xe_gt *gt, struct drm_printer *p) drm_printf(p, "\n"); drm_printf(p, "Whitelist\n"); - for_each_hw_engine(hwe, gt, id) + for_each_hw_engine(hwe, gt, id) { xe_reg_whitelist_dump(&hwe->reg_whitelist, p); + xe_reg_whitelist_dump(&hwe->oa_whitelist, p); + } xe_pm_runtime_put(gt_to_xe(gt)); diff --git a/drivers/gpu/drm/xe/xe_gt_printk.h b/drivers/gpu/drm/xe/xe_gt_printk.h index 5dc71394372d..11da0228cea7 100644 --- a/drivers/gpu/drm/xe/xe_gt_printk.h +++ b/drivers/gpu/drm/xe/xe_gt_printk.h @@ -60,6 +60,21 @@ static inline void __xe_gt_printfn_info(struct drm_printer *p, struct va_format xe_gt_info(gt, "%pV", vaf); } +static inline void __xe_gt_printfn_dbg(struct drm_printer *p, struct va_format *vaf) +{ + struct xe_gt *gt = p->arg; + struct drm_printer dbg; + + /* + * The original xe_gt_dbg() callsite annotations are useless here, + * redirect to the tweaked drm_dbg_printer() instead. + */ + dbg = drm_dbg_printer(>_to_xe(gt)->drm, DRM_UT_DRIVER, NULL); + dbg.origin = p->origin; + + drm_printf(&dbg, "GT%u: %pV", gt->info.id, vaf); +} + /** * xe_gt_err_printer - Construct a &drm_printer that outputs to xe_gt_err() * @gt: the &xe_gt pointer to use in xe_gt_err() @@ -90,4 +105,20 @@ static inline struct drm_printer xe_gt_info_printer(struct xe_gt *gt) return p; } +/** + * xe_gt_dbg_printer - Construct a &drm_printer that outputs like xe_gt_dbg() + * @gt: the &xe_gt pointer to use in xe_gt_dbg() + * + * Return: The &drm_printer object. + */ +static inline struct drm_printer xe_gt_dbg_printer(struct xe_gt *gt) +{ + struct drm_printer p = { + .printfn = __xe_gt_printfn_dbg, + .arg = gt, + .origin = (const void *)_THIS_IP_, + }; + return p; +} + #endif diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c index 45d0ae074e65..9f560ac9e2b8 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine.c +++ b/drivers/gpu/drm/xe/xe_hw_engine.c @@ -532,6 +532,8 @@ static void hw_engine_init_early(struct xe_gt *gt, struct xe_hw_engine *hwe, hw_engine_setup_default_state(hwe); xe_reg_sr_init(&hwe->reg_whitelist, hwe->name, gt_to_xe(gt)); + xe_reg_sr_init(&hwe->oa_whitelist, hwe->name, gt_to_xe(gt)); + xe_reg_sr_init(&hwe->oa_sr, hwe->name, gt_to_xe(gt)); xe_reg_whitelist_process_engine(hwe); } @@ -546,7 +548,6 @@ static int hw_engine_init(struct xe_gt *gt, struct xe_hw_engine *hwe, xe_gt_assert(gt, gt->info.engine_mask & BIT(id)); xe_reg_sr_apply_mmio(&hwe->reg_sr, gt); - xe_reg_sr_apply_whitelist(hwe); hwe->hwsp = xe_managed_bo_create_pin_map(xe, tile, SZ_4K, XE_BO_FLAG_VRAM_IF_DGFX(tile) | diff --git a/drivers/gpu/drm/xe/xe_hw_engine_types.h b/drivers/gpu/drm/xe/xe_hw_engine_types.h index 8be6d420ece4..ac876a8c3eec 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine_types.h +++ b/drivers/gpu/drm/xe/xe_hw_engine_types.h @@ -128,6 +128,14 @@ struct xe_hw_engine { * @reg_whitelist: table with registers to be whitelisted */ struct xe_reg_sr reg_whitelist; + /** + * @oa_whitelist: oa registers to be whitelisted + */ + struct xe_reg_sr oa_whitelist; + /** + * @oa_sr: oa nonpriv whitelist registers, changed on oa stream open/close + */ + struct xe_reg_sr oa_sr; /** * @reg_lrc: LRC workaround registers */ diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c index 5e38afc46e20..476d887749a9 100644 --- a/drivers/gpu/drm/xe/xe_oa.c +++ b/drivers/gpu/drm/xe/xe_oa.c @@ -34,6 +34,7 @@ #include "xe_oa.h" #include "xe_observation.h" #include "xe_pm.h" +#include "xe_reg_whitelist.h" #include "xe_sched_job.h" #include "xe_sriov.h" #include "xe_sync.h" @@ -836,6 +837,9 @@ static void xe_oa_stream_destroy(struct xe_oa_stream *stream) mutex_destroy(&stream->stream_lock); + if (stream->sample) + xe_reg_dewhitelist_oa_regs(stream->gt); + xe_oa_disable_metric_set(stream); xe_exec_queue_put(stream->k_exec_q); @@ -1868,6 +1872,9 @@ static int xe_oa_stream_open_ioctl_locked(struct xe_oa *oa, goto err_disable; } + if (stream->sample) + xe_reg_whitelist_oa_regs(stream->gt); + /* Hold a reference on the drm device till stream_fd is released */ drm_dev_get(&stream->oa->xe->drm); diff --git a/drivers/gpu/drm/xe/xe_oa_types.h b/drivers/gpu/drm/xe/xe_oa_types.h index fea9d981e414..0faec73954b7 100644 --- a/drivers/gpu/drm/xe/xe_oa_types.h +++ b/drivers/gpu/drm/xe/xe_oa_types.h @@ -120,6 +120,9 @@ struct xe_oa_gt { /** @oa_unit: array of oa_units */ struct xe_oa_unit *oa_unit; + + /** @whitelist_count: number of open streams for which oa registers are whitelisted */ + u32 whitelist_count; }; /** diff --git a/drivers/gpu/drm/xe/xe_pagefault.c b/drivers/gpu/drm/xe/xe_pagefault.c new file mode 100644 index 000000000000..d509a80cb1f3 --- /dev/null +++ b/drivers/gpu/drm/xe/xe_pagefault.c @@ -0,0 +1,65 @@ +// SPDX-License-Identifier: MIT +/* + * Copyright © 2025 Intel Corporation + */ + +#include "xe_pagefault.h" +#include "xe_pagefault_types.h" + +/** + * DOC: Xe page faults + * + * Xe page faults are handled in two layers. The producer layer interacts with + * hardware or firmware to receive and parse faults into struct xe_pagefault, + * then forwards them to the consumer. The consumer layer services the faults + * (e.g., memory migration, page table updates) and acknowledges the result back + * to the producer, which then forwards the results to the hardware or firmware. + * The consumer uses a page fault queue sized to absorb all potential faults and + * a multi-threaded worker to process them. Multiple producers are supported, + * with a single shared consumer. + * + * xe_pagefault.c implements the consumer layer. + */ + +/** + * xe_pagefault_init() - Page fault init + * @xe: xe device instance + * + * Initialize Xe page fault state. Must be done after reading fuses. + * + * Return: 0 on Success, errno on failure + */ +int xe_pagefault_init(struct xe_device *xe) +{ + /* TODO - implement */ + return 0; +} + +/** + * xe_pagefault_reset() - Page fault reset for a GT + * @xe: xe device instance + * @gt: GT being reset + * + * Reset the Xe page fault state for a GT; that is, squash any pending faults on + * the GT. + */ +void xe_pagefault_reset(struct xe_device *xe, struct xe_gt *gt) +{ + /* TODO - implement */ +} + +/** + * xe_pagefault_handler() - Page fault handler + * @xe: xe device instance + * @pf: Page fault + * + * Sink the page fault to a queue (i.e., a memory buffer) and queue a worker to + * service it. Safe to be called from IRQ or process context. Reclaim safe. + * + * Return: 0 on success, errno on failure + */ +int xe_pagefault_handler(struct xe_device *xe, struct xe_pagefault *pf) +{ + /* TODO - implement */ + return 0; +} diff --git a/drivers/gpu/drm/xe/xe_pagefault.h b/drivers/gpu/drm/xe/xe_pagefault.h new file mode 100644 index 000000000000..bd0cdf9ed37f --- /dev/null +++ b/drivers/gpu/drm/xe/xe_pagefault.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2025 Intel Corporation + */ + +#ifndef _XE_PAGEFAULT_H_ +#define _XE_PAGEFAULT_H_ + +struct xe_device; +struct xe_gt; +struct xe_pagefault; + +int xe_pagefault_init(struct xe_device *xe); + +void xe_pagefault_reset(struct xe_device *xe, struct xe_gt *gt); + +int xe_pagefault_handler(struct xe_device *xe, struct xe_pagefault *pf); + +#endif diff --git a/drivers/gpu/drm/xe/xe_pagefault_types.h b/drivers/gpu/drm/xe/xe_pagefault_types.h new file mode 100644 index 000000000000..d3b516407d60 --- /dev/null +++ b/drivers/gpu/drm/xe/xe_pagefault_types.h @@ -0,0 +1,136 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2025 Intel Corporation + */ + +#ifndef _XE_PAGEFAULT_TYPES_H_ +#define _XE_PAGEFAULT_TYPES_H_ + +#include + +struct xe_gt; +struct xe_pagefault; + +/** enum xe_pagefault_access_type - Xe page fault access type */ +enum xe_pagefault_access_type { + /** @XE_PAGEFAULT_ACCESS_TYPE_READ: Read access type */ + XE_PAGEFAULT_ACCESS_TYPE_READ = 0, + /** @XE_PAGEFAULT_ACCESS_TYPE_WRITE: Write access type */ + XE_PAGEFAULT_ACCESS_TYPE_WRITE = 1, + /** @XE_PAGEFAULT_ACCESS_TYPE_ATOMIC: Atomic access type */ + XE_PAGEFAULT_ACCESS_TYPE_ATOMIC = 2, +}; + +/** enum xe_pagefault_type - Xe page fault type */ +enum xe_pagefault_type { + /** @XE_PAGEFAULT_TYPE_NOT_PRESENT: Not present */ + XE_PAGEFAULT_TYPE_NOT_PRESENT = 0, + /** @XE_PAGEFAULT_TYPE_WRITE_ACCESS_VIOLATION: Write access violation */ + XE_PAGEFAULT_TYPE_WRITE_ACCESS_VIOLATION = 1, + /** @XE_PAGEFAULT_TYPE_ATOMIC_ACCESS_VIOLATION: Atomic access violation */ + XE_PAGEFAULT_TYPE_ATOMIC_ACCESS_VIOLATION = 2, +}; + +/** struct xe_pagefault_ops - Xe pagefault ops (producer) */ +struct xe_pagefault_ops { + /** + * @ack_fault: Ack fault + * @pf: Page fault + * @err: Error state of fault + * + * Page fault producer receives acknowledgment from the consumer and + * sends the result to the HW/FW interface. + */ + void (*ack_fault)(struct xe_pagefault *pf, int err); +}; + +/** + * struct xe_pagefault - Xe page fault + * + * Generic page fault structure for communication between producer and consumer. + * Carefully sized to be 64 bytes. Upon a device page fault, the producer + * populates this structure, and the consumer copies it into the page-fault + * queue for deferred handling. + */ +struct xe_pagefault { + /** + * @gt: GT of fault + */ + struct xe_gt *gt; + /** + * @consumer: State for the software handling the fault. Populated by + * the producer and may be modified by the consumer to communicate + * information back to the producer upon fault acknowledgment. + */ + struct { + /** @consumer.page_addr: address of page fault */ + u64 page_addr; + /** @consumer.asid: address space ID */ + u32 asid; + /** + * @consumer.access_type: access type, u8 rather than enum to + * keep size compact + */ + u8 access_type; + /** + * @consumer.fault_type: fault type, u8 rather than enum to + * keep size compact + */ + u8 fault_type; +#define XE_PAGEFAULT_LEVEL_NACK 0xff /* Producer indicates nack fault */ + /** @consumer.fault_level: fault level */ + u8 fault_level; + /** @consumer.engine_class: engine class */ + u8 engine_class; + /** @consumer.engine_instance: engine instance */ + u8 engine_instance; + /** consumer.reserved: reserved bits for future expansion */ + u8 reserved[7]; + } consumer; + /** + * @producer: State for the producer (i.e., HW/FW interface). Populated + * by the producer and should not be modified—or even inspected—by the + * consumer, except for calling operations. + */ + struct { + /** @producer.private: private pointer */ + void *private; + /** @producer.ops: operations */ + const struct xe_pagefault_ops *ops; +#define XE_PAGEFAULT_PRODUCER_MSG_LEN_DW 4 + /** + * @producer.msg: page fault message, used by producer in fault + * acknowledgment to formulate response to HW/FW interface. + * Included in the page-fault message because the producer + * typically receives the fault in a context where memory cannot + * be allocated (e.g., atomic context or the reclaim path). + */ + u32 msg[XE_PAGEFAULT_PRODUCER_MSG_LEN_DW]; + } producer; +}; + +/** + * struct xe_pagefault_queue: Xe pagefault queue (consumer) + * + * Used to capture all device page faults for deferred processing. Size this + * queue to absorb the device’s worst-case number of outstanding faults. + */ +struct xe_pagefault_queue { + /** + * @data: Data in queue containing struct xe_pagefault, protected by + * @lock + */ + void *data; + /** @size: Size of queue in bytes */ + u32 size; + /** @head: Head pointer in bytes, moved by producer, protected by @lock */ + u32 head; + /** @tail: Tail pointer in bytes, moved by consumer, protected by @lock */ + u32 tail; + /** @lock: protects page fault queue */ + spinlock_t lock; + /** @worker: to process page faults */ + struct work_struct worker; +}; + +#endif diff --git a/drivers/gpu/drm/xe/xe_pt.c b/drivers/gpu/drm/xe/xe_pt.c index 0025871485cb..9e60b8cccd0f 100644 --- a/drivers/gpu/drm/xe/xe_pt.c +++ b/drivers/gpu/drm/xe/xe_pt.c @@ -1833,8 +1833,11 @@ static void xe_pt_update_ops_init(struct xe_vm_pgtable_update_ops *pt_update_ops) { init_llist_head(&pt_update_ops->deferred); + pt_update_ops->current_op = 0; pt_update_ops->start = ~0x0ull; pt_update_ops->last = 0x0ull; + pt_update_ops->needs_userptr_lock = false; + pt_update_ops->needs_invalidation = false; } /** diff --git a/drivers/gpu/drm/xe/xe_reg_sr.c b/drivers/gpu/drm/xe/xe_reg_sr.c index ae9e6df2f4e1..01ee54d53531 100644 --- a/drivers/gpu/drm/xe/xe_reg_sr.c +++ b/drivers/gpu/drm/xe/xe_reg_sr.c @@ -24,7 +24,6 @@ #include "xe_hw_engine_types.h" #include "xe_macros.h" #include "xe_mmio.h" -#include "xe_reg_whitelist.h" #include "xe_rtp_types.h" static void reg_sr_fini(struct drm_device *drm, void *arg) @@ -194,58 +193,6 @@ void xe_reg_sr_apply_mmio(struct xe_reg_sr *sr, struct xe_gt *gt) xe_gt_err(gt, "Failed to apply, err=%d\n", err); } -void xe_reg_sr_apply_whitelist(struct xe_hw_engine *hwe) -{ - struct xe_reg_sr *sr = &hwe->reg_whitelist; - struct xe_gt *gt = hwe->gt; - struct xe_device *xe = gt_to_xe(gt); - struct xe_reg_sr_entry *entry; - struct drm_printer p; - u32 mmio_base = hwe->mmio_base; - unsigned long reg; - unsigned int slot = 0; - int err; - - if (xa_empty(&sr->xa)) - return; - - drm_dbg(&xe->drm, "Whitelisting %s registers\n", sr->name); - - err = xe_force_wake_get(gt_to_fw(gt), XE_FORCEWAKE_ALL); - if (err) - goto err_force_wake; - - p = drm_dbg_printer(&xe->drm, DRM_UT_DRIVER, NULL); - xa_for_each(&sr->xa, reg, entry) { - if (slot == RING_MAX_NONPRIV_SLOTS) { - xe_gt_err(gt, - "hwe %s: maximum register whitelist slots (%d) reached, refusing to add more\n", - hwe->name, RING_MAX_NONPRIV_SLOTS); - break; - } - - xe_reg_whitelist_print_entry(&p, 0, reg, entry); - xe_mmio_write32(gt, RING_FORCE_TO_NONPRIV(mmio_base, slot), - reg | entry->set_bits); - slot++; - } - - /* And clear the rest just in case of garbage */ - for (; slot < RING_MAX_NONPRIV_SLOTS; slot++) { - u32 addr = RING_NOPID(mmio_base).addr; - - xe_mmio_write32(gt, RING_FORCE_TO_NONPRIV(mmio_base, slot), addr); - } - - err = xe_force_wake_put(gt_to_fw(gt), XE_FORCEWAKE_ALL); - XE_WARN_ON(err); - - return; - -err_force_wake: - drm_err(&xe->drm, "Failed to apply, err=%d\n", err); -} - /** * xe_reg_sr_dump - print all save/restore entries * @sr: Save/restore entries diff --git a/drivers/gpu/drm/xe/xe_reg_whitelist.c b/drivers/gpu/drm/xe/xe_reg_whitelist.c index 3de0a867149d..4b70460d6e00 100644 --- a/drivers/gpu/drm/xe/xe_reg_whitelist.c +++ b/drivers/gpu/drm/xe/xe_reg_whitelist.c @@ -9,8 +9,11 @@ #include "regs/xe_gt_regs.h" #include "regs/xe_oa_regs.h" #include "regs/xe_regs.h" -#include "xe_gt_types.h" +#include "xe_assert.h" +#include "xe_gt.h" +#include "xe_gt_printk.h" #include "xe_platform_types.h" +#include "xe_reg_sr.h" #include "xe_rtp.h" #include "xe_step.h" @@ -64,31 +67,71 @@ static const struct xe_rtp_entry_sr register_whitelist[] = { ENGINE_CLASS(RENDER)), XE_RTP_ACTIONS(WHITELIST(CSBE_DEBUG_STATUS(RENDER_RING_BASE), 0)) }, - { XE_RTP_NAME("oa_reg_render"), + {} +}; + +static const struct xe_rtp_entry_sr oa_whitelist[] = { + +#define WHITELIST_DENY(r, f) WHITELIST(r, (f) | RING_FORCE_TO_NONPRIV_DENY) + +#define WHITELIST_OA_MMIO_TRG(trg, status, head) \ + WHITELIST_DENY(trg, RING_FORCE_TO_NONPRIV_ACCESS_RW), \ + WHITELIST_DENY(status, RING_FORCE_TO_NONPRIV_ACCESS_RD), \ + WHITELIST_DENY(head, RING_FORCE_TO_NONPRIV_ACCESS_RD | RING_FORCE_TO_NONPRIV_RANGE_4) + +#define WHITELIST_OAG_MMIO_TRG \ + WHITELIST_OA_MMIO_TRG(OAG_MMIOTRIGGER, OAG_OASTATUS, OAG_OAHEADPTR) + + { XE_RTP_NAME("oag_mmio_trg_rcs"), XE_RTP_RULES(GRAPHICS_VERSION_RANGE(1200, XE_RTP_END_VERSION_UNDEFINED), ENGINE_CLASS(RENDER)), - XE_RTP_ACTIONS(WHITELIST(OAG_MMIOTRIGGER, - RING_FORCE_TO_NONPRIV_ACCESS_RW), - WHITELIST(OAG_OASTATUS, - RING_FORCE_TO_NONPRIV_ACCESS_RD), - WHITELIST(OAG_OAHEADPTR, - RING_FORCE_TO_NONPRIV_ACCESS_RD | - RING_FORCE_TO_NONPRIV_RANGE_4)) + XE_RTP_ACTIONS(WHITELIST_OAG_MMIO_TRG) }, - { XE_RTP_NAME("oa_reg_compute"), + { XE_RTP_NAME("oag_mmio_trg_ccs"), XE_RTP_RULES(GRAPHICS_VERSION_RANGE(1200, XE_RTP_END_VERSION_UNDEFINED), ENGINE_CLASS(COMPUTE)), - XE_RTP_ACTIONS(WHITELIST(OAG_MMIOTRIGGER, - RING_FORCE_TO_NONPRIV_ACCESS_RW), - WHITELIST(OAG_OASTATUS, - RING_FORCE_TO_NONPRIV_ACCESS_RD), - WHITELIST(OAG_OAHEADPTR, - RING_FORCE_TO_NONPRIV_ACCESS_RD | - RING_FORCE_TO_NONPRIV_RANGE_4)) + XE_RTP_ACTIONS(WHITELIST_OAG_MMIO_TRG) }, {} }; +static int whitelist_apply_to_hwe(struct xe_hw_engine *hwe, struct xe_reg_sr *in, + struct xe_reg_sr *out, int first_slot) +{ + struct xe_reg_sr *sr = in; + struct xe_reg_sr_entry *entry; + struct drm_printer p; + unsigned long reg; + unsigned int slot; + + xe_gt_dbg(hwe->gt, "Add %s whitelist to engine\n", sr->name); + p = xe_gt_dbg_printer(hwe->gt); + + slot = first_slot; + xa_for_each(&sr->xa, reg, entry) { + struct xe_reg_sr_entry hwe_entry = { + .reg = RING_FORCE_TO_NONPRIV(hwe->mmio_base, slot), + .set_bits = entry->reg.addr | entry->set_bits, + .clr_bits = ~0u, + .read_mask = entry->read_mask, + }; + + if (slot == RING_MAX_NONPRIV_SLOTS) { + xe_gt_err(hwe->gt, + "hwe %s: maximum register whitelist slots (%d) reached, refusing to add more\n", + hwe->name, RING_MAX_NONPRIV_SLOTS); + break; + } + + xe_reg_whitelist_print_entry(&p, 0, reg, entry); + xe_reg_sr_add(out, &hwe_entry, hwe->gt); + + slot++; + } + + return slot; +} + /** * xe_reg_whitelist_process_engine - process table of registers to whitelist * @hwe: engine instance to process whitelist for @@ -100,8 +143,78 @@ static const struct xe_rtp_entry_sr register_whitelist[] = { void xe_reg_whitelist_process_engine(struct xe_hw_engine *hwe) { struct xe_rtp_process_ctx ctx = XE_RTP_PROCESS_CTX_INITIALIZER(hwe); + int first_oa_slot; xe_rtp_process_to_sr(&ctx, register_whitelist, &hwe->reg_whitelist); + first_oa_slot = whitelist_apply_to_hwe(hwe, &hwe->reg_whitelist, &hwe->reg_sr, 0); + + xe_rtp_process_to_sr(&ctx, oa_whitelist, &hwe->oa_whitelist); + + /* + * Save oa nonpriv registers to hwe->oa_sr, from which oa registers are whitelisted + * or de-whitelisted, by toggling the 'deny' bit on oa stream open/close + */ + whitelist_apply_to_hwe(hwe, &hwe->oa_whitelist, &hwe->oa_sr, first_oa_slot); + + /* + * Also save oa nonpriv registers to hwe->reg_sr, to ensure oa registers are not + * whitelisted by default after probe, gt reset, resume and engine reset + */ + whitelist_apply_to_hwe(hwe, &hwe->oa_whitelist, &hwe->reg_sr, first_oa_slot); +} + +static void __whitelist_oa_regs(struct xe_hw_engine *hwe, bool whitelist) +{ + struct xe_reg_sr_entry *entry; + unsigned long reg; + + xa_for_each(&hwe->oa_sr.xa, reg, entry) { + if (whitelist) + entry->set_bits &= ~RING_FORCE_TO_NONPRIV_DENY; + else + entry->set_bits |= RING_FORCE_TO_NONPRIV_DENY; + } + + xe_reg_sr_apply_mmio(&hwe->oa_sr, hwe->gt); +} + +/** + * xe_reg_whitelist_oa_regs - whitelist oa registers for gt + * @gt: gt to whitelist oa registers for + * + * Whitelist OA registers by resetting RING_FORCE_TO_NONPRIV_DENY + */ +void xe_reg_whitelist_oa_regs(struct xe_gt *gt) +{ + struct xe_hw_engine *hwe; + enum xe_hw_engine_id id; + + lockdep_assert_held(>->oa.gt_lock); + if (gt->oa.whitelist_count++) + return; + + for_each_hw_engine(hwe, gt, id) + __whitelist_oa_regs(hwe, true); +} + +/** + * xe_reg_dewhitelist_oa_regs - dewhitelist oa registers for gt + * @gt: gt to dewhitelist oa registers for + * + * Dewhitelist OA registers by setting RING_FORCE_TO_NONPRIV_DENY + */ +void xe_reg_dewhitelist_oa_regs(struct xe_gt *gt) +{ + struct xe_hw_engine *hwe; + enum xe_hw_engine_id id; + + lockdep_assert_held(>->oa.gt_lock); + xe_assert(gt_to_xe(gt), gt->oa.whitelist_count); + if (--gt->oa.whitelist_count) + return; + + for_each_hw_engine(hwe, gt, id) + __whitelist_oa_regs(hwe, false); } /** diff --git a/drivers/gpu/drm/xe/xe_reg_whitelist.h b/drivers/gpu/drm/xe/xe_reg_whitelist.h index 69b121d377da..f243b6ac60b7 100644 --- a/drivers/gpu/drm/xe/xe_reg_whitelist.h +++ b/drivers/gpu/drm/xe/xe_reg_whitelist.h @@ -9,12 +9,16 @@ #include struct drm_printer; +struct xe_gt; struct xe_hw_engine; struct xe_reg_sr; struct xe_reg_sr_entry; void xe_reg_whitelist_process_engine(struct xe_hw_engine *hwe); +void xe_reg_whitelist_oa_regs(struct xe_gt *gt); +void xe_reg_dewhitelist_oa_regs(struct xe_gt *gt); + void xe_reg_whitelist_print_entry(struct drm_printer *p, unsigned int indent, u32 reg, struct xe_reg_sr_entry *entry); diff --git a/drivers/gpu/drm/xe/xe_sched_job.c b/drivers/gpu/drm/xe/xe_sched_job.c index eeccc1c318ae..f586a815f1cf 100644 --- a/drivers/gpu/drm/xe/xe_sched_job.c +++ b/drivers/gpu/drm/xe/xe_sched_job.c @@ -113,7 +113,8 @@ struct xe_sched_job *xe_sched_job_create(struct xe_exec_queue *q, kref_init(&job->refcount); xe_exec_queue_get(job->q); - err = drm_sched_job_init(&job->drm, q->entity, 1, NULL); + err = drm_sched_job_init(&job->drm, q->entity, 1, NULL, + q->xef ? q->xef->drm->client_id : 0); if (err) goto err_free; diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c index 7ce6a9d4e5c1..deb4d0314db3 100644 --- a/drivers/gpu/drm/xe/xe_vm.c +++ b/drivers/gpu/drm/xe/xe_vm.c @@ -418,7 +418,6 @@ int xe_vm_validate_rebind(struct xe_vm *vm, struct drm_exec *exec, unsigned int num_fences) { struct drm_gem_object *obj; - unsigned long index; int ret; do { @@ -431,7 +430,7 @@ int xe_vm_validate_rebind(struct xe_vm *vm, struct drm_exec *exec, return ret; } while (!list_empty(&vm->gpuvm.evict.list)); - drm_exec_for_each_locked_object(exec, index, obj) { + drm_exec_for_each_locked_object(exec, obj) { ret = dma_resv_reserve_fences(obj->resv, num_fences); if (ret) return ret; diff --git a/drivers/hid/hid-logitech-dj.c b/drivers/hid/hid-logitech-dj.c index 3b5412541c92..7895f27c9135 100644 --- a/drivers/hid/hid-logitech-dj.c +++ b/drivers/hid/hid-logitech-dj.c @@ -1749,7 +1749,8 @@ static int logi_dj_raw_event(struct hid_device *hdev, static int logi_dj_probe(struct hid_device *hdev, const struct hid_device_id *id) { - struct hid_report_enum *rep_enum; + struct hid_report_enum *input_report_enum; + struct hid_report_enum *output_report_enum; struct hid_report *rep; struct dj_receiver_dev *djrcv_dev; struct usb_interface *intf; @@ -1793,10 +1794,25 @@ static int logi_dj_probe(struct hid_device *hdev, } } - rep_enum = &hdev->report_enum[HID_INPUT_REPORT]; + output_report_enum = &hdev->report_enum[HID_OUTPUT_REPORT]; + rep = output_report_enum->report_id_hash[REPORT_ID_DJ_SHORT]; + + if (rep && rep->maxfield < 1) { + hid_err(hdev, "Expected size of DJ short report is %d, but got 0", + DJREPORT_SHORT_LENGTH - 1); + return -EINVAL; + } + + if (rep && rep->field[0]->report_count != DJREPORT_SHORT_LENGTH - 1) { + hid_err(hdev, "Expected size of DJ short report is %d, but got %d", + DJREPORT_SHORT_LENGTH - 1, rep->field[0]->report_count); + return -EINVAL; + } + + input_report_enum = &hdev->report_enum[HID_INPUT_REPORT]; /* no input reports, bail out */ - if (list_empty(&rep_enum->report_list)) + if (list_empty(&input_report_enum->report_list)) return -ENODEV; /* @@ -1804,7 +1820,7 @@ static int logi_dj_probe(struct hid_device *hdev, * Note: we should theoretically check for HID++ and DJ * collections, but this will do. */ - list_for_each_entry(rep, &rep_enum->report_list, list) { + list_for_each_entry(rep, &input_report_enum->report_list, list) { if (rep->application == 0xff000001) has_hidpp = true; } @@ -1817,7 +1833,7 @@ static int logi_dj_probe(struct hid_device *hdev, return -ENODEV; /* get the current application attached to the node */ - rep = list_first_entry(&rep_enum->report_list, struct hid_report, list); + rep = list_first_entry(&input_report_enum->report_list, struct hid_report, list); djrcv_dev = dj_get_receiver_dev(hdev, id->driver_data, rep->application, has_hidpp); if (!djrcv_dev) { @@ -1825,7 +1841,7 @@ static int logi_dj_probe(struct hid_device *hdev, return -ENOMEM; } - if (!rep_enum->numbered) + if (!input_report_enum->numbered) djrcv_dev->unnumbered_application = rep->application; /* Starts the usb device and connects to upper interfaces hiddev and diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c index 83f3215bc5bd..db1c71832f2a 100644 --- a/drivers/hv/vmbus_drv.c +++ b/drivers/hv/vmbus_drv.c @@ -1320,8 +1320,19 @@ static void vmbus_isr(void) if (IS_ENABLED(CONFIG_PREEMPT_RT)) { vmbus_irqd_wake(); } else { - lockdep_hardirq_threaded(); + static DEFINE_WAIT_OVERRIDE_MAP(vmbus_map, LD_WAIT_CONFIG); + + /* + * vmbus_isr is never force-threaded and always invoked at hard + * IRQ level. __vmbus_isr() below can acquire a spinlock_t + * which becomes a sleeping lock and must not be acquired in + * this context. Therefore on PREEMPT_RT this will be threaded + * via vmbus_irqd_wake(). On non-PREEMPT the annotation lets + * lockdep know that acquiring a spinlock_t is not an issue. + */ + lock_map_acquire_try(&vmbus_map); __vmbus_isr(); + lock_map_release(&vmbus_map); } } diff --git a/drivers/hwmon/Kconfig b/drivers/hwmon/Kconfig index acd0f86cb9b3..848b9cfcb1d8 100644 --- a/drivers/hwmon/Kconfig +++ b/drivers/hwmon/Kconfig @@ -2170,11 +2170,12 @@ config SENSORS_INA2XX select REGMAP_I2C help If you say yes here you get support for INA219, INA220, INA226, - INA230, and INA231 power monitor chips. + INA230, INA231, INA234, INA260, and SY24655 power monitor chips. The INA2xx driver is configured for the default configuration of the part as described in the datasheet. - Default value for Rshunt is 10 mOhms. + Default value for Rshunt is 10 mOhms except for INA260 which has an + internal 2 mOhm shunt resistor. This driver can also be built as a module. If so, the module will be called ina2xx. diff --git a/drivers/hwmon/adt7470.c b/drivers/hwmon/adt7470.c index dbee6926fa05..0fad4bfe53df 100644 --- a/drivers/hwmon/adt7470.c +++ b/drivers/hwmon/adt7470.c @@ -70,8 +70,8 @@ static const unsigned short normal_i2c[] = { 0x2C, 0x2E, 0x2F, I2C_CLIENT_END }; #define ADT7470_PWM1_AUTO_MASK 0x80 #define ADT7470_PWM_AUTO_MASK 0xC0 #define ADT7470_REG_PWM34_CFG 0x69 -#define ADT7470_PWM3_AUTO_MASK 0x40 -#define ADT7470_PWM4_AUTO_MASK 0x80 +#define ADT7470_PWM4_AUTO_MASK 0x40 +#define ADT7470_PWM3_AUTO_MASK 0x80 #define ADT7470_REG_PWM_MIN_BASE_ADDR 0x6A #define ADT7470_REG_PWM_MIN_MAX_ADDR 0x6D #define ADT7470_REG_PWM_TEMP_MIN_BASE_ADDR 0x6E @@ -110,6 +110,21 @@ static const unsigned short normal_i2c[] = { 0x2C, 0x2E, 0x2F, I2C_CLIENT_END }; #define ALARM2(x) ((x) << 8) +/* TEMP1..TEMP7 (ch 0..6) are, respectively BIT(0)..BIT(6) of reg 0x41 and + * 0x72, or BIT(0)..BIT(6) of data->alarm. + * TEMP8..TEMP9 (ch 7..9) are, respectively BIT(0)..BIT(2) of reg 0x42 and + * 0x73, or BIT(8)..BIT(10) of data->alarm. + */ +#define TEMP_ALARM_BIT(ch) ({ \ + typeof(ch) _ch = (ch); \ + (1 << (_ch < 7 ? _ch : _ch + 1)); \ +}) + +/* FAN1..FAN4 (ch 0..3) are respectively BIT(4)..BIT(7) in + * reg 0x42 and 0x73 or BIT(12)..BIT(15) in data->alarm. + */ +#define FAN_ALARM_BIT(ch) (1 << (12 + (ch))) + #define ADT7470_VENDOR 0x41 #define ADT7470_DEVICE 0x70 /* datasheet only mentions a revision 2 */ @@ -167,6 +182,7 @@ struct adt7470_data { u8 pwm_min[ADT7470_PWM_COUNT]; s8 pwm_tmin[ADT7470_PWM_COUNT]; u8 pwm_auto_temp[ADT7470_PWM_COUNT]; + u32 pwm_freq; struct task_struct *auto_update; unsigned int auto_update_interval; @@ -205,11 +221,12 @@ static inline int adt7470_write_word_data(struct adt7470_data *data, unsigned in /* Probe for temperature sensors. Assumes lock is held */ static int adt7470_read_temperatures(struct adt7470_data *data) { - unsigned long res; + struct device *dev = regmap_get_device(data->regmap); + u8 pwm[ADT7470_FAN_COUNT]; unsigned int pwm_cfg[2]; - int err; + unsigned long res; + int err, err2; int i; - u8 pwm[ADT7470_FAN_COUNT]; /* save pwm[1-4] config register */ err = regmap_read(data->regmap, ADT7470_REG_PWM_CFG(0), &pwm_cfg[0]); @@ -233,19 +250,19 @@ static int adt7470_read_temperatures(struct adt7470_data *data) err = regmap_update_bits(data->regmap, ADT7470_REG_PWM_CFG(2), ADT7470_PWM_AUTO_MASK, 0); if (err < 0) - return err; + goto out_restore; /* write pwm control to whatever it was */ err = regmap_bulk_write(data->regmap, ADT7470_REG_PWM(0), &pwm[0], ADT7470_PWM_COUNT); if (err < 0) - return err; + goto out_restore; /* start reading temperature sensors */ err = regmap_update_bits(data->regmap, ADT7470_REG_CFG, ADT7470_T05_STB_MASK, ADT7470_T05_STB_MASK); if (err < 0) - return err; + goto out_restore; /* Delay is 200ms * number of temp sensors. */ res = msleep_interruptible((data->num_temp_sensors >= 0 ? @@ -256,13 +273,30 @@ static int adt7470_read_temperatures(struct adt7470_data *data) err = regmap_update_bits(data->regmap, ADT7470_REG_CFG, ADT7470_T05_STB_MASK, 0); if (err < 0) - return err; + goto out_restore; +out_restore: /* restore pwm[1-4] config registers */ - err = regmap_write(data->regmap, ADT7470_REG_PWM_CFG(0), pwm_cfg[0]); - if (err < 0) - return err; - err = regmap_write(data->regmap, ADT7470_REG_PWM_CFG(2), pwm_cfg[1]); + err2 = regmap_write(data->regmap, ADT7470_REG_PWM_CFG(0), pwm_cfg[0]); + if (err2 < 0) { + dev_warn_ratelimited(dev, + "failed to restore PWM{1,2} config (%d)\n", + err2); + + if (!err) + err = err2; + } + + err2 = regmap_write(data->regmap, ADT7470_REG_PWM_CFG(2), pwm_cfg[1]); + if (err2 < 0) { + dev_warn_ratelimited(dev, + "failed to restore PWM{3,4} config (%d)\n", + err2); + + if (!err) + err = err2; + } + if (err < 0) return err; @@ -491,7 +525,7 @@ static ssize_t auto_update_interval_store(struct device *dev, if (kstrtol(buf, 10, &temp)) return -EINVAL; - temp = clamp_val(temp, 0, 60000); + temp = clamp_val(temp, 500, 60000); mutex_lock(&data->lock); data->auto_update_interval = temp; @@ -551,7 +585,7 @@ static int adt7470_temp_read(struct device *dev, u32 attr, int channel, long *va *val = 1000 * data->temp_max[channel]; break; case hwmon_temp_alarm: - *val = !!(data->alarm & channel); + *val = !!(data->alarm & TEMP_ALARM_BIT(channel)); break; default: return -EOPNOTSUPP; @@ -571,14 +605,16 @@ static int adt7470_temp_write(struct device *dev, u32 attr, int channel, long va switch (attr) { case hwmon_temp_min: mutex_lock(&data->lock); - data->temp_min[channel] = val; err = regmap_write(data->regmap, ADT7470_TEMP_MIN_REG(channel), val); + if (!err) + data->temp_min[channel] = val; mutex_unlock(&data->lock); break; case hwmon_temp_max: mutex_lock(&data->lock); - data->temp_max[channel] = val; err = regmap_write(data->regmap, ADT7470_TEMP_MAX_REG(channel), val); + if (!err) + data->temp_max[channel] = val; mutex_unlock(&data->lock); break; default: @@ -624,36 +660,33 @@ static ssize_t alarm_mask_store(struct device *dev, static int adt7470_fan_read(struct device *dev, u32 attr, int channel, long *val) { struct adt7470_data *data = adt7470_update_device(dev); + u16 fan_data; if (IS_ERR(data)) return PTR_ERR(data); switch (attr) { case hwmon_fan_input: - if (FAN_DATA_VALID(data->fan[channel])) - *val = FAN_PERIOD_TO_RPM(data->fan[channel]); - else - *val = 0; + fan_data = READ_ONCE(data->fan[channel]); break; case hwmon_fan_min: - if (FAN_DATA_VALID(data->fan_min[channel])) - *val = FAN_PERIOD_TO_RPM(data->fan_min[channel]); - else - *val = 0; + fan_data = READ_ONCE(data->fan_min[channel]); break; case hwmon_fan_max: - if (FAN_DATA_VALID(data->fan_max[channel])) - *val = FAN_PERIOD_TO_RPM(data->fan_max[channel]); - else - *val = 0; + fan_data = READ_ONCE(data->fan_max[channel]); break; case hwmon_fan_alarm: - *val = !!(data->alarm & (1 << (12 + channel))); - break; + *val = !!(data->alarm & FAN_ALARM_BIT(channel)); + return 0; default: return -EOPNOTSUPP; } + if (FAN_DATA_VALID(fan_data)) + *val = FAN_PERIOD_TO_RPM(fan_data); + else + *val = 0; + return 0; } @@ -721,7 +754,7 @@ static ssize_t force_pwm_max_store(struct device *dev, } /* These are the valid PWM frequencies to the nearest Hz */ -static const int adt7470_freq_map[] = { +static const u32 adt7470_freq_map[] = { 11, 15, 22, 29, 35, 44, 59, 88, 1400, 22500 }; @@ -761,7 +794,7 @@ static int adt7470_pwm_read(struct device *dev, u32 attr, int channel, long *val *val = 1 + data->pwm_automatic[channel]; break; case hwmon_pwm_freq: - *val = pwm1_freq_get(dev); + *val = data->pwm_freq; break; default: return -EOPNOTSUPP; @@ -774,12 +807,14 @@ static int pwm1_freq_set(struct device *dev, long freq) { struct adt7470_data *data = dev_get_drvdata(dev); unsigned int low_freq = ADT7470_CFG_LF; + u32 closest_freq; int index; int err; /* Round the user value given to the closest available frequency */ index = find_closest(freq, adt7470_freq_map, ARRAY_SIZE(adt7470_freq_map)); + closest_freq = adt7470_freq_map[index]; if (index >= 8) { index -= 8; @@ -797,6 +832,10 @@ static int pwm1_freq_set(struct device *dev, long freq) err = regmap_update_bits(data->regmap, ADT7470_REG_CFG_2, ADT7470_FREQ_MASK, index << ADT7470_FREQ_SHIFT); + if (err < 0) + goto out; + + data->pwm_freq = closest_freq; out: mutex_unlock(&data->lock); @@ -813,9 +852,10 @@ static int adt7470_pwm_write(struct device *dev, u32 attr, int channel, long val case hwmon_pwm_input: val = clamp_val(val, 0, 255); mutex_lock(&data->lock); - data->pwm[channel] = val; err = regmap_write(data->regmap, ADT7470_REG_PWM(channel), - data->pwm[channel]); + val); + if (!err) + data->pwm[channel] = val; mutex_unlock(&data->lock); break; case hwmon_pwm_enable: @@ -829,10 +869,11 @@ static int adt7470_pwm_write(struct device *dev, u32 attr, int channel, long val val--; mutex_lock(&data->lock); - data->pwm_automatic[channel] = val; err = regmap_update_bits(data->regmap, ADT7470_REG_PWM_CFG(channel), pwm_auto_reg_mask, val ? pwm_auto_reg_mask : 0); + if (!err) + data->pwm_automatic[channel] = val; mutex_unlock(&data->lock); break; case hwmon_pwm_freq: @@ -1008,8 +1049,10 @@ static ssize_t pwm_auto_temp_store(struct device *dev, if (temp < 0) return temp; + if (temp > 0xF) + return -EINVAL; + mutex_lock(&data->lock); - data->pwm_automatic[attr->index] = temp; if (!(attr->index % 2)) { mask = 0xF0; @@ -1020,6 +1063,9 @@ static ssize_t pwm_auto_temp_store(struct device *dev, } err = regmap_update_bits(data->regmap, pwm_auto_reg, mask, val); + if (!err) + data->pwm_auto_temp[attr->index] = temp; + mutex_unlock(&data->lock); return err < 0 ? err : count; @@ -1248,6 +1294,7 @@ static int adt7470_probe(struct i2c_client *client) struct device *dev = &client->dev; struct adt7470_data *data; struct device *hwmon_dev; + int freq_val; int err; data = devm_kzalloc(dev, sizeof(struct adt7470_data), GFP_KERNEL); @@ -1272,6 +1319,14 @@ static int adt7470_probe(struct i2c_client *client) if (err < 0) return err; + freq_val = pwm1_freq_get(dev); + if (freq_val <= 0) { + err = freq_val < 0 ? freq_val : -EINVAL; + return err; + } + + data->pwm_freq = (u32)freq_val; + /* Register sysfs hooks */ hwmon_dev = devm_hwmon_device_register_with_info(dev, client->name, data, &adt7470_chip_info, diff --git a/drivers/hwmon/ina2xx.c b/drivers/hwmon/ina2xx.c index f0fa6d073627..c504bb43a2ca 100644 --- a/drivers/hwmon/ina2xx.c +++ b/drivers/hwmon/ina2xx.c @@ -31,6 +31,7 @@ #include #include #include +#include #include #include #include @@ -51,17 +52,26 @@ #define INA226_ALERT_LIMIT 0x07 #define INA226_DIE_ID 0xFF -#define INA2XX_MAX_REGISTERS 8 +/* SY24655 register definitions */ +#define SY24655_EIN 0x0A +#define SY24655_ACCUM_CONFIG 0x0D +#define INA2XX_MAX_REGISTERS 0x0D /* settings - depend on use case */ #define INA219_CONFIG_DEFAULT 0x399F /* PGA=8 */ #define INA226_CONFIG_DEFAULT 0x4527 /* averages=16 */ +#define INA260_CONFIG_DEFAULT 0x6527 /* averages=16 */ +#define SY24655_CONFIG_DEFAULT 0x4527 /* averages=16 */ + +/* (only for sy24655) */ +#define SY24655_ACCUM_CONFIG_DEFAULT 0x044C /* continuous mode, clear after read*/ /* worst case is 68.10 ms (~14.6Hz, ina219) */ #define INA2XX_CONVERSION_RATE 15 #define INA2XX_MAX_DELAY 69 /* worst case delay in ms */ #define INA2XX_RSHUNT_DEFAULT 10000 +#define INA260_RSHUNT 2000 /* bit mask for reading the averaging setting in the configuration register */ #define INA226_AVG_RD_MASK GENMASK(11, 9) @@ -95,6 +105,7 @@ static bool ina2xx_writeable_reg(struct device *dev, unsigned int reg) case INA2XX_CALIBRATION: case INA226_MASK_ENABLE: case INA226_ALERT_LIMIT: + case SY24655_ACCUM_CONFIG: return true; default: return false; @@ -125,15 +136,27 @@ static const struct regmap_config ina2xx_regmap_config = { .writeable_reg = ina2xx_writeable_reg, }; -enum ina2xx_ids { ina219, ina226 }; +enum ina2xx_ids { + ina219, + ina226, + ina234, + ina260, + sy24655 +}; struct ina2xx_config { u16 config_default; + bool has_alerts; /* chip supports alerts and limits */ + bool has_ishunt; /* chip has internal shunt resistor */ + bool has_power_average; /* chip supports average power */ + bool has_update_interval; int calibration_value; int shunt_div; + int shunt_voltage_shift; int bus_voltage_shift; int bus_voltage_lsb; /* uV */ int power_lsb_factor; + int current_shift; }; struct ina2xx_data { @@ -145,6 +168,7 @@ struct ina2xx_data { long power_lsb_uW; struct mutex config_lock; struct regmap *regmap; + struct i2c_client *client; }; static const struct ina2xx_config ina2xx_config[] = { @@ -152,17 +176,70 @@ static const struct ina2xx_config ina2xx_config[] = { .config_default = INA219_CONFIG_DEFAULT, .calibration_value = 4096, .shunt_div = 100, + .shunt_voltage_shift = 0, .bus_voltage_shift = 3, .bus_voltage_lsb = 4000, .power_lsb_factor = 20, + .has_alerts = false, + .has_ishunt = false, + .has_power_average = false, + .current_shift = 0, + .has_update_interval = false, }, [ina226] = { .config_default = INA226_CONFIG_DEFAULT, .calibration_value = 2048, .shunt_div = 400, + .shunt_voltage_shift = 0, .bus_voltage_shift = 0, .bus_voltage_lsb = 1250, .power_lsb_factor = 25, + .has_alerts = true, + .has_ishunt = false, + .has_power_average = false, + .current_shift = 0, + .has_update_interval = true, + }, + [ina234] = { + .config_default = INA226_CONFIG_DEFAULT, + .calibration_value = 2048, + .shunt_div = 25, /* 2.5 µV/LSB raw ADC reading from INA2XX_SHUNT_VOLTAGE */ + .shunt_voltage_shift = 4, + .bus_voltage_shift = 4, + .bus_voltage_lsb = 25600, + .power_lsb_factor = 32, + .has_alerts = true, + .has_ishunt = false, + .has_power_average = false, + .current_shift = 4, + .has_update_interval = true, + }, + [ina260] = { + .config_default = INA260_CONFIG_DEFAULT, + .shunt_div = 400, + .shunt_voltage_shift = 0, + .bus_voltage_shift = 0, + .bus_voltage_lsb = 1250, + .power_lsb_factor = 8, + .has_alerts = true, + .has_ishunt = true, + .has_power_average = false, + .current_shift = 0, + .has_update_interval = true, + }, + [sy24655] = { + .config_default = SY24655_CONFIG_DEFAULT, + .calibration_value = 4096, + .shunt_div = 400, + .shunt_voltage_shift = 0, + .bus_voltage_shift = 0, + .bus_voltage_lsb = 1250, + .power_lsb_factor = 25, + .has_alerts = true, + .has_ishunt = false, + .has_power_average = true, + .current_shift = 0, + .has_update_interval = false, }, }; @@ -207,28 +284,34 @@ static u16 ina226_interval_to_reg(long interval) return FIELD_PREP(INA226_AVG_RD_MASK, avg_bits); } -static int ina2xx_get_value(struct ina2xx_data *data, u8 reg, - unsigned int regval) +static long ina2xx_get_value(struct ina2xx_data *data, u8 reg, + unsigned int regval) { - int val; + s64 val64; + long val; switch (reg) { case INA2XX_SHUNT_VOLTAGE: /* signed register */ - val = DIV_ROUND_CLOSEST((s16)regval, data->config->shunt_div); + val = DIV_ROUND_CLOSEST((s16)regval >> data->config->shunt_voltage_shift, + data->config->shunt_div); break; case INA2XX_BUS_VOLTAGE: - val = (regval >> data->config->bus_voltage_shift) * - data->config->bus_voltage_lsb; - val = DIV_ROUND_CLOSEST(val, 1000); + val = DIV_ROUND_CLOSEST((regval >> data->config->bus_voltage_shift) * + data->config->bus_voltage_lsb, 1000); break; case INA2XX_POWER: - val = regval * data->power_lsb_uW; + val = min_t(u64, (u64)regval * data->power_lsb_uW, LONG_MAX); break; case INA2XX_CURRENT: /* signed register, result in mA */ - val = (s16)regval * data->current_lsb_uA; - val = DIV_ROUND_CLOSEST(val, 1000); + val64 = (s64)((s16)regval >> data->config->current_shift) * + data->current_lsb_uA; + if (val64 < 0) + val64 = -DIV_ROUND_CLOSEST_ULL(-val64, 1000); + else + val64 = DIV_ROUND_CLOSEST_ULL(val64, 1000); + val = clamp_val(val64, LONG_MIN, LONG_MAX); break; case INA2XX_CALIBRATION: val = regval; @@ -254,6 +337,15 @@ static int ina2xx_read_init(struct device *dev, int reg, long *val) unsigned int regval; int ret, retry; + if (data->config->has_ishunt) { + /* No calibration needed */ + ret = regmap_read(regmap, reg, ®val); + if (ret < 0) + return ret; + *val = ina2xx_get_value(data, reg, regval); + return 0; + } + for (retry = 5; retry; retry--) { ret = regmap_read(regmap, reg, ®val); if (ret < 0) @@ -308,25 +400,29 @@ static int ina2xx_read_init(struct device *dev, int reg, long *val) */ static u16 ina226_alert_to_reg(struct ina2xx_data *data, int reg, long val) { + long limit; + switch (reg) { case INA2XX_SHUNT_VOLTAGE: - val = clamp_val(val, 0, SHRT_MAX * data->config->shunt_div); - val *= data->config->shunt_div; - return clamp_val(val, 0, SHRT_MAX); + val = min_t(long, val, DIV_ROUND_CLOSEST(SHRT_MAX, data->config->shunt_div)); + return min_t(long, (val * data->config->shunt_div) << data->config->shunt_voltage_shift, + SHRT_MAX); case INA2XX_BUS_VOLTAGE: - val = clamp_val(val, 0, 200000); - val = (val * 1000) << data->config->bus_voltage_shift; - val = DIV_ROUND_CLOSEST(val, data->config->bus_voltage_lsb); - return clamp_val(val, 0, USHRT_MAX); + val = min_t(long, val, 130000); + return min_t(long, + DIV_ROUND_CLOSEST((val * 1000) << data->config->bus_voltage_shift, + data->config->bus_voltage_lsb), + USHRT_MAX); case INA2XX_POWER: - val = clamp_val(val, 0, UINT_MAX - data->power_lsb_uW); - val = DIV_ROUND_CLOSEST(val, data->power_lsb_uW); - return clamp_val(val, 0, USHRT_MAX); + val = min_t(long, val, LONG_MAX - data->power_lsb_uW); + return min_t(long, DIV_ROUND_CLOSEST(val, data->power_lsb_uW), USHRT_MAX); case INA2XX_CURRENT: - val = clamp_val(val, INT_MIN / 1000, INT_MAX / 1000); + limit = (LONG_MAX - (data->current_lsb_uA / 2)) / 1000; + val = min_t(long, val, limit); /* signed register, result in mA */ val = DIV_ROUND_CLOSEST(val * 1000, data->current_lsb_uA); - return clamp_val(val, SHRT_MIN, SHRT_MAX); + limit = SHRT_MAX >> data->config->current_shift; + return (u16)(min_t(long, val, limit) << data->config->current_shift); default: /* programmer goofed */ WARN_ON_ONCE(1); @@ -459,6 +555,43 @@ static int ina2xx_in_read(struct device *dev, u32 attr, int channel, long *val) return 0; } +/* + * Configuring the READ_EIN (bit 10) of the ACCUM_CONFIG register to 1 + * can clear accumulator and sample_count after reading the EIN register. + * This way, the average power between the last read and the current + * read can be obtained. By combining with accurate time data from + * outside, the energy consumption during that period can be calculated. + */ +static int sy24655_average_power_read(struct ina2xx_data *data, u8 reg, long *val) +{ + u8 template[6]; + int ret; + long accumulator_24, sample_count; + u64 val64; + + /* 48-bit register read */ + ret = i2c_smbus_read_i2c_block_data(data->client, reg, 6, template); + if (ret < 0) + return ret; + if (ret != 6) + return -EIO; + accumulator_24 = ((template[3] << 16) | + (template[4] << 8) | + template[5]); + sample_count = ((template[0] << 16) | + (template[1] << 8) | + template[2]); + if (sample_count <= 0) { + *val = 0; + return 0; + } + + val64 = (u64)DIV_ROUND_CLOSEST(accumulator_24, sample_count) * data->power_lsb_uW; + *val = min_t(u64, val64, LONG_MAX); + + return 0; +} + static int ina2xx_power_read(struct device *dev, u32 attr, long *val) { struct ina2xx_data *data = dev_get_drvdata(dev); @@ -466,6 +599,8 @@ static int ina2xx_power_read(struct device *dev, u32 attr, long *val) switch (attr) { case hwmon_power_input: return ina2xx_read_init(dev, INA2XX_POWER, val); + case hwmon_power_average: + return sy24655_average_power_read(data, SY24655_EIN, val); case hwmon_power_crit: return ina226_alert_limit_read(data, INA226_POWER_OVER_LIMIT_MASK, INA2XX_POWER, val); @@ -624,7 +759,9 @@ static umode_t ina2xx_is_visible(const void *_data, enum hwmon_sensor_types type u32 attr, int channel) { const struct ina2xx_data *data = _data; - enum ina2xx_ids chip = data->chip; + bool has_alerts = data->config->has_alerts; + bool has_power_average = data->config->has_power_average; + bool has_update_interval = data->config->has_update_interval; switch (type) { case hwmon_in: @@ -633,12 +770,12 @@ static umode_t ina2xx_is_visible(const void *_data, enum hwmon_sensor_types type return 0444; case hwmon_in_lcrit: case hwmon_in_crit: - if (chip == ina226) + if (has_alerts) return 0644; break; case hwmon_in_lcrit_alarm: case hwmon_in_crit_alarm: - if (chip == ina226) + if (has_alerts) return 0444; break; default: @@ -651,12 +788,12 @@ static umode_t ina2xx_is_visible(const void *_data, enum hwmon_sensor_types type return 0444; case hwmon_curr_lcrit: case hwmon_curr_crit: - if (chip == ina226) + if (has_alerts) return 0644; break; case hwmon_curr_lcrit_alarm: case hwmon_curr_crit_alarm: - if (chip == ina226) + if (has_alerts) return 0444; break; default: @@ -668,11 +805,15 @@ static umode_t ina2xx_is_visible(const void *_data, enum hwmon_sensor_types type case hwmon_power_input: return 0444; case hwmon_power_crit: - if (chip == ina226) + if (has_alerts) return 0644; break; case hwmon_power_crit_alarm: - if (chip == ina226) + if (has_alerts) + return 0444; + break; + case hwmon_power_average: + if (has_power_average) return 0444; break; default: @@ -682,7 +823,7 @@ static umode_t ina2xx_is_visible(const void *_data, enum hwmon_sensor_types type case hwmon_chip: switch (attr) { case hwmon_chip_update_interval: - if (chip == ina226) + if (has_update_interval) return 0644; break; default: @@ -707,7 +848,8 @@ static const struct hwmon_channel_info * const ina2xx_info[] = { HWMON_CHANNEL_INFO(curr, HWMON_C_INPUT | HWMON_C_CRIT | HWMON_C_CRIT_ALARM | HWMON_C_LCRIT | HWMON_C_LCRIT_ALARM), HWMON_CHANNEL_INFO(power, - HWMON_P_INPUT | HWMON_P_CRIT | HWMON_P_CRIT_ALARM), + HWMON_P_INPUT | HWMON_P_CRIT | HWMON_P_CRIT_ALARM | + HWMON_P_AVERAGE), NULL }; @@ -791,7 +933,9 @@ static int ina2xx_init(struct device *dev, struct ina2xx_data *data) u32 shunt; int ret; - if (device_property_read_u32(dev, "shunt-resistor", &shunt) < 0) + if (data->config->has_ishunt) + shunt = INA260_RSHUNT; + else if (device_property_read_u32(dev, "shunt-resistor", &shunt) < 0) shunt = INA2XX_RSHUNT_DEFAULT; ret = ina2xx_set_shunt(data, shunt); @@ -802,7 +946,7 @@ static int ina2xx_init(struct device *dev, struct ina2xx_data *data) if (ret < 0) return ret; - if (data->chip == ina226) { + if (data->config->has_alerts) { bool active_high = device_property_read_bool(dev, "ti,alert-polarity-active-high"); regmap_update_bits(regmap, INA226_MASK_ENABLE, @@ -810,6 +954,22 @@ static int ina2xx_init(struct device *dev, struct ina2xx_data *data) INA226_ALERT_LATCH_ENABLE | FIELD_PREP(INA226_ALERT_POLARITY, active_high)); } + if (data->config->has_power_average) { + if (data->chip == sy24655) { + /* + * Initialize the power accumulation method to continuous + * mode and clear the EIN register after each read of the + * EIN register + */ + ret = regmap_write(regmap, SY24655_ACCUM_CONFIG, + SY24655_ACCUM_CONFIG_DEFAULT); + if (ret < 0) + return ret; + } + } + + if (data->config->has_ishunt) + return 0; /* * Calibration register is set to the best value, which eliminates @@ -836,6 +996,7 @@ static int ina2xx_probe(struct i2c_client *client) return -ENOMEM; /* set the device type */ + data->client = client; data->config = &ina2xx_config[chip]; data->chip = chip; mutex_init(&data->config_lock); @@ -856,7 +1017,8 @@ static int ina2xx_probe(struct i2c_client *client) hwmon_dev = devm_hwmon_device_register_with_info(dev, client->name, data, &ina2xx_chip_info, - ina2xx_groups); + data->config->has_ishunt ? + NULL : ina2xx_groups); if (IS_ERR(hwmon_dev)) return PTR_ERR(hwmon_dev); @@ -872,11 +1034,18 @@ static const struct i2c_device_id ina2xx_id[] = { { "ina226", ina226 }, { "ina230", ina226 }, { "ina231", ina226 }, + { "ina234", ina234 }, + { "ina260", ina260 }, + { "sy24655", sy24655 }, { } }; MODULE_DEVICE_TABLE(i2c, ina2xx_id); static const struct of_device_id __maybe_unused ina2xx_of_match[] = { + { + .compatible = "silergy,sy24655", + .data = (void *)sy24655 + }, { .compatible = "ti,ina219", .data = (void *)ina219 @@ -897,7 +1066,15 @@ static const struct of_device_id __maybe_unused ina2xx_of_match[] = { .compatible = "ti,ina231", .data = (void *)ina226 }, - { }, + { + .compatible = "ti,ina234", + .data = (void *)ina234 + }, + { + .compatible = "ti,ina260", + .data = (void *)ina260 + }, + { } }; MODULE_DEVICE_TABLE(of, ina2xx_of_match); diff --git a/drivers/hwmon/lm90.c b/drivers/hwmon/lm90.c index 511d95a0efb3..45d5a2427dcb 100644 --- a/drivers/hwmon/lm90.c +++ b/drivers/hwmon/lm90.c @@ -1149,7 +1149,7 @@ static int lm90_update_alarms_locked(struct lm90_data *data, bool force) check_enable = (client->irq || !(data->config_orig & 0x80)) && (data->config & 0x80); - if (force || check_enable) + if (data->hwmon_dev && (force || check_enable)) schedule_work(&data->report_work); /* @@ -1157,7 +1157,7 @@ static int lm90_update_alarms_locked(struct lm90_data *data, bool force) * alarms are all clear, and alerts are currently disabled. * Otherwise (re)schedule worker if needed. */ - if (check_enable) { + if (check_enable && data->hwmon_dev) { if (!(data->current_alarms & data->alert_alarms)) { dev_dbg(&client->dev, "Re-enabling ALERT#\n"); lm90_update_confreg(data, data->config & ~0x80); diff --git a/drivers/hwmon/ltc4282.c b/drivers/hwmon/ltc4282.c index d98c57918ce3..e1c0d02b564b 100644 --- a/drivers/hwmon/ltc4282.c +++ b/drivers/hwmon/ltc4282.c @@ -382,8 +382,8 @@ static int ltc4282_read_in(struct ltc4282_state *st, u32 attr, long *val, channel, val); case hwmon_in_min_alarm: if (channel == LTC4282_CHAN_VGPIO) - ltc4282_read_alarm(st, LTC4282_ADC_ALERT_LOG, - LTC4282_GPIO_ALARM_L_MASK, val); + return ltc4282_read_alarm(st, LTC4282_ADC_ALERT_LOG, + LTC4282_GPIO_ALARM_L_MASK, val); return ltc4282_vdd_source_read_alm(st, LTC4282_VSOURCE_ALARM_L_MASK, diff --git a/drivers/hwmon/nct6775-core.c b/drivers/hwmon/nct6775-core.c index 79bc67ffb998..ce2cf1f22900 100644 --- a/drivers/hwmon/nct6775-core.c +++ b/drivers/hwmon/nct6775-core.c @@ -791,12 +791,12 @@ static const u16 NCT6106_REG_TOLERANCE_H[] = { 0x112, 0x122, 0x132 }; static const u16 NCT6106_REG_TARGET[] = { 0x111, 0x121, 0x131 }; -static const u16 NCT6106_REG_WEIGHT_TEMP_SEL[] = { 0x168, 0x178, 0x188 }; -static const u16 NCT6106_REG_WEIGHT_TEMP_STEP[] = { 0x169, 0x179, 0x189 }; -static const u16 NCT6106_REG_WEIGHT_TEMP_STEP_TOL[] = { 0x16a, 0x17a, 0x18a }; -static const u16 NCT6106_REG_WEIGHT_DUTY_STEP[] = { 0x16b, 0x17b, 0x18b }; -static const u16 NCT6106_REG_WEIGHT_TEMP_BASE[] = { 0x16c, 0x17c, 0x18c }; -static const u16 NCT6106_REG_WEIGHT_DUTY_BASE[] = { 0x16d, 0x17d, 0x18d }; +static const u16 NCT6106_REG_WEIGHT_TEMP_SEL[] = { 0x168, 0x178, 0x188, 0, 0 }; +static const u16 NCT6106_REG_WEIGHT_TEMP_STEP[] = { 0x169, 0x179, 0x189, 0, 0 }; +static const u16 NCT6106_REG_WEIGHT_TEMP_STEP_TOL[] = { 0x16a, 0x17a, 0x18a, 0, 0 }; +static const u16 NCT6106_REG_WEIGHT_DUTY_STEP[] = { 0x16b, 0x17b, 0x18b, 0, 0 }; +static const u16 NCT6106_REG_WEIGHT_TEMP_BASE[] = { 0x16c, 0x17c, 0x18c, 0, 0 }; +static const u16 NCT6106_REG_WEIGHT_DUTY_BASE[] = { 0x16d, 0x17d, 0x18d, 0, 0 }; static const u16 NCT6106_REG_AUTO_TEMP[] = { 0x160, 0x170, 0x180 }; static const u16 NCT6106_REG_AUTO_PWM[] = { 0x164, 0x174, 0x184 }; @@ -846,8 +846,6 @@ static const u16 NCT6116_FAN_PULSE_SHIFT[] = { 0, 2, 4, 6, 6 }; static const u16 NCT6116_REG_PWM[] = { 0x119, 0x129, 0x139, 0x199, 0x1a9 }; static const u16 NCT6116_REG_FAN_MODE[] = { 0x113, 0x123, 0x133, 0x193, 0x1a3 }; static const u16 NCT6116_REG_TEMP_SEL[] = { 0x110, 0x120, 0x130, 0x190, 0x1a0 }; -static const u16 NCT6116_REG_TEMP_SOURCE[] = { - 0xb0, 0xb1, 0xb2 }; static const u16 NCT6116_REG_CRITICAL_TEMP[] = { 0x11a, 0x12a, 0x13a, 0x19a, 0x1aa }; @@ -3650,7 +3648,7 @@ int nct6775_probe(struct device *dev, struct nct6775_data *data, = NCT6106_CRITICAL_PWM_ENABLE_MASK; data->REG_CRITICAL_PWM = NCT6116_REG_CRITICAL_PWM; data->REG_TEMP_OFFSET = NCT6106_REG_TEMP_OFFSET; - data->REG_TEMP_SOURCE = NCT6116_REG_TEMP_SOURCE; + data->REG_TEMP_SOURCE = NCT6106_REG_TEMP_SOURCE; data->REG_TEMP_SEL = NCT6116_REG_TEMP_SEL; data->REG_WEIGHT_TEMP_SEL = NCT6106_REG_WEIGHT_TEMP_SEL; data->REG_WEIGHT_TEMP[0] = NCT6106_REG_WEIGHT_TEMP_STEP; @@ -3664,13 +3662,13 @@ int nct6775_probe(struct device *dev, struct nct6775_data *data, reg_temp = NCT6106_REG_TEMP; reg_temp_mon = NCT6106_REG_TEMP_MON; - num_reg_temp = ARRAY_SIZE(NCT6106_REG_TEMP); + num_reg_temp = 3; num_reg_temp_mon = ARRAY_SIZE(NCT6106_REG_TEMP_MON); num_reg_tsi_temp = ARRAY_SIZE(NCT6116_REG_TSI_TEMP); reg_temp_over = NCT6106_REG_TEMP_OVER; reg_temp_hyst = NCT6106_REG_TEMP_HYST; reg_temp_config = NCT6106_REG_TEMP_CONFIG; - num_reg_temp_config = ARRAY_SIZE(NCT6106_REG_TEMP_CONFIG); + num_reg_temp_config = 3; reg_temp_alternate = NCT6106_REG_TEMP_ALTERNATE; reg_temp_crit = NCT6106_REG_TEMP_CRIT; reg_temp_crit_l = NCT6106_REG_TEMP_CRIT_L; diff --git a/drivers/hwmon/npcm750-pwm-fan.c b/drivers/hwmon/npcm750-pwm-fan.c index db3b551828eb..84d67157c99a 100644 --- a/drivers/hwmon/npcm750-pwm-fan.c +++ b/drivers/hwmon/npcm750-pwm-fan.c @@ -364,6 +364,11 @@ static void npcm7xx_fan_polling(struct timer_list *t) add_timer(&data->fan_timer); } +static void npcm7xx_fan_cleanup(void *timer) +{ + timer_shutdown_sync(timer); +} + static inline void npcm7xx_fan_compute(struct npcm7xx_pwm_fan_data *data, u8 fan, u8 cmp, u8 fan_id, u8 flag_int, u8 flag_mode, u8 flag_clear) @@ -1027,6 +1032,12 @@ static int npcm7xx_pwm_fan_probe(struct platform_device *pdev) msecs_to_jiffies(NPCM7XX_FAN_POLL_TIMER_200MS); timer_setup(&data->fan_timer, npcm7xx_fan_polling, 0); + ret = devm_add_action_or_reset(dev, + npcm7xx_fan_cleanup, + &data->fan_timer); + if (ret) + return ret; + add_timer(&data->fan_timer); break; } diff --git a/drivers/hwmon/nzxt-smart2.c b/drivers/hwmon/nzxt-smart2.c index 6f8febda4277..66c5886f411a 100644 --- a/drivers/hwmon/nzxt-smart2.c +++ b/drivers/hwmon/nzxt-smart2.c @@ -203,7 +203,7 @@ struct drvdata { */ struct mutex mutex; long update_interval; - u8 output_buffer[OUTPUT_REPORT_SIZE]; + u8 output_buffer[OUTPUT_REPORT_SIZE] __aligned(ARCH_DMA_MINALIGN); }; static long scale_pwm_value(long val, long orig_max, long new_max) diff --git a/drivers/hwmon/pmbus/pmbus_core.c b/drivers/hwmon/pmbus/pmbus_core.c index a431cce33603..8804e1d61faa 100644 --- a/drivers/hwmon/pmbus/pmbus_core.c +++ b/drivers/hwmon/pmbus/pmbus_core.c @@ -508,7 +508,7 @@ int pmbus_update_byte_data(struct i2c_client *client, int page, u8 reg, if (tmp != rv) rv = _pmbus_write_byte_data(client, page, reg, tmp); - return rv; + return rv < 0 ? rv : 0; } EXPORT_SYMBOL_NS_GPL(pmbus_update_byte_data, PMBUS); @@ -2930,8 +2930,9 @@ static void pmbus_notify(struct pmbus_data *data, int page, int reg, int flags) if (reg == sreg && page == spage && (smask & flags)) { dev_dbg(data->dev, "sysfs notify: %s", da->attr.name); - sysfs_notify(&data->dev->kobj, NULL, da->attr.name); - kobject_uevent(&data->dev->kobj, KOBJ_CHANGE); + sysfs_notify(&data->hwmon_dev->kobj, NULL, + da->attr.name); + kobject_uevent(&data->hwmon_dev->kobj, KOBJ_CHANGE); flags &= ~smask; } diff --git a/drivers/hwmon/sht3x.c b/drivers/hwmon/sht3x.c index 94466e28dc56..1f63c07cfb08 100644 --- a/drivers/hwmon/sht3x.c +++ b/drivers/hwmon/sht3x.c @@ -21,6 +21,7 @@ #include #include #include +#include /* commands (high repeatability mode) */ static const unsigned char sht3x_cmd_measure_single_hpm[] = { 0x24, 0x00 }; @@ -279,9 +280,9 @@ static struct sht3x_data *sht3x_update_client(struct device *dev) if (ret) goto out; - val = be16_to_cpup((__be16 *)buf); + val = get_unaligned_be16(buf); data->temperature = sht3x_extract_temperature(val); - val = be16_to_cpup((__be16 *)(buf + 3)); + val = get_unaligned_be16(buf + 3); data->humidity = sht3x_extract_humidity(val); data->last_update = jiffies; } @@ -339,7 +340,7 @@ static int limits_update(struct sht3x_data *data) if (ret) return ret; - raw = be16_to_cpup((__be16 *)buffer); + raw = get_unaligned_be16(buffer); temperature = sht3x_extract_temperature((raw & 0x01ff) << 7); humidity = sht3x_extract_humidity(raw & 0xfe00); data->temperature_limits[index] = temperature; @@ -392,7 +393,7 @@ static size_t limit_write(struct device *dev, raw = ((u32)(temperature + 45000) * 24543) >> (16 + 7); raw |= ((humidity * 42950) >> 16) & 0xfe00; - *((__be16 *)position) = cpu_to_be16(raw); + put_unaligned_be16(raw, position); position += SHT3X_WORD_LEN; *position = crc8(sht3x_crc8_table, position - SHT3X_WORD_LEN, diff --git a/drivers/i2c/busses/i2c-amd-mp2-plat.c b/drivers/i2c/busses/i2c-amd-mp2-plat.c index 6f0ef587e76d..c5bddc722248 100644 --- a/drivers/i2c/busses/i2c-amd-mp2-plat.c +++ b/drivers/i2c/busses/i2c-amd-mp2-plat.c @@ -316,8 +316,10 @@ static int i2c_amd_probe(struct platform_device *pdev) amd_mp2_pm_runtime_put(mp2_dev); - if (ret < 0) + if (ret < 0) { dev_err(&pdev->dev, "i2c add adapter failed = %d\n", ret); + amd_mp2_unregister_cb(&i2c_dev->common); + } return ret; } diff --git a/drivers/i2c/busses/i2c-bcm-iproc.c b/drivers/i2c/busses/i2c-bcm-iproc.c index 133d02899c6b..907f66cbf232 100644 --- a/drivers/i2c/busses/i2c-bcm-iproc.c +++ b/drivers/i2c/busses/i2c-bcm-iproc.c @@ -811,6 +811,17 @@ static int bcm_iproc_i2c_xfer_wait(struct bcm_iproc_i2c_dev *iproc_i2c, } if (!time_left && !iproc_i2c->xfer_is_done) { + /* + * The controller may fail to clear START_BUSY after a timeout, + * reset the controller to recover in that case. + */ + if (!!(iproc_i2c_rd_reg(iproc_i2c, M_CMD_OFFSET) & + BIT(M_CMD_START_BUSY_SHIFT))) { + bcm_iproc_i2c_enable_disable(iproc_i2c, false); + bcm_iproc_i2c_init(iproc_i2c); + bcm_iproc_i2c_enable_disable(iproc_i2c, true); + } + /* flush both TX/RX FIFOs */ val = BIT(M_FIFO_RX_FLUSH_SHIFT) | BIT(M_FIFO_TX_FLUSH_SHIFT); iproc_i2c_wr_reg(iproc_i2c, M_FIFO_CTRL_OFFSET, val); diff --git a/drivers/i2c/busses/i2c-imx.c b/drivers/i2c/busses/i2c-imx.c index 45e71120fdc1..2f98ac4ad214 100644 --- a/drivers/i2c/busses/i2c-imx.c +++ b/drivers/i2c/busses/i2c-imx.c @@ -866,9 +866,6 @@ static int i2c_imx_reg_slave(struct i2c_client *client) if (i2c_imx->slave) return -EBUSY; - i2c_imx->slave = client; - i2c_imx->last_slave_event = I2C_SLAVE_STOP; - /* Resume */ ret = pm_runtime_resume_and_get(i2c_imx->adapter.dev.parent); if (ret < 0) { @@ -876,6 +873,11 @@ static int i2c_imx_reg_slave(struct i2c_client *client) return ret; } + scoped_guard(spinlock_irqsave, &i2c_imx->slave_lock) { + i2c_imx->slave = client; + i2c_imx->last_slave_event = I2C_SLAVE_STOP; + } + i2c_imx_slave_init(i2c_imx); return 0; @@ -894,6 +896,7 @@ static int i2c_imx_unreg_slave(struct i2c_client *client) i2c_imx_reset_regs(i2c_imx); + hrtimer_cancel(&i2c_imx->slave_timer); i2c_imx->slave = NULL; /* Suspend */ diff --git a/drivers/i2c/busses/i2c-jz4780.c b/drivers/i2c/busses/i2c-jz4780.c index 92cc5b091137..1bf25f575fbe 100644 --- a/drivers/i2c/busses/i2c-jz4780.c +++ b/drivers/i2c/busses/i2c-jz4780.c @@ -141,6 +141,7 @@ struct jz4780_i2c { void __iomem *iomem; int irq; struct clk *clk; + unsigned long clk_rate_khz; struct i2c_adapter adap; const struct ingenic_i2c_config *cdata; @@ -246,7 +247,7 @@ static int jz4780_i2c_set_target(struct jz4780_i2c *i2c, unsigned char address) static int jz4780_i2c_set_speed(struct jz4780_i2c *i2c) { - int dev_clk_khz = clk_get_rate(i2c->clk) / 1000; + int dev_clk_khz = i2c->clk_rate_khz; int cnt_high = 0; /* HIGH period count of the SCL clock */ int cnt_low = 0; /* LOW period count of the SCL clock */ int cnt_period = 0; /* period count of the SCL clock */ @@ -796,6 +797,8 @@ static int jz4780_i2c_probe(struct platform_device *pdev) if (IS_ERR(i2c->clk)) return PTR_ERR(i2c->clk); + i2c->clk_rate_khz = clk_get_rate(i2c->clk) / 1000; + ret = of_property_read_u32(pdev->dev.of_node, "clock-frequency", &clk_freq); if (ret) { diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index 16c6adff3eb7..5b5400efb657 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -6,6 +6,7 @@ */ #include +#include #include #include #include @@ -673,6 +674,11 @@ static struct iova_magazine *iova_depot_pop(struct iova_rcache *rcache) { struct iova_magazine *mag = rcache->depot; + /* + * As the mag->next pointer is moved to rcache->depot and reset via + * the mag->size assignment, mark it as a transient false positive. + */ + kmemleak_transient_leak(mag->next); rcache->depot = mag->next; mag->size = IOVA_MAG_SIZE; rcache->depot_size--; diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c index d057c74f7172..169040479c23 100644 --- a/drivers/media/i2c/imx219.c +++ b/drivers/media/i2c/imx219.c @@ -71,9 +71,8 @@ #define IMX219_EXPOSURE_MAX 65535 /* V_TIMING internal */ -#define IMX219_REG_VTS CCI_REG16(0x0160) -#define IMX219_VTS_MAX 0xffff - +#define IMX219_REG_FRM_LENGTH_A CCI_REG16(0x0160) +#define IMX219_FLL_MAX 0xfffe #define IMX219_VBLANK_MIN 32 /* HBLANK control - read only */ @@ -156,7 +155,7 @@ struct imx219_mode { unsigned int height; /* V-timing */ - unsigned int vts_def; + unsigned int fll_def; }; static const struct cci_reg_sequence imx219_common_regs[] = { @@ -315,25 +314,25 @@ static const struct imx219_mode supported_modes[] = { /* 8MPix 15fps mode */ .width = 3280, .height = 2464, - .vts_def = 3526, + .fll_def = 3526, }, { /* 1080P 30fps cropped */ .width = 1920, .height = 1080, - .vts_def = 1763, + .fll_def = 1763, }, { /* 2x2 binned 30fps mode */ .width = 1640, .height = 1232, - .vts_def = 1763, + .fll_def = 1763, }, { /* 640x480 30fps mode */ .width = 640, .height = 480, - .vts_def = 1763, + .fll_def = 1763, }, }; @@ -444,7 +443,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl) imx219->hflip->val | imx219->vflip->val << 1, &ret); break; case V4L2_CID_VBLANK: - cci_write(imx219->regmap, IMX219_REG_VTS, + cci_write(imx219->regmap, IMX219_REG_FRM_LENGTH_A, format->height + ctrl->val, &ret); break; case V4L2_CID_TEST_PATTERN_RED: @@ -519,15 +518,15 @@ static int imx219_init_controls(struct imx219 *imx219) /* Initial vblank/hblank/exposure parameters based on current mode */ imx219->vblank = v4l2_ctrl_new_std(ctrl_hdlr, &imx219_ctrl_ops, V4L2_CID_VBLANK, IMX219_VBLANK_MIN, - IMX219_VTS_MAX - mode->height, 1, - mode->vts_def - mode->height); + IMX219_FLL_MAX - mode->height, 1, + mode->fll_def - mode->height); hblank = IMX219_PPL_DEFAULT - mode->width; imx219->hblank = v4l2_ctrl_new_std(ctrl_hdlr, &imx219_ctrl_ops, V4L2_CID_HBLANK, hblank, hblank, 1, hblank); if (imx219->hblank) imx219->hblank->flags |= V4L2_CTRL_FLAG_READ_ONLY; - exposure_max = mode->vts_def - 4; + exposure_max = mode->fll_def - 4; exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ? exposure_max : IMX219_EXPOSURE_DEFAULT; imx219->exposure = v4l2_ctrl_new_std(ctrl_hdlr, &imx219_ctrl_ops, @@ -878,12 +877,12 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd, /* Update limits and set FPS to default */ __v4l2_ctrl_modify_range(imx219->vblank, IMX219_VBLANK_MIN, - IMX219_VTS_MAX - mode->height, 1, - mode->vts_def - mode->height); + IMX219_FLL_MAX - mode->height, 1, + mode->fll_def - mode->height); __v4l2_ctrl_s_ctrl(imx219->vblank, - mode->vts_def - mode->height); + mode->fll_def - mode->height); /* Update max exposure while meeting expected vblanking */ - exposure_max = mode->vts_def - 4; + exposure_max = mode->fll_def - 4; exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ? exposure_max : IMX219_EXPOSURE_DEFAULT; __v4l2_ctrl_modify_range(imx219->exposure, diff --git a/drivers/media/platform/chips-media/wave5/wave5-hw.c b/drivers/media/platform/chips-media/wave5/wave5-hw.c index 710311d85113..1daac56dd1d4 100644 --- a/drivers/media/platform/chips-media/wave5/wave5-hw.c +++ b/drivers/media/platform/chips-media/wave5/wave5-hw.c @@ -1753,6 +1753,9 @@ int wave5_vpu_enc_init_seq(struct vpu_instance *inst) (p_param->skip_intra_trans << 25) | (p_param->strong_intra_smooth_enable << 27) | (p_param->en_still_picture << 30); + else if (inst->std == W_AVC_ENC) + reg_val |= (p_param->constraint_set1_flag << 29); + vpu_write_reg(inst->dev, W5_CMD_ENC_SEQ_SPS_PARAM, reg_val); reg_val = (p_param->lossless_enable) | diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c index b7761d8a7d0a..0e92eca18236 100644 --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c @@ -906,6 +906,8 @@ static int wave5_vpu_enc_s_ctrl(struct v4l2_ctrl *ctrl) case V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE: inst->enc_param.profile = H264_PROFILE_BP; inst->bit_depth = 8; + if (ctrl->val == V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) + inst->enc_param.constraint_set1_flag = 1; break; case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN: inst->enc_param.profile = H264_PROFILE_MP; @@ -1168,6 +1170,7 @@ static void wave5_set_enc_openparam(struct enc_open_param *open_param, open_param->wave_param.intra_period = input.avc_idr_period; } } else { + open_param->wave_param.constraint_set1_flag = input.constraint_set1_flag; open_param->wave_param.avc_idr_period = input.avc_idr_period; } open_param->wave_param.entropy_coding_mode = input.entropy_coding_mode; @@ -1631,7 +1634,7 @@ static int wave5_vpu_open_enc(struct file *filp) -6, 6, 1, 0); v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave5_vpu_enc_ctrl_ops, V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM, - 0, 1, 1, 1); + 0, 1, 1, 0); v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave5_vpu_enc_ctrl_ops, V4L2_CID_MPEG_VIDEO_H264_CONSTRAINED_INTRA_PREDICTION, 0, 1, 1, 0); diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h index d2370511faf8..703b2413dafa 100644 --- a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h +++ b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h @@ -568,6 +568,7 @@ struct enc_wave_param { u32 lambda_scaling_enable: 1; /* enable lambda scaling using custom GOP */ u32 transform8x8_enable: 1; /* enable 8x8 intra prediction and 8x8 transform */ u32 mb_level_rc_enable: 1; /* enable MB-level rate control */ + u32 constraint_set1_flag: 1; /* enable CBP */ }; struct enc_open_param { diff --git a/drivers/net/can/c_can/c_can_main.c b/drivers/net/can/c_can/c_can_main.c index cc371d0c9f3c..2ea5224ebfcb 100644 --- a/drivers/net/can/c_can/c_can_main.c +++ b/drivers/net/can/c_can/c_can_main.c @@ -597,20 +597,20 @@ static int c_can_chip_config(struct net_device *dev) return err; /* enable automatic retransmission */ - priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_ENABLE_AR); + priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_ENABLE_AR | CONTROL_INIT); if ((priv->can.ctrlmode & CAN_CTRLMODE_LISTENONLY) && (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK)) { /* loopback + silent mode : useful for hot self-test */ - priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST); + priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST | CONTROL_INIT); priv->write_reg(priv, C_CAN_TEST_REG, TEST_LBACK | TEST_SILENT); } else if (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK) { /* loopback mode : useful for self-test function */ - priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST); + priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST | CONTROL_INIT); priv->write_reg(priv, C_CAN_TEST_REG, TEST_LBACK); } else if (priv->can.ctrlmode & CAN_CTRLMODE_LISTENONLY) { /* silent mode : bus-monitoring mode */ - priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST); + priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST | CONTROL_INIT); priv->write_reg(priv, C_CAN_TEST_REG, TEST_SILENT); } diff --git a/drivers/net/can/ctucanfd/ctucanfd_base.c b/drivers/net/can/ctucanfd/ctucanfd_base.c index 0d40564febee..31fc0b8573bb 100644 --- a/drivers/net/can/ctucanfd/ctucanfd_base.c +++ b/drivers/net/can/ctucanfd/ctucanfd_base.c @@ -340,8 +340,8 @@ static void ctucan_set_mode(struct ctucan_priv *priv, const struct can_ctrlmode (mode_reg & ~REG_MODE_FDE); mode_reg = (mode->flags & CAN_CTRLMODE_PRESUME_ACK) ? - (mode_reg | REG_MODE_ACF) : - (mode_reg & ~REG_MODE_ACF); + (mode_reg | REG_MODE_STM) : + (mode_reg & ~REG_MODE_STM); mode_reg = (mode->flags & CAN_CTRLMODE_FD_NON_ISO) ? (mode_reg | REG_MODE_NISOFD) : @@ -868,7 +868,7 @@ static void ctucan_err_interrupt(struct net_device *ndev, u32 isr) break; case CAN_STATE_ERROR_ACTIVE: if (skb) { - cf->can_id |= CAN_ERR_CNT; + cf->can_id |= CAN_ERR_CRTL | CAN_ERR_CNT; cf->data[1] = CAN_ERR_CRTL_ACTIVE; cf->data[6] = bec.txerr; cf->data[7] = bec.rxerr; @@ -1135,8 +1135,12 @@ static irqreturn_t ctucan_interrupt(int irq, void *dev_id) /* Error interrupts */ if (FIELD_GET(REG_INT_STAT_EWLI, isr) || FIELD_GET(REG_INT_STAT_FCSI, isr) || - FIELD_GET(REG_INT_STAT_ALI, isr)) { - icr = isr & (REG_INT_STAT_EWLI | REG_INT_STAT_FCSI | REG_INT_STAT_ALI); + FIELD_GET(REG_INT_STAT_ALI, isr) || + FIELD_GET(REG_INT_STAT_BEI, isr)) { + icr = isr & (REG_INT_STAT_EWLI | + REG_INT_STAT_FCSI | + REG_INT_STAT_ALI | + REG_INT_STAT_BEI); ctucan_netdev_dbg(ndev, "some ERR interrupt: clearing 0x%08x\n", icr); ctucan_write32(priv, CTUCANFD_INT_STAT, icr); diff --git a/drivers/net/can/ctucanfd/ctucanfd_pci.c b/drivers/net/can/ctucanfd/ctucanfd_pci.c index 9da09e7dd63a..8cc995649287 100644 --- a/drivers/net/can/ctucanfd/ctucanfd_pci.c +++ b/drivers/net/can/ctucanfd/ctucanfd_pci.c @@ -202,7 +202,7 @@ static int ctucan_pci_probe(struct pci_dev *pdev, pci_set_drvdata(pdev, NULL); kfree(bdata); err_pci_iounmap_bar0: - pci_iounmap(pdev, cra_addr); + pci_iounmap(pdev, bar0_base); err_pci_iounmap_bar1: pci_iounmap(pdev, addr); err_release_regions: @@ -274,6 +274,7 @@ static const struct pci_device_id ctucan_pci_tbl[] = { CTUCAN_WITH_CTUCAN_ID)}, {}, }; +MODULE_DEVICE_TABLE(pci, ctucan_pci_tbl); static struct pci_driver ctucan_pci_driver = { .name = KBUILD_MODNAME, diff --git a/drivers/net/can/dev/skb.c b/drivers/net/can/dev/skb.c index 3ebd4f779b9b..0da615afa04d 100644 --- a/drivers/net/can/dev/skb.c +++ b/drivers/net/can/dev/skb.c @@ -202,7 +202,6 @@ static void init_can_skb_reserve(struct sk_buff *skb) skb_reset_transport_header(skb); can_skb_reserve(skb); - can_skb_prv(skb)->skbcnt = 0; } struct sk_buff *alloc_can_skb(struct net_device *dev, struct can_frame **cf) @@ -312,7 +311,6 @@ static bool can_skb_headroom_valid(struct net_device *dev, struct sk_buff *skb) if (skb->ip_summed == CHECKSUM_NONE) { /* init headroom */ can_skb_prv(skb)->ifindex = dev->ifindex; - can_skb_prv(skb)->skbcnt = 0; skb->ip_summed = CHECKSUM_UNNECESSARY; diff --git a/drivers/net/can/softing/softing_fw.c b/drivers/net/can/softing/softing_fw.c index 721df91cdbfb..282570daf3ef 100644 --- a/drivers/net/can/softing/softing_fw.c +++ b/drivers/net/can/softing/softing_fw.c @@ -91,12 +91,12 @@ int softing_bootloader_command(struct softing *card, int16_t cmd, return ret; } -static int fw_parse(const uint8_t **pmem, uint16_t *ptype, uint32_t *paddr, - uint16_t *plen, const uint8_t **pdat) +static int fw_parse(const u8 **pmem, const u8 *limit, u16 *ptype, + u32 *paddr, u16 *plen, const u8 **pdat) { uint16_t checksum[2]; - const uint8_t *mem; - const uint8_t *end; + const u8 *mem; + const u8 *record_end; /* * firmware records are a binary, unaligned stream composed of: @@ -114,14 +114,21 @@ static int fw_parse(const uint8_t **pmem, uint16_t *ptype, uint32_t *paddr, * endianness & alignment. */ mem = *pmem; + /* A record needs an 8-byte prefix and a 2-byte checksum. */ + if (mem > limit || limit - mem < 10) + return -EINVAL; + *ptype = le16_to_cpup((void *)&mem[0]); *paddr = le32_to_cpup((void *)&mem[2]); *plen = le16_to_cpup((void *)&mem[6]); + if (*plen > limit - mem - 10) + return -EINVAL; + *pdat = &mem[8]; /* verify checksum */ - end = &mem[8 + *plen]; - checksum[0] = le16_to_cpup((void *)end); - for (checksum[1] = 0; mem < end; ++mem) + record_end = &mem[8 + *plen]; + checksum[0] = le16_to_cpup((void *)record_end); + for (checksum[1] = 0; mem < record_end; ++mem) checksum[1] += *mem; if (checksum[0] != checksum[1]) return -EINVAL; @@ -139,6 +146,7 @@ int softing_load_fw(const char *file, struct softing *card, uint16_t type, len; uint32_t addr; uint8_t *buf = NULL, *new_buf; + s64 dpram_offset; int buflen = 0; int8_t type_end = 0; @@ -153,7 +161,7 @@ int softing_load_fw(const char *file, struct softing *card, mem = fw->data; end = &mem[fw->size]; /* look for header record */ - ret = fw_parse(&mem, &type, &addr, &len, &dat); + ret = fw_parse(&mem, end, &type, &addr, &len, &dat); if (ret < 0) goto failed; if (type != 0xffff) @@ -164,7 +172,7 @@ int softing_load_fw(const char *file, struct softing *card, } /* ok, we had a header */ while (mem < end) { - ret = fw_parse(&mem, &type, &addr, &len, &dat); + ret = fw_parse(&mem, end, &type, &addr, &len, &dat); if (ret < 0) goto failed; if (type == 3) { @@ -179,9 +187,13 @@ int softing_load_fw(const char *file, struct softing *card, goto failed; } - if ((addr + len + offset) > size) + dpram_offset = (s64)addr + offset; + if (dpram_offset < 0 || dpram_offset > size || + len > size - dpram_offset) { + ret = -EINVAL; goto failed; - memcpy_toio(&dpram[addr + offset], dat, len); + } + memcpy_toio(&dpram[dpram_offset], dat, len); /* be sure to flush caches from IO space */ mb(); if (len > buflen) { @@ -195,7 +207,7 @@ int softing_load_fw(const char *file, struct softing *card, buf = new_buf; } /* verify record data */ - memcpy_fromio(buf, &dpram[addr + offset], len); + memcpy_fromio(buf, &dpram[dpram_offset], len); if (memcmp(buf, dat, len)) { /* is not ok */ dev_alert(&card->pdev->dev, "DPRAM readback failed\n"); @@ -237,7 +249,7 @@ int softing_load_app_fw(const char *file, struct softing *card) mem = fw->data; end = &mem[fw->size]; /* look for header record */ - ret = fw_parse(&mem, &type, &addr, &len, &dat); + ret = fw_parse(&mem, end, &type, &addr, &len, &dat); if (ret) goto failed; ret = -EINVAL; @@ -253,7 +265,7 @@ int softing_load_app_fw(const char *file, struct softing *card) } /* ok, we had a header */ while (mem < end) { - ret = fw_parse(&mem, &type, &addr, &len, &dat); + ret = fw_parse(&mem, end, &type, &addr, &len, &dat); if (ret) goto failed; @@ -279,6 +291,12 @@ int softing_load_app_fw(const char *file, struct softing *card) /* work in 16bit (target) */ sum &= 0xffff; + if (card->pdat->app.offs > card->dpram_size || + len > card->dpram_size - card->pdat->app.offs) { + ret = -EINVAL; + goto failed; + } + memcpy_toio(&card->dpram[card->pdat->app.offs], dat, len); iowrite32(card->pdat->app.offs + card->pdat->app.addr, &card->dpram[DPRAM_COMMAND + 2]); diff --git a/drivers/net/can/usb/ems_usb.c b/drivers/net/can/usb/ems_usb.c index d8c881130e90..4b130c96c9ab 100644 --- a/drivers/net/can/usb/ems_usb.c +++ b/drivers/net/can/usb/ems_usb.c @@ -409,6 +409,40 @@ static void ems_usb_rx_err(struct ems_usb *dev, struct ems_cpc_msg *msg) netif_rx(skb); } +static bool ems_usb_rx_msg_len_valid(struct ems_cpc_msg *msg) +{ + size_t len = msg->length; + size_t can_len; + + switch (msg->type) { + case CPC_MSG_TYPE_CAN_STATE: + return len >= sizeof(msg->msg.can_state); + + case CPC_MSG_TYPE_CAN_FRAME: + case CPC_MSG_TYPE_EXT_CAN_FRAME: + case CPC_MSG_TYPE_RTR_FRAME: + case CPC_MSG_TYPE_EXT_RTR_FRAME: + if (len < CPC_CAN_MSG_MIN_SIZE) + return false; + + if (msg->type == CPC_MSG_TYPE_RTR_FRAME || + msg->type == CPC_MSG_TYPE_EXT_RTR_FRAME) + return true; + + can_len = can_cc_dlc2len(msg->msg.can_msg.length & 0xf); + return len >= CPC_CAN_MSG_MIN_SIZE + can_len; + + case CPC_MSG_TYPE_CAN_FRAME_ERROR: + return len >= sizeof(msg->msg.error); + + case CPC_MSG_TYPE_OVERRUN: + return len >= sizeof(msg->msg.overrun); + + default: + return true; + } +} + /* * callback for bulk IN urb */ @@ -451,6 +485,15 @@ static void ems_usb_read_bulk_callback(struct urb *urb) } msg = (struct ems_cpc_msg *)&ibuf[start]; + if (msg->length > + urb->actual_length - start - CPC_MSG_HEADER_LEN) { + netdev_err(netdev, "format error\n"); + break; + } + if (!ems_usb_rx_msg_len_valid(msg)) { + netdev_err(netdev, "format error\n"); + break; + } switch (msg->type) { case CPC_MSG_TYPE_CAN_STATE: diff --git a/drivers/net/can/usb/etas_es58x/es58x_core.c b/drivers/net/can/usb/etas_es58x/es58x_core.c index 154588cb7b32..1a17ce02d8b1 100644 --- a/drivers/net/can/usb/etas_es58x/es58x_core.c +++ b/drivers/net/can/usb/etas_es58x/es58x_core.c @@ -1476,7 +1476,6 @@ static void es58x_read_bulk_callback(struct urb *urb) dev_err_ratelimited(dev, "Failed resubmitting read bulk urb: %pe\n", ERR_PTR(ret)); - return; free_urb: usb_free_coherent(urb->dev, urb->transfer_buffer_length, diff --git a/drivers/net/can/usb/gs_usb.c b/drivers/net/can/usb/gs_usb.c index ee3d49a686de..6bc9ab2605c0 100644 --- a/drivers/net/can/usb/gs_usb.c +++ b/drivers/net/can/usb/gs_usb.c @@ -671,7 +671,7 @@ static void gs_usb_receive_bulk_callback(struct urb *urb) if (hf->flags & GS_CAN_FLAG_FD) { skb = alloc_canfd_skb(netdev, &cfd); if (!skb) - return; + goto resubmit_urb; cfd->can_id = le32_to_cpu(hf->can_id); cfd->len = data_length; @@ -684,7 +684,7 @@ static void gs_usb_receive_bulk_callback(struct urb *urb) } else { skb = alloc_can_skb(netdev, &cf); if (!skb) - return; + goto resubmit_urb; cf->can_id = le32_to_cpu(hf->can_id); can_frame_set_cc_len(cf, hf->can_dlc, dev->can.ctrlmode); diff --git a/drivers/net/can/usb/kvaser_usb/kvaser_usb_hydra.c b/drivers/net/can/usb/kvaser_usb/kvaser_usb_hydra.c index 3764b263add3..2792ff7dd06f 100644 --- a/drivers/net/can/usb/kvaser_usb/kvaser_usb_hydra.c +++ b/drivers/net/can/usb/kvaser_usb/kvaser_usb_hydra.c @@ -1625,6 +1625,7 @@ static int kvaser_usb_hydra_get_busparams(struct kvaser_usb_net_priv *priv, reinit_completion(&priv->get_busparams_comp); err = kvaser_usb_send_cmd(dev, cmd, cmd_len); + kfree(cmd); if (err) return err; diff --git a/drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c b/drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c index b7f6935686c9..251d4cd963ae 100644 --- a/drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c +++ b/drivers/net/can/usb/kvaser_usb/kvaser_usb_leaf.c @@ -668,13 +668,22 @@ static int kvaser_usb_leaf_wait_cmd(const struct kvaser_usb *dev, u8 id, continue; } - if (pos + tmp->len > actual_len) { + if (tmp->len < CMD_HEADER_LEN || + tmp->len > actual_len - pos) { dev_err_ratelimited(&dev->intf->dev, "Format error\n"); break; } if (tmp->id == id) { + if (tmp->len > sizeof(*cmd)) { + dev_err_ratelimited(&dev->intf->dev, + "Received command %u too large (%u)\n", + tmp->id, tmp->len); + err = -EIO; + goto end; + } + memcpy(cmd, tmp, tmp->len); goto end; } @@ -1677,7 +1686,7 @@ static void kvaser_usb_leaf_read_bulk_callback(struct kvaser_usb *dev, continue; } - if (pos + cmd->len > len) { + if (cmd->len < CMD_HEADER_LEN || cmd->len > len - pos) { dev_err_ratelimited(&dev->intf->dev, "Format error\n"); break; } diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_core.c b/drivers/net/can/usb/peak_usb/pcan_usb_core.c index 69c44f675ff1..cbde414b023a 100644 --- a/drivers/net/can/usb/peak_usb/pcan_usb_core.c +++ b/drivers/net/can/usb/peak_usb/pcan_usb_core.c @@ -470,7 +470,6 @@ static int peak_usb_start(struct peak_usb_device *dev) netif_device_detach(dev->netdev); usb_unanchor_urb(urb); - kfree(buf); usb_free_urb(urb); break; } diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c index ebefc274b50a..6ebfbc0753d9 100644 --- a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c +++ b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c @@ -565,6 +565,13 @@ static int pcan_usb_fd_decode_canmsg(struct pcan_usb_fd_if *usb_if, dev->can.ctrlmode); } + if (!(rx_msg_flags & PUCAN_MSG_RTR) && + le16_to_cpu(rx_msg->size) - offsetof(struct pucan_rx_msg, d) < + cfd->len) { + kfree_skb(skb); + return -EBADMSG; + } + cfd->can_id = le32_to_cpu(rm->can_id); if (rx_msg_flags & PUCAN_MSG_EXT_ID) @@ -713,6 +720,24 @@ static void pcan_usb_fd_decode_ts(struct pcan_usb_fd_if *usb_if, peak_usb_set_ts_now(&usb_if->time_ref, le32_to_cpu(ts->ts_low)); } +static size_t pcan_usb_fd_rx_msg_min_size(u16 rx_msg_type) +{ + switch (rx_msg_type) { + case PUCAN_MSG_CAN_RX: + return offsetof(struct pucan_rx_msg, d); + case PCAN_UFD_MSG_CALIBRATION: + return sizeof(struct pcan_ufd_ts_msg); + case PUCAN_MSG_ERROR: + return sizeof(struct pucan_error_msg); + case PUCAN_MSG_STATUS: + return sizeof(struct pucan_status_msg); + case PCAN_UFD_MSG_OVERRUN: + return sizeof(struct pcan_ufd_ovr_msg); + default: + return sizeof(struct pucan_msg); + } +} + /* callback for bulk IN urb */ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) { @@ -727,6 +752,12 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) msg_end = urb->transfer_buffer + urb->actual_length; for (; msg_ptr < msg_end;) { u16 rx_msg_type, rx_msg_size; + size_t rx_msg_min_size; + + if (msg_end - msg_ptr < sizeof(*rx_msg)) { + err = -EBADMSG; + break; + } rx_msg = (struct pucan_msg *)msg_ptr; if (!rx_msg->size) { @@ -738,13 +769,20 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) rx_msg_type = le16_to_cpu(rx_msg->type); /* check if the record goes out of current packet */ - if (msg_ptr + rx_msg_size > msg_end) { + if (rx_msg_size > msg_end - msg_ptr) { netdev_err(netdev, "got frag rec: should inc usb rx buf sze\n"); err = -EBADMSG; break; } + rx_msg_min_size = pcan_usb_fd_rx_msg_min_size(rx_msg_type); + if (rx_msg_size < rx_msg_min_size) { + netdev_err(netdev, "got short rec\n"); + err = -EBADMSG; + break; + } + switch (rx_msg_type) { case PUCAN_MSG_CAN_RX: err = pcan_usb_fd_decode_canmsg(usb_if, rx_msg); diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_pro.c b/drivers/net/can/usb/peak_usb/pcan_usb_pro.c index f736196383ac..39c86917cb5e 100644 --- a/drivers/net/can/usb/peak_usb/pcan_usb_pro.c +++ b/drivers/net/can/usb/peak_usb/pcan_usb_pro.c @@ -534,12 +534,18 @@ static int pcan_usb_pro_handle_canmsg(struct pcan_usb_pro_interface *usb_if, struct pcan_usb_pro_rxmsg *rx) { const unsigned int ctrl_idx = (rx->len >> 4) & 0x0f; - struct peak_usb_device *dev = usb_if->dev[ctrl_idx]; - struct net_device *netdev = dev->netdev; + struct peak_usb_device *dev; + struct net_device *netdev; struct can_frame *can_frame; struct sk_buff *skb; struct skb_shared_hwtstamps *hwts; + if (ctrl_idx >= ARRAY_SIZE(usb_if->dev)) + return -EINVAL; + + dev = usb_if->dev[ctrl_idx]; + netdev = dev->netdev; + skb = alloc_can_skb(netdev, &can_frame); if (!skb) return -ENOMEM; @@ -573,14 +579,20 @@ static int pcan_usb_pro_handle_error(struct pcan_usb_pro_interface *usb_if, { const u16 raw_status = le16_to_cpu(er->status); const unsigned int ctrl_idx = (er->channel >> 4) & 0x0f; - struct peak_usb_device *dev = usb_if->dev[ctrl_idx]; - struct net_device *netdev = dev->netdev; + struct peak_usb_device *dev; + struct net_device *netdev; struct can_frame *can_frame; enum can_state new_state = CAN_STATE_ERROR_ACTIVE; u8 err_mask = 0; struct sk_buff *skb; struct skb_shared_hwtstamps *hwts; + if (ctrl_idx >= ARRAY_SIZE(usb_if->dev)) + return -EINVAL; + + dev = usb_if->dev[ctrl_idx]; + netdev = dev->netdev; + /* nothing should be sent while in BUS_OFF state */ if (dev->can.state == CAN_STATE_BUS_OFF) return 0; diff --git a/drivers/net/dsa/mt7530-mdio.c b/drivers/net/dsa/mt7530-mdio.c index 51df42ccdbe6..e19b46449ffe 100644 --- a/drivers/net/dsa/mt7530-mdio.c +++ b/drivers/net/dsa/mt7530-mdio.c @@ -55,8 +55,15 @@ mt7530_regmap_read(void *context, unsigned int reg, unsigned int *val) if (ret < 0) return ret; - lo = bus->read(bus, priv->mdiodev->addr, r); - hi = bus->read(bus, priv->mdiodev->addr, 0x10); + ret = bus->read(bus, priv->mdiodev->addr, r); + if (ret < 0) + return ret; + lo = ret; + + ret = bus->read(bus, priv->mdiodev->addr, 0x10); + if (ret < 0) + return ret; + hi = ret; *val = (hi << 16) | (lo & 0xffff); diff --git a/drivers/net/dsa/mt7530.c b/drivers/net/dsa/mt7530.c index 21437ce57b64..b13e905833c3 100644 --- a/drivers/net/dsa/mt7530.c +++ b/drivers/net/dsa/mt7530.c @@ -219,12 +219,6 @@ mt7530_write(struct mt7530_priv *priv, u32 reg, u32 val) mt7530_mutex_unlock(priv); } -static u32 -_mt7530_unlocked_read(struct mt7530_dummy_poll *p) -{ - return mt7530_mii_read(p->priv, p->reg); -} - static u32 _mt7530_read(struct mt7530_dummy_poll *p) { @@ -577,16 +571,13 @@ static int mt7531_ind_c45_phy_read(struct mt7530_priv *priv, int port, int devad, int regnum) { - struct mt7530_dummy_poll p; u32 reg, val; int ret; - INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); - mt7530_mutex_lock(priv); - ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, - !(val & MT7531_PHY_ACS_ST), 20, 100000); + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, + !(val & MT7531_PHY_ACS_ST), 20, 100000); if (ret < 0) { dev_err(priv->dev, "poll timeout\n"); goto out; @@ -596,8 +587,8 @@ mt7531_ind_c45_phy_read(struct mt7530_priv *priv, int port, int devad, MT7531_MDIO_DEV_ADDR(devad) | regnum; mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); - ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, - !(val & MT7531_PHY_ACS_ST), 20, 100000); + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, + !(val & MT7531_PHY_ACS_ST), 20, 100000); if (ret < 0) { dev_err(priv->dev, "poll timeout\n"); goto out; @@ -607,8 +598,8 @@ mt7531_ind_c45_phy_read(struct mt7530_priv *priv, int port, int devad, MT7531_MDIO_DEV_ADDR(devad); mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); - ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, - !(val & MT7531_PHY_ACS_ST), 20, 100000); + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, + !(val & MT7531_PHY_ACS_ST), 20, 100000); if (ret < 0) { dev_err(priv->dev, "poll timeout\n"); goto out; @@ -625,16 +616,13 @@ static int mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, int regnum, u16 data) { - struct mt7530_dummy_poll p; u32 val, reg; int ret; - INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); - mt7530_mutex_lock(priv); - ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, - !(val & MT7531_PHY_ACS_ST), 20, 100000); + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, + !(val & MT7531_PHY_ACS_ST), 20, 100000); if (ret < 0) { dev_err(priv->dev, "poll timeout\n"); goto out; @@ -644,8 +632,8 @@ mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, MT7531_MDIO_DEV_ADDR(devad) | regnum; mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); - ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, - !(val & MT7531_PHY_ACS_ST), 20, 100000); + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, + !(val & MT7531_PHY_ACS_ST), 20, 100000); if (ret < 0) { dev_err(priv->dev, "poll timeout\n"); goto out; @@ -655,8 +643,8 @@ mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, MT7531_MDIO_DEV_ADDR(devad) | data; mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); - ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, - !(val & MT7531_PHY_ACS_ST), 20, 100000); + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, + !(val & MT7531_PHY_ACS_ST), 20, 100000); if (ret < 0) { dev_err(priv->dev, "poll timeout\n"); goto out; @@ -671,16 +659,13 @@ mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, static int mt7531_ind_c22_phy_read(struct mt7530_priv *priv, int port, int regnum) { - struct mt7530_dummy_poll p; int ret; u32 val; - INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); - mt7530_mutex_lock(priv); - ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, - !(val & MT7531_PHY_ACS_ST), 20, 100000); + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, + !(val & MT7531_PHY_ACS_ST), 20, 100000); if (ret < 0) { dev_err(priv->dev, "poll timeout\n"); goto out; @@ -691,8 +676,8 @@ mt7531_ind_c22_phy_read(struct mt7530_priv *priv, int port, int regnum) mt7530_mii_write(priv, MT7531_PHY_IAC, val | MT7531_PHY_ACS_ST); - ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, - !(val & MT7531_PHY_ACS_ST), 20, 100000); + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, + !(val & MT7531_PHY_ACS_ST), 20, 100000); if (ret < 0) { dev_err(priv->dev, "poll timeout\n"); goto out; @@ -709,16 +694,13 @@ static int mt7531_ind_c22_phy_write(struct mt7530_priv *priv, int port, int regnum, u16 data) { - struct mt7530_dummy_poll p; int ret; u32 reg; - INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); - mt7530_mutex_lock(priv); - ret = readx_poll_timeout(_mt7530_unlocked_read, &p, reg, - !(reg & MT7531_PHY_ACS_ST), 20, 100000); + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, reg, + !(reg & MT7531_PHY_ACS_ST), 20, 100000); if (ret < 0) { dev_err(priv->dev, "poll timeout\n"); goto out; @@ -729,8 +711,8 @@ mt7531_ind_c22_phy_write(struct mt7530_priv *priv, int port, int regnum, mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); - ret = readx_poll_timeout(_mt7530_unlocked_read, &p, reg, - !(reg & MT7531_PHY_ACS_ST), 20, 100000); + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, reg, + !(reg & MT7531_PHY_ACS_ST), 20, 100000); if (ret < 0) { dev_err(priv->dev, "poll timeout\n"); goto out; diff --git a/drivers/net/ethernet/airoha/airoha_eth.c b/drivers/net/ethernet/airoha/airoha_eth.c index 58e5791e9cd4..9ebc9788ded7 100644 --- a/drivers/net/ethernet/airoha/airoha_eth.c +++ b/drivers/net/ethernet/airoha/airoha_eth.c @@ -2022,7 +2022,7 @@ static void airoha_qdma_init_qos_stats(struct airoha_qdma *qdma) FIELD_PREP(CNTR_CHAN_MASK, i)); /* Tx-fwd transferred count */ airoha_qdma_wr(qdma, REG_CNTR_VAL((i << 1) + 1), 0); - airoha_qdma_wr(qdma, REG_CNTR_CFG(i << 1), + airoha_qdma_wr(qdma, REG_CNTR_CFG((i << 1) + 1), CNTR_EN_MASK | CNTR_ALL_QUEUE_EN_MASK | CNTR_ALL_DSCP_RING_EN_MASK | FIELD_PREP(CNTR_SRC_MASK, 1) | diff --git a/drivers/net/ethernet/intel/e1000/e1000_main.c b/drivers/net/ethernet/intel/e1000/e1000_main.c index 8072aa8f05e3..f9be17e64d18 100644 --- a/drivers/net/ethernet/intel/e1000/e1000_main.c +++ b/drivers/net/ethernet/intel/e1000/e1000_main.c @@ -1225,11 +1225,11 @@ static int e1000_probe(struct pci_dev *pdev, const struct pci_device_id *ent) if (hw->flash_address) iounmap(hw->flash_address); +err_mdio_ioremap: kfree(adapter->tx_ring); kfree(adapter->rx_ring); err_dma: err_sw_init: -err_mdio_ioremap: iounmap(hw->ce4100_gbe_mdio_base_virt); iounmap(hw->hw_addr); err_ioremap: diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c index e6e635ad971f..51b35c15827d 100644 --- a/drivers/net/ethernet/intel/ice/ice_ethtool.c +++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c @@ -1067,18 +1067,18 @@ static int ice_lbtest_prepare_rings(struct ice_vsi *vsi) status = ice_vsi_cfg_lan(vsi); if (status) - goto err_setup_rx_ring; + goto err_cfg_lan; status = ice_vsi_start_all_rx_rings(vsi); if (status) - goto err_start_rx_ring; + goto err_cfg_lan; return 0; -err_start_rx_ring: - ice_vsi_free_rx_rings(vsi); -err_setup_rx_ring: +err_cfg_lan: ice_vsi_stop_lan_tx_rings(vsi, ICE_NO_RESET, 0); +err_setup_rx_ring: + ice_vsi_free_rx_rings(vsi); err_setup_tx_ring: ice_vsi_free_tx_rings(vsi); diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index f1c50910b433..5d430ed6ae43 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -5708,6 +5708,16 @@ static int ice_resume(struct device *dev) /* Restart the service task */ mod_timer(&pf->serv_tmr, round_jiffies(jiffies + pf->serv_tmr_period)); + /* Best-effort wait for the scheduled reset to finish so that the + * device is operational before returning. Without this, userspace + * (e.g. NetworkManager) may try to open the net device while the + * asynchronous reset is still in progress, hitting -EBUSY. + */ + ret = ice_wait_for_reset(pf, secs_to_jiffies(10)); + if (ret) + dev_err(dev, "Wait for reset timed out (10s) during resume: %d\n", + ret); + return 0; } diff --git a/drivers/net/ethernet/intel/idpf/idpf_lib.c b/drivers/net/ethernet/intel/idpf/idpf_lib.c index 4973135c346a..c2101f7f6e1b 100644 --- a/drivers/net/ethernet/intel/idpf/idpf_lib.c +++ b/drivers/net/ethernet/intel/idpf/idpf_lib.c @@ -134,7 +134,7 @@ static int idpf_mb_intr_req_irq(struct idpf_adapter *adapter) if (err) { dev_err(&adapter->pdev->dev, "IRQ request for mailbox failed, error: %d\n", err); - + kfree(name); return err; } diff --git a/drivers/net/ethernet/intel/idpf/idpf_txrx.c b/drivers/net/ethernet/intel/idpf/idpf_txrx.c index 82a927265b9c..afb922e8704d 100644 --- a/drivers/net/ethernet/intel/idpf/idpf_txrx.c +++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c @@ -2507,10 +2507,7 @@ static netdev_tx_t idpf_tx_splitq_frame(struct sk_buff *skb, tx_params.dtype = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE; tx_params.eop_cmd = IDPF_TXD_FLEX_FLOW_CMD_EOP; - /* Set the RE bit to periodically "clean" the descriptor ring. - * MIN_GAP is set to MIN_RING size to ensure it will be set at - * least once each time around the ring. - */ + /* Set the RE bit periodically to "clean" the descriptor ring */ if (idpf_tx_splitq_need_re(tx_q)) { tx_params.eop_cmd |= IDPF_TXD_FLEX_FLOW_CMD_RE; tx_q->txq_grp->num_completions_pending++; diff --git a/drivers/net/ethernet/intel/idpf/idpf_txrx.h b/drivers/net/ethernet/intel/idpf/idpf_txrx.h index a34c791c4608..2d9960dd51ad 100644 --- a/drivers/net/ethernet/intel/idpf/idpf_txrx.h +++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h @@ -19,7 +19,7 @@ /* Mailbox Queue */ #define IDPF_MAX_MBXQ 1 -#define IDPF_MIN_TXQ_DESC 64 +#define IDPF_MIN_TXQ_DESC 128 #define IDPF_MIN_RXQ_DESC 64 #define IDPF_MIN_TXQ_COMPLQ_DESC 256 #define IDPF_MAX_QIDS 256 diff --git a/drivers/net/ethernet/intel/igbvf/netdev.c b/drivers/net/ethernet/intel/igbvf/netdev.c index 925d7286a8ee..0bc78a2c78ef 100644 --- a/drivers/net/ethernet/intel/igbvf/netdev.c +++ b/drivers/net/ethernet/intel/igbvf/netdev.c @@ -2201,8 +2201,6 @@ static inline int igbvf_tx_map_adv(struct igbvf_adapter *adapter, buffer_info->time_stamp = 0; buffer_info->length = 0; buffer_info->mapped_as_page = false; - if (count) - count--; /* clear timestamp and dma mappings for remaining portion of packet */ while (count--) { diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c index c314139e0d78..d03399585d01 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -838,8 +838,8 @@ static void otx2_handle_link_event(struct otx2_nic *pf) netif_carrier_on(netdev); netif_tx_start_all_queues(netdev); } else { - netif_tx_stop_all_queues(netdev); netif_carrier_off(netdev); + netif_tx_stop_all_queues(netdev); } } diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.c b/drivers/net/ethernet/mediatek/mtk_eth_soc.c index ebf5432cb328..0d3346baa84e 100644 --- a/drivers/net/ethernet/mediatek/mtk_eth_soc.c +++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.c @@ -3210,6 +3210,37 @@ static void mtk_tx_timeout(struct net_device *dev, unsigned int txqueue) schedule_work(ð->pending_work); } +static int mtk_get_irqs(struct platform_device *pdev, struct mtk_eth *eth) +{ + int i; + + /* future SoCs beginning with MT7988 should use named IRQs in dts */ + eth->irq[MTK_FE_IRQ_TX] = platform_get_irq_byname(pdev, "fe1"); + eth->irq[MTK_FE_IRQ_RX] = platform_get_irq_byname(pdev, "fe2"); + if (eth->irq[MTK_FE_IRQ_TX] >= 0 && eth->irq[MTK_FE_IRQ_RX] >= 0) + return 0; + + /* legacy way: + * On MTK_SHARED_INT SoCs (MT7621 + MT7628) the first IRQ is taken + * from devicetree and used for both RX and TX - it is shared. + * On SoCs with non-shared IRQs the first entry is not used, + * the second is for TX, and the third is for RX. + */ + for (i = 0; i < MTK_FE_IRQ_NUM; i++) { + if (MTK_HAS_CAPS(eth->soc->caps, MTK_SHARED_INT) && i > 0) + eth->irq[i] = eth->irq[MTK_FE_IRQ_SHARED]; + else + eth->irq[i] = platform_get_irq(pdev, i); + + if (eth->irq[i] < 0) { + dev_err(&pdev->dev, "no IRQ%d resource found\n", i); + return -ENXIO; + } + } + + return 0; +} + static irqreturn_t mtk_handle_irq_rx(int irq, void *_eth) { struct mtk_eth *eth = _eth; @@ -3263,7 +3294,7 @@ static void mtk_poll_controller(struct net_device *dev) mtk_tx_irq_disable(eth, MTK_TX_DONE_INT); mtk_rx_irq_disable(eth, eth->soc->rx.irq_done_mask); - mtk_handle_irq_rx(eth->irq[2], dev); + mtk_handle_irq_rx(eth->irq[MTK_FE_IRQ_RX], eth); mtk_tx_irq_enable(eth, MTK_TX_DONE_INT); mtk_rx_irq_enable(eth, eth->soc->rx.irq_done_mask); } @@ -4756,7 +4787,7 @@ static int mtk_add_mac(struct mtk_eth *eth, struct device_node *np) eth->netdev[id]->features |= eth->soc->hw_features; eth->netdev[id]->ethtool_ops = &mtk_ethtool_ops; - eth->netdev[id]->irq = eth->irq[0]; + eth->netdev[id]->irq = eth->irq[MTK_FE_IRQ_SHARED]; eth->netdev[id]->dev.of_node = np; if (MTK_HAS_CAPS(eth->soc->caps, MTK_SOC_MT7628)) @@ -4986,17 +5017,10 @@ static int mtk_probe(struct platform_device *pdev) } } - for (i = 0; i < 3; i++) { - if (MTK_HAS_CAPS(eth->soc->caps, MTK_SHARED_INT) && i > 0) - eth->irq[i] = eth->irq[0]; - else - eth->irq[i] = platform_get_irq(pdev, i); - if (eth->irq[i] < 0) { - dev_err(&pdev->dev, "no IRQ%d resource found\n", i); - err = -ENXIO; - goto err_wed_exit; - } - } + err = mtk_get_irqs(pdev, eth); + if (err) + goto err_wed_exit; + for (i = 0; i < ARRAY_SIZE(eth->clks); i++) { eth->clks[i] = devm_clk_get(eth->dev, mtk_clks_source_name[i]); @@ -5040,17 +5064,17 @@ static int mtk_probe(struct platform_device *pdev) } if (MTK_HAS_CAPS(eth->soc->caps, MTK_SHARED_INT)) { - err = devm_request_irq(eth->dev, eth->irq[0], + err = devm_request_irq(eth->dev, eth->irq[MTK_FE_IRQ_SHARED], mtk_handle_irq, 0, dev_name(eth->dev), eth); } else { - err = devm_request_irq(eth->dev, eth->irq[1], + err = devm_request_irq(eth->dev, eth->irq[MTK_FE_IRQ_TX], mtk_handle_irq_tx, 0, dev_name(eth->dev), eth); if (err) goto err_free_dev; - err = devm_request_irq(eth->dev, eth->irq[2], + err = devm_request_irq(eth->dev, eth->irq[MTK_FE_IRQ_RX], mtk_handle_irq_rx, 0, dev_name(eth->dev), eth); } @@ -5096,7 +5120,7 @@ static int mtk_probe(struct platform_device *pdev) } else netif_info(eth, probe, eth->netdev[i], "mediatek frame engine at 0x%08lx, irq %d\n", - eth->netdev[i]->base_addr, eth->irq[0]); + eth->netdev[i]->base_addr, eth->irq[MTK_FE_IRQ_SHARED]); } /* we run 2 devices on the same DMA ring so we need a dummy device diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.h b/drivers/net/ethernet/mediatek/mtk_eth_soc.h index 0570623e569d..76d60b6a23b2 100644 --- a/drivers/net/ethernet/mediatek/mtk_eth_soc.h +++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.h @@ -611,6 +611,11 @@ #define MTK_MAC_FSM(x) (0x1010C + ((x) * 0x100)) +#define MTK_FE_IRQ_SHARED 0 +#define MTK_FE_IRQ_TX 1 +#define MTK_FE_IRQ_RX 2 +#define MTK_FE_IRQ_NUM (MTK_FE_IRQ_RX + 1) + struct mtk_rx_dma { unsigned int rxd1; unsigned int rxd2; @@ -1248,7 +1253,7 @@ struct mtk_eth { struct net_device *dummy_dev; struct net_device *netdev[MTK_MAX_DEVS]; struct mtk_mac *mac[MTK_MAX_DEVS]; - int irq[3]; + int irq[MTK_FE_IRQ_NUM]; u32 msg_enable; unsigned long sysclk; struct regmap *ethsys; diff --git a/drivers/net/ethernet/nvidia/forcedeth.c b/drivers/net/ethernet/nvidia/forcedeth.c index 720f577929db..11eb3751ad27 100644 --- a/drivers/net/ethernet/nvidia/forcedeth.c +++ b/drivers/net/ethernet/nvidia/forcedeth.c @@ -6199,10 +6199,10 @@ static void nv_remove(struct pci_dev *pci_dev) struct net_device *dev = pci_get_drvdata(pci_dev); struct fe_priv *np = netdev_priv(dev); - free_percpu(np->txrx_stats); - unregister_netdev(dev); + free_percpu(np->txrx_stats); + nv_restore_mac_addr(pci_dev); /* restore any phy related changes */ diff --git a/drivers/net/ethernet/qlogic/qede/qede_main.c b/drivers/net/ethernet/qlogic/qede/qede_main.c index b5d744d2586f..59a80f719372 100644 --- a/drivers/net/ethernet/qlogic/qede/qede_main.c +++ b/drivers/net/ethernet/qlogic/qede/qede_main.c @@ -107,7 +107,7 @@ static void qede_remove(struct pci_dev *pdev); static void qede_shutdown(struct pci_dev *pdev); static void qede_link_update(void *dev, struct qed_link_output *link); static void qede_schedule_recovery_handler(void *dev); -static void qede_recovery_handler(struct qede_dev *edev); +static bool qede_recovery_handler(struct qede_dev *edev); static void qede_schedule_hw_err_handler(void *dev, enum qed_hw_err_type err_type); static void qede_get_eth_tlv_data(void *edev, void *data); @@ -1065,21 +1065,6 @@ void __qede_unlock(struct qede_dev *edev) mutex_unlock(&edev->qede_lock); } -/* This version of the lock should be used when acquiring the RTNL lock is also - * needed in addition to the internal qede lock. - */ -static void qede_lock(struct qede_dev *edev) -{ - rtnl_lock(); - __qede_lock(edev); -} - -static void qede_unlock(struct qede_dev *edev) -{ - __qede_unlock(edev); - rtnl_unlock(); -} - static void qede_periodic_task(struct work_struct *work) { struct qede_dev *edev = container_of(work, struct qede_dev, @@ -1116,6 +1101,8 @@ static void qede_sp_task(struct work_struct *work) */ if (test_and_clear_bit(QEDE_SP_RECOVERY, &edev->sp_flags)) { + bool reloaded; + cancel_delayed_work_sync(&edev->periodic_task); #ifdef CONFIG_QED_SRIOV /* SRIOV must be disabled outside the lock to avoid a deadlock. @@ -1124,9 +1111,17 @@ static void qede_sp_task(struct work_struct *work) if (pci_num_vf(edev->pdev)) qede_sriov_configure(edev->pdev, 0); #endif - qede_lock(edev); - qede_recovery_handler(edev); - qede_unlock(edev); + rtnl_lock(); + __qede_lock(edev); + reloaded = qede_recovery_handler(edev); + __qede_unlock(edev); + + /* The udp_tunnel core synchronously calls back into + * qede_udp_tunnel_sync(), which takes the qede lock. + */ + if (reloaded) + udp_tunnel_nic_reset_ntf(edev->ndev); + rtnl_unlock(); } __qede_lock(edev); @@ -2667,9 +2662,13 @@ static void qede_recovery_failed(struct qede_dev *edev) edev->ops->common->set_power_state(edev->cdev, PCI_D3hot); } -static void qede_recovery_handler(struct qede_dev *edev) +/* Returns true if an open device was successfully reloaded and its + * udp_tunnel ports need to be re-synced by the caller. + */ +static bool qede_recovery_handler(struct qede_dev *edev) { u32 curr_state = edev->state; + bool reloaded = false; int rc; DP_NOTICE(edev, "Starting a recovery process\n"); @@ -2699,17 +2698,18 @@ static void qede_recovery_handler(struct qede_dev *edev) goto err; qede_config_rx_mode(edev->ndev); - udp_tunnel_nic_reset_ntf(edev->ndev); + reloaded = true; } edev->state = curr_state; DP_NOTICE(edev, "Recovery handling is done\n"); - return; + return reloaded; err: qede_recovery_failed(edev); + return false; } static void qede_atomic_hw_err_handler(struct qede_dev *edev) diff --git a/drivers/net/ethernet/realtek/rtase/rtase_main.c b/drivers/net/ethernet/realtek/rtase/rtase_main.c index a565d5fb6b85..afae19cc9931 100644 --- a/drivers/net/ethernet/realtek/rtase/rtase_main.c +++ b/drivers/net/ethernet/realtek/rtase/rtase_main.c @@ -1609,6 +1609,9 @@ static netdev_tx_t rtase_start_xmit(struct sk_buff *skb, err_dma_1: ring->skbuff[entry] = NULL; rtase_tx_clear_range(ring, ring->cur_idx + 1, frags); + if (frags) + /* the frags were cleared above, along with the skb */ + return NETDEV_TX_OK; err_dma_0: tp->stats.tx_dropped++; diff --git a/drivers/net/ethernet/samsung/sxgbe/sxgbe_main.c b/drivers/net/ethernet/samsung/sxgbe/sxgbe_main.c index d2ba283f6123..4afa2b2c67b4 100644 --- a/drivers/net/ethernet/samsung/sxgbe/sxgbe_main.c +++ b/drivers/net/ethernet/samsung/sxgbe/sxgbe_main.c @@ -598,14 +598,13 @@ static int init_dma_desc_rings(struct net_device *netd) return 0; -txalloc_err: - while (queue_num--) - free_tx_ring(priv->device, priv->txq[queue_num], tx_rsize); - return ret; - rxalloc_err: while (queue_num--) free_rx_ring(priv->device, priv->rxq[queue_num], rx_rsize); + queue_num = SXGBE_TX_QUEUES; +txalloc_err: + while (queue_num--) + free_tx_ring(priv->device, priv->txq[queue_num], tx_rsize); return ret; } @@ -1080,7 +1079,9 @@ static int sxgbe_open(struct net_device *dev) priv->dma_buf_sz = SXGBE_ALIGN(DMA_BUFFER_SIZE); priv->tx_tc = TC_DEFAULT; priv->rx_tc = TC_DEFAULT; - init_dma_desc_rings(dev); + ret = init_dma_desc_rings(dev); + if (ret) + goto init_phy_error; /* DMA initialization and SW reset */ ret = sxgbe_init_dma_engine(priv); @@ -1189,6 +1190,7 @@ static int sxgbe_open(struct net_device *dev) init_error: free_dma_desc_resources(priv); +init_phy_error: if (dev->phydev) phy_disconnect(dev->phydev); phy_error: diff --git a/drivers/net/ethernet/wangxun/libwx/wx_lib.c b/drivers/net/ethernet/wangxun/libwx/wx_lib.c index 4c203f4afd68..3666302e3401 100644 --- a/drivers/net/ethernet/wangxun/libwx/wx_lib.c +++ b/drivers/net/ethernet/wangxun/libwx/wx_lib.c @@ -1481,6 +1481,8 @@ static netdev_tx_t wx_xmit_frame_ring(struct sk_buff *skb, if (skb_vlan_tag_present(skb)) { tx_flags |= skb_vlan_tag_get(skb) << WX_TX_FLAGS_VLAN_SHIFT; tx_flags |= WX_TX_FLAGS_HW_VLAN; + } else if (eth_type_vlan(skb->protocol)) { + tx_flags |= WX_TX_FLAGS_SW_VLAN; } /* record initial flags and protocol */ diff --git a/drivers/net/netconsole.c b/drivers/net/netconsole.c index 60375bb814a1..3b76ec3cd49b 100644 --- a/drivers/net/netconsole.c +++ b/drivers/net/netconsole.c @@ -1103,7 +1103,8 @@ static void send_ext_msg_udp(struct netconsole_target *nt, const char *msg, if (msg_len + release_len + userdata_len <= MAX_PRINT_CHUNK) { /* No fragmentation needed */ if (nt->release) { - scnprintf(buf, MAX_PRINT_CHUNK, "%s,%s", release, msg); + scnprintf(buf, MAX_PRINT_CHUNK, "%s,%.*s", release, + msg_len, msg); msg_len += release_len; } else { memcpy(buf, msg, msg_len); diff --git a/drivers/net/phy/phylink.c b/drivers/net/phy/phylink.c index b78dfcbec936..e6f3963739ad 100644 --- a/drivers/net/phy/phylink.c +++ b/drivers/net/phy/phylink.c @@ -1728,8 +1728,8 @@ struct phylink *phylink_create(struct phylink_config *config, } else if (config->type == PHYLINK_DEV) { pl->dev = config->dev; } else { - kfree(pl); - return ERR_PTR(-EINVAL); + ret = -EINVAL; + goto free_pl; } pl->using_mac_select_pcs = using_mac_select_pcs; @@ -1753,28 +1753,29 @@ struct phylink *phylink_create(struct phylink_config *config, phylink_validate(pl, pl->supported, &pl->link_config); ret = phylink_parse_mode(pl, fwnode); - if (ret < 0) { - kfree(pl); - return ERR_PTR(ret); - } + if (ret < 0) + goto free_pl; if (pl->cfg_link_an_mode == MLO_AN_FIXED) { ret = phylink_parse_fixedlink(pl, fwnode); - if (ret < 0) { - kfree(pl); - return ERR_PTR(ret); - } + if (ret < 0) + goto release_link_gpio; } pl->cur_link_an_mode = pl->cfg_link_an_mode; ret = phylink_register_sfp(pl, fwnode); - if (ret < 0) { - kfree(pl); - return ERR_PTR(ret); - } + if (ret < 0) + goto release_link_gpio; return pl; + +release_link_gpio: + if (pl->link_gpio) + gpiod_put(pl->link_gpio); +free_pl: + kfree(pl); + return ERR_PTR(ret); } EXPORT_SYMBOL_GPL(phylink_create); diff --git a/drivers/net/veth.c b/drivers/net/veth.c index eb7ce1211784..918a7130011d 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -756,7 +756,7 @@ static int veth_convert_skb_to_xdp_buff(struct veth_rq *rq, u32 frame_sz; if (skb_shared(skb) || skb_head_is_locked(skb) || - skb_shinfo(skb)->nr_frags || + skb_is_nonlinear(skb) || skb_headroom(skb) < XDP_PACKET_HEADROOM) { if (skb_pp_cow_data(rq->page_pool, pskb, XDP_PACKET_HEADROOM)) goto drop; @@ -771,7 +771,7 @@ static int veth_convert_skb_to_xdp_buff(struct veth_rq *rq, xdp_prepare_buff(xdp, skb->head, skb_headroom(skb), skb_headlen(skb), true); - if (skb_is_nonlinear(skb)) { + if (skb_shinfo(skb)->nr_frags) { skb_shinfo(skb)->xdp_frags_size = skb->data_len; xdp_buff_set_frags_flag(xdp); } else { diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c index c6d80973fd96..fdc46f52c7fc 100644 --- a/drivers/net/vxlan/vxlan_core.c +++ b/drivers/net/vxlan/vxlan_core.c @@ -2138,7 +2138,7 @@ static bool route_shortcircuit(struct net_device *dev, struct sk_buff *skb) { struct iphdr *pip; - if (!pskb_may_pull(skb, sizeof(struct iphdr))) + if (!pskb_network_may_pull(skb, sizeof(struct iphdr))) return false; pip = ip_hdr(skb); n = neigh_lookup(&arp_tbl, &pip->daddr, dev); @@ -2164,7 +2164,7 @@ static bool route_shortcircuit(struct net_device *dev, struct sk_buff *skb) */ if (!ipv6_stub->nd_tbl) return false; - if (!pskb_may_pull(skb, sizeof(struct ipv6hdr))) + if (!pskb_network_may_pull(skb, sizeof(struct ipv6hdr))) return false; pip6 = ipv6_hdr(skb); n = neigh_lookup(ipv6_stub->nd_tbl, &pip6->daddr, dev); @@ -2186,13 +2186,19 @@ static bool route_shortcircuit(struct net_device *dev, struct sk_buff *skb) } if (n) { + u8 haddr[ETH_ALEN]; bool diff; - diff = !ether_addr_equal(eth_hdr(skb)->h_dest, n->ha); + neigh_ha_snapshot(haddr, n, dev); + diff = !ether_addr_equal_unaligned(eth_hdr(skb)->h_dest, haddr); if (diff) { + if (skb_cow_head(skb, 0)) { + neigh_release(n); + return false; + } memcpy(eth_hdr(skb)->h_source, eth_hdr(skb)->h_dest, dev->addr_len); - memcpy(eth_hdr(skb)->h_dest, n->ha, dev->addr_len); + memcpy(eth_hdr(skb)->h_dest, haddr, dev->addr_len); } neigh_release(n); return diff; @@ -2790,6 +2796,7 @@ static netdev_tx_t vxlan_xmit(struct sk_buff *skb, struct net_device *dev) (ntohs(eth->h_proto) == ETH_P_IP || ntohs(eth->h_proto) == ETH_P_IPV6)) { did_rsc = route_shortcircuit(dev, skb); + eth = eth_hdr(skb); if (did_rsc) f = vxlan_find_mac_tx(vxlan, eth->h_dest, vni); } diff --git a/drivers/net/wireless/ath/ath6kl/txrx.c b/drivers/net/wireless/ath/ath6kl/txrx.c index a39c8150587e..24fbb4bb6875 100644 --- a/drivers/net/wireless/ath/ath6kl/txrx.c +++ b/drivers/net/wireless/ath/ath6kl/txrx.c @@ -1829,7 +1829,7 @@ void aggr_reset_state(struct aggr_info_conn *aggr_conn) return; if (aggr_conn->timer_scheduled) { - del_timer(&aggr_conn->timer); + timer_delete_sync(&aggr_conn->timer); aggr_conn->timer_scheduled = false; } diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c index cfdd92564060..45f1d2cae3ca 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c @@ -906,6 +906,7 @@ int brcmf_sdiod_probe(struct brcmf_sdio_dev *sdiodev) return ret; } switch (sdiodev->func2->device) { + case SDIO_DEVICE_ID_BROADCOM_43752: case SDIO_DEVICE_ID_BROADCOM_CYPRESS_4373: f2_blksz = SDIO_4373_FUNC2_BLOCKSIZE; break; @@ -991,9 +992,9 @@ static const struct sdio_device_id brcmf_sdmmc_ids[] = { BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_4354, WCC), BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_4356, WCC), BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_4359, WCC), + BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_43752, WCC), BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_4373, CYW), BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_43012, CYW), - BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_43752, CYW), BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_89359, CYW), CYW_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_CYPRESS_43439, CYW), { /* end: all zeroes */ } @@ -1064,6 +1065,7 @@ static int brcmf_ops_sdio_probe(struct sdio_func *func, bus_if = kzalloc(sizeof(*bus_if), GFP_KERNEL); if (!bus_if) return -ENOMEM; + mutex_init(&bus_if->bus_reset_lock); sdiodev = kzalloc(sizeof(*sdiodev), GFP_KERNEL); if (!sdiodev) { kfree(bus_if); @@ -1125,6 +1127,14 @@ static void brcmf_ops_sdio_remove(struct sdio_func *func) if (func->num != 1) return; + /* Drain bus_reset before the shared brcmf_sdiod_remove() + * teardown, which the SDIO reset callback also reaches. The + * data worker can arm bus_reset via brcmf_fw_crashed(); cancel + * it first. + */ + brcmf_sdio_cancel_datawork(sdiodev->bus); + brcmf_bus_cancel_reset_work(bus_if); + /* only proceed with rest of cleanup if func 1 */ brcmf_sdiod_remove(sdiodev); @@ -1199,6 +1209,8 @@ static int brcmf_ops_sdio_suspend(struct device *dev) } else { /* power will be cut so remove device, probe again in resume */ brcmf_sdiod_intr_unregister(sdiodev); + brcmf_sdio_cancel_datawork(sdiodev->bus); + brcmf_bus_cancel_reset_work(bus_if); ret = brcmf_sdiod_remove(sdiodev); if (ret) brcmf_err("Failed to remove device on suspend\n"); @@ -1224,6 +1236,8 @@ static int brcmf_ops_sdio_resume(struct device *dev) ret = brcmf_sdiod_probe(sdiodev); if (ret) brcmf_err("Failed to probe device on resume\n"); + else + brcmf_bus_allow_reset_work(bus_if); } else { if (sdiodev->wowl_enabled && sdiodev->settings->bus.sdio.oob_irq_supported) disable_irq_wake(sdiodev->settings->bus.sdio.oob_irq_nr); diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bus.h b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bus.h index fe31051a9e11..9371c1489948 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bus.h +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bus.h @@ -9,6 +9,7 @@ #include #include #include +#include #include "debug.h" /* IDs of the 6 default common rings of msgbuf protocol */ @@ -179,6 +180,8 @@ struct brcmf_bus { enum brcmf_fwvendor fwvid; bool always_use_fws_queue; bool wowl_supported; + bool removing; /* device removal in progress; quiesce async work */ + struct mutex bus_reset_lock; const struct brcmf_bus_ops *ops; struct brcmf_bus_msgbuf *msgbuf; @@ -186,6 +189,9 @@ struct brcmf_bus { struct list_head list; }; +void brcmf_bus_cancel_reset_work(struct brcmf_bus *bus_if); +void brcmf_bus_allow_reset_work(struct brcmf_bus *bus_if); + /* * callback wrappers */ diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c index 4dbb1898f306..e3a01a28a182 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c @@ -738,7 +738,7 @@ static u32 brcmf_chip_tcm_rambase(struct brcmf_chip_priv *ci) case BRCM_CC_4364_CHIP_ID: case CY_CC_4373_CHIP_ID: return 0x160000; - case CY_CC_43752_CHIP_ID: + case BRCM_CC_43752_CHIP_ID: case BRCM_CC_4377_CHIP_ID: return 0x170000; case BRCM_CC_4378_CHIP_ID: @@ -1465,7 +1465,7 @@ bool brcmf_chip_sr_capable(struct brcmf_chip *pub) reg = chip->ops->read32(chip->ctx, addr); return (reg & CC_SR_CTL0_ENABLE_MASK) != 0; case BRCM_CC_4359_CHIP_ID: - case CY_CC_43752_CHIP_ID: + case BRCM_CC_43752_CHIP_ID: case CY_CC_43012_CHIP_ID: addr = CORE_CC_REG(pmu->base, retention_ctl); reg = chip->ops->read32(chip->ctx, addr); diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c index 20ab9b1eea28..58eaf08a147b 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c @@ -1162,6 +1162,35 @@ static int brcmf_revinfo_read(struct seq_file *s, void *data) return 0; } +/* + * Serialize arming from debugfs reset and brcmf_fw_crashed() against + * teardown. The remove path sets ->removing and drains the work while + * holding bus_reset_lock, so a racing armer is either drained or skips it. + */ +static void brcmf_bus_schedule_reset(struct brcmf_bus *bus_if) +{ + mutex_lock(&bus_if->bus_reset_lock); + if (bus_if->drvr && bus_if->drvr->bus_reset.func && !bus_if->removing) + schedule_work(&bus_if->drvr->bus_reset); + mutex_unlock(&bus_if->bus_reset_lock); +} + +void brcmf_bus_cancel_reset_work(struct brcmf_bus *bus_if) +{ + mutex_lock(&bus_if->bus_reset_lock); + bus_if->removing = true; + if (bus_if->drvr) + cancel_work_sync(&bus_if->drvr->bus_reset); + mutex_unlock(&bus_if->bus_reset_lock); +} + +void brcmf_bus_allow_reset_work(struct brcmf_bus *bus_if) +{ + mutex_lock(&bus_if->bus_reset_lock); + bus_if->removing = false; + mutex_unlock(&bus_if->bus_reset_lock); +} + static void brcmf_core_bus_reset(struct work_struct *work) { struct brcmf_pub *drvr = container_of(work, struct brcmf_pub, @@ -1182,7 +1211,7 @@ static ssize_t bus_reset_write(struct file *file, const char __user *user_buf, if (value != 1) return -EINVAL; - schedule_work(&drvr->bus_reset); + brcmf_bus_schedule_reset(drvr->bus_if); return count; } @@ -1410,14 +1439,23 @@ void brcmf_dev_coredump(struct device *dev) void brcmf_fw_crashed(struct device *dev) { struct brcmf_bus *bus_if = dev_get_drvdata(dev); - struct brcmf_pub *drvr = bus_if->drvr; + struct brcmf_pub *drvr; + + /* May fire before brcmf_attach() wires up drvr, or after removal + * has cleared it; guard the derefs below (and the arming gate in + * brcmf_bus_schedule_reset() already checks drvr/->removing). + */ + if (!bus_if) + return; + drvr = bus_if->drvr; + if (!drvr) + return; bphy_err(drvr, "Firmware has halted or crashed\n"); brcmf_dev_coredump(dev); - if (drvr->bus_reset.func) - schedule_work(&drvr->bus_reset); + brcmf_bus_schedule_reset(bus_if); } void brcmf_detach(struct device *dev) diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c index c53099a3e250..48224588c8bd 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c @@ -2462,6 +2462,7 @@ brcmf_pcie_probe(struct pci_dev *pdev, const struct pci_device_id *id) ret = -ENOMEM; goto fail; } + mutex_init(&bus->bus_reset_lock); bus->msgbuf = kzalloc(sizeof(*bus->msgbuf), GFP_KERNEL); if (!bus->msgbuf) { ret = -ENOMEM; @@ -2547,6 +2548,11 @@ brcmf_pcie_remove(struct pci_dev *pdev) if (devinfo->ci) brcmf_pcie_intr_disable(devinfo); + if (devinfo->irq_allocated) + synchronize_irq(pdev->irq); + + brcmf_bus_cancel_reset_work(bus); + brcmf_detach(&pdev->dev); brcmf_free(&pdev->dev); diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c index a1b261446076..108e316ef90b 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c @@ -654,10 +654,10 @@ static const struct brcmf_firmware_mapping brcmf_sdio_fwnames[] = { BRCMF_FW_ENTRY(BRCM_CC_4354_CHIP_ID, 0xFFFFFFFF, 4354), BRCMF_FW_ENTRY(BRCM_CC_4356_CHIP_ID, 0xFFFFFFFF, 4356), BRCMF_FW_ENTRY(BRCM_CC_4359_CHIP_ID, 0xFFFFFFFF, 4359), + BRCMF_FW_ENTRY(BRCM_CC_43752_CHIP_ID, 0xFFFFFFFF, 43752), BRCMF_FW_ENTRY(CY_CC_4373_CHIP_ID, 0xFFFFFFFF, 4373), BRCMF_FW_ENTRY(CY_CC_43012_CHIP_ID, 0xFFFFFFFF, 43012), BRCMF_FW_ENTRY(CY_CC_43439_CHIP_ID, 0xFFFFFFFF, 43439), - BRCMF_FW_ENTRY(CY_CC_43752_CHIP_ID, 0xFFFFFFFF, 43752) }; #define TXCTL_CREDITS 2 @@ -3425,8 +3425,8 @@ static int brcmf_sdio_download_firmware(struct brcmf_sdio *bus, static bool brcmf_sdio_aos_no_decode(struct brcmf_sdio *bus) { - if (bus->ci->chip == CY_CC_43012_CHIP_ID || - bus->ci->chip == CY_CC_43752_CHIP_ID) + if (bus->ci->chip == BRCM_CC_43752_CHIP_ID || + bus->ci->chip == CY_CC_43012_CHIP_ID) return true; else return false; @@ -4274,8 +4274,8 @@ static void brcmf_sdio_firmware_callback(struct device *dev, int err, bus->hostintmask, NULL); switch (sdiod->func1->device) { + case SDIO_DEVICE_ID_BROADCOM_43752: case SDIO_DEVICE_ID_BROADCOM_CYPRESS_4373: - case SDIO_DEVICE_ID_BROADCOM_CYPRESS_43752: brcmf_dbg(INFO, "set F2 watermark to 0x%x*4 bytes\n", CY_4373_F2_WATERMARK); brcmf_sdiod_writeb(sdiod, SBSDIO_WATERMARK, @@ -4550,6 +4550,12 @@ struct brcmf_sdio *brcmf_sdio_probe(struct brcmf_sdio_dev *sdiodev) return NULL; } +void brcmf_sdio_cancel_datawork(struct brcmf_sdio *bus) +{ + if (bus) + cancel_work_sync(&bus->datawork); +} + /* Detach and free everything */ void brcmf_sdio_remove(struct brcmf_sdio *bus) { diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.h b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.h index 0d18ed15b403..927fb82dec9c 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.h +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.h @@ -361,6 +361,7 @@ int brcmf_sdiod_remove(struct brcmf_sdio_dev *sdiodev); struct brcmf_sdio *brcmf_sdio_probe(struct brcmf_sdio_dev *sdiodev); void brcmf_sdio_remove(struct brcmf_sdio *bus); void brcmf_sdio_isr(struct brcmf_sdio *bus, bool in_isr); +void brcmf_sdio_cancel_datawork(struct brcmf_sdio *bus); void brcmf_sdio_wd_timer(struct brcmf_sdio *bus, bool active); void brcmf_sdio_wowl_config(struct device *dev, bool enabled); diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/usb.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/usb.c index 0364b81f6e91..56fa7e28e20c 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/usb.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/usb.c @@ -1254,6 +1254,7 @@ static int brcmf_usb_probe_cb(struct brcmf_usbdev_info *devinfo, ret = -ENOMEM; goto fail; } + mutex_init(&bus->bus_reset_lock); bus->dev = dev; bus_pub->bus = bus; @@ -1320,6 +1321,8 @@ brcmf_usb_disconnect_cb(struct brcmf_usbdev_info *devinfo) return; brcmf_dbg(USB, "Enter, bus_pub %p\n", devinfo); + brcmf_bus_cancel_reset_work(devinfo->bus_pub.bus); + brcmf_detach(devinfo->dev); brcmf_free(devinfo->dev); kfree(devinfo->bus_pub.bus); diff --git a/drivers/net/wireless/broadcom/brcm80211/include/brcm_hw_ids.h b/drivers/net/wireless/broadcom/brcm80211/include/brcm_hw_ids.h index 44684bf1b9ac..2f886af068c5 100644 --- a/drivers/net/wireless/broadcom/brcm80211/include/brcm_hw_ids.h +++ b/drivers/net/wireless/broadcom/brcm80211/include/brcm_hw_ids.h @@ -52,13 +52,13 @@ #define BRCM_CC_43664_CHIP_ID 43664 #define BRCM_CC_43666_CHIP_ID 43666 #define BRCM_CC_4371_CHIP_ID 0x4371 +#define BRCM_CC_43752_CHIP_ID 43752 #define BRCM_CC_4377_CHIP_ID 0x4377 #define BRCM_CC_4378_CHIP_ID 0x4378 #define BRCM_CC_4387_CHIP_ID 0x4387 #define CY_CC_4373_CHIP_ID 0x4373 #define CY_CC_43012_CHIP_ID 43012 #define CY_CC_43439_CHIP_ID 43439 -#define CY_CC_43752_CHIP_ID 43752 /* USB Device IDs */ #define BRCM_USB_43143_DEVICE_ID 0xbd1e diff --git a/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c b/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c index 3f4c8c0743fe..43a35a25574b 100644 --- a/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c +++ b/drivers/net/wireless/marvell/mwifiex/11n_rxreorder.c @@ -44,7 +44,7 @@ static int mwifiex_11n_dispatch_amsdu_pkt(struct mwifiex_private *priv, ntohs(rx_hdr->eth803_hdr.h_proto) == ETH_P_TDLS) { mwifiex_process_tdls_action_frame(priv, (u8 *)rx_hdr, - skb->len); + rx_skb->len); } if (priv->bss_role == MWIFIEX_BSS_ROLE_UAP) diff --git a/drivers/of/of_reserved_mem.c b/drivers/of/of_reserved_mem.c index 7b5d6562fe4a..26292d33fd05 100644 --- a/drivers/of/of_reserved_mem.c +++ b/drivers/of/of_reserved_mem.c @@ -28,7 +28,9 @@ #include "of_private.h" -static struct reserved_mem reserved_mem[MAX_RESERVED_REGIONS]; +static struct reserved_mem reserved_mem_array[MAX_RESERVED_REGIONS] __initdata; +static struct reserved_mem *reserved_mem __refdata = reserved_mem_array; +static int total_reserved_mem_cnt = MAX_RESERVED_REGIONS; static int reserved_mem_count; static int __init early_init_dt_alloc_reserved_memory_arch(phys_addr_t size, @@ -57,6 +59,59 @@ static int __init early_init_dt_alloc_reserved_memory_arch(phys_addr_t size, return err; } +/* + * alloc_reserved_mem_array() - allocate memory for the reserved_mem + * array using memblock + * + * This function is used to allocate memory for the reserved_mem + * array according to the total number of reserved memory regions + * defined in the DT. + * After the new array is allocated, the information stored in + * the initial static array is copied over to this new array and + * the new array is used from this point on. + */ +static int __init alloc_reserved_mem_array(void) +{ + struct reserved_mem *new_array; + size_t alloc_size, copy_size, memset_size; + int ret; + + if (!total_reserved_mem_cnt) + return 0; + + alloc_size = array_size(total_reserved_mem_cnt, sizeof(*new_array)); + if (alloc_size == SIZE_MAX) { + ret = -EOVERFLOW; + goto fail; + } + + new_array = memblock_alloc(alloc_size, SMP_CACHE_BYTES); + if (!new_array) { + ret = -ENOMEM; + goto fail; + } + + copy_size = array_size(reserved_mem_count, sizeof(*new_array)); + if (copy_size == SIZE_MAX) { + memblock_free(new_array, alloc_size); + ret = -EOVERFLOW; + goto fail; + } + + memset_size = alloc_size - copy_size; + + memcpy(new_array, reserved_mem, copy_size); + memset(new_array + reserved_mem_count, 0, memset_size); + + reserved_mem = new_array; + return 0; + +fail: + pr_err("Failed to allocate memory for reserved_mem array with err: %d", ret); + reserved_mem_count = 0; + return ret; +} + static void __init fdt_init_reserved_mem_node(struct reserved_mem *rmem); /* * fdt_reserved_mem_save_node() - save fdt node for second pass initialization @@ -66,7 +121,7 @@ static void __init fdt_reserved_mem_save_node(unsigned long node, const char *un { struct reserved_mem *rmem = &reserved_mem[reserved_mem_count]; - if (reserved_mem_count == ARRAY_SIZE(reserved_mem)) { + if (reserved_mem_count == total_reserved_mem_cnt) { pr_err("not enough space for all defined regions.\n"); return; } @@ -199,6 +254,10 @@ void __init fdt_scan_reserved_mem_reg_nodes(void) return; } + /* Attempt dynamic allocation of a new reserved_mem array */ + if (alloc_reserved_mem_array()) + return; + if (__reserved_mem_check_root(node)) { pr_err("Reserved memory: unsupported node format, ignoring\n"); return; @@ -243,7 +302,7 @@ static int __init __reserved_mem_alloc_size(unsigned long node, const char *unam int __init fdt_scan_reserved_mem(void) { int node, child; - int dynamic_nodes_cnt = 0; + int dynamic_nodes_cnt = 0, count = 0; int dynamic_nodes[MAX_RESERVED_REGIONS]; const void *fdt = initial_boot_params; @@ -266,6 +325,9 @@ int __init fdt_scan_reserved_mem(void) uname = fdt_get_name(fdt, child, NULL); err = __reserved_mem_reserve_reg(child, uname); + if (!err) + count++; + /* * Save the nodes for the dynamically-placed regions * into an array which will be used for allocation right @@ -273,18 +335,29 @@ int __init fdt_scan_reserved_mem(void) * or marked as no-map. This is done to avoid dynamically * allocating from one of the statically-placed regions. */ - if (err == -ENOENT && of_get_flat_dt_prop(child, "size", NULL)) { - dynamic_nodes[dynamic_nodes_cnt] = child; - dynamic_nodes_cnt++; + if (err != -ENOENT || !of_get_flat_dt_prop(child, "size", NULL)) + continue; + + if (dynamic_nodes_cnt == MAX_RESERVED_REGIONS) { + pr_err("too many defined dynamic regions, skip '%s'\n", + uname); + continue; } + + dynamic_nodes[dynamic_nodes_cnt] = child; + dynamic_nodes_cnt++; } for (int i = 0; i < dynamic_nodes_cnt; i++) { const char *uname; + int err; child = dynamic_nodes[i]; uname = fdt_get_name(fdt, child, NULL); - __reserved_mem_alloc_size(child, uname); + err = __reserved_mem_alloc_size(child, uname); + if (!err) + count++; } + total_reserved_mem_cnt = count; return 0; } diff --git a/drivers/phy/xilinx/phy-zynqmp.c b/drivers/phy/xilinx/phy-zynqmp.c index e6579002f114..9288247d66f3 100644 --- a/drivers/phy/xilinx/phy-zynqmp.c +++ b/drivers/phy/xilinx/phy-zynqmp.c @@ -53,7 +53,7 @@ #define L0_TM_DIG_6 0x106c #define L0_TM_DIS_DESCRAMBLE_DECODER 0x0f #define L0_TX_DIG_61 0x00f4 -#define L0_TM_DISABLE_SCRAMBLE_ENCODER 0x0f +#define L0_TM_DISABLE_SCRAMBLE_ENCODER (BIT(3) | GENMASK(1, 0)) /* PLL Test Mode register parameters */ #define L0_TM_PLL_DIG_37 0x2094 @@ -222,7 +222,6 @@ struct xpsgtr_phy { * @siou: siou base address * @gtr_mutex: mutex for locking * @phys: PHY lanes - * @refclk_sscs: spread spectrum settings for the reference clocks * @clk: reference clocks * @tx_term_fix: fix for GT issue * @saved_icm_cfg0: stored value of ICM CFG0 register @@ -235,7 +234,6 @@ struct xpsgtr_dev { void __iomem *siou; struct mutex gtr_mutex; /* mutex for locking */ struct xpsgtr_phy phys[NUM_LANES]; - const struct xpsgtr_ssc *refclk_sscs[NUM_LANES]; struct clk *clk[NUM_LANES]; bool tx_term_fix; unsigned int saved_icm_cfg0; @@ -398,13 +396,40 @@ static int xpsgtr_wait_pll_lock(struct phy *phy) return ret; } +/* Get the spread spectrum (SSC) settings for the reference clock rate */ +static const struct xpsgtr_ssc *xpsgtr_find_sscs(struct xpsgtr_phy *gtr_phy) +{ + unsigned long rate; + struct clk *clk; + unsigned int i; + + clk = gtr_phy->dev->clk[gtr_phy->refclk]; + rate = clk_get_rate(clk); + + for (i = 0 ; i < ARRAY_SIZE(ssc_lookup); i++) { + /* Allow an error of 100 ppm */ + unsigned long error = ssc_lookup[i].refclk_rate / 10000; + + if (abs(rate - ssc_lookup[i].refclk_rate) < error) + return &ssc_lookup[i]; + } + + dev_err(gtr_phy->dev->dev, "Invalid rate %lu for reference clock %u\n", + rate, gtr_phy->refclk); + + return NULL; +} + /* Configure PLL and spread-sprectrum clock. */ -static void xpsgtr_configure_pll(struct xpsgtr_phy *gtr_phy) +static int xpsgtr_configure_pll(struct xpsgtr_phy *gtr_phy) { const struct xpsgtr_ssc *ssc; u32 step_size; - ssc = gtr_phy->dev->refclk_sscs[gtr_phy->refclk]; + ssc = xpsgtr_find_sscs(gtr_phy); + if (!ssc) + return -EINVAL; + step_size = ssc->step_size; xpsgtr_clr_set(gtr_phy->dev, PLL_REF_SEL(gtr_phy->lane), @@ -446,6 +471,8 @@ static void xpsgtr_configure_pll(struct xpsgtr_phy *gtr_phy) xpsgtr_clr_set_phy(gtr_phy, L0_PLL_SS_STEP_SIZE_3_MSB, STEP_SIZE_3_MASK, (step_size & STEP_SIZE_3_MASK) | FORCE_STEP_SIZE | FORCE_STEPS); + + return 0; } /* Configure the lane protocol. */ @@ -475,11 +502,30 @@ static void xpsgtr_lane_set_protocol(struct xpsgtr_phy *gtr_phy) } } -/* Bypass (de)scrambler and 8b/10b decoder and encoder. */ -static void xpsgtr_bypass_scrambler_8b10b(struct xpsgtr_phy *gtr_phy) +/** + * xpsgtr_bypass_scrambler_8b10b - Configure scrambler/encoder behavior + * @gtr_phy: pointer to lane context + * @bypass: true to enable scrambler/encoder bypass (SATA/SGMII), + * false to disable scrambler/encoder bypass (USB3) + * + * Uses RMW to preserve reserved and unrelated register fields. + */ +static void xpsgtr_bypass_scrambler_8b10b(struct xpsgtr_phy *gtr_phy, + bool bypass) { - xpsgtr_write_phy(gtr_phy, L0_TM_DIG_6, L0_TM_DIS_DESCRAMBLE_DECODER); - xpsgtr_write_phy(gtr_phy, L0_TX_DIG_61, L0_TM_DISABLE_SCRAMBLE_ENCODER); + if (bypass) { + xpsgtr_clr_set_phy(gtr_phy, L0_TM_DIG_6, + L0_TM_DIS_DESCRAMBLE_DECODER, + L0_TM_DIS_DESCRAMBLE_DECODER); + xpsgtr_clr_set_phy(gtr_phy, L0_TX_DIG_61, + L0_TM_DISABLE_SCRAMBLE_ENCODER, + L0_TM_DISABLE_SCRAMBLE_ENCODER); + } else { + xpsgtr_clr_set_phy(gtr_phy, L0_TM_DIG_6, + L0_TM_DIS_DESCRAMBLE_DECODER, 0); + xpsgtr_clr_set_phy(gtr_phy, L0_TX_DIG_61, + L0_TM_DISABLE_SCRAMBLE_ENCODER, 0); + } } /* DP-specific initialization. */ @@ -500,7 +546,7 @@ static void xpsgtr_phy_init_sata(struct xpsgtr_phy *gtr_phy) { struct xpsgtr_dev *gtr_dev = gtr_phy->dev; - xpsgtr_bypass_scrambler_8b10b(gtr_phy); + xpsgtr_bypass_scrambler_8b10b(gtr_phy, true); writel(gtr_phy->lane, gtr_dev->siou + SATA_CONTROL_OFFSET); } @@ -516,7 +562,7 @@ static void xpsgtr_phy_init_sgmii(struct xpsgtr_phy *gtr_phy) xpsgtr_clr_set(gtr_dev, TX_PROT_BUS_WIDTH, mask, val); xpsgtr_clr_set(gtr_dev, RX_PROT_BUS_WIDTH, mask, val); - xpsgtr_bypass_scrambler_8b10b(gtr_phy); + xpsgtr_bypass_scrambler_8b10b(gtr_phy, true); } /* Configure TX de-emphasis and margining for DP. */ @@ -631,12 +677,13 @@ static int xpsgtr_phy_init(struct phy *phy) { struct xpsgtr_phy *gtr_phy = phy_get_drvdata(phy); struct xpsgtr_dev *gtr_dev = gtr_phy->dev; - int ret = 0; + int ret; mutex_lock(>r_dev->gtr_mutex); /* Configure and enable the clock when peripheral phy_init call */ - if (clk_prepare_enable(gtr_dev->clk[gtr_phy->refclk])) + ret = clk_prepare_enable(gtr_dev->clk[gtr_phy->refclk]); + if (ret) goto out; /* Skip initialization if not required. */ @@ -646,7 +693,7 @@ static int xpsgtr_phy_init(struct phy *phy) if (gtr_dev->tx_term_fix) { ret = xpsgtr_phy_tx_term_fix(gtr_phy); if (ret < 0) - goto out; + goto out_disable_clk; gtr_dev->tx_term_fix = false; } @@ -658,7 +705,10 @@ static int xpsgtr_phy_init(struct phy *phy) * Configure the PLL, the lane protocol, and perform protocol-specific * initialization. */ - xpsgtr_configure_pll(gtr_phy); + ret = xpsgtr_configure_pll(gtr_phy); + if (ret) + goto out_disable_clk; + xpsgtr_lane_set_protocol(gtr_phy); switch (gtr_phy->protocol) { @@ -673,8 +723,16 @@ static int xpsgtr_phy_init(struct phy *phy) case ICM_PROTOCOL_SGMII: xpsgtr_phy_init_sgmii(gtr_phy); break; + + case ICM_PROTOCOL_USB: + xpsgtr_bypass_scrambler_8b10b(gtr_phy, false); + break; } + goto out; + +out_disable_clk: + clk_disable_unprepare(gtr_dev->clk[gtr_phy->refclk]); out: mutex_unlock(>r_dev->gtr_mutex); return ret; @@ -823,8 +881,7 @@ static struct phy *xpsgtr_xlate(struct device *dev, } refclk = args->args[3]; - if (refclk >= ARRAY_SIZE(gtr_dev->refclk_sscs) || - !gtr_dev->refclk_sscs[refclk]) { + if (refclk >= ARRAY_SIZE(gtr_dev->clk)) { dev_err(dev, "Invalid reference clock number %u\n", refclk); return ERR_PTR(-EINVAL); } @@ -928,9 +985,7 @@ static int xpsgtr_get_ref_clocks(struct xpsgtr_dev *gtr_dev) { unsigned int refclk; - for (refclk = 0; refclk < ARRAY_SIZE(gtr_dev->refclk_sscs); ++refclk) { - unsigned long rate; - unsigned int i; + for (refclk = 0; refclk < ARRAY_SIZE(gtr_dev->clk); ++refclk) { struct clk *clk; char name[8]; @@ -946,29 +1001,6 @@ static int xpsgtr_get_ref_clocks(struct xpsgtr_dev *gtr_dev) continue; gtr_dev->clk[refclk] = clk; - - /* - * Get the spread spectrum (SSC) settings for the reference - * clock rate. - */ - rate = clk_get_rate(clk); - - for (i = 0 ; i < ARRAY_SIZE(ssc_lookup); i++) { - /* Allow an error of 100 ppm */ - unsigned long error = ssc_lookup[i].refclk_rate / 10000; - - if (abs(rate - ssc_lookup[i].refclk_rate) < error) { - gtr_dev->refclk_sscs[refclk] = &ssc_lookup[i]; - break; - } - } - - if (i == ARRAY_SIZE(ssc_lookup)) { - dev_err(gtr_dev->dev, - "Invalid rate %lu for reference clock %u\n", - rate, refclk); - return -EINVAL; - } } return 0; @@ -1035,6 +1067,12 @@ static int xpsgtr_probe(struct platform_device *pdev) return PTR_ERR(provider); } + gtr_dev->saved_regs = devm_kmalloc(gtr_dev->dev, + sizeof(save_reg_address), + GFP_KERNEL); + if (!gtr_dev->saved_regs) + return -ENOMEM; + pm_runtime_set_active(gtr_dev->dev); pm_runtime_enable(gtr_dev->dev); @@ -1044,12 +1082,6 @@ static int xpsgtr_probe(struct platform_device *pdev) return ret; } - gtr_dev->saved_regs = devm_kmalloc(gtr_dev->dev, - sizeof(save_reg_address), - GFP_KERNEL); - if (!gtr_dev->saved_regs) - return -ENOMEM; - return 0; } diff --git a/drivers/pinctrl/Kconfig b/drivers/pinctrl/Kconfig index e05174e5efbc..7cb21d3ea02c 100644 --- a/drivers/pinctrl/Kconfig +++ b/drivers/pinctrl/Kconfig @@ -150,6 +150,7 @@ config PINCTRL_BM1880 depends on OF && (ARCH_BITMAIN || COMPILE_TEST) default ARCH_BITMAIN select PINMUX + select GENERIC_PINCONF help Pinctrl driver for Bitmain BM1880 SoC. @@ -366,6 +367,7 @@ config PINCTRL_MICROCHIP_SGPIO select GENERIC_PINCTRL_GROUPS select GENERIC_PINMUX_FUNCTIONS select OF_GPIO + select REGMAP_MMIO help Support for the serial GPIO interface used on Microsemi and Microchip SoCs. By using a serial interface, the SIO diff --git a/drivers/pinctrl/devicetree.c b/drivers/pinctrl/devicetree.c index 0b7f74beb6a6..fdb5ff0e4b0b 100644 --- a/drivers/pinctrl/devicetree.c +++ b/drivers/pinctrl/devicetree.c @@ -69,6 +69,10 @@ static int dt_remember_or_free_map(struct pinctrl *p, const char *statename, int i; struct pinctrl_dt_map *dt_map; + /* Initialize dev_name before any allocation can fail */ + for (i = 0; i < num_maps; i++) + map[i].dev_name = NULL; + /* Initialize common mapping table entry fields */ for (i = 0; i < num_maps; i++) { const char *devname; diff --git a/drivers/pinctrl/pinctrl-amd.c b/drivers/pinctrl/pinctrl-amd.c index debf36ce5785..65006eceaf39 100644 --- a/drivers/pinctrl/pinctrl-amd.c +++ b/drivers/pinctrl/pinctrl-amd.c @@ -869,8 +869,7 @@ static void amd_gpio_irq_init(struct amd_gpio *gpio_dev) u32 pin_reg, mask; int i; - mask = BIT(WAKE_CNTRL_OFF_S0I3) | BIT(WAKE_CNTRL_OFF_S3) | - BIT(WAKE_CNTRL_OFF_S4); + mask = BIT(WAKE_CNTRL_OFF_S0I3) | BIT(WAKE_CNTRL_OFF_S3); for (i = 0; i < desc->npins; i++) { int pin = desc->pins[i].number; diff --git a/drivers/pinctrl/qcom/pinctrl-msm.c b/drivers/pinctrl/qcom/pinctrl-msm.c index 27eb585bf42d..994f46c4c3b6 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm.c +++ b/drivers/pinctrl/qcom/pinctrl-msm.c @@ -1247,12 +1247,12 @@ static int msm_gpio_irq_reqres(struct irq_data *d) /* * If the wakeup_enable bit is present and marked as available for the * requested GPIO, it should be enabled when the GPIO is marked as - * wake irq in order to allow the interrupt event to be transfered to - * the PDC HW. + * wake irq in order to allow the interrupt event to be transferred to + * the PDC/MPM HW. * While the name implies only the wakeup event, it's also required for * the interrupt event. */ - if (test_bit(d->hwirq, pctrl->skip_wake_irqs) && g->intr_wakeup_present_bit) { + if (g->intr_wakeup_present_bit) { u32 intr_cfg; raw_spin_lock_irqsave(&pctrl->lock, flags); @@ -1280,7 +1280,7 @@ static void msm_gpio_irq_relres(struct irq_data *d) unsigned long flags; /* Disable the wakeup_enable bit if it has been set in msm_gpio_irq_reqres() */ - if (test_bit(d->hwirq, pctrl->skip_wake_irqs) && g->intr_wakeup_present_bit) { + if (g->intr_wakeup_present_bit) { u32 intr_cfg; raw_spin_lock_irqsave(&pctrl->lock, flags); diff --git a/drivers/pinctrl/qcom/pinctrl-sc8280xp.c b/drivers/pinctrl/qcom/pinctrl-sc8280xp.c index 4b1c49697698..67945ce867fa 100644 --- a/drivers/pinctrl/qcom/pinctrl-sc8280xp.c +++ b/drivers/pinctrl/qcom/pinctrl-sc8280xp.c @@ -1884,16 +1884,17 @@ static const struct msm_gpio_wakeirq_map sc8280xp_pdc_map[] = { { 126, 200 }, { 127, 225 }, { 128, 262 }, { 129, 201 }, { 130, 209 }, { 131, 173 }, { 132, 202 }, { 136, 210 }, { 138, 171 }, { 139, 226 }, { 140, 227 }, { 142, 228 }, - { 144, 229 }, { 145, 230 }, { 146, 231 }, { 148, 232 }, - { 149, 233 }, { 150, 234 }, { 152, 235 }, { 154, 212 }, - { 157, 213 }, { 161, 219 }, { 170, 236 }, { 171, 221 }, - { 174, 222 }, { 175, 237 }, { 176, 223 }, { 177, 170 }, - { 180, 238 }, { 181, 239 }, { 182, 240 }, { 183, 241 }, - { 184, 242 }, { 185, 243 }, { 190, 178 }, { 193, 184 }, - { 196, 185 }, { 198, 186 }, { 200, 174 }, { 201, 175 }, - { 205, 176 }, { 206, 177 }, { 208, 187 }, { 210, 198 }, - { 211, 199 }, { 212, 204 }, { 215, 205 }, { 220, 188 }, - { 221, 194 }, { 223, 195 }, { 225, 196 }, { 227, 197 }, + { 143, 261 }, { 144, 229 }, { 145, 230 }, { 146, 231 }, + { 148, 232 }, { 149, 233 }, { 150, 234 }, { 151, 264 }, + { 152, 235 }, { 154, 212 }, { 157, 213 }, { 161, 219 }, + { 170, 236 }, { 171, 221 }, { 174, 222 }, { 175, 237 }, + { 176, 223 }, { 177, 170 }, { 180, 238 }, { 181, 239 }, + { 182, 240 }, { 183, 241 }, { 184, 242 }, { 185, 243 }, + { 190, 178 }, { 193, 184 }, { 196, 185 }, { 198, 186 }, + { 200, 174 }, { 201, 175 }, { 205, 176 }, { 206, 177 }, + { 208, 187 }, { 210, 198 }, { 211, 199 }, { 212, 204 }, + { 215, 205 }, { 220, 188 }, { 221, 194 }, { 223, 195 }, + { 225, 196 }, { 227, 197 }, }; static struct msm_pinctrl_soc_data sc8280xp_pinctrl = { diff --git a/drivers/power/supply/bq25890_charger.c b/drivers/power/supply/bq25890_charger.c index 2f5ceaf00b94..ea62220ee155 100644 --- a/drivers/power/supply/bq25890_charger.c +++ b/drivers/power/supply/bq25890_charger.c @@ -320,7 +320,7 @@ static const u32 bq25890_tspct_tbl[] = { 145, 140, 130, 120, 115, 110, 100, 90, 80, 70, 60, 50, 40, 30, 20, 10, 0, -10, -20, -30, -40, -60, -70, -80, - -90, -10, -120, -140, -150, -170, -190, -210, + -90, -100, -120, -140, -150, -170, -190, -210, }; #define BQ25890_TSPCT_TBL_SIZE ARRAY_SIZE(bq25890_tspct_tbl) diff --git a/drivers/power/supply/max17040_battery.c b/drivers/power/supply/max17040_battery.c index 48453508688a..f4748adec092 100644 --- a/drivers/power/supply/max17040_battery.c +++ b/drivers/power/supply/max17040_battery.c @@ -405,7 +405,11 @@ static int max17040_get_property(struct power_supply *psy, val->intval = chip->low_soc_alert; break; case POWER_SUPPLY_PROP_STATUS: - power_supply_get_property_from_supplier(psy, psp, val); + ret = power_supply_get_property_from_supplier(psy, psp, val); + if (ret == -ENODEV) + val->intval = POWER_SUPPLY_STATUS_UNKNOWN; + else if (ret) + return ret; break; case POWER_SUPPLY_PROP_TEMP: if (!chip->channel_temp) diff --git a/drivers/s390/block/dasd_eckd.c b/drivers/s390/block/dasd_eckd.c index 5ff386fabdc9..3ac88a22a9e5 100644 --- a/drivers/s390/block/dasd_eckd.c +++ b/drivers/s390/block/dasd_eckd.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include @@ -3475,11 +3476,11 @@ static int dasd_eckd_check_device_format(struct dasd_device *base, { struct dasd_eckd_private *private = base->private; struct eckd_count *fmt_buffer; - struct irb irb; + size_t fmt_buffer_size; + unsigned int trkcount; int rpt_max, rpt_exp; - int fmt_buffer_size; + struct irb irb; int trk_per_cyl; - int trkcount; int tpm = 0; int rc; @@ -3490,7 +3491,9 @@ static int dasd_eckd_check_device_format(struct dasd_device *base, rpt_exp = recs_per_track(&private->rdc_data, 0, cdata->expect.blksize); trkcount = cdata->expect.stop_unit - cdata->expect.start_unit + 1; - fmt_buffer_size = trkcount * rpt_max * sizeof(struct eckd_count); + if (check_mul_overflow(trkcount, rpt_max, &fmt_buffer_size) || + check_mul_overflow(fmt_buffer_size, sizeof(struct eckd_count), &fmt_buffer_size)) + return -EINVAL; fmt_buffer = kzalloc(fmt_buffer_size, GFP_KERNEL | GFP_DMA); if (!fmt_buffer) diff --git a/drivers/s390/block/dasd_ioctl.c b/drivers/s390/block/dasd_ioctl.c index eb5dcbe37230..3cf2960ce47c 100644 --- a/drivers/s390/block/dasd_ioctl.c +++ b/drivers/s390/block/dasd_ioctl.c @@ -324,7 +324,7 @@ static int dasd_ioctl_check_format(struct block_device *bdev, void __user *argp) static int dasd_release_space(struct dasd_device *device, struct format_data_t *rdata) { - if (!device->discipline->is_ese && !device->discipline->is_ese(device)) + if (!device->discipline->is_ese || !device->discipline->is_ese(device)) return -ENOTSUPP; if (!device->discipline->release_space) return -ENOTSUPP; diff --git a/drivers/s390/crypto/zcrypt_api.c b/drivers/s390/crypto/zcrypt_api.c index 5020696f1379..6bfebc3ef0a7 100644 --- a/drivers/s390/crypto/zcrypt_api.c +++ b/drivers/s390/crypto/zcrypt_api.c @@ -1071,7 +1071,7 @@ static long _zcrypt_send_ep11_cprb(bool userspace, struct ap_perms *perms, print_hex_dump_debug("ep11req: ", DUMP_PREFIX_ADDRESS, 16, 1, ap_msg.msg, ap_msg.len, false); - if (perms != &ap_perms && domain < AUTOSEL_DOM) { + if (perms != &ap_perms && domain < AP_DOMAINS) { if (ap_msg.flags & AP_MSG_FLAG_ADMIN) { if (!test_bit_inv(domain, perms->adm)) { rc = -ENODEV; diff --git a/drivers/s390/crypto/zcrypt_ccamisc.c b/drivers/s390/crypto/zcrypt_ccamisc.c index 9dd4d01e1203..6790afd517ec 100644 --- a/drivers/s390/crypto/zcrypt_ccamisc.c +++ b/drivers/s390/crypto/zcrypt_ccamisc.c @@ -1218,6 +1218,9 @@ int cca_cipher2protkey(u16 cardnr, u16 domain, const u8 *ckey, } __packed * prepparm; int keytoklen = ((struct cipherkeytoken *)ckey)->len; + if (keytoklen > PARMBSIZE - sizeof(struct aureqparm)) + return -EINVAL; + /* get already prepared memory for 2 cprbs with param block each */ rc = alloc_and_prep_cprbmem(PARMBSIZE, &mem, &preqcblk, &prepcblk); if (rc) @@ -1381,6 +1384,9 @@ int cca_ecc2protkey(u16 cardnr, u16 domain, const u8 *key, } __packed * prepparm; int keylen = ((struct eccprivkeytoken *)key)->len; + if (keylen > PARMBSIZE - sizeof(struct aureqparm)) + return -EINVAL; + /* get already prepared memory for 2 cprbs with param block each */ rc = alloc_and_prep_cprbmem(PARMBSIZE, &mem, &preqcblk, &prepcblk); if (rc) diff --git a/drivers/s390/net/qeth_core_main.c b/drivers/s390/net/qeth_core_main.c index a3adaec5504e..2861b6b38cb0 100644 --- a/drivers/s390/net/qeth_core_main.c +++ b/drivers/s390/net/qeth_core_main.c @@ -6526,6 +6526,9 @@ int qeth_siocdevprivate(struct net_device *dev, struct ifreq *rq, void __user *d struct qeth_card *card = dev->ml_priv; int rc = 0; + if (!capable(CAP_NET_ADMIN)) + return -EPERM; + switch (cmd) { case SIOC_QETH_ADP_SET_SNMP_CONTROL: rc = qeth_snmp_command(card, data); diff --git a/drivers/s390/scsi/zfcp_aux.c b/drivers/s390/scsi/zfcp_aux.c index ab2f35bc294d..d3cc884ccd59 100644 --- a/drivers/s390/scsi/zfcp_aux.c +++ b/drivers/s390/scsi/zfcp_aux.c @@ -254,6 +254,7 @@ static int zfcp_allocate_low_mem_buffers(struct zfcp_adapter *adapter) static void zfcp_free_low_mem_buffers(struct zfcp_adapter *adapter) { mempool_destroy(adapter->pool.erp_req); + mempool_destroy(adapter->pool.gid_pn_req); mempool_destroy(adapter->pool.scsi_req); mempool_destroy(adapter->pool.scsi_abort); mempool_destroy(adapter->pool.qtcb_pool); diff --git a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c index e958b588d078..b7dd4efca0c7 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c +++ b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c @@ -5219,15 +5219,7 @@ static int _resume_v3_hw(struct device *device) return rc; } phys_init_v3_hw(hisi_hba); - - /* - * If a directly-attached disk is removed during suspend, a deadlock - * may occur, as the PHYE_RESUME_TIMEOUT processing will require the - * hisi_hba->device to be active, which can only happen when resume - * completes. So don't wait for the HA event workqueue to drain upon - * resume. - */ - sas_resume_ha_no_sync(sha); + sas_resume_ha(sha); clear_bit(HISI_SAS_RESETTING_BIT, &hisi_hba->flags); dev_warn(dev, "end of resuming controller\n"); diff --git a/drivers/scsi/libiscsi.c b/drivers/scsi/libiscsi.c index 29af3722ea22..0f169f243475 100644 --- a/drivers/scsi/libiscsi.c +++ b/drivers/scsi/libiscsi.c @@ -918,7 +918,7 @@ static void iscsi_scsi_cmd_rsp(struct iscsi_conn *conn, struct iscsi_hdr *hdr, } senselen = get_unaligned_be16(data); - if (datalen < senselen) + if (datalen < senselen + 2) goto invalid_datalen; memcpy(sc->sense_buffer, data + 2, diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c index c182aa83f2c9..4d2320512943 100644 --- a/drivers/scsi/libiscsi_tcp.c +++ b/drivers/scsi/libiscsi_tcp.c @@ -763,13 +763,6 @@ iscsi_tcp_hdr_dissect(struct iscsi_conn *conn, struct iscsi_hdr *hdr) rc = __iscsi_complete_pdu(conn, hdr, NULL, 0); spin_unlock(&conn->session->back_lock); break; - case ISCSI_OP_SCSI_CMD_RSP: - if (tcp_conn->in.datalen) { - iscsi_tcp_data_recv_prep(tcp_conn); - return 0; - } - rc = iscsi_complete_pdu(conn, hdr, NULL, 0); - break; case ISCSI_OP_R2T: if (ahslen) { rc = ISCSI_ERR_AHSLEN; @@ -777,6 +770,7 @@ iscsi_tcp_hdr_dissect(struct iscsi_conn *conn, struct iscsi_hdr *hdr) } rc = iscsi_tcp_r2t_rsp(conn, hdr); break; + case ISCSI_OP_SCSI_CMD_RSP: case ISCSI_OP_LOGIN_RSP: case ISCSI_OP_TEXT_RSP: case ISCSI_OP_REJECT: diff --git a/drivers/scsi/libsas/sas_init.c b/drivers/scsi/libsas/sas_init.c index 8566bb1208a0..ac157ab6a301 100644 --- a/drivers/scsi/libsas/sas_init.c +++ b/drivers/scsi/libsas/sas_init.c @@ -409,7 +409,7 @@ static void sas_resume_insert_broadcast_ha(struct sas_ha_struct *ha) } } -static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain) +void sas_resume_ha(struct sas_ha_struct *ha) { const unsigned long tmo = msecs_to_jiffies(25000); int i; @@ -425,6 +425,23 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain) dev_info(ha->dev, "waiting up to 25 seconds for %d phy%s to resume\n", i, i > 1 ? "s" : ""); wait_event_timeout(ha->eh_wait_q, phys_suspended(ha) == 0, tmo); + + /* + * All phys are back up or timed out. Turn on I/O and drain + * pending work. + */ + scsi_unblock_requests(ha->shost); + sas_drain_work(ha); + + /* + * Send PHYE_RESUME_TIMEOUT after sas_drain_work(). The handler + * calls sas_deform_port() -> sas_destruct_devices(), which removes + * SCSI devices and, for LLDDs using device_link() PM sync, waits + * for the host to be runtime-active. Sending it before the drain + * would deadlock: the drain waits for the handler, the handler + * waits for host resume, and host resume waits for the drain to + * finish. + */ for (i = 0; i < ha->num_phys; i++) { struct asd_sas_phy *phy = ha->sas_phy[i]; @@ -435,12 +452,6 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain) } } - /* all phys are back up or timed out, turn on i/o so we can - * flush out disks that did not return - */ - scsi_unblock_requests(ha->shost); - if (drain) - sas_drain_work(ha); clear_bit(SAS_HA_RESUMING, &ha->state); sas_queue_deferred_work(ha); @@ -449,20 +460,8 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain) */ sas_resume_insert_broadcast_ha(ha); } - -void sas_resume_ha(struct sas_ha_struct *ha) -{ - _sas_resume_ha(ha, true); -} EXPORT_SYMBOL(sas_resume_ha); -/* A no-sync variant, which does not call sas_drain_ha(). */ -void sas_resume_ha_no_sync(struct sas_ha_struct *ha) -{ - _sas_resume_ha(ha, false); -} -EXPORT_SYMBOL(sas_resume_ha_no_sync); - void sas_suspend_ha(struct sas_ha_struct *ha) { int i; diff --git a/drivers/scsi/scsi_debug.c b/drivers/scsi/scsi_debug.c index dfe38d34d051..9a194cf64528 100644 --- a/drivers/scsi/scsi_debug.c +++ b/drivers/scsi/scsi_debug.c @@ -5191,6 +5191,7 @@ static int resp_report_zones(struct scsi_cmnd *scp, u32 alloc_len, rep_opts, rep_len; bool partial; u64 lba, zs_lba; + u64 arr_len; u8 *arr = NULL, *desc; u8 *cmd = scp->cmnd; struct sdeb_zone_state *zsp = NULL; @@ -5212,9 +5213,12 @@ static int resp_report_zones(struct scsi_cmnd *scp, return check_condition_result; } - rep_max_zones = (alloc_len - 64) >> ilog2(RZONES_DESC_HD); + rep_max_zones = (ALIGN((u64)alloc_len, RZONES_DESC_HD) - RZONES_DESC_HD) >> + ilog2(RZONES_DESC_HD); + rep_max_zones = min_t(unsigned int, rep_max_zones, devip->nr_zones); + arr_len = (u64)RZONES_DESC_HD * (rep_max_zones + 1); - arr = kzalloc(alloc_len, GFP_ATOMIC | __GFP_NOWARN); + arr = kzalloc(arr_len, GFP_ATOMIC | __GFP_NOWARN); if (!arr) { mk_sense_buffer(scp, ILLEGAL_REQUEST, INSUFF_RES_ASC, INSUFF_RES_ASCQ); diff --git a/drivers/spi/spi-cadence.c b/drivers/spi/spi-cadence.c index 914384348448..a8512ae75a78 100644 --- a/drivers/spi/spi-cadence.c +++ b/drivers/spi/spi-cadence.c @@ -109,6 +109,7 @@ * @rxbuf: Pointer to the RX buffer * @tx_bytes: Number of bytes left to transfer * @rx_bytes: Number of bytes requested + * @n_bytes: Number of bytes per word * @dev_busy: Device busy flag * @is_decoded_cs: Flag for decoder property set or not * @tx_fifo_depth: Depth of the TX FIFO @@ -120,16 +121,24 @@ struct cdns_spi { struct clk *pclk; unsigned int clk_rate; u32 speed_hz; - const u8 *txbuf; - u8 *rxbuf; + const void *txbuf; + void *rxbuf; int tx_bytes; int rx_bytes; + u8 n_bytes; u8 dev_busy; u32 is_decoded_cs; unsigned int tx_fifo_depth; struct reset_control *rstc; }; +enum cdns_spi_frame_n_bytes { + CDNS_SPI_N_BYTES_NULL = 0, + CDNS_SPI_N_BYTES_U8 = 1, + CDNS_SPI_N_BYTES_U16 = 2, + CDNS_SPI_N_BYTES_U32 = 4 +}; + /* Macros for the SPI controller read/write */ static inline u32 cdns_spi_read(struct cdns_spi *xspi, u32 offset) { @@ -305,13 +314,87 @@ static int cdns_spi_setup_transfer(struct spi_device *spi, return 0; } +static u8 cdns_spi_n_bytes(struct spi_transfer *transfer) +{ + if (transfer->bits_per_word <= 8) + return CDNS_SPI_N_BYTES_U8; + else if (transfer->bits_per_word <= 16) + return CDNS_SPI_N_BYTES_U16; + else + return CDNS_SPI_N_BYTES_U32; +} + +static inline void cdns_spi_reader(struct cdns_spi *xspi) +{ + u32 rxw = 0; + + if (xspi->rxbuf && !IS_ALIGNED((uintptr_t)xspi->rxbuf, xspi->n_bytes)) { + pr_err("%s: rxbuf address is not aligned for %d bytes\n", + __func__, xspi->n_bytes); + return; + } + + rxw = cdns_spi_read(xspi, CDNS_SPI_RXD); + if (xspi->rxbuf) { + switch (xspi->n_bytes) { + case CDNS_SPI_N_BYTES_U8: + *(u8 *)xspi->rxbuf = rxw; + break; + case CDNS_SPI_N_BYTES_U16: + *(u16 *)xspi->rxbuf = rxw; + break; + case CDNS_SPI_N_BYTES_U32: + *(u32 *)xspi->rxbuf = rxw; + break; + default: + pr_err("%s invalid n_bytes %d\n", __func__, + xspi->n_bytes); + return; + } + xspi->rxbuf = (u8 *)xspi->rxbuf + xspi->n_bytes; + } +} + +static inline void cdns_spi_writer(struct cdns_spi *xspi) +{ + u32 txw = 0; + + if (xspi->txbuf && !IS_ALIGNED((uintptr_t)xspi->txbuf, xspi->n_bytes)) { + pr_err("%s: txbuf address is not aligned for %d bytes\n", + __func__, xspi->n_bytes); + return; + } + + if (xspi->txbuf) { + switch (xspi->n_bytes) { + case CDNS_SPI_N_BYTES_U8: + txw = *(u8 *)xspi->txbuf; + break; + case CDNS_SPI_N_BYTES_U16: + txw = *(u16 *)xspi->txbuf; + break; + case CDNS_SPI_N_BYTES_U32: + txw = *(u32 *)xspi->txbuf; + break; + default: + pr_err("%s invalid n_bytes %d\n", __func__, + xspi->n_bytes); + return; + } + cdns_spi_write(xspi, CDNS_SPI_TXD, txw); + xspi->txbuf = (u8 *)xspi->txbuf + xspi->n_bytes; + } +} + /** * cdns_spi_process_fifo - Fills the TX FIFO, and drain the RX FIFO + * @ctlr: Pointer to the spi_controller structure * @xspi: Pointer to the cdns_spi structure * @ntx: Number of bytes to pack into the TX FIFO * @nrx: Number of bytes to drain from the RX FIFO */ -static void cdns_spi_process_fifo(struct cdns_spi *xspi, int ntx, int nrx) +static void cdns_spi_process_fifo(struct spi_controller *ctlr, + struct cdns_spi *xspi, int ntx, int nrx) { ntx = clamp(ntx, 0, xspi->tx_bytes); nrx = clamp(nrx, 0, xspi->rx_bytes); @@ -321,23 +404,24 @@ static void cdns_spi_process_fifo(struct cdns_spi *xspi, int ntx, int nrx) while (ntx || nrx) { if (nrx) { - u8 data = cdns_spi_read(xspi, CDNS_SPI_RXD); - - if (xspi->rxbuf) - *xspi->rxbuf++ = data; - + cdns_spi_reader(xspi); nrx--; } if (ntx) { - if (xspi->txbuf) - cdns_spi_write(xspi, CDNS_SPI_TXD, *xspi->txbuf++); - else - cdns_spi_write(xspi, CDNS_SPI_TXD, 0); + /* When xspi in busy condition, bytes may send failed, + * then spi control didn't work thoroughly, add one byte + * delay. Only in host mode; in target mode this delay + * causes data corruption as the target fails to prepare + * data in time. + */ + if (!spi_controller_is_target(ctlr) && + (cdns_spi_read(xspi, CDNS_SPI_ISR) & CDNS_SPI_IXR_TXFULL)) + udelay(10); + cdns_spi_writer(xspi); ntx--; } - } } @@ -388,14 +472,14 @@ static irqreturn_t cdns_spi_irq(int irq, void *dev_id) cdns_spi_write(xspi, CDNS_SPI_THLD, 1); if (xspi->tx_bytes) { - cdns_spi_process_fifo(xspi, trans_cnt, trans_cnt); + cdns_spi_process_fifo(ctlr, xspi, trans_cnt, trans_cnt); } else { /* Fixed delay due to controller limitation with * RX_NEMPTY incorrect status * Xilinx AR:65885 contains more details */ udelay(10); - cdns_spi_process_fifo(xspi, 0, trans_cnt); + cdns_spi_process_fifo(ctlr, xspi, 0, trans_cnt); cdns_spi_write(xspi, CDNS_SPI_IDR, CDNS_SPI_IXR_DEFAULT); spi_finalize_current_transfer(ctlr); @@ -448,13 +532,11 @@ static int cdns_transfer_one(struct spi_controller *ctlr, cdns_spi_write(xspi, CDNS_SPI_THLD, xspi->tx_fifo_depth >> 1); } - /* When xspi in busy condition, bytes may send failed, - * then spi control didn't work thoroughly, add one byte delay - */ - if (cdns_spi_read(xspi, CDNS_SPI_ISR) & CDNS_SPI_IXR_TXFULL) - udelay(10); + xspi->n_bytes = cdns_spi_n_bytes(transfer); + xspi->tx_bytes = DIV_ROUND_UP(xspi->tx_bytes, xspi->n_bytes); + xspi->rx_bytes = DIV_ROUND_UP(xspi->rx_bytes, xspi->n_bytes); - cdns_spi_process_fifo(xspi, xspi->tx_fifo_depth, 0); + cdns_spi_process_fifo(ctlr, xspi, xspi->tx_fifo_depth, 0); cdns_spi_write(xspi, CDNS_SPI_IER, CDNS_SPI_IXR_DEFAULT); return transfer->len; @@ -653,6 +735,10 @@ static int cdns_spi_probe(struct platform_device *pdev) ctlr->unprepare_transfer_hardware = cdns_unprepare_transfer_hardware; ctlr->mode_bits = SPI_CPOL | SPI_CPHA; ctlr->bits_per_word_mask = SPI_BPW_MASK(8); + ctlr->flags = SPI_CONTROLLER_MUST_TX; + + if (of_device_is_compatible(pdev->dev.of_node, "cix,sky1-spi-r1p6")) + ctlr->bits_per_word_mask |= SPI_BPW_MASK(16) | SPI_BPW_MASK(32); if (!spi_controller_is_target(ctlr)) { ctlr->mode_bits |= SPI_CS_HIGH; @@ -809,6 +895,7 @@ static const struct dev_pm_ops cdns_spi_dev_pm_ops = { static const struct of_device_id cdns_spi_of_match[] = { { .compatible = "xlnx,zynq-spi-r1p6" }, + { .compatible = "cix,sky1-spi-r1p6" }, { .compatible = "cdns,spi-r1p6" }, { /* end of table */ } }; diff --git a/drivers/spi/spi-qcom-qspi.c b/drivers/spi/spi-qcom-qspi.c index 49b775134485..591fdd2ec7b8 100644 --- a/drivers/spi/spi-qcom-qspi.c +++ b/drivers/spi/spi-qcom-qspi.c @@ -759,7 +759,8 @@ static int qcom_qspi_probe(struct platform_device *pdev) return dev_err_probe(dev, ret, "could not set DMA mask\n"); host->max_speed_hz = 300000000; - host->max_dma_len = 65536; /* as per HPG */ + /* as per HPG, it is 64KB, limit to 60KB to avoid boundary condition failures */ + host->max_dma_len = 0xf000; host->dma_alignment = QSPI_ALIGN_REQ; host->num_chipselect = QSPI_NUM_CS; host->bus_num = -1; diff --git a/drivers/target/target_core_iblock.c b/drivers/target/target_core_iblock.c index c8dc92a7d63e..0a232abee8f5 100644 --- a/drivers/target/target_core_iblock.c +++ b/drivers/target/target_core_iblock.c @@ -891,7 +891,7 @@ static sense_reason_t iblock_execute_pr_out(struct se_cmd *cmd, u8 sa, u64 key, break; case PRO_PREEMPT: case PRO_PREEMPT_AND_ABORT: - if (!ops->pr_clear) { + if (!ops->pr_preempt) { pr_err("block_device does not support pr_preempt.\n"); return TCM_UNSUPPORTED_SCSI_OPCODE; } @@ -901,8 +901,8 @@ static sense_reason_t iblock_execute_pr_out(struct se_cmd *cmd, u8 sa, u64 key, sa == PRO_PREEMPT_AND_ABORT); break; case PRO_RELEASE: - if (!ops->pr_clear) { - pr_err("block_device does not support pr_pclear.\n"); + if (!ops->pr_release) { + pr_err("block_device does not support pr_release.\n"); return TCM_UNSUPPORTED_SCSI_OPCODE; } diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index cf9834f958c9..3dc49f6c7832 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -1691,6 +1691,7 @@ int target_init_cmd(struct se_cmd *se_cmd, struct se_session *se_sess, u32 data_length, int task_attr, int data_dir, int flags) { struct se_portal_group *se_tpg; + int ret; se_tpg = se_sess->se_tpg; BUG_ON(!se_tpg); @@ -1720,7 +1721,11 @@ int target_init_cmd(struct se_cmd *se_cmd, struct se_session *se_sess, * necessary for fabrics using TARGET_SCF_ACK_KREF that expect a second * kref_put() to happen during fabric packet acknowledgement. */ - return target_get_sess_cmd(se_cmd, flags & TARGET_SCF_ACK_KREF); + ret = target_get_sess_cmd(se_cmd, flags & TARGET_SCF_ACK_KREF); + if (ret) + se_cmd->cmd_cnt = NULL; + + return ret; } EXPORT_SYMBOL_GPL(target_init_cmd); @@ -1996,8 +2001,10 @@ int target_submit_tmr(struct se_cmd *se_cmd, struct se_session *se_sess, * allocation failure. */ ret = core_tmr_alloc_req(se_cmd, fabric_tmr_ptr, tm_type, gfp); - if (ret < 0) + if (ret < 0) { + se_cmd->cmd_cnt = NULL; return -ENOMEM; + } if (tm_type == TMR_ABORT_TASK) se_cmd->se_tmr_req->ref_task_tag = tag; @@ -2005,6 +2012,7 @@ int target_submit_tmr(struct se_cmd *se_cmd, struct se_session *se_sess, /* See target_submit_cmd for commentary */ ret = target_get_sess_cmd(se_cmd, flags & TARGET_SCF_ACK_KREF); if (ret) { + se_cmd->cmd_cnt = NULL; core_tmr_release_req(se_cmd->se_tmr_req); return ret; } diff --git a/drivers/thunderbolt/xdomain.c b/drivers/thunderbolt/xdomain.c index 06357073f4ab..ffe7ce073749 100644 --- a/drivers/thunderbolt/xdomain.c +++ b/drivers/thunderbolt/xdomain.c @@ -783,9 +783,13 @@ static void tb_xdp_handle_request(struct work_struct *work) * the xdomain related to this connection as well in * case there is a change in services it offers. */ - if (xd && device_is_registered(&xd->dev)) - queue_delayed_work(tb->wq, &xd->state_work, - msecs_to_jiffies(XDOMAIN_SHORT_TIMEOUT)); + if (xd) { + mutex_lock(&xd->lock); + if (!xd->removing && device_is_registered(&xd->dev)) + queue_delayed_work(tb->wq, &xd->state_work, + msecs_to_jiffies(XDOMAIN_SHORT_TIMEOUT)); + mutex_unlock(&xd->lock); + } break; case UUID_REQUEST_OLD: @@ -798,8 +802,12 @@ static void tb_xdp_handle_request(struct work_struct *work) * received UUID request from the remote host. */ if (!ret && xd && xd->state == XDOMAIN_STATE_ERROR) { - dev_dbg(&xd->dev, "restarting handshake\n"); - start_handshake(xd); + mutex_lock(&xd->lock); + if (!xd->removing) { + dev_dbg(&xd->dev, "restarting handshake\n"); + start_handshake(xd); + } + mutex_unlock(&xd->lock); } break; @@ -827,9 +835,13 @@ static void tb_xdp_handle_request(struct work_struct *work) ret = tb_xdp_link_state_change_response(ctl, route, sequence, 0); - xd->target_link_width = lsc->tlw; - queue_delayed_work(tb->wq, &xd->state_work, - msecs_to_jiffies(XDOMAIN_SHORT_TIMEOUT)); + mutex_lock(&xd->lock); + if (!xd->removing) { + xd->target_link_width = lsc->tlw; + queue_delayed_work(tb->wq, &xd->state_work, + msecs_to_jiffies(XDOMAIN_SHORT_TIMEOUT)); + } + mutex_unlock(&xd->lock); } else { tb_xdp_error_response(ctl, route, sequence, ERROR_NOT_READY); @@ -2068,6 +2080,10 @@ void tb_xdomain_remove(struct tb_xdomain *xd) { tb_xdomain_debugfs_remove(xd); + mutex_lock(&xd->lock); + xd->removing = true; + mutex_unlock(&xd->lock); + stop_handshake(xd); device_for_each_child_reverse(&xd->dev, xd, unregister_service); @@ -2464,8 +2480,12 @@ static int update_xdomain(struct device *dev, void *data) xd = tb_to_xdomain(dev); if (xd) { - queue_delayed_work(xd->tb->wq, &xd->properties_changed_work, - msecs_to_jiffies(50)); + mutex_lock(&xd->lock); + if (!xd->removing) + queue_delayed_work(xd->tb->wq, + &xd->properties_changed_work, + msecs_to_jiffies(50)); + mutex_unlock(&xd->lock); } return 0; diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 205427f6c465..98bf5bc6f383 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -9761,6 +9761,7 @@ static int __ufshcd_wl_suspend(struct ufs_hba *hba, enum ufs_pm_op pm_op) req_link_state == UIC_LINK_ACTIVE_STATE) { ufshcd_disable_auto_bkops(hba); flush_work(&hba->eeh_work); + cancel_delayed_work_sync(&hba->ufs_rtc_update_work); goto vops_suspend; } @@ -9970,10 +9971,11 @@ static int __ufshcd_wl_resume(struct ufs_hba *hba, enum ufs_pm_op pm_op) if (ret) goto set_old_link_state; ufshcd_set_timestamp_attr(hba); - schedule_delayed_work(&hba->ufs_rtc_update_work, - msecs_to_jiffies(UFS_RTC_UPDATE_INTERVAL_MS)); } + schedule_delayed_work(&hba->ufs_rtc_update_work, + msecs_to_jiffies(UFS_RTC_UPDATE_INTERVAL_MS)); + if (ufshcd_keep_autobkops_enabled_except_suspend(hba)) ufshcd_enable_auto_bkops(hba); else diff --git a/drivers/usb/gadget/function/f_tcm.c b/drivers/usb/gadget/function/f_tcm.c index cca19b465e88..c7678b076271 100644 --- a/drivers/usb/gadget/function/f_tcm.c +++ b/drivers/usb/gadget/function/f_tcm.c @@ -2014,31 +2014,158 @@ static int tcm_bind(struct usb_configuration *c, struct usb_function *f) return -ENOTSUPP; } -struct guas_setup_wq { - struct work_struct work; - struct f_uas *fu; - unsigned int alt; -}; +static void tcm_cleanup_old_alt(struct f_uas *fu) +{ + if (fu->flags & USBG_IS_UAS) + uasp_cleanup_old_alt(fu); + else if (fu->flags & USBG_IS_BOT) + bot_cleanup_old_alt(fu); + fu->flags = 0; +} + +static void tcm_delayed_set_alt_done(struct f_uas *fu) +{ + unsigned long flags; + + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + fu->delayed_set_alt_state = USBG_DELAYED_SET_ALT_IDLE; + fu->delayed_set_alt_cancel = false; + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); +} + +static bool tcm_delayed_set_alt_cancelled(struct f_uas *fu) +{ + bool cancelled; + unsigned long flags; + + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + cancelled = fu->delayed_set_alt_cancel; + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); + + return cancelled; +} + +static bool tcm_complete_delayed_status(struct f_uas *fu) +{ + struct usb_composite_dev *cdev = fu->function.config->cdev; + struct usb_request *req = cdev->req; + unsigned long cdev_flags; + bool cancelled; + int ret; + + spin_lock_irqsave(&cdev->lock, cdev_flags); + spin_lock(&fu->delayed_set_alt_lock); + cancelled = fu->delayed_set_alt_cancel; + if (!cancelled) { + fu->delayed_set_alt_state = USBG_DELAYED_SET_ALT_IDLE; + fu->delayed_set_alt_cancel = false; + } + spin_unlock(&fu->delayed_set_alt_lock); + + if (cancelled) { + spin_unlock_irqrestore(&cdev->lock, cdev_flags); + return false; + } + + if (cdev->delayed_status == 0) { + WARN(cdev, "%s: Unexpected call\n", __func__); + } else if (--cdev->delayed_status == 0) { + req->length = 0; + req->context = cdev; + ret = usb_ep_queue(cdev->gadget->ep0, req, GFP_ATOMIC); + if (ret == 0) { + cdev->setup_pending = true; + } else { + req->status = 0; + req->complete(cdev->gadget->ep0, req); + } + } + + spin_unlock_irqrestore(&cdev->lock, cdev_flags); + + return true; +} + +static bool tcm_cancel_delayed_set_alt(struct f_uas *fu) +{ + bool cleanup = false; + bool cancel = false; + unsigned long flags; + + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + switch (fu->delayed_set_alt_state) { + case USBG_DELAYED_SET_ALT_IDLE: + cleanup = true; + break; + case USBG_DELAYED_SET_ALT_QUEUED: + case USBG_DELAYED_SET_ALT_RUNNING: + fu->delayed_set_alt_cancel = true; + cancel = true; + break; + } + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); + + if (cancel && cancel_work(&fu->delayed_set_alt)) { + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + if (fu->delayed_set_alt_state == USBG_DELAYED_SET_ALT_QUEUED) { + fu->delayed_set_alt_state = USBG_DELAYED_SET_ALT_IDLE; + fu->delayed_set_alt_cancel = false; + cleanup = true; + } + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); + } + + return cleanup; +} + +static void tcm_cancel_delayed_set_alt_sync(struct f_uas *fu) +{ + unsigned long flags; + + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + if (fu->delayed_set_alt_state != USBG_DELAYED_SET_ALT_IDLE) + fu->delayed_set_alt_cancel = true; + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); + + cancel_work_sync(&fu->delayed_set_alt); + + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + fu->delayed_set_alt_state = USBG_DELAYED_SET_ALT_IDLE; + fu->delayed_set_alt_cancel = false; + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); +} static void tcm_delayed_set_alt(struct work_struct *wq) { - struct guas_setup_wq *work = container_of(wq, struct guas_setup_wq, - work); - struct f_uas *fu = work->fu; - int alt = work->alt; + struct f_uas *fu = container_of(wq, struct f_uas, delayed_set_alt); + unsigned long flags; + unsigned int alt; - kfree(work); + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + if (fu->delayed_set_alt_state != USBG_DELAYED_SET_ALT_QUEUED) { + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); + return; + } + fu->delayed_set_alt_state = USBG_DELAYED_SET_ALT_RUNNING; + alt = fu->delayed_alt; + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); - if (fu->flags & USBG_IS_BOT) - bot_cleanup_old_alt(fu); - if (fu->flags & USBG_IS_UAS) - uasp_cleanup_old_alt(fu); + tcm_cleanup_old_alt(fu); + + if (tcm_delayed_set_alt_cancelled(fu)) + goto out_done; if (alt == USB_G_ALT_INT_BBB) bot_set_alt(fu); else if (alt == USB_G_ALT_INT_UAS) uasp_set_alt(fu); - usb_composite_setup_continue(fu->function.config->cdev); + + if (tcm_complete_delayed_status(fu)) + return; + + tcm_cleanup_old_alt(fu); +out_done: + tcm_delayed_set_alt_done(fu); } static int tcm_get_alt(struct usb_function *f, unsigned intf) @@ -2064,15 +2191,20 @@ static int tcm_set_alt(struct usb_function *f, unsigned intf, unsigned alt) return -EOPNOTSUPP; if ((alt == USB_G_ALT_INT_BBB) || (alt == USB_G_ALT_INT_UAS)) { - struct guas_setup_wq *work; + unsigned long flags; - work = kmalloc(sizeof(*work), GFP_ATOMIC); - if (!work) - return -ENOMEM; - INIT_WORK(&work->work, tcm_delayed_set_alt); - work->fu = fu; - work->alt = alt; - schedule_work(&work->work); + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + if (fu->delayed_set_alt_state != USBG_DELAYED_SET_ALT_IDLE) { + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, + flags); + return -EBUSY; + } + fu->delayed_alt = alt; + fu->delayed_set_alt_cancel = false; + fu->delayed_set_alt_state = USBG_DELAYED_SET_ALT_QUEUED; + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); + + schedule_work(&fu->delayed_set_alt); return USB_GADGET_DELAYED_STATUS; } return -EOPNOTSUPP; @@ -2082,11 +2214,8 @@ static void tcm_disable(struct usb_function *f) { struct f_uas *fu = to_f_uas(f); - if (fu->flags & USBG_IS_UAS) - uasp_cleanup_old_alt(fu); - else if (fu->flags & USBG_IS_BOT) - bot_cleanup_old_alt(fu); - fu->flags = 0; + if (tcm_cancel_delayed_set_alt(fu)) + tcm_cleanup_old_alt(fu); } static int tcm_setup(struct usb_function *f, @@ -2234,11 +2363,16 @@ static void tcm_free(struct usb_function *f) { struct f_uas *tcm = to_f_uas(f); + tcm_cancel_delayed_set_alt_sync(tcm); kfree(tcm); } static void tcm_unbind(struct usb_configuration *c, struct usb_function *f) { + struct f_uas *fu = to_f_uas(f); + + tcm_cancel_delayed_set_alt_sync(fu); + tcm_cleanup_old_alt(fu); usb_free_all_descriptors(f); } @@ -2271,6 +2405,8 @@ static struct usb_function *tcm_alloc(struct usb_function_instance *fi) fu->function.disable = tcm_disable; fu->function.free_func = tcm_free; fu->tpg = tpg_instances[i].tpg; + INIT_WORK(&fu->delayed_set_alt, tcm_delayed_set_alt); + spin_lock_init(&fu->delayed_set_alt_lock); mutex_unlock(&tpg_instances_lock); return &fu->function; diff --git a/drivers/usb/gadget/function/tcm.h b/drivers/usb/gadget/function/tcm.h index 3cd565794ad7..6c9bdbeed3c8 100644 --- a/drivers/usb/gadget/function/tcm.h +++ b/drivers/usb/gadget/function/tcm.h @@ -3,6 +3,7 @@ #define __TARGET_USB_GADGET_H__ #include +#include /* #include */ #include #include @@ -26,6 +27,12 @@ enum { #define USB_G_DEFAULT_SESSION_TAGS 128 +enum { + USBG_DELAYED_SET_ALT_IDLE = 0, + USBG_DELAYED_SET_ALT_QUEUED, + USBG_DELAYED_SET_ALT_RUNNING, +}; + struct tcm_usbg_nexus { struct se_session *tvn_se_sess; }; @@ -117,6 +124,12 @@ struct f_uas { #define USBG_IS_BOT (1 << 3) #define USBG_BOT_CMD_PEND (1 << 4) + struct work_struct delayed_set_alt; + spinlock_t delayed_set_alt_lock; /* protects delayed_set_alt_* */ + unsigned int delayed_alt; + unsigned int delayed_set_alt_state; + bool delayed_set_alt_cancel; + struct usbg_cdb cmd; struct usb_ep *ep_in; struct usb_ep *ep_out; diff --git a/drivers/usb/typec/ucsi/ucsi.c b/drivers/usb/typec/ucsi/ucsi.c index 5f7196b936e2..a443efde4804 100644 --- a/drivers/usb/typec/ucsi/ucsi.c +++ b/drivers/usb/typec/ucsi/ucsi.c @@ -1569,6 +1569,7 @@ static int ucsi_register_port(struct ucsi *ucsi, struct ucsi_connector *con) INIT_WORK(&con->work, ucsi_handle_connector_change); init_completion(&con->complete); mutex_init(&con->lock); + lockdep_set_class(&con->lock, &con->lock_key); INIT_LIST_HEAD(&con->partner_tasks); con->ucsi = ucsi; @@ -1718,6 +1719,42 @@ static int ucsi_register_port(struct ucsi *ucsi, struct ucsi_connector *con) return ret; } +static void ucsi_unregister_port(struct ucsi_connector *con) +{ + struct ucsi_work *uwork; + + if (con->wq) { + mutex_lock(&con->lock); + ucsi_unregister_partner(con); + /* + * queue delayed items immediately so they can execute + * and free themselves before the wq is destroyed + */ + list_for_each_entry(uwork, &con->partner_tasks, node) { + if (cancel_delayed_work(&uwork->work)) + queue_delayed_work(con->wq, &uwork->work, 0); + } + mutex_unlock(&con->lock); + + destroy_workqueue(con->wq); + con->wq = NULL; + } else { + ucsi_unregister_partner(con); + } + + ucsi_unregister_altmodes(con, UCSI_RECIPIENT_CON); + ucsi_unregister_port_psy(con); + + usb_power_delivery_unregister_capabilities(con->port_sink_caps); + con->port_sink_caps = NULL; + usb_power_delivery_unregister_capabilities(con->port_source_caps); + con->port_source_caps = NULL; + usb_power_delivery_unregister(con->pd); + con->pd = NULL; + typec_unregister_port(con->port); + con->port = NULL; +} + static u64 ucsi_get_supported_notifications(struct ucsi *ucsi) { u16 features = ucsi->cap.features; @@ -1808,6 +1845,9 @@ static int ucsi_init(struct ucsi *ucsi) goto err_reset; } + for (i = 0; i < ucsi->cap.num_connectors; i++) + lockdep_register_key(&connector[i].lock_key); + /* Register all connectors */ for (i = 0; i < ucsi->cap.num_connectors; i++) { connector[i].num = i + 1; @@ -1837,22 +1877,11 @@ static int ucsi_init(struct ucsi *ucsi) return 0; err_unregister: - for (con = connector; con->port; con++) { - if (con->wq) - destroy_workqueue(con->wq); - ucsi_unregister_partner(con); - ucsi_unregister_altmodes(con, UCSI_RECIPIENT_CON); - ucsi_unregister_port_psy(con); - - usb_power_delivery_unregister_capabilities(con->port_sink_caps); - con->port_sink_caps = NULL; - usb_power_delivery_unregister_capabilities(con->port_source_caps); - con->port_source_caps = NULL; - usb_power_delivery_unregister(con->pd); - con->pd = NULL; - typec_unregister_port(con->port); - con->port = NULL; - } + for (con = connector; con->port; con++) + ucsi_unregister_port(con); + for (i = 0; i < ucsi->cap.num_connectors; i++) + lockdep_unregister_key(&connector[i].lock_key); + kfree(connector); err_reset: memset(&ucsi->cap, 0, sizeof(ucsi->cap)); @@ -2080,33 +2109,8 @@ void ucsi_unregister(struct ucsi *ucsi) for (i = 0; i < ucsi->cap.num_connectors; i++) { cancel_work_sync(&ucsi->connector[i].work); - - if (ucsi->connector[i].wq) { - struct ucsi_work *uwork; - - mutex_lock(&ucsi->connector[i].lock); - /* - * queue delayed items immediately so they can execute - * and free themselves before the wq is destroyed - */ - list_for_each_entry(uwork, &ucsi->connector[i].partner_tasks, node) - mod_delayed_work(ucsi->connector[i].wq, &uwork->work, 0); - mutex_unlock(&ucsi->connector[i].lock); - destroy_workqueue(ucsi->connector[i].wq); - } - - ucsi_unregister_partner(&ucsi->connector[i]); - ucsi_unregister_altmodes(&ucsi->connector[i], - UCSI_RECIPIENT_CON); - ucsi_unregister_port_psy(&ucsi->connector[i]); - - usb_power_delivery_unregister_capabilities(ucsi->connector[i].port_sink_caps); - ucsi->connector[i].port_sink_caps = NULL; - usb_power_delivery_unregister_capabilities(ucsi->connector[i].port_source_caps); - ucsi->connector[i].port_source_caps = NULL; - usb_power_delivery_unregister(ucsi->connector[i].pd); - ucsi->connector[i].pd = NULL; - typec_unregister_port(ucsi->connector[i].port); + ucsi_unregister_port(&ucsi->connector[i]); + lockdep_unregister_key(&ucsi->connector[i].lock_key); } kfree(ucsi->connector); diff --git a/drivers/usb/typec/ucsi/ucsi.h b/drivers/usb/typec/ucsi/ucsi.h index 9d0a9359ccf3..120e5a570385 100644 --- a/drivers/usb/typec/ucsi/ucsi.h +++ b/drivers/usb/typec/ucsi/ucsi.h @@ -422,6 +422,7 @@ struct ucsi_connector { struct ucsi *ucsi; struct mutex lock; /* port lock */ + struct lock_class_key lock_key; struct work_struct work; struct completion complete; struct workqueue_struct *wq; diff --git a/fs/binfmt_misc.c b/fs/binfmt_misc.c index 503eefe3dd1f..f85ee577c34d 100644 --- a/fs/binfmt_misc.c +++ b/fs/binfmt_misc.c @@ -375,6 +375,10 @@ static Node *create_entry(const char __user *buffer, size_t count) pr_debug("register: delim: %#x {%c}\n", del, del); + /* A flag-char delimiter runs the flag scan off the buffer. */ + if (del == 'P' || del == 'O' || del == 'C' || del == 'F') + goto einval; + /* Pad the buffer with the delim to simplify parsing below. */ memset(buf + count, del, 8); @@ -982,6 +986,10 @@ static int bm_fill_super(struct super_block *sb, struct fs_context *fc) if (WARN_ON(user_ns != current_user_ns())) return -EINVAL; + /* Never exec off this instance and never let anything stack on it. */ + sb->s_iflags |= SB_I_NOEXEC | SB_I_NODEV; + sb->s_stack_depth = FILESYSTEM_MAX_STACK_DEPTH; + /* * Lazily allocate a new binfmt_misc instance for this namespace, i.e. * do it here during the first mount of binfmt_misc. We don't need to diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 21419bc33863..c8ad43f94db0 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -1509,12 +1509,14 @@ static int btrfs_reconfigure(struct fs_context *fc) sync_filesystem(sb); set_bit(BTRFS_FS_STATE_REMOUNTING, &fs_info->fs_state); - if (!btrfs_check_options(fs_info, &ctx->mount_opt, fc->sb_flags)) - return -EINVAL; + if (!btrfs_check_options(fs_info, &ctx->mount_opt, fc->sb_flags)) { + ret = -EINVAL; + goto restore; + } ret = btrfs_check_features(fs_info, !(fc->sb_flags & SB_RDONLY)); if (ret < 0) - return ret; + goto restore; btrfs_ctx_to_info(fs_info, ctx); btrfs_remount_begin(fs_info, old_ctx.mount_opt, fc->sb_flags); diff --git a/fs/btrfs/zoned.c b/fs/btrfs/zoned.c index d9c26f4be663..c64840b15597 100644 --- a/fs/btrfs/zoned.c +++ b/fs/btrfs/zoned.c @@ -2122,7 +2122,11 @@ static bool check_bg_is_active(struct btrfs_eb_write_context *ctx, if (fs_info->treelog_bg == block_group->start) { if (!btrfs_zone_activate(block_group)) { - int ret_fin = btrfs_zone_finish_one_bg(fs_info); + int ret_fin; + + btrfs_zoned_meta_io_unlock(fs_info); + ret_fin = btrfs_zone_finish_one_bg(fs_info); + btrfs_zoned_meta_io_lock(fs_info); if (ret_fin != 1 || !btrfs_zone_activate(block_group)) return false; diff --git a/fs/erofs/Kconfig b/fs/erofs/Kconfig index 6ea60661fa55..d7e004f8cfed 100644 --- a/fs/erofs/Kconfig +++ b/fs/erofs/Kconfig @@ -114,6 +114,20 @@ config EROFS_FS_ZIP_LZMA If unsure, say N. +config EROFS_FS_ZIP_LZMA_DEFAULT_MAX_STREAMS + int "EROFS LZMA default maximum decompression streams" + depends on EROFS_FS_ZIP_LZMA + range 1 NR_CPUS + default 16 + help + By default EROFS allocates one LZMA decompression stream per CPU. + Each stream can hold a dictionary of up to 8 MiB taken from the + mounted image, so on systems with many CPUs this can reserve a lot + of memory. This caps the default; the lzma_streams module parameter + still overrides it. + + If unsure, keep the default of 16. + config EROFS_FS_ZIP_DEFLATE bool "EROFS DEFLATE compressed data support" depends on EROFS_FS_ZIP diff --git a/fs/erofs/decompressor_lzma.c b/fs/erofs/decompressor_lzma.c index 832cffb83a66..90bdc8638199 100644 --- a/fs/erofs/decompressor_lzma.c +++ b/fs/erofs/decompressor_lzma.c @@ -51,7 +51,8 @@ static int __init z_erofs_lzma_init(void) /* by default, use # of possible CPUs instead */ if (!z_erofs_lzma_nstrms) - z_erofs_lzma_nstrms = num_possible_cpus(); + z_erofs_lzma_nstrms = min_t(unsigned int, num_possible_cpus(), + CONFIG_EROFS_FS_ZIP_LZMA_DEFAULT_MAX_STREAMS); for (i = 0; i < z_erofs_lzma_nstrms; ++i) { struct z_erofs_lzma *strm = kzalloc(sizeof(*strm), GFP_KERNEL); diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index bc127f1ad4bb..44dc442b4f0e 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -2097,6 +2097,19 @@ static unsigned long pagemap_page_category(struct pagemap_scan_private *p, { unsigned long categories = 0; + if (pte_none(pte)) { + /* + * An unpopulated pte carries no uffd-wp marker, i.e. it is not + * write-protected, the same condition under which the present + * and swap cases below report PAGE_IS_WRITTEN. Report it here + * too so this generic path agrees with the PAGE_IS_WRITTEN fast + * path in pagemap_scan_pmd_entry(), which reports pte_none as + * written and, under PM_SCAN_WP_MATCHING, arms a marker. The + * fast path applies no VMA test, so neither does this. + */ + return PAGE_IS_WRITTEN; + } + if (pte_present(pte)) { struct page *page; @@ -2690,12 +2703,28 @@ static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end, { struct pagemap_scan_private *p = walk->private; struct vm_area_struct *vma = walk->vma; + unsigned long categories; int ret, err; - if (!vma || !pagemap_scan_is_interesting_page(p->cur_vma_category, p)) + if (!vma) + return 0; + + /* + * In a uffd-wp VMA an unpopulated range is treated as written: + * uffd-wp registration populates page tables and installs markers + * with WP_UNPOPULATED, so a missing marker means the range was + * zapped. See the pte_none() handling in pagemap_page_category(). + * + * hugetlb differs, see pagemap_hugetlb_category(). + */ + categories = p->cur_vma_category; + if (userfaultfd_wp(vma) && !is_vm_hugetlb_page(vma)) + categories |= PAGE_IS_WRITTEN; + + if (!pagemap_scan_is_interesting_page(categories, p)) return 0; - ret = pagemap_scan_output(p->cur_vma_category, p, addr, &end); + ret = pagemap_scan_output(categories, p, addr, &end); if (addr == end) return ret; diff --git a/fs/smb/client/cifssmb.c b/fs/smb/client/cifssmb.c index 81a1c116e903..882d535d17a4 100644 --- a/fs/smb/client/cifssmb.c +++ b/fs/smb/client/cifssmb.c @@ -1436,8 +1436,10 @@ CIFSSMBRead(const unsigned int xid, struct cifs_io_parms *io_parms, pSMB->hdr.PidHigh = cpu_to_le16((__u16)(pid >> 16)); /* tcon and ses pointer are checked in smb_init */ - if (tcon->ses->server == NULL) + if (!tcon->ses->server) { + cifs_small_buf_release(pSMB); return -ECONNABORTED; + } pSMB->AndXCommand = 0xFF; /* none */ pSMB->Fid = netfid; @@ -1549,8 +1551,10 @@ CIFSSMBWrite(const unsigned int xid, struct cifs_io_parms *io_parms, pSMB->hdr.PidHigh = cpu_to_le16((__u16)(pid >> 16)); /* tcon and ses pointer are checked in smb_init */ - if (tcon->ses->server == NULL) + if (!tcon->ses->server) { + cifs_buf_release(pSMB); return -ECONNABORTED; + } pSMB->AndXCommand = 0xFF; /* none */ pSMB->Fid = netfid; @@ -1816,8 +1820,10 @@ CIFSSMBWrite2(const unsigned int xid, struct cifs_io_parms *io_parms, pSMB->hdr.PidHigh = cpu_to_le16((__u16)(pid >> 16)); /* tcon and ses pointer are checked in smb_init */ - if (tcon->ses->server == NULL) + if (!tcon->ses->server) { + cifs_small_buf_release(pSMB); return -ECONNABORTED; + } pSMB->AndXCommand = 0xFF; /* none */ pSMB->Fid = netfid; diff --git a/fs/smb/server/vfs_cache.c b/fs/smb/server/vfs_cache.c index 5b6d8bb8edb2..51e37e89d1aa 100644 --- a/fs/smb/server/vfs_cache.c +++ b/fs/smb/server/vfs_cache.c @@ -487,6 +487,7 @@ int ksmbd_close_fd(struct ksmbd_work *work, u64 id) { struct ksmbd_file *fp; struct ksmbd_file_table *ft; + bool closed = false; if (!has_file_id(id)) return 0; @@ -501,6 +502,9 @@ int ksmbd_close_fd(struct ksmbd_work *work, u64 id) fp = NULL; else { fp->f_state = FP_CLOSED; + idr_remove(ft->idr, id); + fp->volatile_id = KSMBD_NO_FID; + closed = true; if (!atomic_dec_and_test(&fp->refcount)) fp = NULL; } @@ -508,7 +512,7 @@ int ksmbd_close_fd(struct ksmbd_work *work, u64 id) write_unlock(&ft->lock); if (!fp) - return -EINVAL; + return closed ? 0 : -EINVAL; __put_fd_final(work, fp); return 0; diff --git a/include/drm/drm_exec.h b/include/drm/drm_exec.h index aa786b828a0a..dee6ebdbe416 100644 --- a/include/drm/drm_exec.h +++ b/include/drm/drm_exec.h @@ -65,31 +65,35 @@ drm_exec_obj(struct drm_exec *exec, unsigned long index) return index < exec->num_objects ? exec->objects[index] : NULL; } +/* Helper for drm_exec_for_each_locked_object(). Internal use only. */ +#define __drm_exec_for_each_locked_object(exec, obj, __index) \ + for (unsigned long __index = 0; ((obj) = drm_exec_obj(exec, __index)); ++__index) /** * drm_exec_for_each_locked_object - iterate over all the locked objects * @exec: drm_exec object - * @index: unsigned long index for the iteration * @obj: the current GEM object * * Iterate over all the locked GEM objects inside the drm_exec object. */ -#define drm_exec_for_each_locked_object(exec, index, obj) \ - for ((index) = 0; ((obj) = drm_exec_obj(exec, index)); ++(index)) +#define drm_exec_for_each_locked_object(exec, obj) \ + __drm_exec_for_each_locked_object(exec, obj, __UNIQUE_ID(drm_exec)) +/* Helper for drm_exec_for_each_locked_object_reverse(). Internal use only. */ +#define __drm_exec_for_each_locked_object_reverse(exec, obj, __index) \ + for (unsigned long __index = (exec)->num_objects - 1; \ + ((obj) = drm_exec_obj(exec, __index)); --__index) /** * drm_exec_for_each_locked_object_reverse - iterate over all the locked * objects in reverse locking order * @exec: drm_exec object - * @index: unsigned long index for the iteration * @obj: the current GEM object * * Iterate over all the locked GEM objects inside the drm_exec object in - * reverse locking order. Note that @index may go below zero and wrap, + * reverse locking order. Note that the internal index may wrap around, * but that will be caught by drm_exec_obj(), returning a NULL object. */ -#define drm_exec_for_each_locked_object_reverse(exec, index, obj) \ - for ((index) = (exec)->num_objects - 1; \ - ((obj) = drm_exec_obj(exec, index)); --(index)) +#define drm_exec_for_each_locked_object_reverse(exec, obj) \ + __drm_exec_for_each_locked_object_reverse(exec, obj, __UNIQUE_ID(drm_exec)) /** * drm_exec_until_all_locked - loop until all GEM objects are locked diff --git a/include/drm/drm_fb_helper.h b/include/drm/drm_fb_helper.h index 699f2790b9ac..eb825acedf74 100644 --- a/include/drm/drm_fb_helper.h +++ b/include/drm/drm_fb_helper.h @@ -246,8 +246,6 @@ int drm_fb_helper_check_var(struct fb_var_screeninfo *var, int drm_fb_helper_restore_fbdev_mode_unlocked(struct drm_fb_helper *fb_helper); -struct fb_info *drm_fb_helper_alloc_info(struct drm_fb_helper *fb_helper); -void drm_fb_helper_release_info(struct drm_fb_helper *fb_helper); void drm_fb_helper_unregister_info(struct drm_fb_helper *fb_helper); void drm_fb_helper_fill_info(struct fb_info *info, struct drm_fb_helper *fb_helper, @@ -328,16 +326,6 @@ drm_fb_helper_restore_fbdev_mode_unlocked(struct drm_fb_helper *fb_helper) return 0; } -static inline struct fb_info * -drm_fb_helper_alloc_info(struct drm_fb_helper *fb_helper) -{ - return NULL; -} - -static inline void drm_fb_helper_release_info(struct drm_fb_helper *fb_helper) -{ -} - static inline void drm_fb_helper_unregister_info(struct drm_fb_helper *fb_helper) { } diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index e78adc7a9195..4246ed5b9703 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -309,6 +309,13 @@ struct drm_sched_fence { * @owner: job owner for debugging */ void *owner; + + /** + * @drm_client_id: + * + * The client_id of the drm_file which owns the job. + */ + uint64_t drm_client_id; }; struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f); @@ -553,7 +560,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched, void drm_sched_fini(struct drm_gpu_scheduler *sched); int drm_sched_job_init(struct drm_sched_job *job, struct drm_sched_entity *entity, - u32 credits, void *owner); + u32 credits, void *owner, + u64 drm_client_id); void drm_sched_job_arm(struct drm_sched_job *job); int drm_sched_job_add_dependency(struct drm_sched_job *job, struct dma_fence *fence); @@ -613,7 +621,7 @@ bool drm_sched_entity_is_ready(struct drm_sched_entity *entity); int drm_sched_entity_error(struct drm_sched_entity *entity); struct drm_sched_fence *drm_sched_fence_alloc( - struct drm_sched_entity *s_entity, void *owner); + struct drm_sched_entity *s_entity, void *owner, u64 drm_client_id); void drm_sched_fence_init(struct drm_sched_fence *fence, struct drm_sched_entity *entity); void drm_sched_fence_free(struct drm_sched_fence *fence); diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h index 6073a8f13c41..70025f8e2499 100644 --- a/include/linux/alloc_tag.h +++ b/include/linux/alloc_tag.h @@ -105,6 +105,8 @@ static inline bool mem_alloc_profiling_enabled(void) &mem_alloc_profiling_key); } +bool __init mem_alloc_profiling_permanently_disabled(void); + static inline struct alloc_tag_counters alloc_tag_read(struct alloc_tag *tag) { struct alloc_tag_counters v = { 0, 0 }; @@ -198,6 +200,7 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) #define DEFINE_ALLOC_TAG(_alloc_tag) static inline bool mem_alloc_profiling_enabled(void) { return false; } +static inline bool mem_alloc_profiling_permanently_disabled(void) { return true; } static inline void alloc_tag_add(union codetag_ref *ref, struct alloc_tag *tag, size_t bytes) {} static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {} diff --git a/include/linux/can/core.h b/include/linux/can/core.h index 5fb8d0e3f9c1..3287232e3cad 100644 --- a/include/linux/can/core.h +++ b/include/linux/can/core.h @@ -58,6 +58,7 @@ extern void can_rx_unregister(struct net *net, struct net_device *dev, void *data); extern int can_send(struct sk_buff *skb, int loop); +void can_set_skb_uid(struct sk_buff *skb); void can_sock_destruct(struct sock *sk); #endif /* !_CAN_CORE_H */ diff --git a/include/linux/can/skb.h b/include/linux/can/skb.h index 1abc25a8d144..869ea574a40a 100644 --- a/include/linux/can/skb.h +++ b/include/linux/can/skb.h @@ -49,13 +49,11 @@ bool can_dropped_invalid_skb(struct net_device *dev, struct sk_buff *skb); /** * struct can_skb_priv - private additional data inside CAN sk_buffs * @ifindex: ifindex of the first interface the CAN frame appeared on - * @skbcnt: atomic counter to have an unique id together with skb pointer * @frame_len: length of CAN frame in data link layer * @cf: align to the following CAN frame at skb->data */ struct can_skb_priv { int ifindex; - int skbcnt; unsigned int frame_len; struct can_frame cf[]; }; diff --git a/include/linux/dma/qcom_bam_dma.h b/include/linux/dma/qcom_bam_dma.h index 68fc0e643b1b..d9d07a9ab313 100644 --- a/include/linux/dma/qcom_bam_dma.h +++ b/include/linux/dma/qcom_bam_dma.h @@ -13,9 +13,12 @@ * supported by BAM DMA Engine. * * @cmd_and_addr - upper 8 bits command and lower 24 bits register address. - * @data - for write command: content to be written into peripheral register. - * for read command: dest addr to write peripheral register value. - * @mask - register mask. + * @data - For write command: content to be written into peripheral register. + * For read command: lower 32 bits of destination address. + * @mask - For write command: register write mask. + * For read command on BAM v1.6.0+: upper 4 bits of destination address. + * For read command on BAM < v1.6.0: ignored by hardware. + * Setting to 0 ensures 32-bit addressing compatibility. * @reserved - for future usage. * */ @@ -42,6 +45,10 @@ enum bam_command_type { * @addr: target address * @cmd: BAM command * @data: actual data for write and dest addr for read in le32 + * + * For BAM v1.6.0+, the mask field behavior depends on command type: + * - Write commands: mask = write mask (typically 0xffffffff) + * - Read commands: mask = upper 4 bits of destination address (0 for 32-bit) */ static inline void bam_prep_ce_le32(struct bam_cmd_element *bam_ce, u32 addr, @@ -50,7 +57,11 @@ bam_prep_ce_le32(struct bam_cmd_element *bam_ce, u32 addr, bam_ce->cmd_and_addr = cpu_to_le32((addr & 0xffffff) | ((cmd & 0xff) << 24)); bam_ce->data = data; - bam_ce->mask = cpu_to_le32(0xffffffff); + if (cmd == BAM_READ_COMMAND) + bam_ce->mask = cpu_to_le32(0x0); /* 32-bit addressing */ + else + bam_ce->mask = cpu_to_le32(0xffffffff); /* Write mask */ + bam_ce->reserved = 0; } /* @@ -60,7 +71,7 @@ bam_prep_ce_le32(struct bam_cmd_element *bam_ce, u32 addr, * @bam_ce: BAM command element * @addr: target address * @cmd: BAM command - * @data: actual data for write and dest addr for read + * @data: actual data for write and destination address for read */ static inline void bam_prep_ce(struct bam_cmd_element *bam_ce, u32 addr, diff --git a/include/linux/kmemleak.h b/include/linux/kmemleak.h index 6a3cd1bf4680..93a73c076d16 100644 --- a/include/linux/kmemleak.h +++ b/include/linux/kmemleak.h @@ -26,6 +26,7 @@ extern void kmemleak_free_part(const void *ptr, size_t size) __ref; extern void kmemleak_free_percpu(const void __percpu *ptr) __ref; extern void kmemleak_update_trace(const void *ptr) __ref; extern void kmemleak_not_leak(const void *ptr) __ref; +extern void kmemleak_transient_leak(const void *ptr) __ref; extern void kmemleak_ignore(const void *ptr) __ref; extern void kmemleak_scan_area(const void *ptr, size_t size, gfp_t gfp) __ref; extern void kmemleak_no_scan(const void *ptr) __ref; @@ -93,6 +94,9 @@ static inline void kmemleak_update_trace(const void *ptr) static inline void kmemleak_not_leak(const void *ptr) { } +static inline void kmemleak_transient_leak(const void *ptr) +{ +} static inline void kmemleak_ignore(const void *ptr) { } diff --git a/include/linux/libata.h b/include/linux/libata.h index a78227fb66a8..c41faa99feb3 100644 --- a/include/linux/libata.h +++ b/include/linux/libata.h @@ -402,7 +402,7 @@ enum { /* This should match the actual table size of * ata_eh_cmd_timeout_table in libata-eh.c. */ - ATA_EH_CMD_TIMEOUT_TABLE_SIZE = 8, + ATA_EH_CMD_TIMEOUT_TABLE_SIZE = 9, /* * Quirk flags: may be set by libata or controller drivers on drives. diff --git a/include/linux/mmc/sdio_ids.h b/include/linux/mmc/sdio_ids.h index 7cddfdac2f57..8fb8e7e87f6e 100644 --- a/include/linux/mmc/sdio_ids.h +++ b/include/linux/mmc/sdio_ids.h @@ -76,7 +76,7 @@ #define SDIO_DEVICE_ID_BROADCOM_43430 0xa9a6 #define SDIO_DEVICE_ID_BROADCOM_43439 0xa9af #define SDIO_DEVICE_ID_BROADCOM_43455 0xa9bf -#define SDIO_DEVICE_ID_BROADCOM_CYPRESS_43752 0xaae8 +#define SDIO_DEVICE_ID_BROADCOM_43752 0xaae8 #define SDIO_VENDOR_ID_CYPRESS 0x04b4 #define SDIO_DEVICE_ID_BROADCOM_CYPRESS_43439 0xbd3d diff --git a/include/linux/netfilter/nf_conntrack_sip.h b/include/linux/netfilter/nf_conntrack_sip.h index dbc614dfe0d5..aafa0c04f917 100644 --- a/include/linux/netfilter/nf_conntrack_sip.h +++ b/include/linux/netfilter/nf_conntrack_sip.h @@ -115,7 +115,7 @@ struct nf_nat_sip_hooks { unsigned int *datalen); void (*seq_adjust)(struct sk_buff *skb, - unsigned int protoff, s16 off); + unsigned int protoff, s32 off); unsigned int (*expect)(struct sk_buff *skb, unsigned int protoff, diff --git a/include/linux/slab.h b/include/linux/slab.h index 773843a71960..28eba6cbb767 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -206,7 +206,7 @@ enum _slab_flag_bits { #endif #define SLAB_TEMPORARY SLAB_RECLAIM_ACCOUNT /* Objects are short-lived */ -/* Slab created using create_boot_cache */ +/* Slab caches without obj_exts array */ #ifdef CONFIG_SLAB_OBJ_EXT #define SLAB_NO_OBJ_EXT __SLAB_FLAG_BIT(_SLAB_NO_OBJ_EXT) #else @@ -578,6 +578,9 @@ enum kmalloc_cache_type { #endif #ifndef CONFIG_MEMCG KMALLOC_CGROUP = KMALLOC_NORMAL, +#endif +#ifndef CONFIG_SLAB_OBJ_EXT + KMALLOC_NO_OBJ_EXT = KMALLOC_NORMAL, #endif KMALLOC_RANDOM_START = KMALLOC_NORMAL, KMALLOC_RANDOM_END = KMALLOC_RANDOM_START + RANDOM_KMALLOC_CACHES_NR, @@ -591,6 +594,9 @@ enum kmalloc_cache_type { #endif #ifdef CONFIG_MEMCG KMALLOC_CGROUP, +#endif +#ifdef CONFIG_SLAB_OBJ_EXT + KMALLOC_NO_OBJ_EXT, #endif NR_KMALLOC_TYPES }; diff --git a/include/linux/thunderbolt.h b/include/linux/thunderbolt.h index 7d902d8c054b..4ef9c4302656 100644 --- a/include/linux/thunderbolt.h +++ b/include/linux/thunderbolt.h @@ -202,6 +202,8 @@ enum tb_link_width { * @link_width: Width of the downstream facing link * @link_usb4: Downstream link is USB4 * @is_unplugged: The XDomain is unplugged + * @removing: Set by tb_xdomain_remove() under @lock to prevent + * concurrent delayed work queueing * @needs_uuid: If the XDomain does not have @remote_uuid it will be * queried first * @service_ids: Used to generate IDs for the services @@ -250,6 +252,7 @@ struct tb_xdomain { enum tb_link_width link_width; bool link_usb4; bool is_unplugged; + bool removing; bool needs_uuid; struct ida service_ids; struct ida in_hopids; diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h index 54ba231ded51..98ee10b417e6 100644 --- a/include/linux/trace_events.h +++ b/include/linux/trace_events.h @@ -327,7 +327,6 @@ void *trace_event_buffer_reserve(struct trace_event_buffer *fbuffer, void trace_event_buffer_commit(struct trace_event_buffer *fbuffer); enum { - TRACE_EVENT_FL_FILTERED_BIT, TRACE_EVENT_FL_CAP_ANY_BIT, TRACE_EVENT_FL_NO_SET_FILTER_BIT, TRACE_EVENT_FL_IGNORE_ENABLE_BIT, @@ -343,7 +342,6 @@ enum { /* * Event flags: - * FILTERED - The event has a filter attached * CAP_ANY - Any user can enable for perf * NO_SET_FILTER - Set when filter has error and is to be ignored * IGNORE_ENABLE - For trace internal events, do not enable with debugfs file @@ -359,7 +357,6 @@ enum { * TEST_STR - The event has a "%s" that points to a string outside the event */ enum { - TRACE_EVENT_FL_FILTERED = (1 << TRACE_EVENT_FL_FILTERED_BIT), TRACE_EVENT_FL_CAP_ANY = (1 << TRACE_EVENT_FL_CAP_ANY_BIT), TRACE_EVENT_FL_NO_SET_FILTER = (1 << TRACE_EVENT_FL_NO_SET_FILTER_BIT), TRACE_EVENT_FL_IGNORE_ENABLE = (1 << TRACE_EVENT_FL_IGNORE_ENABLE_BIT), @@ -385,7 +382,6 @@ struct trace_event_call { }; struct trace_event event; char *print_fmt; - struct event_filter *filter; /* * Static events can disappear with modules, * where as dynamic ones need their own ref count. diff --git a/include/net/ip_vs.h b/include/net/ip_vs.h index 08b90d33acdc..6935ec09af24 100644 --- a/include/net/ip_vs.h +++ b/include/net/ip_vs.h @@ -24,7 +24,9 @@ #include /* for union nf_inet_addr */ #include #include /* for struct ipv6hdr */ +#include #include +#include #if IS_ENABLED(CONFIG_NF_CONNTRACK) #include #endif @@ -1624,8 +1626,8 @@ int ip_vs_tunnel_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, int ip_vs_dr_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, struct ip_vs_protocol *pp, struct ip_vs_iphdr *iph); int ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, - struct ip_vs_protocol *pp, int offset, - unsigned int hooknum, struct ip_vs_iphdr *iph); + struct ip_vs_protocol *pp, unsigned int toff, + unsigned int hooknum, struct ip_vs_iphdr *ciph); void ip_vs_dest_dst_rcu_free(struct rcu_head *head); #ifdef CONFIG_IP_VS_IPV6 @@ -1638,8 +1640,8 @@ int ip_vs_tunnel_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, int ip_vs_dr_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, struct ip_vs_protocol *pp, struct ip_vs_iphdr *iph); int ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, - struct ip_vs_protocol *pp, int offset, - unsigned int hooknum, struct ip_vs_iphdr *iph); + struct ip_vs_protocol *pp, unsigned int toff, + unsigned int hooknum, struct ip_vs_iphdr *ciph); #endif #ifdef CONFIG_SYSCTL @@ -1704,15 +1706,15 @@ static inline char ip_vs_fwd_tag(struct ip_vs_conn *cp) } void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, - struct ip_vs_conn *cp, int dir); + struct ip_vs_conn *cp, int dir, unsigned int toff, + bool has_ports); #ifdef CONFIG_IP_VS_IPV6 void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, - struct ip_vs_conn *cp, int dir); + struct ip_vs_conn *cp, int dir, unsigned int toff, + bool has_ports, struct ip_vs_iphdr *ciph); #endif -__sum16 ip_vs_checksum_complete(struct sk_buff *skb, int offset); - static inline __wsum ip_vs_check_diff4(__be32 old, __be32 new, __wsum oldsum) { __be32 diff[2] = { ~old, new }; @@ -1738,6 +1740,33 @@ static inline __wsum ip_vs_check_diff2(__be16 old, __be16 new, __wsum oldsum) return csum_partial(diff, sizeof(diff), oldsum); } +static inline bool ip_vs_checksum_needed(struct sk_buff *skb, int af) +{ + /* Checksum unnecessary or already validated? */ + if (skb_csum_unnecessary(skb)) + return false; + /* LOCAL_OUT ? */ + if (!skb->dev || skb->dev->flags & IFF_LOOPBACK) + return false; + /* !LOCAL_IN (FORWARD) ? */ + if (af == AF_INET6) { + if (!(dst_rt6_info(skb_dst(skb))->rt6i_flags & RTF_LOCAL)) + return false; + } else { + if (!(skb_rtable(skb)->rt_flags & RTCF_LOCAL)) + return false; + } + return true; +} + +static inline bool ip_vs_checksum_common_check(struct sk_buff *skb, + int offset, int proto, int af) +{ + if (!ip_vs_checksum_needed(skb, af)) + return true; + return !nf_checksum(skb, NF_INET_LOCAL_IN, offset, proto, af); +} + /* Forget current conntrack (unconfirmed) and attach notrack entry */ static inline void ip_vs_notrack(struct sk_buff *skb) { diff --git a/include/net/neighbour.h b/include/net/neighbour.h index cb5f835a5d61..0272d0f0e89d 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -476,11 +476,15 @@ static inline int neigh_event_send(struct neighbour *neigh, struct sk_buff *skb) #if IS_ENABLED(CONFIG_BRIDGE_NETFILTER) static inline int neigh_hh_bridge(struct hh_cache *hh, struct sk_buff *skb) { - unsigned int seq, hh_alen; + unsigned int seq, hh_alen = HH_DATA_ALIGN(ETH_HLEN); + int err; + + err = skb_cow_head(skb, hh_alen); + if (err) + return err; do { seq = read_seqbegin(&hh->hh_lock); - hh_alen = HH_DATA_ALIGN(ETH_HLEN); memcpy(skb->data - hh_alen, hh->hh_data, ETH_ALEN + hh_alen - ETH_HLEN); } while (read_seqretry(&hh->hh_lock, seq)); return 0; diff --git a/include/net/netfilter/nf_conntrack_expect.h b/include/net/netfilter/nf_conntrack_expect.h index e9a8350e7ccf..80f50fd0f7ad 100644 --- a/include/net/netfilter/nf_conntrack_expect.h +++ b/include/net/netfilter/nf_conntrack_expect.h @@ -45,9 +45,12 @@ struct nf_conntrack_expect { void (*expectfn)(struct nf_conn *new, struct nf_conntrack_expect *this); - /* Helper to assign to new connection */ + /* Helper that created this expectation */ struct nf_conntrack_helper __rcu *helper; + /* Helper to assign to new connection */ + struct nf_conntrack_helper __rcu *assign_helper; + /* The conntrack of the master connection */ struct nf_conn *master; diff --git a/include/net/netfilter/nf_tables.h b/include/net/netfilter/nf_tables.h index 032b649658be..f4b59915a49c 100644 --- a/include/net/netfilter/nf_tables.h +++ b/include/net/netfilter/nf_tables.h @@ -1274,6 +1274,7 @@ static inline void nft_use_inc_restore(u32 *use) * @sets: sets in the table * @objects: stateful objects in the table * @flowtables: flow tables in the table + * @objname_ht: hashtable for objects lookup by name * @hgenerator: handle generator state * @handle: table handle * @use: number of chain references to this table @@ -1293,6 +1294,7 @@ struct nft_table { struct list_head sets; struct list_head objects; struct list_head flowtables; + struct rhltable objname_ht; u64 hgenerator; u64 handle; u32 use; @@ -1380,7 +1382,7 @@ static inline void *nft_obj_data(const struct nft_object *obj) #define nft_expr_obj(expr) *((struct nft_object **)nft_expr_priv(expr)) struct nft_object *nft_obj_lookup(const struct net *net, - const struct nft_table *table, + struct nft_table *table, const struct nlattr *nla, u32 objtype, u8 genmask); diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h index 1324068dd950..2e3fcae62e27 100644 --- a/include/scsi/libsas.h +++ b/include/scsi/libsas.h @@ -676,7 +676,6 @@ extern int sas_register_ha(struct sas_ha_struct *); extern int sas_unregister_ha(struct sas_ha_struct *); extern void sas_prep_resume_ha(struct sas_ha_struct *sas_ha); extern void sas_resume_ha(struct sas_ha_struct *sas_ha); -extern void sas_resume_ha_no_sync(struct sas_ha_struct *sas_ha); extern void sas_suspend_ha(struct sas_ha_struct *sas_ha); int sas_phy_reset(struct sas_phy *phy, int hard_reset); diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h index fd913d7f79b8..8c4d791507f1 100644 --- a/include/trace/events/rxrpc.h +++ b/include/trace/events/rxrpc.h @@ -1308,9 +1308,9 @@ TRACE_EVENT(rxrpc_rtt_rx, TP_PROTO(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why, int slot, rxrpc_serial_t send_serial, rxrpc_serial_t resp_serial, - u32 rtt, u32 rto), + u32 rtt, u32 srtt, u32 rto), - TP_ARGS(call, why, slot, send_serial, resp_serial, rtt, rto), + TP_ARGS(call, why, slot, send_serial, resp_serial, rtt, srtt, rto), TP_STRUCT__entry( __field(unsigned int, call) @@ -1319,7 +1319,9 @@ TRACE_EVENT(rxrpc_rtt_rx, __field(rxrpc_serial_t, send_serial) __field(rxrpc_serial_t, resp_serial) __field(u32, rtt) + __field(u32, srtt) __field(u32, rto) + __field(u32, min_rtt) ), TP_fast_assign( @@ -1329,17 +1331,21 @@ TRACE_EVENT(rxrpc_rtt_rx, __entry->send_serial = send_serial; __entry->resp_serial = resp_serial; __entry->rtt = rtt; + __entry->srtt = srtt; __entry->rto = rto; + __entry->min_rtt = minmax_get(&call->min_rtt) ), - TP_printk("c=%08x [%d] %s sr=%08x rr=%08x rtt=%u rto=%u", + TP_printk("c=%08x [%d] %s sr=%08x rr=%08x rtt=%u srtt=%u rto=%u min=%u", __entry->call, __entry->slot, __print_symbolic(__entry->why, rxrpc_rtt_rx_traces), __entry->send_serial, __entry->resp_serial, __entry->rtt, - __entry->rto) + __entry->srtt / 8, + __entry->rto, + __entry->min_rtt) ); TRACE_EVENT(rxrpc_timer_set, diff --git a/include/uapi/linux/rkisp1-config.h b/include/uapi/linux/rkisp1-config.h index 2d995f3c1ca3..963c7a055015 100644 --- a/include/uapi/linux/rkisp1-config.h +++ b/include/uapi/linux/rkisp1-config.h @@ -1487,11 +1487,11 @@ struct rkisp1_ext_params_compand_curve_config { sizeof(struct rkisp1_ext_params_compand_curve_config)) /** - * enum rksip1_ext_param_buffer_version - RkISP1 extensible parameters version + * enum rkisp1_ext_param_buffer_version - RkISP1 extensible parameters version * * @RKISP1_EXT_PARAM_BUFFER_V1: First version of RkISP1 extensible parameters */ -enum rksip1_ext_param_buffer_version { +enum rkisp1_ext_param_buffer_version { RKISP1_EXT_PARAM_BUFFER_V1 = 1, }; @@ -1563,7 +1563,7 @@ enum rksip1_ext_param_buffer_version { * +---------------------------------------------------------------------+ * * @version: The RkISP1 extensible parameters buffer version, see - * :c:type:`rksip1_ext_param_buffer_version` + * :c:type:`rkisp1_ext_param_buffer_version` * @data_size: The RkISP1 configuration data effective size, excluding this * header * @data: The RkISP1 extensible configuration data blocks diff --git a/kernel/audit.c b/kernel/audit.c index a8a0beede152..df279dd2e450 100644 --- a/kernel/audit.c +++ b/kernel/audit.c @@ -2070,7 +2070,8 @@ void audit_log_n_hex(struct audit_buffer *ab, const unsigned char *buf, void audit_log_n_string(struct audit_buffer *ab, const char *string, size_t slen) { - int avail, new_len; + int avail; + size_t new_len; unsigned char *ptr; struct sk_buff *skb; @@ -2080,7 +2081,13 @@ void audit_log_n_string(struct audit_buffer *ab, const char *string, BUG_ON(!ab->skb); skb = ab->skb; avail = skb_tailroom(skb); - new_len = slen + 3; /* enclosing quotes + null terminator */ + + /* enclosing quotes + null terminator */ + if (check_add_overflow(slen, 3, &new_len)) { + audit_log_format(ab, "?"); + return; + } + if (new_len > avail) { avail = audit_expand(ab, new_len); if (!avail) diff --git a/kernel/auditfilter.c b/kernel/auditfilter.c index 88f8ab2b2ffc..de10aeda55a4 100644 --- a/kernel/auditfilter.c +++ b/kernel/auditfilter.c @@ -1047,6 +1047,10 @@ int audit_del_rule(struct audit_entry *entry) goto out; } + list_del_rcu(&e->list); + list_del(&e->rule.list); + synchronize_rcu(); + if (e->rule.watch) audit_remove_watch_rule(&e->rule); @@ -1064,8 +1068,6 @@ int audit_del_rule(struct audit_entry *entry) audit_signals--; #endif - list_del_rcu(&e->list); - list_del(&e->rule.list); call_rcu(&e->rcu, audit_free_rule_rcu); out: diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 8c3c4ab989a0..77183b436da5 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7234,6 +7234,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn */ if (reg_type == SCALAR_VALUE) { if (is_retval && get_func_retval_range(env->prog, &range)) { + mark_reg_unknown(env, regs, value_regno); err = __mark_reg_s32_range(env, regs, value_regno, range.minval, range.maxval); if (err) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index e5ced97d9681..8c5abf67983d 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -880,8 +880,19 @@ static int sugov_start(struct cpufreq_policy *policy) memset(sg_cpu, 0, sizeof(*sg_cpu)); sg_cpu->cpu = cpu; sg_cpu->sg_policy = sg_policy; + } + + /* + * Publish the hooks only after all per-CPU data is initialized, so a + * shared policy's sugov_update_shared() never reads an uninitialized + * sibling sugov_cpu. + */ + for_each_cpu(cpu, policy->cpus) { + struct sugov_cpu *sg_cpu = &per_cpu(sugov_cpu, cpu); + cpufreq_add_update_util_hook(cpu, &sg_cpu->update_util, uu); } + return 0; } diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index cb8eff0ebd22..f5c64d1a019d 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1079,7 +1079,8 @@ static void update_dl_entity(struct sched_dl_entity *dl_se) if (dl_time_before(dl_se->deadline, rq_clock(rq)) || dl_entity_overflow(dl_se, rq_clock(rq))) { - if (unlikely((!dl_is_implicit(dl_se) || dl_se->dl_defer) && + if (unlikely((!dl_is_implicit(dl_se) || + (dl_se->dl_defer && dl_se->dl_defer_running)) && !dl_time_before(dl_se->deadline, rq_clock(rq)) && !is_dl_boosted(dl_se))) { update_dl_revised_wakeup(dl_se, rq); diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 9e5aa468d7e4..fcbeda360df3 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -591,19 +591,6 @@ int tracing_check_open_get_tr(struct trace_array *tr) return 0; } -int call_filter_check_discard(struct trace_event_call *call, void *rec, - struct trace_buffer *buffer, - struct ring_buffer_event *event) -{ - if (unlikely(call->flags & TRACE_EVENT_FL_FILTERED) && - !filter_match_preds(call->filter, rec)) { - __trace_event_discard_commit(buffer, event); - return 1; - } - - return 0; -} - /** * trace_find_filtered_pid - check if a pid exists in a filtered_pid list * @filtered_pids: The list of pids to check @@ -2894,7 +2881,6 @@ void trace_function(struct trace_array *tr, unsigned long ip, unsigned long parent_ip, unsigned int trace_ctx) { - struct trace_event_call *call = &event_function; struct trace_buffer *buffer = tr->array_buffer.buffer; struct ring_buffer_event *event; struct ftrace_entry *entry; @@ -2907,11 +2893,9 @@ trace_function(struct trace_array *tr, unsigned long ip, unsigned long entry->ip = ip; entry->parent_ip = parent_ip; - if (!call_filter_check_discard(call, entry, buffer, event)) { - if (static_branch_unlikely(&trace_function_exports_enabled)) - ftrace_exports(event, TRACE_EXPORT_FUNCTION); - __buffer_unlock_commit(buffer, event); - } + if (static_branch_unlikely(&trace_function_exports_enabled)) + ftrace_exports(event, TRACE_EXPORT_FUNCTION); + __buffer_unlock_commit(buffer, event); } #ifdef CONFIG_STACKTRACE @@ -2938,7 +2922,6 @@ static void __ftrace_trace_stack(struct trace_array *tr, unsigned int trace_ctx, int skip, struct pt_regs *regs) { - struct trace_event_call *call = &event_kernel_stack; struct ring_buffer_event *event; unsigned int size, nr_entries; struct ftrace_stack *fstack; @@ -3011,8 +2994,7 @@ static void __ftrace_trace_stack(struct trace_array *tr, memcpy(&entry->caller, fstack->calls, flex_array_size(entry, caller, nr_entries)); - if (!call_filter_check_discard(call, entry, buffer, event)) - __buffer_unlock_commit(buffer, event); + __buffer_unlock_commit(buffer, event); out: /* Again, don't let gcc optimize things here */ @@ -3086,7 +3068,6 @@ static void ftrace_trace_userstack(struct trace_array *tr, struct trace_buffer *buffer, unsigned int trace_ctx) { - struct trace_event_call *call = &event_user_stack; struct ring_buffer_event *event; struct userstack_entry *entry; @@ -3120,8 +3101,7 @@ ftrace_trace_userstack(struct trace_array *tr, memset(&entry->caller, 0, sizeof(entry->caller)); stack_trace_save_user(entry->caller, FTRACE_STACK_ENTRIES); - if (!call_filter_check_discard(call, entry, buffer, event)) - __buffer_unlock_commit(buffer, event); + __buffer_unlock_commit(buffer, event); out_drop_count: __this_cpu_dec(user_stack_count); @@ -3290,7 +3270,6 @@ static void trace_printk_start_stop_comm(int enabled) */ int trace_vbprintk(unsigned long ip, const char *fmt, va_list args) { - struct trace_event_call *call = &event_bprint; struct ring_buffer_event *event; struct trace_buffer *buffer; struct trace_array *tr = READ_ONCE(printk_trace); @@ -3334,10 +3313,8 @@ int trace_vbprintk(unsigned long ip, const char *fmt, va_list args) entry->fmt = fmt; memcpy(entry->buf, tbuffer, sizeof(u32) * len); - if (!call_filter_check_discard(call, entry, buffer, event)) { - __buffer_unlock_commit(buffer, event); - ftrace_trace_stack(tr, buffer, trace_ctx, 6, NULL); - } + __buffer_unlock_commit(buffer, event); + ftrace_trace_stack(tr, buffer, trace_ctx, 6, NULL); out: ring_buffer_nest_end(buffer); @@ -3356,7 +3333,6 @@ static __printf(3, 0) int __trace_array_vprintk(struct trace_buffer *buffer, unsigned long ip, const char *fmt, va_list args) { - struct trace_event_call *call = &event_print; struct ring_buffer_event *event; int len = 0, size; struct print_entry *entry; @@ -3391,10 +3367,8 @@ int __trace_array_vprintk(struct trace_buffer *buffer, entry->ip = ip; memcpy(&entry->buf, tbuffer, len + 1); - if (!call_filter_check_discard(call, entry, buffer, event)) { - __buffer_unlock_commit(buffer, event); - ftrace_trace_stack(printk_trace, buffer, trace_ctx, 6, NULL); - } + __buffer_unlock_commit(buffer, event); + ftrace_trace_stack(printk_trace, buffer, trace_ctx, 6, NULL); out: ring_buffer_nest_end(buffer); diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h index ed4fcc438a0b..baa39009a25d 100644 --- a/kernel/trace/trace.h +++ b/kernel/trace/trace.h @@ -1440,10 +1440,6 @@ struct trace_subsystem_dir { int nr_events; }; -extern int call_filter_check_discard(struct trace_event_call *call, void *rec, - struct trace_buffer *buffer, - struct ring_buffer_event *event); - void trace_buffer_unlock_commit_regs(struct trace_array *tr, struct trace_buffer *buffer, struct ring_buffer_event *event, diff --git a/kernel/trace/trace_branch.c b/kernel/trace/trace_branch.c index 30f72e0ecb5d..138f69521eb4 100644 --- a/kernel/trace/trace_branch.c +++ b/kernel/trace/trace_branch.c @@ -30,7 +30,6 @@ static struct trace_array *branch_tracer; static void probe_likely_condition(struct ftrace_likely_data *f, int val, int expect) { - struct trace_event_call *call = &event_branch; struct trace_array *tr = branch_tracer; struct trace_buffer *buffer; struct trace_array_cpu *data; @@ -82,8 +81,7 @@ probe_likely_condition(struct ftrace_likely_data *f, int val, int expect) entry->line = f->data.line; entry->correct = val == expect; - if (!call_filter_check_discard(call, entry, buffer, event)) - trace_buffer_unlock_commit_nostack(buffer, event); + trace_buffer_unlock_commit_nostack(buffer, event); out: current->trace_recursion &= ~TRACE_BRANCH_BIT; diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c index d07b8062ded3..1630d2b7fbc4 100644 --- a/kernel/trace/trace_events.c +++ b/kernel/trace/trace_events.c @@ -3319,8 +3319,6 @@ static void __trace_remove_event_call(struct trace_event_call *call) { event_remove(call); trace_destroy_fields(call); - free_event_filter(call->filter); - call->filter = NULL; } static int probe_remove_event_call(struct trace_event_call *call) @@ -3407,8 +3405,8 @@ static void trace_module_add_events(struct module *mod) end = mod->trace_events + mod->num_trace_events; for_each_event(call, start, end) { - __register_event(*call, mod); - __add_event_to_tracers(*call); + if (!__register_event(*call, mod)) + __add_event_to_tracers(*call); } } diff --git a/kernel/trace/trace_events_filter.c b/kernel/trace/trace_events_filter.c index 9a26ac07e324..b6fc33ddb228 100644 --- a/kernel/trace/trace_events_filter.c +++ b/kernel/trace/trace_events_filter.c @@ -1027,6 +1027,9 @@ static int regex_match_full(char *str, struct regex *r, int len) if (!len) return strcmp(str, r->pattern) == 0; + if (len < r->len) + return 0; + return strncmp(str, r->pattern, len) == 0; } diff --git a/kernel/trace/trace_functions_graph.c b/kernel/trace/trace_functions_graph.c index 47ea114ca9f4..04d403a38f04 100644 --- a/kernel/trace/trace_functions_graph.c +++ b/kernel/trace/trace_functions_graph.c @@ -102,7 +102,6 @@ int __trace_graph_entry(struct trace_array *tr, struct ftrace_graph_ent *trace, unsigned int trace_ctx) { - struct trace_event_call *call = &event_funcgraph_entry; struct ring_buffer_event *event; struct trace_buffer *buffer = tr->array_buffer.buffer; struct ftrace_graph_ent_entry *entry; @@ -113,8 +112,7 @@ int __trace_graph_entry(struct trace_array *tr, return 0; entry = ring_buffer_event_data(event); entry->graph_ent = *trace; - if (!call_filter_check_discard(call, entry, buffer, event)) - trace_buffer_unlock_commit_nostack(buffer, event); + trace_buffer_unlock_commit_nostack(buffer, event); return 1; } @@ -223,7 +221,6 @@ void __trace_graph_return(struct trace_array *tr, struct ftrace_graph_ret *trace, unsigned int trace_ctx) { - struct trace_event_call *call = &event_funcgraph_exit; struct ring_buffer_event *event; struct trace_buffer *buffer = tr->array_buffer.buffer; struct ftrace_graph_ret_entry *entry; @@ -234,8 +231,7 @@ void __trace_graph_return(struct trace_array *tr, return; entry = ring_buffer_event_data(event); entry->ret = *trace; - if (!call_filter_check_discard(call, entry, buffer, event)) - trace_buffer_unlock_commit_nostack(buffer, event); + trace_buffer_unlock_commit_nostack(buffer, event); } void trace_graph_return(struct ftrace_graph_ret *trace, diff --git a/kernel/trace/trace_hwlat.c b/kernel/trace/trace_hwlat.c index bc437b6ce896..8703eb86a20c 100644 --- a/kernel/trace/trace_hwlat.c +++ b/kernel/trace/trace_hwlat.c @@ -130,7 +130,6 @@ static bool hwlat_busy; static void trace_hwlat_sample(struct hwlat_sample *sample) { struct trace_array *tr = hwlat_trace; - struct trace_event_call *call = &event_hwlat; struct trace_buffer *buffer = tr->array_buffer.buffer; struct ring_buffer_event *event; struct hwlat_entry *entry; @@ -148,8 +147,7 @@ static void trace_hwlat_sample(struct hwlat_sample *sample) entry->nmi_count = sample->nmi_count; entry->count = sample->count; - if (!call_filter_check_discard(call, entry, buffer, event)) - trace_buffer_unlock_commit_nostack(buffer, event); + trace_buffer_unlock_commit_nostack(buffer, event); } /* Macros to encapsulate the time capturing infrastructure */ diff --git a/kernel/trace/trace_mmiotrace.c b/kernel/trace/trace_mmiotrace.c index 4d9e5c830dbe..32b7e1d972a4 100644 --- a/kernel/trace/trace_mmiotrace.c +++ b/kernel/trace/trace_mmiotrace.c @@ -29,6 +29,7 @@ static void mmio_reset_data(struct trace_array *tr) { overrun_detected = false; prev_overruns = 0; + atomic_set(&dropped_count, 0); tracing_reset_online_cpus(&tr->array_buffer); } @@ -291,15 +292,17 @@ __init static int init_mmio_trace(void) device_initcall(init_mmio_trace); static void __trace_mmiotrace_rw(struct trace_array *tr, - struct trace_array_cpu *data, struct mmiotrace_rw *rw) { - struct trace_event_call *call = &event_mmiotrace_rw; - struct trace_buffer *buffer = tr->array_buffer.buffer; + struct trace_buffer *buffer; struct ring_buffer_event *event; struct trace_mmiotrace_rw *entry; unsigned int trace_ctx; + if (!tr) + return; + + buffer = tr->array_buffer.buffer; trace_ctx = tracing_gen_ctx_flags(0); event = trace_buffer_lock_reserve(buffer, TRACE_MMIO_RW, sizeof(*entry), trace_ctx); @@ -310,27 +313,27 @@ static void __trace_mmiotrace_rw(struct trace_array *tr, entry = ring_buffer_event_data(event); entry->rw = *rw; - if (!call_filter_check_discard(call, entry, buffer, event)) - trace_buffer_unlock_commit(tr, buffer, event, trace_ctx); + trace_buffer_unlock_commit(tr, buffer, event, trace_ctx); } void mmio_trace_rw(struct mmiotrace_rw *rw) { struct trace_array *tr = mmio_trace_array; - struct trace_array_cpu *data = per_cpu_ptr(tr->array_buffer.data, smp_processor_id()); - __trace_mmiotrace_rw(tr, data, rw); + __trace_mmiotrace_rw(tr, rw); } static void __trace_mmiotrace_map(struct trace_array *tr, - struct trace_array_cpu *data, struct mmiotrace_map *map) { - struct trace_event_call *call = &event_mmiotrace_map; - struct trace_buffer *buffer = tr->array_buffer.buffer; + struct trace_buffer *buffer; struct ring_buffer_event *event; struct trace_mmiotrace_map *entry; unsigned int trace_ctx; + if (!tr) + return; + + buffer = tr->array_buffer.buffer; trace_ctx = tracing_gen_ctx_flags(0); event = trace_buffer_lock_reserve(buffer, TRACE_MMIO_MAP, sizeof(*entry), trace_ctx); @@ -341,19 +344,13 @@ static void __trace_mmiotrace_map(struct trace_array *tr, entry = ring_buffer_event_data(event); entry->map = *map; - if (!call_filter_check_discard(call, entry, buffer, event)) - trace_buffer_unlock_commit(tr, buffer, event, trace_ctx); + trace_buffer_unlock_commit(tr, buffer, event, trace_ctx); } void mmio_trace_mapping(struct mmiotrace_map *map) { struct trace_array *tr = mmio_trace_array; - struct trace_array_cpu *data; - - preempt_disable(); - data = per_cpu_ptr(tr->array_buffer.data, smp_processor_id()); - __trace_mmiotrace_map(tr, data, map); - preempt_enable(); + __trace_mmiotrace_map(tr, map); } int mmio_trace_printk(const char *fmt, va_list args) diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c index 4daedf8debdc..549ac1916dc3 100644 --- a/kernel/trace/trace_osnoise.c +++ b/kernel/trace/trace_osnoise.c @@ -501,7 +501,6 @@ static void print_osnoise_headers(struct seq_file *s) static void __trace_osnoise_sample(struct osnoise_sample *sample, struct trace_buffer *buffer) { - struct trace_event_call *call = &event_osnoise; struct ring_buffer_event *event; struct osnoise_entry *entry; @@ -519,8 +518,7 @@ __trace_osnoise_sample(struct osnoise_sample *sample, struct trace_buffer *buffe entry->softirq_count = sample->softirq_count; entry->thread_count = sample->thread_count; - if (!call_filter_check_discard(call, entry, buffer, event)) - trace_buffer_unlock_commit_nostack(buffer, event); + trace_buffer_unlock_commit_nostack(buffer, event); } /* @@ -580,7 +578,6 @@ static void print_timerlat_headers(struct seq_file *s) static void __trace_timerlat_sample(struct timerlat_sample *sample, struct trace_buffer *buffer) { - struct trace_event_call *call = &event_osnoise; struct ring_buffer_event *event; struct timerlat_entry *entry; @@ -593,8 +590,7 @@ __trace_timerlat_sample(struct timerlat_sample *sample, struct trace_buffer *buf entry->context = sample->context; entry->timer_latency = sample->timer_latency; - if (!call_filter_check_discard(call, entry, buffer, event)) - trace_buffer_unlock_commit_nostack(buffer, event); + trace_buffer_unlock_commit_nostack(buffer, event); } /* @@ -656,7 +652,6 @@ static void timerlat_save_stack(int skip) static void __timerlat_dump_stack(struct trace_buffer *buffer, struct trace_stack *fstack, unsigned int size) { - struct trace_event_call *call = &event_osnoise; struct ring_buffer_event *event; struct stack_entry *entry; @@ -670,8 +665,7 @@ __timerlat_dump_stack(struct trace_buffer *buffer, struct trace_stack *fstack, u entry->size = fstack->nr_entries; memcpy(&entry->caller, fstack->calls, size); - if (!call_filter_check_discard(call, entry, buffer, event)) - trace_buffer_unlock_commit_nostack(buffer, event); + trace_buffer_unlock_commit_nostack(buffer, event); } /* diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c index 683c30e93dd1..4d0b563fb3bd 100644 --- a/kernel/trace/trace_probe.c +++ b/kernel/trace/trace_probe.c @@ -1766,7 +1766,11 @@ const char **traceprobe_expand_meta_args(int argc, const char *argv[], ret = -ENOENT; goto error; } - /* Note: $argN starts from $arg1 */ + /* Note: $argN starts from $arg1, so $arg0 is invalid. */ + if (n == 0) { + trace_probe_log_err(0, BAD_ARG_NUM); + return ERR_PTR(-EINVAL); + } ret = sprint_nth_btf_arg(n - 1, type, buf + used, bufsize - used, ctx); if (ret < 0) diff --git a/kernel/trace/trace_sched_wakeup.c b/kernel/trace/trace_sched_wakeup.c index 039382576bc1..59e30d897a9d 100644 --- a/kernel/trace/trace_sched_wakeup.c +++ b/kernel/trace/trace_sched_wakeup.c @@ -376,7 +376,6 @@ tracing_sched_switch_trace(struct trace_array *tr, struct task_struct *next, unsigned int trace_ctx) { - struct trace_event_call *call = &event_context_switch; struct trace_buffer *buffer = tr->array_buffer.buffer; struct ring_buffer_event *event; struct ctx_switch_entry *entry; @@ -394,8 +393,7 @@ tracing_sched_switch_trace(struct trace_array *tr, entry->next_state = task_state_index(next); entry->next_cpu = task_cpu(next); - if (!call_filter_check_discard(call, entry, buffer, event)) - trace_buffer_unlock_commit(tr, buffer, event, trace_ctx); + trace_buffer_unlock_commit(tr, buffer, event, trace_ctx); } static void @@ -404,7 +402,6 @@ tracing_sched_wakeup_trace(struct trace_array *tr, struct task_struct *curr, unsigned int trace_ctx) { - struct trace_event_call *call = &event_wakeup; struct ring_buffer_event *event; struct ctx_switch_entry *entry; struct trace_buffer *buffer = tr->array_buffer.buffer; @@ -422,8 +419,7 @@ tracing_sched_wakeup_trace(struct trace_array *tr, entry->next_state = task_state_index(wakee); entry->next_cpu = task_cpu(wakee); - if (!call_filter_check_discard(call, entry, buffer, event)) - trace_buffer_unlock_commit(tr, buffer, event, trace_ctx); + trace_buffer_unlock_commit(tr, buffer, event, trace_ctx); } static void notrace diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c index e76c40bf29d0..a0d11582029e 100644 --- a/lib/alloc_tag.c +++ b/lib/alloc_tag.c @@ -183,6 +183,15 @@ static bool mem_profiling_support __meminitdata = true; static bool mem_profiling_support __meminitdata; #endif +/* + * Memory allocation profiling is permanently disabled and cannot be enabled. + * Must be called after setup_early_mem_profiling(). + */ +bool __init mem_alloc_profiling_permanently_disabled(void) +{ + return !mem_profiling_support; +} + static int __init setup_early_mem_profiling(char *str) { bool enable; diff --git a/lib/assoc_array.c b/lib/assoc_array.c index 388e656ac974..01619e88a52c 100644 --- a/lib/assoc_array.c +++ b/lib/assoc_array.c @@ -255,7 +255,8 @@ assoc_array_walk(const struct assoc_array *array, sc_segments = shortcut->index_key[sc_level >> ASSOC_ARRAY_KEY_CHUNK_SHIFT]; dissimilarity = segments ^ sc_segments; - if (round_up(sc_level, ASSOC_ARRAY_KEY_CHUNK_SIZE) > shortcut->skip_to_level) { + if (shortcut->skip_to_level < round_down(sc_level, + ASSOC_ARRAY_KEY_CHUNK_SIZE) + ASSOC_ARRAY_KEY_CHUNK_SIZE) { /* Trim segments that are beyond the shortcut */ int shift = shortcut->skip_to_level & ASSOC_ARRAY_KEY_CHUNK_MASK; dissimilarity &= ~(ULONG_MAX << shift); diff --git a/lib/rhashtable.c b/lib/rhashtable.c index 0e9a1d4cf89b..16d2ce87c22e 100644 --- a/lib/rhashtable.c +++ b/lib/rhashtable.c @@ -742,6 +742,7 @@ int rhashtable_walk_start_check(struct rhashtable_iter *iter) iter->walker.tbl = rht_dereference_rcu(ht->tbl, ht); iter->slot = 0; iter->skip = 0; + iter->p = NULL; return -EAGAIN; } diff --git a/lib/test_fortify/Makefile b/lib/test_fortify/Makefile index 1c3f82ad8bb2..afd0908b684e 100644 --- a/lib/test_fortify/Makefile +++ b/lib/test_fortify/Makefile @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 ccflags-y := $(call cc-disable-warning,fortify-source) +ccflags-y += $(call cc-disable-warning,stringop-overread) quiet_cmd_test_fortify = TEST $@ cmd_test_fortify = $(CONFIG_SHELL) $(src)/test_fortify.sh \ diff --git a/lib/win_minmax.c b/lib/win_minmax.c index ec10506834b6..1682e614309c 100644 --- a/lib/win_minmax.c +++ b/lib/win_minmax.c @@ -97,3 +97,4 @@ u32 minmax_running_min(struct minmax *m, u32 win, u32 t, u32 meas) return minmax_subwin_update(m, win, &val); } +EXPORT_SYMBOL(minmax_running_min); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index e60c21b92464..b2daa1767d90 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3206,7 +3206,7 @@ static void __split_huge_page_tail(struct folio *folio, int tail, } static void __split_huge_page(struct page *page, struct list_head *list, - pgoff_t end, unsigned int new_order) + pgoff_t end, unsigned int new_order, struct address_space *mapping) { struct folio *folio = page_folio(page); /* Scan poisoned pages when split a poisoned folio to large folios */ @@ -3303,6 +3303,16 @@ static void __split_huge_page(struct page *page, struct list_head *list, if (new_order) page = compound_head(page); + /* + * Drop the mapping while the head page is still locked and thus pins + * the inode. The loop below may free the after-split subpages -- + * including the head, when @page is a tail beyond EOF that the split + * dropped from the page cache -- which could otherwise let the inode, + * and @mapping, be freed before this unlock. + */ + if (mapping) + i_mmap_unlock_read(mapping); + for (i = 0; i < nr; i += new_nr) { struct page *subpage = head + i; struct folio *new_folio = page_folio(subpage); @@ -3580,7 +3590,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, mod_mthp_stat(order, MTHP_STAT_NR_ANON, -1); mod_mthp_stat(new_order, MTHP_STAT_NR_ANON, 1 << (order - new_order)); } - __split_huge_page(page, list, end, new_order); + __split_huge_page(page, list, end, new_order, mapping); + /* __split_huge_page() dropped the i_mmap lock */ + mapping = NULL; ret = 0; } else { spin_unlock(&ds_queue->split_queue_lock); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 90faa4fd2ca2..e910ed0c43d0 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -676,7 +676,7 @@ static int allocate_file_region_entries(struct resv_map *resv, spin_lock(&resv->lock); - list_splice(&allocated_regions, &resv->region_cache); + list_splice_init(&allocated_regions, &resv->region_cache); resv->region_cache_count += to_allocate; } @@ -5281,8 +5281,12 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, */ ; } else if (unlikely(is_hugetlb_entry_hwpoisoned(entry))) { - if (!userfaultfd_wp(dst_vma)) - entry = huge_pte_clear_uffd_wp(entry); + /* + * A hwpoison entry never carries the uffd-wp bit: it is + * installed fresh by make_hwpoison_entry() and + * hugetlb_change_protection() leaves it untouched, so + * there is nothing to clear for the child. + */ set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(is_hugetlb_entry_migration(entry))) { swp_entry_t swp_entry = pte_to_swp_entry(entry); @@ -5301,7 +5305,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, set_huge_pte_at(src, addr, src_pte, entry, sz); } if (!userfaultfd_wp(dst_vma)) - entry = huge_pte_clear_uffd_wp(entry); + entry = pte_swp_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(is_pte_marker(entry))) { pte_marker marker = copy_pte_marker( diff --git a/mm/kmemleak.c b/mm/kmemleak.c index 0aecd537645a..4092f38d3fc8 100644 --- a/mm/kmemleak.c +++ b/mm/kmemleak.c @@ -671,7 +671,7 @@ static struct kmemleak_object *__alloc_object(gfp_t gfp) atomic_set(&object->use_count, 1); object->excess_ref = 0; object->count = 0; /* white color initially */ - object->checksum = 0; + object->checksum = ~0; object->del_state = 0; /* task information */ @@ -950,6 +950,28 @@ static void make_black_object(unsigned long ptr, unsigned int objflags) paint_ptr(ptr, KMEMLEAK_BLACK, objflags); } +/* + * Reset the checksum of an object. The immediate effect is that it will not + * be reported as a leak during the next scan until its checksum is updated. + */ +static void reset_checksum(unsigned long ptr) +{ + unsigned long flags; + struct kmemleak_object *object; + + object = find_and_get_object(ptr, 0); + if (!object) { + kmemleak_warn("Not resetting the checksum of an unknown object at 0x%08lx\n", + ptr); + return; + } + + raw_spin_lock_irqsave(&object->lock, flags); + object->checksum = ~0; + raw_spin_unlock_irqrestore(&object->lock, flags); + put_object(object); +} + /* * Add a scanning area to the object. If at least one such area is added, * kmemleak will only scan these ranges rather than the whole memory block. @@ -1218,6 +1240,23 @@ void __ref kmemleak_not_leak(const void *ptr) } EXPORT_SYMBOL(kmemleak_not_leak); +/** + * kmemleak_transient_leak - mark an allocated object as transient false positive + * @ptr: pointer to beginning of the object + * + * Calling this function on an object will cause the memory block to not be + * reported as a leak temporarily. This may happen, for example, if the object + * is part of a singly linked list and the ->next reference to it is changed. + */ +void __ref kmemleak_transient_leak(const void *ptr) +{ + pr_debug("%s(0x%px)\n", __func__, ptr); + + if (kmemleak_enabled && ptr && !IS_ERR(ptr)) + reset_checksum((unsigned long)ptr); +} +EXPORT_SYMBOL(kmemleak_transient_leak); + /** * kmemleak_ignore - ignore an allocated object * @ptr: pointer to beginning of the object @@ -1343,7 +1382,8 @@ static bool update_checksum(struct kmemleak_object *object) for_each_possible_cpu(cpu) { void *ptr = per_cpu_ptr((void __percpu *)object->pointer, cpu); - object->checksum ^= crc32(0, kasan_reset_tag((void *)ptr), object->size); + object->checksum = crc32(object->checksum, + kasan_reset_tag((void *)ptr), object->size); } } else { object->checksum = crc32(0, kasan_reset_tag((void *)object->pointer), object->size); diff --git a/mm/memcontrol-v1.h b/mm/memcontrol-v1.h index 6fbc78e0e440..e4c666bff493 100644 --- a/mm/memcontrol-v1.h +++ b/mm/memcontrol-v1.h @@ -133,7 +133,11 @@ static inline void memcg1_soft_limit_reset(struct mem_cgroup *memcg) {} static inline bool memcg1_wait_acct_move(struct mem_cgroup *memcg) { return false; } static inline void memcg1_css_offline(struct mem_cgroup *memcg) {} -static inline bool memcg1_oom_prepare(struct mem_cgroup *memcg, bool *locked) { return true; } +static inline bool memcg1_oom_prepare(struct mem_cgroup *memcg, bool *locked) +{ + *locked = false; + return true; +} static inline void memcg1_oom_finish(struct mem_cgroup *memcg, bool locked) {} static inline void memcg1_oom_recover(struct mem_cgroup *memcg) {} diff --git a/mm/migrate_device.c b/mm/migrate_device.c index 5bd888223cc8..94e0b33bd99c 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -196,7 +196,8 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, bool anon_exclusive; pte_t swp_pte; - flush_cache_page(vma, addr, pte_pfn(pte)); + if (pte_present(pte)) + flush_cache_page(vma, addr, pte_pfn(pte)); anon_exclusive = folio_test_anon(folio) && PageAnonExclusive(page); if (anon_exclusive) { @@ -216,7 +217,7 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, migrate->cpages++; /* Set the dirty flag on the folio now the pte is gone. */ - if (pte_dirty(pte)) + if (pte_present(pte) && pte_dirty(pte)) folio_mark_dirty(folio); /* Setup special migration page table entry */ diff --git a/mm/mm_init.c b/mm/mm_init.c index b57385e76862..21e01537c9f5 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -1524,7 +1524,7 @@ void __ref free_area_init_core_hotplug(struct pglist_data *pgdat) { int nid = pgdat->node_id; enum zone_type z; - int cpu; + int cpu, i; pgdat_init_internals(pgdat); @@ -1542,10 +1542,17 @@ void __ref free_area_init_core_hotplug(struct pglist_data *pgdat) pgdat->node_start_pfn = 0; pgdat->node_present_pages = 0; - for_each_online_cpu(cpu) { - struct per_cpu_nodestat *p; + /* + * Hot-unplug can leave per-cpu vmstat deltas unfolded (folders skip + * offline nodes) - reconcile this at online. Foreign access to counters + * is safe: the node is not online yet and we hold the hotplug lock. + */ + for_each_possible_cpu(cpu) { + struct per_cpu_nodestat *p = per_cpu_ptr(pgdat->per_cpu_nodestats, cpu); - p = per_cpu_ptr(pgdat->per_cpu_nodestats, cpu); + for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) + if (p->vm_node_stat_diff[i]) + node_page_state_add(p->vm_node_stat_diff[i], pgdat, i); memset(p, 0, sizeof(*p)); } diff --git a/mm/page_reporting.c b/mm/page_reporting.c index e4c428e61d8c..68c791d243b1 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -80,7 +80,8 @@ __page_reporting_request(struct page_reporting_dev_info *prdev) * now we are limiting this to running no more than once every * couple of seconds. */ - schedule_delayed_work(&prdev->work, PAGE_REPORTING_DELAY); + queue_delayed_work(system_freezable_wq, &prdev->work, + PAGE_REPORTING_DELAY); } /* notify prdev of free page reporting request */ @@ -343,7 +344,8 @@ static void page_reporting_process(struct work_struct *work) */ state = atomic_cmpxchg(&prdev->state, state, PAGE_REPORTING_IDLE); if (state == PAGE_REPORTING_REQUESTED) - schedule_delayed_work(&prdev->work, PAGE_REPORTING_DELAY); + queue_delayed_work(system_freezable_wq, &prdev->work, + PAGE_REPORTING_DELAY); } static DEFINE_MUTEX(page_reporting_mutex); diff --git a/mm/percpu-km.c b/mm/percpu-km.c index fe31aa19db81..86d31190524d 100644 --- a/mm/percpu-km.c +++ b/mm/percpu-km.c @@ -75,7 +75,7 @@ static struct pcpu_chunk *pcpu_create_chunk(gfp_t gfp) chunk->base_addr = page_address(pages); spin_lock_irqsave(&pcpu_lock, flags); - pcpu_chunk_populated(chunk, 0, nr_pages); + pcpu_chunk_populated(chunk, 0, chunk->nr_pages); spin_unlock_irqrestore(&pcpu_lock, flags); pcpu_stats_chunk_alloc(); diff --git a/mm/slab.h b/mm/slab.h index b65d2462b3fd..8e9fc3efc3fa 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -413,9 +413,13 @@ static inline struct kmem_cache * kmalloc_slab(size_t size, kmem_buckets *b, gfp_t flags, unsigned long caller) { unsigned int index; + enum kmalloc_cache_type type = kmalloc_type(flags, caller); + + if (flags & __GFP_NO_OBJ_EXT) + type = KMALLOC_NO_OBJ_EXT; if (!b) - b = &kmalloc_caches[kmalloc_type(flags, caller)]; + b = &kmalloc_caches[type]; if (size <= 192) index = kmalloc_size_index[size_index_elem(size)]; else @@ -454,7 +458,8 @@ static inline bool is_kmalloc_normal(struct kmem_cache *s) { if (!is_kmalloc_cache(s)) return false; - return !(s->flags & (SLAB_CACHE_DMA|SLAB_ACCOUNT|SLAB_RECLAIM_ACCOUNT)); + + return !(s->flags & (SLAB_CACHE_DMA|SLAB_ACCOUNT|SLAB_RECLAIM_ACCOUNT|SLAB_NO_OBJ_EXT)); } /* Legal flag mask for kmem_cache_create(), for various configurations */ @@ -557,6 +562,25 @@ bool slab_in_kunit_test(void); static inline bool slab_in_kunit_test(void) { return false; } #endif +/* + * Return true if KMALLOC_NORMAL caches may need obj_exts arrays. + * + * Memory allocation profiling requires obj_exts for all caches. + * Memcg usually doesn't need them for normal kmalloc caches, but kmalloc types + * with a priority higher than KMALLOC_CGROUP can be aliased with KMALLOC_NORMAL. + */ +static __always_inline bool need_kmalloc_no_objext(void) +{ + if (!mem_alloc_profiling_permanently_disabled()) + return true; + + if (!mem_cgroup_kmem_disabled() && + (KMALLOC_NORMAL == KMALLOC_RECLAIM)) + return true; + + return false; +} + #ifdef CONFIG_SLAB_OBJ_EXT /* diff --git a/mm/slab_common.c b/mm/slab_common.c index 477fa471da18..4cc96c3ea18e 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -793,6 +793,12 @@ EXPORT_SYMBOL(kmalloc_size_roundup); #define KMALLOC_RANDOM_NAME(N, sz) #endif +#ifdef CONFIG_SLAB_OBJ_EXT +#define KMALLOC_NO_OBJ_EXT_NAME(sz) .name[KMALLOC_NO_OBJ_EXT] = "kmalloc-no-objext-" #sz, +#else +#define KMALLOC_NO_OBJ_EXT_NAME(sz) +#endif + #define INIT_KMALLOC_INFO(__size, __short_size) \ { \ .name[KMALLOC_NORMAL] = "kmalloc-" #__short_size, \ @@ -800,6 +806,7 @@ EXPORT_SYMBOL(kmalloc_size_roundup); KMALLOC_CGROUP_NAME(__short_size) \ KMALLOC_DMA_NAME(__short_size) \ KMALLOC_RANDOM_NAME(RANDOM_KMALLOC_CACHES_NR, __short_size) \ + KMALLOC_NO_OBJ_EXT_NAME(__short_size) \ .size = __size, \ } @@ -907,6 +914,12 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type) return; } flags |= SLAB_ACCOUNT; + } else if (IS_ENABLED(CONFIG_SLAB_OBJ_EXT) && type == KMALLOC_NO_OBJ_EXT) { + if (!need_kmalloc_no_objext()) { + kmalloc_caches[type][idx] = kmalloc_caches[KMALLOC_NORMAL][idx]; + return; + } + flags |= SLAB_NO_OBJ_EXT | SLAB_NO_MERGE; } else if (IS_ENABLED(CONFIG_ZONE_DMA) && (type == KMALLOC_DMA)) { flags |= SLAB_CACHE_DMA; } diff --git a/mm/slub.c b/mm/slub.c index cf678f461725..b61225c3503b 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1997,8 +1997,13 @@ int alloc_slab_obj_exts(struct slab *slab, struct kmem_cache *s, struct slabobj_ext *vec; gfp &= ~OBJCGS_CLEAR_MASK; - /* Prevent recursive extension vector allocation */ - gfp |= __GFP_NO_OBJ_EXT; + /* + * In most cases, obj_exts arrays are allocated from normal kmalloc. + * However, normal kmalloc caches must allocate them from + * KMALLOC_NO_OBJ_EXT caches to prevent recursion. + */ + if (is_kmalloc_normal(s)) + gfp |= __GFP_NO_OBJ_EXT; vec = kcalloc_node(objects, sizeof(struct slabobj_ext), gfp, slab_nid(slab)); if (!vec) { @@ -2014,6 +2019,22 @@ int alloc_slab_obj_exts(struct slab *slab, struct kmem_cache *s, return -ENOMEM; } + if (IS_ENABLED(CONFIG_DEBUG_VM)) { + struct kmem_cache *exts_cache; + struct slab *exts_slab; + + exts_slab = virt_to_slab(vec); + if (exts_slab) { + /* + * The vector must be allocated from either normal or + * KMALLOC_NO_OBJ_EXT kmalloc caches to avoid cycles. + */ + exts_cache = exts_slab->slab_cache; + WARN_ON_ONCE(!is_kmalloc_normal(exts_cache) && + !(exts_cache->flags & SLAB_NO_OBJ_EXT)); + } + } + new_exts = (unsigned long)vec; #ifdef CONFIG_MEMCG new_exts |= MEMCG_DATA_OBJEXTS; @@ -2034,7 +2055,6 @@ int alloc_slab_obj_exts(struct slab *slab, struct kmem_cache *s, * assign slabobj_exts in parallel. In this case the existing * objcg vector should be reused. */ - mark_objexts_empty(vec); kfree(vec); return 0; } else if (cmpxchg(&slab->obj_exts, old_exts, new_exts) != old_exts) { @@ -2061,14 +2081,6 @@ static inline void free_slab_obj_exts(struct slab *slab) return; } - /* - * obj_exts was created with __GFP_NO_OBJ_EXT flag, therefore its - * corresponding extension will be NULL. alloc_tag_sub() will throw a - * warning if slab has extensions but the extension of an object is - * NULL, therefore replace NULL with CODETAG_EMPTY to indicate that - * the extension for obj_exts is expected to be NULL. - */ - mark_objexts_empty(obj_exts); kfree(obj_exts); slab->obj_exts = 0; } diff --git a/net/bluetooth/hci_conn.c b/net/bluetooth/hci_conn.c index ac98e3f2344e..c5b006d9d974 100644 --- a/net/bluetooth/hci_conn.c +++ b/net/bluetooth/hci_conn.c @@ -2925,9 +2925,17 @@ static int abort_conn_sync(struct hci_dev *hdev, void *data) return hci_abort_conn_sync(hdev, conn, conn->abort_reason); } +static void abort_conn_destroy(struct hci_dev *hdev, void *data, int err) +{ + struct hci_conn *conn = data; + + hci_conn_put(conn); +} + int hci_abort_conn(struct hci_conn *conn, u8 reason) { struct hci_dev *hdev = conn->hdev; + int err; /* If abort_reason has already been set it means the connection is * already being aborted so don't attempt to overwrite it. @@ -2949,5 +2957,9 @@ int hci_abort_conn(struct hci_conn *conn, u8 reason) * as a result to MGMT_OP_DISCONNECT/MGMT_OP_UNPAIR which does * already queue its callback on cmd_sync_work. */ - return hci_cmd_sync_run_once(hdev, abort_conn_sync, conn, NULL); + err = hci_cmd_sync_run_once(hdev, abort_conn_sync, hci_conn_get(conn), + abort_conn_destroy); + if (err) + hci_conn_put(conn); + return (err == -EEXIST) ? 0 : err; } diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c index d2e8543b5a51..ec99b0812580 100644 --- a/net/bluetooth/hci_sync.c +++ b/net/bluetooth/hci_sync.c @@ -825,7 +825,7 @@ int hci_cmd_sync_run_once(struct hci_dev *hdev, hci_cmd_sync_work_func_t func, void *data, hci_cmd_sync_work_destroy_t destroy) { if (hci_cmd_sync_lookup_entry(hdev, func, data, destroy)) - return 0; + return -EEXIST; return hci_cmd_sync_run(hdev, func, data, destroy); } @@ -1228,10 +1228,11 @@ static int hci_set_adv_set_random_addr_sync(struct hci_dev *hdev, u8 instance, } static int -hci_set_ext_adv_params_sync(struct hci_dev *hdev, struct adv_info *adv, +hci_set_ext_adv_params_sync(struct hci_dev *hdev, u8 instance, const struct hci_cp_le_set_ext_adv_params *cp, struct hci_rp_le_set_ext_adv_params *rp) { + struct adv_info *adv; struct sk_buff *skb; skb = __hci_cmd_sync(hdev, HCI_OP_LE_SET_EXT_ADV_PARAMS, sizeof(*cp), @@ -1259,11 +1260,15 @@ hci_set_ext_adv_params_sync(struct hci_dev *hdev, struct adv_info *adv, if (!rp->status) { hdev->adv_addr_type = cp->own_addr_type; - if (!cp->handle) { + if (!instance) { /* Store in hdev for instance 0 */ hdev->adv_tx_power = rp->tx_power; - } else if (adv) { - adv->tx_power = rp->tx_power; + } else { + hci_dev_lock(hdev); + adv = hci_find_adv_instance(hdev, instance); + if (adv) + adv->tx_power = rp->tx_power; + hci_dev_unlock(hdev); } } @@ -1279,9 +1284,13 @@ static int hci_set_ext_adv_data_sync(struct hci_dev *hdev, u8 instance) int err; if (instance) { + hci_dev_lock(hdev); + adv = hci_find_adv_instance(hdev, instance); - if (!adv || !adv->adv_data_changed) + if (!adv || !adv->adv_data_changed) { + hci_dev_unlock(hdev); return 0; + } } len = eir_create_adv_data(hdev, instance, pdu->data, @@ -1292,16 +1301,27 @@ static int hci_set_ext_adv_data_sync(struct hci_dev *hdev, u8 instance) pdu->operation = LE_SET_ADV_DATA_OP_COMPLETE; pdu->frag_pref = LE_SET_ADV_DATA_NO_FRAG; + if (adv) { + adv->adv_data_changed = false; + hci_dev_unlock(hdev); + } + err = __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_ADV_DATA, struct_size(pdu, data, len), pdu, HCI_CMD_TIMEOUT); - if (err) + if (err) { + if (instance) { + hci_dev_lock(hdev); + adv = hci_find_adv_instance(hdev, instance); + if (adv) + adv->adv_data_changed = true; + hci_dev_unlock(hdev); + } + return err; + } - /* Update data if the command succeed */ - if (adv) { - adv->adv_data_changed = false; - } else { + if (!instance) { memcpy(hdev->adv_data, pdu->data, len); hdev->adv_data_len = len; } @@ -1355,22 +1375,22 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) struct adv_info *adv; bool secondary_adv; - if (instance > 0) { - adv = hci_find_adv_instance(hdev, instance); - if (!adv) - return -EINVAL; - } else { - adv = NULL; - } - /* Updating parameters of an active instance will return a - * Command Disallowed error, so we must first disable the - * instance if it is active. + * Command Disallowed error, so disable it before taking a snapshot. */ - if (adv) { + if (instance > 0) { err = hci_disable_ext_adv_instance_sync(hdev, instance); if (err) return err; + + hci_dev_lock(hdev); + adv = hci_find_adv_instance(hdev, instance); + if (!adv) { + hci_dev_unlock(hdev); + return -EINVAL; + } + } else { + adv = NULL; } flags = hci_adv_instance_flags(hdev, instance); @@ -1381,8 +1401,11 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) connectable = (flags & MGMT_ADV_FLAG_CONNECTABLE) || mgmt_get_connectable(hdev); - if (!is_advertising_allowed(hdev, connectable)) + if (!is_advertising_allowed(hdev, connectable)) { + if (instance) + hci_dev_unlock(hdev); return -EPERM; + } /* Set require_privacy to true only when non-connectable * advertising is used and it is not periodic. @@ -1393,8 +1416,11 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) err = hci_get_random_address(hdev, require_privacy, adv_use_rpa(hdev, flags), adv, &own_addr_type, &random_addr); - if (err < 0) + if (err < 0) { + if (instance) + hci_dev_unlock(hdev); return err; + } memset(&cp, 0, sizeof(cp)); @@ -1445,6 +1471,9 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) cp.channel_map = hdev->le_adv_channel_map; cp.handle = adv ? adv->handle : instance; + if (instance) + hci_dev_unlock(hdev); + if (flags & MGMT_ADV_FLAG_SEC_2M) { cp.primary_phy = HCI_ADV_PHY_1M; cp.secondary_phy = HCI_ADV_PHY_2M; @@ -1457,12 +1486,12 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) cp.secondary_phy = HCI_ADV_PHY_1M; } - err = hci_set_ext_adv_params_sync(hdev, adv, &cp, &rp); + err = hci_set_ext_adv_params_sync(hdev, instance, &cp, &rp); if (err) return err; /* Update adv data as tx power is known now */ - err = hci_set_ext_adv_data_sync(hdev, cp.handle); + err = hci_set_ext_adv_data_sync(hdev, instance); if (err) return err; @@ -1470,9 +1499,14 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) own_addr_type == ADDR_LE_DEV_RANDOM_RESOLVED) && bacmp(&random_addr, BDADDR_ANY)) { /* Check if random address need to be updated */ - if (adv) { - if (!bacmp(&random_addr, &adv->random_addr)) + if (instance) { + hci_dev_lock(hdev); + adv = hci_find_adv_instance(hdev, instance); + if (!adv || !bacmp(&random_addr, &adv->random_addr)) { + hci_dev_unlock(hdev); return 0; + } + hci_dev_unlock(hdev); } else { if (!bacmp(&random_addr, &hdev->random_addr)) return 0; @@ -1494,9 +1528,13 @@ static int hci_set_ext_scan_rsp_data_sync(struct hci_dev *hdev, u8 instance) int err; if (instance) { + hci_dev_lock(hdev); + adv = hci_find_adv_instance(hdev, instance); - if (!adv || !adv->scan_rsp_changed) + if (!adv || !adv->scan_rsp_changed) { + hci_dev_unlock(hdev); return 0; + } } len = eir_create_scan_rsp(hdev, instance, pdu->data); @@ -1506,15 +1544,27 @@ static int hci_set_ext_scan_rsp_data_sync(struct hci_dev *hdev, u8 instance) pdu->operation = LE_SET_ADV_DATA_OP_COMPLETE; pdu->frag_pref = LE_SET_ADV_DATA_NO_FRAG; + if (adv) { + adv->scan_rsp_changed = false; + hci_dev_unlock(hdev); + } + err = __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_SCAN_RSP_DATA, struct_size(pdu, data, len), pdu, HCI_CMD_TIMEOUT); - if (err) + if (err) { + if (instance) { + hci_dev_lock(hdev); + adv = hci_find_adv_instance(hdev, instance); + if (adv) + adv->scan_rsp_changed = true; + hci_dev_unlock(hdev); + } + return err; + } - if (adv) { - adv->scan_rsp_changed = false; - } else { + if (!instance) { memcpy(hdev->scan_rsp_data, pdu->data, len); hdev->scan_rsp_data_len = len; } @@ -1529,8 +1579,14 @@ static int __hci_set_scan_rsp_data_sync(struct hci_dev *hdev, u8 instance) memset(&cp, 0, sizeof(cp)); + if (instance) + hci_dev_lock(hdev); + len = eir_create_scan_rsp(hdev, instance, cp.data); + if (instance) + hci_dev_unlock(hdev); + if (hdev->scan_rsp_data_len == len && !memcmp(cp.data, hdev->scan_rsp_data, len)) return 0; @@ -1665,9 +1721,13 @@ static int hci_set_per_adv_data_sync(struct hci_dev *hdev, u8 instance) struct adv_info *adv = NULL; if (instance) { + hci_dev_lock(hdev); + adv = hci_find_adv_instance(hdev, instance); - if (!adv || !adv->periodic) + if (!adv || !adv->periodic) { + hci_dev_unlock(hdev); return 0; + } } len = eir_create_per_adv_data(hdev, instance, pdu->data); @@ -1676,6 +1736,9 @@ static int hci_set_per_adv_data_sync(struct hci_dev *hdev, u8 instance) pdu->handle = adv ? adv->handle : instance; pdu->operation = LE_SET_ADV_DATA_OP_COMPLETE; + if (adv) + hci_dev_unlock(hdev); + return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_PER_ADV_DATA, struct_size(pdu, data, len), pdu, HCI_CMD_TIMEOUT); @@ -6422,7 +6485,7 @@ static int hci_le_ext_directed_advertising_sync(struct hci_dev *hdev, if (err) return err; - err = hci_set_ext_adv_params_sync(hdev, NULL, &cp, &rp); + err = hci_set_ext_adv_params_sync(hdev, 0, &cp, &rp); if (err) return err; @@ -6593,7 +6656,9 @@ static int hci_le_create_conn_sync(struct hci_dev *hdev, void *data) if (hci_dev_test_flag(hdev, HCI_LE_SCAN) && hdev->le_scan_type == LE_SCAN_ACTIVE && !hci_dev_test_flag(hdev, HCI_LE_SIMULTANEOUS_ROLES)) { - hci_conn_del(conn); + conn->state = BT_OPEN; + hci_abort_conn_sync(hdev, conn, + HCI_ERROR_REJ_LIMITED_RESOURCES); hci_conn_put(conn); return -EBUSY; } diff --git a/net/bluetooth/hidp/core.c b/net/bluetooth/hidp/core.c index c0c4df8cfbc9..84c930e3ac5e 100644 --- a/net/bluetooth/hidp/core.c +++ b/net/bluetooth/hidp/core.c @@ -546,9 +546,10 @@ static int hidp_process_data(struct hidp_session *session, struct sk_buff *skb, } if (test_bit(HIDP_WAITING_FOR_RETURN, &session->flags) && - param == session->waiting_report_type) { + param == session->waiting_report_type) { if (session->waiting_report_number < 0 || - session->waiting_report_number == skb->data[0]) { + (skb->len && + session->waiting_report_number == skb->data[0])) { /* hidp_get_raw_report() is waiting on this report. */ session->report_return = skb; done_with_skb = 0; @@ -563,16 +564,18 @@ static int hidp_process_data(struct hidp_session *session, struct sk_buff *skb, static void hidp_recv_ctrl_frame(struct hidp_session *session, struct sk_buff *skb) { - unsigned char hdr, type, param; + unsigned char type, param; + u8 *hdr; int free_skb = 1; BT_DBG("session %p skb %p len %u", session, skb, skb->len); - hdr = skb->data[0]; - skb_pull(skb, 1); + hdr = skb_pull_data(skb, 1); + if (!hdr) + goto free; - type = hdr & HIDP_HEADER_TRANS_MASK; - param = hdr & HIDP_HEADER_PARAM_MASK; + type = *hdr & HIDP_HEADER_TRANS_MASK; + param = *hdr & HIDP_HEADER_PARAM_MASK; switch (type) { case HIDP_TRANS_HANDSHAKE: @@ -593,6 +596,7 @@ static void hidp_recv_ctrl_frame(struct hidp_session *session, break; } +free: if (free_skb) kfree_skb(skb); } @@ -600,14 +604,15 @@ static void hidp_recv_ctrl_frame(struct hidp_session *session, static void hidp_recv_intr_frame(struct hidp_session *session, struct sk_buff *skb) { - unsigned char hdr; + u8 *hdr; BT_DBG("session %p skb %p len %u", session, skb, skb->len); - hdr = skb->data[0]; - skb_pull(skb, 1); + hdr = skb_pull_data(skb, 1); + if (!hdr) + goto free; - if (hdr == (HIDP_TRANS_DATA | HIDP_DATA_RTYPE_INPUT)) { + if (*hdr == (HIDP_TRANS_DATA | HIDP_DATA_RTYPE_INPUT)) { hidp_set_timer(session); if (session->input) @@ -619,9 +624,10 @@ static void hidp_recv_intr_frame(struct hidp_session *session, BT_DBG("report len %d", skb->len); } } else { - BT_DBG("Unsupported protocol header 0x%02x", hdr); + BT_DBG("Unsupported protocol header 0x%02x", *hdr); } +free: kfree_skb(skb); } diff --git a/net/bluetooth/iso.c b/net/bluetooth/iso.c index 12a01dc6dbde..7d24544bb301 100644 --- a/net/bluetooth/iso.c +++ b/net/bluetooth/iso.c @@ -23,15 +23,19 @@ static struct bt_sock_list iso_sk_list = { }; /* ---- ISO connections ---- */ +enum { + ISO_CONN_DROPPED, + __ISO_CONN_NUM_FLAGS +}; + struct iso_conn { struct hci_conn *hcon; + DECLARE_BITMAP(flags, __ISO_CONN_NUM_FLAGS); /* @lock: spinlock protecting changes to iso_conn fields */ spinlock_t lock; struct sock *sk; - struct delayed_work timeout_work; - struct sk_buff *rx_skb; __u32 rx_len; __u16 tx_sn; @@ -55,6 +59,7 @@ static void iso_sock_kill(struct sock *sk); enum { BT_SK_BIG_SYNC, BT_SK_PA_SYNC, + BT_SK_KILLED, }; struct iso_pinfo { @@ -73,6 +78,7 @@ struct iso_pinfo { __u8 base_len; __u8 base[BASE_MAX_LENGTH]; struct iso_conn *conn; + struct delayed_work timeout_work; }; static struct bt_iso_qos default_qos; @@ -105,12 +111,10 @@ static void iso_conn_free(struct kref *ref) if (conn->hcon) { conn->hcon->iso_data = NULL; - hci_conn_drop(conn->hcon); + if (!test_and_set_bit(ISO_CONN_DROPPED, conn->flags)) + hci_conn_drop(conn->hcon); } - /* Ensure no more work items will run since hci_conn has been dropped */ - disable_delayed_work_sync(&conn->timeout_work); - kfree_skb(conn->rx_skb); kfree(conn); @@ -151,48 +155,45 @@ static struct sock *iso_sock_hold(struct iso_conn *conn) static void iso_sock_timeout(struct work_struct *work) { - struct iso_conn *conn = container_of(work, struct iso_conn, - timeout_work.work); - struct sock *sk; - - conn = iso_conn_hold_unless_zero(conn); - if (!conn) - return; - - iso_conn_lock(conn); - sk = iso_sock_hold(conn); - iso_conn_unlock(conn); - iso_conn_put(conn); - - if (!sk) - return; + struct iso_pinfo *pi = container_of(work, struct iso_pinfo, + timeout_work.work); + struct sock *sk = &pi->bt.sk; BT_DBG("sock %p state %d", sk, sk->sk_state); lock_sock(sk); - sk->sk_err = ETIMEDOUT; - sk->sk_state_change(sk); + if (!sock_flag(sk, SOCK_ZAPPED)) { + sk->sk_err = ETIMEDOUT; + sk->sk_state_change(sk); + } release_sock(sk); - sock_put(sk); } static void iso_sock_set_timer(struct sock *sk, long timeout) { + lockdep_assert(lockdep_sock_is_held(sk)); + + cancel_delayed_work(&iso_pi(sk)->timeout_work); + if (!iso_pi(sk)->conn) return; BT_DBG("sock %p state %d timeout %ld", sk, sk->sk_state, timeout); - cancel_delayed_work(&iso_pi(sk)->conn->timeout_work); - schedule_delayed_work(&iso_pi(sk)->conn->timeout_work, timeout); + schedule_delayed_work(&iso_pi(sk)->timeout_work, timeout); } static void iso_sock_clear_timer(struct sock *sk) { - if (!iso_pi(sk)->conn) - return; + BT_DBG("sock %p state %d", sk, sk->sk_state); + cancel_delayed_work(&iso_pi(sk)->timeout_work); +} + +static void iso_sock_disable_timer(struct sock *sk) +{ + lockdep_assert(!lockdep_sock_is_held(sk)); BT_DBG("sock %p state %d", sk, sk->sk_state); - cancel_delayed_work(&iso_pi(sk)->conn->timeout_work); + disable_delayed_work_sync(&iso_pi(sk)->timeout_work); } /* ---- ISO connections ---- */ @@ -217,7 +218,6 @@ static struct iso_conn *iso_conn_add(struct hci_conn *hcon) kref_init(&conn->ref); spin_lock_init(&conn->lock); - INIT_DELAYED_WORK(&conn->timeout_work, iso_sock_timeout); hcon->iso_data = conn; conn->hcon = hcon; @@ -282,10 +282,12 @@ static void iso_conn_del(struct hci_conn *hcon, int err) return; } + iso_sock_disable_timer(sk); + lock_sock(sk); - iso_sock_clear_timer(sk); iso_chan_del(sk, err); release_sock(sk); + iso_sock_kill(sk); sock_put(sk); } @@ -304,6 +306,7 @@ static int __iso_chan_add(struct iso_conn *conn, struct sock *sk, iso_pi(sk)->conn = conn; conn->sk = sk; + clear_bit(ISO_CONN_DROPPED, conn->flags); if (parent) bt_accept_enqueue(parent, sk, true); @@ -764,9 +767,15 @@ static void iso_sock_cleanup_listen(struct sock *parent) */ static void iso_sock_kill(struct sock *sk) { + iso_sock_disable_timer(sk); + + lock_sock(sk); + if (!sock_flag(sk, SOCK_ZAPPED) || sk->sk_socket || - sock_flag(sk, SOCK_DEAD)) + test_bit(BT_SK_KILLED, &iso_pi(sk)->flags)) { + release_sock(sk); return; + } BT_DBG("sk %p state %d", sk, sk->sk_state); @@ -780,6 +789,9 @@ static void iso_sock_kill(struct sock *sk) /* Kill poor orphan */ bt_sock_unlink(&iso_sk_list, sk); sock_set_flag(sk, SOCK_DEAD); + set_bit(BT_SK_KILLED, &iso_pi(sk)->flags); + + release_sock(sk); sock_put(sk); } @@ -809,10 +821,8 @@ static void iso_sock_disconn(struct sock *sk) } sk->sk_state = BT_DISCONN; - iso_conn_lock(iso_pi(sk)->conn); - hci_conn_drop(iso_pi(sk)->conn->hcon); - iso_pi(sk)->conn->hcon = NULL; - iso_conn_unlock(iso_pi(sk)->conn); + if (!test_and_set_bit(ISO_CONN_DROPPED, iso_pi(sk)->conn->flags)) + hci_conn_drop(iso_pi(sk)->conn->hcon); } static void __iso_sock_close(struct sock *sk) @@ -854,11 +864,11 @@ static void __iso_sock_close(struct sock *sk) /* Must be called on unlocked socket. */ static void iso_sock_close(struct sock *sk) { + iso_sock_disable_timer(sk); + lock_sock(sk); - iso_sock_clear_timer(sk); __iso_sock_close(sk); release_sock(sk); - iso_sock_kill(sk); } static void iso_sock_init(struct sock *sk, struct sock *parent) @@ -925,6 +935,8 @@ static struct sock *iso_sock_alloc(struct net *net, struct socket *sock, iso_pi(sk)->qos = default_qos; iso_pi(sk)->sync_handle = -1; + INIT_DELAYED_WORK(&iso_pi(sk)->timeout_work, iso_sock_timeout); + bt_sock_link(&iso_sk_list, sk); return sk; } @@ -1013,15 +1025,15 @@ static int iso_sock_bind_pa_sk(struct sock *sk, struct sockaddr_iso *sa, goto done; } - iso_pi(sk)->bc_num_bis = sa->iso_bc->bc_num_bis; - - for (int i = 0; i < iso_pi(sk)->bc_num_bis; i++) + for (int i = 0; i < sa->iso_bc->bc_num_bis; i++) if (sa->iso_bc->bc_bis[i] < 0x01 || sa->iso_bc->bc_bis[i] > 0x1f) { err = -EINVAL; goto done; } + iso_pi(sk)->bc_num_bis = sa->iso_bc->bc_num_bis; + memcpy(iso_pi(sk)->bc_bis, sa->iso_bc->bc_bis, iso_pi(sk)->bc_num_bis); @@ -1635,7 +1647,7 @@ static bool check_bcast_qos(struct bt_iso_qos *qos) return false; if (!qos->bcast.timeout) - qos->bcast.sync_timeout = BT_ISO_SYNC_TIMEOUT; + qos->bcast.timeout = BT_ISO_SYNC_TIMEOUT; if (qos->bcast.timeout < 0x000a || qos->bcast.timeout > 0x4000) return false; @@ -1866,8 +1878,16 @@ static int iso_sock_release(struct socket *sock) release_sock(sk); } + /* Make sure sk is valid even if iso_conn_del() is concurrent */ + sock_hold(sk); + + lock_sock(sk); sock_orphan(sk); + release_sock(sk); + iso_sock_kill(sk); + + sock_put(sk); return err; } @@ -1879,9 +1899,18 @@ static void iso_sock_ready(struct sock *sk) return; lock_sock(sk); + + switch (sk->sk_state) { + case BT_DISCONN: + case BT_CLOSED: + release_sock(sk); + return; + } + iso_sock_clear_timer(sk); sk->sk_state = BT_CONNECTED; sk->sk_state_change(sk); + release_sock(sk); } diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c index 56337f164d30..c876f986b27f 100644 --- a/net/bluetooth/l2cap_core.c +++ b/net/bluetooth/l2cap_core.c @@ -4796,6 +4796,10 @@ static int l2cap_le_connect_rsp(struct l2cap_conn *conn, if (!chan) return -EBADSLT; + chan = l2cap_chan_hold_unless_zero(chan); + if (!chan) + return -EBADSLT; + err = 0; l2cap_chan_lock(chan); @@ -4841,6 +4845,7 @@ static int l2cap_le_connect_rsp(struct l2cap_conn *conn, } l2cap_chan_unlock(chan); + l2cap_chan_put(chan); return err; } diff --git a/net/bluetooth/mgmt.c b/net/bluetooth/mgmt.c index 65d1a25c929e..13b1ecb83e0e 100644 --- a/net/bluetooth/mgmt.c +++ b/net/bluetooth/mgmt.c @@ -2617,18 +2617,28 @@ static int set_le(struct sock *sk, struct hci_dev *hdev, void *data, u16 len) static bool pending_eir_or_class(struct hci_dev *hdev) { struct mgmt_pending_cmd *cmd; + bool pending = false; + + mutex_lock(&hdev->mgmt_pending_lock); list_for_each_entry(cmd, &hdev->mgmt_pending, list) { switch (cmd->opcode) { case MGMT_OP_ADD_UUID: case MGMT_OP_REMOVE_UUID: case MGMT_OP_SET_DEV_CLASS: + case MGMT_OP_SET_LOCAL_NAME: case MGMT_OP_SET_POWERED: - return true; + pending = true; + break; } + + if (pending) + break; } - return false; + mutex_unlock(&hdev->mgmt_pending_lock); + + return pending; } static const u8 bluetooth_base_uuid[] = { @@ -3435,11 +3445,13 @@ static int set_io_capability(struct sock *sk, struct hci_dev *hdev, void *data, NULL, 0); } -static struct mgmt_pending_cmd *find_pairing(struct hci_conn *conn) +static struct mgmt_pending_cmd *remove_pairing(struct hci_conn *conn) { struct hci_dev *hdev = conn->hdev; struct mgmt_pending_cmd *cmd; + mutex_lock(&hdev->mgmt_pending_lock); + list_for_each_entry(cmd, &hdev->mgmt_pending, list) { if (cmd->opcode != MGMT_OP_PAIR_DEVICE) continue; @@ -3447,9 +3459,39 @@ static struct mgmt_pending_cmd *find_pairing(struct hci_conn *conn) if (cmd->user_data != conn) continue; + list_del(&cmd->list); + mutex_unlock(&hdev->mgmt_pending_lock); return cmd; } + mutex_unlock(&hdev->mgmt_pending_lock); + + return NULL; +} + +static struct mgmt_pending_cmd *remove_pairing_by_addr(struct hci_dev *hdev, + bdaddr_t *bdaddr) +{ + struct mgmt_pending_cmd *cmd; + struct hci_conn *conn; + + mutex_lock(&hdev->mgmt_pending_lock); + + list_for_each_entry(cmd, &hdev->mgmt_pending, list) { + if (cmd->opcode != MGMT_OP_PAIR_DEVICE) + continue; + + conn = cmd->user_data; + if (bacmp(bdaddr, &conn->dst) != 0) + continue; + + list_del(&cmd->list); + mutex_unlock(&hdev->mgmt_pending_lock); + return cmd; + } + + mutex_unlock(&hdev->mgmt_pending_lock); + return NULL; } @@ -3487,10 +3529,10 @@ void mgmt_smp_complete(struct hci_conn *conn, bool complete) u8 status = complete ? MGMT_STATUS_SUCCESS : MGMT_STATUS_FAILED; struct mgmt_pending_cmd *cmd; - cmd = find_pairing(conn); + cmd = remove_pairing(conn); if (cmd) { cmd->cmd_complete(cmd, status); - mgmt_pending_remove(cmd); + mgmt_pending_free(cmd); } } @@ -3500,14 +3542,14 @@ static void pairing_complete_cb(struct hci_conn *conn, u8 status) BT_DBG("status %u", status); - cmd = find_pairing(conn); + cmd = remove_pairing(conn); if (!cmd) { BT_DBG("Unable to find a pending command"); return; } cmd->cmd_complete(cmd, mgmt_status(status)); - mgmt_pending_remove(cmd); + mgmt_pending_free(cmd); } static void le_pairing_complete_cb(struct hci_conn *conn, u8 status) @@ -3519,14 +3561,14 @@ static void le_pairing_complete_cb(struct hci_conn *conn, u8 status) if (!status) return; - cmd = find_pairing(conn); + cmd = remove_pairing(conn); if (!cmd) { BT_DBG("Unable to find a pending command"); return; } cmd->cmd_complete(cmd, mgmt_status(status)); - mgmt_pending_remove(cmd); + mgmt_pending_free(cmd); } static int pair_device(struct sock *sk, struct hci_dev *hdev, void *data, @@ -3683,23 +3725,17 @@ static int cancel_pair_device(struct sock *sk, struct hci_dev *hdev, void *data, goto unlock; } - cmd = pending_find(MGMT_OP_PAIR_DEVICE, hdev); + cmd = remove_pairing_by_addr(hdev, &addr->bdaddr); if (!cmd) { err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_CANCEL_PAIR_DEVICE, MGMT_STATUS_INVALID_PARAMS); goto unlock; } - conn = cmd->user_data; - - if (bacmp(&addr->bdaddr, &conn->dst) != 0) { - err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_CANCEL_PAIR_DEVICE, - MGMT_STATUS_INVALID_PARAMS); - goto unlock; - } + conn = hci_conn_get(cmd->user_data); cmd->cmd_complete(cmd, MGMT_STATUS_CANCELLED); - mgmt_pending_remove(cmd); + mgmt_pending_free(cmd); err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_CANCEL_PAIR_DEVICE, 0, addr, sizeof(*addr)); @@ -3717,6 +3753,8 @@ static int cancel_pair_device(struct sock *sk, struct hci_dev *hdev, void *data, if (conn->conn_reason == CONN_REASON_PAIR_DEVICE) hci_abort_conn(conn, HCI_ERROR_REMOTE_USER_TERM); + hci_conn_put(conn); + unlock: hci_dev_unlock(hdev); return err; @@ -3964,6 +4002,12 @@ static int set_local_name(struct sock *sk, struct hci_dev *hdev, void *data, goto failed; } + if (hdev_is_powered(hdev) && pending_eir_or_class(hdev)) { + err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_LOCAL_NAME, + MGMT_STATUS_BUSY); + goto failed; + } + memcpy(hdev->short_name, cp->short_name, sizeof(hdev->short_name)); if (!hdev_is_powered(hdev)) { @@ -10205,14 +10249,14 @@ void mgmt_auth_failed(struct hci_conn *conn, u8 hci_status) ev.addr.type = link_to_bdaddr(conn->type, conn->dst_type); ev.status = status; - cmd = find_pairing(conn); + cmd = remove_pairing(conn); mgmt_event(MGMT_EV_AUTH_FAILED, conn->hdev, &ev, sizeof(ev), cmd ? cmd->sk : NULL); if (cmd) { cmd->cmd_complete(cmd, status); - mgmt_pending_remove(cmd); + mgmt_pending_free(cmd); } } diff --git a/net/bridge/br_mrp.c b/net/bridge/br_mrp.c index fd2de35ffb3c..5fd22bb4f5b6 100644 --- a/net/bridge/br_mrp.c +++ b/net/bridge/br_mrp.c @@ -215,7 +215,7 @@ static struct sk_buff *br_mrp_alloc_test_skb(struct br_mrp *mrp, struct br_mrp_oui_hdr *oui = NULL; u8 length; - length = sizeof(*sub_opt) + sizeof(*sub_tlv) + sizeof(oui) + + length = sizeof(*sub_opt) + sizeof(*sub_tlv) + sizeof(*oui) + MRP_OPT_PADDING; br_mrp_skb_tlv(skb, BR_MRP_TLV_HEADER_OPTION, length); diff --git a/net/bridge/br_multicast.c b/net/bridge/br_multicast.c index 3194344529a5..d43b39ed61c7 100644 --- a/net/bridge/br_multicast.c +++ b/net/bridge/br_multicast.c @@ -3685,6 +3685,7 @@ br_multicast_leave_group(struct net_bridge_mcast *brmctx, p->flags |= MDB_PG_FLAGS_FAST_LEAVE; br_multicast_del_pg(mp, p, pp); + break; } goto out; } diff --git a/net/bridge/br_netfilter_hooks.c b/net/bridge/br_netfilter_hooks.c index 5ad3f3ef4ca7..c550907b92dd 100644 --- a/net/bridge/br_netfilter_hooks.c +++ b/net/bridge/br_netfilter_hooks.c @@ -296,7 +296,11 @@ int br_nf_pre_routing_finish_bridge(struct net *net, struct sock *sk, struct sk_ goto free_skb; } - neigh_hh_bridge(&neigh->hh, skb); + if (neigh_hh_bridge(&neigh->hh, skb)) { + neigh_release(neigh); + goto free_skb; + } + skb->dev = br_indev; ret = br_handle_frame_finish(net, sk, skb); diff --git a/net/can/af_can.c b/net/can/af_can.c index c86cbb3ed9c5..c2c6372f56b7 100644 --- a/net/can/af_can.c +++ b/net/can/af_can.c @@ -639,6 +639,16 @@ static int can_rcv_filter(struct can_dev_rcv_lists *dev_rcv_lists, struct sk_buf return matches; } +void can_set_skb_uid(struct sk_buff *skb) +{ + /* create non-zero unique skb identifier together with *skb */ + while (!(skb->hash)) + skb->hash = atomic_inc_return(&skbcounter); + + skb->sw_hash = 1; +} +EXPORT_SYMBOL(can_set_skb_uid); + static void can_receive(struct sk_buff *skb, struct net_device *dev) { struct can_dev_rcv_lists *dev_rcv_lists; @@ -650,9 +660,7 @@ static void can_receive(struct sk_buff *skb, struct net_device *dev) atomic_long_inc(&pkg_stats->rx_frames); atomic_long_inc(&pkg_stats->rx_frames_delta); - /* create non-zero unique skb identifier together with *skb */ - while (!(can_skb_prv(skb)->skbcnt)) - can_skb_prv(skb)->skbcnt = atomic_inc_return(&skbcounter); + can_set_skb_uid(skb); rcu_read_lock(); diff --git a/net/can/bcm.c b/net/can/bcm.c index 0c7e6203eb44..d35a39d55edd 100644 --- a/net/can/bcm.c +++ b/net/can/bcm.c @@ -350,7 +350,6 @@ static void bcm_can_tx(struct bcm_op *op, struct canfd_frame *cf) can_skb_reserve(skb); can_skb_prv(skb)->ifindex = dev->ifindex; - can_skb_prv(skb)->skbcnt = 0; skb_put_data(skb, cf, op->cfsiz); @@ -1624,7 +1623,6 @@ static int bcm_tx_send(struct msghdr *msg, int ifindex, struct sock *sk, } can_skb_prv(skb)->ifindex = dev->ifindex; - can_skb_prv(skb)->skbcnt = 0; skb->dev = dev; can_skb_set_owner(skb, sk); err = can_send(skb, 1); /* send with loopback */ diff --git a/net/can/isotp.c b/net/can/isotp.c index b2622c881aa3..5792230e5309 100644 --- a/net/can/isotp.c +++ b/net/can/isotp.c @@ -126,6 +126,15 @@ MODULE_PARM_DESC(max_pdu_size, "maximum isotp pdu size (default " #define ISOTP_FC_TIMEOUT 1 /* 1 sec */ #define ISOTP_ECHO_TIMEOUT 2 /* 2 secs */ +/* so->tx_result[so->tx_gen % ISOTP_TX_RESULT_SLOTS] holds the packed value + * (err << ISOTP_TX_RESULT_GEN_BITS | gen) for each tx generation slot, so it + * can be handled with a single READ_ONCE()/WRITE_ONCE() access. + */ +#define ISOTP_TX_RESULT_SLOTS 4 +#define ISOTP_TX_RESULT_GEN_BITS 24 +#define ISOTP_TX_RESULT_GEN_MASK ((1U << ISOTP_TX_RESULT_GEN_BITS) - 1) +#define ISOTP_TX_RESULT_ERR_MASK 0xFF + enum { ISOTP_IDLE = 0, ISOTP_WAIT_FIRST_FC, @@ -164,7 +173,8 @@ struct isotp_sock { u32 force_tx_stmin; u32 force_rx_stmin; u32 cfecho; /* consecutive frame echo tag */ - u32 tx_gen; /* generation, bumped per new tx transfer */ + u32 tx_gen; /* transfer generation, increased per new tx transfer */ + u32 tx_result[ISOTP_TX_RESULT_SLOTS]; /* per-generation result slots */ struct tpcon rx, tx; struct list_head notifier; wait_queue_head_t wait; @@ -175,6 +185,65 @@ static LIST_HEAD(isotp_notifier_list); static DEFINE_SPINLOCK(isotp_notifier_lock); static struct isotp_sock *isotp_busy_notifier; +/* increase (24 bit) tx generation value */ +static u32 isotp_inc_tx_gen(u32 gen) +{ + return (gen + 1) & ISOTP_TX_RESULT_GEN_MASK; +} + +/* store 8 bit error and 24 bit tx generation values in packed u32 element */ +static u32 isotp_pack_tx_result(u32 gen, int err) +{ + return gen | ((u32)err << ISOTP_TX_RESULT_GEN_BITS); +} + +/* get the 24 bit tx generation value from the tx result */ +static u32 isotp_get_tx_gen(u32 gen_err) +{ + return gen_err & ISOTP_TX_RESULT_GEN_MASK; +} + +/* get the 8 bit error value from the tx result */ +static u32 isotp_get_tx_err(u32 gen_err) +{ + return (gen_err >> ISOTP_TX_RESULT_GEN_BITS) & ISOTP_TX_RESULT_ERR_MASK; +} + +/* store transfer result in per-generation%4 so->tx_result[] slot */ +static void isotp_set_tx_result(struct isotp_sock *so, u32 gen, int err) +{ + WRITE_ONCE(so->tx_result[gen % ISOTP_TX_RESULT_SLOTS], + isotp_pack_tx_result(gen, err)); +} + +/* fetch the result recorded for 'gen', as a (negative) errno (0 for success) */ +static int isotp_get_tx_result(struct isotp_sock *so, u32 gen) +{ + u32 result = READ_ONCE(so->tx_result[gen % ISOTP_TX_RESULT_SLOTS]); + + if (isotp_get_tx_gen(result) != gen) { + pr_notice_once("can-isotp: tx_result[] slot reused before read\n"); + + /* report failure rather than risk a false success */ + return -ECOMM; + } + + return -(isotp_get_tx_err(result)); +} + +/* true if done, shut down or superseded ('gen' is no longer the active + * transfer). Reads tx.state first (acquire) so tx_gen/tx_result reads + * below see at least what that state write published (common sequence). + */ +static bool isotp_tx_gen_done(struct isotp_sock *so, u32 gen) +{ + /* read tx.state first for the common sequence */ + u32 state = smp_load_acquire(&so->tx.state); + + return state == ISOTP_IDLE || state == ISOTP_SHUTDOWN || + READ_ONCE(so->tx_gen) != gen; +} + static inline struct isotp_sock *isotp_sk(const struct sock *sk) { return (struct isotp_sock *)sk; @@ -197,7 +266,7 @@ static enum hrtimer_restart isotp_rx_timer_handler(struct hrtimer *hrtimer) rxtimer); struct sock *sk = &so->sk; - if (so->rx.state == ISOTP_WAIT_DATA) { + if (READ_ONCE(so->rx.state) == ISOTP_WAIT_DATA) { /* we did not get new data frames in time */ /* report 'connection timed out' */ @@ -206,7 +275,7 @@ static enum hrtimer_restart isotp_rx_timer_handler(struct hrtimer *hrtimer) sk_error_report(sk); /* reset rx state */ - so->rx.state = ISOTP_IDLE; + WRITE_ONCE(so->rx.state, ISOTP_IDLE); } return HRTIMER_NORESTART; @@ -232,7 +301,6 @@ static int isotp_send_fc(struct sock *sk, int ae, u8 flowstatus) can_skb_reserve(nskb); can_skb_prv(nskb)->ifindex = dev->ifindex; - can_skb_prv(nskb)->skbcnt = 0; nskb->dev = dev; can_skb_set_owner(nskb, sk); @@ -364,20 +432,19 @@ static void isotp_send_cframe(struct isotp_sock *so); static int isotp_rcv_fc(struct isotp_sock *so, struct canfd_frame *cf, int ae) { struct sock *sk = &so->sk; + int tx_err = EBADMSG; /* default for unknown FC status */ - if (so->tx.state != ISOTP_WAIT_FC && - so->tx.state != ISOTP_WAIT_FIRST_FC) + if (READ_ONCE(so->tx.state) != ISOTP_WAIT_FC && + READ_ONCE(so->tx.state) != ISOTP_WAIT_FIRST_FC) return 0; hrtimer_cancel(&so->txtimer); /* isotp_tx_timeout() may have given up on this job while - * hrtimer_cancel() above waited for it to finish; so->rx_lock - * (held by our caller isotp_rcv()) rules out a concurrent claim, - * so a plain recheck is enough here. + * hrtimer_cancel() above waited for it to finish => recheck */ - if (so->tx.state != ISOTP_WAIT_FC && - so->tx.state != ISOTP_WAIT_FIRST_FC) + if (READ_ONCE(so->tx.state) != ISOTP_WAIT_FC && + READ_ONCE(so->tx.state) != ISOTP_WAIT_FIRST_FC) return 1; if ((cf->len < ae + FC_CONTENT_SZ) || @@ -388,13 +455,15 @@ static int isotp_rcv_fc(struct isotp_sock *so, struct canfd_frame *cf, int ae) if (!sock_flag(sk, SOCK_DEAD)) sk_error_report(sk); - so->tx.state = ISOTP_IDLE; + isotp_set_tx_result(so, so->tx_gen, EBADMSG); + /* set to IDLE after publishing tx_result */ + smp_store_release(&so->tx.state, ISOTP_IDLE); wake_up_interruptible(&so->wait); return 1; } /* get static/dynamic communication params from first/every FC frame */ - if (so->tx.state == ISOTP_WAIT_FIRST_FC || + if (READ_ONCE(so->tx.state) == ISOTP_WAIT_FIRST_FC || so->opt.flags & CAN_ISOTP_DYN_FC_PARMS) { so->txfc.bs = cf->data[ae + 1]; so->txfc.stmin = cf->data[ae + 2]; @@ -418,13 +487,13 @@ static int isotp_rcv_fc(struct isotp_sock *so, struct canfd_frame *cf, int ae) so->tx_gap = ktime_add_ns(so->tx_gap, (so->txfc.stmin - 0xF0) * 100000); - so->tx.state = ISOTP_WAIT_FC; + WRITE_ONCE(so->tx.state, ISOTP_WAIT_FC); } switch (cf->data[ae] & 0x0F) { case ISOTP_FC_CTS: so->tx.bs = 0; - so->tx.state = ISOTP_SENDING; + WRITE_ONCE(so->tx.state, ISOTP_SENDING); /* send CF frame and enable echo timeout handling */ hrtimer_start(&so->echotimer, ktime_set(ISOTP_ECHO_TIMEOUT, 0), HRTIMER_MODE_REL_SOFT); @@ -439,14 +508,19 @@ static int isotp_rcv_fc(struct isotp_sock *so, struct canfd_frame *cf, int ae) case ISOTP_FC_OVFLW: /* overflow on receiver side - report 'message too long' */ - sk->sk_err = EMSGSIZE; - if (!sock_flag(sk, SOCK_DEAD)) - sk_error_report(sk); + tx_err = EMSGSIZE; fallthrough; default: - /* stop this tx job */ - so->tx.state = ISOTP_IDLE; + /* reserved/unknown flow status (tx_err defaults to EBADMSG) */ + + sk->sk_err = tx_err; + if (!sock_flag(sk, SOCK_DEAD)) + sk_error_report(sk); + + isotp_set_tx_result(so, so->tx_gen, tx_err); + /* set to IDLE after publishing tx_result */ + smp_store_release(&so->tx.state, ISOTP_IDLE); wake_up_interruptible(&so->wait); } return 0; @@ -459,7 +533,7 @@ static int isotp_rcv_sf(struct sock *sk, struct canfd_frame *cf, int pcilen, struct sk_buff *nskb; hrtimer_cancel(&so->rxtimer); - so->rx.state = ISOTP_IDLE; + WRITE_ONCE(so->rx.state, ISOTP_IDLE); if (!len || len > cf->len - pcilen) return 1; @@ -493,7 +567,7 @@ static int isotp_rcv_ff(struct sock *sk, struct canfd_frame *cf, int ae) int ff_pci_sz; hrtimer_cancel(&so->rxtimer); - so->rx.state = ISOTP_IDLE; + WRITE_ONCE(so->rx.state, ISOTP_IDLE); /* get the used sender LL_DL from the (first) CAN frame data length */ so->rx.ll_dl = padlen(cf->len); @@ -547,7 +621,7 @@ static int isotp_rcv_ff(struct sock *sk, struct canfd_frame *cf, int ae) /* initial setup for this pdu reception */ so->rx.sn = 1; - so->rx.state = ISOTP_WAIT_DATA; + WRITE_ONCE(so->rx.state, ISOTP_WAIT_DATA); /* no creation of flow control frames */ if (so->opt.flags & CAN_ISOTP_LISTEN_MODE) @@ -565,7 +639,7 @@ static int isotp_rcv_cf(struct sock *sk, struct canfd_frame *cf, int ae, struct sk_buff *nskb; int i; - if (so->rx.state != ISOTP_WAIT_DATA) + if (READ_ONCE(so->rx.state) != ISOTP_WAIT_DATA) return 0; /* drop if timestamp gap is less than force_rx_stmin nano secs */ @@ -580,11 +654,9 @@ static int isotp_rcv_cf(struct sock *sk, struct canfd_frame *cf, int ae, hrtimer_cancel(&so->rxtimer); /* isotp_rx_timer_handler() may have raced us for so->rx.state - * while hrtimer_cancel() above waited for it to finish, already - * reporting ETIMEDOUT and resetting the reception; don't process - * this CF into a reassembly that has already been given up on. + * while hrtimer_cancel() above waited for it to finish => recheck */ - if (so->rx.state != ISOTP_WAIT_DATA) + if (READ_ONCE(so->rx.state) != ISOTP_WAIT_DATA) return 1; /* CFs are never longer than the FF */ @@ -605,7 +677,7 @@ static int isotp_rcv_cf(struct sock *sk, struct canfd_frame *cf, int ae, sk_error_report(sk); /* reset rx state */ - so->rx.state = ISOTP_IDLE; + WRITE_ONCE(so->rx.state, ISOTP_IDLE); return 1; } so->rx.sn++; @@ -619,7 +691,7 @@ static int isotp_rcv_cf(struct sock *sk, struct canfd_frame *cf, int ae, if (so->rx.idx >= so->rx.len) { /* we are done */ - so->rx.state = ISOTP_IDLE; + WRITE_ONCE(so->rx.state, ISOTP_IDLE); if ((so->opt.flags & ISOTP_CHECK_PADDING) && check_pad(so, cf, i + 1, so->opt.rxpad_content)) { @@ -690,8 +762,10 @@ static void isotp_rcv(struct sk_buff *skb, void *data) if (so->opt.flags & CAN_ISOTP_HALF_DUPLEX) { /* check rx/tx path half duplex expectations */ - if ((so->tx.state != ISOTP_IDLE && n_pci_type != N_PCI_FC) || - (so->rx.state != ISOTP_IDLE && n_pci_type == N_PCI_FC)) + if ((READ_ONCE(so->tx.state) != ISOTP_IDLE && + n_pci_type != N_PCI_FC) || + (READ_ONCE(so->rx.state) != ISOTP_IDLE && + n_pci_type == N_PCI_FC)) goto out_unlock; } @@ -785,6 +859,7 @@ static void isotp_send_cframe(struct isotp_sock *so) struct canfd_frame *cf; int can_send_ret; int ae = (so->opt.flags & CAN_ISOTP_EXTEND_ADDR) ? 1 : 0; + u32 old_cfecho; dev = dev_get_by_index(sock_net(sk), so->ifindex); if (!dev) @@ -798,7 +873,9 @@ static void isotp_send_cframe(struct isotp_sock *so) can_skb_reserve(skb); can_skb_prv(skb)->ifindex = dev->ifindex; - can_skb_prv(skb)->skbcnt = 0; + + /* set uid in tx skb to identify CF echo frames */ + can_set_skb_uid(skb); cf = (struct canfd_frame *)skb->data; skb_put_zero(skb, so->ll.mtu); @@ -816,12 +893,15 @@ static void isotp_send_cframe(struct isotp_sock *so) skb->dev = dev; can_skb_set_owner(skb, sk); - /* cfecho should have been zero'ed by init/isotp_rcv_echo() */ - if (so->cfecho) - pr_notice_once("can-isotp: cfecho is %08X != 0\n", so->cfecho); + /* zero'ed by init/isotp_rcv_echo(); reached lock-free via + * isotp_txfr_timer_handler() too, so use READ_ONCE()/WRITE_ONCE() + */ + old_cfecho = READ_ONCE(so->cfecho); + if (old_cfecho) + pr_notice_once("can-isotp: cfecho is %08X != 0\n", old_cfecho); /* set consecutive frame echo tag */ - so->cfecho = *(u32 *)cf->data; + WRITE_ONCE(so->cfecho, skb->hash); /* send frame with local echo enabled */ can_send_ret = can_send(skb, 1); @@ -873,7 +953,6 @@ static void isotp_rcv_echo(struct sk_buff *skb, void *data) { struct sock *sk = (struct sock *)data; struct isotp_sock *so = isotp_sk(sk); - struct canfd_frame *cf = (struct canfd_frame *)skb->data; /* only handle my own local echo CF/SF skb's (no FF!) */ if (skb->sk != sk) @@ -885,32 +964,35 @@ static void isotp_rcv_echo(struct sk_buff *skb, void *data) spin_lock(&so->rx_lock); /* so->cfecho may since belong to a new transfer; recheck under lock */ - if (so->cfecho != *(u32 *)cf->data) + if (READ_ONCE(so->cfecho) != skb->hash) goto out_unlock; /* cancel local echo timeout */ hrtimer_cancel(&so->echotimer); /* local echo skb with consecutive frame has been consumed */ - so->cfecho = 0; + WRITE_ONCE(so->cfecho, 0); /* claiming a transfer also takes so->rx_lock, so a plain recheck * is enough: so->tx.state can't have flipped to ISOTP_SENDING for * a new claim while we're still in here */ - if (so->tx.state != ISOTP_SENDING) + if (READ_ONCE(so->tx.state) != ISOTP_SENDING) goto out_unlock; if (so->tx.idx >= so->tx.len) { /* we are done */ - so->tx.state = ISOTP_IDLE; + + isotp_set_tx_result(so, so->tx_gen, 0); + /* set to IDLE after publishing tx_result */ + smp_store_release(&so->tx.state, ISOTP_IDLE); wake_up_interruptible(&so->wait); goto out_unlock; } if (so->txfc.bs && so->tx.bs >= so->txfc.bs) { /* stop and wait for FC with timeout */ - so->tx.state = ISOTP_WAIT_FC; + WRITE_ONCE(so->tx.state, ISOTP_WAIT_FC); hrtimer_start(&so->txtimer, ktime_set(ISOTP_FC_TIMEOUT, 0), HRTIMER_MODE_REL_SOFT); goto out_unlock; @@ -932,16 +1014,20 @@ static void isotp_rcv_echo(struct sk_buff *skb, void *data) spin_unlock(&so->rx_lock); } -/* shared by so->txtimer's and so->echotimer's callbacks. Both timers get - * cancelled under so->rx_lock elsewhere, so this must stay lock-free to - * avoid deadlocking with that; uses so->tx_gen instead to avoid tainting - * a new transfer with an error from the one that just timed out. +/* isotp_tx_timeout: we did not get any flow control or echo frame in time + * + * Shared by so->txtimer's and so->echotimer's callbacks. Both timers get + * cancelled under so->rx_lock elsewhere, so this must stay lock-free. + * + * tx.state is acquired before tx_gen. Common sequence in isotp_tx_gen_done(). + * cmpxchg() only orders itself, not the two preceding loads. */ static enum hrtimer_restart isotp_tx_timeout(struct isotp_sock *so) { struct sock *sk = &so->sk; + /* read tx.state first for the common sequence */ + u32 old_state = smp_load_acquire(&so->tx.state); u32 gen = READ_ONCE(so->tx_gen); - u32 old_state = READ_ONCE(so->tx.state); /* don't handle timeouts in IDLE or SHUTDOWN state */ if (old_state == ISOTP_IDLE || old_state == ISOTP_SHUTDOWN) @@ -951,14 +1037,14 @@ static enum hrtimer_restart isotp_tx_timeout(struct isotp_sock *so) if (cmpxchg(&so->tx.state, old_state, ISOTP_IDLE) != old_state) return HRTIMER_NORESTART; - /* we did not get any flow control or echo frame in time */ + /* detected timeout: report 'communication error on send' */ - if (READ_ONCE(so->tx_gen) == gen) { - /* report 'communication error on send' */ - sk->sk_err = ECOMM; - if (!sock_flag(sk, SOCK_DEAD)) - sk_error_report(sk); - } + /* a stale read of this slot by a waiter still falls back to ECOMM */ + isotp_set_tx_result(so, gen, ECOMM); + + sk->sk_err = ECOMM; + if (!sock_flag(sk, SOCK_DEAD)) + sk_error_report(sk); wake_up_interruptible(&so->wait); @@ -993,7 +1079,7 @@ static enum hrtimer_restart isotp_txfr_timer_handler(struct hrtimer *hrtimer) HRTIMER_MODE_REL_SOFT); /* cfecho should be consumed by isotp_rcv_echo() here */ - if (so->tx.state == ISOTP_SENDING && !so->cfecho) + if (READ_ONCE(so->tx.state) == ISOTP_SENDING && !READ_ONCE(so->cfecho)) isotp_send_cframe(so); return HRTIMER_NORESTART; @@ -1011,10 +1097,12 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) s64 hrtimer_sec = ISOTP_ECHO_TIMEOUT; struct hrtimer *tx_hrt = &so->echotimer; u32 new_state = ISOTP_SENDING; + u32 my_gen; + u32 old_cfecho; int off; int err; - if (!so->bound || so->tx.state == ISOTP_SHUTDOWN) + if (!so->bound || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN) return -EADDRNOTAVAIL; /* claim the socket under so->rx_lock: this serializes the claim @@ -1031,29 +1119,33 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) if (msg->msg_flags & MSG_DONTWAIT) return -EAGAIN; - if (so->tx.state == ISOTP_SHUTDOWN) + if (READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN) return -EADDRNOTAVAIL; /* wait for complete transmission of current pdu */ err = wait_event_interruptible(so->wait, - so->tx.state == ISOTP_IDLE); + READ_ONCE(so->tx.state) == ISOTP_IDLE || + READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN); if (err) return err; } - /* new transfer: bump so->tx_gen and drain the old one's timers, - * still under the so->rx_lock we just claimed the socket with - */ - WRITE_ONCE(so->tx.state, ISOTP_SENDING); - WRITE_ONCE(so->tx_gen, READ_ONCE(so->tx_gen) + 1); + /* txfrtimer's callback re-arms echotimer lock-free: drain it first */ + hrtimer_cancel(&so->txfrtimer); hrtimer_cancel(&so->txtimer); hrtimer_cancel(&so->echotimer); - hrtimer_cancel(&so->txfrtimer); - so->cfecho = 0; + + /* new transfer: increment so->tx_gen and set tx.state after barrier */ + my_gen = isotp_inc_tx_gen(READ_ONCE(so->tx_gen)); + isotp_set_tx_result(so, my_gen, ECOMM); /* prevent stale slot matching */ + WRITE_ONCE(so->tx_gen, my_gen); + smp_wmb(); /* see smp_load_acquire() in isotp_tx_[timeout|gen_done] */ + WRITE_ONCE(so->tx.state, ISOTP_SENDING); + WRITE_ONCE(so->cfecho, 0); spin_unlock_bh(&so->rx_lock); /* so->bound is only checked once above - a wakeup may have - * unbound/rebound the socket meanwhile, so re-validate it + * unbound/rebound the socket meanwhile => recheck */ if (!so->bound) { err = -EADDRNOTAVAIL; @@ -1104,7 +1196,9 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) can_skb_reserve(skb); can_skb_prv(skb)->ifindex = dev->ifindex; - can_skb_prv(skb)->skbcnt = 0; + + /* set uid in tx skb to identify CF echo frames */ + can_set_skb_uid(skb); so->tx.len = size; so->tx.idx = 0; @@ -1113,8 +1207,9 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) skb_put_zero(skb, so->ll.mtu); /* cfecho should have been zero'ed by init / former isotp_rcv_echo() */ - if (so->cfecho) - pr_notice_once("can-isotp: uninit cfecho %08X\n", so->cfecho); + old_cfecho = READ_ONCE(so->cfecho); + if (old_cfecho) + pr_notice_once("can-isotp: uninit cfecho %08X\n", old_cfecho); /* check for single frame transmission depending on TX_DL */ if (size <= so->tx.ll_dl - SF_PCI_SZ4 - ae - off) { @@ -1142,7 +1237,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) cf->data[ae] |= size; /* set CF echo tag for isotp_rcv_echo() (SF-mode) */ - so->cfecho = *(u32 *)cf->data; + WRITE_ONCE(so->cfecho, skb->hash); } else { /* send first frame */ @@ -1159,7 +1254,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) so->txfc.bs = 0; /* set CF echo tag for isotp_rcv_echo() (CF-mode) */ - so->cfecho = *(u32 *)cf->data; + WRITE_ONCE(so->cfecho, skb->hash); } else { /* standard flow control check */ new_state = ISOTP_WAIT_FIRST_FC; @@ -1169,12 +1264,12 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) tx_hrt = &so->txtimer; /* no CF echo tag for isotp_rcv_echo() (FF-mode) */ - so->cfecho = 0; + WRITE_ONCE(so->cfecho, 0); } } spin_lock_bh(&so->rx_lock); - if (so->tx.state == ISOTP_SHUTDOWN) { + if (READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN) { /* isotp_release() has since taken over and already drained * our timers - don't send into a socket that's going away */ @@ -1185,7 +1280,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) return -EADDRNOTAVAIL; } /* WAIT_FIRST_FC for standard FF, else stays ISOTP_SENDING */ - so->tx.state = new_state; + WRITE_ONCE(so->tx.state, new_state); hrtimer_start(tx_hrt, ktime_set(hrtimer_sec, 0), HRTIMER_MODE_REL_SOFT); spin_unlock_bh(&so->rx_lock); @@ -1202,20 +1297,49 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) __func__, ERR_PTR(err)); spin_lock_bh(&so->rx_lock); + + /* new transfer already claimed by a concurrent completion, + * timeout or sendmsg() while we were stuck in can_send()? + */ + if (READ_ONCE(so->tx_gen) != my_gen) { + /* don't touch timers and state of the new transfer */ + spin_unlock_bh(&so->rx_lock); + return err; + } + /* no transmission -> no timeout monitoring */ hrtimer_cancel(tx_hrt); goto err_out_drop_locked; } if (wait_tx_done) { - /* wait for complete transmission of current pdu */ - err = wait_event_interruptible(so->wait, so->tx.state == ISOTP_IDLE); + /* wake up for: + * - concurrent sendmsg() claiming a new transfer + * - complete transmission of current PDU + * - shutdown state change in isotp_release() + * isotp_tx_gen_done() uses common tx.state/tx_gen read sequence + */ + err = wait_event_interruptible(so->wait, + isotp_tx_gen_done(so, my_gen)); if (err) goto err_event_drop; - err = sock_error(sk); - if (err) - return err; + /* still our claim, but isotp_release() force-shut it down */ + if (smp_load_acquire(&so->tx.state) == ISOTP_SHUTDOWN && + READ_ONCE(so->tx_gen) == my_gen) { + err = -EADDRNOTAVAIL; + goto err_event_drop; + } + + /* own completion, or tx_gen moved on - either way this is + * what isotp_get_tx_result() recorded for my_gen + */ + err = isotp_get_tx_result(so, my_gen); + + /* drain to avoid stale error for a later poll()/SO_ERROR */ + sock_error(sk); + + return err ? err : size; } return size; @@ -1225,15 +1349,26 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) spin_lock_bh(&so->rx_lock); goto err_out_drop_locked; err_event_drop: - /* interrupted waiting on our own transfer - drain its timers */ + /* interrupted or shut down while waiting on our own transfer */ spin_lock_bh(&so->rx_lock); + + /* new transfer already started by concurrent sendmsg()? */ + if (READ_ONCE(so->tx_gen) != my_gen) { + /* don't touch timers and states of the new transfer */ + spin_unlock_bh(&so->rx_lock); + return err; + } + hrtimer_cancel(&so->txfrtimer); hrtimer_cancel(&so->txtimer); hrtimer_cancel(&so->echotimer); err_out_drop_locked: /* release the claim; so->rx_lock still held from above */ - so->cfecho = 0; - so->tx.state = ISOTP_IDLE; + WRITE_ONCE(so->cfecho, 0); + + /* only claim to IDLE if isotp_release() has not taken over */ + if (READ_ONCE(so->tx.state) != ISOTP_SHUTDOWN) + WRITE_ONCE(so->tx.state, ISOTP_IDLE); spin_unlock_bh(&so->rx_lock); wake_up_interruptible(&so->wait); @@ -1299,8 +1434,9 @@ static int isotp_release(struct socket *sock) /* best-effort: wait for a running pdu to finish, but don't block on * it forever - give up after the first signal */ - while (so->tx.state != ISOTP_IDLE && - wait_event_interruptible(so->wait, so->tx.state == ISOTP_IDLE) == 0) + while (READ_ONCE(so->tx.state) != ISOTP_IDLE && + wait_event_interruptible(so->wait, + READ_ONCE(so->tx.state) == ISOTP_IDLE) == 0) ; /* claim the socket under so->rx_lock like sendmsg() does, so its @@ -1308,9 +1444,12 @@ static int isotp_release(struct socket *sock) * unconditionally, even when a signal cut the wait above short */ spin_lock_bh(&so->rx_lock); - so->tx.state = ISOTP_SHUTDOWN; + WRITE_ONCE(so->tx.state, ISOTP_SHUTDOWN); spin_unlock_bh(&so->rx_lock); - so->rx.state = ISOTP_IDLE; + WRITE_ONCE(so->rx.state, ISOTP_IDLE); + + /* forced SHUTDOWN may have skipped IDLE (gave up on a signal) */ + wake_up_interruptible(&so->wait); spin_lock(&isotp_notifier_lock); while (isotp_busy_notifier == so) { @@ -1425,7 +1564,8 @@ static int isotp_bind(struct socket *sock, struct sockaddr *uaddr, int len) * with so->bound in the same lock_sock() section above, so there is * no window in which a concurrent isotp_notify() could be missed. */ - if (so->tx.state != ISOTP_IDLE || so->rx.state != ISOTP_IDLE) { + if (READ_ONCE(so->tx.state) != ISOTP_IDLE || + READ_ONCE(so->rx.state) != ISOTP_IDLE) { err = -EAGAIN; goto out; } @@ -1459,7 +1599,7 @@ static int isotp_bind(struct socket *sock, struct sockaddr *uaddr, int len) isotp_rcv, sk, "isotp", sk); /* no consecutive frame echo skb in flight */ - so->cfecho = 0; + WRITE_ONCE(so->cfecho, 0); /* register for echo skb's */ can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id), @@ -1825,7 +1965,7 @@ static __poll_t isotp_poll(struct file *file, struct socket *sock, poll_table *w poll_wait(file, &so->wait, wait); /* Check for false positives due to TX state */ - if ((mask & EPOLLWRNORM) && (so->tx.state != ISOTP_IDLE)) + if ((mask & EPOLLWRNORM) && (READ_ONCE(so->tx.state) != ISOTP_IDLE)) mask &= ~(EPOLLOUT | EPOLLWRNORM); return mask; @@ -1885,13 +2025,18 @@ static __init int isotp_module_init(void) pr_info("can: isotp protocol (max_pdu_size %d)\n", max_pdu_size); + err = register_netdevice_notifier(&canisotp_notifier); + if (err) + return err; + err = can_proto_register(&isotp_can_proto); - if (err < 0) + if (err < 0) { pr_err("can: registration of isotp protocol failed %pe\n", ERR_PTR(err)); - else - register_netdevice_notifier(&canisotp_notifier); + unregister_netdevice_notifier(&canisotp_notifier); + return err; + } - return err; + return 0; } static __exit void isotp_module_exit(void) diff --git a/net/can/j1939/bus.c b/net/can/j1939/bus.c index e0b966c2517c..aabe4fa91543 100644 --- a/net/can/j1939/bus.c +++ b/net/can/j1939/bus.c @@ -20,6 +20,7 @@ static void __j1939_ecu_release(struct kref *kref) struct j1939_priv *priv = ecu->priv; list_del(&ecu->list); + netdev_put(priv->ndev, &ecu->priv_dev_tracker); kfree(ecu); j1939_priv_put(priv); } @@ -155,6 +156,7 @@ struct j1939_ecu *j1939_ecu_create_locked(struct j1939_priv *priv, name_t name) if (!ecu) return ERR_PTR(-ENOMEM); kref_init(&ecu->kref); + netdev_hold(priv->ndev, &ecu->priv_dev_tracker, gfp_any()); ecu->addr = J1939_IDLE_ADDR; ecu->name = name; diff --git a/net/can/j1939/j1939-priv.h b/net/can/j1939/j1939-priv.h index 31a93cae5111..1d3d5d0551e0 100644 --- a/net/can/j1939/j1939-priv.h +++ b/net/can/j1939/j1939-priv.h @@ -38,6 +38,7 @@ struct j1939_ecu { struct hrtimer ac_timer; struct kref kref; struct j1939_priv *priv; + netdevice_tracker priv_dev_tracker; /* count users, to help transport protocol decide for interaction */ int nusers; @@ -60,6 +61,7 @@ struct j1939_priv { rwlock_t lock; struct net_device *ndev; + netdevice_tracker dev_tracker; /* list of 256 ecu ptrs, that cache the claimed addresses. * also protected by the above lock @@ -229,6 +231,7 @@ enum j1939_session_state { struct j1939_session { struct j1939_priv *priv; + netdevice_tracker priv_dev_tracker; struct list_head active_session_list_entry; struct list_head sk_session_queue_entry; struct kref kref; diff --git a/net/can/j1939/main.c b/net/can/j1939/main.c index 7e8a20f2fc42..b45f834ff2cf 100644 --- a/net/can/j1939/main.c +++ b/net/can/j1939/main.c @@ -137,7 +137,7 @@ static struct j1939_priv *j1939_priv_create(struct net_device *ndev) priv->ndev = ndev; kref_init(&priv->kref); kref_init(&priv->rx_kref); - dev_hold(ndev); + netdev_hold(ndev, &priv->dev_tracker, GFP_KERNEL); netdev_dbg(priv->ndev, "%s : 0x%p\n", __func__, priv); @@ -163,7 +163,7 @@ static void __j1939_priv_release(struct kref *kref) WARN_ON_ONCE(!list_empty(&priv->ecus)); WARN_ON_ONCE(!list_empty(&priv->j1939_socks)); - dev_put(ndev); + netdev_put(ndev, &priv->dev_tracker); kfree(priv); } @@ -281,7 +281,7 @@ struct j1939_priv *j1939_netdev_start(struct net_device *ndev) */ kref_get(&priv_new->rx_kref); mutex_unlock(&j1939_netdev_lock); - dev_put(ndev); + netdev_put(ndev, &priv->dev_tracker); kfree(priv); return priv_new; } @@ -298,7 +298,7 @@ struct j1939_priv *j1939_netdev_start(struct net_device *ndev) j1939_priv_set(ndev, NULL); mutex_unlock(&j1939_netdev_lock); - dev_put(ndev); + netdev_put(ndev, &priv->dev_tracker); kfree(priv); return ERR_PTR(ret); diff --git a/net/can/j1939/socket.c b/net/can/j1939/socket.c index 75c2b9b23390..e843891a9323 100644 --- a/net/can/j1939/socket.c +++ b/net/can/j1939/socket.c @@ -889,7 +889,6 @@ static struct sk_buff *j1939_sk_alloc_skb(struct net_device *ndev, can_skb_reserve(skb); can_skb_prv(skb)->ifindex = ndev->ifindex; - can_skb_prv(skb)->skbcnt = 0; skb_reserve(skb, offsetof(struct can_frame, data)); ret = memcpy_from_msg(skb_put(skb, size), msg, size); diff --git a/net/can/j1939/transport.c b/net/can/j1939/transport.c index d8dada03b3cd..a262a58588c2 100644 --- a/net/can/j1939/transport.c +++ b/net/can/j1939/transport.c @@ -282,6 +282,7 @@ static void j1939_session_destroy(struct j1939_session *session) kfree_skb(skb); } __j1939_session_drop(session); + netdev_put(session->priv->ndev, &session->priv_dev_tracker); j1939_priv_put(session->priv); kfree(session); } @@ -612,7 +613,6 @@ sk_buff *j1939_tp_tx_dat_new(struct j1939_priv *priv, skb->dev = priv->ndev; can_skb_reserve(skb); can_skb_prv(skb)->ifindex = priv->ndev->ifindex; - can_skb_prv(skb)->skbcnt = 0; /* reserve CAN header */ skb_reserve(skb, offsetof(struct can_frame, data)); @@ -1510,6 +1510,7 @@ static struct j1939_session *j1939_session_new(struct j1939_priv *priv, INIT_LIST_HEAD(&session->active_session_list_entry); INIT_LIST_HEAD(&session->sk_session_queue_entry); kref_init(&session->kref); + netdev_hold(priv->ndev, &session->priv_dev_tracker, gfp_any()); j1939_priv_get(priv); session->priv = priv; @@ -1551,7 +1552,6 @@ j1939_session *j1939_session_fresh_new(struct j1939_priv *priv, skb->dev = priv->ndev; can_skb_reserve(skb); can_skb_prv(skb)->ifindex = priv->ndev->ifindex; - can_skb_prv(skb)->skbcnt = 0; skcb = j1939_skb_to_cb(skb); memcpy(skcb, rel_skcb, sizeof(*skcb)); @@ -1562,7 +1562,7 @@ j1939_session *j1939_session_fresh_new(struct j1939_priv *priv, } /* alloc data area */ - skb_put(skb, size); + skb_put_zero(skb, size); /* skb is recounted in j1939_session_new() */ return session; } diff --git a/net/can/raw.c b/net/can/raw.c index 3b274db1da61..d0b84bdd0cb5 100644 --- a/net/can/raw.c +++ b/net/can/raw.c @@ -75,8 +75,8 @@ MODULE_ALIAS("can-proto-1"); */ struct uniqframe { - int skbcnt; const struct sk_buff *skb; + u32 hash; unsigned int join_rx_count; }; @@ -163,7 +163,7 @@ static void raw_rcv(struct sk_buff *oskb, void *data) /* eliminate multiple filter matches for the same skb */ if (this_cpu_ptr(ro->uniq)->skb == oskb && - this_cpu_ptr(ro->uniq)->skbcnt == can_skb_prv(oskb)->skbcnt) { + this_cpu_ptr(ro->uniq)->hash == oskb->hash) { if (!ro->join_filters) return; @@ -173,7 +173,7 @@ static void raw_rcv(struct sk_buff *oskb, void *data) return; } else { this_cpu_ptr(ro->uniq)->skb = oskb; - this_cpu_ptr(ro->uniq)->skbcnt = can_skb_prv(oskb)->skbcnt; + this_cpu_ptr(ro->uniq)->hash = oskb->hash; this_cpu_ptr(ro->uniq)->join_rx_count = 1; /* drop first frame to check all enabled filters? */ if (ro->join_filters && ro->count > 1) @@ -945,7 +945,6 @@ static int raw_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) can_skb_reserve(skb); can_skb_prv(skb)->ifindex = dev->ifindex; - can_skb_prv(skb)->skbcnt = 0; /* fill the skb before testing for valid CAN frames */ err = memcpy_from_msg(skb_put(skb, size), msg, size); diff --git a/net/core/lwt_bpf.c b/net/core/lwt_bpf.c index ab23323314b7..f1641ec8284b 100644 --- a/net/core/lwt_bpf.c +++ b/net/core/lwt_bpf.c @@ -248,8 +248,10 @@ static int bpf_lwt_xmit_reroute(struct sk_buff *skb) * if there is enough header space in skb. */ err = skb_cow_head(skb, LL_RESERVED_SPACE(dst->dev)); - if (unlikely(err)) + if (unlikely(err)) { + dst_release(dst); goto err; + } skb_dst_drop(skb); skb_dst_set(skb, dst); diff --git a/net/core/skbuff.c b/net/core/skbuff.c index fede3aa3ddbc..dcb5d025dc08 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -925,6 +925,18 @@ static void skb_clone_fraglist(struct sk_buff *skb) skb_get(list); } +/** + * skb_pp_cow_data() - copy skb data into page-pool backed storage + * @pool: page pool to allocate from + * @pskb: pointer to skb pointer, replaced with the copied skb on success + * @headroom: headroom to reserve in the copied skb + * + * skb_copy_bits() handles both frags[] and frag_list input. If the copied + * skb remains non-linear, it uses frags[], which is the representation used + * by XDP multi-buffer. + * + * Return: 0 on success or a negative errno on failure. + */ int skb_pp_cow_data(struct page_pool *pool, struct sk_buff **pskb, unsigned int headroom) { @@ -934,12 +946,6 @@ int skb_pp_cow_data(struct page_pool *pool, struct sk_buff **pskb, int err, i, head_off; void *data; - /* XDP does not support fraglist so we need to linearize - * the skb. - */ - if (skb_has_frag_list(skb)) - return -EOPNOTSUPP; - max_head_size = SKB_WITH_OVERHEAD(PAGE_SIZE - headroom); if (skb->len > max_head_size + MAX_SKB_FRAGS * PAGE_SIZE) return -ENOMEM; diff --git a/net/ipv4/route.c b/net/ipv4/route.c index 4dce0de6ab89..2b8c29a29c4a 100644 --- a/net/ipv4/route.c +++ b/net/ipv4/route.c @@ -890,8 +890,6 @@ void ip_rt_send_redirect(struct sk_buff *skb) peer = inet_getpeer_v4(net->ipv4.peers, ip_hdr(skb)->saddr, vif); if (!peer) { rcu_read_unlock(); - icmp_send(skb, ICMP_REDIRECT, ICMP_REDIR_HOST, - rt_nexthop(rt, ip_hdr(skb)->daddr)); return; } diff --git a/net/ipv4/udp_tunnel_nic.c b/net/ipv4/udp_tunnel_nic.c index b13e8f7092f4..4360d159ee8a 100644 --- a/net/ipv4/udp_tunnel_nic.c +++ b/net/ipv4/udp_tunnel_nic.c @@ -32,13 +32,12 @@ struct udp_tunnel_nic_table_entry { * @lock: protects all fields * @need_sync: at least one port start changed * @need_replay: space was freed, we need a replay of all ports - * @work_pending: @work is currently scheduled * @n_tables: number of tables under @entries * @missed: bitmap of tables which overflown * @entries: table of tables of ports currently offloaded */ struct udp_tunnel_nic { - struct work_struct work; + struct delayed_work work; struct net_device *dev; @@ -46,7 +45,6 @@ struct udp_tunnel_nic { u8 need_sync:1; u8 need_replay:1; - u8 work_pending:1; unsigned int n_tables; unsigned long missed; @@ -301,11 +299,10 @@ __udp_tunnel_nic_device_sync(struct net_device *dev, struct udp_tunnel_nic *utn) static void udp_tunnel_nic_device_sync(struct net_device *dev, struct udp_tunnel_nic *utn) { - if (!utn->need_sync || utn->work_pending) + if (!utn->need_sync) return; - queue_work(udp_tunnel_nic_workqueue, &utn->work); - utn->work_pending = 1; + queue_delayed_work(udp_tunnel_nic_workqueue, &utn->work, 0); } static bool @@ -731,12 +728,17 @@ udp_tunnel_nic_replay(struct net_device *dev, struct udp_tunnel_nic *utn) static void udp_tunnel_nic_device_sync_work(struct work_struct *work) { struct udp_tunnel_nic *utn = - container_of(work, struct udp_tunnel_nic, work); + container_of(work, struct udp_tunnel_nic, work.work); - rtnl_lock(); + /* We cannot block on RTNL here, otherwise we would deadlock with + * udp_tunnel_nic_unregister() calling cancel_delayed_work_sync() + * while holding RTNL. Requeue with 1 jiffy delay if RTNL is contended. + */ + if (!rtnl_trylock()) { + queue_delayed_work(udp_tunnel_nic_workqueue, &utn->work, 1); + return; + } mutex_lock(&utn->lock); - - utn->work_pending = 0; __udp_tunnel_nic_device_sync(utn->dev, utn); if (utn->need_replay) @@ -757,7 +759,7 @@ udp_tunnel_nic_alloc(const struct udp_tunnel_nic_info *info, if (!utn) return NULL; utn->n_tables = n_tables; - INIT_WORK(&utn->work, udp_tunnel_nic_device_sync_work); + INIT_DELAYED_WORK(&utn->work, udp_tunnel_nic_device_sync_work); mutex_init(&utn->lock); for (i = 0; i < n_tables; i++) { @@ -901,11 +903,11 @@ udp_tunnel_nic_unregister(struct net_device *dev, struct udp_tunnel_nic *utn) udp_tunnel_nic_flush(dev, utn); udp_tunnel_nic_unlock(dev); - /* Wait for the work to be done using the state, netdev core will - * retry unregister until we give up our reference on this device. + /* Make sure no work is running or queued before freeing @utn. + * The work handler uses rtnl_trylock(), so it will not deadlock + * against the RTNL we are holding here. */ - if (utn->work_pending) - return; + cancel_delayed_work_sync(&utn->work); udp_tunnel_nic_free(utn); release_dev: diff --git a/net/ipv6/fib6_rules.c b/net/ipv6/fib6_rules.c index 29185c9ebd02..a7e7ab998871 100644 --- a/net/ipv6/fib6_rules.c +++ b/net/ipv6/fib6_rules.c @@ -302,6 +302,7 @@ INDIRECT_CALLABLE_SCOPE bool fib6_rule_suppress(struct fib_rule *rule, suppress_route: ip6_rt_put_flags(rt, flags); + res->rt6 = NULL; return true; } diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 50b41be5d38d..5d17982cb994 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -627,7 +627,7 @@ int ip6_forward(struct sk_buff *skb) /* Limit redirects both by destination (here) and by source (inside ndisc_send_redirect) */ - if (inet_peer_xrlim_allow(peer, 1*HZ)) + if (peer && inet_peer_xrlim_allow(peer, 1*HZ)) ndisc_send_redirect(skb, target); rcu_read_unlock(); } else { diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c index 73e2e457e30b..1a759519a90f 100644 --- a/net/ipv6/ndisc.c +++ b/net/ipv6/ndisc.c @@ -1739,6 +1739,8 @@ void ndisc_send_redirect(struct sk_buff *skb, const struct in6_addr *target) rcu_read_lock(); peer = inet_getpeer_v6(net->ipv6.peers, &ipv6_hdr(skb)->saddr); + if (!peer) + goto release; ret = inet_peer_xrlim_allow(peer, 1*HZ); rcu_read_unlock(); diff --git a/net/mac80211/s1g.c b/net/mac80211/s1g.c index d4ed0c0a335c..2a23e431cbb5 100644 --- a/net/mac80211/s1g.c +++ b/net/mac80211/s1g.c @@ -101,6 +101,10 @@ ieee80211_s1g_rx_twt_setup(struct ieee80211_sub_if_data *sdata, struct ieee80211_twt_setup *twt = (void *)mgmt->u.action.u.s1g.variable; struct ieee80211_twt_params *twt_agrt = (void *)twt->params; + if (!(twt->control & IEEE80211_TWT_CONTROL_NEG_TYPE_BROADCAST) && + twt->length < sizeof(twt->control) + sizeof(*twt_agrt)) + return; + twt_agrt->req_type &= cpu_to_le16(~IEEE80211_TWT_REQTYPE_REQUEST); /* broadcast TWT not supported yet */ diff --git a/net/mpls/af_mpls.c b/net/mpls/af_mpls.c index 98816e51e01a..0379540d601a 100644 --- a/net/mpls/af_mpls.c +++ b/net/mpls/af_mpls.c @@ -2468,6 +2468,7 @@ static int mpls_getroute(struct sk_buff *in_skb, struct nlmsghdr *in_nlh, r->rtm_family = AF_MPLS; r->rtm_dst_len = 20; r->rtm_src_len = 0; + r->rtm_tos = 0; r->rtm_table = RT_TABLE_MAIN; r->rtm_type = RTN_UNICAST; r->rtm_scope = RT_SCOPE_UNIVERSE; diff --git a/net/mptcp/pm.c b/net/mptcp/pm.c index 3a27afcb4ec8..009020ee1e04 100644 --- a/net/mptcp/pm.c +++ b/net/mptcp/pm.c @@ -427,7 +427,7 @@ bool mptcp_pm_rm_addr_signal(struct mptcp_sock *msk, unsigned int remaining, int mptcp_pm_get_local_id(struct mptcp_sock *msk, struct sock_common *skc) { - struct mptcp_addr_info skc_local; + struct mptcp_pm_addr_entry skc_local = { 0 }; struct mptcp_addr_info msk_local; if (WARN_ON_ONCE(!msk)) @@ -437,10 +437,13 @@ int mptcp_pm_get_local_id(struct mptcp_sock *msk, struct sock_common *skc) * addr */ mptcp_local_address((struct sock_common *)msk, &msk_local); - mptcp_local_address((struct sock_common *)skc, &skc_local); - if (mptcp_addresses_equal(&msk_local, &skc_local, false)) + mptcp_local_address((struct sock_common *)skc, &skc_local.addr); + if (mptcp_addresses_equal(&msk_local, &skc_local.addr, false)) return 0; + skc_local.addr.id = 0; + skc_local.flags = MPTCP_PM_ADDR_FLAG_IMPLICIT; + if (mptcp_pm_is_userspace(msk)) return mptcp_userspace_pm_get_local_id(msk, &skc_local); return mptcp_pm_nl_get_local_id(msk, &skc_local); diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c index 8159ffb8466b..550c4c1b963d 100644 --- a/net/mptcp/pm_netlink.c +++ b/net/mptcp/pm_netlink.c @@ -1274,21 +1274,18 @@ static int mptcp_pm_nl_create_listen_socket(struct sock *sk, return err; } -int mptcp_pm_nl_get_local_id(struct mptcp_sock *msk, struct mptcp_addr_info *skc) +int mptcp_pm_nl_get_local_id(struct mptcp_sock *msk, + struct mptcp_pm_addr_entry *skc) { struct mptcp_pm_addr_entry *entry; struct pm_nl_pernet *pernet; - int ret = -1; + int ret; pernet = pm_nl_get_pernet_from_msk(msk); rcu_read_lock(); - list_for_each_entry_rcu(entry, &pernet->local_addr_list, list) { - if (mptcp_addresses_equal(&entry->addr, skc, entry->addr.port)) { - ret = entry->addr.id; - break; - } - } + entry = __lookup_addr(pernet, &skc->addr); + ret = entry ? entry->addr.id : -1; rcu_read_unlock(); if (ret >= 0) return ret; @@ -1298,12 +1295,8 @@ int mptcp_pm_nl_get_local_id(struct mptcp_sock *msk, struct mptcp_addr_info *skc if (!entry) return -ENOMEM; - entry->addr = *skc; - entry->addr.id = 0; + *entry = *skc; entry->addr.port = 0; - entry->ifindex = 0; - entry->flags = MPTCP_PM_ADDR_FLAG_IMPLICIT; - entry->lsk = NULL; ret = mptcp_pm_nl_append_new_local_addr(pernet, entry, false); if (ret < 0) kfree(entry); @@ -1315,15 +1308,11 @@ bool mptcp_pm_nl_is_backup(struct mptcp_sock *msk, struct mptcp_addr_info *skc) { struct pm_nl_pernet *pernet = pm_nl_get_pernet_from_msk(msk); struct mptcp_pm_addr_entry *entry; - bool backup = false; + bool backup; rcu_read_lock(); - list_for_each_entry_rcu(entry, &pernet->local_addr_list, list) { - if (mptcp_addresses_equal(&entry->addr, skc, entry->addr.port)) { - backup = !!(entry->flags & MPTCP_PM_ADDR_FLAG_BACKUP); - break; - } - } + entry = __lookup_addr(pernet, skc); + backup = entry && !!(entry->flags & MPTCP_PM_ADDR_FLAG_BACKUP); rcu_read_unlock(); return backup; diff --git a/net/mptcp/pm_userspace.c b/net/mptcp/pm_userspace.c index bc3eb242f298..2a219e47392c 100644 --- a/net/mptcp/pm_userspace.c +++ b/net/mptcp/pm_userspace.c @@ -26,6 +26,19 @@ void mptcp_free_local_addr_list(struct mptcp_sock *msk) } } +static struct mptcp_pm_addr_entry * +mptcp_userspace_pm_lookup_addr(struct mptcp_sock *msk, + const struct mptcp_addr_info *addr) +{ + struct mptcp_pm_addr_entry *entry; + + list_for_each_entry(entry, &msk->pm.userspace_pm_local_addr_list, list) { + if (mptcp_addresses_equal(&entry->addr, addr, false)) + return entry; + } + return NULL; +} + static int mptcp_userspace_pm_append_new_local_addr(struct mptcp_sock *msk, struct mptcp_pm_addr_entry *entry, bool needs_id) @@ -90,22 +103,20 @@ static int mptcp_userspace_pm_append_new_local_addr(struct mptcp_sock *msk, static int mptcp_userspace_pm_delete_local_addr(struct mptcp_sock *msk, struct mptcp_pm_addr_entry *addr) { - struct mptcp_pm_addr_entry *entry, *tmp; struct sock *sk = (struct sock *)msk; + struct mptcp_pm_addr_entry *entry; - list_for_each_entry_safe(entry, tmp, &msk->pm.userspace_pm_local_addr_list, list) { - if (mptcp_addresses_equal(&entry->addr, &addr->addr, false)) { - /* TODO: a refcount is needed because the entry can - * be used multiple times (e.g. fullmesh mode). - */ - list_del_rcu(&entry->list); - sock_kfree_s(sk, entry, sizeof(*entry)); - msk->pm.local_addr_used--; - return 0; - } - } - - return -EINVAL; + entry = mptcp_userspace_pm_lookup_addr(msk, &addr->addr); + if (!entry) + return -EINVAL; + + /* TODO: a refcount is needed because the entry can + * be used multiple times (e.g. fullmesh mode). + */ + list_del_rcu(&entry->list); + sock_kfree_s(sk, entry, sizeof(*entry)); + msk->pm.local_addr_used--; + return 0; } static struct mptcp_pm_addr_entry * @@ -121,47 +132,36 @@ mptcp_userspace_pm_lookup_addr_by_id(struct mptcp_sock *msk, unsigned int id) } int mptcp_userspace_pm_get_local_id(struct mptcp_sock *msk, - struct mptcp_addr_info *skc) + struct mptcp_pm_addr_entry *skc) { - struct mptcp_pm_addr_entry *entry = NULL, *e, new_entry; __be16 msk_sport = ((struct inet_sock *) inet_sk((struct sock *)msk))->inet_sport; + struct mptcp_pm_addr_entry *entry; + int id; spin_lock_bh(&msk->pm.lock); - list_for_each_entry(e, &msk->pm.userspace_pm_local_addr_list, list) { - if (mptcp_addresses_equal(&e->addr, skc, false)) { - entry = e; - break; - } - } + entry = mptcp_userspace_pm_lookup_addr(msk, &skc->addr); + id = entry ? entry->addr.id : -1; spin_unlock_bh(&msk->pm.lock); - if (entry) - return entry->addr.id; - memset(&new_entry, 0, sizeof(struct mptcp_pm_addr_entry)); - new_entry.addr = *skc; - new_entry.addr.id = 0; - new_entry.flags = MPTCP_PM_ADDR_FLAG_IMPLICIT; + if (id != -1) + return id; - if (new_entry.addr.port == msk_sport) - new_entry.addr.port = 0; + if (skc->addr.port == msk_sport) + skc->addr.port = 0; - return mptcp_userspace_pm_append_new_local_addr(msk, &new_entry, true); + return mptcp_userspace_pm_append_new_local_addr(msk, skc, true); } bool mptcp_userspace_pm_is_backup(struct mptcp_sock *msk, struct mptcp_addr_info *skc) { struct mptcp_pm_addr_entry *entry; - bool backup = false; + bool backup; spin_lock_bh(&msk->pm.lock); - list_for_each_entry(entry, &msk->pm.userspace_pm_local_addr_list, list) { - if (mptcp_addresses_equal(&entry->addr, skc, false)) { - backup = !!(entry->flags & MPTCP_PM_ADDR_FLAG_BACKUP); - break; - } - } + entry = mptcp_userspace_pm_lookup_addr(msk, skc); + backup = entry && !!(entry->flags & MPTCP_PM_ADDR_FLAG_BACKUP); spin_unlock_bh(&msk->pm.lock); return backup; @@ -607,13 +607,12 @@ int mptcp_userspace_pm_set_flags(struct sk_buff *skb, struct genl_info *info) bkup = 1; spin_lock_bh(&msk->pm.lock); - list_for_each_entry(entry, &msk->pm.userspace_pm_local_addr_list, list) { - if (mptcp_addresses_equal(&entry->addr, &loc.addr, false)) { - if (bkup) - entry->flags |= MPTCP_PM_ADDR_FLAG_BACKUP; - else - entry->flags &= ~MPTCP_PM_ADDR_FLAG_BACKUP; - } + entry = mptcp_userspace_pm_lookup_addr(msk, &loc.addr); + if (entry) { + if (bkup) + entry->flags |= MPTCP_PM_ADDR_FLAG_BACKUP; + else + entry->flags &= ~MPTCP_PM_ADDR_FLAG_BACKUP; } spin_unlock_bh(&msk->pm.lock); diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h index ad30b4b48143..04f33abd410d 100644 --- a/net/mptcp/protocol.h +++ b/net/mptcp/protocol.h @@ -1136,8 +1136,10 @@ bool mptcp_pm_add_addr_signal(struct mptcp_sock *msk, unsigned int opt_size, bool mptcp_pm_rm_addr_signal(struct mptcp_sock *msk, unsigned int remaining, struct mptcp_rm_list *rm_list); int mptcp_pm_get_local_id(struct mptcp_sock *msk, struct sock_common *skc); -int mptcp_pm_nl_get_local_id(struct mptcp_sock *msk, struct mptcp_addr_info *skc); -int mptcp_userspace_pm_get_local_id(struct mptcp_sock *msk, struct mptcp_addr_info *skc); +int mptcp_pm_nl_get_local_id(struct mptcp_sock *msk, + struct mptcp_pm_addr_entry *skc); +int mptcp_userspace_pm_get_local_id(struct mptcp_sock *msk, + struct mptcp_pm_addr_entry *skc); bool mptcp_pm_is_backup(struct mptcp_sock *msk, struct sock_common *skc); bool mptcp_pm_nl_is_backup(struct mptcp_sock *msk, struct mptcp_addr_info *skc); bool mptcp_userspace_pm_is_backup(struct mptcp_sock *msk, struct mptcp_addr_info *skc); diff --git a/net/netfilter/ipset/ip_set_hash_gen.h b/net/netfilter/ipset/ip_set_hash_gen.h index 0426d012185d..4c1082e38e3d 100644 --- a/net/netfilter/ipset/ip_set_hash_gen.h +++ b/net/netfilter/ipset/ip_set_hash_gen.h @@ -984,7 +984,7 @@ mtype_add(struct ip_set *set, void *value, const struct ip_set_ext *ext, #endif if (SET_WITH_COUNTER(set)) ip_set_init_counter(ext_counter(data, set), ext); - if (SET_WITH_COMMENT(set)) + if (SET_WITH_COMMENT(set) && !ext->target) ip_set_init_comment(set, ext_comment(data, set), ext); if (SET_WITH_SKBINFO(set)) ip_set_init_skbinfo(ext_skbinfo(data, set), ext); diff --git a/net/netfilter/ipvs/ip_vs_app.c b/net/netfilter/ipvs/ip_vs_app.c index f9132b359f0c..0c690a30a85d 100644 --- a/net/netfilter/ipvs/ip_vs_app.c +++ b/net/netfilter/ipvs/ip_vs_app.c @@ -368,7 +368,7 @@ static inline int app_tcp_pkt_out(struct ip_vs_conn *cp, struct sk_buff *skb, if (skb_ensure_writable(skb, ipvsh->len + sizeof(*th))) return 0; - th = (struct tcphdr *)(skb_network_header(skb) + ipvsh->len); + th = (struct tcphdr *)(skb->data + ipvsh->len); /* * Remember seq number in case this pkt gets resized @@ -444,7 +444,7 @@ static inline int app_tcp_pkt_in(struct ip_vs_conn *cp, struct sk_buff *skb, if (skb_ensure_writable(skb, ipvsh->len + sizeof(*th))) return 0; - th = (struct tcphdr *)(skb_network_header(skb) + ipvsh->len); + th = (struct tcphdr *)(skb->data + ipvsh->len); /* * Remember seq number in case this pkt gets resized diff --git a/net/netfilter/ipvs/ip_vs_conn.c b/net/netfilter/ipvs/ip_vs_conn.c index 7ab2b482d4bf..5ed91a8a8fdf 100644 --- a/net/netfilter/ipvs/ip_vs_conn.c +++ b/net/netfilter/ipvs/ip_vs_conn.c @@ -599,6 +599,9 @@ ip_vs_bind_dest(struct ip_vs_conn *cp, struct ip_vs_dest *dest) flags = cp->flags; /* Bind with the destination and its corresponding transmitter */ if (flags & IP_VS_CONN_F_SYNC) { + /* Synced conns are hashed, so they can not get this flag */ + conn_flags &= ~IP_VS_CONN_F_ONE_PACKET; + /* if the connection is not template and is created * by sync, preserve the activity flag. */ diff --git a/net/netfilter/ipvs/ip_vs_core.c b/net/netfilter/ipvs/ip_vs_core.c index 90619453cb6f..6207a91e93f3 100644 --- a/net/netfilter/ipvs/ip_vs_core.c +++ b/net/netfilter/ipvs/ip_vs_core.c @@ -689,7 +689,7 @@ static int sysctl_nat_icmp_send(struct netns_ipvs *ipvs) { return 0; } #endif -__sum16 ip_vs_checksum_complete(struct sk_buff *skb, int offset) +static __sum16 ip_vs_checksum_complete(struct sk_buff *skb, int offset) { return csum_fold(skb_checksum(skb, offset, skb->len - offset, 0)); } @@ -746,13 +746,13 @@ static int ip_vs_route_me_harder(struct netns_ipvs *ipvs, int af, * - inout: 1=in->out, 0=out->in */ void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, - struct ip_vs_conn *cp, int inout) + struct ip_vs_conn *cp, int inout, unsigned int toff, + bool has_ports) { struct iphdr *iph = ip_hdr(skb); - unsigned int icmp_offset = iph->ihl*4; - struct icmphdr *icmph = (struct icmphdr *)(skb_network_header(skb) + - icmp_offset); + struct icmphdr *icmph = (struct icmphdr *)(skb->data + toff); struct iphdr *ciph = (struct iphdr *)(icmph + 1); + unsigned int coff __maybe_unused = toff + sizeof(struct icmphdr); if (inout) { iph->saddr = cp->vaddr.ip; @@ -767,8 +767,7 @@ void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, } /* the TCP/UDP/SCTP port */ - if (IPPROTO_TCP == ciph->protocol || IPPROTO_UDP == ciph->protocol || - IPPROTO_SCTP == ciph->protocol) { + if (has_ports) { __be16 *ports = (void *)ciph + ciph->ihl*4; if (inout) @@ -779,48 +778,40 @@ void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, /* And finally the ICMP checksum */ icmph->checksum = 0; - icmph->checksum = ip_vs_checksum_complete(skb, icmp_offset); + icmph->checksum = ip_vs_checksum_complete(skb, toff); skb->ip_summed = CHECKSUM_UNNECESSARY; if (inout) - IP_VS_DBG_PKT(11, AF_INET, pp, skb, (void *)ciph - (void *)iph, - "Forwarding altered outgoing ICMP"); + IP_VS_DBG_PKT(11, AF_INET, pp, skb, coff, + "Forwarding altered outgoing ICMP"); else - IP_VS_DBG_PKT(11, AF_INET, pp, skb, (void *)ciph - (void *)iph, - "Forwarding altered incoming ICMP"); + IP_VS_DBG_PKT(11, AF_INET, pp, skb, coff, + "Forwarding altered incoming ICMP"); } #ifdef CONFIG_IP_VS_IPV6 void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, - struct ip_vs_conn *cp, int inout) + struct ip_vs_conn *cp, int inout, unsigned int toff, + bool has_ports, struct ip_vs_iphdr *ciph) { struct ipv6hdr *iph = ipv6_hdr(skb); - unsigned int icmp_offset = 0; - unsigned int offs = 0; /* header offset*/ - int protocol; struct icmp6hdr *icmph; - struct ipv6hdr *ciph; - unsigned short fragoffs; + struct ipv6hdr *cih; - ipv6_find_hdr(skb, &icmp_offset, IPPROTO_ICMPV6, &fragoffs, NULL); - icmph = (struct icmp6hdr *)(skb_network_header(skb) + icmp_offset); - offs = icmp_offset + sizeof(struct icmp6hdr); - ciph = (struct ipv6hdr *)(skb_network_header(skb) + offs); - - protocol = ipv6_find_hdr(skb, &offs, -1, &fragoffs, NULL); + icmph = (struct icmp6hdr *)(skb->data + toff); + cih = (struct ipv6hdr *)(skb->data + ciph->off); if (inout) { iph->saddr = cp->vaddr.in6; - ciph->daddr = cp->vaddr.in6; + cih->daddr = cp->vaddr.in6; } else { iph->daddr = cp->daddr.in6; - ciph->saddr = cp->daddr.in6; + cih->saddr = cp->daddr.in6; } /* the TCP/UDP/SCTP port */ - if (!fragoffs && (IPPROTO_TCP == protocol || IPPROTO_UDP == protocol || - IPPROTO_SCTP == protocol)) { - __be16 *ports = (void *)(skb_network_header(skb) + offs); + if (has_ports) { + __be16 *ports = (void *)(skb->data + ciph->len); IP_VS_DBG(11, "%s() changed port %d to %d\n", __func__, ntohs(inout ? ports[1] : ports[0]), @@ -833,19 +824,17 @@ void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, /* And finally the ICMP checksum */ icmph->icmp6_cksum = ~csum_ipv6_magic(&iph->saddr, &iph->daddr, - skb->len - icmp_offset, + skb->len - toff, IPPROTO_ICMPV6, 0); - skb->csum_start = skb_network_header(skb) - skb->head + icmp_offset; + skb->csum_start = skb_headroom(skb) + toff; skb->csum_offset = offsetof(struct icmp6hdr, icmp6_cksum); skb->ip_summed = CHECKSUM_PARTIAL; if (inout) - IP_VS_DBG_PKT(11, AF_INET6, pp, skb, - (void *)ciph - (void *)iph, + IP_VS_DBG_PKT(11, AF_INET6, pp, skb, ciph->off, "Forwarding altered outgoing ICMPv6"); else - IP_VS_DBG_PKT(11, AF_INET6, pp, skb, - (void *)ciph - (void *)iph, + IP_VS_DBG_PKT(11, AF_INET6, pp, skb, ciph->off, "Forwarding altered incoming ICMPv6"); } #endif @@ -855,36 +844,41 @@ void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, */ static int handle_response_icmp(int af, struct sk_buff *skb, union nf_inet_addr *snet, - __u8 protocol, struct ip_vs_conn *cp, + struct ip_vs_conn *cp, struct ip_vs_protocol *pp, - unsigned int offset, unsigned int ihl, - unsigned int hooknum) + struct ip_vs_iphdr *ciph, + unsigned int toff, unsigned int hooknum) { + int iproto = af == AF_INET6 ? IPPROTO_ICMPV6 : IPPROTO_ICMP; unsigned int verdict = NF_DROP; + unsigned int ctoff = ciph->len; + bool has_ports = false; if (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ) goto after_nat; /* Ensure the checksum is correct */ - if (!skb_csum_unnecessary(skb) && ip_vs_checksum_complete(skb, ihl)) { + if (!ip_vs_checksum_common_check(skb, toff, iproto, af)) { /* Failed checksum! */ IP_VS_DBG_BUF(1, "Forward ICMP: failed checksum from %s!\n", IP_VS_DBG_ADDR(af, snet)); goto out; } - if (IPPROTO_TCP == protocol || IPPROTO_UDP == protocol || - IPPROTO_SCTP == protocol) - offset += 2 * sizeof(__u16); - if (skb_ensure_writable(skb, offset)) + if (ciph->protocol == IPPROTO_TCP || ciph->protocol == IPPROTO_UDP || + ciph->protocol == IPPROTO_SCTP) { + ctoff += 2 * sizeof(__u16); + has_ports = true; + } + if (skb_ensure_writable(skb, ctoff)) goto out; #ifdef CONFIG_IP_VS_IPV6 if (af == AF_INET6) - ip_vs_nat_icmp_v6(skb, pp, cp, 1); + ip_vs_nat_icmp_v6(skb, pp, cp, 1, toff, has_ports, ciph); else #endif - ip_vs_nat_icmp(skb, pp, cp, 1); + ip_vs_nat_icmp(skb, pp, cp, 1, toff, has_ports); if (ip_vs_route_me_harder(cp->ipvs, af, skb, hooknum)) goto out; @@ -912,9 +906,9 @@ static int handle_response_icmp(int af, struct sk_buff *skb, * Currently handles error types - unreachable, quench, ttl exceeded. */ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, - int *related, unsigned int hooknum) + int *related, unsigned int hooknum, + struct ip_vs_iphdr *ipvsh) { - struct iphdr *iph; struct icmphdr _icmph, *ic; struct iphdr _ciph, *cih; /* The ip header contained within the ICMP */ struct ip_vs_iphdr ciph; @@ -929,17 +923,19 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, if (ip_is_fragment(ip_hdr(skb))) { if (ip_vs_gather_frags(ipvs, skb, ip_vs_defrag_user(hooknum))) return NF_STOLEN; + if (!ip_vs_fill_iph_skb(AF_INET, skb, false, ipvsh)) + return NF_ACCEPT; } - iph = ip_hdr(skb); - offset = ihl = iph->ihl * 4; + ihl = ipvsh->len; + offset = ipvsh->len; ic = skb_header_pointer(skb, offset, sizeof(_icmph), &_icmph); if (ic == NULL) return NF_DROP; IP_VS_DBG(12, "Outgoing ICMP (%d,%d) %pI4->%pI4\n", ic->type, ntohs(icmp_id(ic)), - &iph->saddr, &iph->daddr); + &ipvsh->saddr.ip, &ipvsh->daddr.ip); /* * Work through seeing if this is for us. @@ -958,7 +954,7 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, /* Now find the contained IP header */ offset += sizeof(_icmph); cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph); - if (cih == NULL) + if (!(cih && cih->version == 4 && cih->ihl >= 5)) return NF_ACCEPT; /* The packet looks wrong, ignore */ pp = ip_vs_proto_get(cih->protocol); @@ -966,8 +962,7 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, return NF_ACCEPT; /* Is the embedded protocol header present? */ - if (unlikely(cih->frag_off & htons(IP_OFFSET) && - pp->dont_defrag)) + if (unlikely(cih->frag_off & htons(IP_OFFSET) && !pp->dont_defrag)) return NF_ACCEPT; IP_VS_DBG_PKT(11, AF_INET, pp, skb, offset, @@ -981,9 +976,9 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, if (!cp) return NF_ACCEPT; - snet.ip = iph->saddr; - return handle_response_icmp(AF_INET, skb, &snet, cih->protocol, cp, - pp, ciph.len, ihl, hooknum); + snet.ip = ipvsh->saddr.ip; + return handle_response_icmp(AF_INET, skb, &snet, cp, pp, &ciph, ihl, + hooknum); } #ifdef CONFIG_IP_VS_IPV6 @@ -996,7 +991,6 @@ static int ip_vs_out_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, struct ip_vs_conn *cp; struct ip_vs_protocol *pp; union nf_inet_addr snet; - unsigned int offset; *related = 1; ic = frag_safe_skb_hp(skb, ipvsh->len, sizeof(_icmph), &_icmph); @@ -1032,6 +1026,10 @@ static int ip_vs_out_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, if (!pp) return NF_ACCEPT; + /* Is the embedded protocol header present? */ + if (unlikely(ciph.fragoffs && !pp->dont_defrag)) + return NF_ACCEPT; + /* The embedded headers contain source and dest in reverse order */ cp = INDIRECT_CALL_1(pp->conn_out_get, ip_vs_conn_out_get_proto, ipvs, AF_INET6, skb, &ciph); @@ -1039,9 +1037,8 @@ static int ip_vs_out_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, return NF_ACCEPT; snet.in6 = ciph.saddr.in6; - offset = ciph.len; - return handle_response_icmp(AF_INET6, skb, &snet, ciph.protocol, cp, - pp, offset, ipvsh->len, hooknum); + return handle_response_icmp(AF_INET6, skb, &snet, cp, pp, &ciph, + ipvsh->len, hooknum); } #endif @@ -1367,7 +1364,8 @@ ip_vs_out_hook(void *priv, struct sk_buff *skb, const struct nf_hook_state *stat #endif if (unlikely(iph.protocol == IPPROTO_ICMP)) { int related; - int verdict = ip_vs_out_icmp(ipvs, skb, &related, hooknum); + int verdict = ip_vs_out_icmp(ipvs, skb, &related, + hooknum, &iph); if (related) return verdict; @@ -1575,9 +1573,8 @@ static int ipvs_gre_decap(struct netns_ipvs *ipvs, struct sk_buff *skb, */ static int ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, - unsigned int hooknum) + unsigned int hooknum, struct ip_vs_iphdr *iph) { - struct iphdr *iph; struct icmphdr _icmph, *ic; struct iphdr _ciph, *cih; /* The ip header contained within the ICMP */ struct ip_vs_iphdr ciph; @@ -1587,7 +1584,7 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, unsigned int offset, offset2, ihl, verdict; bool tunnel, new_cp = false; union nf_inet_addr *raddr; - char *outer_proto = "IPIP"; + char *outer_proto __maybe_unused = "IPIP"; unsigned int hlen_ipip; int ulen = 0; @@ -1597,17 +1594,19 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, if (ip_is_fragment(ip_hdr(skb))) { if (ip_vs_gather_frags(ipvs, skb, ip_vs_defrag_user(hooknum))) return NF_STOLEN; + if (!ip_vs_fill_iph_skb(AF_INET, skb, false, iph)) + return NF_ACCEPT; } - iph = ip_hdr(skb); - offset = ihl = iph->ihl * 4; + ihl = iph->len; + offset = iph->len; ic = skb_header_pointer(skb, offset, sizeof(_icmph), &_icmph); if (ic == NULL) return NF_DROP; IP_VS_DBG(12, "Incoming ICMP (%d,%d) %pI4->%pI4\n", ic->type, ntohs(icmp_id(ic)), - &iph->saddr, &iph->daddr); + &iph->saddr.ip, &iph->daddr.ip); /* * Work through seeing if this is for us. @@ -1689,8 +1688,7 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, pp = pd->pp; /* Is the embedded protocol header present? */ - if (unlikely(cih->frag_off & htons(IP_OFFSET) && - pp->dont_defrag)) + if (unlikely(cih->frag_off & htons(IP_OFFSET) && !pp->dont_defrag)) return NF_ACCEPT; IP_VS_DBG_PKT(11, AF_INET, pp, skb, offset, @@ -1698,7 +1696,6 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, offset2 = offset; ip_vs_fill_iph_skb_icmp(AF_INET, skb, offset, !tunnel, &ciph); - offset = ciph.len; /* The embedded headers contain source and dest in reverse order. * For IPIP/UDP/GRE tunnel this is error for request, not for reply. @@ -1720,10 +1717,11 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, verdict = NF_DROP; /* Ensure the checksum is correct */ - if (!skb_csum_unnecessary(skb) && ip_vs_checksum_complete(skb, ihl)) { + if ((IP_VS_FWD_METHOD(cp) == IP_VS_CONN_F_MASQ || tunnel) && + !ip_vs_checksum_common_check(skb, ihl, IPPROTO_ICMP, AF_INET)) { /* Failed checksum! */ IP_VS_DBG(1, "Incoming ICMP: failed checksum from %pI4!\n", - &iph->saddr); + &iph->saddr.ip); goto out; } @@ -1791,10 +1789,7 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, /* do the statistics and put it back */ ip_vs_in_stats(cp, skb); - if (IPPROTO_TCP == cih->protocol || IPPROTO_UDP == cih->protocol || - IPPROTO_SCTP == cih->protocol) - offset += 2 * sizeof(__u16); - verdict = ip_vs_icmp_xmit(skb, cp, pp, offset, hooknum, &ciph); + verdict = ip_vs_icmp_xmit(skb, cp, pp, iph->len, hooknum, &ciph); out: if (likely(!new_cp)) @@ -1854,8 +1849,8 @@ static int ip_vs_in_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, return NF_ACCEPT; pp = pd->pp; - /* Cannot handle fragmented embedded protocol */ - if (ciph.fragoffs) + /* Is the embedded protocol header present? */ + if (ciph.fragoffs && !pp->dont_defrag) return NF_ACCEPT; IP_VS_DBG_PKT(11, AF_INET6, pp, skb, offset, @@ -1879,23 +1874,22 @@ static int ip_vs_in_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, new_cp = true; } - /* VS/TUN, VS/DR and LOCALNODE just let it go */ - if ((hooknum == NF_INET_LOCAL_OUT) && - (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ)) { - verdict = NF_ACCEPT; + verdict = NF_DROP; + + /* Ensure the checksum is correct */ + if (IP_VS_FWD_METHOD(cp) == IP_VS_CONN_F_MASQ && + !ip_vs_checksum_common_check(skb, iph->len, IPPROTO_ICMPV6, + AF_INET6)) { + /* Failed checksum! */ + IP_VS_DBG(1, "Incoming ICMPv6: failed checksum from %pI6c!\n", + &iph->saddr); goto out; } /* do the statistics and put it back */ ip_vs_in_stats(cp, skb); - /* Need to mangle contained IPv6 header in ICMPv6 packet */ - offset = ciph.len; - if (IPPROTO_TCP == ciph.protocol || IPPROTO_UDP == ciph.protocol || - IPPROTO_SCTP == ciph.protocol) - offset += 2 * sizeof(__u16); /* Also mangle ports */ - - verdict = ip_vs_icmp_xmit_v6(skb, cp, pp, offset, hooknum, &ciph); + verdict = ip_vs_icmp_xmit_v6(skb, cp, pp, iph->len, hooknum, &ciph); out: if (likely(!new_cp)) @@ -1974,7 +1968,7 @@ ip_vs_in_hook(void *priv, struct sk_buff *skb, const struct nf_hook_state *state if (unlikely(iph.protocol == IPPROTO_ICMP)) { int related; int verdict = ip_vs_in_icmp(ipvs, skb, &related, - hooknum); + hooknum, &iph); if (related) return verdict; @@ -2110,6 +2104,7 @@ ip_vs_forward_icmp(void *priv, struct sk_buff *skb, const struct nf_hook_state *state) { struct netns_ipvs *ipvs = net_ipvs(state->net); + struct ip_vs_iphdr iphdr; int r; /* ipvs enabled in this netns ? */ @@ -2119,10 +2114,9 @@ ip_vs_forward_icmp(void *priv, struct sk_buff *skb, if (state->pf == NFPROTO_IPV4) { if (ip_hdr(skb)->protocol != IPPROTO_ICMP) return NF_ACCEPT; + ip_vs_fill_iph_skb(AF_INET, skb, false, &iphdr); #ifdef CONFIG_IP_VS_IPV6 } else { - struct ip_vs_iphdr iphdr; - ip_vs_fill_iph_skb(AF_INET6, skb, false, &iphdr); if (iphdr.protocol != IPPROTO_ICMPV6) @@ -2132,7 +2126,7 @@ ip_vs_forward_icmp(void *priv, struct sk_buff *skb, #endif } - return ip_vs_in_icmp(ipvs, skb, &r, state->hook); + return ip_vs_in_icmp(ipvs, skb, &r, state->hook, &iphdr); } static const struct nf_hook_ops ip_vs_ops4[] = { diff --git a/net/netfilter/ipvs/ip_vs_proto_sctp.c b/net/netfilter/ipvs/ip_vs_proto_sctp.c index c67317be17df..3dbd3096e163 100644 --- a/net/netfilter/ipvs/ip_vs_proto_sctp.c +++ b/net/netfilter/ipvs/ip_vs_proto_sctp.c @@ -11,7 +11,7 @@ static int sctp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, - unsigned int sctphoff); + struct ip_vs_iphdr *iph); static int sctp_conn_schedule(struct netns_ipvs *ipvs, int af, struct sk_buff *skb, @@ -109,7 +109,7 @@ sctp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, int ret; /* Some checks before mangling */ - if (!sctp_csum_check(cp->af, skb, pp, sctphoff)) + if (!sctp_csum_check(cp->af, skb, pp, iph)) return 0; /* Call application helper if needed */ @@ -121,7 +121,7 @@ sctp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, payload_csum = true; } - sctph = (void *) skb_network_header(skb) + sctphoff; + sctph = (void *)skb->data + sctphoff; /* Only update csum if we really have to */ if (sctph->source != cp->vport || payload_csum || @@ -157,7 +157,7 @@ sctp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, int ret; /* Some checks before mangling */ - if (!sctp_csum_check(cp->af, skb, pp, sctphoff)) + if (!sctp_csum_check(cp->af, skb, pp, iph)) return 0; /* Call application helper if needed */ @@ -169,7 +169,7 @@ sctp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, payload_csum = true; } - sctph = (void *) skb_network_header(skb) + sctphoff; + sctph = (void *)skb->data + sctphoff; /* Only update csum if we really have to */ if (sctph->dest != cp->dport || payload_csum || @@ -187,19 +187,22 @@ sctp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, static int sctp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, - unsigned int sctphoff) + struct ip_vs_iphdr *iph) { + unsigned int sctphoff = iph->len; struct sctphdr *sh; __le32 cmp, val; + if (!ip_vs_checksum_needed(skb, af)) + return 1; sh = (struct sctphdr *)(skb->data + sctphoff); cmp = sh->checksum; val = sctp_compute_cksum(skb, sctphoff); if (val != cmp) { /* CRC failure, dump it. */ - IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, - "Failed checksum for"); + IP_VS_DBG_RL_PKT(0, af, pp, skb, iph->off, + "Failed checksum for"); return 0; } return 1; diff --git a/net/netfilter/ipvs/ip_vs_proto_tcp.c b/net/netfilter/ipvs/ip_vs_proto_tcp.c index b382810156b2..1ac9c233537d 100644 --- a/net/netfilter/ipvs/ip_vs_proto_tcp.c +++ b/net/netfilter/ipvs/ip_vs_proto_tcp.c @@ -30,7 +30,7 @@ static int tcp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, - unsigned int tcphoff); + struct ip_vs_iphdr *iph); static int tcp_conn_schedule(struct netns_ipvs *ipvs, int af, struct sk_buff *skb, @@ -167,7 +167,7 @@ tcp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, int ret; /* Some checks before mangling */ - if (!tcp_csum_check(cp->af, skb, pp, tcphoff)) + if (!tcp_csum_check(cp->af, skb, pp, iph)) return 0; /* Call application helper if needed */ @@ -180,7 +180,7 @@ tcp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, payload_csum = true; } - tcph = (void *)skb_network_header(skb) + tcphoff; + tcph = (void *)skb->data + tcphoff; tcph->source = cp->vport; /* Adjust TCP checksums */ @@ -245,7 +245,7 @@ tcp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, int ret; /* Some checks before mangling */ - if (!tcp_csum_check(cp->af, skb, pp, tcphoff)) + if (!tcp_csum_check(cp->af, skb, pp, iph)) return 0; /* @@ -261,7 +261,7 @@ tcp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, payload_csum = true; } - tcph = (void *)skb_network_header(skb) + tcphoff; + tcph = (void *)skb->data + tcphoff; tcph->dest = cp->dport; /* @@ -303,41 +303,13 @@ tcp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, static int tcp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, - unsigned int tcphoff) + struct ip_vs_iphdr *iph) { - switch (skb->ip_summed) { - case CHECKSUM_NONE: - skb->csum = skb_checksum(skb, tcphoff, skb->len - tcphoff, 0); - fallthrough; - case CHECKSUM_COMPLETE: -#ifdef CONFIG_IP_VS_IPV6 - if (af == AF_INET6) { - if (csum_ipv6_magic(&ipv6_hdr(skb)->saddr, - &ipv6_hdr(skb)->daddr, - skb->len - tcphoff, - IPPROTO_TCP, - skb->csum)) { - IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, - "Failed checksum for"); - return 0; - } - } else -#endif - if (csum_tcpudp_magic(ip_hdr(skb)->saddr, - ip_hdr(skb)->daddr, - skb->len - tcphoff, - ip_hdr(skb)->protocol, - skb->csum)) { - IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, - "Failed checksum for"); - return 0; - } - break; - default: - /* No need to checksum. */ - break; + if (!ip_vs_checksum_common_check(skb, iph->len, IPPROTO_TCP, af)) { + IP_VS_DBG_RL_PKT(0, af, pp, skb, iph->off, + "Failed checksum for"); + return 0; } - return 1; } diff --git a/net/netfilter/ipvs/ip_vs_proto_udp.c b/net/netfilter/ipvs/ip_vs_proto_udp.c index dbd4155bb075..96ac882df15c 100644 --- a/net/netfilter/ipvs/ip_vs_proto_udp.c +++ b/net/netfilter/ipvs/ip_vs_proto_udp.c @@ -26,7 +26,7 @@ static int udp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, - unsigned int udphoff); + struct ip_vs_iphdr *iph); static int udp_conn_schedule(struct netns_ipvs *ipvs, int af, struct sk_buff *skb, @@ -156,7 +156,7 @@ udp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, int ret; /* Some checks before mangling */ - if (!udp_csum_check(cp->af, skb, pp, udphoff)) + if (!udp_csum_check(cp->af, skb, pp, iph)) return 0; /* @@ -171,7 +171,7 @@ udp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, payload_csum = true; } - udph = (void *)skb_network_header(skb) + udphoff; + udph = (void *)skb->data + udphoff; udph->source = cp->vport; /* @@ -239,7 +239,7 @@ udp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, int ret; /* Some checks before mangling */ - if (!udp_csum_check(cp->af, skb, pp, udphoff)) + if (!udp_csum_check(cp->af, skb, pp, iph)) return 0; /* @@ -255,7 +255,7 @@ udp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, payload_csum = true; } - udph = (void *)skb_network_header(skb) + udphoff; + udph = (void *)skb->data + udphoff; udph->dest = cp->dport; /* @@ -299,48 +299,20 @@ udp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, static int udp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, - unsigned int udphoff) + struct ip_vs_iphdr *iph) { struct udphdr _udph, *uh; - uh = skb_header_pointer(skb, udphoff, sizeof(_udph), &_udph); + uh = skb_header_pointer(skb, iph->len, sizeof(_udph), &_udph); if (uh == NULL) return 0; - if (uh->check != 0) { - switch (skb->ip_summed) { - case CHECKSUM_NONE: - skb->csum = skb_checksum(skb, udphoff, - skb->len - udphoff, 0); - fallthrough; - case CHECKSUM_COMPLETE: -#ifdef CONFIG_IP_VS_IPV6 - if (af == AF_INET6) { - if (csum_ipv6_magic(&ipv6_hdr(skb)->saddr, - &ipv6_hdr(skb)->daddr, - skb->len - udphoff, - IPPROTO_UDP, - skb->csum)) { - IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, - "Failed checksum for"); - return 0; - } - } else -#endif - if (csum_tcpudp_magic(ip_hdr(skb)->saddr, - ip_hdr(skb)->daddr, - skb->len - udphoff, - ip_hdr(skb)->protocol, - skb->csum)) { - IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, - "Failed checksum for"); - return 0; - } - break; - default: - /* No need to checksum. */ - break; - } + if (!uh->check) + return 1; + if (!ip_vs_checksum_common_check(skb, iph->len, IPPROTO_UDP, af)) { + IP_VS_DBG_RL_PKT(0, af, pp, skb, iph->off, + "Failed checksum for"); + return 0; } return 1; } diff --git a/net/netfilter/ipvs/ip_vs_xmit.c b/net/netfilter/ipvs/ip_vs_xmit.c index fa1b03c3201f..b85420a3a14c 100644 --- a/net/netfilter/ipvs/ip_vs_xmit.c +++ b/net/netfilter/ipvs/ip_vs_xmit.c @@ -1499,20 +1499,22 @@ ip_vs_dr_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, */ int ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, - struct ip_vs_protocol *pp, int offset, unsigned int hooknum, - struct ip_vs_iphdr *iph) + struct ip_vs_protocol *pp, unsigned int toff, + unsigned int hooknum, struct ip_vs_iphdr *ciph) { struct rtable *rt; /* Route to the other host */ int rc; int local; int rt_mode, was_input; + bool has_ports = false; + unsigned int wlen; /* The ICMP packet for VS/TUN, VS/DR and LOCALNODE will be forwarded directly here, because there is no need to translate address/port back */ if (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ) { if (cp->packet_xmit) - rc = cp->packet_xmit(skb, cp, pp, iph); + rc = cp->packet_xmit(skb, cp, pp, ciph); else rc = NF_ACCEPT; /* do not touch skb anymore */ @@ -1530,7 +1532,7 @@ ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, IP_VS_RT_MODE_LOCAL | IP_VS_RT_MODE_NON_LOCAL | IP_VS_RT_MODE_RDR : IP_VS_RT_MODE_NON_LOCAL; local = __ip_vs_get_out_rt(cp->ipvs, cp->af, skb, cp->dest, cp->daddr.ip, rt_mode, - NULL, iph); + NULL, ciph); if (local < 0) goto tx_error; rt = skb_rtable(skb); @@ -1561,14 +1563,21 @@ ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, goto tx_error; } + wlen = ciph->len; + if (ciph->protocol == IPPROTO_TCP || ciph->protocol == IPPROTO_UDP || + ciph->protocol == IPPROTO_SCTP) { + wlen += 2 * sizeof(__u16); /* Also mangle ports */ + has_ports = true; + } + /* copy-on-write the packet before mangling it */ - if (skb_ensure_writable(skb, offset)) + if (skb_ensure_writable(skb, wlen)) goto tx_error; if (skb_cow(skb, rt->dst.dev->hard_header_len)) goto tx_error; - ip_vs_nat_icmp(skb, pp, cp, 0); + ip_vs_nat_icmp(skb, pp, cp, 0, toff, has_ports); /* Another hack: avoid icmp_send in ip_fragment */ skb->ignore_df = 1; @@ -1584,10 +1593,12 @@ ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, #ifdef CONFIG_IP_VS_IPV6 int ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, - struct ip_vs_protocol *pp, int offset, unsigned int hooknum, - struct ip_vs_iphdr *ipvsh) + struct ip_vs_protocol *pp, unsigned int toff, + unsigned int hooknum, struct ip_vs_iphdr *ciph) { + bool has_ports = false; struct rt6_info *rt; /* Route to the other host */ + unsigned int wlen; int rc; int local; int rt_mode; @@ -1597,7 +1608,7 @@ ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, translate address/port back */ if (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ) { if (cp->packet_xmit) - rc = cp->packet_xmit(skb, cp, pp, ipvsh); + rc = cp->packet_xmit(skb, cp, pp, ciph); else rc = NF_ACCEPT; /* do not touch skb anymore */ @@ -1614,7 +1625,7 @@ ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, IP_VS_RT_MODE_LOCAL | IP_VS_RT_MODE_NON_LOCAL | IP_VS_RT_MODE_RDR : IP_VS_RT_MODE_NON_LOCAL; local = __ip_vs_get_out_rt_v6(cp->ipvs, cp->af, skb, cp->dest, - &cp->daddr.in6, NULL, ipvsh, 0, rt_mode); + &cp->daddr.in6, NULL, ciph, 0, rt_mode); if (local < 0) goto tx_error; rt = dst_rt6_info(skb_dst(skb)); @@ -1645,14 +1656,21 @@ ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, goto tx_error; } + wlen = ciph->len; + if (ciph->protocol == IPPROTO_TCP || ciph->protocol == IPPROTO_UDP || + ciph->protocol == IPPROTO_SCTP) { + wlen += 2 * sizeof(__u16); /* Also mangle ports */ + has_ports = true; + } + /* copy-on-write the packet before mangling it */ - if (skb_ensure_writable(skb, offset)) + if (skb_ensure_writable(skb, wlen)) goto tx_error; if (skb_cow(skb, rt->dst.dev->hard_header_len)) goto tx_error; - ip_vs_nat_icmp_v6(skb, pp, cp, 0); + ip_vs_nat_icmp_v6(skb, pp, cp, 0, toff, has_ports, ciph); /* Another hack: avoid icmp_send in ip_fragment */ skb->ignore_df = 1; diff --git a/net/netfilter/nf_conntrack_broadcast.c b/net/netfilter/nf_conntrack_broadcast.c index f9528d4db0a8..93c501d9d399 100644 --- a/net/netfilter/nf_conntrack_broadcast.c +++ b/net/netfilter/nf_conntrack_broadcast.c @@ -72,6 +72,7 @@ int nf_conntrack_broadcast_help(struct sk_buff *skb, exp->flags = NF_CT_EXPECT_PERMANENT; exp->class = NF_CT_EXPECT_CLASS_DEFAULT; rcu_assign_pointer(exp->helper, helper); + rcu_assign_pointer(exp->assign_helper, NULL); write_pnet(&exp->net, net); #ifdef CONFIG_NF_CONNTRACK_ZONES exp->zone = ct->zone; diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c index 423080cf86a4..0c457e159727 100644 --- a/net/netfilter/nf_conntrack_core.c +++ b/net/netfilter/nf_conntrack_core.c @@ -1773,14 +1773,17 @@ init_conntrack(struct net *net, struct nf_conn *tmpl, spin_lock_bh(&nf_conntrack_expect_lock); exp = nf_ct_find_expectation(net, zone, tuple, !tmpl || nf_ct_is_confirmed(tmpl)); if (exp) { + struct nf_conntrack_helper *assign_helper; + /* Welcome, Mr. Bond. We've been expecting you... */ __set_bit(IPS_EXPECTED_BIT, &ct->status); /* exp->master safe, refcnt bumped in nf_ct_find_expectation */ ct->master = exp->master; - if (exp->helper) { + assign_helper = rcu_dereference(exp->assign_helper); + if (assign_helper) { help = nf_ct_helper_ext_add(ct, GFP_ATOMIC); if (help) - rcu_assign_pointer(help->helper, exp->helper); + rcu_assign_pointer(help->helper, assign_helper); } #ifdef CONFIG_NF_CONNTRACK_MARK diff --git a/net/netfilter/nf_conntrack_expect.c b/net/netfilter/nf_conntrack_expect.c index bb8b87f9ee50..a8929885485b 100644 --- a/net/netfilter/nf_conntrack_expect.c +++ b/net/netfilter/nf_conntrack_expect.c @@ -344,6 +344,7 @@ void nf_ct_expect_init(struct nf_conntrack_expect *exp, unsigned int class, helper = rcu_dereference(help->helper); rcu_assign_pointer(exp->helper, helper); + rcu_assign_pointer(exp->assign_helper, NULL); write_pnet(&exp->net, net); #ifdef CONFIG_NF_CONNTRACK_ZONES exp->zone = ct->zone; diff --git a/net/netfilter/nf_conntrack_h323_main.c b/net/netfilter/nf_conntrack_h323_main.c index 791aafe9f396..c42547284f35 100644 --- a/net/netfilter/nf_conntrack_h323_main.c +++ b/net/netfilter/nf_conntrack_h323_main.c @@ -642,7 +642,7 @@ static int expect_h245(struct sk_buff *skb, struct nf_conn *ct, &ct->tuplehash[!dir].tuple.src.u3, &ct->tuplehash[!dir].tuple.dst.u3, IPPROTO_TCP, NULL, &port); - rcu_assign_pointer(exp->helper, &nf_conntrack_helper_h245); + rcu_assign_pointer(exp->assign_helper, &nf_conntrack_helper_h245); nathook = rcu_dereference(nfct_h323_nat_hook); if (memcmp(&ct->tuplehash[dir].tuple.src.u3, @@ -766,7 +766,7 @@ static int expect_callforwarding(struct sk_buff *skb, nf_ct_expect_init(exp, NF_CT_EXPECT_CLASS_DEFAULT, nf_ct_l3num(ct), &ct->tuplehash[!dir].tuple.src.u3, &addr, IPPROTO_TCP, NULL, &port); - rcu_assign_pointer(exp->helper, nf_conntrack_helper_q931); + rcu_assign_pointer(exp->assign_helper, nf_conntrack_helper_q931); nathook = rcu_dereference(nfct_h323_nat_hook); if (memcmp(&ct->tuplehash[dir].tuple.src.u3, @@ -1233,7 +1233,7 @@ static int expect_q931(struct sk_buff *skb, struct nf_conn *ct, &ct->tuplehash[!dir].tuple.src.u3 : NULL, &ct->tuplehash[!dir].tuple.dst.u3, IPPROTO_TCP, NULL, &port); - rcu_assign_pointer(exp->helper, nf_conntrack_helper_q931); + rcu_assign_pointer(exp->assign_helper, nf_conntrack_helper_q931); exp->flags = NF_CT_EXPECT_PERMANENT; /* Accept multiple calls */ nathook = rcu_dereference(nfct_h323_nat_hook); @@ -1305,7 +1305,7 @@ static int process_gcf(struct sk_buff *skb, struct nf_conn *ct, nf_ct_expect_init(exp, NF_CT_EXPECT_CLASS_DEFAULT, nf_ct_l3num(ct), &ct->tuplehash[!dir].tuple.src.u3, &addr, IPPROTO_UDP, NULL, &port); - rcu_assign_pointer(exp->helper, nf_conntrack_helper_ras); + rcu_assign_pointer(exp->assign_helper, nf_conntrack_helper_ras); if (nf_ct_expect_related(exp, 0) == 0) { pr_debug("nf_ct_ras: expect RAS "); @@ -1522,7 +1522,7 @@ static int process_acf(struct sk_buff *skb, struct nf_conn *ct, &ct->tuplehash[!dir].tuple.src.u3, &addr, IPPROTO_TCP, NULL, &port); exp->flags = NF_CT_EXPECT_PERMANENT; - rcu_assign_pointer(exp->helper, nf_conntrack_helper_q931); + rcu_assign_pointer(exp->assign_helper, nf_conntrack_helper_q931); if (nf_ct_expect_related(exp, 0) == 0) { pr_debug("nf_ct_ras: expect Q.931 "); @@ -1576,7 +1576,7 @@ static int process_lcf(struct sk_buff *skb, struct nf_conn *ct, &ct->tuplehash[!dir].tuple.src.u3, &addr, IPPROTO_TCP, NULL, &port); exp->flags = NF_CT_EXPECT_PERMANENT; - rcu_assign_pointer(exp->helper, nf_conntrack_helper_q931); + rcu_assign_pointer(exp->assign_helper, nf_conntrack_helper_q931); if (nf_ct_expect_related(exp, 0) == 0) { pr_debug("nf_ct_ras: expect Q.931 "); diff --git a/net/netfilter/nf_conntrack_helper.c b/net/netfilter/nf_conntrack_helper.c index 9150bcfd7ca8..ea0cdb7ec915 100644 --- a/net/netfilter/nf_conntrack_helper.c +++ b/net/netfilter/nf_conntrack_helper.c @@ -419,6 +419,11 @@ static bool expect_iter_me(struct nf_conntrack_expect *exp, void *data) this = rcu_dereference_protected(exp->helper, lockdep_is_held(&nf_conntrack_expect_lock)); + if (this == me) + return true; + + this = rcu_dereference_protected(exp->assign_helper, + lockdep_is_held(&nf_conntrack_expect_lock)); return this == me; } diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c index eacbbc342c3f..80fdb875c977 100644 --- a/net/netfilter/nf_conntrack_netlink.c +++ b/net/netfilter/nf_conntrack_netlink.c @@ -2630,6 +2630,7 @@ static const struct nla_policy exp_nla_policy[CTA_EXPECT_MAX+1] = { static struct nf_conntrack_expect * ctnetlink_alloc_expect(const struct nlattr *const cda[], struct nf_conn *ct, + const struct nf_conntrack_helper *assign_helper, struct nf_conntrack_tuple *tuple, struct nf_conntrack_tuple *mask); @@ -2856,6 +2857,7 @@ static int ctnetlink_glue_attach_expect(const struct nlattr *attr, struct nf_conn *ct, u32 portid, u32 report) { + struct nf_conntrack_helper *assign_helper = NULL; struct nlattr *cda[CTA_EXPECT_MAX+1]; struct nf_conntrack_tuple tuple, mask; struct nf_conntrack_expect *exp; @@ -2871,8 +2873,18 @@ ctnetlink_glue_attach_expect(const struct nlattr *attr, struct nf_conn *ct, if (err < 0) return err; + if (cda[CTA_EXPECT_HELP_NAME]) { + const char *helpname = nla_data(cda[CTA_EXPECT_HELP_NAME]); + + assign_helper = __nf_conntrack_helper_find(helpname, + nf_ct_l3num(ct), + tuple.dst.protonum); + if (!assign_helper) + return -EOPNOTSUPP; + } + exp = ctnetlink_alloc_expect((const struct nlattr * const *)cda, ct, - &tuple, &mask); + assign_helper, &tuple, &mask); if (IS_ERR(exp)) return PTR_ERR(exp); @@ -3511,6 +3523,7 @@ ctnetlink_parse_expect_nat(const struct nlattr *attr, static struct nf_conntrack_expect * ctnetlink_alloc_expect(const struct nlattr * const cda[], struct nf_conn *ct, + const struct nf_conntrack_helper *assign_helper, struct nf_conntrack_tuple *tuple, struct nf_conntrack_tuple *mask) { @@ -3564,6 +3577,7 @@ ctnetlink_alloc_expect(const struct nlattr * const cda[], struct nf_conn *ct, exp->zone = ct->zone; #endif rcu_assign_pointer(exp->helper, helper); + rcu_assign_pointer(exp->assign_helper, assign_helper); exp->tuple = *tuple; exp->mask.src.u3 = mask->src.u3; exp->mask.src.u.all = mask->src.u.all; @@ -3619,7 +3633,7 @@ ctnetlink_create_expect(struct net *net, ct = nf_ct_tuplehash_to_ctrack(h); rcu_read_lock(); - exp = ctnetlink_alloc_expect(cda, ct, &tuple, &mask); + exp = ctnetlink_alloc_expect(cda, ct, NULL, &tuple, &mask); if (IS_ERR(exp)) { err = PTR_ERR(exp); goto err_rcu; diff --git a/net/netfilter/nf_conntrack_sip.c b/net/netfilter/nf_conntrack_sip.c index bd91b8b47f4b..4f975b83c84f 100644 --- a/net/netfilter/nf_conntrack_sip.c +++ b/net/netfilter/nf_conntrack_sip.c @@ -1386,7 +1386,7 @@ static int process_register_request(struct sk_buff *skb, unsigned int protoff, nf_ct_expect_init(exp, SIP_EXPECT_SIGNALLING, nf_ct_l3num(ct), saddr, &daddr, proto, NULL, &port); exp->timeout.expires = sip_timeout * HZ; - rcu_assign_pointer(exp->helper, helper); + rcu_assign_pointer(exp->assign_helper, helper); exp->flags = NF_CT_EXPECT_PERMANENT | NF_CT_EXPECT_INACTIVE; hooks = rcu_dereference(nf_nat_sip_hooks); @@ -1629,7 +1629,7 @@ static int sip_help_tcp(struct sk_buff *skb, unsigned int protoff, unsigned int matchoff, matchlen; unsigned int msglen, origlen; const char *dptr, *end; - s16 diff, tdiff = 0; + s32 diff, tdiff = 0; int ret = NF_ACCEPT; unsigned long clen; bool term; diff --git a/net/netfilter/nf_nat_sip.c b/net/netfilter/nf_nat_sip.c index f7be30c69b5c..a1c41defaf22 100644 --- a/net/netfilter/nf_nat_sip.c +++ b/net/netfilter/nf_nat_sip.c @@ -315,7 +315,7 @@ static unsigned int nf_nat_sip(struct sk_buff *skb, unsigned int protoff, } static void nf_nat_sip_seq_adjust(struct sk_buff *skb, unsigned int protoff, - s16 off) + s32 off) { enum ip_conntrack_info ctinfo; struct nf_conn *ct = nf_ct_get(skb, &ctinfo); diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c index 4cf565686a51..03dd9d1db12f 100644 --- a/net/netfilter/nf_tables_api.c +++ b/net/netfilter/nf_tables_api.c @@ -41,8 +41,6 @@ enum { NFT_VALIDATE_DO, }; -static struct rhltable nft_objname_ht; - static u32 nft_chain_hash(const void *data, u32 len, u32 seed); static u32 nft_chain_hash_obj(const void *data, u32 len, u32 seed); static int nft_chain_hash_cmp(struct rhashtable_compare_arg *, const void *); @@ -1457,6 +1455,10 @@ static int nf_tables_newtable(struct sk_buff *skb, const struct nfnl_info *info, if (err) goto err_chain_ht; + err = rhltable_init(&table->objname_ht, &nft_objname_ht_params); + if (err < 0) + goto err_obj_ht; + INIT_LIST_HEAD(&table->chains); INIT_LIST_HEAD(&table->sets); INIT_LIST_HEAD(&table->objects); @@ -1475,6 +1477,8 @@ static int nf_tables_newtable(struct sk_buff *skb, const struct nfnl_info *info, list_add_tail_rcu(&table->list, &nft_net->tables); return 0; err_trans: + rhltable_destroy(&table->objname_ht); +err_obj_ht: rhltable_destroy(&table->chains_ht); err_chain_ht: kfree(table->udata); @@ -1641,6 +1645,7 @@ static void nf_tables_table_destroy(struct nft_table *table) return; rhltable_destroy(&table->chains_ht); + rhltable_destroy(&table->objname_ht); kfree(table->name); kfree(table->udata); kfree(table); @@ -7578,7 +7583,7 @@ void nft_unregister_obj(struct nft_object_type *obj_type) EXPORT_SYMBOL_GPL(nft_unregister_obj); struct nft_object *nft_obj_lookup(const struct net *net, - const struct nft_table *table, + struct nft_table *table, const struct nlattr *nla, u32 objtype, u8 genmask) { @@ -7594,7 +7599,7 @@ struct nft_object *nft_obj_lookup(const struct net *net, !lockdep_commit_lock_is_held(net)); rcu_read_lock(); - list = rhltable_lookup(&nft_objname_ht, &k, nft_objname_ht_params); + list = rhltable_lookup(&table->objname_ht, &k, nft_objname_ht_params); if (!list) goto out; @@ -7873,7 +7878,7 @@ static int nf_tables_newobj(struct sk_buff *skb, const struct nfnl_info *info, if (err < 0) goto err_trans; - err = rhltable_insert(&nft_objname_ht, &obj->rhlhead, + err = rhltable_insert(&table->objname_ht, &obj->rhlhead, nft_objname_ht_params); if (err < 0) goto err_obj_ht; @@ -8052,8 +8057,8 @@ nf_tables_getobj_single(u32 portid, const struct nfnl_info *info, struct netlink_ext_ack *extack = info->extack; u8 genmask = nft_genmask_cur(info->net); u8 family = info->nfmsg->nfgen_family; - const struct nft_table *table; struct net *net = info->net; + struct nft_table *table; struct nft_object *obj; struct sk_buff *skb2; u32 objtype; @@ -9782,9 +9787,9 @@ static void nf_tables_commit_chain(struct net *net, struct nft_chain *chain) nf_tables_commit_chain_free_rules_old(g0); } -static void nft_obj_del(struct nft_object *obj) +static void nft_obj_del(struct nft_table *table, struct nft_object *obj) { - rhltable_remove(&nft_objname_ht, &obj->rhlhead, nft_objname_ht_params); + rhltable_remove(&table->objname_ht, &obj->rhlhead, nft_objname_ht_params); list_del_rcu(&obj->list); } @@ -10480,7 +10485,7 @@ static int nf_tables_commit(struct net *net, struct sk_buff *skb) break; case NFT_MSG_DELOBJ: case NFT_MSG_DESTROYOBJ: - nft_obj_del(nft_trans_obj(trans)); + nft_obj_del(table, nft_trans_obj(trans)); nf_tables_obj_notify(&ctx, nft_trans_obj(trans), trans->msg_type); break; @@ -10780,7 +10785,7 @@ static int __nf_tables_abort(struct net *net, enum nfnl_abort_action action) nft_trans_destroy(trans); } else { nft_use_dec_restore(&table->use); - nft_obj_del(nft_trans_obj(trans)); + nft_obj_del(table, nft_trans_obj(trans)); } break; case NFT_MSG_DELOBJ: @@ -11447,7 +11452,7 @@ static void __nft_release_table(struct net *net, struct nft_table *table) nft_set_destroy(&ctx, set); } list_for_each_entry_safe(obj, ne, &table->objects, list) { - nft_obj_del(obj); + nft_obj_del(table, obj); nft_use_dec(&table->use); nft_obj_destroy(&ctx, obj); } @@ -11629,10 +11634,6 @@ static int __init nf_tables_module_init(void) if (err < 0) goto err_netdev_notifier; - err = rhltable_init(&nft_objname_ht, &nft_objname_ht_params); - if (err < 0) - goto err_rht_objname; - err = nft_offload_init(); if (err < 0) goto err_offload; @@ -11655,8 +11656,6 @@ static int __init nf_tables_module_init(void) err_netlink_notifier: nft_offload_exit(); err_offload: - rhltable_destroy(&nft_objname_ht); -err_rht_objname: unregister_netdevice_notifier(&nf_tables_flowtable_notifier); err_netdev_notifier: nf_tables_core_module_exit(); @@ -11678,7 +11677,6 @@ static void __exit nf_tables_module_exit(void) unregister_pernet_subsys(&nf_tables_net_ops); cancel_work_sync(&trans_gc_work); rcu_barrier(); - rhltable_destroy(&nft_objname_ht); nf_tables_core_module_exit(); } diff --git a/net/netfilter/nft_payload.c b/net/netfilter/nft_payload.c index 36c31ad2d64c..15feb49e0b66 100644 --- a/net/netfilter/nft_payload.c +++ b/net/netfilter/nft_payload.c @@ -253,9 +253,7 @@ static int nft_payload_dump(struct sk_buff *skb, static bool nft_payload_offload_mask(struct nft_offload_reg *reg, u32 priv_len, u32 field_len) { - unsigned int remainder, delta, k; struct nft_data mask = {}; - __be32 remainder_mask; if (priv_len == field_len) { memset(®->mask, 0xff, priv_len); @@ -264,15 +262,7 @@ static bool nft_payload_offload_mask(struct nft_offload_reg *reg, return false; } - memset(&mask, 0xff, field_len); - remainder = priv_len % sizeof(u32); - if (remainder) { - k = priv_len / sizeof(u32); - delta = field_len - priv_len; - remainder_mask = htonl(~((1 << (delta * BITS_PER_BYTE)) - 1)); - mask.data[k] = (__force u32)remainder_mask; - } - + memset(&mask, 0xff, priv_len); memcpy(®->mask, &mask, field_len); return true; diff --git a/net/netfilter/xt_hashlimit.c b/net/netfilter/xt_hashlimit.c index 0859b8f76764..61813010cd31 100644 --- a/net/netfilter/xt_hashlimit.c +++ b/net/netfilter/xt_hashlimit.c @@ -118,6 +118,7 @@ struct xt_hashlimit_htable { refcount_t use; u_int8_t family; bool rnd_initialized; + bool ratematch; struct hashlimit_cfg3 cfg; /* config */ @@ -325,6 +326,7 @@ static int htable_create(struct net *net, struct hashlimit_cfg3 *cfg, vfree(hinfo); return -ENOMEM; } + hinfo->ratematch = !!(cfg->mode & XT_HASHLIMIT_RATE_MATCH); spin_lock_init(&hinfo->lock); switch (revision) { @@ -868,7 +870,10 @@ static int hashlimit_mt_check_common(const struct xt_mtchk_param *par, } /* Check for overflow. */ - if (revision >= 3 && cfg->mode & XT_HASHLIMIT_RATE_MATCH) { + if (cfg->mode & XT_HASHLIMIT_RATE_MATCH) { + if (revision < 3) + return -EINVAL; + if (cfg->avg == 0 || cfg->avg > U32_MAX) { pr_info_ratelimited("invalid rate\n"); return -ERANGE; @@ -901,6 +906,15 @@ static int hashlimit_mt_check_common(const struct xt_mtchk_param *par, mutex_unlock(&hashlimit_mutex); return ret; } + } else { + if ((cfg->mode & XT_HASHLIMIT_RATE_MATCH && + !(*hinfo)->ratematch) || + (!(cfg->mode & XT_HASHLIMIT_RATE_MATCH) && + (*hinfo)->ratematch)) { + mutex_unlock(&hashlimit_mutex); + htable_put(*hinfo); + return -EINVAL; + } } mutex_unlock(&hashlimit_mutex); diff --git a/net/openvswitch/actions.c b/net/openvswitch/actions.c index 3dcd42727f70..de126616d1cb 100644 --- a/net/openvswitch/actions.c +++ b/net/openvswitch/actions.c @@ -1146,6 +1146,10 @@ static int execute_masked_set_action(struct sk_buff *skb, return err; } +/* When 'last' is true, recirc() should always consume the 'skb'. + * Otherwise, recirc() should keep 'skb' intact regardless what + * actions are executed on recirculation. + */ static int execute_recirc(struct datapath *dp, struct sk_buff *skb, struct sw_flow_key *key, const struct nlattr *a, bool last) @@ -1156,8 +1160,12 @@ static int execute_recirc(struct datapath *dp, struct sk_buff *skb, int err; err = ovs_flow_key_update(skb, key); - if (err) + if (err) { + if (last) + ovs_kfree_skb_reason(skb, + OVS_DROP_ACTION_ERROR); return err; + } } BUG_ON(!is_flow_key_valid(key)); @@ -1410,7 +1418,7 @@ static int do_execute_actions(struct datapath *dp, struct sk_buff *skb, if (!is_flow_key_valid(key)) { err = ovs_flow_key_update(skb, key); if (err) - return err; + break; } err = ovs_ct_execute(ovs_dp_get_net(dp), skb, key, diff --git a/net/openvswitch/meter.c b/net/openvswitch/meter.c index cc08e0403909..671984edfe1e 100644 --- a/net/openvswitch/meter.c +++ b/net/openvswitch/meter.c @@ -133,18 +133,10 @@ static void dp_meter_instance_remove(struct dp_meter_instance *ti, static int attach_meter(struct dp_meter_table *tbl, struct dp_meter *meter) { - struct dp_meter_instance *ti = rcu_dereference_ovsl(tbl->ti); - u32 hash = meter_hash(ti, meter->id); + struct dp_meter_instance *ti; + u32 hash; int err; - /* In generally, slots selected should be empty, because - * OvS uses id-pool to fetch a available id. - */ - if (unlikely(rcu_dereference_ovsl(ti->dp_meters[hash]))) - return -EBUSY; - - dp_meter_instance_insert(ti, meter); - /* That function is thread-safe. */ tbl->count++; if (tbl->count >= tbl->max_meters_allowed) { @@ -152,16 +144,29 @@ static int attach_meter(struct dp_meter_table *tbl, struct dp_meter *meter) goto attach_err; } - if (tbl->count >= ti->n_meters && - dp_meter_instance_realloc(tbl, ti->n_meters * 2)) { - err = -ENOMEM; + ti = rcu_dereference_ovsl(tbl->ti); + if (tbl->count >= ti->n_meters) { + err = dp_meter_instance_realloc(tbl, ti->n_meters * 2); + if (err) + goto attach_err; + + ti = rcu_dereference_ovsl(tbl->ti); + } + + hash = meter_hash(ti, meter->id); + + /* In general, selected slots should be empty, because + * OvS uses id-pool to fetch available ids. + */ + if (unlikely(rcu_dereference_ovsl(ti->dp_meters[hash]))) { + err = -EBUSY; goto attach_err; } + dp_meter_instance_insert(ti, meter); return 0; attach_err: - dp_meter_instance_remove(ti, meter); tbl->count--; return err; } diff --git a/net/rds/ib.c b/net/rds/ib.c index ce5be43c5fba..1061bcf7d131 100644 --- a/net/rds/ib.c +++ b/net/rds/ib.c @@ -431,6 +431,10 @@ static int rds_ib_laddr_check_cm(struct net *net, const struct in6_addr *addr, sa = (struct sockaddr *)&sin; } else { #if IS_ENABLED(CONFIG_IPV6) + if (!ipv6_mod_enabled()) { + ret = -EADDRNOTAVAIL; + goto out; + } memset(&sin6, 0, sizeof(sin6)); sin6.sin6_family = AF_INET6; sin6.sin6_addr = *addr; diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c index 5289afbb61aa..e50e01abb079 100644 --- a/net/rds/ib_cm.c +++ b/net/rds/ib_cm.c @@ -810,6 +810,10 @@ int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id, dp = event->param.conn.private_data; if (isv6) { #if IS_ENABLED(CONFIG_IPV6) + if (!ipv6_mod_enabled()) { + err = -EOPNOTSUPP; + goto out; + } dp_cmn = &dp->ricp_v6.dp_cmn; saddr6 = &dp->ricp_v6.dp_saddr; daddr6 = &dp->ricp_v6.dp_daddr; diff --git a/net/rds/tcp.c b/net/rds/tcp.c index 06a2d8d48bba..1980a197034b 100644 --- a/net/rds/tcp.c +++ b/net/rds/tcp.c @@ -331,21 +331,25 @@ int rds_tcp_laddr_check(struct net *net, const struct in6_addr *addr, /* If the scope_id is specified, check only those addresses * hosted on the specified interface. */ + rcu_read_lock(); if (scope_id != 0) { - rcu_read_lock(); dev = dev_get_by_index_rcu(net, scope_id); /* scope_id is not valid... */ if (!dev) { rcu_read_unlock(); return -EADDRNOTAVAIL; } - rcu_read_unlock(); } #if IS_ENABLED(CONFIG_IPV6) - ret = ipv6_chk_addr(net, addr, dev, 0); - if (ret) - return 0; + if (ipv6_mod_enabled()) { + ret = ipv6_chk_addr(net, addr, dev, 0); + if (ret) { + rcu_read_unlock(); + return 0; + } + } #endif + rcu_read_unlock(); return -EADDRNOTAVAIL; } diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h index 79a377f1b551..48202528a243 100644 --- a/net/rxrpc/ar-internal.h +++ b/net/rxrpc/ar-internal.h @@ -344,18 +344,9 @@ struct rxrpc_peer { int debug_id; /* debug ID for printks */ struct sockaddr_rxrpc srx; /* remote address */ - /* calculated RTT cache */ -#define RXRPC_RTT_CACHE_SIZE 32 - spinlock_t rtt_input_lock; /* RTT lock for input routine */ - ktime_t rtt_last_req; /* Time of last RTT request */ - unsigned int rtt_count; /* Number of samples we've got */ - - u32 srtt_us; /* smoothed round trip time << 3 in usecs */ - u32 mdev_us; /* medium deviation */ - u32 mdev_max_us; /* maximal mdev for the last rtt period */ - u32 rttvar_us; /* smoothed mdev_max */ - u32 rto_us; /* Retransmission timeout in usec */ - u8 backoff; /* Backoff timeout (as shift) */ + /* Calculated RTT cache */ + unsigned int recent_srtt_us; + unsigned int recent_rto_us; u8 cong_ssthresh; /* Congestion slow-start threshold */ }; @@ -740,6 +731,18 @@ struct rxrpc_call { rxrpc_seq_t acks_hard_ack; /* Latest hard-ack point */ rxrpc_seq_t acks_lowest_nak; /* Lowest NACK in the buffer (or ==tx_hard_ack) */ rxrpc_serial_t acks_highest_serial; /* Highest serial number ACK'd */ + + /* Calculated RTT cache */ + ktime_t rtt_last_req; /* Time of last RTT request */ + unsigned int rtt_count; /* Number of samples we've got */ + unsigned int rtt_taken; /* Number of samples taken (wrapping) */ + struct minmax min_rtt; /* Estimated minimum RTT */ + u32 srtt_us; /* smoothed round trip time << 3 in usecs */ + u32 mdev_us; /* medium deviation */ + u32 mdev_max_us; /* maximal mdev for the last rtt period */ + u32 rttvar_us; /* smoothed mdev_max */ + u32 rto_us; /* Retransmission timeout in usec */ + u8 backoff; /* Backoff timeout (as shift) */ }; /* @@ -1180,6 +1183,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu(struct rxrpc_local *, const struct sockaddr_rxrpc *); struct rxrpc_peer *rxrpc_lookup_peer(struct rxrpc_local *local, struct sockaddr_rxrpc *srx, gfp_t gfp); +void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer); struct rxrpc_peer *rxrpc_alloc_peer(struct rxrpc_local *, gfp_t, enum rxrpc_peer_trace); void rxrpc_new_incoming_peer(struct rxrpc_local *local, struct rxrpc_peer *peer); @@ -1220,10 +1224,12 @@ static inline int rxrpc_abort_eproto(struct rxrpc_call *call, /* * rtt.c */ -void rxrpc_peer_add_rtt(struct rxrpc_call *, enum rxrpc_rtt_rx_trace, int, - rxrpc_serial_t, rxrpc_serial_t, ktime_t, ktime_t); -ktime_t rxrpc_get_rto_backoff(struct rxrpc_peer *peer, bool retrans); -void rxrpc_peer_init_rtt(struct rxrpc_peer *); +void rxrpc_call_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why, + int rtt_slot, + rxrpc_serial_t send_serial, rxrpc_serial_t resp_serial, + ktime_t send_time, ktime_t resp_time); +ktime_t rxrpc_get_rto_backoff(struct rxrpc_call *call, bool retrans); +void rxrpc_call_init_rtt(struct rxrpc_call *call); /* * rxkad.c diff --git a/net/rxrpc/call_accept.c b/net/rxrpc/call_accept.c index 15e63b1ae713..82766feb34e5 100644 --- a/net/rxrpc/call_accept.c +++ b/net/rxrpc/call_accept.c @@ -407,6 +407,7 @@ bool rxrpc_new_incoming_call(struct rxrpc_local *local, spin_unlock(&rx->incoming_lock); read_unlock_irq(&local->services_lock); + rxrpc_assess_MTU_size(local, call->peer); if (hlist_unhashed(&call->error_link)) { spin_lock_irq(&call->peer->lock); diff --git a/net/rxrpc/call_event.c b/net/rxrpc/call_event.c index 1181976972a3..3425e778d24f 100644 --- a/net/rxrpc/call_event.c +++ b/net/rxrpc/call_event.c @@ -44,8 +44,8 @@ void rxrpc_propose_delay_ACK(struct rxrpc_call *call, rxrpc_serial_t serial, trace_rxrpc_propose_ack(call, why, RXRPC_ACK_DELAY, serial); - if (call->peer->srtt_us) - delay = (call->peer->srtt_us >> 3) * NSEC_PER_USEC; + if (call->srtt_us) + delay = (call->srtt_us >> 3) * NSEC_PER_USEC; else delay = ms_to_ktime(READ_ONCE(rxrpc_soft_ack_delay)); ktime_add_ms(delay, call->tx_backoff); @@ -71,7 +71,8 @@ void rxrpc_resend(struct rxrpc_call *call, struct sk_buff *ack_skb) struct rxrpc_skb_priv *sp; struct rxrpc_txbuf *txb; rxrpc_seq_t transmitted = call->tx_transmitted; - ktime_t next_resend = KTIME_MAX, rto = ns_to_ktime(call->peer->rto_us * NSEC_PER_USEC); + ktime_t next_resend = KTIME_MAX; + ktime_t rto = rxrpc_get_rto_backoff(call, false); ktime_t resend_at = KTIME_MAX, now, delay; bool unacked = false, did_send = false; unsigned int i; @@ -173,7 +174,7 @@ void rxrpc_resend(struct rxrpc_call *call, struct sk_buff *ack_skb) no_further_resend: no_resend: if (resend_at < KTIME_MAX) { - delay = rxrpc_get_rto_backoff(call->peer, did_send); + delay = rxrpc_get_rto_backoff(call, did_send); resend_at = ktime_add(resend_at, delay); trace_rxrpc_timer_set(call, resend_at - now, rxrpc_timer_trace_resend_reset); } @@ -188,7 +189,7 @@ void rxrpc_resend(struct rxrpc_call *call, struct sk_buff *ack_skb) */ if (!did_send) { ktime_t next_ping = ktime_add_us(call->acks_latest_ts, - call->peer->srtt_us >> 3); + call->srtt_us >> 3); if (ktime_sub(next_ping, now) <= 0) rxrpc_send_ACK(call, RXRPC_ACK_PING, 0, @@ -305,8 +306,8 @@ static void rxrpc_transmit_some_data(struct rxrpc_call *call) */ static void rxrpc_send_initial_ping(struct rxrpc_call *call) { - if (call->peer->rtt_count < 3 || - ktime_before(ktime_add_ms(call->peer->rtt_last_req, 1000), + if (call->rtt_count < 3 || + ktime_before(ktime_add_ms(call->rtt_last_req, 1000), ktime_get_real())) rxrpc_send_ACK(call, RXRPC_ACK_PING, 0, rxrpc_propose_ack_ping_for_params); @@ -437,10 +438,10 @@ bool rxrpc_input_call_event(struct rxrpc_call *call) rxrpc_propose_ack_rx_idle); if (call->ackr_nr_unacked > 2) { - if (call->peer->rtt_count < 3) + if (call->rtt_count < 3) rxrpc_send_ACK(call, RXRPC_ACK_PING, 0, rxrpc_propose_ack_ping_for_rtt); - else if (ktime_before(ktime_add_ms(call->peer->rtt_last_req, 1000), + else if (ktime_before(ktime_add_ms(call->rtt_last_req, 1000), ktime_get_real())) rxrpc_send_ACK(call, RXRPC_ACK_PING, 0, rxrpc_propose_ack_ping_for_old_rtt); diff --git a/net/rxrpc/call_object.c b/net/rxrpc/call_object.c index 6e011a1e8e71..485c481481fc 100644 --- a/net/rxrpc/call_object.c +++ b/net/rxrpc/call_object.c @@ -178,6 +178,8 @@ struct rxrpc_call *rxrpc_alloc_call(struct rxrpc_sock *rx, gfp_t gfp, call->cong_cwnd = RXRPC_MIN_CWND; call->cong_ssthresh = RXRPC_TX_MAX_WINDOW; + rxrpc_call_init_rtt(call); + call->rxnet = rxnet; call->rtt_avail = RXRPC_CALL_RTT_AVAIL_MASK; atomic_inc(&rxnet->nr_calls); diff --git a/net/rxrpc/input.c b/net/rxrpc/input.c index c2a2fb7210c3..d312feb4aaf4 100644 --- a/net/rxrpc/input.c +++ b/net/rxrpc/input.c @@ -83,11 +83,11 @@ static void rxrpc_congestion_management(struct rxrpc_call *call, /* We analyse the number of packets that get ACK'd per RTT * period and increase the window if we managed to fill it. */ - if (call->peer->rtt_count == 0) + if (call->rtt_count == 0) goto out; if (ktime_before(skb->tstamp, ktime_add_us(call->cong_tstamp, - call->peer->srtt_us >> 3))) + call->srtt_us >> 3))) goto out_no_clear_ca; change = rxrpc_cong_rtt_window_end; call->cong_tstamp = skb->tstamp; @@ -197,7 +197,7 @@ void rxrpc_congestion_degrade(struct rxrpc_call *call) if (__rxrpc_call_state(call) == RXRPC_CALL_CLIENT_AWAIT_REPLY) return; - rtt = ns_to_ktime(call->peer->srtt_us * (1000 / 8)); + rtt = ns_to_ktime(call->srtt_us * (NSEC_PER_USEC / 8)); now = ktime_get_real(); if (!ktime_before(ktime_add(call->tx_last_sent, rtt), now)) return; @@ -664,7 +664,7 @@ static void rxrpc_complete_rtt_probe(struct rxrpc_call *call, clear_bit(i + RXRPC_CALL_RTT_PEND_SHIFT, &call->rtt_avail); smp_mb(); /* Read data before setting avail bit */ set_bit(i, &call->rtt_avail); - rxrpc_peer_add_rtt(call, type, i, acked_serial, ack_serial, + rxrpc_call_add_rtt(call, type, i, acked_serial, ack_serial, sent_at, resp_time); matched = true; } @@ -674,7 +674,7 @@ static void rxrpc_complete_rtt_probe(struct rxrpc_call *call, */ if (after(acked_serial, orig_serial)) { trace_rxrpc_rtt_rx(call, rxrpc_rtt_rx_obsolete, i, - orig_serial, acked_serial, 0, 0); + orig_serial, acked_serial, 0, 0, 0); clear_bit(i + RXRPC_CALL_RTT_PEND_SHIFT, &call->rtt_avail); smp_wmb(); set_bit(i, &call->rtt_avail); @@ -682,7 +682,7 @@ static void rxrpc_complete_rtt_probe(struct rxrpc_call *call, } if (!matched) - trace_rxrpc_rtt_rx(call, rxrpc_rtt_rx_lost, 9, 0, acked_serial, 0, 0); + trace_rxrpc_rtt_rx(call, rxrpc_rtt_rx_lost, 9, 0, acked_serial, 0, 0, 0); } /* diff --git a/net/rxrpc/output.c b/net/rxrpc/output.c index ad7e61066d2b..20085175471f 100644 --- a/net/rxrpc/output.c +++ b/net/rxrpc/output.c @@ -220,7 +220,7 @@ static void rxrpc_send_ack_packet(struct rxrpc_call *call, struct rxrpc_txbuf *t if (ack->reason == RXRPC_ACK_PING) rxrpc_begin_rtt_probe(call, txb->serial, now, rxrpc_rtt_tx_ping); if (txb->flags & RXRPC_REQUEST_ACK) - call->peer->rtt_last_req = now; + call->rtt_last_req = now; rxrpc_set_keepalive(call, now); } rxrpc_tx_backoff(call, ret); @@ -358,9 +358,9 @@ static void rxrpc_prepare_data_subpacket(struct rxrpc_call *call, struct rxrpc_t why = rxrpc_reqack_slow_start; else if (call->tx_winsize <= 2) why = rxrpc_reqack_small_txwin; - else if (call->peer->rtt_count < 3 && txb->seq & 1) + else if (call->rtt_count < 3) why = rxrpc_reqack_more_rtt; - else if (ktime_before(ktime_add_ms(call->peer->rtt_last_req, 1000), ktime_get_real())) + else if (ktime_before(ktime_add_ms(call->rtt_last_req, 1000), ktime_get_real())) why = rxrpc_reqack_old_rtt; else goto dont_set_request_ack; @@ -407,9 +407,9 @@ static void rxrpc_tstamp_data_packets(struct rxrpc_call *call, struct rxrpc_txbu if (ack_requested) { rxrpc_begin_rtt_probe(call, txb->serial, now, rxrpc_rtt_tx_data); - call->peer->rtt_last_req = now; - if (call->peer->rtt_count > 1) { - ktime_t delay = rxrpc_get_rto_backoff(call->peer, false); + call->rtt_last_req = now; + if (call->rtt_count > 1) { + ktime_t delay = rxrpc_get_rto_backoff(call, false); call->ack_lost_at = ktime_add(now, delay); trace_rxrpc_timer_set(call, delay, rxrpc_timer_trace_lost_ack); @@ -727,7 +727,7 @@ void rxrpc_transmit_one(struct rxrpc_call *call, struct rxrpc_txbuf *txb) rxrpc_instant_resend(call, txb); } } else { - ktime_t delay = ns_to_ktime(call->peer->rto_us * NSEC_PER_USEC); + ktime_t delay = ns_to_ktime(call->rto_us * NSEC_PER_USEC); call->resend_at = ktime_add(ktime_get_real(), delay); trace_rxrpc_timer_set(call, delay, rxrpc_timer_trace_resend_tx); diff --git a/net/rxrpc/peer_object.c b/net/rxrpc/peer_object.c index a09b8aaf9bcd..3af39fc67d34 100644 --- a/net/rxrpc/peer_object.c +++ b/net/rxrpc/peer_object.c @@ -149,8 +149,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu(struct rxrpc_local *local, * assess the MTU size for the network interface through which this peer is * reached */ -static void rxrpc_assess_MTU_size(struct rxrpc_local *local, - struct rxrpc_peer *peer) +void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer) { struct net *net = local->net; struct dst_entry *dst; @@ -162,6 +161,8 @@ static void rxrpc_assess_MTU_size(struct rxrpc_local *local, #endif peer->if_mtu = 1500; + peer->mtu = peer->if_mtu; + peer->maxdata = peer->mtu - peer->hdrsize; memset(&fl, 0, sizeof(fl)); switch (peer->srx.transport.family) { @@ -201,6 +202,9 @@ static void rxrpc_assess_MTU_size(struct rxrpc_local *local, peer->if_mtu = dst_mtu(dst); dst_release(dst); + peer->mtu = peer->if_mtu; + peer->maxdata = peer->mtu - peer->hdrsize; + _leave(" [if_mtu %u]", peer->if_mtu); } @@ -222,11 +226,8 @@ struct rxrpc_peer *rxrpc_alloc_peer(struct rxrpc_local *local, gfp_t gfp, peer->service_conns = RB_ROOT; seqlock_init(&peer->service_conn_lock); spin_lock_init(&peer->lock); - spin_lock_init(&peer->rtt_input_lock); peer->debug_id = atomic_inc_return(&rxrpc_debug_id); - - rxrpc_peer_init_rtt(peer); - + peer->recent_srtt_us = UINT_MAX; peer->cong_ssthresh = RXRPC_TX_MAX_WINDOW; trace_rxrpc_peer(peer->debug_id, 1, why); } @@ -242,9 +243,6 @@ static void rxrpc_init_peer(struct rxrpc_local *local, struct rxrpc_peer *peer, unsigned long hash_key) { peer->hash_key = hash_key; - rxrpc_assess_MTU_size(local, peer); - peer->mtu = peer->if_mtu; - peer->rtt_last_req = ktime_get_real(); switch (peer->srx.transport.family) { case AF_INET: @@ -268,7 +266,6 @@ static void rxrpc_init_peer(struct rxrpc_local *local, struct rxrpc_peer *peer, } peer->hdrsize += sizeof(struct rxrpc_wire_header); - peer->maxdata = peer->mtu - peer->hdrsize; } /* @@ -287,6 +284,7 @@ static struct rxrpc_peer *rxrpc_create_peer(struct rxrpc_local *local, if (peer) { memcpy(&peer->srx, srx, sizeof(*srx)); rxrpc_init_peer(local, peer, hash_key); + rxrpc_assess_MTU_size(local, peer); } _leave(" = %p", peer); @@ -480,7 +478,7 @@ EXPORT_SYMBOL(rxrpc_kernel_get_call_peer); */ unsigned int rxrpc_kernel_get_srtt(const struct rxrpc_peer *peer) { - return peer->rtt_count > 0 ? peer->srtt_us >> 3 : UINT_MAX; + return READ_ONCE(peer->recent_srtt_us); } EXPORT_SYMBOL(rxrpc_kernel_get_srtt); diff --git a/net/rxrpc/proc.c b/net/rxrpc/proc.c index ca85ac764f82..2dd9567cb1cf 100644 --- a/net/rxrpc/proc.c +++ b/net/rxrpc/proc.c @@ -299,15 +299,15 @@ static int rxrpc_peer_seq_show(struct seq_file *seq, void *v) now = ktime_get_seconds(); seq_printf(seq, "UDP %-47.47s %-47.47s %3u" - " %3u %5u %6ds %8u %8u\n", + " %3u %5u %6ds %8d %8d\n", lbuff, rbuff, refcount_read(&peer->ref), peer->cong_ssthresh, peer->mtu, (s32)now - (s32)READ_ONCE(peer->last_tx_at), - peer->srtt_us >> 3, - peer->rto_us); + READ_ONCE(peer->recent_srtt_us), + READ_ONCE(peer->recent_rto_us)); return 0; } diff --git a/net/rxrpc/rtt.c b/net/rxrpc/rtt.c index cdab7b7d08a0..7474f88d7b18 100644 --- a/net/rxrpc/rtt.c +++ b/net/rxrpc/rtt.c @@ -12,22 +12,22 @@ #include "ar-internal.h" #define RXRPC_RTO_MAX (120 * USEC_PER_SEC) -#define RXRPC_TIMEOUT_INIT ((unsigned int)(1 * MSEC_PER_SEC)) /* RFC6298 2.1 initial RTO value */ +#define RXRPC_TIMEOUT_INIT ((unsigned int)(1 * USEC_PER_SEC)) /* RFC6298 2.1 initial RTO value */ #define rxrpc_jiffies32 ((u32)jiffies) /* As rxrpc_jiffies32 */ -static u32 rxrpc_rto_min_us(struct rxrpc_peer *peer) +static u32 rxrpc_rto_min_us(struct rxrpc_call *call) { return 200; } -static u32 __rxrpc_set_rto(const struct rxrpc_peer *peer) +static u32 __rxrpc_set_rto(const struct rxrpc_call *call) { - return (peer->srtt_us >> 3) + peer->rttvar_us; + return (call->srtt_us >> 3) + call->rttvar_us; } static u32 rxrpc_bound_rto(u32 rto) { - return min(rto, RXRPC_RTO_MAX); + return clamp(200000, rto + 100000, RXRPC_RTO_MAX); } /* @@ -40,10 +40,10 @@ static u32 rxrpc_bound_rto(u32 rto) * To save cycles in the RFC 1323 implementation it was better to break * it up into three procedures. -- erics */ -static void rxrpc_rtt_estimator(struct rxrpc_peer *peer, long sample_rtt_us) +static void rxrpc_rtt_estimator(struct rxrpc_call *call, long sample_rtt_us) { long m = sample_rtt_us; /* RTT */ - u32 srtt = peer->srtt_us; + u32 srtt = call->srtt_us; /* The following amusing code comes from Jacobson's * article in SIGCOMM '88. Note that rtt and mdev @@ -66,7 +66,7 @@ static void rxrpc_rtt_estimator(struct rxrpc_peer *peer, long sample_rtt_us) srtt += m; /* rtt = 7/8 rtt + 1/8 new */ if (m < 0) { m = -m; /* m is now abs(error) */ - m -= (peer->mdev_us >> 2); /* similar update on mdev */ + m -= (call->mdev_us >> 2); /* similar update on mdev */ /* This is similar to one of Eifel findings. * Eifel blocks mdev updates when rtt decreases. * This solution is a bit different: we use finer gain @@ -78,31 +78,31 @@ static void rxrpc_rtt_estimator(struct rxrpc_peer *peer, long sample_rtt_us) if (m > 0) m >>= 3; } else { - m -= (peer->mdev_us >> 2); /* similar update on mdev */ + m -= (call->mdev_us >> 2); /* similar update on mdev */ } - peer->mdev_us += m; /* mdev = 3/4 mdev + 1/4 new */ - if (peer->mdev_us > peer->mdev_max_us) { - peer->mdev_max_us = peer->mdev_us; - if (peer->mdev_max_us > peer->rttvar_us) - peer->rttvar_us = peer->mdev_max_us; + call->mdev_us += m; /* mdev = 3/4 mdev + 1/4 new */ + if (call->mdev_us > call->mdev_max_us) { + call->mdev_max_us = call->mdev_us; + if (call->mdev_max_us > call->rttvar_us) + call->rttvar_us = call->mdev_max_us; } } else { /* no previous measure. */ srtt = m << 3; /* take the measured time to be rtt */ - peer->mdev_us = m << 1; /* make sure rto = 3*rtt */ - peer->rttvar_us = max(peer->mdev_us, rxrpc_rto_min_us(peer)); - peer->mdev_max_us = peer->rttvar_us; + call->mdev_us = m << 1; /* make sure rto = 3*rtt */ + call->rttvar_us = umax(call->mdev_us, rxrpc_rto_min_us(call)); + call->mdev_max_us = call->rttvar_us; } - peer->srtt_us = max(1U, srtt); + call->srtt_us = umax(srtt, 1); } /* * Calculate rto without backoff. This is the second half of Van Jacobson's * routine referred to above. */ -static void rxrpc_set_rto(struct rxrpc_peer *peer) +static void rxrpc_set_rto(struct rxrpc_call *call) { u32 rto; @@ -113,7 +113,7 @@ static void rxrpc_set_rto(struct rxrpc_peer *peer) * is invisible. Actually, Linux-2.4 also generates erratic * ACKs in some circumstances. */ - rto = __rxrpc_set_rto(peer); + rto = __rxrpc_set_rto(call); /* 2. Fixups made earlier cannot be right. * If we do not estimate RTO correctly without them, @@ -124,61 +124,73 @@ static void rxrpc_set_rto(struct rxrpc_peer *peer) /* NOTE: clamping at RXRPC_RTO_MIN is not required, current algo * guarantees that rto is higher. */ - peer->rto_us = rxrpc_bound_rto(rto); + call->rto_us = rxrpc_bound_rto(rto); } -static void rxrpc_ack_update_rtt(struct rxrpc_peer *peer, long rtt_us) +static void rxrpc_update_rtt_min(struct rxrpc_call *call, ktime_t resp_time, long rtt_us) +{ + /* Window size 5mins in approx usec (ipv4.sysctl_tcp_min_rtt_wlen) */ + u32 wlen_us = 5ULL * NSEC_PER_SEC / 1024; + + minmax_running_min(&call->min_rtt, wlen_us, resp_time / 1024, + (u32)rtt_us ? : jiffies_to_usecs(1)); +} + +static void rxrpc_ack_update_rtt(struct rxrpc_call *call, ktime_t resp_time, long rtt_us) { if (rtt_us < 0) return; - //rxrpc_update_rtt_min(peer, rtt_us); - rxrpc_rtt_estimator(peer, rtt_us); - rxrpc_set_rto(peer); + /* Update RACK min RTT [RFC8985 6.1 Step 1]. */ + rxrpc_update_rtt_min(call, resp_time, rtt_us); + + rxrpc_rtt_estimator(call, rtt_us); + rxrpc_set_rto(call); - /* RFC6298: only reset backoff on valid RTT measurement. */ - peer->backoff = 0; + /* Only reset backoff on valid RTT measurement [RFC6298]. */ + call->backoff = 0; } /* * Add RTT information to cache. This is called in softirq mode and has - * exclusive access to the peer RTT data. + * exclusive access to the call RTT data. */ -void rxrpc_peer_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why, +void rxrpc_call_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why, int rtt_slot, rxrpc_serial_t send_serial, rxrpc_serial_t resp_serial, ktime_t send_time, ktime_t resp_time) { - struct rxrpc_peer *peer = call->peer; s64 rtt_us; rtt_us = ktime_to_us(ktime_sub(resp_time, send_time)); if (rtt_us < 0) return; - spin_lock(&peer->rtt_input_lock); - rxrpc_ack_update_rtt(peer, rtt_us); - if (peer->rtt_count < 3) - peer->rtt_count++; - spin_unlock(&peer->rtt_input_lock); + rxrpc_ack_update_rtt(call, resp_time, rtt_us); + if (call->rtt_count < 3) + call->rtt_count++; + call->rtt_taken++; + + WRITE_ONCE(call->peer->recent_srtt_us, call->srtt_us / 8); + WRITE_ONCE(call->peer->recent_rto_us, call->rto_us); trace_rxrpc_rtt_rx(call, why, rtt_slot, send_serial, resp_serial, - peer->srtt_us >> 3, peer->rto_us); + rtt_us, call->srtt_us, call->rto_us); } /* * Get the retransmission timeout to set in nanoseconds, backing it off each * time we retransmit. */ -ktime_t rxrpc_get_rto_backoff(struct rxrpc_peer *peer, bool retrans) +ktime_t rxrpc_get_rto_backoff(struct rxrpc_call *call, bool retrans) { u64 timo_us; - u32 backoff = READ_ONCE(peer->backoff); + u32 backoff = READ_ONCE(call->backoff); - timo_us = peer->rto_us; + timo_us = call->rto_us; timo_us <<= backoff; if (retrans && timo_us * 2 <= RXRPC_RTO_MAX) - WRITE_ONCE(peer->backoff, backoff + 1); + WRITE_ONCE(call->backoff, backoff + 1); if (timo_us < 1) timo_us = 1; @@ -186,10 +198,11 @@ ktime_t rxrpc_get_rto_backoff(struct rxrpc_peer *peer, bool retrans) return ns_to_ktime(timo_us * NSEC_PER_USEC); } -void rxrpc_peer_init_rtt(struct rxrpc_peer *peer) +void rxrpc_call_init_rtt(struct rxrpc_call *call) { - peer->rto_us = RXRPC_TIMEOUT_INIT; - peer->mdev_us = RXRPC_TIMEOUT_INIT; - peer->backoff = 0; - //minmax_reset(&peer->rtt_min, rxrpc_jiffies32, ~0U); + call->rtt_last_req = KTIME_MIN; + call->rto_us = RXRPC_TIMEOUT_INIT; + call->mdev_us = RXRPC_TIMEOUT_INIT; + call->backoff = 0; + //minmax_reset(&call->rtt_min, rxrpc_jiffies32, ~0U); } diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c index dc43bf8cbac9..1d8065468e90 100644 --- a/net/rxrpc/sendmsg.c +++ b/net/rxrpc/sendmsg.c @@ -133,7 +133,7 @@ static int rxrpc_wait_for_tx_window_waitall(struct rxrpc_sock *rx, rxrpc_seq_t tx_start, tx_win; signed long rtt, timeout; - rtt = READ_ONCE(call->peer->srtt_us) >> 3; + rtt = READ_ONCE(call->srtt_us) >> 3; rtt = usecs_to_jiffies(rtt) * 2; if (rtt < 2) rtt = 2; diff --git a/net/sctp/associola.c b/net/sctp/associola.c index 0b0794f164cf..44f06c5f1f25 100644 --- a/net/sctp/associola.c +++ b/net/sctp/associola.c @@ -614,6 +614,9 @@ struct sctp_transport *sctp_assoc_add_peer(struct sctp_association *asoc, return peer; } + if (asoc->peer.transport_count == U16_MAX) + return NULL; + peer = sctp_transport_new(asoc->base.net, addr, gfp); if (!peer) return NULL; diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c index 187304b43e67..27a7739a7d42 100644 --- a/net/sctp/sm_make_chunk.c +++ b/net/sctp/sm_make_chunk.c @@ -1821,9 +1821,9 @@ struct sctp_association *sctp_unpack_cookie( goto fail; } - /* Check to see if the cookie is stale. If there is already - * an association, there is no need to check cookie's expiration - * for init collision case of lost COOKIE ACK. + /* Check to see if the cookie is stale. RFC 9260 Section 5.2.4 + * exempts an expired cookie only when both Verification Tags match + * the current association. * If skb has been timestamped, then use the stamp, otherwise * use current time. This introduces a small possibility that * a cookie may be considered expired, but this would only slow @@ -1834,7 +1834,10 @@ struct sctp_association *sctp_unpack_cookie( else kt = ktime_get_real(); - if (!asoc && ktime_before(bear_cookie->expiration, kt)) { + if ((!asoc || + asoc->c.my_vtag != bear_cookie->my_vtag || + asoc->c.peer_vtag != bear_cookie->peer_vtag) && + ktime_before(bear_cookie->expiration, kt)) { suseconds_t usecs = ktime_to_us(ktime_sub(kt, bear_cookie->expiration)); __be32 n = htonl(usecs); @@ -2187,7 +2190,13 @@ static enum sctp_ierror sctp_verify_param(struct net *net, case SCTP_PARAM_HEARTBEAT_INFO: case SCTP_PARAM_UNRECOGNIZED_PARAMETERS: case SCTP_PARAM_ECN_CAPABLE: + break; case SCTP_PARAM_ADAPTATION_LAYER_IND: + if (ntohs(param.p->length) != sizeof(*param.aind)) { + sctp_process_inv_paramlength(asoc, param.p, + chunk, err_chunk); + retval = SCTP_IERROR_ABORT; + } break; case SCTP_PARAM_SUPPORTED_EXT: diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c index 68515a41d776..15d2c69f7ebc 100644 --- a/net/smc/smc_core.c +++ b/net/smc/smc_core.c @@ -1553,10 +1553,10 @@ static void __smc_lgr_terminate(struct smc_link_group *lgr, bool soft) read_lock_bh(&lgr->conns_lock); node = rb_first(&lgr->conns_all); while (node) { - read_unlock_bh(&lgr->conns_lock); conn = rb_entry(node, struct smc_connection, alert_node); smc = container_of(conn, struct smc_sock, conn); sock_hold(&smc->sk); /* sock_put below */ + read_unlock_bh(&lgr->conns_lock); lock_sock(&smc->sk); smc_conn_kill(conn, soft); release_sock(&smc->sk); diff --git a/net/tipc/socket.c b/net/tipc/socket.c index c129f4f6c605..97ac69e77cee 100644 --- a/net/tipc/socket.c +++ b/net/tipc/socket.c @@ -795,7 +795,7 @@ static __poll_t tipc_poll(struct file *file, struct socket *sock, __poll_t revents = 0; sock_poll_wait(file, sock, wait); - trace_tipc_sk_poll(sk, NULL, TIPC_DUMP_ALL, " "); + trace_tipc_sk_poll(sk, NULL, TIPC_DUMP_NONE, " "); if (sk->sk_shutdown & RCV_SHUTDOWN) revents |= EPOLLRDHUP | EPOLLIN | EPOLLRDNORM; diff --git a/security/keys/keyring.c b/security/keys/keyring.c index e105349794f2..fd95a0eb7a46 100644 --- a/security/keys/keyring.c +++ b/security/keys/keyring.c @@ -271,6 +271,7 @@ static unsigned long keyring_get_key_chunk(const void *data, int level) unsigned long chunk = 0; const u8 *d; int desc_len = index_key->desc_len, n = sizeof(chunk); + unsigned int offset; level /= ASSOC_ARRAY_KEY_CHUNK_SIZE; switch (level) { @@ -284,17 +285,18 @@ static unsigned long keyring_get_key_chunk(const void *data, int level) return (unsigned long)index_key->domain_tag; default: level -= 4; - if (desc_len <= sizeof(index_key->desc)) + offset = sizeof(index_key->desc) + level * sizeof(long); + if (desc_len <= offset) return 0; - d = index_key->description + sizeof(index_key->desc); - d += level * sizeof(long); - desc_len -= sizeof(index_key->desc); + d = index_key->description + offset; + desc_len -= offset; if (desc_len > n) desc_len = n; + d += desc_len; do { chunk <<= 8; - chunk |= *d++; + chunk |= *--d; } while (--desc_len > 0); return chunk; } @@ -375,7 +377,7 @@ static int keyring_diff_objects(const void *object, const void *data) return -1; differ_plus_i: - level += i; + level += i - (int)sizeof(a->desc); differ: i = level * 8 + __ffs(seg_a ^ seg_b); return i; diff --git a/security/keys/trusted-keys/trusted_dcp.c b/security/keys/trusted-keys/trusted_dcp.c index 7b6eb655df0c..c078adebe190 100644 --- a/security/keys/trusted-keys/trusted_dcp.c +++ b/security/keys/trusted-keys/trusted_dcp.c @@ -69,7 +69,7 @@ static bool skip_zk_test; module_param_named(dcp_skip_zk_test, skip_zk_test, bool, 0); MODULE_PARM_DESC(dcp_skip_zk_test, "Don't test whether device keys are zero'ed"); -static unsigned int calc_blob_len(unsigned int payload_len) +static size_t calc_blob_len(unsigned int payload_len) { return sizeof(struct dcp_blob_fmt) + payload_len + DCP_BLOB_AUTHLEN; } @@ -200,7 +200,8 @@ static int encrypt_blob_key(u8 *plain_key, u8 *encrypted_key) static int trusted_dcp_seal(struct trusted_key_payload *p, char *datablob) { struct dcp_blob_fmt *b = (struct dcp_blob_fmt *)p->blob; - int blen, ret; + size_t blen; + int ret; u8 *plain_blob_key; blen = calc_blob_len(p->key_len); @@ -242,7 +243,8 @@ static int trusted_dcp_seal(struct trusted_key_payload *p, char *datablob) static int trusted_dcp_unseal(struct trusted_key_payload *p, char *datablob) { struct dcp_blob_fmt *b = (struct dcp_blob_fmt *)p->blob; - int blen, ret; + size_t blen; + int ret; u8 *plain_blob_key = NULL; if (b->fmt_version != DCP_BLOB_VERSION) { @@ -253,9 +255,14 @@ static int trusted_dcp_unseal(struct trusted_key_payload *p, char *datablob) } p->key_len = le32_to_cpu(b->payload_len); + if (p->key_len < MIN_KEY_SIZE || p->key_len > MAX_KEY_SIZE) { + ret = -EINVAL; + goto out; + } + blen = calc_blob_len(p->key_len); if (blen != p->blob_len) { - pr_err("DCP blob has bad length: %i != %i\n", blen, + pr_err("DCP blob has bad length: %zu != %u\n", blen, p->blob_len); ret = -EINVAL; goto out; diff --git a/sound/core/pcm_native.c b/sound/core/pcm_native.c index a57123b1d336..706fab8ee038 100644 --- a/sound/core/pcm_native.c +++ b/sound/core/pcm_native.c @@ -2334,6 +2334,7 @@ static void relink_to_local(struct snd_pcm_substream *substream) static int snd_pcm_unlink(struct snd_pcm_substream *substream) { + struct snd_pcm_substream *s; struct snd_pcm_group *group; bool nonatomic = substream->pcm->nonatomic; bool do_free = false; @@ -2346,6 +2347,12 @@ static int snd_pcm_unlink(struct snd_pcm_substream *substream) group = substream->group; snd_pcm_group_lock_irq(group, nonatomic); + /* release drain waiters before changing membership, else snd_pcm_drain() + * leaves its on-stack wait entry queued on a member's sleep list + */ + snd_pcm_group_for_each_entry(s, substream) + wake_up(&s->runtime->sleep); + relink_to_local(substream); refcount_dec(&group->refs); diff --git a/sound/core/seq/seq_timer.c b/sound/core/seq/seq_timer.c index c27bea6a3bd7..ef66e350e7b9 100644 --- a/sound/core/seq/seq_timer.c +++ b/sound/core/seq/seq_timer.c @@ -362,11 +362,10 @@ static int initialize_timer(struct snd_seq_timer *tmr) tmr->ticks = 1; if (!(t->hw.flags & SNDRV_TIMER_HW_SLAVE)) { unsigned long r = snd_timer_resolution(tmr->timeri); - if (r) { - tmr->ticks = (unsigned int)(1000000000uL / (r * freq)); - if (! tmr->ticks) - tmr->ticks = 1; - } + unsigned long den; + + if (r && !check_mul_overflow(r, freq, &den)) + tmr->ticks = max(1U, (unsigned int)(1000000000uL / den)); } tmr->initialized = 1; return 0; diff --git a/sound/core/timer.c b/sound/core/timer.c index 403bc62df8f6..a548885e8178 100644 --- a/sound/core/timer.c +++ b/sound/core/timer.c @@ -409,6 +409,8 @@ static void remove_slave_links(struct snd_timer_instance *timeri, list_del_init(&slave->ack_list); list_del_init(&slave->active_list); } + /* the close is done; a reopen must not see the mark */ + timeri->flags &= ~SNDRV_TIMER_IFLG_DEAD; } /* diff --git a/sound/core/ump.c b/sound/core/ump.c index abc537d54b73..f69b9e70de01 100644 --- a/sound/core/ump.c +++ b/sound/core/ump.c @@ -1297,6 +1297,7 @@ int snd_ump_attach_legacy_rawmidi(struct snd_ump_endpoint *ump, &rmidi); if (err < 0) { kfree(ump->out_cvts); + ump->out_cvts = NULL; return err; } diff --git a/sound/pci/hda/patch_hdmi.c b/sound/pci/hda/patch_hdmi.c index a5ebd2a9a111..8b68386e1ac9 100644 --- a/sound/pci/hda/patch_hdmi.c +++ b/sound/pci/hda/patch_hdmi.c @@ -111,6 +111,15 @@ struct hdmi_ops { hda_nid_t pin_nid, int dev_id, u32 stream_tag, int format); + /* + * Optional hook invoked at the beginning of the PCM prepare + * sequence, before the audio infoframe and stream format are + * (re)programmed. Used to disable keep-alive / silent stream so + * that the format change is not done while keep-alive is active. + */ + void (*prepare)(struct hda_codec *codec, + struct hdmi_spec_per_pin *per_pin); + void (*pin_cvt_fixup)(struct hda_codec *codec, struct hdmi_spec_per_pin *per_pin, hda_nid_t cvt_nid); @@ -2134,6 +2143,9 @@ static int generic_hdmi_playback_pcm_prepare(struct hda_pcm_stream *hinfo, per_pin->channels = substream->runtime->channels; per_pin->setup = true; + if (spec->ops.prepare) + spec->ops.prepare(codec, per_pin); + if (get_wcaps(codec, cvt_nid) & AC_WCAP_STRIPE) { stripe = snd_hdac_get_stream_stripe_ctl(&codec->bus->core, substream); @@ -2901,6 +2913,28 @@ static void register_i915_notifier(struct hda_codec *codec) codec->relaxed_resume = 1; } +/* + * prepare ops override for HSW+ + * + * Disable keep-alive before the converter format and audio infoframe are + * reprogrammed by the PCM prepare sequence. Changing the audio format (e.g. + * the channel count when switching to multichannel PCM) while a keep-alive + * stream is active is not safe, so release keep-alive here, early in the + * sequence. It is re-enabled once the new stream has been set up, in + * i915_hsw_setup_stream(). + */ +static void i915_hsw_prepare(struct hda_codec *codec, + struct hdmi_spec_per_pin *per_pin) +{ + struct hdmi_spec *spec = codec->spec; + + if (spec->silent_stream_type == SILENT_STREAM_KAE && per_pin->silent_stream) { + silent_stream_set_kae(codec, per_pin, false); + /* wait for pending transfers in codec to clear */ + usleep_range(100, 200); + } +} + /* setup_stream ops override for HSW+ */ static int i915_hsw_setup_stream(struct hda_codec *codec, hda_nid_t cvt_nid, hda_nid_t pin_nid, int dev_id, u32 stream_tag, @@ -2918,15 +2952,16 @@ static int i915_hsw_setup_stream(struct hda_codec *codec, hda_nid_t cvt_nid, haswell_verify_D0(codec, cvt_nid, pin_nid); - if (spec->silent_stream_type == SILENT_STREAM_KAE && per_pin && per_pin->silent_stream) { - silent_stream_set_kae(codec, per_pin, false); - /* wait for pending transfers in codec to clear */ - usleep_range(100, 200); - } - res = hdmi_setup_stream(codec, cvt_nid, pin_nid, dev_id, stream_tag, format); + /* + * Keep-alive was disabled in i915_hsw_prepare(), re-enable it now. + * The pin lookup above resolves to the same per_pin that prepare + * used (pin_nid comes from that per_pin), so this stays balanced; a + * NULL per_pin only occurs on a lookup failure that also implies no + * active keep-alive stream to restore. + */ if (spec->silent_stream_type == SILENT_STREAM_KAE && per_pin && per_pin->silent_stream) { usleep_range(100, 200); silent_stream_set_kae(codec, per_pin, true); @@ -3100,6 +3135,7 @@ static int intel_hsw_common_init(struct hda_codec *codec, hda_nid_t vendor_nid, codec->depop_delay = 0; codec->auto_runtime_pm = 1; + spec->ops.prepare = i915_hsw_prepare; spec->ops.setup_stream = i915_hsw_setup_stream; spec->ops.pin_cvt_fixup = i915_pin_cvt_fixup; diff --git a/sound/pci/lx6464es/lx6464es.c b/sound/pci/lx6464es/lx6464es.c index bd9b6148dd6f..45ef869ae9e5 100644 --- a/sound/pci/lx6464es/lx6464es.c +++ b/sound/pci/lx6464es/lx6464es.c @@ -410,11 +410,8 @@ static void lx_trigger_start(struct lx6464es *chip, struct lx_stream *lx_stream) int err; - const u32 channels = substream->runtime->channels; - const u32 bytes_per_frame = channels * 3; - const u32 period_size = substream->runtime->period_size; const u32 periods = substream->runtime->periods; - const u32 period_bytes = period_size * bytes_per_frame; + const u32 period_bytes = snd_pcm_lib_period_bytes(substream); dma_addr_t buf = substream->dma_buffer.addr; int i; diff --git a/sound/pci/lx6464es/lx_core.c b/sound/pci/lx6464es/lx_core.c index a99acd1125e7..2680e18cfc2b 100644 --- a/sound/pci/lx6464es/lx_core.c +++ b/sound/pci/lx6464es/lx_core.c @@ -1015,10 +1015,7 @@ static int lx_interrupt_request_new_buffer(struct lx6464es *chip, const unsigned int is_capture = lx_stream->is_capture; int err; - const u32 channels = substream->runtime->channels; - const u32 bytes_per_frame = channels * 3; - const u32 period_size = substream->runtime->period_size; - const u32 period_bytes = period_size * bytes_per_frame; + const u32 period_bytes = snd_pcm_lib_period_bytes(substream); const u32 pos = lx_stream->frame_pos; const u32 next_pos = ((pos+1) == substream->runtime->periods) ? 0 : pos + 1; diff --git a/sound/soc/codecs/max98090.c b/sound/soc/codecs/max98090.c index 4023b88e7bc1..9253707bccde 100644 --- a/sound/soc/codecs/max98090.c +++ b/sound/soc/codecs/max98090.c @@ -2388,8 +2388,9 @@ static int max98090_probe(struct snd_soc_component *component) dev_dbg(component->dev, "max98090_probe\n"); max98090->mclk = devm_clk_get(component->dev, "mclk"); - if (PTR_ERR(max98090->mclk) == -EPROBE_DEFER) - return -EPROBE_DEFER; + if (IS_ERR(max98090->mclk)) + if (PTR_ERR(max98090->mclk) == -EPROBE_DEFER) + return -EPROBE_DEFER; max98090->component = component; diff --git a/sound/soc/codecs/max98095.c b/sound/soc/codecs/max98095.c index 7e525d49328d..3b37f4c4235e 100644 --- a/sound/soc/codecs/max98095.c +++ b/sound/soc/codecs/max98095.c @@ -1984,8 +1984,9 @@ static int max98095_probe(struct snd_soc_component *component) int ret = 0; max98095->mclk = devm_clk_get(component->dev, "mclk"); - if (PTR_ERR(max98095->mclk) == -EPROBE_DEFER) - return -EPROBE_DEFER; + if (IS_ERR(max98095->mclk)) + if (PTR_ERR(max98095->mclk) == -EPROBE_DEFER) + return -EPROBE_DEFER; /* reset the codec, the DSP core, and disable all interrupts */ max98095_reset(component); diff --git a/sound/soc/codecs/tas2562.c b/sound/soc/codecs/tas2562.c index 7daa9e9e0f60..6051027c8266 100644 --- a/sound/soc/codecs/tas2562.c +++ b/sound/soc/codecs/tas2562.c @@ -32,15 +32,16 @@ static const unsigned int float_vol_db_lookup[] = { 0x00000d43, 0x000010b2, 0x00001505, 0x00001a67, 0x00002151, 0x000029f1, 0x000034cd, 0x00004279, 0x000053af, 0x0000695b, -0x0000695b, 0x0000a6fa, 0x0000d236, 0x000108a4, 0x00014d2a, +0x000084a3, 0x0000a6fa, 0x0000d236, 0x000108a4, 0x00014d2a, 0x0001a36e, 0x00021008, 0x000298c0, 0x000344df, 0x00041d8f, 0x00052e5a, 0x000685c8, 0x00083621, 0x000a566d, 0x000d03a7, 0x0010624d, 0x0014a050, 0x0019f786, 0x0020b0bc, 0x0029279d, 0x0033cf8d, 0x004139d3, 0x00521d50, 0x00676044, 0x0082248a, 0x00a3d70a, 0x00ce4328, 0x0103ab3d, 0x0146e75d, 0x019b8c27, 0x02061b89, 0x028c423f, 0x03352529, 0x0409c2b0, 0x05156d68, -0x080e9f96, 0x0a24b062, 0x0cc509ab, 0x10137987, 0x143d1362, -0x197a967f, 0x2013739e, 0x28619ae9, 0x32d64617, 0x40000000 +0x06666666, 0x080e9f96, 0x0a24b062, 0x0cc509ab, 0x10137987, +0x143d1362, 0x197a967f, 0x2013739e, 0x28619ae9, 0x32d64617, +0x40000000 }; struct tas2562_data { @@ -475,20 +476,27 @@ static int tas2562_volume_control_put(struct snd_kcontrol *kcontrol, u32 reg_val; reg_val = float_vol_db_lookup[ucontrol->value.integer.value[0]/2]; - ret = snd_soc_component_write(component, TAS2562_DVC_CFG4, - (reg_val & 0xff)); - if (ret) - return ret; - ret = snd_soc_component_write(component, TAS2562_DVC_CFG3, - ((reg_val >> 8) & 0xff)); + /* + * The device applies the 32-bit coefficient to the playback path on + * the write to DVC_CFG4 (the LSB, book 0 page 2 reg 0x0F), so the + * bytes must be written MSB first and DVC_CFG4 last. Writing CFG4 + * first latches a mix of the previous coefficient's upper bytes and + * the new LSB instead of the requested value. + */ + ret = snd_soc_component_write(component, TAS2562_DVC_CFG1, + ((reg_val >> 24) & 0xff)); if (ret) return ret; ret = snd_soc_component_write(component, TAS2562_DVC_CFG2, ((reg_val >> 16) & 0xff)); if (ret) return ret; - ret = snd_soc_component_write(component, TAS2562_DVC_CFG1, - ((reg_val >> 24) & 0xff)); + ret = snd_soc_component_write(component, TAS2562_DVC_CFG3, + ((reg_val >> 8) & 0xff)); + if (ret) + return ret; + ret = snd_soc_component_write(component, TAS2562_DVC_CFG4, + (reg_val & 0xff)); if (ret) return ret; diff --git a/sound/usb/6fire/chip.c b/sound/usb/6fire/chip.c index 835295e0807d..e5916c6b75ae 100644 --- a/sound/usb/6fire/chip.c +++ b/sound/usb/6fire/chip.c @@ -158,6 +158,10 @@ static int usb6fire_chip_probe(struct usb_interface *intf, return 0; destroy_chip: + chip->shutdown = true; + if (card) + snd_card_disconnect(card); + usb6fire_chip_abort(chip); snd_card_free(card); return ret; } diff --git a/sound/usb/endpoint.c b/sound/usb/endpoint.c index 7eb4c639761a..eeee2886dc21 100644 --- a/sound/usb/endpoint.c +++ b/sound/usb/endpoint.c @@ -1188,10 +1188,12 @@ static int data_ep_set_params(struct snd_usb_endpoint *ep) << (16 - ep->datainterval); } - if (ep->fill_max) + if (ep->fill_max) { ep->curpacksize = ep->maxpacksize; - else + maxsize = ep->curpacksize; + } else { ep->curpacksize = maxsize; + } if (snd_usb_get_speed(chip->dev) != USB_SPEED_FULL) { packs_per_ms = 8 >> ep->datainterval; @@ -1835,11 +1837,13 @@ static void snd_usb_handle_sync_urb(struct snd_usb_endpoint *ep, out_packet->packets = in_ctx->packets; for (i = 0; i < in_ctx->packets; i++) { - if (urb->iso_frame_desc[i].status == 0) - out_packet->packet_size[i] = + if (urb->iso_frame_desc[i].status == 0) { + unsigned int frames = urb->iso_frame_desc[i].actual_length / sender->stride; - else + out_packet->packet_size[i] = min(frames, ep->maxframesize); + } else { out_packet->packet_size[i] = 0; + } } spin_unlock_irqrestore(&ep->lock, flags); diff --git a/sound/usb/midi.c b/sound/usb/midi.c index 5917f4da9776..da6c55a5fe88 100644 --- a/sound/usb/midi.c +++ b/sound/usb/midi.c @@ -800,6 +800,8 @@ static void snd_usbmidi_akai_output(struct snd_usb_midi_out_endpoint *ep, msg = urb->transfer_buffer + urb->transfer_buffer_length; buf_end = ep->max_transfer - MAX_AKAI_SYSEX_LEN - 1; + if (buf_end <= 0) + return; /* only try adding more data when there's space for at least 1 SysEx */ while (urb->transfer_buffer_length < buf_end) { diff --git a/sound/usb/midi2.c b/sound/usb/midi2.c index 95c21c9bf906..19c979df36bb 100644 --- a/sound/usb/midi2.c +++ b/sound/usb/midi2.c @@ -337,7 +337,7 @@ ump_to_endpoint(struct snd_ump_endpoint *ump, int dir) { struct snd_usb_midi2_ump *rmidi = ump->private_data; - return rmidi->eps[dir]; + return rmidi ? rmidi->eps[dir] : NULL; } /* ump open callback */ @@ -693,6 +693,8 @@ static void free_all_midi2_umps(struct snd_usb_midi2_interface *umidi) rmidi = list_first_entry(&umidi->rawmidi_list, struct snd_usb_midi2_ump, list); list_del(&rmidi->list); + if (rmidi->ump) + rmidi->ump->private_data = NULL; kfree(rmidi); } } diff --git a/sound/usb/mixer_quirks.c b/sound/usb/mixer_quirks.c index 487396db19ff..48f6b53fefd8 100644 --- a/sound/usb/mixer_quirks.c +++ b/sound/usb/mixer_quirks.c @@ -3413,7 +3413,7 @@ static int snd_rme_digiface_read_status(struct snd_kcontrol *kcontrol, u32 statu struct usb_mixer_elem_list *list = snd_kcontrol_chip(kcontrol); struct snd_usb_audio *chip = list->mixer->chip; struct usb_device *dev = chip->dev; - __le32 buf[4]; + __le32 buf[4] = {}; int err; err = snd_usb_ctl_msg(dev, usb_rcvctrlpipe(dev, 0), diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index 12b0f2ee5665..c09b66ba734b 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -14,8 +14,9 @@ import os import shlex import shutil import signal +import sys import threading -from typing import Iterator, List, Optional, Tuple +from typing import Iterator, List, Optional, Tuple, Any from types import FrameType import kunit_config @@ -252,6 +253,7 @@ class LinuxSourceTree: if kconfig_add: kconfig = kunit_config.parse_from_string('\n'.join(kconfig_add)) self._kconfig.merge_in_entries(kconfig) + self._process : Optional[subprocess.Popen[Any]] = None def arch(self) -> str: return self._arch @@ -332,6 +334,12 @@ class LinuxSourceTree: return False return self.validate_config(build_dir) + def _restore_terminal_if_tty(self) -> None: + # stty requires a controlling terminal; skip headless runs. + if sys.stdin is None or not sys.stdin.isatty(): + return + subprocess.call(['stty', 'sane']) + def run_kernel(self, args: Optional[List[str]]=None, build_dir: str='', filter_glob: str='', filter: str='', filter_action: Optional[str]=None, timeout: Optional[int]=None) -> Iterator[str]: # Copy to avoid mutating the caller-supplied list. exec_tests() reuses # the same args across repeated run_kernel() calls (e.g. --run_isolated), @@ -345,36 +353,45 @@ class LinuxSourceTree: args.append('kunit.filter_action=' + filter_action) args.append('kunit.enable=1') - process = self._ops.start(args, build_dir) - assert process.stdout is not None # tell mypy it's set + self._process = self._ops.start(args, build_dir) + assert self._process is not None # tell mypy it's set + assert self._process.stdout is not None # tell mypy it's set # Enforce the timeout in a background thread. def _wait_proc() -> None: try: - process.wait(timeout=timeout) + if self._process: + self._process.wait(timeout=timeout) except Exception as e: print(e) - process.terminate() - process.wait() + if self._process: + self._process.terminate() + self._process.wait() waiter = threading.Thread(target=_wait_proc) waiter.start() output = open(get_outfile_path(build_dir), 'w') try: # Tee the output to the file and to our caller in real time. - for line in process.stdout: + for line in self._process.stdout: output.write(line) yield line # This runs even if our caller doesn't consume every line. finally: # Flush any leftover output to the file - output.write(process.stdout.read()) + if self._process: + if self._process.stdout: + output.write(self._process.stdout.read()) + self._process.stdout.close() + self._process = None output.close() - process.stdout.close() waiter.join() - subprocess.call(['stty', 'sane']) + self._restore_terminal_if_tty() def signal_handler(self, unused_sig: int, unused_frame: Optional[FrameType]) -> None: logging.error('Build interruption occurred. Cleaning console.') - subprocess.call(['stty', 'sane']) + if self._process: + self._process.terminate() + self._process.wait() + self._restore_terminal_if_tty() diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index 70e5d0abe87f..6e6200d52540 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -503,6 +503,48 @@ class LinuxSourceTreeTest(unittest.TestCase): self.assertIn('kunit.filter_glob=suite.test1', start_calls[0]) self.assertIn('kunit.filter_glob=suite.test2', start_calls[1]) + def test_run_kernel_skips_terminal_reset_without_tty(self): + def fake_start(unused_args, unused_build_dir): + return subprocess.Popen(['printf', 'KTAP version 1\n'], + text=True, stdout=subprocess.PIPE) + + non_tty_stdin = mock.Mock() + non_tty_stdin.isatty.return_value = False + + with tempfile.TemporaryDirectory('') as build_dir: + tree = kunit_kernel.LinuxSourceTree(build_dir, kunitconfig_paths=[os.devnull]) + with mock.patch.object(tree._ops, 'start', side_effect=fake_start), \ + mock.patch.object(kunit_kernel.sys, 'stdin', non_tty_stdin), \ + mock.patch.object(kunit_kernel.subprocess, 'call') as mock_call: + for _ in tree.run_kernel(build_dir=build_dir): + pass + + mock_call.assert_not_called() + + def test_signal_handler_skips_terminal_reset_without_tty(self): + non_tty_stdin = mock.Mock() + non_tty_stdin.isatty.return_value = False + tree = kunit_kernel.LinuxSourceTree('', kunitconfig_paths=[os.devnull]) + + with mock.patch.object(kunit_kernel.sys, 'stdin', non_tty_stdin), \ + mock.patch.object(kunit_kernel.subprocess, 'call') as mock_call, \ + mock.patch.object(kunit_kernel.logging, 'error') as mock_error: + tree.signal_handler(signal.SIGINT, None) + mock_error.assert_called_once() + mock_call.assert_not_called() + + def test_signal_handler_resets_terminal_with_tty(self): + tty_stdin = mock.Mock() + tty_stdin.isatty.return_value = True + tree = kunit_kernel.LinuxSourceTree('', kunitconfig_paths=[os.devnull]) + + with mock.patch.object(kunit_kernel.sys, 'stdin', tty_stdin), \ + mock.patch.object(kunit_kernel.subprocess, 'call') as mock_call, \ + mock.patch.object(kunit_kernel.logging, 'error') as mock_error: + tree.signal_handler(signal.SIGINT, None) + mock_error.assert_called_once() + mock_call.assert_called_once_with(['stty', 'sane']) + def test_build_reconfig_no_config(self): with tempfile.TemporaryDirectory('') as build_dir: with open(kunit_kernel.get_kunitconfig_path(build_dir), 'w') as f: diff --git a/tools/testing/selftests/clone3/clone3_set_tid.c b/tools/testing/selftests/clone3/clone3_set_tid.c index bfb0da2b4fdd..4debe0756b67 100644 --- a/tools/testing/selftests/clone3/clone3_set_tid.c +++ b/tools/testing/selftests/clone3/clone3_set_tid.c @@ -141,7 +141,7 @@ int main(int argc, char *argv[]) { FILE *f; char buf; - char *line; + char *line = NULL; int status; int ret = -1; size_t len = 0; diff --git a/tools/testing/selftests/mm/mlock-random-test.c b/tools/testing/selftests/mm/mlock-random-test.c index 1cd80b0f76c3..f551bc799399 100644 --- a/tools/testing/selftests/mm/mlock-random-test.c +++ b/tools/testing/selftests/mm/mlock-random-test.c @@ -84,7 +84,7 @@ int get_proc_locked_vm_size(void) int get_proc_page_size(unsigned long addr) { FILE *smaps; - char *line; + char *line = NULL; unsigned long mmupage_size = 0; size_t size;