stable.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Bart Van Assche <bart.vanassche@wdc.com>,
	Tejun Heo <tj@kernel.org>,
	Johannes Thumshirn <jthumshirn@suse.de>,
	"Martin K. Petersen" <martin.petersen@oracle.com>
Subject: [PATCH 4.9 100/107] scsi: core: Avoid that SCSI device removal through sysfs triggers a deadlock
Date: Mon,  3 Sep 2018 18:57:04 +0200	[thread overview]
Message-ID: <20180903165642.465724701@linuxfoundation.org> (raw)
In-Reply-To: <20180903165637.293735109@linuxfoundation.org>

4.9-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Bart Van Assche <bart.vanassche@wdc.com>

commit 0ee223b2e1f67cb2de9c0e3247c510d846e74d63 upstream.

A long time ago the unfortunate decision was taken to add a self-deletion
attribute to the sysfs SCSI device directory. That decision was unfortunate
because self-deletion is really tricky. We can't drop that attribute
because widely used user space software depends on it, namely the
rescan-scsi-bus.sh script. Hence this patch that avoids that writing into
that attribute triggers a deadlock. See also commit 7973cbd9fbd9 ("[PATCH]
add sysfs attributes to scan and delete scsi_devices").

This patch avoids that self-removal triggers the following deadlock:

======================================================
WARNING: possible circular locking dependency detected
4.18.0-rc2-dbg+ #5 Not tainted
------------------------------------------------------
modprobe/6539 is trying to acquire lock:
000000008323c4cd (kn->count#202){++++}, at: kernfs_remove_by_name_ns+0x45/0x90

but task is already holding lock:
00000000a6ec2c69 (&shost->scan_mutex){+.+.}, at: scsi_remove_host+0x21/0x150 [scsi_mod]

which lock already depends on the new lock.

the existing dependency chain (in reverse order) is:

-> #1 (&shost->scan_mutex){+.+.}:
       __mutex_lock+0xfe/0xc70
       mutex_lock_nested+0x1b/0x20
       scsi_remove_device+0x26/0x40 [scsi_mod]
       sdev_store_delete+0x27/0x30 [scsi_mod]
       dev_attr_store+0x3e/0x50
       sysfs_kf_write+0x87/0xa0
       kernfs_fop_write+0x190/0x230
       __vfs_write+0xd2/0x3b0
       vfs_write+0x101/0x270
       ksys_write+0xab/0x120
       __x64_sys_write+0x43/0x50
       do_syscall_64+0x77/0x230
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #0 (kn->count#202){++++}:
       lock_acquire+0xd2/0x260
       __kernfs_remove+0x424/0x4a0
       kernfs_remove_by_name_ns+0x45/0x90
       remove_files.isra.1+0x3a/0x90
       sysfs_remove_group+0x5c/0xc0
       sysfs_remove_groups+0x39/0x60
       device_remove_attrs+0x82/0xb0
       device_del+0x251/0x580
       __scsi_remove_device+0x19f/0x1d0 [scsi_mod]
       scsi_forget_host+0x37/0xb0 [scsi_mod]
       scsi_remove_host+0x9b/0x150 [scsi_mod]
       sdebug_driver_remove+0x4b/0x150 [scsi_debug]
       device_release_driver_internal+0x241/0x360
       device_release_driver+0x12/0x20
       bus_remove_device+0x1bc/0x290
       device_del+0x259/0x580
       device_unregister+0x1a/0x70
       sdebug_remove_adapter+0x8b/0xf0 [scsi_debug]
       scsi_debug_exit+0x76/0xe8 [scsi_debug]
       __x64_sys_delete_module+0x1c1/0x280
       do_syscall_64+0x77/0x230
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&shost->scan_mutex);
                               lock(kn->count#202);
                               lock(&shost->scan_mutex);
  lock(kn->count#202);

 *** DEADLOCK ***

2 locks held by modprobe/6539:
 #0: 00000000efaf9298 (&dev->mutex){....}, at: device_release_driver_internal+0x68/0x360
 #1: 00000000a6ec2c69 (&shost->scan_mutex){+.+.}, at: scsi_remove_host+0x21/0x150 [scsi_mod]

stack backtrace:
CPU: 10 PID: 6539 Comm: modprobe Not tainted 4.18.0-rc2-dbg+ #5
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.0.0-prebuilt.qemu-project.org 04/01/2014
Call Trace:
 dump_stack+0xa4/0xf5
 print_circular_bug.isra.34+0x213/0x221
 __lock_acquire+0x1a7e/0x1b50
 lock_acquire+0xd2/0x260
 __kernfs_remove+0x424/0x4a0
 kernfs_remove_by_name_ns+0x45/0x90
 remove_files.isra.1+0x3a/0x90
 sysfs_remove_group+0x5c/0xc0
 sysfs_remove_groups+0x39/0x60
 device_remove_attrs+0x82/0xb0
 device_del+0x251/0x580
 __scsi_remove_device+0x19f/0x1d0 [scsi_mod]
 scsi_forget_host+0x37/0xb0 [scsi_mod]
 scsi_remove_host+0x9b/0x150 [scsi_mod]
 sdebug_driver_remove+0x4b/0x150 [scsi_debug]
 device_release_driver_internal+0x241/0x360
 device_release_driver+0x12/0x20
 bus_remove_device+0x1bc/0x290
 device_del+0x259/0x580
 device_unregister+0x1a/0x70
 sdebug_remove_adapter+0x8b/0xf0 [scsi_debug]
 scsi_debug_exit+0x76/0xe8 [scsi_debug]
 __x64_sys_delete_module+0x1c1/0x280
 do_syscall_64+0x77/0x230
 entry_SYSCALL_64_after_hwframe+0x49/0xbe

See also https://www.mail-archive.com/linux-scsi@vger.kernel.org/msg54525.html.

Fixes: ac0ece9174ac ("scsi: use device_remove_file_self() instead of device_schedule_callback()")
Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Tejun Heo <tj@kernel.org>
Cc: Johannes Thumshirn <jthumshirn@suse.de>
Cc: <stable@vger.kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

---
 drivers/scsi/scsi_sysfs.c |   20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -709,8 +709,24 @@ static ssize_t
 sdev_store_delete(struct device *dev, struct device_attribute *attr,
 		  const char *buf, size_t count)
 {
-	if (device_remove_file_self(dev, attr))
-		scsi_remove_device(to_scsi_device(dev));
+	struct kernfs_node *kn;
+
+	kn = sysfs_break_active_protection(&dev->kobj, &attr->attr);
+	WARN_ON_ONCE(!kn);
+	/*
+	 * Concurrent writes into the "delete" sysfs attribute may trigger
+	 * concurrent calls to device_remove_file() and scsi_remove_device().
+	 * device_remove_file() handles concurrent removal calls by
+	 * serializing these and by ignoring the second and later removal
+	 * attempts.  Concurrent calls of scsi_remove_device() are
+	 * serialized. The second and later calls of scsi_remove_device() are
+	 * ignored because the first call of that function changes the device
+	 * state into SDEV_DEL.
+	 */
+	device_remove_file(dev, attr);
+	scsi_remove_device(to_scsi_device(dev));
+	if (kn)
+		sysfs_unbreak_active_protection(kn);
 	return count;
 };
 static DEVICE_ATTR(delete, S_IWUSR, NULL, sdev_store_delete);

  parent reply	other threads:[~2018-09-03 21:36 UTC|newest]

Thread overview: 105+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-09-03 16:55 [PATCH 4.9 000/107] 4.9.125-stable review Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 001/107] vti6: fix PMTU caching and reporting on xmit Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 002/107] xfrm: fix missing dst_release() after policy blocking lbcast and multicast Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 003/107] xfrm: free skb if nlsk pointer is NULL Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 004/107] mac80211: add stations tied to AP_VLANs during hw reconfig Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 005/107] nl80211: Add a missing break in parse_station_flags Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 006/107] drm/bridge: adv7511: Reset registers on hotplug Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 007/107] scsi: libiscsi: fix possible NULL pointer dereference in case of TMF Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 008/107] drm/imx: imx-ldb: disable LDB on driver bind Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 009/107] drm/imx: imx-ldb: check if channel is enabled before printing warning Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 010/107] usb: gadget: r8a66597: Fix two possible sleep-in-atomic-context bugs in init_controller() Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 011/107] usb: gadget: r8a66597: Fix a possible sleep-in-atomic-context bugs in r8a66597_queue() Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 012/107] usb/phy: fix PPC64 build errors in phy-fsl-usb.c Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 014/107] usb: gadget: f_uac2: fix endianness of struct cntrl_*_lay3 Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 015/107] bpf, ppc64: fix unexpected r0=0 exit path inside bpf_xadd Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 016/107] tools/power turbostat: fix -S on UP systems Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 017/107] net: caif: Add a missing rcu_read_unlock() in caif_flow_cb Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 018/107] qed: Fix possible race for the link state value Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 019/107] qed: Correct Multicast API to reflect existence of 256 approximate buckets Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 020/107] atl1c: reserve min skb headroom Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 021/107] net: prevent ISA drivers from building on PPC32 Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 022/107] can: mpc5xxx_can: check of_iomap return before use Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 023/107] i2c: davinci: Avoid zero value of CLKH Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 024/107] perf/x86/amd/ibs: Dont access non-started event Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 025/107] media: staging: omap4iss: Include asm/cacheflush.h after generic includes Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 026/107] bnx2x: Fix invalid memory access in rss hash config path Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 028/107] net: axienet: Fix double deregister of mdio Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 029/107] x86/boot: Fix if_changed build flip/flop bug Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 030/107] fscache: Allow cancelled operations to be enqueued Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 031/107] cachefiles: Fix refcounting bug in backing-file read monitoring Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 032/107] cachefiles: Wait rather than BUGing on "Unexpected object collision" Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 033/107] selftests/ftrace: Add snapshot and tracing_on test case Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 034/107] zswap: re-check zswap_is_full() after do zswap_shrink() Greg Kroah-Hartman
2018-09-03 16:55 ` [PATCH 4.9 035/107] tools/power turbostat: Read extended processor family from CPUID Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 037/107] enic: handle mtu change for vf properly Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 038/107] arc: [plat-eznps] fix data type errors in platform headers Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 039/107] arc: fix build errors in arc/include/asm/delay.h Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 040/107] arc: fix type warnings in arc/mm/cache.c Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 041/107] squashfs metadata 2: electric boogaloo Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 043/107] drivers: net: lmc: fix case value for target abort error Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 044/107] memcg: remove memcg_cgroup::id from IDR on mem_cgroup_css_alloc() failure Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 045/107] scsi: fcoe: drop frames in ELS LOGO error path Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 046/107] scsi: fcoe: clear FC_RP_STARTED flags when receiving a LOGO Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 047/107] scsi: vmw_pvscsi: Return DID_RESET for status SAM_STAT_COMMAND_TERMINATED Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 048/107] mm/memory.c: check return value of ioremap_prot Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 049/107] sched/sysctl: Check user input value of sysctl_sched_time_avg Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 050/107] Cipso: cipso_v4_optptr enter infinite loop Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 051/107] mei: dont update offset in write Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 052/107] cifs: add missing debug entries for kconfig options Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 053/107] cifs: check kmalloc before use Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 054/107] smb3: enumerating snapshots was leaving part of the data off end Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 055/107] smb3: Do not send SMB3 SET_INFO if nothing changed Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 056/107] smb3: dont request leases in symlink creation and query Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 057/107] kprobes/arm64: Fix %p uses in error messages Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 058/107] arm64: mm: check for upper PAGE_SHIFT bits in pfn_valid() Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 059/107] s390/kvm: fix deadlock when killed by oom Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 060/107] ext4: check for NUL characters in extended attributes name Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 061/107] ext4: sysfs: print ext4_super_block fields as little-endian Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 062/107] ext4: reset error code in ext4_find_entry in fallback Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 063/107] staging: android: ion: fix ION_IOC_{MAP,SHARE} use-after-free Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 064/107] KVM: arm/arm64: Skip updating PTE entry if no change Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 065/107] KVM: arm/arm64: Skip updating PMD " Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 067/107] x86/speculation/l1tf: Fix overflow in l1tf_pfn_limit() on 32bit Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 068/107] x86/speculation/l1tf: Fix off-by-one error when warning that system has too much RAM Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 069/107] x86/speculation/l1tf: Suggest what to do on systems with " Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 070/107] x86/process: Re-export start_thread() Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 072/107] x86/kvm/vmx: Remove duplicate l1d flush definitions Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 073/107] fuse: Dont access pipe->buffers without pipe_lock() Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 074/107] fuse: fix initial parallel dirops Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 075/107] fuse: fix double request_end() Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 076/107] fuse: fix unlocked access to processing queue Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 077/107] fuse: umount should wait for all requests Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 078/107] fuse: Fix oops at process_init_reply() Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 079/107] fuse: Add missed unlock_page() to fuse_readpages_fill() Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 080/107] udl-kms: change down_interruptible to down Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 081/107] udl-kms: handle allocation failure Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 082/107] udl-kms: fix crash due to uninitialized memory Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 083/107] b43legacy/leds: Ensure NUL-termination of LED name string Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 084/107] b43/leds: " Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 085/107] ASoC: dpcm: dont merge format from invalid codec dai Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 086/107] ASoC: sirf: Fix potential NULL pointer dereference Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 088/107] x86/irqflags: Mark native_restore_fl extern inline Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 089/107] x86/spectre: Add missing family 6 check to microcode check Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 090/107] x86/speculation/l1tf: Increase l1tf memory limit for Nehalem+ Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 091/107] x86/entry/64: Wipe KASAN stack shadow before rewind_stack_do_exit() Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 092/107] s390: fix br_r1_trampoline for machines without exrl Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 093/107] s390/qdio: reset old sbal_state flags Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 094/107] s390/numa: move initial setup of node_to_cpumask_map Greg Kroah-Hartman
2018-09-03 16:56 ` [PATCH 4.9 095/107] s390/pci: fix out of bounds access during irq setup Greg Kroah-Hartman
2018-09-03 16:57 ` [PATCH 4.9 096/107] kprobes: Make list and blacklist root user read only Greg Kroah-Hartman
2018-09-03 16:57 ` [PATCH 4.9 097/107] MIPS: Correct the 64-bit DSP accumulator register size Greg Kroah-Hartman
2018-09-03 16:57 ` [PATCH 4.9 098/107] MIPS: lib: Provide MIPS64r6 __multi3() for GCC < 7 Greg Kroah-Hartman
2018-09-03 16:57 ` [PATCH 4.9 099/107] scsi: sysfs: Introduce sysfs_{un,}break_active_protection() Greg Kroah-Hartman
2018-09-03 16:57 ` Greg Kroah-Hartman [this message]
2018-09-03 16:57 ` [PATCH 4.9 101/107] iscsi target: fix session creation failure handling Greg Kroah-Hartman
2018-09-03 16:57 ` [PATCH 4.9 102/107] clk: rockchip: fix clk_i2sout parent selection bits on rk3399 Greg Kroah-Hartman
2018-09-03 16:57 ` [PATCH 4.9 103/107] PM / clk: signedness bug in of_pm_clk_add_clks() Greg Kroah-Hartman
2018-09-03 16:57 ` [PATCH 4.9 104/107] power: generic-adc-battery: fix out-of-bounds write when copying channel properties Greg Kroah-Hartman
2018-09-03 16:57 ` [PATCH 4.9 105/107] power: generic-adc-battery: check for duplicate properties copied from iio channels Greg Kroah-Hartman
2018-09-03 16:57 ` [PATCH 4.9 106/107] cdrom: Fix info leak/OOB read in cdrom_ioctl_drive_status Greg Kroah-Hartman
2018-09-03 16:57 ` [PATCH 4.9 107/107] staging: android: ion: check for kref overflow Greg Kroah-Hartman
2018-09-04  8:22 ` [PATCH 4.9 000/107] 4.9.125-stable review Naresh Kamboju
2018-09-04 19:33   ` Greg Kroah-Hartman
2018-09-04 19:29 ` Shuah Khan
2018-09-04 22:51 ` Guenter Roeck

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180903165642.465724701@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=bart.vanassche@wdc.com \
    --cc=jthumshirn@suse.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=martin.petersen@oracle.com \
    --cc=stable@vger.kernel.org \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).