From: lizf@kernel.org
To: stable@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, "Seymour,
Shane M" <shane.seymour@hp.com>,
Darren Lavender <darren.lavender@hp.com>,
James Bottomley <JBottomley@Odin.com>,
Zefan Li <lizefan@huawei.com>
Subject: [PATCH 3.4 023/107] st: null pointer dereference panic caused by use after kref_put by st_open
Date: Wed, 16 Mar 2016 16:05:17 +0800 [thread overview]
Message-ID: <1458115601-5762-23-git-send-email-lizf@kernel.org> (raw)
In-Reply-To: <1458115541-5712-1-git-send-email-lizf@kernel.org>
From: "Seymour, Shane M" <shane.seymour@hp.com>
3.4.111-rc1 review patch. If anyone has any objections, please let me know.
------------------
commit e7ac6c6666bec0a354758a1298d3231e4a635362 upstream.
Two SLES11 SP3 servers encountered similar crashes simultaneously
following some kind of SAN/tape target issue:
...
qla2xxx [0000:81:00.0]-801c:3: Abort command issued nexus=3:0:2 -- 1 2002.
qla2xxx [0000:81:00.0]-801c:3: Abort command issued nexus=3:0:2 -- 1 2002.
qla2xxx [0000:81:00.0]-8009:3: DEVICE RESET ISSUED nexus=3:0:2 cmd=ffff882f89c2c7c0.
qla2xxx [0000:81:00.0]-800c:3: do_reset failed for cmd=ffff882f89c2c7c0.
qla2xxx [0000:81:00.0]-800f:3: DEVICE RESET FAILED: Task management failed nexus=3:0:2 cmd=ffff882f89c2c7c0.
qla2xxx [0000:81:00.0]-8009:3: TARGET RESET ISSUED nexus=3:0:2 cmd=ffff882f89c2c7c0.
qla2xxx [0000:81:00.0]-800c:3: do_reset failed for cmd=ffff882f89c2c7c0.
qla2xxx [0000:81:00.0]-800f:3: TARGET RESET FAILED: Task management failed nexus=3:0:2 cmd=ffff882f89c2c7c0.
qla2xxx [0000:81:00.0]-8012:3: BUS RESET ISSUED nexus=3:0:2.
qla2xxx [0000:81:00.0]-802b:3: BUS RESET SUCCEEDED nexus=3:0:2.
qla2xxx [0000:81:00.0]-505f:3: Link is operational (8 Gbps).
qla2xxx [0000:81:00.0]-8018:3: ADAPTER RESET ISSUED nexus=3:0:2.
qla2xxx [0000:81:00.0]-00af:3: Performing ISP error recovery - ha=ffff88bf04d18000.
rport-3:0-0: blocked FC remote port time out: removing target and saving binding
qla2xxx [0000:81:00.0]-505f:3: Link is operational (8 Gbps).
qla2xxx [0000:81:00.0]-8017:3: ADAPTER RESET SUCCEEDED nexus=3:0:2.
rport-2:0-0: blocked FC remote port time out: removing target and saving binding
sg_rq_end_io: device detached
BUG: unable to handle kernel NULL pointer dereference at 00000000000002a8
IP: [<ffffffff8133b268>] __pm_runtime_idle+0x28/0x90
PGD 7e6586f067 PUD 7e5af06067 PMD 0 [1739975.390354] Oops: 0002 [#1] SMP
CPU 0
...
Supported: No, Proprietary modules are loaded [1739975.390463]
Pid: 27965, comm: ABCD Tainted: PF X 3.0.101-0.29-default #1 HP ProLiant DL580 Gen8
RIP: 0010:[<ffffffff8133b268>] [<ffffffff8133b268>] __pm_runtime_idle+0x28/0x90
RSP: 0018:ffff8839dc1e7c68 EFLAGS: 00010202
RAX: 0000000000000000 RBX: ffff883f0592fc00 RCX: 0000000000000090
RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000138
RBP: 0000000000000138 R08: 0000000000000010 R09: ffffffff81bd39d0
R10: 00000000000009c0 R11: ffffffff81025790 R12: 0000000000000001
R13: ffff883022212b80 R14: 0000000000000004 R15: ffff883022212b80
FS: 00007f8e54560720(0000) GS:ffff88407f800000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00000000000002a8 CR3: 0000007e6ced6000 CR4: 00000000001407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process ABCD (pid: 27965, threadinfo ffff8839dc1e6000, task ffff883592e0c640)
Stack:
ffff883f0592fc00 00000000fffffffa 0000000000000001 ffff883022212b80
ffff883eff772400 ffffffffa03fa309 0000000000000000 0000000000000000
ffffffffa04003a0 ffff883f063196c0 ffff887f0379a930 ffffffff8115ea1e
Call Trace:
[<ffffffffa03fa309>] st_open+0x129/0x240 [st]
[<ffffffff8115ea1e>] chrdev_open+0x13e/0x200
[<ffffffff811588a8>] __dentry_open+0x198/0x310
[<ffffffff81167d74>] do_last+0x1f4/0x800
[<ffffffff81168fe9>] path_openat+0xd9/0x420
[<ffffffff8116946c>] do_filp_open+0x4c/0xc0
[<ffffffff8115a00f>] do_sys_open+0x17f/0x250
[<ffffffff81468d92>] system_call_fastpath+0x16/0x1b
[<00007f8e4f617fd0>] 0x7f8e4f617fcf
Code: eb d3 90 48 83 ec 28 40 f6 c6 04 48 89 6c 24 08 4c 89 74 24 20 48 89 fd 48 89 1c 24 4c 89 64 24 10 41 89 f6 4c 89 6c 24 18 74 11 <f0> ff 8f 70 01 00 00 0f 94 c0 45 31 ed 84 c0 74 2b 4c 8d a5 a0
RIP [<ffffffff8133b268>] __pm_runtime_idle+0x28/0x90
RSP <ffff8839dc1e7c68>
CR2: 00000000000002a8
Analysis reveals the cause of the crash to be due to STp->device
being NULL. The pointer was NULLed via scsi_tape_put(STp) when it
calls scsi_tape_release(). In st_open() we jump to err_out after
scsi_block_when_processing_errors() completes and returns the
device as offline (sdev_state was SDEV_DEL):
1180 /* Open the device. Needs to take the BKL only because of incrementing the SCSI host
1181 module count. */
1182 static int st_open(struct inode *inode, struct file *filp)
1183 {
1184 int i, retval = (-EIO);
1185 int resumed = 0;
1186 struct scsi_tape *STp;
1187 struct st_partstat *STps;
1188 int dev = TAPE_NR(inode);
1189 char *name;
...
1217 if (scsi_autopm_get_device(STp->device) < 0) {
1218 retval = -EIO;
1219 goto err_out;
1220 }
1221 resumed = 1;
1222 if (!scsi_block_when_processing_errors(STp->device)) {
1223 retval = (-ENXIO);
1224 goto err_out;
1225 }
...
1264 err_out:
1265 normalize_buffer(STp->buffer);
1266 spin_lock(&st_use_lock);
1267 STp->in_use = 0;
1268 spin_unlock(&st_use_lock);
1269 scsi_tape_put(STp); <-- STp->device = 0 after this
1270 if (resumed)
1271 scsi_autopm_put_device(STp->device);
1272 return retval;
The ref count for the struct scsi_tape had already been reduced
to 1 when the .remove method of the st module had been called.
The kref_put() in scsi_tape_put() caused scsi_tape_release()
to be called:
0266 static void scsi_tape_put(struct scsi_tape *STp)
0267 {
0268 struct scsi_device *sdev = STp->device;
0269
0270 mutex_lock(&st_ref_mutex);
0271 kref_put(&STp->kref, scsi_tape_release); <-- calls this
0272 scsi_device_put(sdev);
0273 mutex_unlock(&st_ref_mutex);
0274 }
In scsi_tape_release() the struct scsi_device in the struct
scsi_tape gets set to NULL:
4273 static void scsi_tape_release(struct kref *kref)
4274 {
4275 struct scsi_tape *tpnt = to_scsi_tape(kref);
4276 struct gendisk *disk = tpnt->disk;
4277
4278 tpnt->device = NULL; <<<---- where the dev is nulled
4279
4280 if (tpnt->buffer) {
4281 normalize_buffer(tpnt->buffer);
4282 kfree(tpnt->buffer->reserved_pages);
4283 kfree(tpnt->buffer);
4284 }
4285
4286 disk->private_data = NULL;
4287 put_disk(disk);
4288 kfree(tpnt);
4289 return;
4290 }
Although the problem was reported on SLES11.3 the problem appears
in linux-next as well.
The crash is fixed by reordering the code so we no longer access
the struct scsi_tape after the kref_put() is done on it in st_open().
Signed-off-by: Shane Seymour <shane.seymour@hp.com>
Signed-off-by: Darren Lavender <darren.lavender@hp.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.com>
Acked-by: Kai Mäkisara <kai.makisara@kolumbus.fi>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
Signed-off-by: Zefan Li <lizefan@huawei.com>
---
drivers/scsi/st.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c
index e41998c..a3eb263 100644
--- a/drivers/scsi/st.c
+++ b/drivers/scsi/st.c
@@ -1268,9 +1268,9 @@ static int st_open(struct inode *inode, struct file *filp)
err_out:
normalize_buffer(STp->buffer);
STp->in_use = 0;
- scsi_tape_put(STp);
if (resumed)
scsi_autopm_put_device(STp->device);
+ scsi_tape_put(STp);
mutex_unlock(&st_mutex);
return retval;
--
1.9.1
next prev parent reply other threads:[~2016-03-16 8:40 UTC|newest]
Thread overview: 114+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-03-16 8:05 [PATCH 3.4 000/107] 3.4.111-rc1 review lizf
2016-03-16 8:04 ` [PATCH 3.4 001/107] Btrfs: use kmem_cache_free when freeing entry in inode cache lizf
2016-03-16 8:04 ` [PATCH 3.4 002/107] fs/buffer.c: support buffer cache allocations with gfp modifiers lizf
2016-03-16 8:04 ` [PATCH 3.4 003/107] bufferhead: Add _gfp version for sb_getblk() lizf
2016-03-16 8:04 ` [PATCH 3.4 004/107] ext4: avoid deadlocks in the writeback path by using sb_getblk_gfp lizf
2016-03-16 8:04 ` [PATCH 3.4 005/107] ext4: replace open coded nofail allocation in ext4_free_blocks() lizf
2016-03-16 8:05 ` [PATCH 3.4 006/107] mm: avoid setting up anonymous pages into file mapping lizf
2016-03-16 8:05 ` [PATCH 3.4 007/107] hpfs: kstrdup() out of memory handling lizf
2016-03-16 8:05 ` [PATCH 3.4 008/107] hpfs: hpfs_error: Remove static buffer, use vsprintf extension %pV instead lizf
2016-03-16 8:05 ` [PATCH 3.4 009/107] 9p: don't leave a half-initialized inode sitting around lizf
2016-03-16 8:05 ` [PATCH 3.4 010/107] ALSA: usb-audio: Add MIDI support for Steinberg MI2/MI4 lizf
2016-03-16 8:05 ` [PATCH 3.4 011/107] dm btree remove: fix bug in redistribute3 lizf
2016-03-16 8:05 ` [PATCH 3.4 012/107] dm thin: allocate the cell_sort_array dynamically lizf
2016-03-16 8:05 ` [PATCH 3.4 013/107] USB: option: add 2020:4000 ID lizf
2016-03-16 8:05 ` [PATCH 3.4 014/107] USB: cp210x: add ID for Aruba Networks controllers lizf
2016-03-16 8:05 ` [PATCH 3.4 015/107] dm btree: silence lockdep lock inversion in dm_btree_del() lizf
2016-03-16 8:05 ` [PATCH 3.4 016/107] s390/sclp: clear upper register halves in _sclp_print_early lizf
2016-03-16 8:05 ` [PATCH 3.4 017/107] drm: add a check for x/y in drm_mode_setcrtc lizf
2016-03-16 8:05 ` [PATCH 3.4 018/107] rtnetlink: verify IFLA_VF_INFO attributes before passing them to driver lizf
2016-03-16 8:05 ` [PATCH 3.4 019/107] net: do not process device backlog during unregistration lizf
2016-03-16 8:05 ` [PATCH 3.4 020/107] net: call rcu_read_lock early in process_backlog lizf
2016-03-16 8:05 ` [PATCH 3.4 021/107] s390/process: fix sfpc inline assembly lizf
2016-03-16 8:05 ` [PATCH 3.4 022/107] rds: rds_ib_device.refcount overflow lizf
2016-03-16 8:05 ` lizf [this message]
2016-03-16 8:05 ` [PATCH 3.4 024/107] ata: pmp: add quirk for Marvell 4140 SATA PMP lizf
2016-03-16 8:05 ` [PATCH 3.4 025/107] libata: add ATA_HORKAGE_BROKEN_FPDMA_AA quirk for HP 250GB SATA disk VB0250EAVER lizf
2016-03-16 8:05 ` [PATCH 3.4 026/107] libata: add ATA_HORKAGE_NOTRIM lizf
2016-03-16 8:05 ` [PATCH 3.4 027/107] libata: force disable trim for SuperSSpeed S238 lizf
2016-03-16 8:05 ` [PATCH 3.4 028/107] libata: increase the timeout when setting transfer mode lizf
2016-03-16 8:05 ` [PATCH 3.4 029/107] net: Clone skb before setting peeked flag lizf
2016-03-16 8:05 ` [PATCH 3.4 030/107] NET: AX.25: Stop heartbeat timer on disconnect lizf
2016-03-16 10:40 ` Richard Stearn
2016-03-17 1:18 ` Zefan Li
2016-03-16 8:05 ` [PATCH 3.4 031/107] can: mcp251x: fix resume when device is down lizf
2016-03-16 8:05 ` [PATCH 3.4 032/107] mac80211: clear subdir_stations when removing debugfs lizf
2016-03-16 8:05 ` [PATCH 3.4 033/107] inet: frags: fix defragmented packet's IP header for af_packet lizf
2016-03-16 8:05 ` [PATCH 3.4 034/107] md: make sure everything is freed when dm-raid stops an array lizf
2016-03-16 8:05 ` [PATCH 3.4 035/107] md: flush ->event_work before stopping array lizf
2016-03-16 8:05 ` [PATCH 3.4 036/107] usb: dwc3: Reset the transfer resource index on SET_INTERFACE lizf
2016-03-16 8:05 ` [PATCH 3.4 037/107] usb: xhci: Bugfix for NULL pointer deference in xhci_endpoint_init() function lizf
2016-03-16 8:05 ` [PATCH 3.4 038/107] xhci: Calculate old endpoints correctly on device reset lizf
2016-03-16 8:05 ` [PATCH 3.4 039/107] xhci: report U3 when link is in resume state lizf
2016-03-16 8:05 ` [PATCH 3.4 040/107] xhci: prevent bus_suspend if SS port resuming in phase 1 lizf
2016-03-16 8:05 ` [PATCH 3.4 041/107] usb-storage: ignore ZTE MF 823 card reader in mode 0x1225 lizf
2016-03-16 8:05 ` [PATCH 3.4 042/107] tile: use free_bootmem_late() for initrd lizf
2016-03-16 8:05 ` [PATCH 3.4 043/107] Input: usbtouchscreen - avoid unresponsive TSC-30 touch screen lizf
2016-03-16 8:05 ` [PATCH 3.4 044/107] md/raid1: fix test for 'was read error from last working device' lizf
2016-03-16 8:05 ` [PATCH 3.4 045/107] mmc: block: Add missing mmc_blk_put() in power_ro_lock_show() lizf
2016-03-16 8:05 ` [PATCH 3.4 046/107] netfilter: nf_conntrack: Support expectations in different zones lizf
2016-03-16 8:05 ` [PATCH 3.4 047/107] crypto: ixp4xx - Remove bogus BUG_ON on scattered dst buffer lizf
2016-03-16 8:05 ` [PATCH 3.4 048/107] iscsi-target: Fix use-after-free during TPG session shutdown lizf
2016-03-16 8:05 ` [PATCH 3.4 049/107] niu: don't count tx error twice in case of headroom realloc fails lizf
2016-03-16 8:05 ` [PATCH 3.4 050/107] vhost: actually track log eventfd file lizf
2016-03-16 8:05 ` [PATCH 3.4 051/107] USB: sierra: add 1199:68AB device ID lizf
2016-03-16 8:05 ` [PATCH 3.4 052/107] ALSA: usb-audio: add dB range mapping for some devices lizf
2016-03-16 8:05 ` [PATCH 3.4 053/107] drm/radeon/combios: add some validation of lvds values lizf
2016-03-16 8:05 ` [PATCH 3.4 054/107] target/iscsi: Fix double free of a TUR followed by a solicited NOPOUT lizf
2016-03-16 8:05 ` [PATCH 3.4 055/107] x86/xen: Probe target addresses in set_aliased_prot() before the hypercall lizf
2016-03-16 8:05 ` [PATCH 3.4 056/107] x86/ldt: Make modify_ldt synchronous lizf
2016-03-16 8:05 ` [PATCH 3.4 057/107] md/raid1: extend spinlock to protect raid1_end_read_request against inconsistencies lizf
2016-03-16 8:05 ` [PATCH 3.4 058/107] MIPS: Fix sched_getaffinity with MT FPAFF enabled lizf
2016-03-16 8:05 ` [PATCH 3.4 059/107] xhci: fix off by one error in TRB DMA address boundary check lizf
2016-03-16 8:05 ` [PATCH 3.4 060/107] rds: fix an integer overflow test in rds_info_getsockopt() lizf
2016-03-16 8:05 ` [PATCH 3.4 061/107] perf: Fix fasync handling on inherited events lizf
2016-03-16 8:05 ` [PATCH 3.4 062/107] MIPS: Make set_pte() SMP safe lizf
2016-03-16 8:05 ` [PATCH 3.4 063/107] ocfs2: fix BUG in ocfs2_downconvert_thread_do_work() lizf
2016-03-16 8:05 ` [PATCH 3.4 064/107] x86/ldt: Correct LDT access in single stepping logic lizf
2016-03-16 8:05 ` [PATCH 3.4 065/107] x86/ldt: Correct FPU emulation access to LDT lizf
2016-03-16 8:06 ` [PATCH 3.4 066/107] localmodconfig: Use Kbuild files too lizf
2016-03-16 8:06 ` [PATCH 3.4 067/107] dm btree: add ref counting ops for the leaves of top level btrees lizf
2016-03-16 8:06 ` [PATCH 3.4 068/107] libiscsi: Fix host busy blocking during connection teardown lizf
2016-03-16 8:06 ` [PATCH 3.4 069/107] libfc: Fix fc_fcp_cleanup_each_cmd() lizf
2016-03-16 8:06 ` [PATCH 3.4 070/107] EDAC, ppc4xx: Access mci->csrows array elements properly lizf
2016-03-16 8:06 ` [PATCH 3.4 071/107] ipc,sem: fix use after free on IPC_RMID after a task using same semaphore set exits lizf
2016-03-16 8:06 ` [PATCH 3.4 072/107] net: Fix RCU splat in af_key lizf
2016-03-16 8:06 ` [PATCH 3.4 073/107] sctp: donot reset the overall_error_count in SHUTDOWN_RECEIVE state lizf
2016-03-16 8:06 ` [PATCH 3.4 074/107] Revert "usb: dwc3: Reset the transfer resource index on SET_INTERFACE" lizf
2016-03-16 8:06 ` [PATCH 3.4 075/107] unix: avoid use-after-free in ep_remove_wait_queue lizf
2016-03-16 8:06 ` [PATCH 3.4 076/107] pptp: verify sockaddr_len in pptp_bind() and pptp_connect() lizf
2016-03-16 8:06 ` [PATCH 3.4 077/107] net: add validation for the socket syscall protocol argument lizf
2016-03-16 8:06 ` [PATCH 3.4 078/107] RDS: verify the underlying transport exists before creating a connection lizf
2016-03-16 8:06 ` [PATCH 3.4 079/107] RDS: fix race condition when sending a message on unbound socket lizf
2016-03-16 8:06 ` [PATCH 3.4 080/107] sg_start_req(): make sure that there's not too many elements in iovec lizf
2016-03-16 8:06 ` [PATCH 3.4 081/107] virtio-net: drop NETIF_F_FRAGLIST lizf
2016-03-16 8:06 ` [PATCH 3.4 082/107] isdn_ppp: Add checks for allocation failure in isdn_ppp_open() lizf
2016-03-16 8:06 ` [PATCH 3.4 083/107] ppp, slip: Validate VJ compression slot parameters completely lizf
2016-03-16 8:06 ` [PATCH 3.4 084/107] USB: whiteheat: fix potential null-deref at probe lizf
2016-03-16 8:06 ` [PATCH 3.4 085/107] KEYS: Fix race between key destruction and finding a keyring by name lizf
2016-03-16 8:06 ` [PATCH 3.4 086/107] KEYS: Fix crash when attempt to garbage collect an uninstantiated keyring lizf
2016-03-16 8:06 ` [PATCH 3.4 087/107] ipv6: addrconf: validate new MTU before applying it lizf
2016-03-16 8:06 ` [PATCH 3.4 088/107] KVM: x86: work around infinite loop in microcode when #AC is delivered lizf
2016-03-16 8:06 ` [PATCH 3.4 089/107] KVM: svm: unconditionally intercept #DB lizf
2016-03-16 8:06 ` [PATCH 3.4 090/107] get rid of s_files and files_lock lizf
2016-03-16 8:06 ` [PATCH 3.4 091/107] Initialize msg/shm IPC objects before doing ipc_addid() lizf
2016-03-16 8:06 ` [PATCH 3.4 092/107] net: avoid to hang up on sending due to sysctl configuration overflow lizf
2016-03-16 8:06 ` [PATCH 3.4 093/107] ipv6: probe routes asynchronous in rt6_probe lizf
2016-03-16 8:06 ` [PATCH 3.4 094/107] netfilter: nf_conntrack: fix RCU race in nf_conntrack_find_get lizf
2016-03-16 8:06 ` [PATCH 3.4 095/107] atm: deal with setting entry before mkip was called lizf
2016-03-16 8:06 ` [PATCH 3.4 096/107] SUNRPC: never enqueue a ->rq_cong request on ->sending lizf
2016-03-16 8:06 ` [PATCH 3.4 097/107] ipv6: prevent fib6_run_gc() contention lizf
2016-03-16 8:06 ` [PATCH 3.4 098/107] kernel/watchdog.c: touch_nmi_watchdog should only touch local cpu not every one lizf
2016-03-16 14:09 ` Don Zickus
2016-03-17 1:20 ` Zefan Li
2016-03-16 8:06 ` [PATCH 3.4 099/107] net: fix warnings in 'make htmldocs' by moving macro definition out of field declaration lizf
2016-03-16 8:06 ` [PATCH 3.4 100/107] af_unix: Guard against other == sk in unix_dgram_sendmsg lizf
2016-03-16 8:06 ` [PATCH 3.4 101/107] x86/LDT: Print the real LDT base address lizf
2016-03-16 8:06 ` [PATCH 3.4 102/107] ALSA: tlv: compute TLV_*_ITEM lengths automatically lizf
2016-03-16 8:06 ` [PATCH 3.4 103/107] ALSA: tlv: add DECLARE_TLV_DB_RANGE() lizf
2016-03-16 8:06 ` [PATCH 3.4 104/107] ALSA: usb-audio: Add a more accurate volume quirk for AudioQuest DragonFly lizf
2016-03-16 8:06 ` [PATCH 3.4 105/107] usb: dwc3: Fix assignment of EP transfer resources lizf
2016-03-16 8:06 ` [PATCH 3.4 106/107] dm btree remove: fix a bug when rebalancing nodes after removal lizf
2016-03-16 8:06 ` [PATCH 3.4 107/107] KVM: x86: move steal time initialization to vcpu entry time lizf
2016-03-16 17:51 ` [PATCH 3.4 000/107] 3.4.111-rc1 review Guenter Roeck
2016-03-17 1:21 ` Zefan Li
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1458115601-5762-23-git-send-email-lizf@kernel.org \
--to=lizf@kernel.org \
--cc=JBottomley@Odin.com \
--cc=darren.lavender@hp.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan@huawei.com \
--cc=shane.seymour@hp.com \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox