From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org,
Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
Guenter Roeck <linux@roeck-us.net>,
Raul Rangel <rrangel@google.com>,
Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Subject: [PATCH 4.14 68/91] serial: 8250: change lock order in serial8250_do_startup()
Date: Tue, 1 Sep 2020 17:10:42 +0200 [thread overview]
Message-ID: <20200901150931.549274292@linuxfoundation.org> (raw)
In-Reply-To: <20200901150928.096174795@linuxfoundation.org>
From: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
commit 205d300aea75623e1ae4aa43e0d265ab9cf195fd upstream.
We have a number of "uart.port->desc.lock vs desc.lock->uart.port"
lockdep reports coming from 8250 driver; this causes a bit of trouble
to people, so let's fix it.
The problem is reverse lock order in two different call paths:
chain #1:
serial8250_do_startup()
spin_lock_irqsave(&port->lock);
disable_irq_nosync(port->irq);
raw_spin_lock_irqsave(&desc->lock)
chain #2:
__report_bad_irq()
raw_spin_lock_irqsave(&desc->lock)
for_each_action_of_desc()
printk()
spin_lock_irqsave(&port->lock);
Fix this by changing the order of locks in serial8250_do_startup():
do disable_irq_nosync() first, which grabs desc->lock, and grab
uart->port after that, so that chain #1 and chain #2 have same lock
order.
Full lockdep splat:
======================================================
WARNING: possible circular locking dependency detected
5.4.39 #55 Not tainted
======================================================
swapper/0/0 is trying to acquire lock:
ffffffffab65b6c0 (console_owner){-...}, at: console_lock_spinning_enable+0x31/0x57
but task is already holding lock:
ffff88810a8e34c0 (&irq_desc_lock_class){-.-.}, at: __report_bad_irq+0x5b/0xba
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #2 (&irq_desc_lock_class){-.-.}:
_raw_spin_lock_irqsave+0x61/0x8d
__irq_get_desc_lock+0x65/0x89
__disable_irq_nosync+0x3b/0x93
serial8250_do_startup+0x451/0x75c
uart_startup+0x1b4/0x2ff
uart_port_activate+0x73/0xa0
tty_port_open+0xae/0x10a
uart_open+0x1b/0x26
tty_open+0x24d/0x3a0
chrdev_open+0xd5/0x1cc
do_dentry_open+0x299/0x3c8
path_openat+0x434/0x1100
do_filp_open+0x9b/0x10a
do_sys_open+0x15f/0x3d7
kernel_init_freeable+0x157/0x1dd
kernel_init+0xe/0x105
ret_from_fork+0x27/0x50
-> #1 (&port_lock_key){-.-.}:
_raw_spin_lock_irqsave+0x61/0x8d
serial8250_console_write+0xa7/0x2a0
console_unlock+0x3b7/0x528
vprintk_emit+0x111/0x17f
printk+0x59/0x73
register_console+0x336/0x3a4
uart_add_one_port+0x51b/0x5be
serial8250_register_8250_port+0x454/0x55e
dw8250_probe+0x4dc/0x5b9
platform_drv_probe+0x67/0x8b
really_probe+0x14a/0x422
driver_probe_device+0x66/0x130
device_driver_attach+0x42/0x5b
__driver_attach+0xca/0x139
bus_for_each_dev+0x97/0xc9
bus_add_driver+0x12b/0x228
driver_register+0x64/0xed
do_one_initcall+0x20c/0x4a6
do_initcall_level+0xb5/0xc5
do_basic_setup+0x4c/0x58
kernel_init_freeable+0x13f/0x1dd
kernel_init+0xe/0x105
ret_from_fork+0x27/0x50
-> #0 (console_owner){-...}:
__lock_acquire+0x118d/0x2714
lock_acquire+0x203/0x258
console_lock_spinning_enable+0x51/0x57
console_unlock+0x25d/0x528
vprintk_emit+0x111/0x17f
printk+0x59/0x73
__report_bad_irq+0xa3/0xba
note_interrupt+0x19a/0x1d6
handle_irq_event_percpu+0x57/0x79
handle_irq_event+0x36/0x55
handle_fasteoi_irq+0xc2/0x18a
do_IRQ+0xb3/0x157
ret_from_intr+0x0/0x1d
cpuidle_enter_state+0x12f/0x1fd
cpuidle_enter+0x2e/0x3d
do_idle+0x1ce/0x2ce
cpu_startup_entry+0x1d/0x1f
start_kernel+0x406/0x46a
secondary_startup_64+0xa4/0xb0
other info that might help us debug this:
Chain exists of:
console_owner --> &port_lock_key --> &irq_desc_lock_class
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&irq_desc_lock_class);
lock(&port_lock_key);
lock(&irq_desc_lock_class);
lock(console_owner);
*** DEADLOCK ***
2 locks held by swapper/0/0:
#0: ffff88810a8e34c0 (&irq_desc_lock_class){-.-.}, at: __report_bad_irq+0x5b/0xba
#1: ffffffffab65b5c0 (console_lock){+.+.}, at: console_trylock_spinning+0x20/0x181
stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.4.39 #55
Hardware name: XXXXXX
Call Trace:
<IRQ>
dump_stack+0xbf/0x133
? print_circular_bug+0xd6/0xe9
check_noncircular+0x1b9/0x1c3
__lock_acquire+0x118d/0x2714
lock_acquire+0x203/0x258
? console_lock_spinning_enable+0x31/0x57
console_lock_spinning_enable+0x51/0x57
? console_lock_spinning_enable+0x31/0x57
console_unlock+0x25d/0x528
? console_trylock+0x18/0x4e
vprintk_emit+0x111/0x17f
? lock_acquire+0x203/0x258
printk+0x59/0x73
__report_bad_irq+0xa3/0xba
note_interrupt+0x19a/0x1d6
handle_irq_event_percpu+0x57/0x79
handle_irq_event+0x36/0x55
handle_fasteoi_irq+0xc2/0x18a
do_IRQ+0xb3/0x157
common_interrupt+0xf/0xf
</IRQ>
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Fixes: 768aec0b5bcc ("serial: 8250: fix shared interrupts issues with SMP and RT kernels")
Reported-by: Guenter Roeck <linux@roeck-us.net>
Reported-by: Raul Rangel <rrangel@google.com>
BugLink: https://bugs.chromium.org/p/chromium/issues/detail?id=1114800
Link: https://lore.kernel.org/lkml/CAHQZ30BnfX+gxjPm1DUd5psOTqbyDh4EJE=2=VAMW_VDafctkA@mail.gmail.com/T/#u
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: Guenter Roeck <linux@roeck-us.net>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Cc: stable <stable@vger.kernel.org>
Link: https://lore.kernel.org/r/20200817022646.1484638-1-sergey.senozhatsky@gmail.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/tty/serial/8250/8250_port.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
--- a/drivers/tty/serial/8250/8250_port.c
+++ b/drivers/tty/serial/8250/8250_port.c
@@ -2264,6 +2264,10 @@ int serial8250_do_startup(struct uart_po
if (port->irq && !(up->port.flags & UPF_NO_THRE_TEST)) {
unsigned char iir1;
+
+ if (port->irqflags & IRQF_SHARED)
+ disable_irq_nosync(port->irq);
+
/*
* Test for UARTs that do not reassert THRE when the
* transmitter is idle and the interrupt has already
@@ -2273,8 +2277,6 @@ int serial8250_do_startup(struct uart_po
* allow register changes to become visible.
*/
spin_lock_irqsave(&port->lock, flags);
- if (up->port.irqflags & IRQF_SHARED)
- disable_irq_nosync(port->irq);
wait_for_xmitr(up, UART_LSR_THRE);
serial_port_out_sync(port, UART_IER, UART_IER_THRI);
@@ -2286,9 +2288,10 @@ int serial8250_do_startup(struct uart_po
iir = serial_port_in(port, UART_IIR);
serial_port_out(port, UART_IER, 0);
+ spin_unlock_irqrestore(&port->lock, flags);
+
if (port->irqflags & IRQF_SHARED)
enable_irq(port->irq);
- spin_unlock_irqrestore(&port->lock, flags);
/*
* If the interrupt is not reasserted, or we otherwise
next prev parent reply other threads:[~2020-09-01 16:59 UTC|newest]
Thread overview: 95+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-01 15:09 [PATCH 4.14 00/91] 4.14.196-rc1 review Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 01/91] powerpc/64s: Dont init FSCR_DSCR in __init_FSCR() Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 02/91] net: Fix potential wrong skb->protocol in skb_vlan_untag() Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 03/91] tipc: fix uninit skb->data in tipc_nl_compat_dumpit() Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 04/91] ipvlan: fix device features Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 05/91] gre6: Fix reception with IP6_TNL_F_RCV_DSCP_COPY Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 06/91] ALSA: pci: delete repeated words in comments Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 07/91] ASoC: tegra: Fix reference count leaks Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 08/91] mfd: intel-lpss: Add Intel Emmitsburg PCH PCI IDs Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 09/91] arm64: dts: qcom: msm8916: Pull down PDM GPIOs during sleep Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 10/91] powerpc/xive: Ignore kmemleak false positives Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 11/91] media: pci: ttpci: av7110: fix possible buffer overflow caused by bad DMA value in debiirq() Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 12/91] blktrace: ensure our debugfs dir exists Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 13/91] scsi: target: tcmu: Fix crash on ARM during cmd completion Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 14/91] iommu/iova: Dont BUG on invalid PFNs Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 15/91] drm/amdkfd: Fix reference count leaks Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 16/91] drm/radeon: fix multiple reference count leak Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 17/91] drm/amdgpu: fix ref count leak in amdgpu_driver_open_kms Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 18/91] drm/amd/display: fix ref count leak in amdgpu_drm_ioctl Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 19/91] drm/amdgpu: fix ref count leak in amdgpu_display_crtc_set_config Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 20/91] drm/amdgpu/display: fix ref count leak when pm_runtime_get_sync fails Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 21/91] scsi: lpfc: Fix shost refcount mismatch when deleting vport Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 22/91] selftests/powerpc: Purge extra count_pmc() calls of ebb selftests Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 23/91] omapfb: fix multiple reference count leaks due to pm_runtime_get_sync Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 24/91] PCI: Fix pci_create_slot() reference count leak Greg Kroah-Hartman
2020-09-01 15:09 ` [PATCH 4.14 25/91] rtlwifi: rtl8192cu: Prevent leaking urb Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 26/91] mips/vdso: Fix resource leaks in genvdso.c Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 27/91] cec-api: prevent leaking memory through hole in structure Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 28/91] f2fs: fix use-after-free issue Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 29/91] drm/nouveau/drm/noveau: fix reference count leak in nouveau_fbcon_open Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 30/91] drm/nouveau: Fix reference count leak in nouveau_connector_detect Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 31/91] locking/lockdep: Fix overflow in presentation of average lock-time Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 32/91] scsi: iscsi: Do not put host in iscsi_set_flashnode_param() Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 33/91] ceph: fix potential mdsc use-after-free crash Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 34/91] scsi: fcoe: Memory leak fix in fcoe_sysfs_fcf_del() Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 35/91] EDAC/ie31200: Fallback if host bridge device is already initialized Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 36/91] media: davinci: vpif_capture: fix potential double free Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 37/91] KVM: arm64: Fix symbol dependency in __hyp_call_panic_nvhe Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 38/91] powerpc/spufs: add CONFIG_COREDUMP dependency Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 39/91] USB: sisusbvga: Fix a potential UB casued by left shifting a negative value Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 40/91] efi: provide empty efi_enter_virtual_mode implementation Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 41/91] Revert "ath10k: fix DMA related firmware crashes on multiple devices" Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 42/91] media: gpio-ir-tx: improve precision of transmitted signal due to scheduling Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 43/91] nvme-fc: Fix wrong return value in __nvme_fc_init_request() Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 44/91] null_blk: fix passing of REQ_FUA flag in null_handle_rq Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 45/91] i2c: rcar: in slave mode, clear NACK earlier Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 46/91] usb: gadget: f_tcm: Fix some resource leaks in some error paths Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 47/91] jbd2: make sure jh have b_transaction set in refile/unfile_buffer Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 48/91] ext4: dont BUG on inconsistent journal feature Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 49/91] jbd2: abort journal if free a async write error metadata buffer Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 50/91] fs: prevent BUG_ON in submit_bh_wbc() Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 51/91] spi: stm32: fix stm32_spi_prepare_mbr in case of odd clk_rate Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 52/91] s390/cio: add cond_resched() in the slow_eval_known_fn() loop Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 53/91] scsi: ufs: Fix possible infinite loop in ufshcd_hold Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 54/91] scsi: ufs: Improve interrupt handling for shared interrupts Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 55/91] scsi: ufs: Clean up completed request without interrupt notification Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 56/91] net: gianfar: Add of_node_put() before goto statement Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 57/91] powerpc/perf: Fix soft lockups due to missed interrupt accounting Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 58/91] HID: i2c-hid: Always sleep 60ms after I2C_HID_PWR_ON commands Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 59/91] btrfs: fix space cache memory leak after transaction abort Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 60/91] fbcon: prevent user font height or width change from causing potential out-of-bounds access Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 61/91] USB: lvtest: return proper error code in probe Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 62/91] vt: defer kfree() of vc_screenbuf in vc_do_resize() Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 63/91] vt_ioctl: change VT_RESIZEX ioctl to check for error return from vc_resize() Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 64/91] serial: samsung: Removes the IRQ not found warning Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 65/91] serial: pl011: Fix oops on -EPROBE_DEFER Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 66/91] serial: pl011: Dont leak amba_ports entry on driver register error Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 67/91] serial: 8250_exar: Fix number of ports for Commtech PCIe cards Greg Kroah-Hartman
2020-09-01 15:10 ` Greg Kroah-Hartman [this message]
2020-09-01 15:10 ` [PATCH 4.14 69/91] writeback: Protect inode->i_io_list with inode->i_lock Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 70/91] writeback: Avoid skipping inode writeback Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 71/91] writeback: Fix sync livelock due to b_dirty_time processing Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 72/91] XEN uses irqdesc::irq_data_common::handler_data to store a per interrupt XEN data pointer which contains XEN specific information Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 73/91] xhci: Do warm-reset when both CAS and XDEV_RESUME are set Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 74/91] PM: sleep: core: Fix the handling of pending runtime resume requests Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 75/91] device property: Fix the secondary firmware node handling in set_primary_fwnode() Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 76/91] drm/amdgpu: Fix buffer overflow in INFO ioctl Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 77/91] USB: yurex: Fix bad gfp argument Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 78/91] usb: uas: Add quirk for PNY Pro Elite Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 79/91] USB: quirks: Add no-lpm quirk for another Raydium touchscreen Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 80/91] USB: Ignore UAS for JMicron JMS567 ATA/ATAPI Bridge Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 81/91] usb: host: ohci-exynos: Fix error handling in exynos_ohci_probe() Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 82/91] overflow.h: Add allocation size calculation helpers Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 83/91] USB: gadget: u_f: add overflow checks to VLA macros Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 84/91] USB: gadget: f_ncm: add bounds checks to ncm_unwrap_ntb() Greg Kroah-Hartman
2020-09-01 15:10 ` [PATCH 4.14 85/91] USB: gadget: u_f: Unbreak offset calculation in VLAs Greg Kroah-Hartman
2020-09-01 15:11 ` [PATCH 4.14 86/91] USB: cdc-acm: rework notification_buffer resizing Greg Kroah-Hartman
2020-09-01 15:11 ` [PATCH 4.14 87/91] usb: storage: Add unusual_uas entry for Sony PSZ drives Greg Kroah-Hartman
2020-09-01 15:11 ` [PATCH 4.14 88/91] btrfs: check the right error variable in btrfs_del_dir_entries_in_log Greg Kroah-Hartman
2020-09-01 15:11 ` [PATCH 4.14 89/91] tpm: Unify the mismatching TPM space buffer sizes Greg Kroah-Hartman
2020-09-01 15:11 ` [PATCH 4.14 90/91] HID: hiddev: Fix slab-out-of-bounds write in hiddev_ioctl_usage() Greg Kroah-Hartman
2020-09-01 15:11 ` [PATCH 4.14 91/91] ALSA: usb-audio: Update documentation comment for MS2109 quirk Greg Kroah-Hartman
2020-09-01 22:22 ` [PATCH 4.14 00/91] 4.14.196-rc1 review Shuah Khan
2020-09-02 7:13 ` Naresh Kamboju
2020-09-02 16:46 ` Guenter Roeck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200901150931.549274292@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=andriy.shevchenko@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@roeck-us.net \
--cc=rrangel@google.com \
--cc=sergey.senozhatsky@gmail.com \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox