From: syzbot <syzbot+51244a05705883616c95@syzkaller.appspotmail.com>
To: jlbec@evilplan.org, joseph.qi@linux.alibaba.com,
linux-kernel@vger.kernel.org, mark@fasheh.com,
ocfs2-devel@lists.linux.dev, syzkaller-bugs@googlegroups.com
Subject: Re: [syzbot] [ocfs2?] possible deadlock in ocfs2_acquire_dquot
Date: Thu, 24 Oct 2024 18:08:25 -0700 [thread overview]
Message-ID: <671aef89.050a0220.381c35.0006.GAE@google.com> (raw)
In-Reply-To: <66fa0328.050a0220.6bad9.002c.GAE@google.com>
syzbot has found a reproducer for the following issue on:
HEAD commit: ce8e69898653 Merge remote-tracking branch 'tip/irq/core' i..
git tree: git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci
console output: https://syzkaller.appspot.com/x/log.txt?x=12bfe65f980000
kernel config: https://syzkaller.appspot.com/x/.config?x=cf20d4aad28662c6
dashboard link: https://syzkaller.appspot.com/bug?extid=51244a05705883616c95
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
userspace arch: arm64
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=15ca5e40580000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=14583287980000
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/c4140162bfaa/disk-ce8e6989.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/9cc02a082c01/vmlinux-ce8e6989.xz
kernel image: https://storage.googleapis.com/syzbot-assets/4ca0d2cc7b00/Image-ce8e6989.gz.xz
mounted in repro #1: https://storage.googleapis.com/syzbot-assets/fae7f4b73536/mount_0.gz
mounted in repro #2: https://storage.googleapis.com/syzbot-assets/60abe9617ec0/mount_3.gz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+51244a05705883616c95@syzkaller.appspotmail.com
======================================================
WARNING: possible circular locking dependency detected
6.12.0-rc4-syzkaller-gce8e69898653 #0 Not tainted
------------------------------------------------------
syz-executor217/6496 is trying to acquire lock:
ffff0000d451e610 (sb_internal#2){.+.+}-{0:0}, at: ocfs2_acquire_dquot+0x3c4/0xa8c fs/ocfs2/quota_global.c:855
but task is already holding lock:
ffff0000e202dbe0 (&ocfs2_quota_ip_alloc_sem_key){++++}-{3:3}, at: ocfs2_lock_global_qf+0x1d4/0x260 fs/ocfs2/quota_global.c:314
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #6 (&ocfs2_quota_ip_alloc_sem_key){++++}-{3:3}:
down_write+0x50/0xc0 kernel/locking/rwsem.c:1577
ocfs2_lock_global_qf+0x1d4/0x260 fs/ocfs2/quota_global.c:314
ocfs2_acquire_dquot+0x268/0xa8c fs/ocfs2/quota_global.c:823
dqget+0x660/0xcec fs/quota/dquot.c:976
__dquot_initialize+0x344/0xc04 fs/quota/dquot.c:1504
dquot_initialize+0x24/0x34 fs/quota/dquot.c:1566
ocfs2_get_init_inode+0x14c/0x1b8 fs/ocfs2/namei.c:202
ocfs2_mknod+0x878/0x243c fs/ocfs2/namei.c:308
ocfs2_create+0x194/0x4e0 fs/ocfs2/namei.c:672
lookup_open fs/namei.c:3595 [inline]
open_last_lookups fs/namei.c:3694 [inline]
path_openat+0x13e4/0x2b14 fs/namei.c:3930
do_filp_open+0x1bc/0x3cc fs/namei.c:3960
do_sys_openat2+0x124/0x1b8 fs/open.c:1415
do_sys_open fs/open.c:1430 [inline]
__do_sys_openat fs/open.c:1446 [inline]
__se_sys_openat fs/open.c:1441 [inline]
__arm64_sys_openat+0x1f0/0x240 fs/open.c:1441
__invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49
el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132
do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151
el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:744
el0t_64_sync_handler+0x84/0x108 arch/arm64/kernel/entry-common.c:762
el0t_64_sync+0x198/0x19c arch/arm64/kernel/entry.S:600
-> #5 (&ocfs2_sysfile_lock_key[args->fi_sysfile_type]#3){+.+.}-{3:3}:
down_write+0x50/0xc0 kernel/locking/rwsem.c:1577
inode_lock include/linux/fs.h:815 [inline]
ocfs2_lock_global_qf+0x1b8/0x260 fs/ocfs2/quota_global.c:313
ocfs2_acquire_dquot+0x268/0xa8c fs/ocfs2/quota_global.c:823
dqget+0x660/0xcec fs/quota/dquot.c:976
__dquot_initialize+0x344/0xc04 fs/quota/dquot.c:1504
dquot_initialize+0x24/0x34 fs/quota/dquot.c:1566
ocfs2_get_init_inode+0x14c/0x1b8 fs/ocfs2/namei.c:202
ocfs2_mknod+0x878/0x243c fs/ocfs2/namei.c:308
ocfs2_create+0x194/0x4e0 fs/ocfs2/namei.c:672
lookup_open fs/namei.c:3595 [inline]
open_last_lookups fs/namei.c:3694 [inline]
path_openat+0x13e4/0x2b14 fs/namei.c:3930
do_filp_open+0x1bc/0x3cc fs/namei.c:3960
do_sys_openat2+0x124/0x1b8 fs/open.c:1415
do_sys_open fs/open.c:1430 [inline]
__do_sys_openat fs/open.c:1446 [inline]
__se_sys_openat fs/open.c:1441 [inline]
__arm64_sys_openat+0x1f0/0x240 fs/open.c:1441
__invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49
el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132
do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151
el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:744
el0t_64_sync_handler+0x84/0x108 arch/arm64/kernel/entry-common.c:762
el0t_64_sync+0x198/0x19c arch/arm64/kernel/entry.S:600
-> #4 (&dquot->dq_lock){+.+.}-{3:3}:
__mutex_lock_common+0x190/0x21a0 kernel/locking/mutex.c:608
__mutex_lock kernel/locking/mutex.c:752 [inline]
mutex_lock_nested+0x2c/0x38 kernel/locking/mutex.c:804
wait_on_dquot fs/quota/dquot.c:355 [inline]
dqget+0x5f0/0xcec fs/quota/dquot.c:971
dquot_transfer+0x3a8/0x5c0 fs/quota/dquot.c:2153
ext4_setattr+0x8bc/0x1624 fs/ext4/inode.c:5368
notify_change+0x9f0/0xca0 fs/attr.c:503
chown_common+0x438/0x700 fs/open.c:793
vfs_fchown fs/open.c:861 [inline]
ksys_fchown+0xe0/0x158 fs/open.c:872
__do_sys_fchown fs/open.c:880 [inline]
__se_sys_fchown fs/open.c:878 [inline]
__arm64_sys_fchown+0x7c/0x94 fs/open.c:878
__invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49
el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132
do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151
el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:744
el0t_64_sync_handler+0x84/0x108 arch/arm64/kernel/entry-common.c:762
el0t_64_sync+0x198/0x19c arch/arm64/kernel/entry.S:600
-> #3 (&ei->xattr_sem){++++}-{3:3}:
down_write+0x50/0xc0 kernel/locking/rwsem.c:1577
ext4_write_lock_xattr fs/ext4/xattr.h:154 [inline]
ext4_xattr_set_handle+0x1dc/0x12d0 fs/ext4/xattr.c:2373
ext4_initxattrs+0xa4/0x11c fs/ext4/xattr_security.c:44
security_inode_init_security+0x73c/0x908 security/security.c:1848
ext4_init_security+0x44/0x58 fs/ext4/xattr_security.c:58
__ext4_new_inode+0x2be8/0x3830 fs/ext4/ialloc.c:1323
ext4_create+0x234/0x480 fs/ext4/namei.c:2834
lookup_open fs/namei.c:3595 [inline]
open_last_lookups fs/namei.c:3694 [inline]
path_openat+0x13e4/0x2b14 fs/namei.c:3930
do_filp_open+0x1bc/0x3cc fs/namei.c:3960
do_sys_openat2+0x124/0x1b8 fs/open.c:1415
do_sys_open fs/open.c:1430 [inline]
__do_sys_openat fs/open.c:1446 [inline]
__se_sys_openat fs/open.c:1441 [inline]
__arm64_sys_openat+0x1f0/0x240 fs/open.c:1441
__invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49
el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132
do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151
el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:744
el0t_64_sync_handler+0x84/0x108 arch/arm64/kernel/entry-common.c:762
el0t_64_sync+0x198/0x19c arch/arm64/kernel/entry.S:600
-> #2 (jbd2_handle){.+.+}-{0:0}:
start_this_handle+0xf34/0x11c4 fs/jbd2/transaction.c:448
jbd2__journal_start+0x298/0x544 fs/jbd2/transaction.c:505
jbd2_journal_start+0x3c/0x4c fs/jbd2/transaction.c:544
ocfs2_start_trans+0x3d0/0x71c fs/ocfs2/journal.c:352
ocfs2_modify_bh+0xe4/0x484 fs/ocfs2/quota_local.c:101
ocfs2_local_read_info+0x1220/0x15bc fs/ocfs2/quota_local.c:771
dquot_load_quota_sb+0x6e4/0xb24 fs/quota/dquot.c:2458
dquot_load_quota_inode+0x280/0x4f4 fs/quota/dquot.c:2495
ocfs2_enable_quotas+0x17c/0x3cc fs/ocfs2/super.c:926
ocfs2_fill_super+0x3f04/0x49d0 fs/ocfs2/super.c:1141
mount_bdev+0x1d4/0x2a0 fs/super.c:1679
ocfs2_mount+0x44/0x58 fs/ocfs2/super.c:1188
legacy_get_tree+0xd4/0x16c fs/fs_context.c:662
vfs_get_tree+0x90/0x28c fs/super.c:1800
do_new_mount+0x278/0x900 fs/namespace.c:3507
path_mount+0x590/0xe04 fs/namespace.c:3834
do_mount fs/namespace.c:3847 [inline]
__do_sys_mount fs/namespace.c:4055 [inline]
__se_sys_mount fs/namespace.c:4032 [inline]
__arm64_sys_mount+0x45c/0x5a8 fs/namespace.c:4032
__invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49
el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132
do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151
el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:744
el0t_64_sync_handler+0x84/0x108 arch/arm64/kernel/entry-common.c:762
el0t_64_sync+0x198/0x19c arch/arm64/kernel/entry.S:600
-> #1 (&journal->j_trans_barrier){.+.+}-{3:3}:
down_read+0x58/0x2fc kernel/locking/rwsem.c:1524
ocfs2_start_trans+0x3c4/0x71c fs/ocfs2/journal.c:350
ocfs2_modify_bh+0xe4/0x484 fs/ocfs2/quota_local.c:101
ocfs2_local_read_info+0x1220/0x15bc fs/ocfs2/quota_local.c:771
dquot_load_quota_sb+0x6e4/0xb24 fs/quota/dquot.c:2458
dquot_load_quota_inode+0x280/0x4f4 fs/quota/dquot.c:2495
ocfs2_enable_quotas+0x17c/0x3cc fs/ocfs2/super.c:926
ocfs2_fill_super+0x3f04/0x49d0 fs/ocfs2/super.c:1141
mount_bdev+0x1d4/0x2a0 fs/super.c:1679
ocfs2_mount+0x44/0x58 fs/ocfs2/super.c:1188
legacy_get_tree+0xd4/0x16c fs/fs_context.c:662
vfs_get_tree+0x90/0x28c fs/super.c:1800
do_new_mount+0x278/0x900 fs/namespace.c:3507
path_mount+0x590/0xe04 fs/namespace.c:3834
do_mount fs/namespace.c:3847 [inline]
__do_sys_mount fs/namespace.c:4055 [inline]
__se_sys_mount fs/namespace.c:4032 [inline]
__arm64_sys_mount+0x45c/0x5a8 fs/namespace.c:4032
__invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49
el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132
do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151
el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:744
el0t_64_sync_handler+0x84/0x108 arch/arm64/kernel/entry-common.c:762
el0t_64_sync+0x198/0x19c arch/arm64/kernel/entry.S:600
-> #0 (sb_internal#2){.+.+}-{0:0}:
check_prev_add kernel/locking/lockdep.c:3161 [inline]
check_prevs_add kernel/locking/lockdep.c:3280 [inline]
validate_chain kernel/locking/lockdep.c:3904 [inline]
__lock_acquire+0x33f8/0x77c8 kernel/locking/lockdep.c:5202
lock_acquire+0x240/0x728 kernel/locking/lockdep.c:5825
percpu_down_read include/linux/percpu-rwsem.h:51 [inline]
__sb_start_write include/linux/fs.h:1716 [inline]
sb_start_intwrite include/linux/fs.h:1899 [inline]
ocfs2_start_trans+0x244/0x71c fs/ocfs2/journal.c:348
ocfs2_acquire_dquot+0x3c4/0xa8c fs/ocfs2/quota_global.c:855
dqget+0x660/0xcec fs/quota/dquot.c:976
__dquot_initialize+0x344/0xc04 fs/quota/dquot.c:1504
dquot_initialize+0x24/0x34 fs/quota/dquot.c:1566
ocfs2_get_init_inode+0x14c/0x1b8 fs/ocfs2/namei.c:202
ocfs2_mknod+0x878/0x243c fs/ocfs2/namei.c:308
ocfs2_create+0x194/0x4e0 fs/ocfs2/namei.c:672
lookup_open fs/namei.c:3595 [inline]
open_last_lookups fs/namei.c:3694 [inline]
path_openat+0x13e4/0x2b14 fs/namei.c:3930
do_filp_open+0x1bc/0x3cc fs/namei.c:3960
do_sys_openat2+0x124/0x1b8 fs/open.c:1415
do_sys_open fs/open.c:1430 [inline]
__do_sys_openat fs/open.c:1446 [inline]
__se_sys_openat fs/open.c:1441 [inline]
__arm64_sys_openat+0x1f0/0x240 fs/open.c:1441
__invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49
el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132
do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151
el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:744
el0t_64_sync_handler+0x84/0x108 arch/arm64/kernel/entry-common.c:762
el0t_64_sync+0x198/0x19c arch/arm64/kernel/entry.S:600
other info that might help us debug this:
Chain exists of:
sb_internal#2 --> &ocfs2_sysfile_lock_key[args->fi_sysfile_type]#3 --> &ocfs2_quota_ip_alloc_sem_key
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&ocfs2_quota_ip_alloc_sem_key);
lock(&ocfs2_sysfile_lock_key[args->fi_sysfile_type]#3);
lock(&ocfs2_quota_ip_alloc_sem_key);
rlock(sb_internal#2);
*** DEADLOCK ***
6 locks held by syz-executor217/6496:
#0: ffff0000d451e420 (sb_writers#10){.+.+}-{0:0}, at: mnt_want_write+0x44/0x9c fs/namespace.c:515
#1: ffff0000e20289c0 (&type->i_mutex_dir_key#8){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:815 [inline]
#1: ffff0000e20289c0 (&type->i_mutex_dir_key#8){+.+.}-{3:3}, at: open_last_lookups fs/namei.c:3691 [inline]
#1: ffff0000e20289c0 (&type->i_mutex_dir_key#8){+.+.}-{3:3}, at: path_openat+0x684/0x2b14 fs/namei.c:3930
#2: ffff0000e21a9800 (&ocfs2_sysfile_lock_key[args->fi_sysfile_type]#2){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:815 [inline]
#2: ffff0000e21a9800 (&ocfs2_sysfile_lock_key[args->fi_sysfile_type]#2){+.+.}-{3:3}, at: ocfs2_reserve_suballoc_bits+0x164/0x4288 fs/ocfs2/suballoc.c:786
#3: ffff0000e20400a8 (&dquot->dq_lock){+.+.}-{3:3}, at: ocfs2_acquire_dquot+0x25c/0xa8c fs/ocfs2/quota_global.c:818
#4: ffff0000e202df40 (&ocfs2_sysfile_lock_key[args->fi_sysfile_type]#3){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:815 [inline]
#4: ffff0000e202df40 (&ocfs2_sysfile_lock_key[args->fi_sysfile_type]#3){+.+.}-{3:3}, at: ocfs2_lock_global_qf+0x1b8/0x260 fs/ocfs2/quota_global.c:313
#5: ffff0000e202dbe0 (&ocfs2_quota_ip_alloc_sem_key){++++}-{3:3}, at: ocfs2_lock_global_qf+0x1d4/0x260 fs/ocfs2/quota_global.c:314
stack backtrace:
CPU: 1 UID: 0 PID: 6496 Comm: syz-executor217 Not tainted 6.12.0-rc4-syzkaller-gce8e69898653 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call trace:
show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:484 (C)
__dump_stack lib/dump_stack.c:94 [inline]
dump_stack_lvl+0xe4/0x150 lib/dump_stack.c:120
dump_stack+0x1c/0x28 lib/dump_stack.c:129
print_circular_bug+0x154/0x1c0 kernel/locking/lockdep.c:2074
check_noncircular+0x310/0x404 kernel/locking/lockdep.c:2206
check_prev_add kernel/locking/lockdep.c:3161 [inline]
check_prevs_add kernel/locking/lockdep.c:3280 [inline]
validate_chain kernel/locking/lockdep.c:3904 [inline]
__lock_acquire+0x33f8/0x77c8 kernel/locking/lockdep.c:5202
lock_acquire+0x240/0x728 kernel/locking/lockdep.c:5825
percpu_down_read include/linux/percpu-rwsem.h:51 [inline]
__sb_start_write include/linux/fs.h:1716 [inline]
sb_start_intwrite include/linux/fs.h:1899 [inline]
ocfs2_start_trans+0x244/0x71c fs/ocfs2/journal.c:348
ocfs2_acquire_dquot+0x3c4/0xa8c fs/ocfs2/quota_global.c:855
dqget+0x660/0xcec fs/quota/dquot.c:976
__dquot_initialize+0x344/0xc04 fs/quota/dquot.c:1504
dquot_initialize+0x24/0x34 fs/quota/dquot.c:1566
ocfs2_get_init_inode+0x14c/0x1b8 fs/ocfs2/namei.c:202
ocfs2_mknod+0x878/0x243c fs/ocfs2/namei.c:308
ocfs2_create+0x194/0x4e0 fs/ocfs2/namei.c:672
lookup_open fs/namei.c:3595 [inline]
open_last_lookups fs/namei.c:3694 [inline]
path_openat+0x13e4/0x2b14 fs/namei.c:3930
do_filp_open+0x1bc/0x3cc fs/namei.c:3960
do_sys_openat2+0x124/0x1b8 fs/open.c:1415
do_sys_open fs/open.c:1430 [inline]
__do_sys_openat fs/open.c:1446 [inline]
__se_sys_openat fs/open.c:1441 [inline]
__arm64_sys_openat+0x1f0/0x240 fs/open.c:1441
__invoke_syscall arch/arm64/kernel/syscall.c:35 [inline]
invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49
el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132
do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151
el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:744
el0t_64_sync_handler+0x84/0x108 arch/arm64/kernel/entry-common.c:762
el0t_64_sync+0x198/0x19c arch/arm64/kernel/entry.S:600
---
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.
next prev parent reply other threads:[~2024-10-25 1:08 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-09-30 1:47 [syzbot] [ocfs2?] possible deadlock in ocfs2_acquire_dquot syzbot
2024-10-25 1:08 ` syzbot [this message]
2025-12-27 16:44 ` Forwarded: OCFS2 fix test syzbot
[not found] <CAJRoVwhAjDOoaFERGYdpiMTHgAu2=9H27Yjb1LbXtGtN80rXkw@mail.gmail.com>
2025-12-27 17:35 ` [syzbot] [ocfs2?] possible deadlock in ocfs2_acquire_dquot syzbot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=671aef89.050a0220.381c35.0006.GAE@google.com \
--to=syzbot+51244a05705883616c95@syzkaller.appspotmail.com \
--cc=jlbec@evilplan.org \
--cc=joseph.qi@linux.alibaba.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mark@fasheh.com \
--cc=ocfs2-devel@lists.linux.dev \
--cc=syzkaller-bugs@googlegroups.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.