From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Greg KH <gregkh@linuxfoundation.org>,
Ben Hutchings <bhutchings@solarflare.com>,
Bruce Allan <bruce.w.allan@intel.com>,
Aaron Brown <aaron.f.brown@intel.com>,
Jeff Kirsher <jeffrey.t.kirsher@intel.com>,
"David S. Miller" <davem@davemloft.net>
Subject: [ 100/108] e1000e: DoS while TSO enabled caused by link partner with small MSS
Date: Wed, 12 Sep 2012 16:29:38 -0700 [thread overview]
Message-ID: <20120912232500.985321825@linuxfoundation.org> (raw)
In-Reply-To: <20120912232450.500619493@linuxfoundation.org>
From: Greg KH <gregkh@linuxfoundation.org>
3.5-stable review patch. If anyone has any objections, please let me know.
------------------
From: Bruce Allan <bruce.w.allan@intel.com>
commit d821a4c4d11ad160925dab2bb009b8444beff484 upstream.
With a low enough MSS on the link partner and TSO enabled locally, the
networking stack can periodically send a very large (e.g. 64KB) TCP
message for which the driver will attempt to use more Tx descriptors than
are available by default in the Tx ring. This is due to a workaround in
the code that imposes a limit of only 4 MSS-sized segments per descriptor
which appears to be a carry-over from the older e1000 driver and may be
applicable only to some older PCI or PCIx parts which are not supported in
e1000e. When the driver gets a message that is too large to fit across the
configured number of Tx descriptors, it stops the upper stack from queueing
any more and gets stuck in this state. After a timeout, the upper stack
assumes the adapter is hung and calls the driver to reset it.
Remove the unnecessary limitation of using up to only 4 MSS-sized segments
per Tx descriptor, and put in a hard failure test to catch when attempting
to check for message sizes larger than would fit in the whole Tx ring.
Refactor the remaining logic that limits the size of data per Tx descriptor
from a seemingly arbitrary 8KB to a limit based on the dynamic size of the
Tx packet buffer as described in the hardware specification.
Also, fix the logic in the check for space in the Tx ring for the next
largest possible packet after the current one has been successfully queued
for transmit, and use the appropriate defines for default ring sizes in
e1000_probe instead of magic values.
This issue goes back to the introduction of e1000e in 2.6.24 when it was
split off from e1000.
Reported-by: Ben Hutchings <bhutchings@solarflare.com>
Signed-off-by: Bruce Allan <bruce.w.allan@intel.com>
Tested-by: Aaron Brown <aaron.f.brown@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/net/ethernet/intel/e1000e/e1000.h | 1
drivers/net/ethernet/intel/e1000e/netdev.c | 48 +++++++++++++----------------
2 files changed, 24 insertions(+), 25 deletions(-)
--- a/drivers/net/ethernet/intel/e1000e/e1000.h
+++ b/drivers/net/ethernet/intel/e1000e/e1000.h
@@ -310,6 +310,7 @@ struct e1000_adapter {
*/
struct e1000_ring *tx_ring /* One per active queue */
____cacheline_aligned_in_smp;
+ u32 tx_fifo_limit;
struct napi_struct napi;
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -3477,6 +3477,15 @@ void e1000e_reset(struct e1000_adapter *
}
/*
+ * Alignment of Tx data is on an arbitrary byte boundary with the
+ * maximum size per Tx descriptor limited only to the transmit
+ * allocation of the packet buffer minus 96 bytes with an upper
+ * limit of 24KB due to receive synchronization limitations.
+ */
+ adapter->tx_fifo_limit = min_t(u32, ((er32(PBA) >> 16) << 10) - 96,
+ 24 << 10);
+
+ /*
* Disable Adaptive Interrupt Moderation if 2 full packets cannot
* fit in receive buffer.
*/
@@ -4745,12 +4754,9 @@ static bool e1000_tx_csum(struct e1000_r
return 1;
}
-#define E1000_MAX_PER_TXD 8192
-#define E1000_MAX_TXD_PWR 12
-
static int e1000_tx_map(struct e1000_ring *tx_ring, struct sk_buff *skb,
unsigned int first, unsigned int max_per_txd,
- unsigned int nr_frags, unsigned int mss)
+ unsigned int nr_frags)
{
struct e1000_adapter *adapter = tx_ring->adapter;
struct pci_dev *pdev = adapter->pdev;
@@ -4983,20 +4989,19 @@ static int __e1000_maybe_stop_tx(struct
static int e1000_maybe_stop_tx(struct e1000_ring *tx_ring, int size)
{
+ BUG_ON(size > tx_ring->count);
+
if (e1000_desc_unused(tx_ring) >= size)
return 0;
return __e1000_maybe_stop_tx(tx_ring, size);
}
-#define TXD_USE_COUNT(S, X) (((S) >> (X)) + 1)
static netdev_tx_t e1000_xmit_frame(struct sk_buff *skb,
struct net_device *netdev)
{
struct e1000_adapter *adapter = netdev_priv(netdev);
struct e1000_ring *tx_ring = adapter->tx_ring;
unsigned int first;
- unsigned int max_per_txd = E1000_MAX_PER_TXD;
- unsigned int max_txd_pwr = E1000_MAX_TXD_PWR;
unsigned int tx_flags = 0;
unsigned int len = skb_headlen(skb);
unsigned int nr_frags;
@@ -5016,18 +5021,8 @@ static netdev_tx_t e1000_xmit_frame(stru
}
mss = skb_shinfo(skb)->gso_size;
- /*
- * The controller does a simple calculation to
- * make sure there is enough room in the FIFO before
- * initiating the DMA for each buffer. The calc is:
- * 4 = ceil(buffer len/mss). To make sure we don't
- * overrun the FIFO, adjust the max buffer len if mss
- * drops.
- */
if (mss) {
u8 hdr_len;
- max_per_txd = min(mss << 2, max_per_txd);
- max_txd_pwr = fls(max_per_txd) - 1;
/*
* TSO Workaround for 82571/2/3 Controllers -- if skb->data
@@ -5057,12 +5052,12 @@ static netdev_tx_t e1000_xmit_frame(stru
count++;
count++;
- count += TXD_USE_COUNT(len, max_txd_pwr);
+ count += DIV_ROUND_UP(len, adapter->tx_fifo_limit);
nr_frags = skb_shinfo(skb)->nr_frags;
for (f = 0; f < nr_frags; f++)
- count += TXD_USE_COUNT(skb_frag_size(&skb_shinfo(skb)->frags[f]),
- max_txd_pwr);
+ count += DIV_ROUND_UP(skb_frag_size(&skb_shinfo(skb)->frags[f]),
+ adapter->tx_fifo_limit);
if (adapter->hw.mac.tx_pkt_filtering)
e1000_transfer_dhcp_info(adapter, skb);
@@ -5104,15 +5099,18 @@ static netdev_tx_t e1000_xmit_frame(stru
tx_flags |= E1000_TX_FLAGS_NO_FCS;
/* if count is 0 then mapping error has occurred */
- count = e1000_tx_map(tx_ring, skb, first, max_per_txd, nr_frags, mss);
+ count = e1000_tx_map(tx_ring, skb, first, adapter->tx_fifo_limit,
+ nr_frags);
if (count) {
skb_tx_timestamp(skb);
netdev_sent_queue(netdev, skb->len);
e1000_tx_queue(tx_ring, tx_flags, count);
/* Make sure there is space in the ring for the next send. */
- e1000_maybe_stop_tx(tx_ring, MAX_SKB_FRAGS + 2);
-
+ e1000_maybe_stop_tx(tx_ring,
+ (MAX_SKB_FRAGS *
+ DIV_ROUND_UP(PAGE_SIZE,
+ adapter->tx_fifo_limit) + 2));
} else {
dev_kfree_skb_any(skb);
tx_ring->buffer_info[first].time_stamp = 0;
@@ -6284,8 +6282,8 @@ static int __devinit e1000_probe(struct
adapter->hw.phy.autoneg_advertised = 0x2f;
/* ring size defaults */
- adapter->rx_ring->count = 256;
- adapter->tx_ring->count = 256;
+ adapter->rx_ring->count = E1000_DEFAULT_RXD;
+ adapter->tx_ring->count = E1000_DEFAULT_TXD;
/*
* Initial Wake on LAN setting - If APM wake is enabled in
next prev parent reply other threads:[~2012-09-13 0:20 UTC|newest]
Thread overview: 112+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-09-12 23:28 [ 000/108] 3.5.4-stable review Greg KH
2012-09-12 23:27 ` [ 001/108] USB: vt6656: remove __devinit* from the struct usb_device_id table Greg Kroah-Hartman
2012-09-12 23:28 ` [ 002/108] USB: emi62: " Greg Kroah-Hartman
2012-09-12 23:28 ` [ 003/108] ALSA: hda - fix Copyright debug message Greg Kroah-Hartman
2012-09-12 23:28 ` [ 004/108] ARM: 7483/1: vfp: only advertise VFPv4 in hwcaps if CONFIG_VFPv3 is enabled Greg Kroah-Hartman
2012-09-12 23:28 ` [ 005/108] ARM: 7487/1: mm: avoid setting nG bit for user mappings that arent present Greg Kroah-Hartman
2012-09-12 23:28 ` [ 006/108] ARM: 7488/1: mm: use 5 bits for swapfile type encoding Greg Kroah-Hartman
2012-09-12 23:28 ` [ 007/108] ARM: 7489/1: errata: fix workaround for erratum #720789 on UP systems Greg Kroah-Hartman
2012-09-12 23:28 ` [ 008/108] ARM: OMAP2+: Fix dmtimer set source clock failure Greg Kroah-Hartman
2012-09-12 23:28 ` [ 009/108] ARM: S3C24XX: Add missing DMACH_DT_PROP Greg Kroah-Hartman
2012-09-12 23:28 ` [ 010/108] ARM: S3C24XX: Fix s3c2410_dma_enqueue parameters Greg Kroah-Hartman
2012-09-12 23:28 ` [ 011/108] xen/p2m: Reuse existing P2M leafs if they are filled with 1:1 PFNs or INVALID Greg Kroah-Hartman
2012-09-12 23:28 ` [ 012/108] xen/setup: Fix one-off error when adding for-balloon PFNs to the P2M Greg Kroah-Hartman
2012-09-12 23:28 ` [ 013/108] ARM: imx6: spin the cpu until hardware takes it down Greg Kroah-Hartman
2012-09-12 23:28 ` [ 014/108] ARM: imx: build pm-imx5 code only when PM is enabled Greg Kroah-Hartman
2012-09-12 23:28 ` [ 015/108] ARM: imx: select CPU_FREQ_TABLE when needed Greg Kroah-Hartman
2012-09-12 23:28 ` [ 016/108] ARM: imx: build i.MX6 functions only " Greg Kroah-Hartman
2012-09-12 23:28 ` [ 017/108] MIPS: pci-ar724x: avoid data bus error due to a missing PCIe module Greg Kroah-Hartman
2012-09-12 23:28 ` [ 018/108] ASoC: wm9712: Fix microphone source selection Greg Kroah-Hartman
2012-09-12 23:28 ` [ 019/108] ASoC: omap-mcbsp: Fix 6pin mux configuration Greg Kroah-Hartman
2012-09-12 23:28 ` [ 020/108] vfs: missed source of ->f_pos races Greg Kroah-Hartman
2012-09-12 23:28 ` [ 021/108] vfs: canonicalize create mode in build_open_flags() Greg Kroah-Hartman
2012-09-12 23:28 ` [ 022/108] alpha: fix fpu.h usage in userspace Greg Kroah-Hartman
2012-09-12 23:28 ` [ 023/108] alpha: Dont export SOCK_NONBLOCK to user space Greg Kroah-Hartman
2012-09-12 23:28 ` [ 024/108] USB: winbond: remove __devinit* from the struct usb_device_id table Greg Kroah-Hartman
2012-09-12 23:28 ` [ 025/108] mm: hugetlbfs: correctly populate shared pmd Greg Kroah-Hartman
2012-09-12 23:28 ` [ 026/108] ALSA: hda - dont create dysfunctional mixer controls for ca0132 Greg Kroah-Hartman
2012-09-12 23:28 ` [ 027/108] ALSA: hda - Fix leftover codec->power_transition Greg Kroah-Hartman
2012-09-12 23:28 ` [ 028/108] target: fix NULL pointer dereference bug alloc_page() fails to get memory Greg Kroah-Hartman
2012-09-12 23:28 ` [ 029/108] NFSv3: Ensure that do_proc_get_root() reports errors correctly Greg Kroah-Hartman
2012-09-12 23:28 ` [ 030/108] pnfs: defer release of pages in layoutget Greg Kroah-Hartman
2012-09-12 23:28 ` [ 031/108] nfs: tear down caches in nfs_init_writepagecache when allocation fails Greg Kroah-Hartman
2012-09-12 23:28 ` [ 032/108] NFSv4.1: Remove a bogus BUG_ON() in nfs4_layoutreturn_done Greg Kroah-Hartman
2012-09-12 23:28 ` [ 033/108] NFS: Clear key construction data if the idmap upcall fails Greg Kroah-Hartman
2012-09-12 23:28 ` [ 034/108] NFS: return -ENOKEY when the upcall fails to map the name Greg Kroah-Hartman
2012-09-12 23:28 ` [ 035/108] nfsd4: fix security flavor of NFSv4.0 callback Greg Kroah-Hartman
2012-09-12 23:28 ` [ 036/108] UBIFS: fix crash on error path Greg Kroah-Hartman
2012-09-12 23:28 ` [ 037/108] UBIFS: fix complaints about too small debug buffer size Greg Kroah-Hartman
2012-09-12 23:28 ` [ 038/108] Bluetooth: Fix using NULL inquiry entry Greg Kroah-Hartman
2012-09-12 23:28 ` [ 039/108] Bluetooth: Fix using a NULL inquiry cache entry Greg Kroah-Hartman
2012-09-12 23:28 ` [ 040/108] Bluetooth: Set name_state to unknown when entry name is empty Greg Kroah-Hartman
2012-09-12 23:28 ` [ 041/108] Bluetooth: Fix legacy pairing with some devices Greg Kroah-Hartman
2012-09-12 23:28 ` [ 042/108] NFS: Alias the nfs module to nfs4 Greg Kroah-Hartman
2012-09-13 1:08 ` Josh Boyer
2012-09-13 1:24 ` Myklebust, Trond
2012-09-13 2:35 ` Josh Boyer
2012-09-12 23:28 ` [ 043/108] audit: dont free_chunk() after fsnotify_add_mark() Greg Kroah-Hartman
2012-09-12 23:28 ` [ 044/108] audit: fix refcounting in audit-tree Greg Kroah-Hartman
2012-09-12 23:28 ` [ 045/108] drm: stop vmgfx driver explosion Greg Kroah-Hartman
2012-09-12 23:28 ` [ 046/108] Revert "drm/radeon: fix bo creation retry path" Greg Kroah-Hartman
2012-09-12 23:28 ` [ 047/108] Btrfs: revert checksum error statistic which can cause a BUG() Greg Kroah-Hartman
2012-09-12 23:28 ` [ 048/108] svcrpc: fix BUG() in svc_tcp_clear_pages Greg Kroah-Hartman
2012-09-12 23:28 ` [ 049/108] svcrpc: fix svc_xprt_enqueue/svc_recv busy-looping Greg Kroah-Hartman
2012-09-12 23:28 ` [ 050/108] svcrpc: sends on closed socket should stop immediately Greg Kroah-Hartman
2012-09-12 23:28 ` [ 051/108] cciss: fix incorrect scsi status reporting Greg Kroah-Hartman
2012-09-12 23:28 ` [ 052/108] ACPI: export symbol acpi_get_table_with_size Greg Kroah-Hartman
2012-09-12 23:28 ` [ 053/108] ath9k: stop btcoex on device suspend Greg Kroah-Hartman
2012-09-12 23:28 ` [ 054/108] ath9k: fix decrypt_error initialization in ath_rx_tasklet() Greg Kroah-Hartman
2012-09-12 23:28 ` [ 055/108] PCI: EHCI: Fix crash during hibernation on ASUS computers Greg Kroah-Hartman
2012-09-12 23:28 ` [ 056/108] gma500: Consider CRTC initially active Greg Kroah-Hartman
2012-09-12 23:28 ` [ 057/108] block: replace __getblk_slow misfix by grow_dev_page fix Greg Kroah-Hartman
2012-09-12 23:28 ` [ 058/108] jbd: dont write superblock when unmounting an ro filesystem Greg Kroah-Hartman
2012-09-12 23:28 ` [ 059/108] ALSA: usb-audio: Fix scheduling-while-atomic bug in PCM capture stream Greg Kroah-Hartman
2012-09-12 23:28 ` [ 060/108] sched,cgroup: Fix up task_groups list Greg Kroah-Hartman
2012-09-12 23:28 ` [ 061/108] sched: fix divide by zero at {thread_group,task}_times Greg Kroah-Hartman
2012-09-12 23:29 ` [ 062/108] [media] uvcvideo: Reset the bytesused field when recycling an erroneous buffer Greg Kroah-Hartman
2012-09-12 23:29 ` [ 063/108] rapidio/tsi721: fix inbound doorbell interrupt handling Greg Kroah-Hartman
2012-09-12 23:29 ` [ 064/108] rapidio/tsi721: fix unused variable compiler warning Greg Kroah-Hartman
2012-09-12 23:29 ` [ 065/108] regulator: twl-regulator: fix up VINTANA1/VINTANA2 Greg Kroah-Hartman
2012-09-12 23:29 ` [ 066/108] x32: Use compat shims for {g,s}etsockopt Greg Kroah-Hartman
2012-09-12 23:29 ` [ 067/108] USB: spca506: remove __devinit* from the struct usb_device_id table Greg Kroah-Hartman
2012-09-12 23:29 ` [ 068/108] USB: jl2005bcd: " Greg Kroah-Hartman
2012-09-12 23:29 ` [ 069/108] USB: p54usb: " Greg Kroah-Hartman
2012-09-12 23:29 ` [ 070/108] USB: rtl8187: " Greg Kroah-Hartman
2012-09-12 23:29 ` [ 071/108] USB: smsusb: " Greg Kroah-Hartman
2012-09-12 23:29 ` [ 072/108] USB: CDC ACM: Fix NULL pointer dereference Greg Kroah-Hartman
2012-09-12 23:29 ` [ 073/108] ALSA: snd-usb: Fix URB cancellation at stream start Greg Kroah-Hartman
2012-09-12 23:29 ` [ 074/108] ALSA: snd-usb: use list_for_each_safe for endpoint resources Greg Kroah-Hartman
2012-09-12 23:29 ` [ 075/108] ALSA: snd-usb: restore delay information Greg Kroah-Hartman
2012-09-12 23:29 ` [ 076/108] ALSA: snd-usb: fix calls to next_packet_size Greg Kroah-Hartman
2012-09-12 23:29 ` [ 077/108] ALSA: snd-usb: fix cross-interface streaming devices Greg Kroah-Hartman
2012-09-12 23:29 ` [ 078/108] powerpc: Update DSCR on all CPUs when writing sysfs dscr_default Greg Kroah-Hartman
2012-09-12 23:29 ` [ 079/108] powerpc: Keep thread.dscr and thread.dscr_inherit in sync Greg Kroah-Hartman
2012-09-12 23:29 ` [ 080/108] powerpc: Fix DSCR inheritance in copy_thread() Greg Kroah-Hartman
2012-09-12 23:29 ` [ 081/108] powerpc: Restore correct DSCR in context switch Greg Kroah-Hartman
2012-09-12 23:29 ` [ 082/108] powerpc: Make sure IPI handlers see data written by IPI senders Greg Kroah-Hartman
2012-09-12 23:29 ` [ 083/108] Remove user-triggerable BUG from mpol_to_str Greg Kroah-Hartman
2012-09-12 23:29 ` [ 084/108] Fix order of arguments to compat_put_time[spec|val] Greg Kroah-Hartman
2012-09-12 23:29 ` [ 085/108] usbnet: fix deadlock in resume Greg Kroah-Hartman
2012-09-12 23:29 ` [ 086/108] SCSI: megaraid_sas: Move poll_aen_lock initializer Greg Kroah-Hartman
2012-09-12 23:29 ` [ 087/108] SCSI: scsi_lib: fix scsi_io_completions SG_IO error propagation Greg Kroah-Hartman
2012-09-12 23:29 ` [ 088/108] SCSI: mpt2sas: Fix for Driver oops, when loading driver with max_queue_depth command line option to a very small value Greg Kroah-Hartman
2012-09-12 23:29 ` [ 089/108] SCSI: Fix Device not ready issue on mpt2sas Greg Kroah-Hartman
2012-09-12 23:29 ` [ 090/108] udf: Fix data corruption for files in ICB Greg Kroah-Hartman
2012-09-12 23:29 ` [ 091/108] ext3: Fix fdatasync() for files with only i_size changes Greg Kroah-Hartman
2012-09-12 23:29 ` [ 092/108] fuse: fix retrieve length Greg Kroah-Hartman
2012-09-12 23:29 ` [ 093/108] i2c-designware: Fix build error if CONFIG_I2C_DESIGNWARE_PLATFORM=y && CONFIG_I2C_DESIGNWARE_PCI=y Greg Kroah-Hartman
2012-09-12 23:29 ` [ 094/108] i2c-i801: Add Device IDs for Intel Lynx Point-LP PCH Greg Kroah-Hartman
2012-09-12 23:29 ` [ 095/108] HID: add NOGET quirk for Eaton Ellipse MAX UPS Greg Kroah-Hartman
2012-09-12 23:29 ` [ 096/108] Input: i8042 - add Gigabyte T1005 series netbooks to noloop table Greg Kroah-Hartman
2012-09-12 23:29 ` [ 097/108] UBI: fix a horrible memory deallocation bug Greg Kroah-Hartman
2012-09-12 23:29 ` [ 098/108] drm/vmwgfx: add MODULE_DEVICE_TABLE so vmwgfx loads at boot Greg Kroah-Hartman
2012-09-12 23:29 ` [ 099/108] OMAPFB: fix framebuffer console colors Greg Kroah-Hartman
2012-09-12 23:29 ` Greg Kroah-Hartman [this message]
2012-09-12 23:29 ` [ 101/108] PARISC: Redefine ATOMIC_INIT and ATOMIC64_INIT to drop the casts Greg Kroah-Hartman
2012-09-12 23:29 ` [ 102/108] xen: Use correct masking in xen_swiotlb_alloc_coherent Greg Kroah-Hartman
2012-09-12 23:29 ` [ 103/108] xen/p2m: Fix one-off error in checking the P2M tree directory Greg Kroah-Hartman
2012-09-12 23:29 ` [ 104/108] xen/pciback: Fix proper FLR steps Greg Kroah-Hartman
2012-09-12 23:29 ` [ 105/108] uprobes: Fix mmap_region()s mm->mm_rb corruption if uprobe_mmap() fails Greg Kroah-Hartman
2012-09-12 23:29 ` [ 106/108] x86, microcode, AMD: Fix broken ucode patch size check Greg Kroah-Hartman
2012-09-12 23:29 ` [ 107/108] dccp: check ccid before dereferencing Greg Kroah-Hartman
2012-09-12 23:29 ` [ 108/108] hwmon: (asus_atk0110) Add quirk for Asus M5A78L Greg Kroah-Hartman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120912232500.985321825@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=aaron.f.brown@intel.com \
--cc=bhutchings@solarflare.com \
--cc=bruce.w.allan@intel.com \
--cc=davem@davemloft.net \
--cc=jeffrey.t.kirsher@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.