All of lore.kernel.org
 help / color / mirror / Atom feed
From: Wu Fengguang <fengguang.wu@intel.com>
To: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Jan Kara <jack@suse.cz>, Chris Mason <chris.mason@oracle.com>,
	Dave Chinner <david@fromorbit.com>, Jens Axboe <axboe@kernel.dk>,
	Christoph Hellwig <hch@lst.de>, "Theodore Ts'o" <tytso@mit.edu>,
	Mel Gorman <mel@csn.ul.ie>, Rik van Riel <riel@redhat.com>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	linux-mm <linux-mm@kvack.org>,
	"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>, tglx <tglx@linutronix.de>
Subject: Re: [PATCH 01/13] writeback: IO-less balance_dirty_pages()
Date: Thu, 18 Nov 2010 21:26:17 +0800	[thread overview]
Message-ID: <20101118132617.GA9307@localhost> (raw)
In-Reply-To: <1290085474.2109.1480.camel@laptop>

On Thu, Nov 18, 2010 at 09:04:34PM +0800, Peter Zijlstra wrote:
> On Wed, 2010-11-17 at 12:27 +0800, Wu Fengguang wrote:
> > - avoid useless (eg. zero pause time) balance_dirty_pages() calls
> > - avoid too small pause time (less than  10ms, which burns CPU power)
> > - avoid too large pause time (more than 100ms, which hurts responsiveness)
> > - avoid big fluctuations of pause times 
> 
> If you feel like playing with sub-jiffies timeouts (a way to avoid that
> HZ=>100 assumption), the below (totally untested) patch might be of
> help..

Assuming there are HZ=10 users.

- when choosing such a coarse granularity, do they really care about
  responsiveness? :)

- will the use of hrtimer add a little code size and/or runtime
  overheads, and hence hurt the majority HZ=100 users?

Thanks,
Fengguang

> 
> ---
> Subject: hrtimer: Provide io_schedule_timeout*() functions
> 
> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> ---
>  include/linux/hrtimer.h |    7 +++++++
>  kernel/hrtimer.c        |   15 +++++++++++++++
>  kernel/sched.c          |   17 +++++++++++++++++
>  3 files changed, 39 insertions(+), 0 deletions(-)
> 
> diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
> index dd9954b..9e0f67e 100644
> --- a/include/linux/hrtimer.h
> +++ b/include/linux/hrtimer.h
> @@ -419,6 +419,13 @@ extern long hrtimer_nanosleep_restart(struct restart_block *restart_block);
>  extern void hrtimer_init_sleeper(struct hrtimer_sleeper *sl,
>  				 struct task_struct *tsk);
>  
> +extern int io_schedule_hrtimeout_range(ktime_t *expires, unsigned long delta,
> +						const enum hrtimer_mode mode);
> +extern int io_schedule_hrtimeout_range_clock(ktime_t *expires,
> +		unsigned long delta, const enum hrtimer_mode mode, int clock);
> +extern int io_schedule_hrtimeout(ktime_t *expires, const enum hrtimer_mode mode);
> +
> +
>  extern int schedule_hrtimeout_range(ktime_t *expires, unsigned long delta,
>  						const enum hrtimer_mode mode);
>  extern int schedule_hrtimeout_range_clock(ktime_t *expires,
> diff --git a/kernel/hrtimer.c b/kernel/hrtimer.c
> index 72206cf..ef2d93c 100644
> --- a/kernel/hrtimer.c
> +++ b/kernel/hrtimer.c
> @@ -1838,6 +1838,14 @@ int __sched schedule_hrtimeout_range(ktime_t *expires, unsigned long delta,
>  }
>  EXPORT_SYMBOL_GPL(schedule_hrtimeout_range);
>  
> +int __sched io_schedule_hrtimeout_range(ktime_t *expires, unsigned long delta,
> +				     const enum hrtimer_mode mode)
> +{
> +	return io_schedule_hrtimeout_range_clock(expires, delta, mode,
> +					      CLOCK_MONOTONIC);
> +}
> +EXPORT_SYMBOL_GPL(io_schedule_hrtimeout_range);
> +
>  /**
>   * schedule_hrtimeout - sleep until timeout
>   * @expires:	timeout value (ktime_t)
> @@ -1866,3 +1874,10 @@ int __sched schedule_hrtimeout(ktime_t *expires,
>  	return schedule_hrtimeout_range(expires, 0, mode);
>  }
>  EXPORT_SYMBOL_GPL(schedule_hrtimeout);
> +
> +int __sched io_schedule_hrtimeout(ktime_t *expires,
> +			       const enum hrtimer_mode mode)
> +{
> +	return io_schedule_hrtimeout_range(expires, 0, mode);
> +}
> +EXPORT_SYMBOL_GPL(io_schedule_hrtimeout);
> diff --git a/kernel/sched.c b/kernel/sched.c
> index d5564a8..ac84455 100644
> --- a/kernel/sched.c
> +++ b/kernel/sched.c
> @@ -5303,6 +5303,23 @@ long __sched io_schedule_timeout(long timeout)
>  	return ret;
>  }
>  
> +int __sched
> +io_schedule_hrtimeout_range_clock(ktime_t *expires, unsigned long delta,
> +			       const enum hrtimer_mode mode, int clock)
> +{
> +	struct rq *rq = raw_rq();
> +	long ret;
> +
> +	delayacct_blkio_start();
> +	atomic_inc(&rq->nr_iowait);
> +	current->in_iowait = 1;
> +	ret = schedule_hrtimeout_range_clock(expires, delta, mode, clock);
> +	current->in_iowait = 0;
> +	atomic_dec(&rq->nr_iowait);
> +	delayacct_blkio_end();
> +	return ret;
> +}
> +
>  /**
>   * sys_sched_get_priority_max - return maximum RT priority.
>   * @policy: scheduling class.
> 

WARNING: multiple messages have this Message-ID (diff)
From: Wu Fengguang <fengguang.wu@intel.com>
To: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Jan Kara <jack@suse.cz>, Chris Mason <chris.mason@oracle.com>,
	Dave Chinner <david@fromorbit.com>, Jens Axboe <axboe@kernel.dk>,
	Christoph Hellwig <hch@lst.de>, Theodore Ts'o <tytso@mit.edu>,
	Mel Gorman <mel@csn.ul.ie>, Rik van Riel <riel@redhat.com>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	linux-mm <linux-mm@kvack.org>,
	"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>, tglx <tglx@linutronix.de>
Subject: Re: [PATCH 01/13] writeback: IO-less balance_dirty_pages()
Date: Thu, 18 Nov 2010 21:26:17 +0800	[thread overview]
Message-ID: <20101118132617.GA9307@localhost> (raw)
In-Reply-To: <1290085474.2109.1480.camel@laptop>

On Thu, Nov 18, 2010 at 09:04:34PM +0800, Peter Zijlstra wrote:
> On Wed, 2010-11-17 at 12:27 +0800, Wu Fengguang wrote:
> > - avoid useless (eg. zero pause time) balance_dirty_pages() calls
> > - avoid too small pause time (less than  10ms, which burns CPU power)
> > - avoid too large pause time (more than 100ms, which hurts responsiveness)
> > - avoid big fluctuations of pause times 
> 
> If you feel like playing with sub-jiffies timeouts (a way to avoid that
> HZ=>100 assumption), the below (totally untested) patch might be of
> help..

Assuming there are HZ=10 users.

- when choosing such a coarse granularity, do they really care about
  responsiveness? :)

- will the use of hrtimer add a little code size and/or runtime
  overheads, and hence hurt the majority HZ=100 users?

Thanks,
Fengguang

> 
> ---
> Subject: hrtimer: Provide io_schedule_timeout*() functions
> 
> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> ---
>  include/linux/hrtimer.h |    7 +++++++
>  kernel/hrtimer.c        |   15 +++++++++++++++
>  kernel/sched.c          |   17 +++++++++++++++++
>  3 files changed, 39 insertions(+), 0 deletions(-)
> 
> diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h
> index dd9954b..9e0f67e 100644
> --- a/include/linux/hrtimer.h
> +++ b/include/linux/hrtimer.h
> @@ -419,6 +419,13 @@ extern long hrtimer_nanosleep_restart(struct restart_block *restart_block);
>  extern void hrtimer_init_sleeper(struct hrtimer_sleeper *sl,
>  				 struct task_struct *tsk);
>  
> +extern int io_schedule_hrtimeout_range(ktime_t *expires, unsigned long delta,
> +						const enum hrtimer_mode mode);
> +extern int io_schedule_hrtimeout_range_clock(ktime_t *expires,
> +		unsigned long delta, const enum hrtimer_mode mode, int clock);
> +extern int io_schedule_hrtimeout(ktime_t *expires, const enum hrtimer_mode mode);
> +
> +
>  extern int schedule_hrtimeout_range(ktime_t *expires, unsigned long delta,
>  						const enum hrtimer_mode mode);
>  extern int schedule_hrtimeout_range_clock(ktime_t *expires,
> diff --git a/kernel/hrtimer.c b/kernel/hrtimer.c
> index 72206cf..ef2d93c 100644
> --- a/kernel/hrtimer.c
> +++ b/kernel/hrtimer.c
> @@ -1838,6 +1838,14 @@ int __sched schedule_hrtimeout_range(ktime_t *expires, unsigned long delta,
>  }
>  EXPORT_SYMBOL_GPL(schedule_hrtimeout_range);
>  
> +int __sched io_schedule_hrtimeout_range(ktime_t *expires, unsigned long delta,
> +				     const enum hrtimer_mode mode)
> +{
> +	return io_schedule_hrtimeout_range_clock(expires, delta, mode,
> +					      CLOCK_MONOTONIC);
> +}
> +EXPORT_SYMBOL_GPL(io_schedule_hrtimeout_range);
> +
>  /**
>   * schedule_hrtimeout - sleep until timeout
>   * @expires:	timeout value (ktime_t)
> @@ -1866,3 +1874,10 @@ int __sched schedule_hrtimeout(ktime_t *expires,
>  	return schedule_hrtimeout_range(expires, 0, mode);
>  }
>  EXPORT_SYMBOL_GPL(schedule_hrtimeout);
> +
> +int __sched io_schedule_hrtimeout(ktime_t *expires,
> +			       const enum hrtimer_mode mode)
> +{
> +	return io_schedule_hrtimeout_range(expires, 0, mode);
> +}
> +EXPORT_SYMBOL_GPL(io_schedule_hrtimeout);
> diff --git a/kernel/sched.c b/kernel/sched.c
> index d5564a8..ac84455 100644
> --- a/kernel/sched.c
> +++ b/kernel/sched.c
> @@ -5303,6 +5303,23 @@ long __sched io_schedule_timeout(long timeout)
>  	return ret;
>  }
>  
> +int __sched
> +io_schedule_hrtimeout_range_clock(ktime_t *expires, unsigned long delta,
> +			       const enum hrtimer_mode mode, int clock)
> +{
> +	struct rq *rq = raw_rq();
> +	long ret;
> +
> +	delayacct_blkio_start();
> +	atomic_inc(&rq->nr_iowait);
> +	current->in_iowait = 1;
> +	ret = schedule_hrtimeout_range_clock(expires, delta, mode, clock);
> +	current->in_iowait = 0;
> +	atomic_dec(&rq->nr_iowait);
> +	delayacct_blkio_end();
> +	return ret;
> +}
> +
>  /**
>   * sys_sched_get_priority_max - return maximum RT priority.
>   * @policy: scheduling class.
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2010-11-18 13:26 UTC|newest]

Thread overview: 168+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-11-17  4:27 [PATCH 00/13] IO-less dirty throttling v2 Wu Fengguang
2010-11-17  4:27 ` Wu Fengguang
2010-11-17  4:27 ` Wu Fengguang
2010-11-17  4:27 ` [PATCH 01/13] writeback: IO-less balance_dirty_pages() Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17 10:34   ` Minchan Kim
2010-11-17 10:34     ` Minchan Kim
2010-11-22  2:01     ` Wu Fengguang
2010-11-22  2:01       ` Wu Fengguang
2010-11-22  2:01       ` Wu Fengguang
2010-11-17 23:08   ` Andrew Morton
2010-11-17 23:08     ` Andrew Morton
2010-11-17 23:08     ` Andrew Morton
2010-11-18 13:04   ` Peter Zijlstra
2010-11-18 13:04     ` Peter Zijlstra
2010-11-18 13:26     ` Wu Fengguang [this message]
2010-11-18 13:26       ` Wu Fengguang
2010-11-18 13:40       ` Peter Zijlstra
2010-11-18 13:40         ` Peter Zijlstra
2010-11-18 14:02         ` Wu Fengguang
2010-11-18 14:02           ` Wu Fengguang
     [not found]     ` <20101129151719.GA30590@localhost>
     [not found]       ` <1291064013.32004.393.camel@laptop>
     [not found]         ` <20101130043735.GA22947@localhost>
     [not found]           ` <1291156522.32004.1359.camel@laptop>
     [not found]             ` <1291156765.32004.1365.camel@laptop>
     [not found]               ` <20101201133818.GA13377@localhost>
2010-12-01 23:03                 ` Andrew Morton
2010-12-01 23:03                   ` Andrew Morton
2010-12-02  1:56                   ` Wu Fengguang
2010-12-02  1:56                     ` Wu Fengguang
2010-12-05 16:14                 ` Wu Fengguang
2010-12-06  2:42                   ` Ted Ts'o
2010-12-06  2:42                     ` Ted Ts'o
2010-12-06  9:52                     ` Dmitry
2010-12-06  9:52                       ` Dmitry
2010-12-06  9:52                       ` Dmitry
2010-12-06 12:34                       ` Ted Ts'o
2010-12-06 12:34                         ` Ted Ts'o
2010-11-17  4:27 ` [PATCH 02/13] writeback: consolidate variable names in balance_dirty_pages() Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27 ` [PATCH 03/13] writeback: per-task rate limit on balance_dirty_pages() Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17 14:39   ` Wu Fengguang
2010-11-17 14:39     ` Wu Fengguang
2010-11-24 10:23   ` Peter Zijlstra
2010-11-24 10:23     ` Peter Zijlstra
2010-11-24 10:43     ` Wu Fengguang
2010-11-24 10:43       ` Wu Fengguang
2010-11-24 10:49       ` Peter Zijlstra
2010-11-24 10:49         ` Peter Zijlstra
2010-11-17  4:27 ` [PATCH 04/13] writeback: prevent duplicate balance_dirty_pages_ratelimited() calls Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27 ` [PATCH 05/13] writeback: account per-bdi accumulated written pages Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-24 10:26   ` Peter Zijlstra
2010-11-24 10:26     ` Peter Zijlstra
2010-11-24 10:44     ` Wu Fengguang
2010-11-24 10:44       ` Wu Fengguang
2010-11-17  4:27 ` [PATCH 06/13] writeback: bdi write bandwidth estimation Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17 23:08   ` Andrew Morton
2010-11-17 23:08     ` Andrew Morton
2010-11-17 23:08     ` Andrew Morton
2010-11-17 23:24     ` Peter Zijlstra
2010-11-17 23:24       ` Peter Zijlstra
2010-11-17 23:38       ` Andrew Morton
2010-11-17 23:38         ` Andrew Morton
2010-11-17 23:43         ` Peter Zijlstra
2010-11-17 23:43           ` Peter Zijlstra
2010-11-18  6:51     ` Wu Fengguang
2010-11-18  6:51       ` Wu Fengguang
2010-11-24 10:58   ` Peter Zijlstra
2010-11-24 10:58     ` Peter Zijlstra
2010-11-24 14:06     ` Wu Fengguang
2010-11-24 14:06       ` Wu Fengguang
2010-11-24 11:05   ` Peter Zijlstra
2010-11-24 11:05     ` Peter Zijlstra
2010-11-24 12:10     ` Wu Fengguang
2010-11-24 12:10       ` Wu Fengguang
2010-11-24 12:50       ` Peter Zijlstra
2010-11-24 12:50         ` Peter Zijlstra
2010-11-24 13:14         ` Wu Fengguang
2010-11-24 13:14           ` Wu Fengguang
2010-11-24 13:20           ` Wu Fengguang
2010-11-24 13:20             ` Wu Fengguang
2010-11-24 13:42             ` Peter Zijlstra
2010-11-24 13:42               ` Peter Zijlstra
2010-11-24 13:46               ` Wu Fengguang
2010-11-24 13:46                 ` Wu Fengguang
2010-11-24 14:12                 ` Peter Zijlstra
2010-11-24 14:12                   ` Peter Zijlstra
2010-11-24 14:21                   ` Wu Fengguang
2010-11-24 14:21                     ` Wu Fengguang
2010-11-24 14:31                     ` Peter Zijlstra
2010-11-24 14:31                       ` Peter Zijlstra
2010-11-24 14:38                       ` Wu Fengguang
2010-11-24 14:38                         ` Wu Fengguang
2010-11-24 14:34                   ` Wu Fengguang
2010-11-24 14:34                     ` Wu Fengguang
2010-11-17  4:27 ` [PATCH 07/13] writeback: show bdi write bandwidth in debugfs Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27 ` [PATCH 08/13] writeback: quit throttling when bdi dirty pages dropped low Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-24 11:13   ` Peter Zijlstra
2010-11-24 11:13     ` Peter Zijlstra
2010-11-24 12:30     ` Wu Fengguang
2010-11-24 12:30       ` Wu Fengguang
2010-11-24 12:46       ` Peter Zijlstra
2010-11-24 12:46         ` Peter Zijlstra
2010-11-24 12:59         ` Wu Fengguang
2010-11-24 12:59           ` Wu Fengguang
2010-11-17  4:27 ` [PATCH 09/13] writeback: reduce per-bdi dirty threshold ramp up time Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-24 11:15   ` Peter Zijlstra
2010-11-24 11:15     ` Peter Zijlstra
2010-11-24 12:39     ` Wu Fengguang
2010-11-24 12:39       ` Wu Fengguang
2010-11-24 12:56       ` Peter Zijlstra
2010-11-24 12:56         ` Peter Zijlstra
2010-11-17  4:27 ` [PATCH 10/13] writeback: make reasonable gap between the dirty/background thresholds Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-24 11:18   ` Peter Zijlstra
2010-11-24 11:18     ` Peter Zijlstra
2010-11-24 12:48     ` Wu Fengguang
2010-11-24 12:48       ` Wu Fengguang
2010-11-17  4:27 ` [PATCH 11/13] writeback: scale down max throttle bandwidth on concurrent dirtiers Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27 ` [PATCH 12/13] writeback: add trace event for balance_dirty_pages() Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:41   ` Wu Fengguang
2010-11-17  4:41     ` Wu Fengguang
2010-11-17  4:27 ` [PATCH 13/13] writeback: make nr_to_write a per-file limit Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17  4:27   ` Wu Fengguang
2010-11-17 23:03 ` [PATCH 00/13] IO-less dirty throttling v2 Andrew Morton
2010-11-17 23:03   ` Andrew Morton
2010-11-17 23:03   ` Andrew Morton
2010-11-18  2:06   ` Dave Chinner
2010-11-18  2:06     ` Dave Chinner
2010-11-18  2:09     ` Andrew Morton
2010-11-18  2:09       ` Andrew Morton
2010-11-18  3:21       ` Dave Chinner
2010-11-18  3:21         ` Dave Chinner
2010-11-18  3:34         ` Andrew Morton
2010-11-18  3:34           ` Andrew Morton
2010-11-18  7:27           ` Dave Chinner
2010-11-18  7:27             ` Dave Chinner
2010-11-18  7:33             ` Andrew Morton
2010-11-18  7:33               ` Andrew Morton
2010-11-19  3:11               ` Dave Chinner
2010-11-19  3:11                 ` Dave Chinner
2010-11-24 11:12       ` Avi Kivity
2010-11-24 11:12         ` Avi Kivity
  -- strict thread matches above, loose matches on Subject: below --
2010-11-17  3:58 [PATCH 01/13] writeback: IO-less balance_dirty_pages() Wu Fengguang
2010-11-17  3:58 ` Wu Fengguang
2010-11-17  3:58 ` Wu Fengguang
2010-11-17  4:19 ` Wu Fengguang
2010-11-17  4:19   ` Wu Fengguang
2010-11-17  8:33   ` Wu Fengguang
2010-11-17  8:33     ` Wu Fengguang
2010-11-17  4:30 ` Wu Fengguang
2010-11-17  4:30   ` Wu Fengguang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20101118132617.GA9307@localhost \
    --to=fengguang.wu@intel.com \
    --cc=a.p.zijlstra@chello.nl \
    --cc=akpm@linux-foundation.org \
    --cc=axboe@kernel.dk \
    --cc=chris.mason@oracle.com \
    --cc=david@fromorbit.com \
    --cc=hch@lst.de \
    --cc=jack@suse.cz \
    --cc=kosaki.motohiro@jp.fujitsu.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mel@csn.ul.ie \
    --cc=riel@redhat.com \
    --cc=tglx@linutronix.de \
    --cc=tytso@mit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.