All of lore.kernel.org
 help / color / mirror / Atom feed
From: Marc Zyngier <maz@kernel.org>
To: Jon Hunter <jonathanh@nvidia.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>,
	linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org, Sumit Garg <sumit.garg@linaro.org>,
	kernel-team@android.com, Florian Fainelli <f.fainelli@gmail.com>,
	Russell King <linux@arm.linux.org.uk>,
	Jason Cooper <jason@lakedaemon.net>,
	Saravana Kannan <saravanak@google.com>,
	Andrew Lunn <andrew@lunn.ch>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Gregory Clement <gregory.clement@bootlin.com>,
	Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>,
	Krzysztof Kozlowski <krzk@kernel.org>,
	'Linux Samsung SOC' <linux-samsung-soc@vger.kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Will Deacon <will@kernel.org>,
	Valentin Schneider <Valentin.Schneider@arm.com>,
	linux-tegra <linux-tegra@vger.kernel.org>,
	linus.walleij@linaro.org
Subject: Re: [PATCH v3 08/16] irqchip/gic: Configure SGIs as standard interrupts
Date: Wed, 16 Sep 2020 16:10:57 +0100	[thread overview]
Message-ID: <3378cd07b92e87a24f1db75f708424ee@kernel.org> (raw)
In-Reply-To: <933bc43e-3cd7-10ec-b9ec-58afaa619fb7@nvidia.com>

Hi Jon,

+Linus, who is facing a similar issue.

On 2020-09-16 15:16, Jon Hunter wrote:
> Hi Marc,
> 
> On 14/09/2020 14:06, Marek Szyprowski wrote:
>> Hi Marc,
>> 
>> On 01.09.2020 16:43, Marc Zyngier wrote:
>>> Change the way we deal with GIC SGIs by turning them into proper
>>> IRQs, and calling into the arch code to register the interrupt range
>>> instead of a callback.
>>> 
>>> Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
>>> Signed-off-by: Marc Zyngier <maz@kernel.org>
>> This patch landed in linux next-20200914 as commit ac063232d4b0
>> ("irqchip/gic: Configure SGIs as standard interrupts"). Sadly it 
>> breaks
>> booting of all Samsung Exynos 4210/4412 based boards (dual/quad ARM
>> Cortex A9 based). Here are the last lines from the bootlog:
> 
> I am observing the same thing on several Tegra boards (both arm and
> arm64). Bisect is pointing to this commit. Reverting this alone does 
> not
> appear to be enough to fix the issue.

Right, I am just massively by the GICv3 spec, and failed to remember
that ye olde GIC exposes the source CPU in AIR *and* wants it back, 
while
newer GICs deal with that transparently.

Can you try the patch below and let me know?

         M.

diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index 98743afdaea6..56492bf8b6f9 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -121,9 +121,10 @@ static struct gic_chip_data 
gic_data[CONFIG_ARM_GIC_MAX_NR] __read_mostly;

  static struct gic_kvm_info gic_v2_kvm_info;

+static DEFINE_PER_CPU(u32, sgi_intid);
+
  #ifdef CONFIG_GIC_NON_BANKED
  static DEFINE_STATIC_KEY_FALSE(frankengic_key);
-static DEFINE_PER_CPU(u32, sgi_intid);

  static void enable_frankengic(void)
  {
@@ -135,16 +136,6 @@ static inline bool is_frankengic(void)
  	return static_branch_unlikely(&frankengic_key);
  }

-static inline void set_sgi_intid(u32 intid)
-{
-	this_cpu_write(sgi_intid, intid);
-}
-
-static inline u32 get_sgi_intid(void)
-{
-	return this_cpu_read(sgi_intid);
-}
-
  static inline void __iomem *__get_base(union gic_base *base)
  {
  	if (is_frankengic())
@@ -160,8 +151,6 @@ static inline void __iomem *__get_base(union 
gic_base *base)
  #define gic_data_cpu_base(d)	((d)->cpu_base.common_base)
  #define enable_frankengic()	do { } while(0)
  #define is_frankengic()		false
-#define set_sgi_intid(i)	do { } while(0)
-#define get_sgi_intid()		0
  #endif

  static inline void __iomem *gic_dist_base(struct irq_data *d)
@@ -236,8 +225,8 @@ static void gic_eoi_irq(struct irq_data *d)
  {
  	u32 hwirq = gic_irq(d);

-	if (is_frankengic() && hwirq < 16)
-		hwirq = get_sgi_intid();
+	if (hwirq < 16)
+		hwirq = this_cpu_read(sgi_intid);

  	writel_relaxed(hwirq, gic_cpu_base(d) + GIC_CPU_EOI);
  }
@@ -365,14 +354,13 @@ static void __exception_irq_entry 
gic_handle_irq(struct pt_regs *regs)
  			smp_rmb();

  			/*
-			 * Samsung's funky GIC encodes the source CPU in
-			 * GICC_IAR, leading to the deactivation to fail if
-			 * not written back as is to GICC_EOI.  Stash the
-			 * INTID away for gic_eoi_irq() to write back.
-			 * This only works because we don't nest SGIs...
+			 * The GIC encodes the source CPU in GICC_IAR,
+			 * leading to the deactivation to fail if not
+			 * written back as is to GICC_EOI.  Stash the INTID
+			 * away for gic_eoi_irq() to write back.  This only
+			 * works because we don't nest SGIs...
  			 */
-			if (is_frankengic())
-				set_sgi_intid(irqstat);
+			this_cpu_write(sgi_intid, intid);
  		}

  		handle_domain_irq(gic->domain, irqnr, regs);

-- 
Jazz is not dead. It just smells funny...

WARNING: multiple messages have this Message-ID (diff)
From: Marc Zyngier <maz@kernel.org>
To: Jon Hunter <jonathanh@nvidia.com>
Cc: Sumit Garg <sumit.garg@linaro.org>,
	linus.walleij@linaro.org, Florian Fainelli <f.fainelli@gmail.com>,
	Russell King <linux@arm.linux.org.uk>,
	Jason Cooper <jason@lakedaemon.net>,
	Saravana Kannan <saravanak@google.com>,
	Andrew Lunn <andrew@lunn.ch>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Gregory Clement <gregory.clement@bootlin.com>,
	Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>,
	linux-kernel@vger.kernel.org,
	Krzysztof Kozlowski <krzk@kernel.org>,
	Will Deacon <will@kernel.org>,
	'Linux Samsung SOC' <linux-samsung-soc@vger.kernel.org>,
	linux-tegra <linux-tegra@vger.kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	kernel-team@android.com,
	Valentin Schneider <Valentin.Schneider@arm.com>,
	linux-arm-kernel@lists.infradead.org,
	Marek Szyprowski <m.szyprowski@samsung.com>
Subject: Re: [PATCH v3 08/16] irqchip/gic: Configure SGIs as standard interrupts
Date: Wed, 16 Sep 2020 16:10:57 +0100	[thread overview]
Message-ID: <3378cd07b92e87a24f1db75f708424ee@kernel.org> (raw)
In-Reply-To: <933bc43e-3cd7-10ec-b9ec-58afaa619fb7@nvidia.com>

Hi Jon,

+Linus, who is facing a similar issue.

On 2020-09-16 15:16, Jon Hunter wrote:
> Hi Marc,
> 
> On 14/09/2020 14:06, Marek Szyprowski wrote:
>> Hi Marc,
>> 
>> On 01.09.2020 16:43, Marc Zyngier wrote:
>>> Change the way we deal with GIC SGIs by turning them into proper
>>> IRQs, and calling into the arch code to register the interrupt range
>>> instead of a callback.
>>> 
>>> Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
>>> Signed-off-by: Marc Zyngier <maz@kernel.org>
>> This patch landed in linux next-20200914 as commit ac063232d4b0
>> ("irqchip/gic: Configure SGIs as standard interrupts"). Sadly it 
>> breaks
>> booting of all Samsung Exynos 4210/4412 based boards (dual/quad ARM
>> Cortex A9 based). Here are the last lines from the bootlog:
> 
> I am observing the same thing on several Tegra boards (both arm and
> arm64). Bisect is pointing to this commit. Reverting this alone does 
> not
> appear to be enough to fix the issue.

Right, I am just massively by the GICv3 spec, and failed to remember
that ye olde GIC exposes the source CPU in AIR *and* wants it back, 
while
newer GICs deal with that transparently.

Can you try the patch below and let me know?

         M.

diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index 98743afdaea6..56492bf8b6f9 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -121,9 +121,10 @@ static struct gic_chip_data 
gic_data[CONFIG_ARM_GIC_MAX_NR] __read_mostly;

  static struct gic_kvm_info gic_v2_kvm_info;

+static DEFINE_PER_CPU(u32, sgi_intid);
+
  #ifdef CONFIG_GIC_NON_BANKED
  static DEFINE_STATIC_KEY_FALSE(frankengic_key);
-static DEFINE_PER_CPU(u32, sgi_intid);

  static void enable_frankengic(void)
  {
@@ -135,16 +136,6 @@ static inline bool is_frankengic(void)
  	return static_branch_unlikely(&frankengic_key);
  }

-static inline void set_sgi_intid(u32 intid)
-{
-	this_cpu_write(sgi_intid, intid);
-}
-
-static inline u32 get_sgi_intid(void)
-{
-	return this_cpu_read(sgi_intid);
-}
-
  static inline void __iomem *__get_base(union gic_base *base)
  {
  	if (is_frankengic())
@@ -160,8 +151,6 @@ static inline void __iomem *__get_base(union 
gic_base *base)
  #define gic_data_cpu_base(d)	((d)->cpu_base.common_base)
  #define enable_frankengic()	do { } while(0)
  #define is_frankengic()		false
-#define set_sgi_intid(i)	do { } while(0)
-#define get_sgi_intid()		0
  #endif

  static inline void __iomem *gic_dist_base(struct irq_data *d)
@@ -236,8 +225,8 @@ static void gic_eoi_irq(struct irq_data *d)
  {
  	u32 hwirq = gic_irq(d);

-	if (is_frankengic() && hwirq < 16)
-		hwirq = get_sgi_intid();
+	if (hwirq < 16)
+		hwirq = this_cpu_read(sgi_intid);

  	writel_relaxed(hwirq, gic_cpu_base(d) + GIC_CPU_EOI);
  }
@@ -365,14 +354,13 @@ static void __exception_irq_entry 
gic_handle_irq(struct pt_regs *regs)
  			smp_rmb();

  			/*
-			 * Samsung's funky GIC encodes the source CPU in
-			 * GICC_IAR, leading to the deactivation to fail if
-			 * not written back as is to GICC_EOI.  Stash the
-			 * INTID away for gic_eoi_irq() to write back.
-			 * This only works because we don't nest SGIs...
+			 * The GIC encodes the source CPU in GICC_IAR,
+			 * leading to the deactivation to fail if not
+			 * written back as is to GICC_EOI.  Stash the INTID
+			 * away for gic_eoi_irq() to write back.  This only
+			 * works because we don't nest SGIs...
  			 */
-			if (is_frankengic())
-				set_sgi_intid(irqstat);
+			this_cpu_write(sgi_intid, intid);
  		}

  		handle_domain_irq(gic->domain, irqnr, regs);

-- 
Jazz is not dead. It just smells funny...

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

  reply	other threads:[~2020-09-16 20:59 UTC|newest]

Thread overview: 170+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-01 14:43 [PATCH v3 00/16] arm/arm64: Turning IPIs into normal interrupts Marc Zyngier
2020-09-01 14:43 ` Marc Zyngier
2020-09-01 14:43 ` [PATCH v3 01/16] genirq: Add fasteoi IPI flow Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-01 14:43 ` [PATCH v3 02/16] genirq: Allow interrupts to be excluded from /proc/interrupts Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-01 14:43 ` [PATCH v3 03/16] arm64: Allow IPIs to be handled as normal interrupts Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-11 15:05   ` Catalin Marinas
2020-09-11 15:05     ` Catalin Marinas
2020-10-19 12:42   ` Vincent Guittot
2020-10-19 12:42     ` Vincent Guittot
2020-10-19 13:04     ` Marc Zyngier
2020-10-19 13:04       ` Marc Zyngier
2020-10-19 15:43       ` Vincent Guittot
2020-10-19 15:43         ` Vincent Guittot
2020-10-19 16:00         ` Valentin Schneider
2020-10-19 16:00           ` Valentin Schneider
2020-10-27 10:12         ` Vincent Guittot
2020-10-27 10:12           ` Vincent Guittot
2020-10-27 10:37           ` Marc Zyngier
2020-10-27 10:37             ` Marc Zyngier
2020-10-27 10:50             ` Vincent Guittot
2020-10-27 10:50               ` Vincent Guittot
2020-10-27 11:21               ` Vincent Guittot
2020-10-27 11:21                 ` Vincent Guittot
2020-10-27 12:06                 ` Marc Zyngier
2020-10-27 12:06                   ` Marc Zyngier
2020-10-27 13:17                   ` Vincent Guittot
2020-10-27 13:17                     ` Vincent Guittot
     [not found]                     ` <c66367b0-e8a0-2b7b-13c3-c9413462357c@huawei.com>
2021-05-06 11:44                       ` Marc Zyngier
2021-05-06 11:44                         ` Marc Zyngier
2021-05-07  7:30                         ` He Ying
2021-05-07  7:30                           ` He Ying
2021-05-07  8:56                           ` Marc Zyngier
2021-05-07  8:56                             ` Marc Zyngier
2021-05-07  9:31                             ` He Ying
2021-05-07  9:31                               ` He Ying
2020-09-01 14:43 ` [PATCH v3 04/16] ARM: " Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-01 14:43 ` [PATCH v3 05/16] irqchip/gic-v3: Describe the SGI range Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-01 14:43 ` [PATCH v3 06/16] irqchip/gic-v3: Configure SGIs as standard interrupts Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-01 14:43 ` [PATCH v3 07/16] irqchip/gic: Refactor SMP configuration Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-01 14:43 ` [PATCH v3 08/16] irqchip/gic: Configure SGIs as standard interrupts Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-14 13:06   ` Marek Szyprowski
2020-09-14 13:06     ` Marek Szyprowski
2020-09-14 13:13     ` Marc Zyngier
2020-09-14 13:13       ` Marc Zyngier
2020-09-14 13:26       ` Marek Szyprowski
2020-09-14 13:26         ` Marek Szyprowski
2020-09-14 15:09         ` Marc Zyngier
2020-09-14 15:09           ` Marc Zyngier
2020-09-15  6:48           ` Marek Szyprowski
2020-09-15  6:48             ` Marek Szyprowski
2020-09-15  8:07             ` Marc Zyngier
2020-09-15  8:07               ` Marc Zyngier
2020-09-15  8:35               ` Marek Szyprowski
2020-09-15  8:35                 ` Marek Szyprowski
2020-09-15  9:48                 ` Marc Zyngier
2020-09-15  9:48                   ` Marc Zyngier
2020-09-16 14:16     ` Jon Hunter
2020-09-16 14:16       ` Jon Hunter
2020-09-16 15:10       ` Marc Zyngier [this message]
2020-09-16 15:10         ` Marc Zyngier
2020-09-16 15:46         ` Jon Hunter
2020-09-16 15:46           ` Jon Hunter
2020-09-16 15:55           ` Marc Zyngier
2020-09-16 15:55             ` Marc Zyngier
2020-09-16 15:58             ` Jon Hunter
2020-09-16 15:58               ` Jon Hunter
2020-09-16 16:22               ` Marc Zyngier
2020-09-16 16:22                 ` Marc Zyngier
2020-09-16 16:28                 ` Marc Zyngier
2020-09-16 16:28                   ` Marc Zyngier
2020-09-16 19:08                   ` Jon Hunter
2020-09-16 19:08                     ` Jon Hunter
2020-09-16 19:06                 ` Jon Hunter
2020-09-16 19:06                   ` Jon Hunter
2020-09-16 19:26                   ` Mikko Perttunen
2020-09-16 19:26                     ` Mikko Perttunen
2020-09-16 19:39                     ` Jon Hunter
2020-09-16 19:39                       ` Jon Hunter
2020-09-17  7:40         ` Linus Walleij
2020-09-17  7:40           ` Linus Walleij
2020-09-17  7:50           ` Marc Zyngier
2020-09-17  7:50             ` Marc Zyngier
2020-09-17  7:54             ` Jon Hunter
2020-09-17  7:54               ` Jon Hunter
2020-09-17  8:45               ` Marc Zyngier
2020-09-17  8:45                 ` Marc Zyngier
2020-09-17  8:49                 ` Jon Hunter
2020-09-17  8:49                   ` Jon Hunter
2020-09-17  8:54                   ` Marek Szyprowski
2020-09-17  8:54                     ` Marek Szyprowski
2020-09-17  9:09                     ` Jon Hunter
2020-09-17  9:09                       ` Jon Hunter
2020-09-17  9:13                       ` Marek Szyprowski
2020-09-17  9:13                         ` Marek Szyprowski
2020-09-17  9:29                         ` Marc Zyngier
2020-09-17  9:29                           ` Marc Zyngier
2020-09-17 14:53                     ` Jon Hunter
2020-09-17 14:53                       ` Jon Hunter
2020-09-17 18:24                       ` Jon Hunter
2020-09-17 18:24                         ` Jon Hunter
2020-09-18  8:24                         ` Marc Zyngier
2020-09-18  8:24                           ` Marc Zyngier
2020-09-17  8:56                   ` Marc Zyngier
2020-09-17  8:56                     ` Marc Zyngier
2020-09-17 10:11                   ` Linus Walleij
2020-09-17 10:11                     ` Linus Walleij
2020-09-16 14:03   ` Linus Walleij
2020-09-16 14:03     ` Linus Walleij
2020-09-16 14:14     ` Marc Zyngier
2020-09-16 14:14       ` Marc Zyngier
2020-09-18  9:58   ` James Morse
2020-09-18  9:58     ` James Morse
2020-09-18 10:21     ` Marc Zyngier
2020-09-18 10:21       ` Marc Zyngier
2020-09-01 14:43 ` [PATCH v3 09/16] irqchip/gic-common: Don't enable SGIs by default Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-01 14:43 ` [PATCH v3 10/16] irqchip/bcm2836: Configure mailbox interrupts as standard interrupts Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-14 14:32   ` Marek Szyprowski
2020-09-14 14:32     ` Marek Szyprowski
2020-09-14 16:10     ` Marc Zyngier
2020-09-14 16:10       ` Marc Zyngier
2020-09-14 19:13       ` Marek Szyprowski
2020-09-14 19:13         ` Marek Szyprowski
2020-09-01 14:43 ` [PATCH v3 11/16] irqchip/hip04: Configure IPIs " Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-01 14:43 ` [PATCH v3 12/16] irqchip/armada-370-xp: " Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-01 14:43 ` [PATCH v3 13/16] arm64: Kill __smp_cross_call and co Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-11 15:06   ` Catalin Marinas
2020-09-11 15:06     ` Catalin Marinas
2020-09-01 14:43 ` [PATCH v3 14/16] arm64: Remove custom IRQ stat accounting Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-11 15:06   ` Catalin Marinas
2020-09-11 15:06     ` Catalin Marinas
2020-09-01 14:43 ` [PATCH v3 15/16] ARM: Kill __smp_cross_call and co Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-01 14:43 ` [PATCH v3 16/16] ARM: Remove custom IRQ stat accounting Marc Zyngier
2020-09-01 14:43   ` Marc Zyngier
2020-09-02  7:41   ` kernel test robot
2020-09-02  7:41     ` kernel test robot
2020-09-02  7:41     ` kernel test robot
2020-09-02 20:20     ` Marc Zyngier
2020-09-02 20:20       ` Marc Zyngier
2020-09-02 20:20       ` Marc Zyngier
2020-09-24  9:00   ` Guillaume Tucker
2020-09-24  9:00     ` Guillaume Tucker
2020-09-24  9:29     ` Marc Zyngier
2020-09-24  9:29       ` Marc Zyngier
2020-09-24 13:09       ` Guillaume Tucker
2020-09-24 13:09         ` Guillaume Tucker
2020-09-28  9:00         ` Guillaume Tucker
2020-09-28  9:00           ` Guillaume Tucker
2020-09-24 13:34     ` Fabio Estevam
2020-09-24 13:34       ` Fabio Estevam
2020-09-24 14:19       ` Guillaume Tucker
2020-09-24 14:19         ` Guillaume Tucker
2020-09-07  6:06 ` [PATCH v3 00/16] arm/arm64: Turning IPIs into normal interrupts hasegawa-hitomi
2020-09-07  6:06   ` hasegawa-hitomi
2020-09-16 16:54 ` Florian Fainelli
2020-09-16 16:54   ` Florian Fainelli

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3378cd07b92e87a24f1db75f708424ee@kernel.org \
    --to=maz@kernel.org \
    --cc=Valentin.Schneider@arm.com \
    --cc=andrew@lunn.ch \
    --cc=b.zolnierkie@samsung.com \
    --cc=catalin.marinas@arm.com \
    --cc=f.fainelli@gmail.com \
    --cc=gregory.clement@bootlin.com \
    --cc=jason@lakedaemon.net \
    --cc=jonathanh@nvidia.com \
    --cc=kernel-team@android.com \
    --cc=krzk@kernel.org \
    --cc=linus.walleij@linaro.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-samsung-soc@vger.kernel.org \
    --cc=linux-tegra@vger.kernel.org \
    --cc=linux@arm.linux.org.uk \
    --cc=m.szyprowski@samsung.com \
    --cc=saravanak@google.com \
    --cc=sumit.garg@linaro.org \
    --cc=tglx@linutronix.de \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.