From: Ed Smith <esmith@virtualiron.com>
To: Xen Devel <xen-devel@lists.xensource.com>
Subject: Testing status of HVM (Intel VT) on 64bit XEN unstable c/s 12308
Date: Thu, 09 Nov 2006 18:22:59 -0500 [thread overview]
Message-ID: <4553B853.6040409@virtualiron.com> (raw)
[-- Attachment #1: Type: text/plain, Size: 4336 bytes --]
Summary:
Changeset 12308
- nanosleep01/02 tests run much better, taking many passes before
failing. The gettimeofday02 ltp clock test still fails (failure.1)
- After a 256MB guest has been up and running for some time
on a 2GB node, launching another 256MB guest fails with:
Error: (12, 'Cannot allocate memory') (failure.2)
Test Configuration:
Dell Precision WorkStation 380, Dual Core, 2GB, 3 SATA (Intel VT)
64bit XEN RELEASE Build Hypervisor on a RHEL4U2 64bit root (/dev/sda)
32bit fully virtualized (HVM) guest RHEL4U2 256MB (/dev/sdb)
pae=1(smp) pae=0(up), acpi=1, apic=1
64bit fully virtualized (HVM) guest RHEL4U2 256MB (/dev/sdc)
pae=1, acpi=1, apic=1
Boot Tests:
Boot a fully virtualized (HVM) guest to the login prompt
Results are marked Pass|Fail where (n) points to a failure description
Regression Tests:
852 tests (851 ltp tests and one 30 minute user load test)
Tests are marked #Pass/#Fail where (n) points to a failure description
XEN 64bit 2 CPU Hypervisor (booted smp):
----------------------------------------------------------------------
| XEN | Guest Kernel (SMP kernels booted with 2 CPUs) |
| Changeset|-----------------------------------------------------------|
| | 32bit UP | 32bit SMP | 64bit UP | 64bit SMP |
| |--------------|--------------|--------------|--------------|
| | Boot | Test | Boot | Test | Boot | Test | Boot | Test |
|----------|------|-------|------|-------|------|-------|------|-------|
| 12308 | Pass | | Pass | 851/1 | Pass | | Pass | 851/1 |
| | | | | (1) | | | | (1) |
|----------|------|-------|------|-------|------|-------|------|-------|
| 12217 | Fail | | Fail | | Fail | | Fail | |
| | (4) | | (4) | | (4) | | (4) | |
|----------|------|-------|------|-------|------|-------|------|-------|
| 12053 | Pass | | Pass | 851/1 | Pass | | Pass | 851/1 |
| | | | |(1) | | | | (1) |
|----------|------|-------|------|-------|------|-------|------|-------|
| 12025 | Pass | | Pass | 851/1 | Pass | | Pass | 851/1 |
| | (3) | | (3) |(1) | | | | (1) |
|----------|------|-------|------|-------|------|-------|------|-------|
| 12009 | Pass | | Pass | 851/1 | Pass | | Pass | 851/1 |
| | | | |(1) | | | | (1) |
----------------------------------------------------------------------
Multiple Guest Boot Test
Test is a 30 minute user load on both Guests
XEN 64bit 2 CPU Hypervisor (booted smp):
--------------------------------------------
| XEN | Guest Kernel |
| Changeset|---------------------------------|
| | 32bit 1CPU UP | 32bit 2CPU SMP |
| | 64bit 1CPU UP | 64bit 2CPU SMP |
| |----------------|----------------|
| | Boot | Test | Boot | Test |
|----------|------|---------|------|---------|
| 12308 | Pass | Pass | Pass | Pass |
| | (2) | | (2) | |
|----------|------|---------|------|---------|
| 12217 | Fail | | Fail | |
| | (4) | | (4) | |
|----------|------|---------|------|---------|
| 12053 | Pass | Pass | Pass | Pass |
| | (2) | | (2) | |
|----------|------|---------|------|---------|
| 12025 | Pass | Pass | Pass | Pass |
| | (2) | | (2) | |
|----------|------|---------|------|---------|
| 12009 | Pass | Pass | Pass | Pass |
| | (2) | | (2) | |
--------------------------------------------
Failures:
1. BUG 666: 32bit guests fail ltp gettimeofday02
and nanosleep01/02 with clock problems
2. After a 256MB guest has been up and running for some time
on a 2GB node, launching another 256MB guest fails with:
Error: (12, 'Cannot allocate memory')
3. FIXED: 32bit guest coming up paused when launched, reporting this
error: WARNING: device 'vif1.0' has non-zero queue length (32)!
Unpausing the guest allows it to boot.
4. FIXED: HVM guests boot up but do not see the network
[-- Attachment #2: failure.1 --]
[-- Type: text/plain, Size: 16616 bytes --]
File: failure.1
Red Hat Enterprise Linux ES release 4 (Nahant Update 2)
Kernel 2.6.16.29-xen on an x86_64
tst079 login: root
Password:
Last login: Thu Nov 9 13:32:37 from 10.1.2.13
You have new mail.
[root@tst079 ~]# xm info
host : tst079
release : 2.6.16.29-xen
version : #1 SMP Thu Nov 9 01:43:03 EST 2006
machine : x86_64
nr_cpus : 2
nr_nodes : 1
sockets_per_node : 1
cores_per_socket : 2
threads_per_core : 1
cpu_mhz : 2793
hw_caps : bfebfbff:20100800:00000000:00000180:0000e43d:00000000:00000001
total_memory : 1022
free_memory : 1
xen_major : 3
xen_minor : 0
xen_extra : -unstable
xen_caps : xen-3.0-x86_64 hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : Wed Nov 08 19:48:46 2006 +0000 12308:3280d6211213
cc_compiler : gcc version 4.0.0 20050519 (Red Hat 4.0.0-8)
cc_compile_by : build
cc_compile_domain : virtualiron.com
cc_compile_date : Thu Nov 9 01:02:44 EST 2006
xend_config_format : 2
[root@tst079 ~]# xm list
Name ID Mem VCPUs State Time(s)
Domain-0 0 461 2 r----- 2041.2
Error: No domain named '-1'.
Usage: xm list [options] [Domain, ...]
List information about all/some domains.
-l, --long Output all VM details in SXP
--label Include security labels
[root@tst079 ~]#
Guest:
root@vs161's password:
Last login: Thu Nov 9 10:30:41 2006
[root@vs161 ~]# cd /qa/conductor/tests/ltp/testcases/bin/
[root@vs161 bin]# while [ 1 ]; do gettimeofday02; sleep 2; done
gettimeofday02 0 INFO : checking if gettimeofday is monotonous, takes 30s
gettimeofday02 1 FAIL : Time is going backwards (old 1163109553.451024 vs new 1163109553.380048!
gettimeofday02 0 INFO : checking if gettimeofday is monotonous, takes 30s
gettimeofday02 1 FAIL : Time is going backwards (old 1163109557.541991 vs new 1163109557.446435!
gettimeofday02 0 INFO : checking if gettimeofday is monotonous, takes 30s
gettimeofday02 1 FAIL : Time is going backwards (old 1163109559.471831 vs new 1163109559.471716!
gettimeofday02 0 INFO : checking if gettimeofday is monotonous, takes 30s
gettimeofday02 1 FAIL : Time is going backwards (old 1163109561.511749 vs new 1163109561.511305!
gettimeofday02 0 INFO : checking if gettimeofday is monotonous, takes 30s
gettimeofday02 1 FAIL : Time is going backwards (old 1163109563.545659 vs new 1163109563.544892!
[root@vs161 bin]# while [ 1 ]; do nanosleep01; sleep 2; done
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 FAIL : Child execution not suspended for 2 seconds. (Wanted 2000 ms, got 3535 ms)
nanosleep01 1 FAIL : Failures reported above
nanosleep01 1 PASS : nanosleep functionality is correct
nanosleep01 1 PASS : nanosleep functionality is correct
[root@vs161 bin]# while [ 1 ]; do nanosleep02; sleep 2; done
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
nanosleep02 1 PASS : Functionality of nanosleep() is correct
[-- Attachment #3: failure.2 --]
[-- Type: text/plain, Size: 2373 bytes --]
File: failure.2
[root@tst078 ~]# xm info
host : tst078
release : 2.6.16.29-xen
version : #1 SMP Thu Nov 9 01:43:03 EST 2006
machine : x86_64
nr_cpus : 2
nr_nodes : 1
sockets_per_node : 1
cores_per_socket : 2
threads_per_core : 1
cpu_mhz : 2793
hw_caps : bfebfbff:20100800:00000000:00000180:0000e43d:00000000:00000001
total_memory : 2046
free_memory : 1
xen_major : 3
xen_minor : 0
xen_extra : -unstable
xen_caps : xen-3.0-x86_64 hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : Wed Nov 08 19:48:46 2006 +0000 12308:3280d6211213
cc_compiler : gcc version 4.0.0 20050519 (Red Hat 4.0.0-8)
cc_compile_by : build
cc_compile_domain : virtualiron.com
cc_compile_date : Thu Nov 9 01:02:44 EST 2006
xend_config_format : 2
[root@tst078 ~]# xm list
Name ID Mem VCPUs State Time(s)
Domain-0 0 1737 2 r----- 7523.9
vs170 1 263 2 -b---- 15214.6
[root@tst078 ~]# cat /proc/meminfo
MemTotal: 1778688 kB
MemFree: 11496 kB
Buffers: 1601500 kB
Cached: 15960 kB
SwapCached: 0 kB
Active: 597804 kB
Inactive: 1047020 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 1778688 kB
LowFree: 11496 kB
SwapTotal: 6450056 kB
SwapFree: 6449784 kB
Dirty: 484 kB
Writeback: 0 kB
Mapped: 36976 kB
Slab: 60708 kB
CommitLimit: 7339400 kB
Committed_AS: 115400 kB
PageTables: 2660 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 85492 kB
VmallocChunk: 34359651751 kB
[root@tst078 ~]# cat /proc/xen/balloon
Current allocation: 1778688 kB
Requested target: 1778688 kB
Low-mem balloon: 149108 kB
High-mem balloon: 0 kB
Driver pages: 1024 kB
Xen hard limit: ??? kB
[root@tst078 ~]# xm create /boot/vs160_sdl.hvm
Using config file "/boot/vs160_sdl.hvm".
Error: (12, 'Cannot allocate memory')
[root@tst078 ~]#
[-- Attachment #4: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
reply other threads:[~2006-11-09 23:22 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4553B853.6040409@virtualiron.com \
--to=esmith@virtualiron.com \
--cc=xen-devel@lists.xensource.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.