xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: osstest service owner <osstest-admin@xenproject.org>
To: xen-devel@lists.xenproject.org, osstest-admin@xenproject.org
Subject: [linux-3.18 bisection] complete test-amd64-i386-examine
Date: Wed, 03 Apr 2019 23:21:29 +0000	[thread overview]
Message-ID: <E1hBpCD-0007rr-0t@osstest.test-lab.xenproject.org> (raw)

branch xen-unstable
xenbranch xen-unstable
job test-amd64-i386-examine
testid reboot

Tree: linux git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  linux git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
  Bug introduced:  7b8052e19304865477e03a0047062d977309a22f
  Bug not present: d255d18a34a8d53ccc4a019dc07e17b6e8cf6bd1
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/134363/


  commit 7b8052e19304865477e03a0047062d977309a22f
  Author: Jan Beulich <JBeulich@suse.com>
  Date:   Mon Oct 19 04:23:29 2015 -0600
  
      igb: fix NULL derefs due to skipped SR-IOV enabling
      
      [ Upstream commit be06998f96ecb93938ad2cce46c4289bf7cf45bc ]
      
      The combined effect of commits 6423fc3416 ("igb: do not re-init SR-IOV
      during probe") and ceee3450b3 ("igb: make sure SR-IOV init uses the
      right number of queues") causes VFs no longer getting set up, leading
      to NULL pointer dereferences due to the adapter's ->vf_data being NULL
      while ->vfs_allocated_count is non-zero. The first commit not only
      neglected the side effect of igb_sriov_reinit() that the second commit
      tried to account for, but also that of setting IGB_FLAG_HAS_MSIX,
      without which igb_enable_sriov() is effectively a no-op. Calling
      igb_{,re}set_interrupt_capability() as done here seems to address this,
      but I'm not sure whether this is better than sinply reverting the other
      two commits.
      
      Signed-off-by: Jan Beulich <jbeulich@suse.com>
      Tested-by: Aaron Brown <aaron.f.brown@intel.com>
      Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
      Signed-off-by: Sasha Levin <sashal@kernel.org>


For bisection revision-tuple graph see:
   http://logs.test-lab.xenproject.org/osstest/results/bisect/linux-3.18/test-amd64-i386-examine.reboot.html
Revision IDs in each graph node refer, respectively, to the Trees above.

----------------------------------------
Running cs-bisection-step --graph-out=/home/logs/results/bisect/linux-3.18/test-amd64-i386-examine.reboot --summary-out=tmp/134363.bisection-summary --basis-template=128858 --blessings=real,real-bisect linux-3.18 test-amd64-i386-examine reboot
Searching for failure / basis pass:
 133953 fail [host=debina1] / 132456 [host=baroque1] 128858 [host=albana1] 128841 [host=chardonnay0] 128807 [host=albana0] 128691 [host=elbling0] 128258 [host=rimava1] 128232 [host=pinot0] 128177 ok.
Failure / basis pass flights: 133953 / 128177
(tree with no url: minios)
(tree with no url: ovmf)
(tree with no url: seabios)
Tree: linux git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest e128f16fddb66db5a73c30d4f3558d037d119e58 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 1e780ef5a527661d1d6106ccacf65706e3ed664d
Basis pass 921b2fed6a79439ef1609ef4af0ada5cccb3555c c530a75c1e6a472b0eb9558310b518f0dfcd8860 9c0eed618f37dd5b4a57c8b3fbc48ef8913e3149 de5b678ca4dcdfa83e322491d478d66df56c1986 940185b2f6f343251c2b83bd96e599398cea51ec
Generating revisions with ./adhoc-revtuple-generator  git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git#921b2fed6a79439ef1609ef4af0ada5cccb3555c-e128f16fddb66db5a73c30d4f3558d037d119e58 git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860 git://xenbits.xen.org/qemu-xen-traditional.git#9c0eed618f37dd5b4a57c8b3fbc48ef8913e3149-d0d8ad39ecb51cd7497cd524484fe09f50876798 git://xenbits.xen.org/qemu-xen.git\
 #de5b678ca4dcdfa83e322491d478d66df56c1986-de5b678ca4dcdfa83e322491d478d66df56c1986 git://xenbits.xen.org/xen.git#940185b2f6f343251c2b83bd96e599398cea51ec-1e780ef5a527661d1d6106ccacf65706e3ed664d
Loaded 3004 nodes in revision graph
Searching for test results:
 128096 [host=albana1]
 128177 pass 921b2fed6a79439ef1609ef4af0ada5cccb3555c c530a75c1e6a472b0eb9558310b518f0dfcd8860 9c0eed618f37dd5b4a57c8b3fbc48ef8913e3149 de5b678ca4dcdfa83e322491d478d66df56c1986 940185b2f6f343251c2b83bd96e599398cea51ec
 128232 [host=pinot0]
 128258 [host=rimava1]
 128691 [host=elbling0]
 128807 [host=albana0]
 128858 [host=albana1]
 128841 [host=chardonnay0]
 129760 [host=debina0]
 129845 [host=debina0]
 129857 [host=debina0]
 129876 [host=debina0]
 129860 [host=debina0]
 129844 [host=debina0]
 129863 [host=debina0]
 129866 [host=debina0]
 129883 [host=debina0]
 129895 [host=debina0]
 129889 [host=debina0]
 129903 [host=debina0]
 129907 [host=debina0]
 129910 [host=debina0]
 129915 [host=debina0]
 129919 [host=debina0]
 129921 [host=debina0]
 130067 [host=debina0]
 130203 [host=debina0]
 130367 [host=debina0]
 130843 [host=debina0]
 130876 [host=debina0]
 130939 [host=debina0]
 131095 [host=debina0]
 131035 [host=debina0]
 131149 [host=debina0]
 131192 [host=debina0]
 131279 [host=debina0]
 131231 [host=debina0]
 131307 [host=debina0]
 131336 [host=debina0]
 131420 [host=debina0]
 131370 [host=debina0]
 131442 [host=debina0]
 131479 [host=debina0]
 131512 [host=debina0]
 131535 [host=debina0]
 131563 [host=debina0]
 131580 [host=debina0]
 131593 [host=debina0]
 131641 [host=debina0]
 131619 [host=debina0]
 131666 [host=debina0]
 131673 [host=debina0]
 131705 [host=debina0]
 131749 [host=debina0]
 131729 [host=debina0]
 131769 [host=debina0]
 131990 [host=debina0]
 131969 [host=debina0]
 132065 [host=debina0]
 132066 [host=debina0]
 132176 [host=debina0]
 132141 [host=debina0]
 132162 [host=debina0]
 132146 [host=debina0]
 132149 [host=debina0]
 132125 [host=debina0]
 132130 [host=debina0]
 132180 [host=debina0]
 132150 [host=debina0]
 132132 [host=debina0]
 132139 [host=debina0]
 132165 [host=debina0]
 132154 [host=debina0]
 132157 [host=debina0]
 132171 [host=debina0]
 132159 [host=debina0]
 132181 [host=debina0]
 132200 [host=debina0]
 132173 [host=debina0]
 132189 [host=debina0]
 132185 [host=debina0]
 132179 [host=debina0]
 132196 [host=debina0]
 132186 [host=debina0]
 132211 [host=debina0]
 132193 [host=debina0]
 132207 [host=debina0]
 132205 [host=debina0]
 132213 [host=debina0]
 132248 [host=debina0]
 132219 [host=debina0]
 132254 [host=debina0]
 132224 [host=debina0]
 132256 [host=debina0]
 132228 [host=debina0]
 132259 [host=debina0]
 132234 [host=debina0]
 132238 [host=debina0]
 132246 [host=debina0]
 132261 [host=debina0]
 132290 [host=debina0]
 132408 [host=debina0]
 132456 [host=baroque1]
 132579 [host=debina0]
 132652 [host=debina0]
 132741 [host=debina0]
 132798 [host=debina0]
 132969 [host=debina0]
 133275 [host=debina0]
 133288 [host=debina0]
 133464 fail irrelevant
 133499 fail irrelevant
 133565 fail irrelevant
 133539 fail irrelevant
 133630 fail e128f16fddb66db5a73c30d4f3558d037d119e58 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 f393b82fe5ba3ed9cfe2b306ffa53368e55b75af
 133577 fail e128f16fddb66db5a73c30d4f3558d037d119e58 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 f393b82fe5ba3ed9cfe2b306ffa53368e55b75af
 133671 fail e128f16fddb66db5a73c30d4f3558d037d119e58 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 eeb31ee522c7bb8541eb4c037be2c42bfcf0a3c3
 133693 fail e128f16fddb66db5a73c30d4f3558d037d119e58 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 4deeaf2a3ee50b096426eea41a4c9b96ded0f029
 133774 fail e128f16fddb66db5a73c30d4f3558d037d119e58 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 4deeaf2a3ee50b096426eea41a4c9b96ded0f029
 133736 []
 133819 fail e128f16fddb66db5a73c30d4f3558d037d119e58 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 4deeaf2a3ee50b096426eea41a4c9b96ded0f029
 133856 fail e128f16fddb66db5a73c30d4f3558d037d119e58 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 4deeaf2a3ee50b096426eea41a4c9b96ded0f029
 133891 fail e128f16fddb66db5a73c30d4f3558d037d119e58 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 4deeaf2a3ee50b096426eea41a4c9b96ded0f029
 133918 fail e128f16fddb66db5a73c30d4f3558d037d119e58 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 1e780ef5a527661d1d6106ccacf65706e3ed664d
 133953 fail e128f16fddb66db5a73c30d4f3558d037d119e58 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 1e780ef5a527661d1d6106ccacf65706e3ed664d
 134324 blocked 78e0897dd8b321ba1b4a2137778ab7ae7d400af5 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 011319e9ce110c70a3d52f2ea05e5eeb538c9e9e
 134273 pass 921b2fed6a79439ef1609ef4af0ada5cccb3555c c530a75c1e6a472b0eb9558310b518f0dfcd8860 9c0eed618f37dd5b4a57c8b3fbc48ef8913e3149 de5b678ca4dcdfa83e322491d478d66df56c1986 940185b2f6f343251c2b83bd96e599398cea51ec
 134311 fail 9b5eed105a45ac0557af113b4096132ae7e3e47f c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 68292c94a60eab24514ab4a8e4772af24dead807
 134316 fail 78e0897dd8b321ba1b4a2137778ab7ae7d400af5 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 f6b6ae78679b363ff670a9c125077c436dabd608
 134328 pass ce2c2e07337e98eb42904ef27c3ce62972d22a60 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 6d8ffac1f7a782dc2c7f8df3871a294729ae36bd
 134318 pass 0d63979c1bc9c85578be4c589768a13dc0a7c5eb c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 9f7c1e765d04c277ee762b41bf7d4ae469f9b9fb
 134297 fail e128f16fddb66db5a73c30d4f3558d037d119e58 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 1e780ef5a527661d1d6106ccacf65706e3ed664d
 134333 fail e29dae46550b33ab914c78960da6f26fc8522867 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 6d8ffac1f7a782dc2c7f8df3871a294729ae36bd
 134331 blocked 78e0897dd8b321ba1b4a2137778ab7ae7d400af5 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 c7d4847a430867102c483b81c5d222a3cded14d8
 134321 pass 21b25db9247985c6dc891221cb267e819967ba4d c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 6d8ffac1f7a782dc2c7f8df3871a294729ae36bd
 134343 pass 61fde28f92ab9a329ef06e0ab3f41269e3930213 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 6d8ffac1f7a782dc2c7f8df3871a294729ae36bd
 134340 fail 3237836242f704e95c2e255620e1a12c98b55a2b c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 6d8ffac1f7a782dc2c7f8df3871a294729ae36bd
 134348 fail 7b8052e19304865477e03a0047062d977309a22f c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 6d8ffac1f7a782dc2c7f8df3871a294729ae36bd
 134341 fail 7b8052e19304865477e03a0047062d977309a22f c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 6d8ffac1f7a782dc2c7f8df3871a294729ae36bd
 134345 pass d255d18a34a8d53ccc4a019dc07e17b6e8cf6bd1 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 6d8ffac1f7a782dc2c7f8df3871a294729ae36bd
 134356 fail 7b8052e19304865477e03a0047062d977309a22f c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 6d8ffac1f7a782dc2c7f8df3871a294729ae36bd
 134352 pass d255d18a34a8d53ccc4a019dc07e17b6e8cf6bd1 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 6d8ffac1f7a782dc2c7f8df3871a294729ae36bd
 134363 fail 7b8052e19304865477e03a0047062d977309a22f c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 6d8ffac1f7a782dc2c7f8df3871a294729ae36bd
 134358 pass d255d18a34a8d53ccc4a019dc07e17b6e8cf6bd1 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 6d8ffac1f7a782dc2c7f8df3871a294729ae36bd
Searching for interesting versions
 Result found: flight 128177 (pass), for basis pass
 Result found: flight 133918 (fail), for basis failure
 Repro found: flight 134273 (pass), for basis pass
 Repro found: flight 134297 (fail), for basis failure
 0 revisions at d255d18a34a8d53ccc4a019dc07e17b6e8cf6bd1 c530a75c1e6a472b0eb9558310b518f0dfcd8860 d0d8ad39ecb51cd7497cd524484fe09f50876798 de5b678ca4dcdfa83e322491d478d66df56c1986 6d8ffac1f7a782dc2c7f8df3871a294729ae36bd
No revisions left to test, checking graph state.
 Result found: flight 134345 (pass), for last pass
 Result found: flight 134348 (fail), for first failure
 Repro found: flight 134352 (pass), for last pass
 Repro found: flight 134356 (fail), for first failure
 Repro found: flight 134358 (pass), for last pass
 Repro found: flight 134363 (fail), for first failure

*** Found and reproduced problem changeset ***

  Bug is in tree:  linux git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
  Bug introduced:  7b8052e19304865477e03a0047062d977309a22f
  Bug not present: d255d18a34a8d53ccc4a019dc07e17b6e8cf6bd1
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/134363/


  commit 7b8052e19304865477e03a0047062d977309a22f
  Author: Jan Beulich <JBeulich@suse.com>
  Date:   Mon Oct 19 04:23:29 2015 -0600
  
      igb: fix NULL derefs due to skipped SR-IOV enabling
      
      [ Upstream commit be06998f96ecb93938ad2cce46c4289bf7cf45bc ]
      
      The combined effect of commits 6423fc3416 ("igb: do not re-init SR-IOV
      during probe") and ceee3450b3 ("igb: make sure SR-IOV init uses the
      right number of queues") causes VFs no longer getting set up, leading
      to NULL pointer dereferences due to the adapter's ->vf_data being NULL
      while ->vfs_allocated_count is non-zero. The first commit not only
      neglected the side effect of igb_sriov_reinit() that the second commit
      tried to account for, but also that of setting IGB_FLAG_HAS_MSIX,
      without which igb_enable_sriov() is effectively a no-op. Calling
      igb_{,re}set_interrupt_capability() as done here seems to address this,
      but I'm not sure whether this is better than sinply reverting the other
      two commits.
      
      Signed-off-by: Jan Beulich <jbeulich@suse.com>
      Tested-by: Aaron Brown <aaron.f.brown@intel.com>
      Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
      Signed-off-by: Sasha Levin <sashal@kernel.org>

dot: graph is too large for cairo-renderer bitmaps. Scaling by 0.340674 to fit
pnmtopng: 38 colors found
Revision graph left in /home/logs/results/bisect/linux-3.18/test-amd64-i386-examine.reboot.{dot,ps,png,html,svg}.
----------------------------------------
134363: tolerable ALL FAIL

flight 134363 linux-3.18 real-bisect [real]
http://logs.test-lab.xenproject.org/osstest/logs/134363/

Failures :-/ but no regressions.

Tests which did not succeed,
including tests which could not be run:
 test-amd64-i386-examine       8 reboot                  fail baseline untested


jobs:
 test-amd64-i386-examine                                      fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

             reply	other threads:[~2019-04-03 23:21 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-03 23:21 osstest service owner [this message]
  -- strict thread matches above, loose matches on Subject: below --
2018-11-13  6:14 [linux-3.18 bisection] complete test-amd64-i386-examine osstest service owner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=E1hBpCD-0007rr-0t@osstest.test-lab.xenproject.org \
    --to=osstest-admin@xenproject.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).