All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>, "Jeff King" <peff@peff.net>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH v2 0/8] Improvements for t/README
Date: Thu,  1 Jul 2010 20:17:50 +0000	[thread overview]
Message-ID: <1278015478-6920-1-git-send-email-avarab@gmail.com> (raw)

This is v2 of the t/README series. It should address the concerns
Junio and Jeff had about it.

Jeff:

  - Keep the ./ way of running tests, but elaborate a bit in the
    commit message

  - Mention --root's effects on the trash directory location

Junio:

  - The bit about ok/not ok moved out of the section on running
    tests.

  - Integrate a section that covers all the points in Junio's "Test
    your stuff" section, but does so as an itemized list, so it's
    easier to maintain it.

Other:

  - Made it diff --check friendly.

There's a diff against v1 of the series after this diffstat:

Ævar Arnfjörð Bjarmason (8):
  t/README: Tests are all +x, ./test, not sh ./test
  t/README: The trash is in 't/trash directory.$name'
  t/README: Typo: paralell -> parallel
  t/README: Document the prereq functions, and 3-arg test_*
  t/README: Document test_external*
  t/README: Document test_expect_code
  t/README: Add a section about skipping tests
  t/README: Document the do's and don'ts of tests

---

 t/README |  172 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 files changed, 164 insertions(+), 8 deletions(-)

diff --git a/t/README b/t/README
index 29f795e..4a1cfc9 100644
--- a/t/README
+++ b/t/README
@@ -50,12 +50,6 @@ prove and other harnesses come with a lot of useful options. The
     # Repeat until no more failures
     $ prove -j 15 --state=failed,save ./t[0-9]*.sh
 
-The TAP support is completely incidental. A TAP harness is just a
-program that interprets the "ok"/"not ok" (and some other strings) in
-a special way. The only limitation this imposes is that you shouldn't
-echo "ok" or "not ok" by yourself at the beginning of a line, that'll
-confuse the TAP harness.
-
 You can also run each test individually from command line, like this:
 
     $ ./t3010-ls-files-killed-modified.sh
@@ -229,14 +223,92 @@ This test harness library does the following things:
 
  - Creates an empty test directory with an empty .git/objects database
    and chdir(2) into it.  This directory is 't/trash
-   directory.$test_name_without_dotsh' if you must know, but I do not
-   think you care.
+   directory.$test_name_without_dotsh', with t/ subject to change by
+   the --root option documented above.
 
  - Defines standard test helper functions for your scripts to
    use.  These functions are designed to make all scripts behave
    consistently when command line arguments --verbose (or -v),
    --debug (or -d), and --immediate (or -i) is given.
 
+Do's, don'ts & things to keep in mind
+-------------------------------------
+
+Here's a few examples of things you probably should and shouldn't do
+when writing tests.
+
+Do:
+
+ - Put as much code as possible inside test_expect_success and other
+   assertions.
+
+   Even code that isn't a test per se, but merely some setup code
+   should be inside a test assertion if at all possible. Test scripts
+   should only have trivial code outside of their assertions.
+
+ - Chain your test assertions
+
+   Write test code like this:
+
+	git merge foo &&
+	git push bar &&
+	test ...
+
+   Instead of:
+
+	git merge hla
+	git push gh
+	test ...
+
+   That way all of the commands in your tests will succeed or fail. If
+   you must ignore the return value of something (e.g. the return
+   value of export is unportable) it's best to indicate so explicitly
+   with a semicolon:
+
+	export HLAGH;
+	git merge hla &&
+	git push gh &&
+	test ...
+
+Don't:
+
+ - exit() within a <script> part.
+
+   The harness will catch this as a programming error of the test.
+   Use test_done instead if you need to stop the tests early (see
+   "Skipping tests" below).
+
+ - Break the TAP output
+
+   The raw output from your test might be interpreted by a TAP
+   harness. You usually don't have to worry about that. TAP harnesses
+   will ignore everything they don't know about, but don't step on
+   their toes in these areas:
+
+   - Don't print lines like "$x..$y" where $x and $y are integers.
+
+   - Don't print lines that begin with "ok" or "not ok".
+
+   A TAP harness expect a line that begins with either "ok" and "not
+   ok" to signal a test passed or failed (and our harness already
+   produces such lines), so your script shouldn't emit such lines to
+   their output.
+
+   You can gleam some further possible issues from the TAP grammar
+   (see http://search.cpan.org/perldoc?TAP::Parser::Grammar#TAP_Grammar)
+   but the best indication is to just run the tests with prove(1),
+   it'll complain if anything is amiss.
+
+Keep in mind:
+
+ - That what you print to stderr and stdout is usually ignored
+
+   Inside <script> part, the standard output and standard error
+   streams are discarded, and the test harness only reports "ok" or
+   "not ok" to the end user running the tests. Under --verbose, they
+   are shown to help debugging the tests.
+
+
 Skipping tests
 --------------
 

             reply	other threads:[~2010-07-01 20:18 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-07-01 20:17 Ævar Arnfjörð Bjarmason [this message]
2010-07-01 20:17 ` [PATCH v2 1/8] t/README: Tests are all +x, ./test, not sh ./test Ævar Arnfjörð Bjarmason
2010-07-01 23:54   ` Junio C Hamano
2010-07-02  1:00     ` Ævar Arnfjörð Bjarmason
2010-07-01 20:17 ` [PATCH v2 2/8] t/README: The trash is in 't/trash directory.$name' Ævar Arnfjörð Bjarmason
2010-07-01 20:17 ` [PATCH v2 3/8] t/README: Typo: paralell -> parallel Ævar Arnfjörð Bjarmason
2010-07-01 20:17 ` [PATCH v2 4/8] t/README: Document the prereq functions, and 3-arg test_* Ævar Arnfjörð Bjarmason
2010-07-01 20:17 ` [PATCH v2 5/8] t/README: Document test_external* Ævar Arnfjörð Bjarmason
2010-07-01 20:52   ` Jakub Narebski
2010-07-01 21:18     ` [PATCH v3 " Ævar Arnfjörð Bjarmason
2010-07-01 20:17 ` [PATCH v2 6/8] t/README: Document test_expect_code Ævar Arnfjörð Bjarmason
2010-07-01 20:17 ` [PATCH v2 7/8] t/README: Add a section about skipping tests Ævar Arnfjörð Bjarmason
2010-07-01 20:17 ` [PATCH v2 8/8] t/README: Document the do's and don'ts of tests Ævar Arnfjörð Bjarmason
2010-07-02  7:28 ` [PATCH v2 0/8] Improvements for t/README Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1278015478-6920-1-git-send-email-avarab@gmail.com \
    --to=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.