test

Yet another testing utility.

  1. test
    1. Usage
    2. Interface
    3. Parameters
    4. Environment
    5. License
    6. Author
    7. Repository
    8. History

Usage

 
(import test)

;; Simple test
(test 4 (+ 2 2))

;; group
(test-group "A group"
  (test "A test with description" 5 (+ 2 3))
  (test-assert "This should always be true" (string? "foo")))

;; IMPORTANT! The following ensures nightly automated tests can
;; distinguish failure from success.  Always end tests/run.scm with this.
(test-exit)

Interface

This is designed to be as simple and friendly as possible.

[syntax] (test [<name>] <expected-value> <expression>)

The basic test interface; really all you need to use. It will catch errors and print informative failure messages, including the name (which must be a string if present, and defaults to an abbreviated form of the test expression), the source, and the line number information of the failed expression. Equality is checked with EQUAL?, unless the expected value is inexact, in which case it checks to be sure the percentage difference (or absolute difference if one value is zero) between the result and expected value fall within the TEST-EPSILON parameter of each other. This is because it almost never makes sense to test inexact numbers with EQUAL? or =, and usually you'll want a single epsilon throughout a range of tests. However, you can always test = manually with TEST-ASSERT.

[syntax] (test-assert [<name>] <expression>)
[syntax] (test-error [<name>] <expression>)

Convenience wrappers around test. test-assert asserts that the expression is non-false; test-error asserts that the expression results in an error.

[syntax] (test-begin [<name>])
[syntax] (test-end [<name>])

Test group reporting, including elapsed time and pass percentages, can be achieved using

 (test-begin [<name>])
 (test ...)
 ...
 (test-end [<name>])

where the group name is optional in either case.

[syntax] (test-group <name> ...)

You can group tests in a single lexical scope (and also establish nested groups) with the test-group macro:

 (test-group <name>
   (test ...)
   ...
 )

The name for test-group is not optional.

[procedure] (test-exit [<failure-exit-code>])

As a convenience there is a test-exit procedure, which exits the process with a status of 0 if all tests in all groups have passed, and with an optional numeric status (defaulting to 1), if there have been any failures or errors.

Parameters

All other aspects of testing are controlled by parameters.

[parameter] current-test-verbosity

Default to #t, prints full diagnostics for failures

[parameter] current-test-epsilon

Percentage difference allowed for inexact comparisons

[parameter] current-test-comparator

TEST-EQUAL?

[parameter] current-test-filters

List of predicates to filter (not run) tests

[parameter] current-test-group-filters

List of predicates to filter entire test groups

[parameter] current-test-applier
[parameter] current-test-handler
[parameter] current-test-skipper
[parameter] current-test-group-reporter

These parameters can be used to override the reporting behavior, for example if you wanted a GUI interface instead of the default textual reporting.

Environment

Often you'll be working on a change or new feature, and while the code is changing the test suite remains relatively constant. However, it's a pain to recompile the test suite every time, and to re-run the entire suite when just working on one area, or even when you just want to run a single test by name.

The parameters allow filtering, but it can be a pain to setup an interface to this or to recompile with different settings. So for easy use, by default the parameters can be initialized from environment variables.

CURRENT-TEST-FILTERS can be set by the environment variables TEST_FILTER, a regexp which test names must match to be run, and TEST_REMOVE, a regexp of tests not to run. For example, if you have your test suite in the file test-foo.scm, compiled to test-foo, and you only want to run the test named "test-foo-with-bells-on," you could do so with

 $ TEST_FILTER=bells-on ./test-foo

The test name defaults to the source expression, which can also be handy for quick filtering. For instance, if in the same test suite you knew that you had broken the "flurble" data structure, you could filter out all tests using it with

 $ TEST_REMOVE=flurble- ./test-foo

For more structured filtering you can also filter by group. The CURRENT-TEST-GROUP-FILTERS parameter can be initialized with the TEST_GROUP_FILTER and TEST_GROUP_REMOVE environment variables in the same way as above.

You can also default the CURRENT-TEST-VERBOSITY parameter to #f by setting the TEST_QUIET environment variable.

test tries to automatically determine if your terminal supports ANSI colors and uses them (sparingly, mostly so that failures and errors stand out). You can force this on or off by setting the environment variable TEST_USE_ANSI variable to 0 or 1.

License

BSD

Author

Alex Shinn

Repository

This egg is hosted on the CHICKEN Subversion repository:

https://anonymous@code.call-cc.org/svn/chicken-eggs/release/5/test

If you want to check out the source code repository of this egg and you are not familiar with Subversion, see this page.

History
1.2
display floating-point number instead of a fraction for the time count
1.1
Pull up fixes made to CHICKEN 4 branch
1.0
Release for CHICKEN 5
0.9.9.11
Added urxvt to the list of supported color terminals
0.9.9.9
Removed regex dependency [Arthur Maciel and Evan Hanson].
0.9.9.8
Bugfix for approx-equal? with zero value.
0.9.9.7
Added test-total-count parameter.
0.9.9.6
Fixed indentation for nested group reporting.
0.9.9.5
Improved approx-equal? definition.
0.9.9.4
Fixed bug introduced with last change to re-allow test with no group for simple cases.
0.9.9.3
Added support for less verbose tests w/ TEST_QUIET=1 env var.
0.9.9.2
Added missing require-library statements for libraries that are imported by test [Peter Bex].
0.9.9.1
Renamed use of deprecated getenv [felix].
0.9.9
Added regex dependency.
0.8
Port to hygienic chicken [Peter Bex]
0.7
Fix for dynamic test names w/o syntax-rules [Peter Bex].
0.6
Added several TERM settings to those recognized as ANSI.
0.5
Fixed spurious warning in nested groups.
0.4
Fixed time report (was off).
0.3
minor bugfixes
0.2
minor bugfixes
0.1
initial release