Outdated egg!

This is an egg for CHICKEN 3, the unsupported old release. You're almost certainly looking for the CHICKEN 4 version of this egg, if it exists.

If it does not exist, there may be equivalent functionality provided by another egg; have a look at the egg index. Otherwise, please consider porting this egg to the current version of CHICKEN.

testbase

Outdated egg!
testbase

Author

Kon Lovett, with portions from test-infrastructure 1.0 by Peter Keller

This extension provides a macro based unit testing facility based upon expectations concerning evaluations of expressions. Expectations are grouped into test containers, such as test-suite and test/case. In addition to expectation and container test forms a set of annotation forms are supplied. All test forms, be they containers, annotations, or expectations, return a test result object. Result objects generated by the test forms enclosed by a test container are composed by the container into the container's test result object. The set of test results forms a tree.

This result tree is then passed to either user defined functions which traverse the tree manipulating it in any way desired. API functions to deal with the result types are supplied. Examples of existing functions that traverse the result are the test results output routines.

Test File

A test-file is a Scheme source file consisting of testbase forms and necessary auxiliary definitions. The general template is:

(require-extension testbase)
.. other extensions, includes, imports, etc. as needed ..

#| TestBase
.. see "testbase-driver" documentation ..
|#

.. any expressions necessary to testing ..

(define-test foo-test ... )

(run-test "Foo Tests")

While possible to use testbase forms without invoking run-test command line processing is not performed.

Command Line Options:

---failure-exit CODE: Specifiy an exit code & set the early-exit-upon-failure-option. See test::exit-on-failure?, test::run, and run-test.
-b ---batch: Specify batch mode. See run-test.
-i ---interactive: Specify interactive (or driven) mode. See run-test.
-V ---version: Display the version.
-h ---help: Display help message.

Macro and Procedure Naming Conventions

A distinction is made between forms that are to be used within a test, and those that are to be used outside of a test. In general, those forms that are for use within a test are macros, and those for use outside a test are procedures.

Procedures have a test:: namespace prefix, such as test::timing

Macros do not have a test:: namespace prefix. Since they can only pollute the namespace of a single file a looser naming convention is used.

Option Expansion Control

The expanded source of the test code is controlled by a set of syntax constants. To drop or add an option place the appropriate syntax form below in a test file.

FLAG is a boolean, #t or #f.

test::timing

[syntax] (define-for-syntax test::timing FLAG)

test::timing controls the expectation timing option. Default is #f.

test::filtering

[syntax] (define-for-syntax test::filtering FLAG)

test::filtering controls the test result filtering option. Default is #f.

test::catching

[syntax] (define-for-syntax test::catching FLAG)

test::catching controls the expectation error/exception trapping option. Default is #t.

Dropping error/exception trapping is not suggested in the general case, since any intermediate errors will abort the entire test. Does not affect expect-exception.

test::selecting

[syntax] (define-for-syntax test::selecting FLAG)

test::selecting controls the test selective evaluation option. Default is #f.

test::limiting

[syntax] (define-for-syntax test::limiting FLAG)

test::limiting controls the test resource limit option. Default is #f.

Multiline String Construction

Due to a bug the Chicken multiline string forms cannot be used. This macro is provided as a workaround. For use in a test-file.

MULTILINE-STRING

[syntax] (MULTILINE-STRING STRING ...)

Yes, the macro name is uppercase.

STRING is any scheme string. The strings are suffixed with a newline and concatenated.

Immediate Test

This macro will evaluate in a left to right fashion the clauses inside it and output the results. The clauses are a sequence of (Expression Expectation) forms. A clause is evaluated as (expect-equal EXPRESSION EXPECTED). For use with a REPL (Read-Eval-Print-Loop).

The results are not kept and no other forms are allowed in the body.

test

[syntax] (test (EXPECTED EXPRESSION) ...)

EXPRESSION is any scheme expression.

EXPECTED is any scheme expression, the expected result of EXPRESSION.

Test Procedure Definition

All test forms are grouped into a test procedure. Test forms cannot be used outside of a test procedure without generating an error.

define-test

[syntax] (define-test PROCEDURE-VARIABLE [TEST-NAME] [(initial FORM ...)] [(final FORM ...)] CLAUSE ...)
[syntax] (define-test (PROCEDURE-VARIABLE VARIABLE ...) [TEST-NAME] [(initial FORM ...)] [(final FORM ...)] CLAUSE ...)

Defines a variable, PROCEDURE-VARIABLE, with a TEST-PROCEDURE value.

TEST-NAME must evaluate to a Scheme atom. Must be unique in the test-file. When missing a string name is formed from the PROCEDURE-VARIABLE.

VARIABLE is a lambda variable, a formal argument to the test-procedure.

When supplied the (initial FORM ...) will cause each FORM to be expanded at the beginning of the test procedure. FORM can be any Scheme form legal at the beginning of a lambda body. The initial section will be evaluated before the first CLAUSE. If define forms occur in this section, the variables will be in scope for the whole test.

When supplied the (final FORM ...) will cause each FORM to be expanded at the end of the test procedure. FORM can be any Scheme form legal at the end of a lambda body. The final section will be evaluated after the last CLAUSE and before the destructor is activated!

The forms in the initial and final sections do not need to return test result objects; they are not considered part of the test body. In other words, a FORM is not a CLAUSE.

CLAUSE is any of the test API elements. Actually CLAUSE can be any Scheme FORM but be careful. A CLAUSE must return a test result object. Unless this can be guaranteed for a FORM a runtime error can be generated.

The DTOR-NAME will be test-dtor. See test-suite.

The ESCR-NAME will be test-escr. See test-suite.

Can be called directly, as (PROCEDURE-VARIABLE) or (PROCEDURE-VARIABLE ARGUMENT ...), and returns a test-result.

A parameterized test definition should be complemented by one or more test declarations, a declare-test form. Otherwise the test cannot be invoked by test::run.

declare-test

[syntax] (declare-test PROCEDURE-VARIABLE TEST-NAME ARGUMENT ...)

Declares an actual test for the PROCEDURE-VARIABLE.

ARGUMENT is a scheme object. An actual argument to the corresponding test-procedure formal argument.

TEST-NAME must evaluate to a Scheme atom.

Must occur lexically after the PROCEDURE-VARIABLE definition in the test-file!

An actual-test is not a test-procedure and cannot be called directly. The actual-test is tied to the PROCEDURE-VARIABLE, and can only be invoked by test::run.

Test Procedure Operations

TEST-IDENTIFIER is either a PROCEDURE-VARIABLE bound to a TEST-PROCEDURE or a TEST-NAME. When a TEST-NAME the corresponding TEST-PROCEDURE will be substituted.

test::procedure?

[procedure] (test::procedure? OBJECT)

Is the OBJECT a defined test-procedure object?

test::formal-arguments

[procedure] (test::formal-arguments TEST-IDENTIFIER)

Returns the formal argument list of the test-procedure.

test::declarations

[procedure] (test::declarations TEST-IDENTIFIER)

Returns a list of (test-procedure actual-test-name actual-test-arguments) for the test-procedure.

TEST-IDENTIFIER should identify a parameterized test-procedure.

test::procedure

[procedure] (test::procedure TEST-IDENTIFIER)

Returns a test-procedure or #f.

test::name

[procedure] (test::name TEST-IDENTIFIER)

Returns the test-name of the test-procedure.

test::procedure-variable

[procedure] (test::procedure-variable TEST-IDENTIFIER)

Returns the defined symbol of the test-procedure.

test::styler

[procedure] (test::styler TEST-IDENTIFIER)

Returns the default output style procedure for the test-procedure, or #f when no default styler is set.

test::styler-set!

[procedure] (test::styler-set! TEST-IDENTIFIER [STYLER])

Sets the default output style procedure for the test-procedure. When missing or #f then no default styler is set.

test::tolerance

[procedure] (test::tolerance TEST-IDENTIFIER)

Returns the current in-exact number comparison tolerance of the test-procedure.

test::tolerance-set!

[procedure] (test::tolerance-set! TEST-IDENTIFIER TOLERANCE)

Sets the current in-exact number comparison tolerance of the test-procedure to TOLERANCE.

test::reset!

[procedure] (test::reset! TEST-IDENTIFIER)

Sets the state of the test-procedure to the initial settings.

test::structure

[procedure] (test::structure TEST-IDENTIFIER)

Returns the internal structure of test-procedure as a tree.

The tree is the test hierarchy, where each node of the tree is (suite | case | expect TEST-NAME ...), where ... are any sub-trees.

The top-level suite element value refers to the test-procedure.

test::add-filter!

[procedure] (test::add-filter! TEST-IDENTIFIER FILTER-PROCEDURE)

Places FILTER-PROCEDURE as the head of the result filter chain for the test-procedure.

FILTER-PROCEDURE must be a procedure of one argument, a test result object, and it must return a test result object. Not necessarily the same one.

Filtering occurs during test element evaluation.

test::remove-filter!

[procedure] (test::remove-filter! TEST-IDENTIFIER FILTER-PROCEDURE)

Removes FILTER-PROCEDURE from the result filter chain for the test-procedure.

test::add-echo!

[procedure] (test::add-echo! TEST-IDENTIFIER [PORT] [KEY VALUE ...])

Causes all test results for the test-procedure to be passed to the echo-procedure. The signature of an echo-procedure is (result-object port association-list -> unspecified). The association-list holds options for the echo-procedure.

When PORT missing the (current-output-port) is assumed.

The PORT is unique. Only one echo per PORT.

KEY is a symbol. The reserved keys are:

echo-procedure, where the VALUE is a procedure with an echo-procedure signature.
include-expectation-specifics, where the VALUE is a boolean indicating whether the value fields for an expectation are to be echoed, default is #f.

When the KEY echo-procedure is missing the default echo-procedure is used. The test results are echoed to the PORT as an association list. One list per line.

The echo-procedure is invoked after each test element evaluation. As such it must deal with test-container result objects in an incomplete state. Specifically the result-list field will be #f when a test-container is entered. It also must deal with ignore-results, which are stripped from the test-container result-list upon exit from the container.

test::remove-echo!

[procedure] (test::remove-echo! TEST-IDENTIFIER [PORT])

Removes the PORT echo for the test-procedure.

test::set-echo-option!

[procedure] (test::set-echo-option! TEST-IDENTIFIER [PORT] [KEY VALUE ...])

Modifies the echo specification for the specified PORT.

The PORT, KEY, and VALUE parameters have the same interpretation as the test::add-echo! procedure.

test::run-mode

[procedure] (test::run-mode)

Returns the test run mode, either batch, interactive or driven.

test::selection-mode

[procedure] (test::selection-mode TEST-IDENTIFIER)

Returns the test selection mode, either take or skip.

When the mode is take then unless a test is to be specifically skipped, it is taken.

When the mode is skip then unless a test is to be specifically taken, it is skipped.

The test procedure is always taken.

test::select!

[procedure] (test::select! SELECTION-MODE TEST-IDENTIFIER [TEST-NAME | (TEST-TYPE TEST-NAME)] ...)

SELECTION-MODE is one of 'take or 'skip.

TEST-TYPE is one of 'suite, 'case, or 'expect.

Select the test element(s) identified by (TEST-TYPE TEST-NAME) of the test-procedure. When only TEST-NAME is provided any test type will match.

When no tests are supplied all existing selections are removed and the test selection mode is set to the SELECTION-MODE.

(test::select! 'skip my-test-proc)
;; Now every test in 'my-test-proc' will be skipped.

(test::select! 'take my-test-proc '(case "Top Case") "Foo")
;; Now every test in 'my-test-proc' except the test-case "Top Case",
;; and any test named "Foo".

When a test container is selected all contained tests are similarly selected.

Independent of the (skip CLAUSE ...) form.

test::selections

[procedure] (test::selections TEST-IDENTIFIER)

Returns a list of the test selections, or #f when there are no selections.

test::resource-limit

[procedure] (test::resource-limit TEST-IDENTIFIER RLIMIT)

Returns the current value of RLIMIT for the test-procedure. The return value will either be a number or #f, indicating no set limit.

RLIMIT is one of 'cpu, 'fsize, 'nofile, 'nproc, 'data.

Only operational on *NIX platforms.

test::resource-limit-set!

[procedure] (test::resource-limit-set! TEST-IDENTIFIER RLIMIT VALUE)

Sets the current value of RLIMIT for the test-procedure to VALUE.

RLIMIT is one of 'cpu, 'fsize, 'nofile, 'nproc, 'data.

VALUE is a positive integer or #f, indicating no set limit.

When a 'cpu or 'fsize limit is reached the test will terminate with a termination result indicating the resource limit exceeded.

When a 'nofile, 'nproc, or 'data limit is reached the test will terminate with a termination result.

Only operational on *NIX platforms.

test::procedures

[procedure] (test::procedures)

Returns a list of all the test-procedures.

test::for-each

[procedure] (test::for-each PROCEDURE [TEST-IDENTIFIER ...])

Calls PROCEDURE with every supplied test procedure, or all test procedures when nothing supplied.

PROCEDURE takes one argument, a test-procedure.

test::map

[procedure] (test::map PROCEDURE [TEST-IDENTIFIER ...])

Calls PROCEDURE with every supplied test procedure, or all test procedures when nothing supplied. Returns a list of the procedure results.

PROCEDURE takes one argument, a test-procedure.

test::forget!

[procedure] (test::forget! [TEST-IDENTIFIER ...])

Causes the specified test-procedures, or all when nothing specified, to be dropped.

Really only for use during test development with a REPL.

test::exit-on-failure?

[procedure] (test::exit-on-failure? [CODE-OR-FLAG])

When CODE-OR-FLAG is supplied sets the early-exit-upon-failure-option. Otherwise returns the current setting.

CODE-OR-FLAG is either #f, to reset the option, or an integer, to set the option.

The initial setting is #f.

test::run

[procedure] (test::run [MESSAGE [TEST-IDENTIFIER ...]])

MESSAGE is some identity for the set of the test-procedures. When missing or #f the process command name is used.

Displays a header with MESSAGE, and outputs the result of calling each test-procedure with the test::styler. When a test::styler is not set a simple success/failure message is given. The test-procedures are called in left to right order of the TEST-IDENTIFIER arguments. When a TEST-IDENTIFIER is not specified all the defined test-procedures are called in the order of definition in the test-file.

For parameterized test-procedures the corresponding test declarations are run.

When (test::exit-on-failure?) and some test fails then testbase will exit with the exit-code.

When interactive mode without a console there is no output by default. See test::add-echo!.

Run Tests

Run test-procedures and output test results.

run-test

[syntax] (run-test [MESSAGE] [FORM ...])

Usually the last expression in a test file. Performs command-line processing.

MESSAGE is some identity for the test. When missing the process command name is used. Use "" for no message.

FORM ... are instances of Scheme forms. When missing (test::run MESSAGE) is used.

This macro will check the process command line for a batch mode (-b or --batch) or interactive mode (-i --interactive) option. When interactive mode without an attached console the mode is called driven. The default is batch mode.

When batch mode the FORM ... will be evaluated. When FORM ... are missing then every test-procedure is run with test result output using the test-procedure styler. See test::run.

When interactive mode a REPL will be entered.

When driven mode a REPL will be entered. Test results will be reported as for test::add-echo!.

Test macros are not automatically available in the REPL. They must be explicitly loaded.

Test Macros

Test macros are used to generate a test result object. A test procedure body is composed of a sequence of test macros.

The test macros are useful only within a test procedure or a test container. A runtime error will result when used outside of a test procedure.

Test Containers

Test containers are test forms that group test forms. Test container are available in long and short forms. The short form provides fewer options. A test container imposes an evaluation termination mode on the contained tests. Test containers may nest.

test-suite

[syntax] (test-suite TEST-NAME DTOR-NAME ESCR-NAME [(warn MESSAGE)] [((VARIABLE VALUE) ...)] CLAUSE ...)
[syntax] (test/suite TEST-NAME [((VARIABLE VALUE) ...)] CLAUSE ...)

Evaluates all contained test forms, CLAUSE, in a left to right order, thru any failures.

TEST-NAME must evaluate to a Scheme atom. Note that #f is interpreted as a missing name. Further, #f is assumed when a name is missing. Thus, the name of all unnamed test elements is the same - #f.

DTOR-NAME is an unquoted symbol for an automatic destructor object that gets called when the test suite completes for any reason. This symbol is bound to a destructor object and is available to you in the CLAUSE ... section of the test suite. See below for the description of the destructor object interface.

test/suite-dtor is the test/suite destructor name.

ESCR-NAME is an unquoted symbol for an escape procedure available in the body of the test suite. Usually this escape procedure is passed to (terminate ...) which calls it for you and performs other tasks. It is not recommended to call the escape procedure directly.

test/suite-escr is the test/suite escape procedure name.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the test suite. The warn function name is actually a syntax reserved word in the macro.

((VARIABLE VALUE) ...) are let-style bindings that you may create and exist in the lexical scope of the test suite.

CLAUSE ... are uses of a test container, annotation, or expectation macros. The clauses are evaluated in a left to right fashion.

While you may use the expectation macros directly in a test suite, doing so has a drawback. If the expectation fails, the test suite macro will continue evaluating until all clauses are evaluated or the escape procedure mechanism is activated. This is different from a test-case macro where upon discovery of a failed expectation, evaluation stops immediately.

test-case

[syntax] (test-case TEST-NAME DTOR-NAME ESCR-NAME [(warn MESSAGE)] [((VARIABLE VALUE) ...)] CLAUSE ...)
[syntax] (test/case TEST-NAME (warn MESSAGE)] [((VARIABLE VALUE) ...)] CLAUSE ...)

Evaluates all contained test forms, CLAUSE, in a left to right order, until failure.

TEST-NAME must evaluate to a Scheme atom.

DTOR-NAME is an unquoted symbol for an automatic destructor object that gets called when the test case completes for any reason. This symbol is bound to a destructor object and is available to you in the CLAUSE ... section of the test case. See below for the description of the destructor object interface.

test/case-dtor is the test/case destructor name.

ESCR-NAME is an unquoted symbol for an escape procedure available in the body of the test case. Usually this escape procedure is passed to (terminate ...) which calls it for you and performs other tasks. It is not recommended to call the escape procedure directly.

test/case-escr is the test/case escape procedure name.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the test case. The warn function name is actually a syntax reserved word in the macro.

((VARIABLE VALUE) ...) are let-style bindings that you may create and exist in the lexical scope of the test case.

CLAUSE ... are uses of a test container, annotation, or expectation macros. The clauses are evaluated in a left to right fashion.

Note: Upon discovery of a failed expectation, the test case stops its evaluation and returns with the test results up thru the failed expectation. This behavior can be changed using the test-case-evaluate-thru-failure macro. Reverts to default at the start of each test-case.

test-collect

[syntax] (test-collect TEST-NAME DTOR-NAME ESCR-NAME [(warn MESSAGE)] [((VARIABLE VALUE) ...)] EXPRESSION ...)
[syntax] (test/collect TEST-NAME (warn MESSAGE)] [((VARIABLE VALUE) ...)] EXPRESSION ...)

A special version of test-case. Since every test form must evaluate to a test-result the use of looping forms to generate test values is very difficult. The collect container provides a method to gather the results of such generated tests.

TEST-NAME must evaluate to a Scheme atom.

DTOR-NAME is an unquoted symbol for an automatic destructor object that gets called when the test collect completes for any reason. This symbol is bound to a destructor object and is available to you in the CLAUSE ... section of the test collect. See below for the description of the destructor object interface.

test/collect-dtor is the test/collect destructor name.

ESCR-NAME is an unquoted symbol for an escape procedure available in the body of the test collect. Usually this escape procedure is passed to (terminate ...) which calls it for you and performs other tasks. It is not recommended to call the escape procedure directly.

test/collect-escr is the test/collect escape procedure name.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the test collect. The warn function name is actually a syntax reserved word in the macro.

((VARIABLE VALUE) ...) are let-style bindings that you may create and exist in the lexical scope of the test collect.

EXPRESSION is either an arbitrary Scheme expression or the form (collect-test CLAUSE), where CLAUSE is a use of a test container, annotation, or expectation macro. The expressions are evaluated in a left to right fashion.

The collect-test macro "appends" the test result of the CLAUSE to the results of the enclosing test collect container.

Note: Upon discovery of a failed expectation, the test collect stops its evaluation and returns with the test results up thru the failed expectation. This behavior can be changed using the test-case-evaluate-thru-failure macro. Reverts to default at the start of each test-case.

Test Flags

These macros set test execution flags.

test-case-evaluate-thru-failure

[syntax] (test-case-evaluate-thru-failure BOOLEAN)

This macro will set the test-case evaluation continuation flag.

The starting value is #f.

test-timing

[syntax] (test-timing BOOLEAN)

This macro will set the expectation timing flag.

The starting value is #f.

Note that if timing expansion is turned off setting this flag has no effect.

test-evaluation-mode?

[syntax] (test-evaluation-mode?)

This macro will expand into a test for the evaluation mode.

Potentially useful within test procedure initial and final forms.

Test Binding Forms

test-letrec

[syntax] (test-letrec ((VARIABLE VALUE) ...) CLAUSE ...)

Defines VARIABLE, in a new lexical scope, as the result of evaluating EXPRESSION.

CLAUSE ... are as for test-suite. Evaluated in the scope of the definitions.

The last CLAUSE ... in the clause body must be a test form of some kind, such as an expectation.

test-let*

[syntax] (test-let* ((VARIABLE VALUE) ...) CLAUSE ...)

Defines VARIABLE, in a new lexical scope, as the result of evaluating EXPRESSION.

CLAUSE ... are as for test-suite. Evaluated in the scope of the definitions.

The last CLAUSE ... in the clause body must be a test form of some kind, such as an expectation.

test-let

[syntax] (test-let ((VARIABLE VALUE) ...) CLAUSE ...)

Defines VARIABLE, in a new lexical scope, as the result of evaluating EXPRESSION.

CLAUSE ... are as for test-suite. Evaluated in the scope of the definitions.

The last CLAUSE ... in the clause body must be a test form of some kind, such as an expectation.

Test Expectations

An expectation at its core simply evaluates its arguments and check to see if it matches the expectation. The positive or negative result is encapsulated, along with other things such as the unevaluated expressions being checked and some messages supplied with each expectation into a particular type of black box object that one can query with the appropriate API calls (detailed below).

Expectations all have a descriptive message that can be bound to them, along with an optional warning syntax detailed below. A design decision was made to supply expectation macros for the usual types of expectations a user needs because this reduced the abstractness of an expectation into something more manageable.

Expectations are normally evaluated with an exception catcher. An unexpected exception is treated as expectation failure. The option may be defeated by redefining test::if-catching.

Any exceptions in setup or teardown expressions will be caught and treated as an abnormal termination of the entire test.

insist

[syntax] (insist [TEST-NAME] [(warn MESSAGE)] EXPRESSION => EXPECTED)
[syntax] (insist [TEST-NAME] [(warn MESSAGE)] EXPRESSION (=> PREDICATE) EXPECTED)

This expectation checks to see if the evaluated expression passed to it meets the predicate condition.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

PREDICATE is a procedure of two arguments to perform the test. The default is equal?.

EXPECTED is evaluated and represents the value the EXPRESSION must match via PREDICATE in order for this expectation to return a positive result.

EXPRESSION should return a value meeting the conditions of the predicate.

expect-unary

[syntax] (expect-unary [TEST-NAME] [(warn MESSAGE)] KIND PREDICATE EXPRESSION)

This expectation checks to see if the evaluated expression passed to it meets the predicate condition. Allows unary and binary predicates.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

KIND is a string stating the kind of test.

PREDICATE is a procedure of one argument to perform the test.

EXPRESSION should return a value meeting the conditions of the predicate.

expect-binary

[syntax] (expect-binary [TEST-NAME] [(warn MESSAGE)] KIND PREDICATE EXPECTED EXPRESSION)

This expectation checks to see if the evaluated expression passed to it meets the predicate condition. Allows unary and binary predicates.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

KIND is a string stating the kind of test.

PREDICATE is a procedure of two arguments to perform the test.

EXPECTED is evaluated and represents the value the EXPRESSION must match via PREDICATE in order for this expectation to return a positive result.

EXPRESSION should return a value meeting the conditions of the predicate.

expect-zero

[syntax] (expect-zero [TEST-NAME] [(warn MESSAGE)] EXPRESSION)

This expectation checks to see if the evaluated expression passed to it is numerically equal to zero.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPRESSION should return a number.

expect-nonzero

[syntax] (expect-nonzero [TEST-NAME] [(warn MESSAGE)] EXPRESSION)

This expectation checks to see if the evaluated expression passed to it is numerically not equal to zero.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPRESSION should return a number.

expect-positive

[syntax] (expect-positive [TEST-NAME] [(warn MESSAGE)] EXPRESSION)

This expectation checks to see if the evaluated expression passed to it is a positive value greater than zero.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPRESSION should return a number.

expect-negative

[syntax] (expect-negative [TEST-NAME] [(warn MESSAGE)] EXPRESSION)

This expectation checks to see if the evaluated expression passed to it is a negative value less than zero.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPRESSION should return a number.

expect-near

[syntax] (expect-near [TEST-NAME] [(warn MESSAGE)] EXPECTED [TOL] EXPRESSION)

This expectation checks to see if (< (abs (- EXPECTED EXPRESSION)) TOL))) is true.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPECTED is evaluated and represents the value the EXPRESSION must be "near" to in order for this expectation to return a positive result.

EXPRESSION should return a number.

TOL is a single expression which, when evaluated, must return a tolerance value, a small inexact number. When missing the current tolerance value is used, usually 0.0001.

The (expect-near EXPECTED TOL EXPRESSION) variant is not supported!

expect-true

[syntax] (expect-true [TEST-NAME] [(warn MESSAGE)] EXPRESSION)

This expectation checks to see if the evaluated expression passed to it is the value #t.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPRESSION should return #t.

expect-false

[syntax] (expect-false [TEST-NAME] [(warn MESSAGE)] EXPRESSION)

This expectation checks to see if the evaluated expression passed to it is the value #f.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPRESSION should return #f.

expect-success

[syntax] (expect-success [TEST-NAME] [(warn MESSAGE)] EXPRESSION)

This expectation checks to see if the evaluated expression passed to it is not the value #f, or a non-error exception occurred.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPRESSION should return #t, or generate a non-error exception.

expect-failure

[syntax] (expect-failure [TEST-NAME] [(warn MESSAGE)] EXPRESSION)

This expectation checks to see if the evaluated expression passed to it is the value #f, or an error exception occurred.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPRESSION should return #f, or generate an error exception.

expect-not-false

[syntax] (expect-not-false [TEST-NAME] [(warn MESSAGE)] EXPRESSION)

This expectation checks to see if the evaluated expression passed to it is not the value #f.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPRESSION should not return #f, any other value is accepted.

expect-not-null

[syntax] (expect-not-null [TEST-NAME] [(warn MESSAGE)] EXPRESSION)

This expectation checks to see if the evaluated expression passed to it is not the value #f.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPRESSION should not return #f, any other value is accepted.

expect-eq

[syntax] (expect-eq [TEST-NAME] [(warn MESSAGE)] EXPECTED EXPRESSION)

This expectation checks to see if (eq? EXPECTED EXPRESSION) is true.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPECTED is evaluated and represents the value the EXPRESSION must be eq? to in order for this expectation to return a positive result.

EXPRESSION is a single expression which, when evaluated must return an object where an eq? of this result and the EXPECTED expression is #t.

The result object this macro produce shall contain the unevaluated EXPRESSION expression as a field, but not an unevaluated EXPECTED expression.

expect-eqv

[syntax] (expect-eqv [TEST-NAME] [(warn MESSAGE)] EXPECTED EXPRESSION)

This expectation checks to see if (eqv? EXPECTED EXPRESSION) is true.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPECTED is evaluated and represents the value the EXPRESSION must be eqv? to in order for this expectation to return a positive result.

EXPRESSION is a single expression which, when evaluated must return an object where an eqv? of this result and the EXPECTED expression is #t.

The result object this macro produce shall contain the unevaluated EXPRESSION expression as a field, but not an unevaluated EXPECTED expression.

expect-equal

[syntax] (expect-equal [TEST-NAME] [(warn MESSAGE)] EXPECTED EXPRESSION)

This expectation checks to see if (equal? EXPECTED EXPRESSION) is true.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPECTED is evaluated and represents the value the EXPRESSION must be equal? to in order for this expectation to return a positive result.

EXPRESSION is a single expression which, when evaluated must return an object where an equal? of this result and the EXPECTED expression is #t.

The result object this macro produce shall contain the unevaluated EXPRESSION expression as a field, but not an unevaluated EXPECTED expression.

expect-unary/values

[syntax] (expect-unary/values [TEST-NAME] [(warn MESSAGE)] KIND (PREDICATE ...) EXPRESSION)

This expectation checks to see if the evaluated multi-valued expression passed to it meets the predicate condition. Allows unary and binary predicates.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

KIND is a string stating the kind of test.

PREDICATE is a procedure taking one argument and returning #t or #f.

EXPRESSION has a multi-valued result.

expect-near/values

[syntax] (expect-near/values [TEST-NAME] [(warn MESSAGE)] KIND (PREDICATE ...) VALUES EXPRESSION)

This expectation checks to see if the evaluated multi-valued expression passed to it meets the predicate condition. Allows unary and binary predicates.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

KIND is a string stating the kind of test.

PREDICATE is a procedure taking two arguments and returning #t or #f.

VALUES are the expected values.

EXPRESSION has a multi-valued result.

expect-near/values

[syntax] (expect-near/values [TEST-NAME] [(warn MESSAGE)] EXPECTED [TOL] EXPRESSION)

This expectation checks to see if (< (abs (- EXPECTED[i] EXPRESSION[i])) TOL))) is true.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPECTED is evaluated and represents the values the EXPRESSION must be "near" to in order for this expectation to return a positive result.

EXPRESSION should return one or more inexact or exact number.

TOL is a single expression which, when evaluated, must return a tolerance value, a small inexact number. When missing the current tolerance value is used, usually 0.0001.

The (expect-near/values EXPECTED TOL EXPRESSION) variant is not supported!

expect-eq/values

[syntax] (expect-eq/values [TEST-NAME] [(warn MESSAGE)] VALUES EXPRESSION)

This expectation checks to see if the result of the evaluated multi-valued expression passed to it is element-wise eq? to the VALUES list. The count of returned multiple values must match the length of the VALUES list.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

VALUES is the expected values list.

EXPRESSION has a multi-valued result.

expect-eqv/values

[syntax] (expect-eqv/values [TEST-NAME] [(warn MESSAGE)] VALUES EXPRESSION)

This expectation checks to see if the result of the evaluated multi-valued expression passed to it is element-wise eqv? to the VALUES list. The count of returned multiple values must match the length of the VALUES list.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

VALUES is the expected values list.

EXPRESSION has a multi-valued result.

expect-equal/values

[syntax] (expect-equal/values [TEST-NAME] [(warn MESSAGE)] VALUES EXPRESSION)

This expectation checks to see if the result of the evaluated multi-valued expression passed to it is element-wise equal? to the VALUES list. The count of returned multiple values must match the length of the VALUES list.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

VALUES is the expected values list.

EXPRESSION has a multi-valued result.

expect-exception

[syntax] (expect-exception [TEST-NAME] [(warn MESSAGE)] EXCEPTION-PATTERN EXPRESSION)

This expectation checks to see if the evaluated expression passed to it signals an exception that matches the supplied exception pattern EXCEPTION-PATTERN.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXCEPTION-PATTERN is a specialized logical expression describing the expected exception, a condition as defined by SRFI-12.

EXPRESSION should signal an exception matching EXCEPTION-PATTERN.

 EXCEPTION-PATTERN : COND-EXN
                   | SIMP-EXN
                   | ()
 
 SIMP-EXN : KINDKEY
          | (property KINDKEY PROP-EXPR ...)
          | (SIMP-EXN ...)
 
 COND-EXN : (and EXN ...)
          | (or EXN ...)
          | (not EXN)
 
 PROP-EXPR : COND-PROP-EXPR
           | SIMP-PROP-EXPR
           | ()
 
 SIMP-PROP-EXPR : PROPKEY
                | (PROPKEY ...)
                | (PROPKEY VALUE)
                | (PREDICATE PROPKEY VALUE)
 
 COND-PROP-EXPR : (and PROP-EXPR ...)
                | (or PROP-EXPR ...)
                | (not PROP-EXPR)

An example:

(define-test ...
  ...
  (test-case "exn1" d e (
      [exn1 (make-composite-condition
              (make-property-condition 'abc 'cbs "pbs")
              (make-property-condition 'foo 'bar "zip"))])
  
    (expect-exception "1" (foo abc) (signal exn1))
  
      ;; These are the same test, the 2nd has an explicit predicate
    (expect-exception "2" ((property foo (bar "zip"))
                           (property abc (cbs "pbs"))) (signal exn1))
    (expect-exception "3" ((property foo (equal? bar "zip"))
                           (property abc (equal? cbs "pbs"))) (signal exn1))
  )
  ...
)

expect-ec

[syntax] (expect-ec [TEST-NAME] [(warn MESSAGE)] QUALIFIER ... EXPECTED [=> PREDICATE] EXPRESSION)

This expectation checks to see if (PREDICATE EXPECTED EXPRESSION) is true for every state produced by the set of QUALIFIER.

TEST-NAME must evaluate to a Scheme atom.

(warn MESSAGE) allows you to specify a warning object, usually a string, that gets associated with the expectation. The warn function name is actually a syntax reserved word in the macro.

EXPRESSION and EXPECTED are expressions.

PREDICATE is a two argument procedure returning a boolean, equal? is used when missing.

QUALIFIER is a SRFI-42 <qualifier>. An eager comprehension implementation, as described by SRFI-42 must be provided to use this expectation.

expect-set!

[syntax] (expect-set! [TEST-NAME] VARIABLE EXPRESSION)

Evaluates EXPRESSION as in expect-success. If successful the variable VARIABLE is bound to the result of evaluating the EXPRESSION.

EXPRESSION cannot evaluate to #f or a condition.

Define New Expectations

An API to define new expectation macros. May only be used at the toplevel.

define-expect-unary

[syntax] (define-expect-unary PREDICATE [SUFFIX [KIND]])

Defines a new unary expectation.

PREDICATE is the name of a procedure of one argument to perform the test.

SUFFIX is a symbol to create the expectation macro name. When missing the PREDICATE is used, without a ? suffix, if any.

KIND is a string stating the kind of test. When missing the string form of the SUFFIX is used.

define-expect-binary

[syntax] (define-expect-binary PREDICATE [SUFFIX [KIND]])

Defines a new binary expectation.

PREDICATE is the name of a procedure of two arguments to perform the test.

SUFFIX is a symbol to create the expectation macro name. When missing the PREDICATE is used, without a ? suffix, if any.

KIND is a string stating the kind of test. When missing the string form of the SUFFIX is used.

define-expect-unary/values

[syntax] (define-expect-unary/values PREDICATE [SUFFIX [KIND]])

Defines a new multivalued unary expectation.

PREDICATE is the name of a procedure of one list argument to perform the test.

SUFFIX is a symbol to create the expectation macro name. When missing the PREDICATE is used, without a ? suffix, if any.

KIND is a string stating the kind of test. When missing the string form of the SUFFIX is used.

define-expect-binary/values

[syntax] (define-expect-binary/values PREDICATE [SUFFIX [KIND]])

Defines a new multivalued binary expectation.

PREDICATE is the name of a procedure of two list arguments to perform the test.

SUFFIX is a symbol to create the expectation macro name. When missing the PREDICATE is used, without a ? suffix, if any.

KIND is a string stating the kind of test. When missing the string form of the SUFFIX is used.

define-expect-nary

[syntax] (define-expect-nary (SUFFIX ARG ...) EXPR ...)

Defines a new n-ary expectation with the name expect-SUFFIX.

(expect-SUFFIX MESSAGE [(warning WARNING)] VAL ...). Unlike other expectations either the MESSAGE or a WARNING must be supplied.

SUFFIX is a symbol.

ARG ... are the parameter variable symbols.

EXPR ... is the body, which must return #f for failure.

Testeez Synonyms

Syntax forms mimicking some of the testeez test framework forms. Expanded into the corresponding expect-* macros.

The formal argument names match those in the testeez documentation.

Following the testeez semantics all test forms are always evaluated as having possible multiple valued returns.

test-eval

[syntax] (test-eval [TEST-NAME] EXPRESSION)

test/eq

[syntax] (test/eq [TEST-NAME] EXPRESSION EXPECTED)

test/eqv

[syntax] (test/eqv [TEST-NAME] EXPRESSION EXPECTED)

test/equal

[syntax] (test/equal [TEST-NAME] EXPRESSION EXPECTED)

Test Setup (Side-Effect)

The side effecting evaluates all of its arguments as in a (begin ...) form, it returns a result that is completely ignored by the system and unavailable to the output analysis code.

side-effect

[syntax] (side-effect CLAUSES)

This macro expands into a begin form with the clauses in order. When it finishes evaluating them, a result is returned that is silently ignored by the testbase system.

Usually this is used in conjunction with (set! ...) or with complicated situations where a lot of setup work must happen for an expectation to be performed.

Define forms cannot be expressed here.

setup

[syntax] (setup CLAUSES)

A synonym for side-effect.

Test Teardown (Destructor)

The destructor object allows for you to create helper functions which clean up in case of aborting from a test container.

For example, suppose you are testing whether or not file writing to a file works correctly in a test case, so, you'd perform an expectation to open the file, and then queue a function in the destructor to remove the file, and then perform the expectation of the write. If the write (or subsequent) expectation fails, then the test case will automatically invoke the helper cleanup function specified in the destructor object that removes the file.

destructor-atexit!

[syntax] (destructor-atexit! [DTOR-NAME] CALL ...)
[syntax] (destructor-atexit! DTOR-NAME PROC ARGUMENT ...)

DTOR-NAME is the destructor to use, test-dtor when missing.

A CALL is a list where the first element is a PROC and the tail elements are the argument list; i.e. (PROC ARGUMENT ...).

Each CALL is added to the destructor queue, promised to be invoked when the DTOR-NAME destructor object is activated.

The second form will insert a promise to invoke (PROC ARGUMENT ...) into a queue in the DTOR-NAME destructor object.

Multiple invocations of this API call will continue to queue up (PROC ARGS ...) promises indefinitely.

Note: The PROC and ARGUMENT ... parameters are evaluated at the point of enqueue; i.e. the current lexical, and dynamic, environment of the macro expansion. This means a variable referenced will have this value when the destructor is activated, and not the "current" value! So no "forward" references.

This macro returns a special ignore-result type that is ignored by the testbase system.

teardown

[syntax] (teardown [DTOR-NAME] (FUNC ARGUMENT ...) ...)

A (FUNC ARGUMENT ...) for is also refered to as a CALL.

A synonym of destructor-atexit!.

[syntax] (destructor-activate! [DTOR-NAME])

DTOR-NAME is the destructor to use, test-dtor when missing.

This macro will call, in order of queueing, all the promises embedded into this destructor object, and then delete the queue. The destructor is always called at the completion of a test container; so be careful that the destructor object doesn't contain anything harmful.

This macro returns a special ignore-result type that is ignored by the testbase system.

destructor-clear!

[syntax] (destructor-clear! [DTOR-NAME])

DTOR-NAME is the destructor to use, test-dtor when missing.

This macro completely removes all of the promises associated with the destructor object DTOR-NAME.

This macro returns a special ignore-result type that is ignored by the testbase system.

destructor-dump

[syntax] (destructor-dump [DTOR-NAME])

DTOR-NAME is the destructor to use, test-dtor when missing.

This macro, mostly used for debugging purposes, prints out a simple representation of the queued atexit functions to the current port.

This macro returns a special ignore-result type that is ignored by the testbase system.

Gloss

The purpose of the gloss API is to allow the author of a test suite the ability to record messages into the result tree purely for documentation purposes.

gloss

[syntax] (gloss MESSAGE [(warn WARNING)])

MESSAGE can be any scheme object, though usually it is a string.

(warn WARNING) allows you to specify a warning object, usually a string, that gets associated with the gloss. The warn function name is actually a syntax reserved word in the macro.

Skip

The purpose of the skip API is to allow the author of a test suite to completely skip evaluation of a set of expressions.

skip

[syntax] (skip MESSAGE [(warn WARNING)] CLAUSES)

MESSAGE can be any scheme object, though usually it is a string.

(warn WARNING) allows you to specify a warning object, usually a string, that gets associated with the gloss. The warn function name is actually a syntax reserved word in the macro.

CLAUSES can be more than one expression (as in a lambda form) that does NOT get evaluated at any time.

Termination

When executing in a test suite or a test case, one might discover some catastrophic failure of such proportions that it is utterly impossible to continue executing the test case or test suite. When that happens you can use the termination facility to exit the test case or test suite. Of course, no more expressions will be evaluated in the scope of the termination. It is recommended that you use this method of terminating the test procedure, test case or test suite evaluation since it wraps some contextual information up into the termination result so you can figure out what happened (and where) later when analyzing the result tree.

terminate

[syntax] (terminate MESSAGE)
[syntax] (terminate MESSAGE FINDING)
[syntax] (terminate ESCR-NAME MESSAGE)
[syntax] (terminate ESCR-NAME MESSAGE FINDING)

This is the recommended termination method for a test procedure, test case, or a test suite.

ESCR-NAME is the name of the termination procedure that was specified in a test procedure, test case or test suite. You may pass any test suite or test case termination function available to you in the lexical scope in which you call this function. The termination will take effect in the scope of the created termination function.

MESSAGE can be any scheme object, though usually it is a string.

FINDING can be any scheme object, though usually it is a boolean.

Todo

The purpose of the todo API is to allow the author of a test suite the ability to record into the result tree for later analysis that something still needs to be done. This way you can count/manipulate this information at a later date.

todo

[syntax] (todo MESSAGE [(warn WARNING)])

MESSAGE can be any scheme object, though usually it is a string.

(warn WARNING) allows you to specify a warning object, usually a string, that gets associated with the todo. The warn function name is actually a syntax reserved word in the macro.

Test Result Object

Test procedures, test containers, expectations, and helper macros (gloss, todo, etc) all return an object that contains the results and other various aspects of the action performed which ultimately get wired together to form the result tree. This collection of functions forming the rest of the testbase API allows manipulation of these results in an abstracted way as to allow changing of the representation in the future.

Any Test Result Object

This section contains a few functions whose purpose is to simplify certain kinds of manipulations of result objects. Will generate an error for an invalid result object.

test::result?

[procedure] (test::result? RESULT-OBJ)

This function will return #t if RESULT-OBJ is any kind of an evaluated result object. Otherwise, it shall return #f.

test::ignore-result?

[procedure] (test::ignore-result? RESULT-OBJ)

This function will return #t if RESULT-OBJ is an ignore result object. Otherwise, it shall return #f. An ignore object is not considered an evaluated result object.

test::result-passed?

[procedure] (test::result-passed? RESULT-OBJ)

This function will return whether the boolean> finding of any kind of RESULT-OBJ passed to it if it is indeed a true result object.

test::result-finding

[procedure] (test::result-finding RESULT-OBJ)

This function will return the finding of any kind of RESULT-OBJ passed to it if it is indeed a true result object.

The finding is a variant type: boolean | condition.

test::result-name

[procedure] (test::result-name RESULT-OBJ)

This function will return the TEST-NAME of any kind of RESULT-OBJ passed to it if it is indeed a true result object.

test::result-message

[procedure] (test::result-message RESULT-OBJ)

This function will return the MESSAGE of any kind of RESULT-OBJ passed to it if it is indeed a true result object.

test::result-warning?

[procedure] (test::result-warning? RESULT-OBJ)

This function will return whether a warning is defined if RESULT-OBJ is any kind of an evaluated result object.

test::result-warning

[procedure] (test::result-warning RESULT-OBJ)

This function will return the warning of any kind of RESULT-OBJ passed to it if it is indeed a true result object.

test::result-timing?

[procedure] (test::result-timing? RESULT-OBJ)

This function will return whether a timing is defined if RESULT-OBJ is any kind of an evaluated result object.

test::result-timing

[procedure] (test::result-timing RESULT-OBJ)

This function will return the timing values of any kind of RESULT-OBJ passed to it if it is indeed a true result object.

The timing values will be a pair of the milliseconds spent in user code and in system code, for the test corresponding to this test::result-object.

For example, the list (6 . 1) indicates 6 milliseconds spent in user code and 1 millisecond spent in system code.

test::result-timing-message

[procedure] (test::result-timing-message RESULT-OBJ)

This function will return the timing string representation of any kind of RESULT-OBJ passed to it if it is indeed a true result object.

test::result-kind

[procedure] (test::result-kind RESULT-OBJ)

This function will return the kind of any RESULT-OBJ passed to it if it is indeed a true result object.

Test Container Result Object

Test Suite Result Object

If any of these API functions, except test::test-suite-result?, are passed something that isn't a test suite result object, they will return an undefined value.

test::test-suite-result?

[procedure] (test::test-suite-result? RESULT)

If RESULT is a result object from the invocation of a test suite macro, then this function will return #t. Otherwise, it will return #f.

test::test-suite-result-passed?

[procedure] (test::test-suite-result-passed? RESULT)

Returns the boolean result associated with the test suite RESULT object.

test::test-suite-result-finding

[procedure] (test::test-suite-result-finding RESULT)

Returns the finding result associated with the test suite RESULT object.

test::test-suite-result-name

[procedure] (test::test-suite-result-name RESULT)

Returns the message object associated with the test suite RESULT object.

test::test-suite-result-list

[procedure] (test::test-suite-result-list RESULT)

Returns the list of result objects associated with the test suite RESULT object.

test::test-suite-result-warning?

[procedure] (test::test-suite-result-warning? RESULT)

If a warning had been attached to this test suite, this function will return #t, otherwise it will be #f.

test::test-suite-result-warning

[procedure] (test::test-suite-result-warning RESULT)

If a warning had been attached to this test suite, this function will return the warning message object supplied by the user, otherwise it shall return '().

test::test-suite-result-timing?

[procedure] (test::test-suite-result-timing? RESULT)

If a timing had been attached to this test suite, this function will return #t, otherwise it will be #f.

test::test-suite-result-timing

[procedure] (test::test-suite-result-timing RESULT)

If a timing had been attached to this test suite, this function will return the timing pair, otherwise it shall return '(0 . 0).

test::test-suite-results-passed?

[procedure] (test::test-suite-results-passed? RESULT-LIST)

Returns #t if every single contained result in the RESULT-LIST passed, or #f otherwise.

Test Case Result Object

If any of these API functions, except test::test-case-result?, are passed something that isn't a test case result object, they will return an undefined value.

test::test-case-result?

[procedure] (test::test-case-result? RESULT)

If RESULT is a result object from the invocation of a test case macro, then this function will return #t. Otherwise, it will return #f.

test::test-case-result-passed?

[procedure] (test::test-case-result-passed? RESULT)

Returns the boolean result associated with the test case RESULT object.

test::test-case-result-finding

[procedure] (test::test-case-result-finding RESULT)

Returns the finding result associated with the test case RESULT object.

test::test-case-result-name

[procedure] (test::test-case-result-name RESULT)

Returns the message object associated with the test case RESULT object.

test::test-case-result-list

[procedure] (test::test-case-result-list RESULT)

Returns the list of result objects associated with the test case RESULT object.

test::test-case-result-warning?

[procedure] (test::test-case-result-warning? RESULT)

If a warning had been attached to this test case, this function will return #t, otherwise it will be #f.

test::test-case-result-warning

[procedure] (test::test-case-result-warning RESULT)

If a warning had been attached to this test case, this function will return the warning message object supplied by the user, otherwise it shall return '().

test::test-case-result-timing?

[procedure] (test::test-case-result-timing? RESULT)

If a timing had been attached to this test case, this function will return #t, otherwise it will be #f.

test::test-case-result-timing

[procedure] (test::test-case-result-timing RESULT)

If a timing had been attached to this test case, this function will return the timing pair, otherwise it shall return '(0 . 0).

test::test-case-results-passed?

[procedure] (test::test-case-results-passed? RESULT-LIST)

Returns #t if every single contained result in the RESULT-LIST passed, or #f otherwise.

Expectation Result Objects

Procedures used to access the various kinds of expectation result object.

Unary Expectation

These expectations all take the form of passing a single expression to them to see if they match some a priori expectation. If any of these API functions, except test::expect-result?, are passed something that isn't a single clause style expectation result object, they will return 'not-an-expect-result.

test::expect-result?

[procedure] (test::expect-result? RESULT)

If RESULT is a single clause style result object from the invocation of an expectation macro, then this function will return #t. Otherwise, it will return #f.

test::expect-result-finding

[procedure] (test::expect-result-finding RESULT)

Returns the boolean result associated with the single clause style expectation RESULT object.

test::expect-result-kind

[procedure] (test::expect-result-kind RESULT)

This retrieves the "kind" field of a particular single clause style expectation. For example, if you had a result object from an invocation of a (expect-zero "foobar" (- 1 1)) expectation, then the "kind" field of the expectation result object will be the string "zero". Here is a table describing what the "kind" fields are for each kind of single clause style expectation:

Unary Expectation	Associated Specific String
expect-zero	"zero"
expect-nonzero	"nonzero"
expect-true	"true"
expect-false	"false"
expect-not-false	"not-false"
expect-not-null	"not-null"
expect-positive	"positive"
expect-negative	"negative"
expect-unary	`KIND`
expect-unary/values	`KIND`

test::expect-result-name

[procedure] (test::expect-result-name RESULT)

Returns the message object associated with the single clause style expectation RESULT object.

test::expect-result-unevaled

[procedure] (test::expect-result-unevaled RESULT)

Returns the unevaluated expression supplied to a single clause style expectation macro.

test::expect-result-evaled

[procedure] (test::expect-result-evaled RESULT)

Returns the evaluated expression supplied to a single clause style expectation macro.

test::expect-result-warning?

[procedure] (test::expect-result-warning? RESULT)

If a warning had been attached to this expectation, this function will return #t, otherwise it will be #f.

test::expect-result-warning

[procedure] (test::expect-result-warning RESULT)

If a warning had been attached to this expectation, this function will return the warning message object supplied by the user, otherwise it shall return '().

test::expect-result-timing?

[procedure] (test::expect-result-timing? RESULT)

If a timing had been attached to this expectation, this function will return #t, otherwise it will be #f.

test::expect-result-timing

[procedure] (test::expect-result-timing RESULT)

If a timing had been attached to this expectation, this function will return the timing pair, otherwise it shall return '(0 . 0).

Binary Expectation

These expectations all take the form of passing a two expressions, the "left hand side" and the "right hand side" to them to see if they match some a priori equivalence. The left hand side is that which you expect the right hand side to be equivalent. If any of these API functions, except test::expect-equivalence-result?, are passed something that isn't a single clause style expectation result object, they will return 'not-an-expect-equivalence-result.

test::expect-equivalence-result?

[procedure] (test::expect-equivalence-result? RESULT)

If RESULT is a comparison style result object from the invocation of an expectation macro, then this function will return #t. Otherwise, it will return #f.

test::expect-equivalence-result-finding

[procedure] (test::expect-equivalence-result-finding RESULT)

Returns the boolean result associated with the comparison style expectation RESULT object.

test::expect-equivalence-result-kind

[procedure] (test::expect-equivalence-result-kind RESULT)

This retrieves the "kind" field of a particular equivalence style expectation. For example, if you had a result object from an invocation of a (expect-equal? "foobar" 0 (- 1 1)) expectation, then the "kind" field of the expectation result object will be the string "equal". Here is a table describing what the "kind" fields are for each kind of equivalence style expectation:

Binary Expectation	Associated Specific String
expect-eq	"eq"
expect-eqv	"eqv"
expect-equal	"equal"
expect-exception	"exception"
expect-binary	`KIND`
expect/values	`"values"`
expect-eq/values	"eq/values"
expect-eqv/values	"eqv/values"
expect-equal/values	"equal/values"
expect-binary/values	`KIND`

test::expect-equivalence-result-name

[procedure] (test::expect-equivalence-result-name RESULT)

Returns the message object associated with the equivalence style expectation RESULT object.

test::expect-equivalence-result-lhs-evaled

[procedure] (test::expect-equivalence-result-lhs-evaled RESULT)

Returns the evaluated "left hand side" expression supplied to an equivalence style expectation.

test::expect-equivalence-result-rhs-unevaled

[procedure] (test::expect-equivalence-result-rhs-unevaled RESULT)

Returns the unevaluated "right hand side" expression supplied to an equivalence style expectation.

test::expect-equivalence-result-rhs-evaled

[procedure] (test::expect-equivalence-result-rhs-evaled RESULT)

Returns the evaluated "right hand side" expression supplied to an equivalence style expectation.

test::expect-equivalence-result-warning?

[procedure] (test::expect-equivalence-result-warning? RESULT)

If a warning had been attached to this expectation, this function will return #t, otherwise it will be #f.

test::expect-equivalence-result-warning

[procedure] (test::expect-equivalence-result-warning RESULT)

If a warning had been attached to this expectation, this function will return the warning message object supplied by the user, otherwise it shall return '().

test::expect-equivalence-result-timing?

[procedure] (test::expect-equivalence-result-timing? RESULT)

If a timing had been attached to this expectation, this function will return #t, otherwise it will be #f.

test::expect-equivalence-result-timing

[procedure] (test::expect-equivalence-result-timing RESULT)

If a timing had been attached to this expectation, this function will return the timing pair, otherwise it shall return '(0 . 0).

Tolerance Style Expectation

This is a specialized expectation which accepts three expressions and checks to see if the "right hand side" is within a "tolerance" of the "left hand side". There is only one expectation in the tolerance style currently. If any of these API functions, except test::expect-tolerance-result?, are passed something that isn't a tolerance style expectation result object the result is unspecified.

test::expect-tolerance-result?

[procedure] (test::expect-tolerance-result? RESULT)

If RESULT is a tolerance style result object from the invocation of an expectation macro, then this function will return #t. Otherwise, it will return #f.

test::expect-tolerance-result-finding

[procedure] (test::expect-tolerance-result-finding RESULT)

Returns the boolean result associated with the tolerance style expectation RESULT object.

test::expect-tolerance-result-kind

[procedure] (test::expect-tolerance-result-kind RESULT)

This retrieves the "kind" field of a particular tolerance style expectation. For example, if you had a result object from an invocation of a (expect-near? "foobar" 100 .01 100.001) expectation, then the "kind" field of the expectation result object will be the string "near". Here is a table describing what the "kind" fields are for each kind of tolerance style expectation:

Tolerance Style Expectation	Associated Specific String
expect-near	"near"

test::expect-tolerance-result-name

[procedure] (test::expect-tolerance-result-name RESULT)

Returns the message object associated with a tolerance style expectation RESULT object.

test::expect-tolerance-result-lhs-evaled

[procedure] (test::expect-tolerance-result-lhs-evaled RESULT)

Returns the evaluated "left hand side" expression supplied to a tolerance style expectation.

test::expect-tolerance-result-lhs-tol-evaled

[procedure] (test::expect-tolerance-result-lhs-tol-evaled RESULT)

Returns the evaluated "tolerance" expression supplied to a tolerance style expectation.

test::expect-tolerance-result-rhs-unevaled

[procedure] (test::expect-tolerance-result-rhs-unevaled RESULT)

Returns the unevaluated "right hand side" expression supplied to a tolerance style expectation.

test::expect-tolerance-result-rhs-evaled

[procedure] (test::expect-tolerance-result-rhs-evaled RESULT)

Returns the evaluated "right hand side" expression supplied to a tolerance style expectation.

test::expect-tolerance-result-warning?

[procedure] (test::expect-tolerance-result-warning? RESULT)

If a warning had been attached to this expectation, this function will return #t, otherwise it will be #f.

test::expect-tolerance-result-warning

[procedure] (test::expect-tolerance-result-warning RESULT)

If a warning had been attached to this expectation, this function will return the warning message object supplied by the user, otherwise it shall return '().

test::expect-tolerance-result-timing?

[procedure] (test::expect-tolerance-result-timing? RESULT)

If a timing had been attached to this expectation, this function will return #t, otherwise it will be #f.

test::expect-tolerance-result-timing

[procedure] (test::expect-tolerance-result-timing RESULT)

If a timing had been attached to this expectation, this function will return the timing pair, otherwise it shall return '(0 . 0).

Gloss Result Object

test::gloss-result?

[procedure] (test::gloss-result? RESULT)

If RESULT is a gloss result object from the invocation of the gloss macro, then this function will return #t. Otherwise, it will return #f.

test::gloss-result-message

[procedure] (test::gloss-result-message RESULT)

Returns the message object associated with the gloss RESULT object.

test::gloss-result-warning?

[procedure] (test::gloss-result-warning? RESULT)

If a warning had been attached to this gloss, this function will return #t, otherwise it will be #f.

test::gloss-result-warning

[procedure] (test::gloss-result-warning RESULT)

If a warning had been attached to this gloss, this function will return the warning message object supplied by the user, otherwise it shall return '().

Skip Result Object

test::skip-result?

[procedure] (test::skip-result? RESULT)

If RESULT is a skip result object from the invocation of the skip macro, then this function will return #t. Otherwise, it will return #f.

test::skip-result-message

[procedure] (test::skip-result-message RESULT)

Returns the message object associated with the skip RESULT object. Hopefully, it was stated why this set of clauses had been skipped.

test::skip-result-warning?

[procedure] (test::skip-result-warning? RESULT)

If a warning had been attached to this skip, this function will return #t, otherwise it will be #f.

test::skip-result-warning

[procedure] (test::skip-result-warning RESULT)

If a warning had been attached to this skip, this function will return the warning message object supplied by the user, otherwise it shall return '().

Termination Result Object

When using the manipulation API for a terminate result, if you pass a result to one of these function that is not a terminate result, it will return an unspecified value.

test::terminate-result?

[procedure] (test::terminate-result? RESULT)

If RESULT is a termination result object from the invocation of a termination function, then this function will return #t. Otherwise, it will return #f.

test::terminate-result-finding

[procedure] (test::terminate-result-finding RESULT)

Returns the result associated with the termination function RESULT object.

test::terminate-result-scope

[procedure] (test::terminate-result-scope RESULT)

The "scope" of the termination result is exactly the TEST-NAME parameter supplied to the test case or test suite associated with the TERMFUNC.

test::terminate-result-container

[procedure] (test::terminate-result-container RESULT)

The "container" of the termination result is going to be either 'test-suite or 'test-case depending upon which the TERMFUNC was associated.

test::terminate-result-message

[procedure] (test::terminate-result-message RESULT)

Returns the message object associated with the termination RESULT object.

Todo Result Object

test::todo-result?

[procedure] (test::todo-result? RESULT)

If RESULT is a todo result object from the invocation of a todo macro, then this function will return #t. Otherwise, it will return #f.

test::todo-result-message

[procedure] (test::todo-result-message RESULT)

Returns the message object associated with the todo RESULT object.

test::todo-result-warning?

[procedure] (test::todo-result-warning? RESULT)

If a warning had been attached to this todo, this function will return #t, otherwise it will be #f.

test::todo-result-warning

[procedure] (test::todo-result-warning RESULT)

If a warning had been attached to this todo, this function will return the warning message object supplied by the user, otherwise it shall return '().

Miscellaneous

test::walk-structure

[procedure] (test::walk-structure WALKER TEST-STRUCTURE)

This procedure will enumerate the TEST-STRUCTURE association list, invoking the WALKER for each element. The hierarchy level value is the tree depth at the point of call.

WALKER is a three argument procedure. The first argument is the hierarchy level, counting from 0, the outermost level. The second argument is the type of the test element. The third argument is the name of the test element.

Statistics Generation

test::stat-result-statistics

[procedure] (test::stat-result-statistics RESULT-TREE)

This function will compute and return a vector of result STATISTICS. Use the procedures below to access the elements by name.

test::stat-test-suites

[procedure] (test::stat-test-suites STATISTICS)

test::stat-test-suite-warnings

[procedure] (test::stat-test-suite-warnings STATISTICS)

test::stat-test-suites-passed

[procedure] (test::stat-test-suites-passed STATISTICS)

test::stat-test-suites-failed

[procedure] (test::stat-test-suites-failed STATISTICS)

test::stat-test-suites-timing

[procedure] (test::stat-test-suites-timing STATISTICS)

test::stat-test-suites-terminated

[procedure] (test::stat-test-suites-terminated STATISTICS)

test::stat-test-cases

[procedure] (test::stat-test-cases STATISTICS)

test::stat-test-case-warnings

[procedure] (test::stat-test-case-warnings STATISTICS)

test::stat-test-cases-passed

[procedure] (test::stat-test-cases-passed STATISTICS)

test::stat-test-cases-failed

[procedure] (test::stat-test-cases-failed STATISTICS)

test::stat-test-cases-timing

[procedure] (test::stat-test-cases-timing STATISTICS)

test::stat-test-cases-terminated

[procedure] (test::stat-test-cases-terminated STATISTICS)

test::stat-all-expectations

[procedure] (test::stat-all-expectations STATISTICS)

test::stat-all-expectation-warnings

[procedure] (test::stat-all-expectation-warnings STATISTICS)

test::stat-all-expectations-passed

[procedure] (test::stat-all-expectations-passed STATISTICS)

test::stat-all-expectations-failed

[procedure] (test::stat-all-expectations-failed STATISTICS)

test::stat-all-expectations-timing

[procedure] (test::stat-all-expectations-timing STATISTICS)

test::stat-single-expectations

[procedure] (test::stat-single-expectations STATISTICS)

test::stat-single-expectation-warnings

[procedure] (test::stat-single-expectation-warnings STATISTICS)

test::stat-single-expectations-passed

[procedure] (test::stat-single-expectations-passed STATISTICS)

test::stat-single-expectations-failed

[procedure] (test::stat-single-expectations-failed STATISTICS)

test::stat-single-expectations-timing

[procedure] (test::stat-single-expectations-timing STATISTICS)

test::stat-tol-expectations

[procedure] (test::stat-tol-expectations STATISTICS)

test::stat-tol-expectation-warnings

[procedure] (test::stat-tol-expectation-warnings STATISTICS)

test::stat-tol-expectations-passed

[procedure] (test::stat-tol-expectations-passed STATISTICS)

test::stat-tol-expectations-failed

[procedure] (test::stat-tol-expectations-failed STATISTICS)

test::stat-tol-expectations-timing

[procedure] (test::stat-tol-expectations-timing STATISTICS)

test::stat-equiv-expectations

[procedure] (test::stat-equiv-expectations STATISTICS)

test::stat-equiv-expectation-warnings

[procedure] (test::stat-equiv-expectation-warnings STATISTICS)

test::stat-equiv-expectations-passed

[procedure] (test::stat-equiv-expectations-passed STATISTICS)

test::stat-equiv-expectations-failed

[procedure] (test::stat-equiv-expectations-failed STATISTICS)

test::stat-equiv-expectations-timing

[procedure] (test::stat-equiv-expectations-timing STATISTICS)

test::stat-todos

[procedure] (test::stat-todos STATISTICS)

test::stat-todo-warnings

[procedure] (test::stat-todo-warnings STATISTICS)

test::stat-skips

[procedure] (test::stat-skips STATISTICS)

test::stat-skip-warnings

[procedure] (test::stat-skip-warnings STATISTICS)

test::stat-glosses

[procedure] (test::stat-glosses STATISTICS)

test::stat-gloss-warnings

[procedure] (test::stat-gloss-warnings STATISTICS)

test::stat-terminations

[procedure] (test::stat-terminations STATISTICS)

Printing Routines

These procedures are used in the generation of test result reports.

test::write-object

[procedure] (test::write-object OBJECT)

test::display-objects

[procedure] (test::display-objects [OBJECT ...])

test::display-objects-newline

[procedure] (test::display-objects-newline [OBJECT ...])

test::display-indent

[procedure] (test::display-indent INDENT)

test::display-indented-objects

[procedure] (test::display-indented-objects INDENT [OBJECT ...])

test::display-indented-objects-newline

[procedure] (test::display-indented-objects-newline INDENT [OBJECT ...])

INDENT is the number of spaces to the left.

test::display-underlined

[procedure] (test::display-underlined [OBJECT ...])

Prints each supplied OBJECT to the (current-output-port), with a #\space interspersed, on a single line. Then prints a line of underscores, #\_.

test::display-structure

[procedure] (test::display-structure TEST-STRUCTURE)

Prints the supplied TEST-STRUCTURE tree to the (current-output-port) with indentation.

Test Report Generation

These functions will display a rendering of the RESULT-TREE and returns the RESULT-TREE.

test::output-style-compact

<procedure>(test::output-style-compact RESULT-TREE [PORT|FILE-NAME|#t])

Prints a simple listing of the test results. Details are only printed for failed tests.

test::output-style-human

[procedure] (test::output-style-human RESULT-TREE [PORT|FILE-NAME|#t])

Prints a verbose listing of the test results.

test::output-style-html

[procedure] (test::output-style-html RESULT-TREE [PORT|FILE-NAME|#t])

Prints a verbose listing of the test results as HTML.

test::output-style-minimal

[procedure] (test::output-style-minimal RESULT-TREE [PORT|FILE-NAME|#t])

Prints a summary of the test result statistics.

The output style procedures are not loaded automatically. They must be loaded explicitly, as in:

(require-extension testbase-output-human)

(require-extension testbase-output-html)

(require-extension testbase-output-compact)

(require-extension testbase-output-minimal)

This procedure is useful with a test report procedure. The actual report procedures of the above report generators are examples.

test::with-output-style

[procedure] (test::with-output-style STYLER RESULT-TREE [PORT|FILE-NAME|#t])

STYLER is a procedure with a single argument, a RESULT-TREE, to print a report.

The default destination is the (current-output-port). Otherwise the destination specifies how the output is targeted.

A PORT destination will print to the port.

A FILE-NAME destination will print to the specified file. Should the file exist the result is implementation dependent.

A #t destination will return the output as a string.

The STYLER will be called with RESULT-TREE as the only actual argument.

Notes & Caveats

To generate the tree structure of a test procedure the test body is evaluated using a special test mode, report. The expectation expressions, test container let bindings, and test-letrec bindings are not evaluated. This means there should not be any expressions in the test body that will be evaluated, such as the construction of a TEST-NAME, that depend on these bindings.

However, test procedure initial and final forms are evaluated.

Be careful when using test selection that dependencies between tests are not broken. For example, an expectation expression could depend on the result of an expect-set!, but the associated variable may never be bound because the expect-set! was skipped.

Examples

Assume the following are the contents of the file "testbase-example.scm":

(require-extension testbase testbase-output-human)

(define-expect-binary =)

(define-test example "Arithmetic Operators"
  (test/case "Testing '+'"
    (expect-= "Adding two positive numbers" 2 (+ 1 1))
    (expect-= "Adding two negative numbers" -2 (+ -1 -1))
    (expect-zero "Adding positive and negative" (+ -1 1)) )
  (test/case "Testing '-'"
    (expect-zero "Subtracting two positive numbers" (- 1 1))
    (expect-zero "Subtracting two negative numbers" (- -1 -1))
    (expect-= "Subtracting positive and negative" -2 (- -1 1)) ) )

(test::styler-set! example test::output-style-human)
(run-test "TestBase Example")

Then the following entered from a command line interface:

 csi -n -s testbase-example.scm

Produces the following output:

 ** Running TestBase Example **
 
 Test Suite: Arithmetic Operators
 
   Test Case: Testing '+'
 
     Expectation: Adding two positive numbers
     Expect =
        Expected: 2
     Unevaluated: (+ 1 1)
       Evaluated: 2
     Pass: Adding two positive numbers
 
     Expectation: Adding two negative numbers
     Expect =
        Expected: -2
     Unevaluated: (+ -1 -1)
       Evaluated: -2
     Pass: Adding two negative numbers
 
     Expectation: Adding positive and negative
     Expect zero
     Unevaluated: (+ -1 1)
       Evaluated: 0
     Pass: Adding positive and negative
 
   Pass: Testing '+'
 
   Test Case: Testing '-'
 
     Expectation: Subtracting two positive numbers
     Expect zero
     Unevaluated: (- 1 1)
       Evaluated: 0
     Pass: Subtracting two positive numbers
 
     Expectation: Subtracting two negative numbers
     Expect zero
     Unevaluated: (- -1 -1)
       Evaluated: 0
     Pass: Subtracting two negative numbers
 
     Expectation: Subtracting positive and negative
     Expect =
        Expected: -2
     Unevaluated: (- -1 1)
       Evaluated: -2
     Pass: Subtracting positive and negative
 
   Pass: Testing '-'
 
 Pass: Arithmetic Operators
 
 ALL TESTS SUCCESSFUL!

And the following entered from a command line interface:

 echo "(test::resource-limit-set! example 'cpu 0) (example)" | csi -n -s testbase-example.scm -- -i

Produces the following output:

 (begin (id tr0) (name "Arithmetic Operators") (result #f) (kind "suite"))
 (begin (id tr1) (name "Testing '+'") (result #f) (kind "case"))
 (expectation (id tr2) (name "Adding two positive numbers") (result #t) (kind "=") (unevaluated (+ 1 1)) (expected 2) (actual 2))
 (expectation (id tr3) (name "Adding two negative numbers") (result #t) (kind "=") (unevaluated (+ -1 -1)) (expected -2) (actual -2))
 (expectation (id tr4) (name "Adding positive and negative") (result #t) (kind "zero") (unevaluated (+ -1 1)) (actual 0))
 (end (id tr1) (name "Testing '+'") (result #t) (kind "case"))
 (begin (id tr5) (name "Testing '-'") (result #f) (kind "case"))
 (expectation (id tr6) (name "Subtracting two positive numbers") (result #t) (kind "zero") (unevaluated (- 1 1)) (actual 0))
 (error (id tr7) (message "Resource Limit Exceeded: Processor-Time") (result #f) (kind "terminate") (container test-suite) (location "Arithmetic Operators"))
 (end (id tr0) (name "Arithmetic Operators") (result #f) (kind "suite"))

And the following entered from a command line interface:

 echo "(test::run)" | csi -n -s testbase-example.scm -- -i

Produces the following output:

 (begin (id tr0) (name "Arithmetic Operators") (result #f) (kind "suite"))
 (begin (id tr1) (name "Testing '+'") (result #f) (kind "case"))
 (expectation (id tr2) (name "Adding two positive numbers") (result #t) (kind "=") (unevaluated (+ 1 1)) (expected 2) (actual 2))
 (expectation (id tr3) (name "Adding two negative numbers") (result #t) (kind "=") (unevaluated (+ -1 -1)) (expected -2) (actual -2))
 (expectation (id tr4) (name "Adding positive and negative") (result #t) (kind "zero") (unevaluated (+ -1 1)) (actual 0))
 (end (id tr1) (name "Testing '+'") (result #t) (kind "case"))
 (begin (id tr5) (name "Testing '-'") (result #f) (kind "case"))
 (expectation (id tr6) (name "Subtracting two positive numbers") (result #t) (kind "zero") (unevaluated (- 1 1)) (actual 0))
 (expectation (id tr7) (name "Subtracting two negative numbers") (result #t) (kind "zero") (unevaluated (- -1 -1)) (actual 0))
 (expectation (id tr8) (name "Subtracting positive and negative") (result #t) (kind "=") (unevaluated (- -1 1)) (expected -2) (actual -2))
 (end (id tr5) (name "Testing '-'") (result #t) (kind "case"))
 (end (id tr0) (name "Arithmetic Operators") (result #t) (kind "suite"))

A test-collect Example:

(require-extension testbase testbase-output-human)

(define-test tcx "A test-collect example"
  (test/collect "Are 0 - 9 integers?"
    (do ([i 0 (add1 i)])
        [(= i 10)]
      (collect-test (expect-true (integer? i))) ) ) )

(test::styler-set! tcx test::output-style-human)
(run-test "TestBase Example")

Produces the following output:

 ** Running TestBase Example **
 Test Suite: A test-collect example
 
   Test Case: Are 0 - 9 integers?
 
     Expectation:
     Expect true
     Unevaluated: (integer? i)
       Evaluated: #t
     Passed:
 
     Expectation:
     Expect true
     Unevaluated: (integer? i)
       Evaluated: #t
     Passed:
 
     Expectation:
     Expect true
     Unevaluated: (integer? i)
       Evaluated: #t
     Passed:
 
     Expectation:
     Expect true
     Unevaluated: (integer? i)
       Evaluated: #t
     Passed:
 
     Expectation:
     Expect true
     Unevaluated: (integer? i)
       Evaluated: #t
     Passed:
 
     Expectation:
     Expect true
     Unevaluated: (integer? i)
       Evaluated: #t
     Passed:
 
     Expectation:
     Expect true
     Unevaluated: (integer? i)
       Evaluated: #t
     Passed:
 
     Expectation:
     Expect true
     Unevaluated: (integer? i)
       Evaluated: #t
     Passed:
 
     Expectation:
     Expect true
     Unevaluated: (integer? i)
       Evaluated: #t
     Passed:
 
     Expectation:
     Expect true
     Unevaluated: (integer? i)
       Evaluated: #t
     Passed:
 
   Passed: Are 0 - 9 integers?
 
 Passed: A test-collect example
 
 ALL TESTS SUCCESSFUL!
 
 ** Passed TestBase Example **

Tips, Tricks, And Idioms

This documentation is pretty large, so put here any things that weren't obvious to you that seem like they might be useful to others.

General Tips

test/case and friends can take a let-style list of local variables, right after the test description, i.e. (test/case "foo" ([var #t]) ...)
Note the existence of expect-set!
It is possible to map over all test procedures, at least up to that point in the file, so you can, for example, set all tests to the "human" report format like so: (test::for-each (cut test::styler-set! <> test::output-style-human))
chicken-testbase-driver (see testbase-driver) has --test-take and --test-skip options, for narrowing down your tests as you're in the middle of particular bits of code

Automated Egg Tests

chicken-setup, at least when run with the -t flag, will treat the "tests" directory as a directory full of tests, and run the "run.scm" file in that directory. Here's a run.scm file, modified from one provided by Kon, that seems to work well. You'll need testbase-driver for it to work. If any tests fail, the setup will abort (although by that time it's already installed the egg, but at least it'll be quite cranky about the failure). This run.scm will run everything in the tests directory named "*/test.scm". It will put the results in "results/" under the tests directory.

;;;; run-testbase.scm (run.scm)
;
; Thanks to Kon Lovett for the code.

(use utils posix)

;;

(define *egg-name* (car (command-line-arguments)))

(define *verbose* #t)

(define-constant TEST-DRIVER "chicken-testbase-driver")
(define-constant TEST-DRIVER-OPTIONS "--indent 2 --test-interpret --failure-exit=1")

(define *test-driver-arguments*
  (conc
    TEST-DRIVER-OPTIONS
    " --results-repository="
    (current-directory)
    "/results "
    (if
      *verbose* " -v" "")))

;;

(system* "~A ~A ~A" TEST-DRIVER *test-driver-arguments* (string-append (current-directory) "/*-test.scm"))

Migration from test-infrastructure

Changes:

test-group removed.
test-apply removed.
test-it -> test.
test-package -> test-suite.
expect -> expect-unary.
expect-values -> expect-unary/values.
expect-equiv -> expect-binary.
expect-equiv-values -> expect-binary/values.
expect-values-foo -> expect-foo/values.
test:foo -> test::foo.
foo-ref -> foo.
*-foo -> test::result-foo.
Test containers cannot be used outside of a define-test!

Forms like:

 (test::apply test::output-style-compact
     (skip ...)
     test-proc)

become

 (test::take! test-proc '(skip ...))
 (test::styler-set! test-proc test::output-style-compact)
 (test-proc)

Forms like

 (define (test-something)
   (test-suite "Something" pd pe
     ... test elements ...
   )
 )

become

 (define-test test-something
   (test-suite "Something" pd pe
     ... test elements ...
   )
 )

or even

 (define-test test-something "Something"
   ... test elements ...
   <where any use of 'pd' and 'pe' becomes 'test-dtor' and 'test-escr'>
 )

Forms like

 (let ((result
         (test-suite "Something" pd pe
           ... test elements ...)))
   ...)

become

 (define-test something
   (test-suite "Something" pd pe
     ... test elements ...
   )
 )
 
 (let ((result (something)))
   ...)

Bugs & Limitations

Because a sharp-syntax reader is created to handle "unprintable" results the "#<<TAG" and "#<#TAG" multiline string constant read syntax cannot be used within a test-file. Use (MULTILINE-STRING STRING ...).

Contributions

output-text-compact by Patrick Brannan.

Changelog

1.5.2 Added early-exit-upon-failure-option
1.5.1 Fix for non-false expect predicate result
1.5 Added insist expectation form
1.41 Cygwin has resource limiting
1.4 Expansion options are #f by default, expcept test::catching!
1.304 Bug fix for test::forget!
1.303 Removed macro expansion time atom check for TEST-NAME
1.302 Added test-let & test-let*
1.301 Dropped :optional
1.3 Added define-expect-nary and test-collect
1.2 Bugfix for define-expect-*/values
1.1 Added echo filter options
1.0 Initial release

License

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

 Redistributions of source code must retain the above copyright notice, this list of conditions and the following
   disclaimer.
 Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following
   disclaimer in the documentation and/or other materials provided with the distribution.
 Neither the name of the author nor the names of its contributors may be used to endorse or promote
   products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Outdated egg!

testbase

Description

Author

Requirements

Download

Documentation

Test File

Macro and Procedure Naming Conventions

Option Expansion Control

test::timing

test::filtering

test::catching

test::selecting

test::limiting

Multiline String Construction

MULTILINE-STRING

Immediate Test

test

Test Procedure Definition

define-test

declare-test

Test Procedure Operations

test::procedure?

test::formal-arguments

test::declarations

test::procedure

test::name

test::procedure-variable

test::styler

test::styler-set!

test::tolerance

test::tolerance-set!

test::reset!

test::structure

test::add-filter!

test::remove-filter!

test::add-echo!

test::remove-echo!

test::set-echo-option!

test::run-mode

test::selection-mode

test::select!

test::selections

test::resource-limit

test::resource-limit-set!

test::procedures

test::for-each

test::map

test::forget!

test::exit-on-failure?

test::run

Run Tests

run-test

Test Macros

Test Containers

test-suite

test-case

test-collect

Test Flags

test-case-evaluate-thru-failure

test-timing

test-evaluation-mode?

Test Binding Forms

test-letrec

test-let*

test-let

Test Expectations

insist

expect-unary

expect-binary

expect-zero

expect-nonzero

expect-positive

expect-negative

expect-near

expect-true

expect-false

expect-success

expect-failure