Introduction (historical revision 4213)

You are looking at historical revision 4213 of this page. It may differ significantly from its current revision.

eggs

Introduction

Introduction

A test runner utility for use with TestBase tests.

The testbase driver runs TestBase test-files, with optional configuration. The test-results are stored in the testbase test results database. The test-files are stored in the testbase test files directory.

Usage

The testbase-driver command:

chicken-testbase-driver [options...] test-file

   --indent=INTEGER                   Amount to indent messages [default: 0]
   --test-compile                     Run compiled test
   --compiler=PATHNAME                Compiler command pathname
   --compiler-option=OPTION(S)        Option(s) for the compiler
   --compiler-extend=FORM(S)          Form(s) for compiler to load before compiling
   --test-interpret                   Run interpreted test
   --interpreter=PATHNAME             Interpreter command pathname
   --interpreter-option=OPTION(S)     Option(s) for the interpreter
   --test-ignore=TESTNAME             Ignore the result(s) for the specified test
   --test-skip=TESTNAME               Skip the specified test
   --test-directory=PATHNAME          Directory in which to run the test
   --test-setup=FORM(S)               Form(s) to pass on to the running test
   --test-reader-extension=ID(S)      Module(s) for test results reader to load before reading
   --files-repository=PATHNAME        Directory where test files are stored
   --results-repository=DIRECTORY     Directory where test results are stored
   --results-expectation-specifics    Enable inclusion of the actual values for expectation test results
   --results-mode=all|errors|none     How are test results stored [default: errors]
   -q, --quiet                        Do not print messages
   -v, --verbose                      Print phase messages
   -n, --dry-run                      Just show commands to be executed, don't run them
   -V, --version                      Display version
   -h, --help                         Display this text

test-file: Pathname of test file; missing directory and extension components default to "files-repository" and "scm".
files-repository: default (repository-path)/testbase/tests
results-repository: default (repository-path)/testbase/results

TestBase Test Results Database

Currently the testbase results database is a suite of files in the results-repository directory, . Each file is a collection of test results records, one per test-file.

An API, TestBase Results, is provided to manipulate test results. The programmer is strongly encouraged to use the provided API.

TestBase Test Configuration

A test-file may have one or more configuration sections. The sections are represented as a Scheme block comment.

The configuation section header must be on a single line. The trailer must be on a single line. The configuation section contents must follow Scheme read-syntax, and comments but not block-comments, are allowed.

Two execution models are recognized, compile and interpret.

#| TestBase ...ignored...

(compile

 (command <pathname>)
 (option <atom> ...)
 (extend <form> ...)
 (directory <pathname>)
 (setup <form> ...)
 (startup <form> ...)
 (cleanup <form> ...)
 (ignore <test-identification> ...)
 (skip (<test-procedure-name> <test-identification> ...) ...) )

(interpret

 (command <pathname>)
 (option <atom> ...)
 (directory <pathname>)
 (setup <form> ...)
 (startup <form> ...)
 (cleanup <form> ...)
 (ignore <test-identification> ...)
 (skip (<test-procedure-name> <test-identification> ...) ...) )

(test

 (purpose <docu-object> ...)
 (feature <module-id> ...)
 (reader-extension <module-id> ...)
 (reader-setup <form> ...)
 (compile <flag>)
 (interpret <flag>)
 (directory <pathname>)
 (ignore <test-identification> ...)
 (skip (<test-procedure-name> <test-identification> ...) ...)
 (setup <form> ...) )

pathname: absolute pathname
flag: boolean
form: string or syntax
atom: string or number or symbol
test-name: test-name
test-type: test | suite | case | expect
test-procedure-name: test-name
test-identification: test-name or (<test-type> <test-name>)
module-id: extension or library symbolic name
docu-object: sxml
directory: Directory in which to run test
command: Path to the command to execute
option: Arguments for the command
extend: Forms to extend the execution environment of the command
setup: Forms to extend the execution environment of the test
reader-extension: Extension module(s) for the test results reader execution environment
reader-setup: Extension expression(s) for the test results reader execution environment
startup: Forms to perform before command
cleanup: Forms to perform after command
ignore: Results are not saved
skip: Tests are not performed
purpose: What the test is to achieve
feature: What the modules are tested

The elements of the 'test subsection are combined with the execution model subsections, compile and interpret. When an element is single-valued the subsection element takes precedence.

The command-line argument values are combined with the corresponding subsection elements. When an element is single-valued the command-line argument takes precedence.

1.0: Initial release

Introduction

Usage

TestBase Test Results Database

TestBase Test Configuration

TestBase Results API

TestBase Test API

Requirements

Authors

License

Bugs

Version History