You can edit this page using wiki syntax for markup.

Article contents:== Outdated egg! This is an egg for CHICKEN 3, the unsupported old release. You're almost certainly looking for [[/eggref/4/testbase|the CHICKEN 4 version of this egg]], if it exists. If it does not exist, there may be equivalent functionality provided by another egg; have a look at the [[https://wiki.call-cc.org/chicken-projects/egg-index-4.html|egg index]]. Otherwise, please consider porting this egg to the current version of CHICKEN. [[tags: egg]] == testbase [[toc:]] === Description Macro based unit-testing facility === Author [[Kon Lovett]], with portions from test-infrastructure 1.0 by Peter Keller === Requirements * [[vector-lib]] * [[syntax-case]] * [[rlimit]] * [[srfi-37]] * [[srfi-42]] * [[args]] * [[testbase-results]] === Download [[http://code.call-cc.org/legacy-eggs/3/testbase.egg|testbase.egg]] === Documentation This extension provides a macro based unit testing facility based upon expectations concerning evaluations of expressions. Expectations are grouped into test containers, such as test-suite and test/case. In addition to expectation and container test forms a set of annotation forms are supplied. All test forms, be they containers, annotations, or expectations, return a test result object. Result objects generated by the test forms enclosed by a test container are composed by the container into the container's test result object. The set of test results forms a tree. This result tree is then passed to either user defined functions which traverse the tree manipulating it in any way desired. API functions to deal with the result types are supplied. Examples of existing functions that traverse the result are the test results output routines. ==== Test File A {{test-file}} is a Scheme source file consisting of testbase forms and necessary auxiliary definitions. The general template is: <enscript highlight=scheme> (require-extension testbase) .. other extensions, includes, imports, etc. as needed .. #| TestBase .. see "testbase-driver" documentation .. |# .. any expressions necessary to testing .. (define-test foo-test ... ) (run-test "Foo Tests") </enscript> While possible to use testbase forms without invoking {{run-test}} command line processing is not performed. Command Line Options: ; ---failure-exit CODE : Specifiy an exit code & set the early-exit-upon-failure-option. See {{test::exit-on-failure?}}, {{test::run}}, and {{run-test}}. ; -b ---batch : Specify batch mode. See {{run-test}}. ; -i ---interactive : Specify interactive (or driven) mode. See {{run-test}}. ; -V ---version : Display the version. ; -h ---help : Display help message. ==== Macro and Procedure Naming Conventions A distinction is made between forms that are to be used within a test, and those that are to be used outside of a test. In general, those forms that are for use within a test are macros, and those for use outside a test are procedures. Procedures have a {{test::}} namespace prefix, such as {{test::timing}} Macros do not have a {{test::}} namespace prefix. Since they can only pollute the namespace of a single file a looser naming convention is used. ==== Option Expansion Control The expanded source of the test code is controlled by a set of syntax constants. To drop or add an option place the appropriate syntax form below in a test file. {{FLAG}} is a boolean, {{#t}} or {{#f}}. ===== test::timing <macro>(define-for-syntax test::timing FLAG)</macro> {{test::timing}} controls the expectation timing option. Default is {{#f}}. ===== test::filtering <macro>(define-for-syntax test::filtering FLAG)</macro> {{test::filtering}} controls the test result filtering option. Default is {{#f}}. ===== test::catching <macro>(define-for-syntax test::catching FLAG)</macro> {{test::catching}} controls the expectation error/exception trapping option. Default is {{#t}}. Dropping error/exception trapping is not suggested in the general case, since any intermediate errors will abort the entire test. Does not affect {{expect-exception}}. ===== test::selecting <macro>(define-for-syntax test::selecting FLAG)</macro> {{test::selecting}} controls the test selective evaluation option. Default is {{#f}}. ===== test::limiting <macro>(define-for-syntax test::limiting FLAG)</macro> {{test::limiting}} controls the test resource limit option. Default is {{#f}}. ==== Multiline String Construction Due to a bug the Chicken multiline string forms cannot be used. This macro is provided as a workaround. For use in a test-file. ===== MULTILINE-STRING <macro>(MULTILINE-STRING STRING ...)</macro> Yes, the macro name is uppercase. {{STRING}} is any scheme string. The strings are suffixed with a newline and concatenated. ==== Immediate Test This macro will evaluate in a left to right fashion the clauses inside it and output the results. The clauses are a sequence of {{(Expression Expectation)}} forms. A clause is evaluated as {{(expect-equal EXPRESSION EXPECTED)}}. For use with a REPL (Read-Eval-Print-Loop). The results are not kept and no other forms are allowed in the body. ===== test <macro>(test (EXPECTED EXPRESSION) ...)</macro> {{EXPRESSION}} is any scheme expression. {{EXPECTED}} is any scheme expression, the expected result of {{EXPRESSION}}. ==== Test Procedure Definition All test forms are grouped into a test procedure. Test forms cannot be used outside of a test procedure without generating an error. ===== define-test <macro>(define-test PROCEDURE-VARIABLE [TEST-NAME] [(initial FORM ...)] [(final FORM ...)] CLAUSE ...)</macro> <macro>(define-test (PROCEDURE-VARIABLE VARIABLE ...) [TEST-NAME] [(initial FORM ...)] [(final FORM ...)] CLAUSE ...)</macro> Defines a variable, {{PROCEDURE-VARIABLE}}, with a {{TEST-PROCEDURE}} value. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. Must be unique in the test-file. When missing a string name is formed from the {{PROCEDURE-VARIABLE}}. {{VARIABLE}} is a lambda variable, a formal argument to the test-procedure. When supplied the {{(initial FORM ...)}} will cause each {{FORM}} to be expanded at the beginning of the test procedure. {{FORM}} can be any Scheme form legal at the beginning of a {{lambda}} body. The {{initial}} section will be evaluated before the first {{CLAUSE}}. If define forms occur in this section, the variables will be in scope for the whole test. When supplied the {{(final FORM ...)}} will cause each {{FORM}} to be expanded at the end of the test procedure. {{FORM}} can be any Scheme form legal at the end of a {{lambda}} body. The {{final}} section will be evaluated after the last {{CLAUSE}} and before the destructor is activated! The forms in the {{initial}} and {{final}} sections do not need to return test result objects; they are not considered part of the test body. In other words, a {{FORM}} is not a {{CLAUSE}}. {{CLAUSE}} is any of the test API elements. Actually {{CLAUSE}} can be any Scheme {{FORM}} but be careful. A {{CLAUSE}} must return a test result object. Unless this can be guaranteed for a {{FORM}} a runtime error can be generated. The {{DTOR-NAME}} will be {{test-dtor}}. See {{test-suite}}. The {{ESCR-NAME}} will be {{test-escr}}. See {{test-suite}}. Can be called directly, as {{(PROCEDURE-VARIABLE)}} or {{(PROCEDURE-VARIABLE ARGUMENT ...)}}, and returns a test-result. A parameterized test definition should be complemented by one or more test declarations, a {{declare-test}} form. Otherwise the test cannot be invoked by {{test::run}}. ===== declare-test <macro>(declare-test PROCEDURE-VARIABLE TEST-NAME ARGUMENT ...)</macro> Declares an actual test for the {{PROCEDURE-VARIABLE}}. {{ARGUMENT}} is a scheme object. An actual argument to the corresponding test-procedure formal argument. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. Must occur lexically after the {{PROCEDURE-VARIABLE}} definition in the test-file! An actual-test is not a test-procedure and cannot be called directly. The actual-test is tied to the {{PROCEDURE-VARIABLE}}, and can only be invoked by {{test::run}}. ==== Test Procedure Operations {{TEST-IDENTIFIER}} is either a {{PROCEDURE-VARIABLE}} bound to a {{TEST-PROCEDURE}} or a {{TEST-NAME}}. When a {{TEST-NAME}} the corresponding {{TEST-PROCEDURE}} will be substituted. ===== test::procedure? <procedure>(test::procedure? OBJECT)</procedure> Is the {{OBJECT}} a defined test-procedure object? ===== test::formal-arguments <procedure>(test::formal-arguments TEST-IDENTIFIER)</procedure> Returns the formal argument list of the test-procedure. ===== test::declarations <procedure>(test::declarations TEST-IDENTIFIER)</procedure> Returns a list of {{(test-procedure actual-test-name actual-test-arguments)}} for the test-procedure. {{TEST-IDENTIFIER}} should identify a parameterized test-procedure. ===== test::procedure <procedure>(test::procedure TEST-IDENTIFIER)</procedure> Returns a test-procedure or {{#f}}. ===== test::name <procedure>(test::name TEST-IDENTIFIER)</procedure> Returns the test-name of the test-procedure. ===== test::procedure-variable <procedure>(test::procedure-variable TEST-IDENTIFIER)</procedure> Returns the defined symbol of the test-procedure. ===== test::styler <procedure>(test::styler TEST-IDENTIFIER)</procedure> Returns the default output style procedure for the test-procedure, or {{#f}} when no default styler is set. ===== test::styler-set! <procedure>(test::styler-set! TEST-IDENTIFIER [STYLER])</procedure> Sets the default output style procedure for the test-procedure. When missing or {{#f}} then no default styler is set. ===== test::tolerance <procedure>(test::tolerance TEST-IDENTIFIER)</procedure> Returns the current in-exact number comparison tolerance of the test-procedure. ===== test::tolerance-set! <procedure>(test::tolerance-set! TEST-IDENTIFIER TOLERANCE)</procedure> Sets the current in-exact number comparison tolerance of the test-procedure to {{TOLERANCE}}. ===== test::reset! <procedure>(test::reset! TEST-IDENTIFIER)</procedure> Sets the state of the test-procedure to the initial settings. ===== test::structure <procedure>(test::structure TEST-IDENTIFIER)</procedure> Returns the internal structure of test-procedure as a tree. The tree is the test hierarchy, where each node of the tree is {{(suite | case | expect TEST-NAME ...)}}, where {{...}} are any sub-trees. The top-level {{suite}} element value refers to the test-procedure. ===== test::add-filter! <procedure>(test::add-filter! TEST-IDENTIFIER FILTER-PROCEDURE)</procedure> Places {{FILTER-PROCEDURE}} as the head of the result filter chain for the test-procedure. {{FILTER-PROCEDURE}} must be a procedure of one argument, a test result object, and it must return a test result object. Not necessarily the same one. Filtering occurs ''during'' test element evaluation. ===== test::remove-filter! <procedure>(test::remove-filter! TEST-IDENTIFIER FILTER-PROCEDURE)</procedure> Removes {{FILTER-PROCEDURE}} from the result filter chain for the test-procedure. ===== test::add-echo! <procedure>(test::add-echo! TEST-IDENTIFIER [PORT] [KEY VALUE ...])</procedure> Causes all test results for the test-procedure to be passed to the ''echo-procedure''. The signature of an echo-procedure is {{(result-object port association-list -> unspecified)}}. The {{association-list}} holds options for the echo-procedure. When {{PORT}} missing the {{(current-output-port)}} is assumed. The {{PORT}} is unique. Only one echo per {{PORT}}. {{KEY}} is a symbol. The reserved keys are: * {{echo-procedure}}, where the {{VALUE}} is a procedure with an echo-procedure signature. * {{include-expectation-specifics}}, where the {{VALUE}} is a boolean indicating whether the value fields for an expectation are to be echoed, default is {{#f}}. When the {{KEY}} {{echo-procedure}} is missing the default echo-procedure is used. The test results are echoed to the {{PORT}} as an association list. One list per line. The echo-procedure is invoked after each test element evaluation. As such it must deal with test-container result objects in an incomplete state. Specifically the result-list field will be {{#f}} when a test-container is entered. It also must deal with ignore-results, which are stripped from the test-container result-list upon exit from the container. ===== test::remove-echo! <procedure>(test::remove-echo! TEST-IDENTIFIER [PORT])</procedure> Removes the {{PORT}} echo for the test-procedure. ===== test::set-echo-option! <procedure>(test::set-echo-option! TEST-IDENTIFIER [PORT] [KEY VALUE ...])</procedure> Modifies the echo specification for the specified {{PORT}}. The {{PORT}}, {{KEY}}, and {{VALUE}} parameters have the same interpretation as the {{test::add-echo!}} procedure. ===== test::run-mode <procedure>(test::run-mode)</procedure> Returns the test run mode, either {{batch}}, {{interactive}} or {{driven}}. ===== test::selection-mode <procedure>(test::selection-mode TEST-IDENTIFIER)</procedure> Returns the test selection mode, either {{take}} or {{skip}}. When the mode is {{take}} then unless a test is to be specifically skipped, it is taken. When the mode is {{skip}} then unless a test is to be specifically taken, it is skipped. The test procedure is always taken. ===== test::select! <procedure>(test::select! SELECTION-MODE TEST-IDENTIFIER [TEST-NAME | (TEST-TYPE TEST-NAME)] ...)</procedure> {{SELECTION-MODE}} is one of {{'take}} or {{'skip}}. {{TEST-TYPE}} is one of {{'suite}}, {{'case}}, or {{'expect}}. Select the test element(s) identified by {{(TEST-TYPE TEST-NAME)}} of the test-procedure. When only {{TEST-NAME}} is provided any test type will match. When no tests are supplied all existing selections are removed and the test selection mode is set to the {{SELECTION-MODE}}. <enscript highlight=scheme> (test::select! 'skip my-test-proc) ;; Now every test in 'my-test-proc' will be skipped. (test::select! 'take my-test-proc '(case "Top Case") "Foo") ;; Now every test in 'my-test-proc' except the test-case "Top Case", ;; and any test named "Foo". </enscript> When a test container is selected all contained tests are similarly selected. '''Independent of the {{(skip CLAUSE ...)}} form.''' ===== test::selections <procedure>(test::selections TEST-IDENTIFIER)</procedure> Returns a list of the test selections, or {{#f}} when there are no selections. The list is composed of sublists - {{(#f|test|suite|case|expect TEST-NAME take|skip)}}. ===== test::resource-limit <procedure>(test::resource-limit TEST-IDENTIFIER RLIMIT)</procedure> Returns the current value of {{RLIMIT}} for the test-procedure. The return value will either be a number or {{#f}}, indicating no set limit. RLIMIT is one of {{'cpu}}, {{'fsize}}, {{'nofile}}, {{'nproc}}, {{'data}}. '''Only operational on *NIX platforms.''' ===== test::resource-limit-set! <procedure>(test::resource-limit-set! TEST-IDENTIFIER RLIMIT VALUE)</procedure> Sets the current value of {{RLIMIT}} for the test-procedure to {{VALUE}}. RLIMIT is one of {{'cpu}}, {{'fsize}}, {{'nofile}}, {{'nproc}}, {{'data}}. {{VALUE}} is a positive integer or {{#f}}, indicating no set limit. When a {{'cpu}} or {{'fsize}} limit is reached the test will terminate with a termination result indicating the resource limit exceeded. When a {{'nofile}}, {{'nproc}}, or {{'data}} limit is reached the test will terminate with a termination result. '''Only operational on *NIX platforms.''' ===== test::procedures <procedure>(test::procedures)</procedure> Returns a list of all the test-procedures. ===== test::for-each <procedure>(test::for-each PROCEDURE [TEST-IDENTIFIER ...])</procedure> Calls {{PROCEDURE}} with every supplied test procedure, or all test procedures when nothing supplied. {{PROCEDURE}} takes one argument, a test-procedure. ===== test::map <procedure>(test::map PROCEDURE [TEST-IDENTIFIER ...])</procedure> Calls {{PROCEDURE}} with every supplied test procedure, or all test procedures when nothing supplied. Returns a list of the procedure results. {{PROCEDURE}} takes one argument, a test-procedure. ===== test::forget! <procedure>(test::forget! [TEST-IDENTIFIER ...])</procedure> Causes the specified test-procedures, or all when nothing specified, to be dropped. '''Really only for use during test development with a REPL.''' ===== test::exit-on-failure? <procedure>(test::exit-on-failure? [CODE-OR-FLAG])</procedure> When {{CODE-OR-FLAG}} is supplied sets the early-exit-upon-failure-option. Otherwise returns the current setting. {{CODE-OR-FLAG}} is either {{#f}}, to reset the option, or an integer, to set the option. The initial setting is {{#f}}. ===== test::run <procedure>(test::run [MESSAGE [TEST-IDENTIFIER ...]])</procedure> {{MESSAGE}} is some identity for the set of the test-procedures. When missing or {{#f}} the process command name is used. Displays a header with {{MESSAGE}}, and outputs the result of calling each test-procedure with the {{test::styler}}. When a {{test::styler}} is not set a simple success/failure message is given. The test-procedures are called in left to right order of the {{TEST-IDENTIFIER}} arguments. When a {{TEST-IDENTIFIER}} is not specified all the defined test-procedures are called in the order of definition in the test-file. For parameterized test-procedures the corresponding test declarations are run. When {{(test::exit-on-failure?)}} and some test fails then testbase will exit with the exit-code. When interactive mode without a console there is no output by default. See {{test::add-echo!}}. ==== Run Tests Run test-procedures and output test results. ===== run-test <macro>(run-test [MESSAGE] [FORM ...])</macro> Usually the last expression in a test file. Performs command-line processing. {{MESSAGE}} is some identity for the test. When missing the process command name is used. Use {{""}} for no message. {{FORM ...}} are instances of Scheme forms. When missing {{(test::run MESSAGE)}} is used. This macro will check the process command line for a {{batch}} mode (-b or --batch) or {{interactive}} mode (-i --interactive) option. When {{interactive}} mode without an attached console the mode is called {{driven}}. The default is {{batch}} mode. When {{batch}} mode the {{FORM ...}} will be evaluated. When {{FORM ...}} are missing then <em>every</em> test-procedure is run with test result output using the test-procedure styler. See {{test::run}}. When {{interactive}} mode a REPL will be entered. When {{driven}} mode a REPL will be entered. Test results will be reported as for {{test::add-echo!}}. '''Test macros are not automatically available in the REPL. They must be explicitly loaded.''' ==== Test Macros Test macros are used to generate a test result object. A test procedure body is composed of a sequence of test macros. The test macros are useful only within a test procedure or a test container. A runtime error will result when used outside of a test procedure. ==== Test Containers Test containers are test forms that group test forms. Test container are available in long and short forms. The short form provides fewer options. A test container imposes an evaluation termination mode on the contained tests. Test containers may nest. ===== test-suite <macro>(test-suite TEST-NAME DTOR-NAME ESCR-NAME [(warn MESSAGE)] [((VARIABLE VALUE) ...)] CLAUSE ...)</macro> <macro>(test/suite TEST-NAME [((VARIABLE VALUE) ...)] CLAUSE ...)</macro> Evaluates all contained test forms, {{CLAUSE}}, in a left to right order, thru any failures. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. Note that {{#f}} is interpreted as a missing name. Further, {{#f}} is assumed when a name is missing. Thus, the name of all unnamed test elements is the same - {{#f}}. {{DTOR-NAME}} is an unquoted symbol for an automatic destructor object that gets called when the test suite completes for any reason. This symbol is bound to a destructor object and is available to you in the {{CLAUSE ...}} section of the test suite. See below for the description of the destructor object interface. {{test/suite-dtor}} is the {{test/suite}} destructor name. {{ESCR-NAME}} is an unquoted symbol for an escape procedure available in the body of the test suite. Usually this escape procedure is passed to {{(terminate ...)}} which calls it for you and performs other tasks. It is not recommended to call the escape procedure directly. {{test/suite-escr}} is the {{test/suite}} escape procedure name. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the test suite. The {{warn}} function name is actually a syntax reserved word in the macro. {{((VARIABLE VALUE) ...)}} are let-style bindings that you may create and exist in the lexical scope of the test suite. {{CLAUSE ...}} are uses of a test container, annotation, or expectation macros. The clauses are evaluated in a left to right fashion. While you may use the expectation macros directly in a test suite, doing so has a drawback. If the expectation fails, the test suite macro will continue evaluating until all clauses are evaluated or the escape procedure mechanism is activated. This is different from a {{test-case}} macro where upon discovery of a failed expectation, evaluation stops immediately. ===== test-case <macro>(test-case TEST-NAME DTOR-NAME ESCR-NAME [(warn MESSAGE)] [((VARIABLE VALUE) ...)] CLAUSE ...)</macro> <macro>(test/case TEST-NAME (warn MESSAGE)] [((VARIABLE VALUE) ...)] CLAUSE ...)</macro> Evaluates all contained test forms, {{CLAUSE}}, in a left to right order, until failure. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{DTOR-NAME}} is an unquoted symbol for an automatic destructor object that gets called when the test case completes for any reason. This symbol is bound to a destructor object and is available to you in the {{CLAUSE ...}} section of the test case. See below for the description of the destructor object interface. {{test/case-dtor}} is the {{test/case}} destructor name. {{ESCR-NAME}} is an unquoted symbol for an escape procedure available in the body of the test case. Usually this escape procedure is passed to {{(terminate ...)}} which calls it for you and performs other tasks. It is not recommended to call the escape procedure directly. {{test/case-escr}} is the {{test/case}} escape procedure name. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the test case. The {{warn}} function name is actually a syntax reserved word in the macro. {{((VARIABLE VALUE) ...)}} are let-style bindings that you may create and exist in the lexical scope of the test case. {{CLAUSE ...}} are uses of a test container, annotation, or expectation macros. The clauses are evaluated in a left to right fashion. '''Note''': Upon discovery of a failed expectation, the test case stops its evaluation and returns with the test results up thru the failed expectation. This behavior can be changed using the {{test-case-evaluate-thru-failure}} macro. Reverts to default at the start of each {{test-case}}. ===== test-collect <macro>(test-collect TEST-NAME DTOR-NAME ESCR-NAME [(warn MESSAGE)] [((VARIABLE VALUE) ...)] EXPRESSION ...)</macro> <macro>(test/collect TEST-NAME (warn MESSAGE)] [((VARIABLE VALUE) ...)] EXPRESSION ...)</macro> A special version of {{test-case}}. Since every test form must evaluate to a test-result the use of looping forms to generate test values is very difficult. The collect container provides a method to gather the results of such generated tests. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{DTOR-NAME}} is an unquoted symbol for an automatic destructor object that gets called when the test collect completes for any reason. This symbol is bound to a destructor object and is available to you in the {{CLAUSE ...}} section of the test collect. See below for the description of the destructor object interface. {{test/collect-dtor}} is the {{test/collect}} destructor name. {{ESCR-NAME}} is an unquoted symbol for an escape procedure available in the body of the test collect. Usually this escape procedure is passed to {{(terminate ...)}} which calls it for you and performs other tasks. It is not recommended to call the escape procedure directly. {{test/collect-escr}} is the {{test/collect}} escape procedure name. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the test collect. The {{warn}} function name is actually a syntax reserved word in the macro. {{((VARIABLE VALUE) ...)}} are let-style bindings that you may create and exist in the lexical scope of the test collect. {{EXPRESSION}} is either an arbitrary Scheme expression or the form {{(collect-test CLAUSE)}}, where {{CLAUSE}} is a use of a test container, annotation, or expectation macro. The expressions are evaluated in a left to right fashion. The {{collect-test}} macro "appends" the test result of the {{CLAUSE}} to the results of the enclosing test collect container. '''Note''': Upon discovery of a failed expectation, the test collect stops its evaluation and returns with the test results up thru the failed expectation. This behavior can be changed using the {{test-case-evaluate-thru-failure}} macro. Reverts to default at the start of each {{test-case}}. ==== Test Flags These macros set test execution flags. ===== test-case-evaluate-thru-failure <macro>(test-case-evaluate-thru-failure BOOLEAN)</macro> This macro will set the test-case evaluation continuation flag. The starting value is {{#f}}. ===== test-timing <macro>(test-timing BOOLEAN)</macro> This macro will set the expectation timing flag. The starting value is {{#f}}. Note that if timing expansion is turned off setting this flag has no effect. ===== test-evaluation-mode? <macro>(test-evaluation-mode?)</macro> This macro will expand into a test for the evaluation mode. Potentially useful within test procedure {{initial}} and {{final}} forms. ==== Test Binding Forms ===== test-letrec <macro>(test-letrec ((VARIABLE VALUE) ...) CLAUSE ...)</macro> Defines {{VARIABLE}}, in a new lexical scope, as the result of evaluating {{EXPRESSION}}. {{CLAUSE ...}} are as for {{test-suite}}. Evaluated in the scope of the definitions. The last {{CLAUSE ...}} in the clause body <em>must</em> be a test form of some kind, such as an expectation. ===== test-let* <macro>(test-let* ((VARIABLE VALUE) ...) CLAUSE ...)</macro> Defines {{VARIABLE}}, in a new lexical scope, as the result of evaluating {{EXPRESSION}}. {{CLAUSE ...}} are as for {{test-suite}}. Evaluated in the scope of the definitions. The last {{CLAUSE ...}} in the clause body ''must'' be a test form of some kind, such as an expectation. ===== test-let <macro>(test-let ((VARIABLE VALUE) ...) CLAUSE ...)</macro> Defines {{VARIABLE}}, in a new lexical scope, as the result of evaluating {{EXPRESSION}}. {{CLAUSE ...}} are as for {{test-suite}}. Evaluated in the scope of the definitions. The last {{CLAUSE ...}} in the clause body <em>must</em> be a test form of some kind, such as an expectation. ==== Test Expectations An expectation at its core simply evaluates its arguments and check to see if it matches the expectation. The positive or negative result is encapsulated, along with other things such as the unevaluated expressions being checked and some messages supplied with each expectation into a particular type of black box object that one can query with the appropriate API calls (detailed below). Expectations all have a descriptive message that can be bound to them, along with an optional warning syntax detailed below. A design decision was made to supply expectation macros for the usual types of expectations a user needs because this reduced the abstractness of an expectation into something more manageable. Expectations are normally evaluated with an exception catcher. An unexpected exception is treated as expectation failure. The option may be defeated by redefining {{test::if-catching}}. Any exceptions in setup or teardown expressions will be caught and treated as an abnormal termination of the entire test. ===== insist <macro>(insist [TEST-NAME] [(warn MESSAGE)] EXPRESSION => EXPECTED)</macro> <macro>(insist [TEST-NAME] [(warn MESSAGE)] EXPRESSION (=> PREDICATE) EXPECTED)</macro> This expectation checks to see if the evaluated expression passed to it meets the predicate condition. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{PREDICATE}} is a procedure of two arguments to perform the test. The default is {{equal?}}. {{EXPECTED}} is evaluated and represents the value the {{EXPRESSION}} <em>must</em> match via {{PREDICATE}} in order for this expectation to return a positive result. {{EXPRESSION}} should return a value meeting the conditions of the predicate. ===== expect-unary <macro>(expect-unary [TEST-NAME] [(warn MESSAGE)] KIND PREDICATE EXPRESSION)</macro> This expectation checks to see if the evaluated expression passed to it meets the predicate condition. Allows unary and binary predicates. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{KIND}} is a string stating the kind of test. {{PREDICATE}} is a procedure of one argument to perform the test. {{EXPRESSION}} should return a value meeting the conditions of the predicate. ===== expect-binary <macro>(expect-binary [TEST-NAME] [(warn MESSAGE)] KIND PREDICATE EXPECTED EXPRESSION)</macro> This expectation checks to see if the evaluated expression passed to it meets the predicate condition. Allows unary and binary predicates. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{KIND}} is a string stating the kind of test. {{PREDICATE}} is a procedure of two arguments to perform the test. {{EXPECTED}} is evaluated and represents the value the {{EXPRESSION}} <em>must</em> match via {{PREDICATE}} in order for this expectation to return a positive result. {{EXPRESSION}} should return a value meeting the conditions of the predicate. ===== expect-zero <macro>(expect-zero [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro> This expectation checks to see if the evaluated expression passed to it is numerically equal to zero. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPRESSION}} should return a number. ===== expect-nonzero <macro>(expect-nonzero [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro> This expectation checks to see if the evaluated expression passed to it is numerically not equal to zero. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPRESSION}} should return a number. ===== expect-positive <macro>(expect-positive [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro> This expectation checks to see if the evaluated expression passed to it is a positive value greater than zero. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPRESSION}} should return a number. ===== expect-negative <macro>(expect-negative [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro> This expectation checks to see if the evaluated expression passed to it is a negative value less than zero. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPRESSION}} should return a number. ===== expect-near <macro>(expect-near [TEST-NAME] [(warn MESSAGE)] EXPECTED [TOL] EXPRESSION)</macro> This expectation checks to see if {{(< (abs (- EXPECTED EXPRESSION)) TOL)))}} is true. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPECTED}} is evaluated and represents the value the {{EXPRESSION}} <em>must</em> be "near" to in order for this expectation to return a positive result. {{EXPRESSION}} should return a number. {{TOL}} is a single expression which, when evaluated, must return a tolerance value, a small inexact number. When missing the current tolerance value is used, usually {{0.0001}}. The {{(expect-near EXPECTED TOL EXPRESSION)}} variant is not supported! ===== expect-true <macro>(expect-true [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro> This expectation checks to see if the evaluated expression passed to it is the value {{#t}}. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPRESSION}} should return {{#t}}. ===== expect-false <macro>(expect-false [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro> This expectation checks to see if the evaluated expression passed to it is the value {{#f}}. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPRESSION}} should return {{#f}}. ===== expect-success <macro>(expect-success [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro> This expectation checks to see if the evaluated expression passed to it is not the value {{#f}}, or a non-error exception occurred. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPRESSION}} should return {{#t}}, or generate a non-error exception. ===== expect-failure <macro>(expect-failure [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro> This expectation checks to see if the evaluated expression passed to it is the value {{#f}}, or an error exception occurred. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPRESSION}} should return {{#f}}, or generate an error exception. ===== expect-not-false <macro>(expect-not-false [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro> This expectation checks to see if the evaluated expression passed to it is not the value {{#f}}. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPRESSION}} should not return {{#f}}, any other value is accepted. ===== expect-not-null <macro>(expect-not-null [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro> This expectation checks to see if the evaluated expression passed to it is not the value {{#f}}. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPRESSION}} should not return {{#f}}, any other value is accepted. ===== expect-eq <macro>(expect-eq [TEST-NAME] [(warn MESSAGE)] EXPECTED EXPRESSION)</macro> This expectation checks to see if {{(eq? EXPECTED EXPRESSION)}} is true. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPECTED}} is evaluated and represents the value the {{EXPRESSION}} ''must'' be eq? to in order for this expectation to return a positive result. {{EXPRESSION}} is a single expression which, when evaluated must return an object where an eq? of this result and the {{EXPECTED}} expression is {{#t}}. The result object this macro produce shall contain the unevaluated {{EXPRESSION}} expression as a field, but not an unevaluated {{EXPECTED}} expression. ===== expect-eqv <macro>(expect-eqv [TEST-NAME] [(warn MESSAGE)] EXPECTED EXPRESSION)</macro> This expectation checks to see if {{(eqv? EXPECTED EXPRESSION)}} is true. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPECTED}} is evaluated and represents the value the {{EXPRESSION}} <em>must</em> be eqv? to in order for this expectation to return a positive result. {{EXPRESSION}} is a single expression which, when evaluated must return an object where an eqv? of this result and the {{EXPECTED}} expression is {{#t}}. The result object this macro produce shall contain the unevaluated {{EXPRESSION}} expression as a field, but not an unevaluated {{EXPECTED}} expression. ===== expect-equal <macro>(expect-equal [TEST-NAME] [(warn MESSAGE)] EXPECTED EXPRESSION)</macro> This expectation checks to see if {{(equal? EXPECTED EXPRESSION)}} is true. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPECTED}} is evaluated and represents the value the {{EXPRESSION}} <em>must</em> be equal? to in order for this expectation to return a positive result. {{EXPRESSION}} is a single expression which, when evaluated must return an object where an equal? of this result and the {{EXPECTED}} expression is {{#t}}. The result object this macro produce shall contain the unevaluated {{EXPRESSION}} expression as a field, but not an unevaluated {{EXPECTED}} expression. ===== expect-unary/values <macro>(expect-unary/values [TEST-NAME] [(warn MESSAGE)] KIND (PREDICATE ...) EXPRESSION)</macro> This expectation checks to see if the evaluated multi-valued expression passed to it meets the predicate condition. Allows unary and binary predicates. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{KIND}} is a string stating the kind of test. {{PREDICATE}} is a procedure taking one argument and returning {{#t}} or {{#f}}. {{EXPRESSION}} has a multi-valued result. ===== expect-near/values <macro>(expect-near/values [TEST-NAME] [(warn MESSAGE)] KIND (PREDICATE ...) VALUES EXPRESSION)</macro> This expectation checks to see if the evaluated multi-valued expression passed to it meets the predicate condition. Allows unary and binary predicates. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{KIND}} is a string stating the kind of test. {{PREDICATE}} is a procedure taking two arguments and returning {{#t}} or {{#f}}. {{VALUES}} are the expected values. {{EXPRESSION}} has a multi-valued result. ===== expect-near/values <macro>(expect-near/values [TEST-NAME] [(warn MESSAGE)] EXPECTED [TOL] EXPRESSION)</macro> This expectation checks to see if {{(< (abs (- EXPECTED[i] EXPRESSION[i])) TOL)))}} is true. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPECTED}} is evaluated and represents the values the {{EXPRESSION}} <em>must</em> be "near" to in order for this expectation to return a positive result. {{EXPRESSION}} should return one or more inexact or exact number. {{TOL}} is a single expression which, when evaluated, must return a tolerance value, a small inexact number. When missing the current tolerance value is used, usually {{0.0001}}. The {{(expect-near/values EXPECTED TOL EXPRESSION)}} variant is not supported! ===== expect-eq/values <macro>(expect-eq/values [TEST-NAME] [(warn MESSAGE)] VALUES EXPRESSION)</macro> This expectation checks to see if the result of the evaluated multi-valued expression passed to it is element-wise eq? to the VALUES list. The count of returned multiple values must match the length of the VALUES list. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{VALUES}} is the expected values list. {{EXPRESSION}} has a multi-valued result. ===== expect-eqv/values <macro>(expect-eqv/values [TEST-NAME] [(warn MESSAGE)] VALUES EXPRESSION)</macro> This expectation checks to see if the result of the evaluated multi-valued expression passed to it is element-wise eqv? to the VALUES list. The count of returned multiple values must match the length of the VALUES list. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{VALUES}} is the expected values list. {{EXPRESSION}} has a multi-valued result. ===== expect-equal/values <macro>(expect-equal/values [TEST-NAME] [(warn MESSAGE)] VALUES EXPRESSION)</macro> This expectation checks to see if the result of the evaluated multi-valued expression passed to it is element-wise equal? to the VALUES list. The count of returned multiple values must match the length of the VALUES list. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{VALUES}} is the expected values list. {{EXPRESSION}} has a multi-valued result. ===== expect-exception <macro>(expect-exception [TEST-NAME] [(warn MESSAGE)] EXCEPTION-PATTERN EXPRESSION)</macro> This expectation checks to see if the evaluated expression passed to it signals an exception that matches the supplied exception pattern {{EXCEPTION-PATTERN}}. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXCEPTION-PATTERN}} is a specialized logical expression describing the expected exception, a condition as defined by [[http://srfi.schemers.org/srfi-12/srfi-12.html|SRFI-12]]. {{EXPRESSION}} should signal an exception matching {{EXCEPTION-PATTERN}}. EXCEPTION-PATTERN : COND-EXN | SIMP-EXN | () SIMP-EXN : KINDKEY | (property KINDKEY PROP-EXPR ...) | (SIMP-EXN ...) COND-EXN : (and EXN ...) | (or EXN ...) | (not EXN) PROP-EXPR : COND-PROP-EXPR | SIMP-PROP-EXPR | () SIMP-PROP-EXPR : PROPKEY | (PROPKEY ...) | (PROPKEY VALUE) | (PREDICATE PROPKEY VALUE) COND-PROP-EXPR : (and PROP-EXPR ...) | (or PROP-EXPR ...) | (not PROP-EXPR) An example: <enscript highlight=scheme> (define-test ... ... (test-case "exn1" d e ( [exn1 (make-composite-condition (make-property-condition 'abc 'cbs "pbs") (make-property-condition 'foo 'bar "zip"))]) (expect-exception "1" (foo abc) (signal exn1)) ;; These are the same test, the 2nd has an explicit predicate (expect-exception "2" ((property foo (bar "zip")) (property abc (cbs "pbs"))) (signal exn1)) (expect-exception "3" ((property foo (equal? bar "zip")) (property abc (equal? cbs "pbs"))) (signal exn1)) ) ... ) </enscript> ===== expect-ec <macro>(expect-ec [TEST-NAME] [(warn MESSAGE)] QUALIFIER ... EXPECTED [=> PREDICATE] EXPRESSION)</macro> This expectation checks to see if {{(PREDICATE EXPECTED EXPRESSION)}} is true for every state produced by the set of {{QUALIFIER}}. {{TEST-NAME}} must evaluate to a Scheme {{atom}}. {{(warn MESSAGE)}} allows you to specify a warning object, usually a string, that gets associated with the expectation. The {{warn}} function name is actually a syntax reserved word in the macro. {{EXPRESSION}} and {{EXPECTED}} are expressions. {{PREDICATE}} is a two argument procedure returning a boolean, {{equal?}} is used when missing. {{QUALIFIER}} is a SRFI-42 <qualifier>. An eager comprehension implementation, as described by [[http://srfi.schemers.org/srfi-42/srfi-42.html|SRFI-42]] must be provided to use this expectation. ===== expect-set! <macro>(expect-set! [TEST-NAME] VARIABLE EXPRESSION)</macro> Evaluates {{EXPRESSION}} as in {{expect-success}}. If successful the variable {{VARIABLE}} is bound to the result of evaluating the {{EXPRESSION}}. {{EXPRESSION}} cannot evaluate to {{#f}} or a {{condition}}. ==== Define New Expectations An API to define new expectation macros. May only be used at the toplevel. ===== define-expect-unary <macro>(define-expect-unary PREDICATE [SUFFIX [KIND]])</macro> Defines a new unary expectation. {{PREDICATE}} is the name of a procedure of one argument to perform the test. {{SUFFIX}} is a symbol to create the expectation macro name. When missing the {{PREDICATE}} is used, without a {{?}} suffix, if any. {{KIND}} is a string stating the kind of test. When missing the string form of the {{SUFFIX}} is used. ===== define-expect-binary <macro>(define-expect-binary PREDICATE [SUFFIX [KIND]])</macro> Defines a new binary expectation. {{PREDICATE}} is the name of a procedure of two arguments to perform the test. {{SUFFIX}} is a symbol to create the expectation macro name. When missing the {{PREDICATE}} is used, without a {{?}} suffix, if any. {{KIND}} is a string stating the kind of test. When missing the string form of the {{SUFFIX}} is used. ===== define-expect-unary/values <macro>(define-expect-unary/values PREDICATE [SUFFIX [KIND]])</macro> Defines a new multivalued unary expectation. {{PREDICATE}} is the name of a procedure of one list argument to perform the test. {{SUFFIX}} is a symbol to create the expectation macro name. When missing the {{PREDICATE}} is used, without a {{?}} suffix, if any. {{KIND}} is a string stating the kind of test. When missing the string form of the {{SUFFIX}} is used. ===== define-expect-binary/values <macro>(define-expect-binary/values PREDICATE [SUFFIX [KIND]])</macro> Defines a new multivalued binary expectation. {{PREDICATE}} is the name of a procedure of two list arguments to perform the test. {{SUFFIX}} is a symbol to create the expectation macro name. When missing the {{PREDICATE}} is used, without a {{?}} suffix, if any. {{KIND}} is a string stating the kind of test. When missing the string form of the {{SUFFIX}} is used. ===== define-expect-nary <macro>(define-expect-nary (SUFFIX ARG ...) EXPR ...)</macro> Defines a new n-ary expectation with the name {{expect-SUFFIX}}. {{(expect-SUFFIX MESSAGE [(warning WARNING)] VAL ...)}}. Unlike other expectations either the MESSAGE or a WARNING must be supplied. {{SUFFIX}} is a symbol. {{ARG ...}} are the parameter variable symbols. {{EXPR ...}} is the body, which must return {{#f}} for failure. ==== Testeez Synonyms Syntax forms mimicking some of the '''testeez''' test framework forms. Expanded into the corresponding {{expect-*}} macros. The formal argument names match those in the testeez documentation. Following the testeez semantics all test forms are always evaluated as having possible multiple valued returns. ===== test-eval <macro>(test-eval [TEST-NAME] EXPRESSION)</macro> ===== test/eq <macro>(test/eq [TEST-NAME] EXPRESSION EXPECTED)</macro> ===== test/eqv <macro>(test/eqv [TEST-NAME] EXPRESSION EXPECTED)</macro> ===== test/equal <macro>(test/equal [TEST-NAME] EXPRESSION EXPECTED)</macro> ==== Test Setup (Side-Effect) The side effecting evaluates all of its arguments as in a {{(begin ...)}} form, it returns a result that is completely ignored by the system and unavailable to the output analysis code. ===== side-effect <macro>(side-effect CLAUSES)</macro> This macro expands into a begin form with the clauses in order. When it finishes evaluating them, a result is returned that is silently ignored by the testbase system. Usually this is used in conjunction with {{(set! ...)}} or with complicated situations where a lot of setup work must happen for an expectation to be performed. Define forms cannot be expressed here. ===== setup <macro>(setup CLAUSES)</macro> A synonym for {{side-effect}}. ==== Test Teardown (Destructor) The destructor object allows for you to create helper functions which clean up in case of aborting from a test container. For example, suppose you are testing whether or not file writing to a file works correctly in a test case, so, you'd perform an expectation to open the file, and then queue a function in the destructor to remove the file, and then perform the expectation of the write. If the write (or subsequent) expectation fails, then the test case will ''automatically'' invoke the helper cleanup function specified in the destructor object that removes the file. ===== destructor-atexit! <macro>(destructor-atexit! [DTOR-NAME] CALL ...)</macro> <macro>(destructor-atexit! DTOR-NAME PROC ARGUMENT ...)</macro> {{DTOR-NAME}} is the destructor to use, {{test-dtor}} when missing. A {{CALL}} is a list where the first element is a {{PROC}} and the tail elements are the argument list; i.e. {{(PROC ARGUMENT ...)}}. Each {{CALL}} is added to the destructor queue, promised to be invoked when the {{DTOR-NAME}} destructor object is activated. The second form will insert a promise to invoke {{(PROC ARGUMENT ...)}} into a queue in the {{DTOR-NAME}} destructor object. Multiple invocations of this API call will continue to queue up {{(PROC ARGS ...)}} promises indefinitely. '''Note''': The {{PROC}} and {{ARGUMENT ...}} parameters are evaluated at the point of enqueue; i.e. the current lexical, and dynamic, environment of the macro expansion. This means a variable referenced will have this value when the destructor is activated, and not the "current" value! So no "forward" references. This macro returns a special ''ignore-result'' type that is ignored by the testbase system. ===== teardown <macro>(teardown [DTOR-NAME] (FUNC ARGUMENT ...) ...)</macro> A {{(FUNC ARGUMENT ...)}} for is also refered to as a {{CALL}}. A synonym of {{destructor-atexit!}}. <macro>(destructor-activate! [DTOR-NAME])</macro> {{DTOR-NAME}} is the destructor to use, {{test-dtor}} when missing. This macro will call, in order of queueing, all the promises embedded into this destructor object, and then delete the queue. The destructor is ''always'' called at the completion of a test container; so be careful that the destructor object doesn't contain anything harmful. This macro returns a special ''ignore-result'' type that is ignored by the testbase system. ===== destructor-clear! <macro>(destructor-clear! [DTOR-NAME])</macro> {{DTOR-NAME}} is the destructor to use, {{test-dtor}} when missing. This macro completely removes all of the promises associated with the destructor object {{DTOR-NAME}}. This macro returns a special ''ignore-result'' type that is ignored by the testbase system. ===== destructor-dump <macro>(destructor-dump [DTOR-NAME])</macro> {{DTOR-NAME}} is the destructor to use, {{test-dtor}} when missing. This macro, mostly used for debugging purposes, prints out a simple representation of the queued atexit functions to the current port. This macro returns a special ''ignore-result'' type that is ignored by the testbase system. ==== Gloss The purpose of the gloss API is to allow the author of a test suite the ability to record messages into the result tree purely for documentation purposes. ===== gloss <macro>(gloss MESSAGE [(warn WARNING)])</macro> {{MESSAGE}} can be any scheme object, though usually it is a string. {{(warn WARNING)}} allows you to specify a warning object, usually a string, that gets associated with the gloss. The {{warn}} function name is actually a syntax reserved word in the macro. ==== Skip The purpose of the skip API is to allow the author of a test suite to completely skip evaluation of a set of expressions. ===== skip <macro>(skip MESSAGE [(warn WARNING)] CLAUSES)</macro> {{MESSAGE}} can be any scheme object, though usually it is a string. {{(warn WARNING)}} allows you to specify a warning object, usually a string, that gets associated with the gloss. The {{warn}} function name is actually a syntax reserved word in the macro. {{CLAUSES}} can be more than one expression (as in a lambda form) that does <em>NOT</em> get evaluated at any time. ==== Termination When executing in a test suite or a test case, one might discover some catastrophic failure of such proportions that it is utterly impossible to continue executing the test case or test suite. When that happens you can use the termination facility to exit the test case or test suite. Of course, no more expressions will be evaluated in the scope of the termination. It is recommended that you use this method of terminating the test procedure, test case or test suite evaluation since it wraps some contextual information up into the termination result so you can figure out what happened (and where) later when analyzing the result tree. ===== terminate <macro>(terminate MESSAGE)</macro> <macro>(terminate MESSAGE FINDING)</macro> <macro>(terminate ESCR-NAME MESSAGE)</macro> <macro>(terminate ESCR-NAME MESSAGE FINDING)</macro> This is the recommended termination method for a test procedure, test case, or a test suite. {{ESCR-NAME}} is the name of the termination procedure that was specified in a test procedure, test case or test suite. You may pass any test suite or test case termination function available to you in the lexical scope in which you call this function. The termination will take effect in the scope of the created termination function. {{MESSAGE}} can be any scheme object, though usually it is a string. {{FINDING}} can be any scheme object, though usually it is a boolean. ==== Todo The purpose of the todo API is to allow the author of a test suite the ability to record into the result tree for later analysis that something still needs to be done. This way you can count/manipulate this information at a later date. ===== todo <macro>(todo MESSAGE [(warn WARNING)])</macro> {{MESSAGE}} can be any scheme object, though usually it is a string. {{(warn WARNING)}} allows you to specify a warning object, usually a string, that gets associated with the todo. The {{warn}} function name is actually a syntax reserved word in the macro. ==== Test Result Object Test procedures, test containers, expectations, and helper macros ({{gloss}}, {{todo}}, etc) all return an object that contains the results and other various aspects of the action performed which ultimately get wired together to form the result tree. This collection of functions forming the rest of the testbase API allows manipulation of these results in an abstracted way as to allow changing of the representation in the future. ==== Any Test Result Object This section contains a few functions whose purpose is to simplify certain kinds of manipulations of result objects. Will generate an error for an invalid result object. ===== test::result? <procedure>(test::result? RESULT-OBJ)</procedure> This function will return {{#t}} if {{RESULT-OBJ}} is any kind of an evaluated result object. Otherwise, it shall return {{#f}}. ===== test::ignore-result? <procedure>(test::ignore-result? RESULT-OBJ)</procedure> This function will return {{#t}} if {{RESULT-OBJ}} is an ignore result object. Otherwise, it shall return {{#f}}. An ignore object is not considered an evaluated result object. ===== test::result-passed? <procedure>(test::result-passed? RESULT-OBJ)</procedure> This function will return whether the {{boolean>}} finding of any kind of {{RESULT-OBJ}} passed to it if it is indeed a true result object. ===== test::result-finding <procedure>(test::result-finding RESULT-OBJ)</procedure> This function will return the finding of any kind of {{RESULT-OBJ}} passed to it if it is indeed a true result object. The finding is a variant type: {{boolean}} | {{condition}}. ===== test::result-name <procedure>(test::result-name RESULT-OBJ)</procedure> This function will return the {{TEST-NAME}} of any kind of {{RESULT-OBJ}} passed to it if it is indeed a true result object. ===== test::result-message <procedure>(test::result-message RESULT-OBJ)</procedure> This function will return the {{MESSAGE}} of any kind of {{RESULT-OBJ}} passed to it if it is indeed a true result object. ===== test::result-warning? <procedure>(test::result-warning? RESULT-OBJ)</procedure> This function will return whether a warning is defined if {{RESULT-OBJ}} is any kind of an evaluated result object. ===== test::result-warning <procedure>(test::result-warning RESULT-OBJ)</procedure> This function will return the warning of any kind of {{RESULT-OBJ}} passed to it if it is indeed a true result object. ===== test::result-timing? <procedure>(test::result-timing? RESULT-OBJ)</procedure> This function will return whether a timing is defined if {{RESULT-OBJ}} is any kind of an evaluated result object. ===== test::result-timing <procedure>(test::result-timing RESULT-OBJ)</procedure> This function will return the timing values of any kind of {{RESULT-OBJ}} passed to it if it is indeed a true result object. The timing values will be a pair of the milliseconds spent in user code and in system code, for the test corresponding to this test::result-object. For example, the list {{(6 . 1)}} indicates 6 milliseconds spent in user code and 1 millisecond spent in system code. ===== test::result-timing-message <procedure>(test::result-timing-message RESULT-OBJ)</procedure> This function will return the timing string representation of any kind of {{RESULT-OBJ}} passed to it if it is indeed a true result object. ===== test::result-kind <procedure>(test::result-kind RESULT-OBJ)</procedure> This function will return the kind of any {{RESULT-OBJ}} passed to it if it is indeed a true result object. ==== Test Container Result Object ==== Test Suite Result Object If any of these API functions, except ''test::test-suite-result?'', are passed something that isn't a test suite result object, they will return an undefined value. ===== test::test-suite-result? <procedure>(test::test-suite-result? RESULT)</procedure> If {{RESULT}} is a result object from the invocation of a test suite macro, then this function will return {{#t}}. Otherwise, it will return {{#f}}. ===== test::test-suite-result-passed? <procedure>(test::test-suite-result-passed? RESULT)</procedure> Returns the boolean result associated with the test suite {{RESULT}} object. ===== test::test-suite-result-finding <procedure>(test::test-suite-result-finding RESULT)</procedure> Returns the finding result associated with the test suite {{RESULT}} object. ===== test::test-suite-result-name <procedure>(test::test-suite-result-name RESULT)</procedure> Returns the message object associated with the test suite {{RESULT}} object. ===== test::test-suite-result-list <procedure>(test::test-suite-result-list RESULT)</procedure> Returns the list of result objects associated with the test suite {{RESULT}} object. ===== test::test-suite-result-warning? <procedure>(test::test-suite-result-warning? RESULT)</procedure> If a warning had been attached to this test suite, this function will return {{#t}}, otherwise it will be {{#f}}. ===== test::test-suite-result-warning <procedure>(test::test-suite-result-warning RESULT)</procedure> If a warning had been attached to this test suite, this function will return the warning message object supplied by the user, otherwise it shall return {{'()}}. ===== test::test-suite-result-timing? <procedure>(test::test-suite-result-timing? RESULT)</procedure> If a timing had been attached to this test suite, this function will return {{#t}}, otherwise it will be {{#f}}. ===== test::test-suite-result-timing <procedure>(test::test-suite-result-timing RESULT)</procedure> If a timing had been attached to this test suite, this function will return the timing pair, otherwise it shall return {{'(0 . 0)}}. ===== test::test-suite-results-passed? <procedure>(test::test-suite-results-passed? RESULT-LIST)</procedure> Returns {{#t}} if every single contained result in the {{RESULT-LIST}} passed, or {{#f}} otherwise. ==== Test Case Result Object If any of these API functions, except {{test::test-case-result?}}, are passed something that isn't a test case result object, they will return an undefined value. ===== test::test-case-result? <procedure>(test::test-case-result? RESULT)</procedure> If {{RESULT}} is a result object from the invocation of a test case macro, then this function will return {{#t}}. Otherwise, it will return {{#f}}. ===== test::test-case-result-passed? <procedure>(test::test-case-result-passed? RESULT)</procedure> Returns the boolean result associated with the test case {{RESULT}} object. ===== test::test-case-result-finding <procedure>(test::test-case-result-finding RESULT)</procedure> Returns the finding result associated with the test case {{RESULT}} object. ===== test::test-case-result-name <procedure>(test::test-case-result-name RESULT)</procedure> Returns the message object associated with the test case {{RESULT}} object. ===== test::test-case-result-list <procedure>(test::test-case-result-list RESULT)</procedure> Returns the list of result objects associated with the test case {{RESULT}} object. ===== test::test-case-result-warning? <procedure>(test::test-case-result-warning? RESULT)</procedure> If a warning had been attached to this test case, this function will return {{#t}}, otherwise it will be {{#f}}. ===== test::test-case-result-warning <procedure>(test::test-case-result-warning RESULT)</procedure> If a warning had been attached to this test case, this function will return the warning message object supplied by the user, otherwise it shall return {{'()}}. ===== test::test-case-result-timing? <procedure>(test::test-case-result-timing? RESULT)</procedure> If a timing had been attached to this test case, this function will return {{#t}}, otherwise it will be {{#f}}. ===== test::test-case-result-timing <procedure>(test::test-case-result-timing RESULT)</procedure> If a timing had been attached to this test case, this function will return the timing pair, otherwise it shall return {{'(0 . 0)}}. ===== test::test-case-results-passed? <procedure>(test::test-case-results-passed? RESULT-LIST)</procedure> Returns {{#t}} if every single contained result in the {{RESULT-LIST}} passed, or {{#f}} otherwise. ==== Expectation Result Objects Procedures used to access the various kinds of expectation result object. ==== Unary Expectation These expectations all take the form of passing a single expression to them to see if they match some a priori expectation. If any of these API functions, except {{test::expect-result?}}, are passed something that isn't a single clause style expectation result object, they will return {{'not-an-expect-result}}. ===== test::expect-result? <procedure>(test::expect-result? RESULT)</procedure> If {{RESULT}} is a single clause style result object from the invocation of an expectation macro, then this function will return {{#t}}. Otherwise, it will return {{#f}}. ===== test::expect-result-finding <procedure>(test::expect-result-finding RESULT)</procedure> Returns the boolean result associated with the single clause style expectation {{RESULT}} object. ===== test::expect-result-kind <procedure>(test::expect-result-kind RESULT)</procedure> This retrieves the "kind" field of a particular single clause style expectation. For example, if you had a result object from an invocation of a {{(expect-zero "foobar" (- 1 1))}} expectation, then the "kind" field of the expectation result object will be the string {{"zero"}}. Here is a table describing what the "kind" fields are for each kind of single clause style expectation: <table border="1"> <tr><th>Unary Expectation</th><th>Associated Specific String</th></tr> <tr><td>expect-zero</td> <td>"zero"</td></tr> <tr><td>expect-nonzero</td> <td>"nonzero"</td></tr> <tr><td>expect-true</td> <td>"true"</td></tr> <tr><td>expect-false</td> <td>"false"</td></tr> <tr><td>expect-not-false</td> <td>"not-false"</td></tr> <tr><td>expect-not-null</td> <td>"not-null"</td></tr> <tr><td>expect-positive</td> <td>"positive"</td></tr> <tr><td>expect-negative</td> <td>"negative"</td></tr> <tr><td>expect-unary</td> <td>{{KIND}}</td></tr> <tr><td>expect-unary/values</td> <td>{{KIND}}</td></tr> </table> ===== test::expect-result-name <procedure>(test::expect-result-name RESULT)</procedure> Returns the message object associated with the single clause style expectation {{RESULT}} object. ===== test::expect-result-unevaled <procedure>(test::expect-result-unevaled RESULT)</procedure> Returns the unevaluated expression supplied to a single clause style expectation macro. ===== test::expect-result-evaled <procedure>(test::expect-result-evaled RESULT)</procedure> Returns the evaluated expression supplied to a single clause style expectation macro. ===== test::expect-result-warning? <procedure>(test::expect-result-warning? RESULT)</procedure> If a warning had been attached to this expectation, this function will return {{#t}}, otherwise it will be {{#f}}. ===== test::expect-result-warning <procedure>(test::expect-result-warning RESULT)</procedure> If a warning had been attached to this expectation, this function will return the warning message object supplied by the user, otherwise it shall return {{'()}}. ===== test::expect-result-timing? <procedure>(test::expect-result-timing? RESULT)</procedure> If a timing had been attached to this expectation, this function will return {{#t}}, otherwise it will be {{#f}}. ===== test::expect-result-timing <procedure>(test::expect-result-timing RESULT)</procedure> If a timing had been attached to this expectation, this function will return the timing pair, otherwise it shall return {{'(0 . 0)}}. ==== Binary Expectation These expectations all take the form of passing a two expressions, the "left hand side" and the "right hand side" to them to see if they match some a priori equivalence. The left hand side is that which you expect the right hand side to be equivalent. If any of these API functions, except {{test::expect-equivalence-result?}}, are passed something that isn't a single clause style expectation result object, they will return {{'not-an-expect-equivalence-result}}. ===== test::expect-equivalence-result? <procedure>(test::expect-equivalence-result? RESULT)</procedure> If {{RESULT}} is a comparison style result object from the invocation of an expectation macro, then this function will return {{#t}}. Otherwise, it will return {{#f}}. ===== test::expect-equivalence-result-finding <procedure>(test::expect-equivalence-result-finding RESULT)</procedure> Returns the boolean result associated with the comparison style expectation {{RESULT}} object. ===== test::expect-equivalence-result-kind <procedure>(test::expect-equivalence-result-kind RESULT)</procedure> This retrieves the "kind" field of a particular equivalence style expectation. For example, if you had a result object from an invocation of a {{(expect-equal? "foobar" 0 (- 1 1))}} expectation, then the "kind" field of the expectation result object will be the string {{"equal"}}. Here is a table describing what the "kind" fields are for each kind of equivalence style expectation: <table border="1"><tr><th>Binary Expectation</th><th>Associated Specific String</th></tr> <tr><td>expect-eq</td> <td>"eq"</td></tr> <tr><td>expect-eqv</td> <td>"eqv"</td></tr> <tr><td>expect-equal</td> <td>"equal"</td></tr> <tr><td>expect-exception</td> <td>"exception"</td></tr> <tr><td>expect-binary</td> <td>{{KIND}}</td></tr> <tr><td>expect/values</td> <td>{{"values"}}</td></tr> <tr><td>expect-eq/values</td> <td>"eq/values"</td></tr> <tr><td>expect-eqv/values</td> <td>"eqv/values"</td></tr> <tr><td>expect-equal/values</td> <td>"equal/values"</td></tr> <tr><td>expect-binary/values</td> <td>{{KIND}}</td></tr> </table> ===== test::expect-equivalence-result-name <procedure>(test::expect-equivalence-result-name RESULT)</procedure> Returns the message object associated with the equivalence style expectation {{RESULT}} object. ===== test::expect-equivalence-result-lhs-evaled <procedure>(test::expect-equivalence-result-lhs-evaled RESULT)</procedure> Returns the evaluated "left hand side" expression supplied to an equivalence style expectation. ===== test::expect-equivalence-result-rhs-unevaled <procedure>(test::expect-equivalence-result-rhs-unevaled RESULT)</procedure> Returns the unevaluated "right hand side" expression supplied to an equivalence style expectation. ===== test::expect-equivalence-result-rhs-evaled <procedure>(test::expect-equivalence-result-rhs-evaled RESULT)</procedure> Returns the evaluated "right hand side" expression supplied to an equivalence style expectation. ===== test::expect-equivalence-result-warning? <procedure>(test::expect-equivalence-result-warning? RESULT)</procedure> If a warning had been attached to this expectation, this function will return {{#t}}, otherwise it will be {{#f}}. ===== test::expect-equivalence-result-warning <procedure>(test::expect-equivalence-result-warning RESULT)</procedure> If a warning had been attached to this expectation, this function will return the warning message object supplied by the user, otherwise it shall return {{'()}}. ===== test::expect-equivalence-result-timing? <procedure>(test::expect-equivalence-result-timing? RESULT)</procedure> If a timing had been attached to this expectation, this function will return {{#t}}, otherwise it will be {{#f}}. ===== test::expect-equivalence-result-timing <procedure>(test::expect-equivalence-result-timing RESULT)</procedure> If a timing had been attached to this expectation, this function will return the timing pair, otherwise it shall return {{'(0 . 0)}}. ==== Tolerance Style Expectation This is a specialized expectation which accepts three expressions and checks to see if the "right hand side" is within a "tolerance" of the "left hand side". There is only one expectation in the tolerance style currently. If any of these API functions, except {{test::expect-tolerance-result?}}, are passed something that isn't a tolerance style expectation result object the result is unspecified. ===== test::expect-tolerance-result? <procedure>(test::expect-tolerance-result? RESULT)</procedure> If {{RESULT}} is a tolerance style result object from the invocation of an expectation macro, then this function will return {{#t}}. Otherwise, it will return {{#f}}. ===== test::expect-tolerance-result-finding <procedure>(test::expect-tolerance-result-finding RESULT)</procedure> Returns the boolean result associated with the tolerance style expectation {{RESULT}} object. ===== test::expect-tolerance-result-kind <procedure>(test::expect-tolerance-result-kind RESULT)</procedure> This retrieves the "kind" field of a particular tolerance style expectation. For example, if you had a result object from an invocation of a {{(expect-near? "foobar" 100 .01 100.001)}} expectation, then the "kind" field of the expectation result object will be the string {{"near"}}. Here is a table describing what the "kind" fields are for each kind of tolerance style expectation: <table border="1"><tr><th>Tolerance Style Expectation</th><th>Associated Specific String</th></tr> <tr><td>expect-near</td> <td>"near"</td></tr> </table> ===== test::expect-tolerance-result-name <procedure>(test::expect-tolerance-result-name RESULT)</procedure> Returns the message object associated with a tolerance style expectation {{RESULT}} object. ===== test::expect-tolerance-result-lhs-evaled <procedure>(test::expect-tolerance-result-lhs-evaled RESULT)</procedure> Returns the evaluated "left hand side" expression supplied to a tolerance style expectation. ===== test::expect-tolerance-result-lhs-tol-evaled <procedure>(test::expect-tolerance-result-lhs-tol-evaled RESULT)</procedure> Returns the evaluated "tolerance" expression supplied to a tolerance style expectation. ===== test::expect-tolerance-result-rhs-unevaled <procedure>(test::expect-tolerance-result-rhs-unevaled RESULT)</procedure> Returns the unevaluated "right hand side" expression supplied to a tolerance style expectation. ===== test::expect-tolerance-result-rhs-evaled <procedure>(test::expect-tolerance-result-rhs-evaled RESULT)</procedure> Returns the evaluated "right hand side" expression supplied to a tolerance style expectation. ===== test::expect-tolerance-result-warning? <procedure>(test::expect-tolerance-result-warning? RESULT)</procedure> If a warning had been attached to this expectation, this function will return {{#t}}, otherwise it will be {{#f}}. ===== test::expect-tolerance-result-warning <procedure>(test::expect-tolerance-result-warning RESULT)</procedure> If a warning had been attached to this expectation, this function will return the warning message object supplied by the user, otherwise it shall return {{'()}}. ===== test::expect-tolerance-result-timing? <procedure>(test::expect-tolerance-result-timing? RESULT)</procedure> If a timing had been attached to this expectation, this function will return {{#t}}, otherwise it will be {{#f}}. ===== test::expect-tolerance-result-timing <procedure>(test::expect-tolerance-result-timing RESULT)</procedure> If a timing had been attached to this expectation, this function will return the timing pair, otherwise it shall return {{'(0 . 0)}}. ===== Gloss Result Object ===== test::gloss-result? <procedure>(test::gloss-result? RESULT)</procedure> If {{RESULT}} is a gloss result object from the invocation of the gloss macro, then this function will return {{#t}}. Otherwise, it will return {{#f}}. ===== test::gloss-result-message <procedure>(test::gloss-result-message RESULT)</procedure> Returns the message object associated with the gloss {{RESULT}} object. ===== test::gloss-result-warning? <procedure>(test::gloss-result-warning? RESULT)</procedure> If a warning had been attached to this gloss, this function will return {{#t}}, otherwise it will be {{#f}}. ===== test::gloss-result-warning <procedure>(test::gloss-result-warning RESULT)</procedure> If a warning had been attached to this gloss, this function will return the warning message object supplied by the user, otherwise it shall return {{'()}}. ===== Skip Result Object ===== test::skip-result? <procedure>(test::skip-result? RESULT)</procedure> If {{RESULT}} is a skip result object from the invocation of the skip macro, then this function will return {{#t}}. Otherwise, it will return {{#f}}. ===== test::skip-result-message <procedure>(test::skip-result-message RESULT)</procedure> Returns the message object associated with the skip {{RESULT}} object. Hopefully, it was stated why this set of clauses had been skipped. ===== test::skip-result-warning? <procedure>(test::skip-result-warning? RESULT)</procedure> If a warning had been attached to this skip, this function will return {{#t}}, otherwise it will be {{#f}}. ===== test::skip-result-warning <procedure>(test::skip-result-warning RESULT)</procedure> If a warning had been attached to this skip, this function will return the warning message object supplied by the user, otherwise it shall return {{'()}}. ===== Termination Result Object When using the manipulation API for a terminate result, if you pass a result to one of these function that is not a terminate result, it will return an unspecified value. ===== test::terminate-result? <procedure>(test::terminate-result? RESULT)</procedure> If {{RESULT}} is a termination result object from the invocation of a termination function, then this function will return {{#t}}. Otherwise, it will return {{#f}}. ===== test::terminate-result-finding <procedure>(test::terminate-result-finding RESULT)</procedure> Returns the result associated with the termination function {{RESULT}} object. ===== test::terminate-result-scope <procedure>(test::terminate-result-scope RESULT)</procedure> The "scope" of the termination result is exactly the {{TEST-NAME}} parameter supplied to the test case or test suite associated with the {{TERMFUNC}}. ===== test::terminate-result-container <procedure>(test::terminate-result-container RESULT)</procedure> The "container" of the termination result is going to be either {{'test-suite}} or {{'test-case}} depending upon which the {{TERMFUNC}} was associated. ===== test::terminate-result-message <procedure>(test::terminate-result-message RESULT)</procedure> Returns the message object associated with the termination {{RESULT}} object. ===== Todo Result Object ===== test::todo-result? <procedure>(test::todo-result? RESULT)</procedure> If {{RESULT}} is a todo result object from the invocation of a todo macro, then this function will return {{#t}}. Otherwise, it will return {{#f}}. ===== test::todo-result-message <procedure>(test::todo-result-message RESULT)</procedure> Returns the message object associated with the todo {{RESULT}} object. ===== test::todo-result-warning? <procedure>(test::todo-result-warning? RESULT)</procedure> If a warning had been attached to this todo, this function will return {{#t}}, otherwise it will be {{#f}}. ===== test::todo-result-warning <procedure>(test::todo-result-warning RESULT)</procedure> If a warning had been attached to this todo, this function will return the warning message object supplied by the user, otherwise it shall return {{'()}}. ==== Miscellaneous ===== test::walk-structure <procedure>(test::walk-structure WALKER TEST-STRUCTURE)</procedure> This procedure will enumerate the {{TEST-STRUCTURE}} association list, invoking the {{WALKER}} for each element. The hierarchy level value is the tree depth at the point of call. {{WALKER}} is a three argument procedure. The first argument is the hierarchy level, counting from 0, the outermost level. The second argument is the type of the test element. The third argument is the name of the test element. ==== Statistics Generation ===== test::stat-result-statistics <procedure>(test::stat-result-statistics RESULT-TREE)</procedure> This function will compute and return a vector of result {{STATISTICS}}. Use the procedures below to access the elements by name. ===== test::stat-test-suites <procedure>(test::stat-test-suites STATISTICS)</procedure> ===== test::stat-test-suite-warnings <procedure>(test::stat-test-suite-warnings STATISTICS)</procedure> ===== test::stat-test-suites-passed <procedure>(test::stat-test-suites-passed STATISTICS)</procedure> ===== test::stat-test-suites-failed <procedure>(test::stat-test-suites-failed STATISTICS)</procedure> ===== test::stat-test-suites-timing <procedure>(test::stat-test-suites-timing STATISTICS)</procedure> ===== test::stat-test-suites-terminated <procedure>(test::stat-test-suites-terminated STATISTICS)</procedure> ===== test::stat-test-cases <procedure>(test::stat-test-cases STATISTICS)</procedure> ===== test::stat-test-case-warnings <procedure>(test::stat-test-case-warnings STATISTICS)</procedure> ===== test::stat-test-cases-passed <procedure>(test::stat-test-cases-passed STATISTICS)</procedure> ===== test::stat-test-cases-failed <procedure>(test::stat-test-cases-failed STATISTICS)</procedure> ===== test::stat-test-cases-timing <procedure>(test::stat-test-cases-timing STATISTICS)</procedure> ===== test::stat-test-cases-terminated <procedure>(test::stat-test-cases-terminated STATISTICS)</procedure> ===== test::stat-all-expectations <procedure>(test::stat-all-expectations STATISTICS)</procedure> ===== test::stat-all-expectation-warnings <procedure>(test::stat-all-expectation-warnings STATISTICS)</procedure> ===== test::stat-all-expectations-passed <procedure>(test::stat-all-expectations-passed STATISTICS)</procedure> ===== test::stat-all-expectations-failed <procedure>(test::stat-all-expectations-failed STATISTICS)</procedure> ===== test::stat-all-expectations-timing <procedure>(test::stat-all-expectations-timing STATISTICS)</procedure> ===== test::stat-single-expectations <procedure>(test::stat-single-expectations STATISTICS)</procedure> ===== test::stat-single-expectation-warnings <procedure>(test::stat-single-expectation-warnings STATISTICS)</procedure> ===== test::stat-single-expectations-passed <procedure>(test::stat-single-expectations-passed STATISTICS)</procedure> ===== test::stat-single-expectations-failed <procedure>(test::stat-single-expectations-failed STATISTICS)</procedure> ===== test::stat-single-expectations-timing <procedure>(test::stat-single-expectations-timing STATISTICS)</procedure> ===== test::stat-tol-expectations <procedure>(test::stat-tol-expectations STATISTICS)</procedure> ===== test::stat-tol-expectation-warnings <procedure>(test::stat-tol-expectation-warnings STATISTICS)</procedure> ===== test::stat-tol-expectations-passed <procedure>(test::stat-tol-expectations-passed STATISTICS)</procedure> ===== test::stat-tol-expectations-failed <procedure>(test::stat-tol-expectations-failed STATISTICS)</procedure> ===== test::stat-tol-expectations-timing <procedure>(test::stat-tol-expectations-timing STATISTICS)</procedure> ===== test::stat-equiv-expectations <procedure>(test::stat-equiv-expectations STATISTICS)</procedure> ===== test::stat-equiv-expectation-warnings <procedure>(test::stat-equiv-expectation-warnings STATISTICS)</procedure> ===== test::stat-equiv-expectations-passed <procedure>(test::stat-equiv-expectations-passed STATISTICS)</procedure> ===== test::stat-equiv-expectations-failed <procedure>(test::stat-equiv-expectations-failed STATISTICS)</procedure> ===== test::stat-equiv-expectations-timing <procedure>(test::stat-equiv-expectations-timing STATISTICS)</procedure> ===== test::stat-todos <procedure>(test::stat-todos STATISTICS)</procedure> ===== test::stat-todo-warnings <procedure>(test::stat-todo-warnings STATISTICS)</procedure> ===== test::stat-skips <procedure>(test::stat-skips STATISTICS)</procedure> ===== test::stat-skip-warnings <procedure>(test::stat-skip-warnings STATISTICS)</procedure> ===== test::stat-glosses <procedure>(test::stat-glosses STATISTICS)</procedure> ===== test::stat-gloss-warnings <procedure>(test::stat-gloss-warnings STATISTICS)</procedure> ===== test::stat-terminations <procedure>(test::stat-terminations STATISTICS)</procedure> ==== Printing Routines These procedures are used in the generation of test result reports. ===== test::write-object <procedure>(test::write-object OBJECT)</procedure> ===== test::display-objects <procedure>(test::display-objects [OBJECT ...])</procedure> ===== test::display-objects-newline <procedure>(test::display-objects-newline [OBJECT ...])</procedure> ===== test::display-indent <procedure>(test::display-indent INDENT)</procedure> ===== test::display-indented-objects <procedure>(test::display-indented-objects INDENT [OBJECT ...])</procedure> ===== test::display-indented-objects-newline <procedure>(test::display-indented-objects-newline INDENT [OBJECT ...])</procedure> {{INDENT}} is the number of spaces to the left. ===== test::display-underlined <procedure>(test::display-underlined [OBJECT ...])</procedure> Prints each supplied {{OBJECT}} to the {{(current-output-port)}}, with a {{#\space}} interspersed, on a single line. Then prints a line of underscores, {{#\_}}. ===== test::display-structure <procedure>(test::display-structure TEST-STRUCTURE)</procedure> Prints the supplied {{TEST-STRUCTURE}} tree to the {{(current-output-port)}} with indentation. ==== Test Report Generation These functions will display a rendering of the {{RESULT-TREE}} and returns the {{RESULT-TREE}}. ===== test::output-style-compact <procedure>(test::output-style-compact RESULT-TREE [PORT|FILE-NAME|#t]) Prints a simple listing of the test results. Details are only printed for failed tests. ===== test::output-style-human <procedure>(test::output-style-human RESULT-TREE [PORT|FILE-NAME|#t])</procedure> Prints a verbose listing of the test results. ===== test::output-style-html <procedure>(test::output-style-html RESULT-TREE [PORT|FILE-NAME|#t])</procedure> Prints a verbose listing of the test results as HTML. ===== test::output-style-minimal <procedure>(test::output-style-minimal RESULT-TREE [PORT|FILE-NAME|#t])</procedure> Prints a summary of the test result statistics. The output style procedures are ''not'' loaded automatically. They must be loaded explicitly, as in: <enscript highlight=scheme> (require-extension testbase-output-human) (require-extension testbase-output-html) (require-extension testbase-output-compact) (require-extension testbase-output-minimal) </enscript> This procedure is useful with a test report procedure. The actual report procedures of the above report generators are examples. ===== test::with-output-style <procedure>(test::with-output-style STYLER RESULT-TREE [PORT|FILE-NAME|#t])</procedure> {{STYLER}} is a procedure with a single argument, a {{RESULT-TREE}}, to print a report. The default destination is the {{(current-output-port)}}. Otherwise the destination specifies how the output is targeted. A {{PORT}} destination will print to the port. A {{FILE-NAME}} destination will print to the specified file. Should the file exist the result is implementation dependent. A {{#t}} destination will return the output as a string. The {{STYLER}} will be called with {{RESULT-TREE}} as the only actual argument. ==== Notes & Caveats To generate the tree structure of a test procedure the test body is evaluated using a special test mode, {{report}}. The expectation expressions, test container {{let}} bindings, and {{test-letrec}} bindings <em>are not</em> evaluated. This means there should not be any expressions in the test body that will be evaluated, such as the construction of a {{TEST-NAME}}, that depend on these bindings. However, test procedure {{initial}} and {{final}} forms ''are'' evaluated. Be careful when using test selection that dependencies between tests are not broken. For example, an expectation expression could depend on the result of an {{expect-set!}}, but the associated variable may never be bound because the {{expect-set!}} was skipped. ==== Examples Assume the following are the contents of the file "testbase-example.scm": <enscript highlight=scheme> (require-extension testbase testbase-output-human) (define-expect-binary =) (define-test example "Arithmetic Operators" (test/case "Testing '+'" (expect-= "Adding two positive numbers" 2 (+ 1 1)) (expect-= "Adding two negative numbers" -2 (+ -1 -1)) (expect-zero "Adding positive and negative" (+ -1 1)) ) (test/case "Testing '-'" (expect-zero "Subtracting two positive numbers" (- 1 1)) (expect-zero "Subtracting two negative numbers" (- -1 -1)) (expect-= "Subtracting positive and negative" -2 (- -1 1)) ) ) (test::styler-set! example test::output-style-human) (run-test "TestBase Example") </enscript> Then the following entered from a command line interface: csi -n -s testbase-example.scm Produces the following output: ** Running TestBase Example ** Test Suite: Arithmetic Operators Test Case: Testing '+' Expectation: Adding two positive numbers Expect = Expected: 2 Unevaluated: (+ 1 1) Evaluated: 2 Pass: Adding two positive numbers Expectation: Adding two negative numbers Expect = Expected: -2 Unevaluated: (+ -1 -1) Evaluated: -2 Pass: Adding two negative numbers Expectation: Adding positive and negative Expect zero Unevaluated: (+ -1 1) Evaluated: 0 Pass: Adding positive and negative Pass: Testing '+' Test Case: Testing '-' Expectation: Subtracting two positive numbers Expect zero Unevaluated: (- 1 1) Evaluated: 0 Pass: Subtracting two positive numbers Expectation: Subtracting two negative numbers Expect zero Unevaluated: (- -1 -1) Evaluated: 0 Pass: Subtracting two negative numbers Expectation: Subtracting positive and negative Expect = Expected: -2 Unevaluated: (- -1 1) Evaluated: -2 Pass: Subtracting positive and negative Pass: Testing '-' Pass: Arithmetic Operators ALL TESTS SUCCESSFUL! And the following entered from a command line interface: echo "(test::resource-limit-set! example 'cpu 0) (example)" | csi -n -s testbase-example.scm -- -i Produces the following output: (begin (id tr0) (name "Arithmetic Operators") (result #f) (kind "suite")) (begin (id tr1) (name "Testing '+'") (result #f) (kind "case")) (expectation (id tr2) (name "Adding two positive numbers") (result #t) (kind "=") (unevaluated (+ 1 1)) (expected 2) (actual 2)) (expectation (id tr3) (name "Adding two negative numbers") (result #t) (kind "=") (unevaluated (+ -1 -1)) (expected -2) (actual -2)) (expectation (id tr4) (name "Adding positive and negative") (result #t) (kind "zero") (unevaluated (+ -1 1)) (actual 0)) (end (id tr1) (name "Testing '+'") (result #t) (kind "case")) (begin (id tr5) (name "Testing '-'") (result #f) (kind "case")) (expectation (id tr6) (name "Subtracting two positive numbers") (result #t) (kind "zero") (unevaluated (- 1 1)) (actual 0)) (error (id tr7) (message "Resource Limit Exceeded: Processor-Time") (result #f) (kind "terminate") (container test-suite) (location "Arithmetic Operators")) (end (id tr0) (name "Arithmetic Operators") (result #f) (kind "suite")) And the following entered from a command line interface: echo "(test::run)" | csi -n -s testbase-example.scm -- -i Produces the following output: (begin (id tr0) (name "Arithmetic Operators") (result #f) (kind "suite")) (begin (id tr1) (name "Testing '+'") (result #f) (kind "case")) (expectation (id tr2) (name "Adding two positive numbers") (result #t) (kind "=") (unevaluated (+ 1 1)) (expected 2) (actual 2)) (expectation (id tr3) (name "Adding two negative numbers") (result #t) (kind "=") (unevaluated (+ -1 -1)) (expected -2) (actual -2)) (expectation (id tr4) (name "Adding positive and negative") (result #t) (kind "zero") (unevaluated (+ -1 1)) (actual 0)) (end (id tr1) (name "Testing '+'") (result #t) (kind "case")) (begin (id tr5) (name "Testing '-'") (result #f) (kind "case")) (expectation (id tr6) (name "Subtracting two positive numbers") (result #t) (kind "zero") (unevaluated (- 1 1)) (actual 0)) (expectation (id tr7) (name "Subtracting two negative numbers") (result #t) (kind "zero") (unevaluated (- -1 -1)) (actual 0)) (expectation (id tr8) (name "Subtracting positive and negative") (result #t) (kind "=") (unevaluated (- -1 1)) (expected -2) (actual -2)) (end (id tr5) (name "Testing '-'") (result #t) (kind "case")) (end (id tr0) (name "Arithmetic Operators") (result #t) (kind "suite")) A {{test-collect}} Example: <enscript highlight=scheme> (require-extension testbase testbase-output-human) (define-test tcx "A test-collect example" (test/collect "Are 0 - 9 integers?" (do ([i 0 (add1 i)]) [(= i 10)] (collect-test (expect-true (integer? i))) ) ) ) (test::styler-set! tcx test::output-style-human) (run-test "TestBase Example") </enscript> Produces the following output: ** Running TestBase Example ** Test Suite: A test-collect example Test Case: Are 0 - 9 integers? Expectation: Expect true Unevaluated: (integer? i) Evaluated: #t Passed: Expectation: Expect true Unevaluated: (integer? i) Evaluated: #t Passed: Expectation: Expect true Unevaluated: (integer? i) Evaluated: #t Passed: Expectation: Expect true Unevaluated: (integer? i) Evaluated: #t Passed: Expectation: Expect true Unevaluated: (integer? i) Evaluated: #t Passed: Expectation: Expect true Unevaluated: (integer? i) Evaluated: #t Passed: Expectation: Expect true Unevaluated: (integer? i) Evaluated: #t Passed: Expectation: Expect true Unevaluated: (integer? i) Evaluated: #t Passed: Expectation: Expect true Unevaluated: (integer? i) Evaluated: #t Passed: Expectation: Expect true Unevaluated: (integer? i) Evaluated: #t Passed: Passed: Are 0 - 9 integers? Passed: A test-collect example ALL TESTS SUCCESSFUL! ** Passed TestBase Example ** ==== Tips, Tricks, And Idioms This documentation is pretty large, so put here any things that weren't obvious to you that seem like they might be useful to others. ===== General Tips * test/case and friends can take a let-style list of local variables, right after the test description, i.e. (test/case "foo" ([var #t]) ...) * Note the existence of expect-set! * It is possible to map over all test procedures, at least up to that point in the file, so you can, for example, set all tests to the "human" report format like so: (test::for-each (cut test::styler-set! <> test::output-style-human)) * chicken-testbase-driver (see [[testbase-driver]]) has --test-take and --test-skip options, for narrowing down your tests as you're in the middle of particular bits of code ===== Automated Egg Tests chicken-setup, at least when run with the -t flag, will treat the "tests" directory as a directory full of tests, and run the "run.scm" file in that directory. Here's a run.scm file, modified from one provided by Kon, that seems to work well. You'll need testbase-driver for it to work. If any tests fail, the setup will abort (although by that time it's already installed the egg, but at least it'll be quite cranky about the failure). This run.scm will run everything in the tests directory named "*/test.scm". It will put the results in "results/" under the tests directory. <enscript highlight=scheme> ;;;; run-testbase.scm (run.scm) ; ; Thanks to Kon Lovett for the code. (use utils posix) ;; (define *egg-name* (car (command-line-arguments))) (define *verbose* #t) (define-constant TEST-DRIVER "chicken-testbase-driver") (define-constant TEST-DRIVER-OPTIONS "--indent 2 --test-interpret --failure-exit=1") (define *test-driver-arguments* (conc TEST-DRIVER-OPTIONS " --results-repository=" (current-directory) "/results " (if *verbose* " -v" ""))) ;; (system* "~A ~A ~A" TEST-DRIVER *test-driver-arguments* (string-append (current-directory) "/*-test.scm")) </enscript> ==== Migration from test-infrastructure Changes: * {{test-group}} removed. * {{test-apply}} removed. * {{test-it}} -> {{test}}. * {{test-package}} -> {{test-suite}}. * {{expect}} -> {{expect-unary}}. * {{expect-values}} -> {{expect-unary/values}}. * {{expect-equiv}} -> {{expect-binary}}. * {{expect-equiv-values}} -> {{expect-binary/values}}. * {{expect-values-foo}} -> {{expect-foo/values}}. * {{test:foo}} -> {{test::foo}}. * {{foo-ref}} -> {{foo}}. * {{*-foo}} -> {{test::result-foo}}. * Test containers cannot be used outside of a {{define-test}}! Forms like: (test::apply test::output-style-compact (skip ...) test-proc) become (test::take! test-proc '(skip ...)) (test::styler-set! test-proc test::output-style-compact) (test-proc) Forms like (define (test-something) (test-suite "Something" pd pe ... test elements ... ) ) become (define-test test-something (test-suite "Something" pd pe ... test elements ... ) ) or even (define-test test-something "Something" ... test elements ... <where any use of 'pd' and 'pe' becomes 'test-dtor' and 'test-escr'> ) Forms like (let ((result (test-suite "Something" pd pe ... test elements ...))) ...) become (define-test something (test-suite "Something" pd pe ... test elements ... ) ) (let ((result (something))) ...) ==== Bugs & Limitations Because a sharp-syntax reader is created to handle "unprintable" results the {{"#<<TAG"}} and {{"#<#TAG"}} multiline string constant read syntax cannot be used within a test-file. Use {{(MULTILINE-STRING STRING ...)}}. === Contributions {{output-text-compact}} by [[Patrick Brannan]]. === Changelog * 1.5.2 Added early-exit-upon-failure-option * 1.5.1 Fix for non-false expect predicate result * 1.5 Added {{insist}} expectation form * 1.41 Cygwin has resource limiting * 1.4 Expansion options are {{#f}} by default, expcept {{test::catching!}} * 1.304 Bug fix for {{test::forget!}} * 1.303 Removed macro expansion time atom check for {{TEST-NAME}} * 1.302 Added {{test-let}} & {{test-let*}} * 1.301 Dropped {{:optional}} * 1.3 Added {{define-expect-nary}} and {{test-collect}} * 1.2 Bugfix for {{define-expect-*/values}} * 1.1 Added echo filter options * 1.0 Initial release === License Portions Copyright (c) 2007, Kon Lovett Portions Copyright (c) 2000-2005, Peter Keller - All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. Neither the name of the author nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Description of your changes:

I would like to authenticate

Spam control

What do you get when you multiply 3 by 1?