Dynamic Test

Getting Started with Dynamic Test

The Cangjie testing framework supports dynamic tests. It allows constructing test cases when test data is unknown at the compilation time. Key use cases include:

• Creating test suites based on external data.
• Creating test suites based on parameters or configuration files.

By comparing with ordinary test cases, you can see how dynamic test cases are constructed using @TestBuilder.

The following is a simple test suite that is constructed using @Test/@TestCase:

@Test
class A {
    @TestCase
    func f() { @Assert(false) }

    @TestCase[x in [1, 2]]
    func g(x: Int) {
        @Assert( x >= 1 )
    }
}

Using @TestBuilder, you can create a dynamic test suite with the same logic as the preceding test suite.

@TestBuilder
public func buildCustomTestSuite(): TestSuite {
   let suiteBuilder = TestSuite.builder("A")
   let caseConfiguration = Configuration()
   suiteBuilder.add(
       UnitTestCase.create("f", configuration: caseConfiguration) { @Assert(false) })
   suiteBuilder.add(
       UnitTestCase.createParameterized("g", [1, 2]) { value => @Assert( value >= 1 ) })
   suiteBuilder.build()
}

TestSuite creates a TestSuiteBuilder object, which supports adding test cases. Test cases are constructed using static functions in the UnitTestCase class. This class supports constructing simple test cases or parameterized test cases. Once the TestSuiteBuilder is fully configured, it finally generates a TestSuite object, which is the return value of the function marked with @TestBuilder.

After the preceding code is compiled using --test and executed, the output matches that of the test suite constructed with @Test/@TestCase.

--------------------------------------------------------------------------------------------------
TP: default, time elapsed: 121592 ns, RESULT:
    TCS: A, time elapsed: 121592 ns, RESULT:
    [ PASSED ] CASE: g (13969 ns)
    [ FAILED ] CASE: f (91641 ns)
    Assert Failed: `(false == true)`
       left: false
      right: true

Summary: TOTAL: 2
    PASSED: 1, SKIPPED: 0, ERROR: 0
    FAILED: 1, listed below:
            TCS: A, CASE: f
--------------------------------------------------------------------------------------------------

@TestBuilder is subject to the following constraints:

• It can only be applied to top-level functions that are not foreign.
• The return type must be explicitly specified and must be of the TestSuite type.
• It can be combined with the @Configure, @Timeout, and @Parallel macros but cannot be combined with other macros from the unittest.testmacro package.

Running Dynamic Test and Obtaining Output

If you need to run test cases by running the program without using the unit test framework, you can use the TestSuite and runTests() functions.

Examples:

import std.unittest.*

main() {
   let suiteBuilder = TestSuite.builder("A")
   let caseConfiguration = Configuration()
   suiteBuilder.add(
       UnitTestCase.create("f", configuration: caseConfiguration) { @Assert(false) })
   suiteBuilder.add(
       UnitTestCase.createParameterized("g", [1, 2]) { value => @Assert( value >= 1 ) })
   let suite = suiteBuilder.build()
   let report = suite.runTests()
}

The runTests function returns an instance of the Report class. You can use the Report object to perform the following operations:

• Use the ConsoleReporter class to print to the console: report.reportTo(ConsoleReporter(colored: true)).
• Use the TextReporter class to print to any PrettyPrinter implementation: report.reportTo(TextReporter(into: PrettyText())).
• Output the result in one of the supported formats, such as XML (XmlReporter, used only for unit tests, not for performance tests) or CSV (CsvReporter or CsvRawReporter, used only for performance tests).