Cangjie Testing Infrastructure

Cangjie standard library offers state-of-the-art testing experience for its users, allowing for both conventional and simple testing techniques and more advanced techniques for more advanced testing scenarios. Cangjie testing infrastructure includes three main components: the unit test framework, the mocking framework and the benchmarking framework.

Cangjie Unit Test Framework

Cangjie unit test framework, as the name suggests, allows users to create unit tests in their Cangjie projects. In addition to simple unit tests that are as simple to write as a single Cangjie function, it provides a variety of more advanced techniques:

• Parameterized test: used to run the test code on multiple inputs.
• Data-driven test: used to read multiple groups of test data from a file and use the data as inputs to test the code.
• Randomized property-based test: used to test the code on structurally-constructed random data. Compared with other programming languages in the industry that can generate only random values of the basic type and character array type for tests, Cangjie can generate various types of random values for randomized property-based test.
• Generic type–parameterized test: Generic library developers can, based on the same test code, test the implementation of a generic function on different data types by passing different types of parameters.
• Death test: Underlying library developers can use the death testing capability to capture unwanted signals, segmentation faults, and other failures that may happen in underlying libraries.

To improve user experience even more, the unit test framework introduces built-in support for power assertions and difference assertions, providing exhaustive insight into the testing data and reasoning behind failures.

All these features are highly configurable and may be used together. For example, when testing a generic function, create randomly-generated generic data types, test the algorithm implementation on the data, and use power assertions and difference assertions to obtain detailed information.

Cangjie Mocking Framework

Cangjie mocking framework allows users to change the behaviour of Cangjie classes used in tests using so-called mocks and spies: objects that can capture and modify the behaviour of particular objects to test how the rest of the program interacts with these objects. Mocking is an advanced technique mainly used for testing bigger applications that are formed from a big number of interacting components.

Users of mocking framework from other languages may find our mocking DSL very similar to what they are used in their previous experience. The DSL allows you to specify, verify and modify the behavior of objects passed to the test code and generate readable error messages. Unlike other mocking frameworks, Cangjie mocking framework is implemented based on a unique instrumentation-based compiler technique, allowing users to mock not only interfaces and open types, but final classes as well.

Cangjie mocking framework is fully integrated with the unit test framework, and any of the features of either can be used together to make the testing capability even more powerful.

Cangjie Benchmarking Framework

Cangjie testing infrastructure provides a state-of-the-art benchmarking experience, including linear regression–based statistical value calculation, warm-up, and precise measurement. Most of the features provided by the unit test framework are also available to the benchmarking framework, allowing the use of benchmark with parameters, random value generation, and generic types.

In addition, the benchmarking framework comes with its own set of features, such as calculations based on a given baseline, getting access to the raw benchmarking data (to make users' own calculations if needed), and accurate error estimation both for micro-benchmarking and macro-benchmarking scenarios.