Guide to Stubs

Mock objects, spy objects, and stubs can be used in various manners. This document introduces different modes and cases so that you can compile maintainable and simple test cases for the mock framework.

Operating Principles of Stubs

Stubs are declared by calling the @On macro within a test case. The declaration is valid before the execution of a specified test case is completed. Test cases can share a stub.

The mock framework processes the call of a mock or spy object member (or a static member or a top-level function or top-level variable) in the following sequence:

• Search for stubs of a specific declaration. The stubs declared later take precedence over the stubs declared earlier. The stubs declared inside the test case take precedence over the shared stubs.
• Apply the parameter matchers of each stub. If all parameters are successfully matched, the operation defined by the stub is performed.
• If no stub is found or no stub matches actual parameters, the default behavior applies.
  • For a mock object, an error of no stub call is reported.
  • For a spy object, the original member of the monitored instance is called.
  • For a static member, top-level function, or top-level variable, the original declaration is called.

Regardless whether multiple stubs are defined for a single member, each stub has its own expectations. A test can be passed when these expectations are satisfied.

@On(foo.bar(1)).returns(1)
@On(foo.bar(2)).returns(2)

foo.bar(2)
//The first stub has been defined but is never used. The test fails.

Redefining a Stub

To change the behavior of a stub in a test, you can redefine the stub.

@On(service.request()).returns(testData)
//Use a service.

@On(service.request()).throws(Exception())
//Test the consequence of a service failure.

Defining Multiple Stubs Using One Declaration

Multiple stubs can define different behavior by using different parameters.

Examples:

@On(storage.get(_)).returns(None)                  // 1
@On(storage.get(TEST_ID)).returns(Some(TEST_DATA)) // 2

In this example, storage returns None for all parameters except TEST_ID. If get is never called using the parameter TEST_ID, the test fails because stub 2 is not used. If get is called always using the parameter TEST_ID, the test fails because stub 1 is not used. These restrictions help ensure pure test code, and developers learn about when a stub becomes unused. If a use case does not require this function, the cardinality specifier anyTimes() is used to improve these expectations.

//Implementation is changed frequently, but the test is expected to proceed.
//anyTimes is used to improve the expectations irrelevant to the test.
@On(storage.get(_)).returns(None).anyTimes()
@On(storage.get(TEST_ID)).returns(Some(TEST_DATA)) // It is a must to call the content under test.

Given that the stub priority becomes higher from the bottom up, the following uses are all incorrect.

@On(storage.get(TEST_ID)).returns(Some(TEST_DATA)) // Incorrect. This stub will never be triggered.
@On(storage.get(_)).returns(None)                  // A higher stub is always hidden.

Expectations can also be used to check the called parameters.

let renderer = spy(Renderer())

@On(renderer.render(_)).fails()
let isVisible = { c: Component => c.isVisible }
@On(renderer.render(argThat(isVisible))).callsOriginal() // Only visible components are permitted.

Sharing Mock Objects and Stubs

When a large number of mock objects are required for testing, multiple test cases can share mock objects and/or stubs. A mock or spy object can be created anywhere. However, if a mock object is leaked from one test case to another by mistake, sequential dependency problems or unstable tests may be caused. Therefore, such operation is not recommended, and the mock framework also detects this type of problems. To share mock or spy objects between test cases of one test class, the objects can be placed in instance variables of the class.

A stub declaration implies expectations, making it more difficult to process shared stubs. Expectations cannot be shared between test cases. A stub can be declared in only two positions:

• principal part of test case (regardless of the @Test function or @TestCase in the @Test class): check expectations; and
• beforeAll in the @Test class: share stubs between test cases. Such a stub cannot declare an expectation, and the expectation is not checked. Cardinality specifiers are not allowed. Only stateless operations such as returns(value), throws(exception), fails(), and callsOriginal() are allowed. These stubs can be considered to have implicit anyTimes() cardinality.

If test cases have the same expectations, you can extract and call functions (non-test case member functions in the test class) in the principal part of the test cases.

Using beforeAll:

@Test
class TestFoo {
    let foo = mock<Foo>()

    //The unit test framework calls the content below before executing test cases.
    public func beforeAll(): Unit {
        //Default behavior is shared among all test cases.
        //This stub does not need to be used in each test case.
        @On(foo.bar(_)).returns("default")
    }

    @TestCase
    func testZero() {
        @On(foo.bar(0)).returns("zero") //This stub needs to be used in this test case.
        foo.bar(0) //"zero" is returned.
        foo.bar(1) //"default" is returned.
    }

    @TestCase
    func testOne() {
        @On(foo.bar(0)).returns("one")
        foo.bar(0) //"one" is returned.
    }
}

Using the function:

@Test
class TestFoo {
    let foo = mock<Foo>()

    func setupDefaultStubs() {
        @On(foo.bar(_)).returns("default")
    }

    @TestCase
    func testZero() {
        setupDefaultStubs()
        @On(foo.bar(0)).returns("zero")

        foo.bar(0) //"zero" is returned.
        foo.bar(1) //"default" is returned.
    }

    @TestCase
    func testOne() {
        setupDefaultStubs()
        @On(foo.bar(0)).returns("zero")
        foo.bar(0) //"zero" is returned.

        //Expectation fails. The stub has declared the expectation but never uses it.
    }
}

Do not declare stubs in constructors of the test class. When to call a constructor of the test class cannot be guaranteed.

Capturing Parameters

The mock framework uses the captor(ValueListener) parameter matcher to capture parameters for checking actual parameters passed to a stub declaration.Once a stub is triggered, ValueListener intercepts corresponding parameters, and checks parameters and/or adds verification parameters.

On each call, the ValueListener.onEach static function can also be used to verify a condition. After lambda is accepted, lambda is called when the stub is triggered. lambda is used to receive the value of a parameter.

let renderer = spy(TextRenderer())
let markUpRenderer = MarkupRenderer(renderer)

// Create a validator.
let validator = ValueListener.onEach { str: String =>
    @Assert(str == "must be bold")
}

// Use the "capture" parameter matcher to bind parameters to the validator.
@On(renderer.renderBold(capture(validator))).callsOriginal() // If this function has never been called, the test fails.

markUpRenderer.render("text inside tag <b>must be bold</b>")

In addition, ValueListener also provides allValues() and lastValue() functions for checking parameters in the following mode:

//Create a capturer.
let captor = ValueListener<String>.new()

//Use the "capture" parameter matcher to bind parameters to the validator.
@On(renderer.renderBold(capture(captor))).callsOriginal()

markUpRenderer.render("text inside tag <b>must be bold</b>")

let argumentValues = captor.allValues()
@Assert(argumentValues.size == 1 && argumentValues[0] == "must be bold")

The argThat matcher is an overloaded function that combines parameter filtering and capturing. argThat(listener, filter) accepts ValueListener instances and filter predicates. listener collects only the parameters that have passed the filter check.

let filter = { arg: String => arg.contains("bold") }
let captor = ValueListener<String>.new()

// Failed unless parameters are intercepted. However, the stub is declared below.
@On(renderer.renderBold(_)).fails()
// Collect only the strings containing "bold".
@On(renderer.renderBold(argThat(captor, filter))).callsOriginal()

markUpRenderer.render("text inside tag <b>must be bold</b>")

// The "captor" object can be used to check all filtered parameters.
@Assert(captor.lastValue() == "must be bold")

The parameter capturer can be used with mock and spy objects. However, such parameter matchers are not allowed in the @Called macro.

Customizing and Using a Parameter Matcher

To prevent reuse of a parameter matcher, you can customize parameter matchers.

The following example shows how to share a matcher between test cases:

@On(foo.bar(oddNumbers())).returns("Odd")
@On(foo.bar(evenNumbers())).returns("Even")
foo.bar(0) // "Even"
foo.bar(1) // "Odd"

Since each matcher is just a static function of the Matchers class, extensions can be used to customize parameter matchers. New parameter matchers need to call existing instances.

extend Matchers {
    static func evenNumbers(): TypedMatcher<Int> {
        argThat { arg: Int => arg % 2 == 0}
    }

    static func oddNumbers(): TypedMatcher<Int> {
        argThat { arg: Int => arg % 2 == 1}
    }
}

A function parameter matcher may contain parameters.

extend Matchers {
    //Accept Int parameters only.
    static func isDivisibleBy(n: Int): TypedMatcher<Int> {
        argThat { arg: Int => arg % n == 0}
    }
}

Most matcher functions specify the return type TypedMatcher<T>. Such a matcher accepts only the T type. When a parameter matcher is used in a stub declaration, values of the T type must be valid parameters for a stubbed function or attribute setter. In other words, the T type should be a parameter subtype or the same as the actual type of parameters.

Setting Attributes, Fields, and Top-Level Variables

The method of stubbing fields, attributes, and top-level variables is the same as that of stubbing functions. The return value can be configured based on the same operation.

A setter is similar to a function that returns Unit. The special operation doesNothing() can be used for setters.

The common mode of stubbing variable attributes is as follows:

@On(foo.prop).returns("value")  //Configure a getter.
@On(foo.prop = _).doesNothing() //Ignore the call of a setter.

In rare cases, the behavior of a variable attribute is expected to be the same as that of a field. To create a synthetic field (a field generated by the framework), use the static function SyntheticField.create. The storage of synthetic fields is managed by the mock framework. It is applicable to the scenario of mocking interfaces or abstract classes that contain variable attributes and fields.

Perform the getsField and setsField stub operations to bind the field or top-level variable to a specific call, during which expectation configuration can be any other operation.

interface Foo {
    mut prop bar: String
}

@Test
func test() {
    let foo = mock<Foo>()
    let syntheticField = SyntheticField.create(initialValue: "initial")
    @On(foo.bar).getsField(syntheticField)     // Reading attributes is to read a synthetic field.
    @On(foo.bar = _).setsField(syntheticField) // Write a new value for the attribute.

    //The 'bar' attribute is represented as a field.
}

NOTE

If multiple test cases share the SyntheticField object, the value of this field is reset to initialValue before each test case to prevent sharing a variable state between tests.

Stub Mode

Usually, an exception is thrown when some calls do not match any stub. However, in some common situations, the mock object can be configured to add default behavior. In this way, when no stub is matched, the default behavior is executed. This is implemented by enabling the stub mode. Two modes are available: ReturnsDefaults and SyntheticFields. These modes are represented by the enumeration type StubMode. During creation of a mock object, these modes can be passed to the mock function so that the stub mode is enabled for specified mock objects.

public func mock<T>(modes: Array<StubMode>): T

NOTE

No expectations are imposed on the mock object members by using the stub mode. When the case is used to check whether only certain members of the mock object are called, the stub mode should be used with caution. The behavior of the tested object may change in an unexpected manner, but the test may still be passed. The current stub mode does not support the stub of static members and top-level functions or variables.

ReturnsDefaults Mode

In this mode, if the return type of a member is listed in the following table, the member can be called without explicit stub configuration.

let foo = mock<Foo>(ReturnsDefaults)
@Assert(foo.isPretty(), false)

The default values returned by this type of members are also shown in the following table.

The ReturnsDefaults mode takes effect only for the following members:

• Member functions whose return values are of a supported type (as shown in the table above)
• Attribute readers and fields of a supported type (as shown in the table above)

SyntheticFields Mode

The SyntheticFields mode can simplify the configuration actions of SyntheticField. For details, see Setting Attributes, Fields, and Top-Level Variables.SyntheticFields implicitly creates synthetic fields of the corresponding type for all attributes and fields by using the mock framework. However, these fields can be read only after being assigned values. This mode is effective only for variable attributes and fields.

let foo = mock<Foo>(SyntheticFields)
// can simply assign a value to a mutable property
foo.bar = "Hello"
@Assert(foo.bar, "Hello")

Values assigned to attributes and fields are visible only in the corresponding test cases. When both SyntheticFields and ReturnsDefaultsare enabled, the assigned value precedes the default value. However, the default value can be used as long as the field or attribute is not assigned a value.

let foo = mock<Foo>(ReturnsDefaults, SyntheticFields)
@Assert(foo.bar, "")
foo.bar = "Hello"
@Assert(foo.bar, "Hello")