Compiling, Error Reporting, and Debugging

Compiling and Using Macros

The current compiler restricts that the definition and invocation of a macro cannot be in the same package. The macro package must be compiled before the package calling the macro. The macro definition is not allowed in the package calling the macro. Since macros need to be exported from a package for use by another package, the compiler mandates that macro definitions must be declared with the public modifier.

The following is a simple example.

The structure of the source code directory is as follows.

// Directory layout.
root_path
├── macros
│    └── m.cj
├── src
│    └── demo.cj
└─ target

The macro definition is stored in the macros subdirectory.

// macros/m.cj
// In this file, we define the macro Inner, Outer.
macro package define
import std.ast.*

public macro Inner(input: Tokens) {
    return input
}

public macro Outer(input: Tokens) {
    return input
}

The macro call is in the src subdirectory.

// src/demo.cj
import define.*
@Outer
class Demo {
    @Inner var state = 1
    @Inner var cnt = 42
}

main() {
    println("test macro")
    0
}

If the compilation product of the macro definition file and the file using the macro are not located in the same directory, you need to add the compilation option --import-path to specify a path for the compilation product of the macro definition file. The following are the compilation commands for the Linux platform. (The specific compilation options will evolve with the cjc update. Use the latest cjc compilation options.)

# Compile the macro definition file to generate the default dynamic library file in the specified directory. (You can specify a path for the dynamic library, but not the name of the library.)
cjc macros/m.cj --compile-macro --output-dir ./target

# Compile a file that uses the macro. After the macro is replaced, an executable file is generated.
cjc src/demo.cj -o demo --import-path ./target --output-dir ./target

# Run the executable file.
./target/demo

On the Linux platform, the macro_define.cjo and dynamic library files are generated for package management.

The following are the compilation commands for the Windows platform.

# Current directory: src

# Compile the macro definition file to generate the default dynamic library file in the specified directory. (You can specify a path for the dynamic library, but not the name of the library.)
cjc macros/m.cj --compile-macro --output-dir ./target

# Compile a file that uses the macro. After the macro is replaced, an executable file is generated.
cjc src/demo.cj -o demo.exe --import-path ./target --output-dir ./target

If the macro package depends on other dynamic libraries, ensure that the macro package can find these dependencies at runtime. (Macro expansion depends on the execution of methods in the macro package.) On Linux, you can set the environment variable LD_LIBRAYR_PATH (PATH on Windows) to add the path to the dependent dynamic library.

NOTE

Macro invocation depends on Cangjie Runtime. During the invocation, the default Cangjie Runtime configuration is used. The configuration parameters can be queried through Cangjie Runtime O&M logs. Only cjHeapSize and cjStackSize can be modified by users. For details about Cangjie Runtime configuration initialization, see Runtime Initialization Configuration Options.

Parallel Macro Expansion

You can add the --parallel-macro-expansion option when compiling a file with macro calls to enable the parallel macro expansion capability. The compiler automatically analyzes the dependency between macro calls. Macro calls that do not depend on each other can be executed in parallel. For example, two @Inner calls can be executed in parallel to shorten the overall compilation time.

NOTE

If a macro function depends on some global variables, it is risky to use parallel macro expansion.

macro package define
import std.ast.*
import std.collection.*

var Counts = HashMap<String, Int64>()

public macro Inner(input: Tokens) {
    for (t in input) {
        if (t.value.size == 0) {
            continue
        }
        // Count the number of occurrences of all valid token values.
        if (!Counts.contains(t.value)) {
            Counts[t.value] = 0
        }
        Counts[t.value] = Counts[t.value] + 1
    }
    return input
}

public macro B(input: Tokens) {
    return input
}

According to the preceding code, if the @Inner macro is called in multiple places and the parallel macro expansion option is enabled, a conflict may occur when the global variable Counts is accessed. As a result, the obtained result is incorrect.

You are advised not to use global variables in macro functions. If you cannot avoid using them, either disable parallel macro expansion or protect global variables using Cangjie thread locks.

diagReport Error Reporting Mechanism

The std.ast package of the Cangjie standard library provides the user-defined error reporting API, diagReport.Users who define macros can customize the error messages for incorrect Tokens content when passed tokens are parsed.

The user-defined error reporting API provides the same output format as the native compiler error reporting API and allows users to report warning and error messages.

The prototype of the diagReport function is as follows.

public func diagReport(level: DiagReportLevel, tokens: Tokens, message: String, hint: String): Unit

The parameters are described as follows.

• level: severity level of the error message.
• tokens: Tokens from the source code that are referenced in the error message.
• message: main error message.
• hint: auxiliary information.

For details, see the following example.

Macro definition file:

// macro_definition.cj
macro package macro_definition

import std.ast.*

public macro testDef(input: Tokens): Tokens {
    for (i in 0..input.size) {
        if (input[i].kind == IDENTIFIER) {
            diagReport(DiagReportLevel.ERROR, input[i..(i + 1)],
                       "This expression is not allowed to contain identifier",
                       "Here is the illegal identifier")
        }
    }
    return input
}

Macro call file:

// macro_call.cj
package macro_calling

import std.ast.*
import macro_definition.*

main(): Int64 {
    let a = @testDef(1)
    let b = @testDef(a)
    let c = @testDef(1 + a)
    return 0
}

The following error information is displayed during the compilation of the macro call file:

error: This expression is not allowed to contain identifier
 ==> call.cj:9:22:
  |
9 |     let b = @testDef(a)
  |                      ^ Here is the illegal identifier
  |

error: This expression is not allowed to contain identifier
  ==> call.cj:10:26:
   |
10 |     let c = @testDef(1 + a)
   |                          ^ Here is the illegal identifier
   |

2 errors generated, 2 errors printed.

Using --debug-macro to Output the Macro Expansion Result

When macros are used to generate code during compilation, debugging errors can be particularly difficult. This is a common problem for developers, as the source code written by the developer is transformed into different code snippets after macro expansion. The error messages thrown by the compiler are based on the final generated code, but this code is not directly visible in the developer's source code.

To solve this problem, Cangjie macros offer a debug mode. In this mode, developers can view the complete expanded macro code in the debug file generated by the compiler. An example is provided below:

Macro definition file:

macro package define

import std.ast.*

public macro Outer(input: Tokens): Tokens {
    let messages = getChildMessages("Inner")

    let getTotalFunc = quote(public func getCnt() {
                       )
    for (m in messages) {
        let identName = m.getString("identifierName")
        getTotalFunc.append(Token(TokenKind.IDENTIFIER, identName))
        getTotalFunc.append(quote(+))
    }
    getTotalFunc.append(quote(0))
    getTotalFunc.append(quote(}))
    let funcDecl = parseDecl(getTotalFunc)

    let decl = (parseDecl(input) as ClassDecl).getOrThrow()
    decl.body.decls.add(funcDecl)
    return decl.toTokens()

}

public macro Inner(input: Tokens): Tokens {
    assertParentContext("Outer")
    let decl = parseDecl(input)
    setItem("identifierName", decl.identifier.value)
    return input
}

Macro call file demo.cj:

import define.*

@Outer
class Demo {
    @Inner var state = 1
    @Inner var cnt = 42
}

main(): Int64 {
    let d = Demo()
    println("${d.getCnt()}")
    return 0
}

When compiling the file that uses the macro, add --debug-macro to the option, that is, use the debug mode of the Cangjie macro.

cjc --debug-macro demo.cj --import-path ./target

NOTE

If you compile with the Cangjie package management tool, CJPM, add the compilation option --debug-macro to the configuration file cjpm.toml to use the debug mode of the macro.

compile-option = "--debug-macro"

In debug mode, a temporary file demo.cj.macrocall is generated. The corresponding macro expansion is as follows.

// demo.cj.macrocall
/* ===== Emitted by MacroCall @Outer in demo.cj:3:1 ===== */
/* 3.1 */class Demo {
/* 3.2 */    var state = 1
/* 3.3 */    var cnt = 42
/* 3.4 */    public func getCnt() {
/* 3.5 */        state + cnt + 0
/* 3.6 */    }
/* 3.7 */}
/* 3.8 */
/* ===== End of the Emit ===== */

If there is a semantic error in the expanded code, the compiler's error messages will trace back to the specific line and column of the expanded macro code. Pay attention to the following points when using the debug mode of the Cangjie macro:

• In debug mode, the source code's line and column numbers are rearranged, which may not be suitable for certain special scenarios, such as handling line breaks. For example:

  // before expansion
  @M{} - 2 // macro M return 2
  
  // after expansion
  // ===== Emmitted my Macro M at line 1 ===
  2
  // ===== End of the Emit =====
  - 2
  

  The debug mode should not be used in cases where semantics is changed due to newline characters.

• Macro calls cannot be debugged in the macro definition. Otherwise, a compilation error is reported.

  public macro M(input: Tokens) {
      let a = @M2(1+2) // M2 is in macro M, not suitable for debug mode.
      return input + quote($a)
  }
  

• The debugging of macros with parentheses is not supported.

  // main.cj
  
  main() {
      // For macro with parenthesis, newline introduced by debug will change the semantics
      // of the expression, so it is not suitable for debug mode.
      let t = @M(1+2)
      0
  }