Locate AST read/write usage markers and builtin CallOp usage rules in LuisaCompute. Use when investigating or modifying AST variable usage propagation, FunctionBuilder internals, or CallOp argument marking.
Locate AST read/write usage markers and builtin CallOp usage rules in LuisaCompute. Use when investigating or modifying AST variable usage propagation, FunctionBuilder internals, or CallOp argument marking.
LuisaCompute AST Usage Markers
Quick reference for how manual FunctionBuilder AST tracks variable read/write usage and how builtin CallOp calls propagate usage to their arguments.
voidExpression::mark(Usage usage)constnoexcept{
if (auto a = to_underlying(_usage), u = a | to_underlying(usage); a != u) {
_usage = static_cast<Usage>(u);
_mark(usage);
}
}
Propagation is idempotent: it only forwards when new bits are added.
Only marks when the current builder owns the expression, preventing stale marking across function boundaries.
Manual API Example
auto &cur = *FunctionBuilder::current();
auto ref = cur.reference(Type::of<float4>());
cur.mark_variable_usage(ref->variable().uid(), Usage::READ_WRITE);
Builtin CallOp Usage Marking
Builtin detection
include/luisa/ast/op.h
[[nodiscard]] constexprautois_builtin_operation(CallOp op)noexcept{
return op != CallOp::CUSTOM && op != CallOp::EXTERNAL;
}
void CallExpr::_mark() constnoexcept {
if (is_builtin()) {
switch (_op) {
case CallOp::BUFFER_VOLATILE_WRITE:
case CallOp::BUFFER_WRITE:
case CallOp::BINDLESS_BUFFER_WRITE:
case CallOp::BYTE_BUFFER_VOLATILE_WRITE:
case CallOp::BYTE_BUFFER_WRITE:
case CallOp::TEXTURE_WRITE:
case CallOp::RAY_TRACING_SET_INSTANCE_TRANSFORM:
case CallOp::RAY_TRACING_SET_INSTANCE_VISIBILITY:
case CallOp::RAY_TRACING_SET_INSTANCE_OPACITY:
case CallOp::RAY_TRACING_SET_INSTANCE_USER_ID:
case CallOp::RAY_TRACING_SET_INSTANCE_MOTION_MATRIX:
case CallOp::RAY_TRACING_SET_INSTANCE_MOTION_SRT:
case CallOp::RAY_QUERY_COMMIT_TRIANGLE:
case CallOp::RAY_QUERY_COMMIT_PROCEDURAL:
case CallOp::RAY_QUERY_TERMINATE:
case CallOp::RAY_QUERY_PROCEED:
case CallOp::GRADIENT_MARKER:
case CallOp::ACCUMULATE_GRADIENT:
case CallOp::ATOMIC_EXCHANGE:
case CallOp::ATOMIC_COMPARE_EXCHANGE:
case CallOp::ATOMIC_FETCH_ADD:
case CallOp::ATOMIC_FETCH_SUB:
case CallOp::ATOMIC_FETCH_AND:
case CallOp::ATOMIC_FETCH_OR:
case CallOp::ATOMIC_FETCH_XOR:
case CallOp::ATOMIC_FETCH_MIN:
case CallOp::ATOMIC_FETCH_MAX:
case CallOp::INDIRECT_SET_DISPATCH_KERNEL:
case CallOp::INDIRECT_SET_DISPATCH_COUNT:
case CallOp::COOPERATIVE_OUTER_PRODUCT_ACCUMULATE:
case CallOp::COOPERATIVE_VECTOR_ACCUMULATE:
case CallOp::COOPERATIVE_VECTOR_STORE:
case CallOp::COOPERATIVE_VECTOR_WORKGROUP_STORE:
_arguments[0]->mark(Usage::WRITE);
for (size_t i = 1; i < _arguments.size(); i++) {
_arguments[i]->mark(Usage::READ);
}
break;
default:
for (auto arg : _arguments) {
arg->mark(Usage::READ);
}
}
} elseif (is_external()) {
auto f = external();
for (size_t i = 0; i < _arguments.size(); i++) {
_arguments[i]->mark(f->argument_usages()[i]);
}
} else {
// custom callableauto args = custom().arguments();
for (size_t i = 0; i < args.size(); i++) {
auto arg = args[i];
_arguments[i]->mark(
arg.is_reference() || arg.is_resource() ?
custom().variable_usage(arg.uid()) :
Usage::READ);
}
}
}
Atomic ops mark their target reference (argument 0) as WRITE; AtomicRefNode::operate() builds the CallExpr with the target as _arguments[0] (src/ast/atomic_ref_node.cpp).
Add a new write-style builtin op: extend the switch in src/ast/expression.cppCallExpr::_mark() so argument 0 is WRITE.
Query usage after building: call Function::variable_usage(uid) or FunctionBuilder::variable_usage(uid).
Custom callable reference/resource args: explicitly mark the reference variable READ_WRITE via mark_variable_usage() so callers propagate usage correctly.
Appendix: Full AST C++ Structure
File Inventory
Headers (include/luisa/ast/)
File
Main Class(es)
Description
usage.h
Usage (enum)
NONE, READ, WRITE, READ_WRITE flags
attribute.h
Attribute
Key-value pair struct for type/variable metadata
variable.h
Variable
Typed variable with Tag (LOCAL, SHARED, REFERENCE, BUFFER, TEXTURE, BINDLESS_ARRAY, ACCEL, and builtins like THREAD_ID, BLOCK_ID, DISPATCH_ID, etc.)
Ownership: FunctionBuilder owns all Expression and Statement objects via unique_ptr vectors. All raw pointers are non-owning views.
Builder stack: Thread-local _function_stack() enables Expression constructors to automatically capture their owning builder. FunctionStackGuard pushes/pops on definition.
Expression internalization (_internalize()): When a callable references a variable from an outer scope, the builder clones/captures the expression chain into the current function. Lvalue locals become reference arguments; resources become new resource arguments; builtins become new builtins; statically-evaluable expressions are recursively cloned.
Usage propagation: Two-phase: (a) Expression::_usage bitfield caches the aggregate usage at each expression node; (b) RefExpr::_mark() writes through to FunctionBuilder::_variable_usages[uid] for final variable-level query.
CallOp semantics: CallOpSet (bitset) tracks which builtins a function directly/propagatedly uses.
Serialization: CallableLibrary provides a custom binary serialization format for distributing callable function graphs.
Duplication: FunctionDuplicator creates a deep copy of a FunctionBuilder graph, remapping variable UIDs and hoisting leaked references.
AtomicRefNode: Chains buffer/array/structure access paths into a flat argument list for atomic CallExpr construction.
Visitor Helpers
traverse_subexpressions(expr, enter, exit) — walks all expression nodes recursively.
traverse_expressions<recurse_subexpr>(stmt, visit, enter_stmt, exit_stmt) — walks all expressions nested in a statement tree.
ExprVisitor — abstract visitor with virtual methods for each expression type.
StmtVisitor — abstract visitor with virtual methods for each statement type.
Type System Details
Type::from(description) parses string descriptions like "array<struct<16,int,float>,10>" into interned Type objects.
TypeRegistry (singleton) manages type pool and deduplication via unordered_set.
TypeImpl extends Type with concrete storage for hash, tag, size, alignment, dimension, members, member_attributes.
TypeDesc<T> maps C++ types to their string descriptions at compile time.
struct_member_tuple<T> decomposes structs into std::tuple of member types with offset validation.
⚠️ DO NOT reorder existing values — enum integer values are embedded in serialized function hashes and are assumed by call_op_count. Append your new op in the appropriate category section before CLOCK (the last enumerator). If you must add after CLOCK, update call_op_count and LUISA_MAGIC_ENUM_RANGE accordingly.
Also update:
call_op_count (line ~522): static constexpr size_t call_op_count = to_underlying(CallOp::CLOCK) + 1u; — This defines the size of the CallOpSet bitset. If your new op is added BEFORE CLOCK, call_op_count already covers it. If added AFTER CLOCK, increment this value.
LUISA_MAGIC_ENUM_RANGE (line ~664): LUISA_MAGIC_ENUM_RANGE(luisa::compute::CallOp, CUSTOM, CLOCK) — Enables to_string/from_string for the range [CUSTOM, CLOCK]. If your new op is after CLOCK, extend the range to include it.
Step 2: Update usage propagation
File:src/ast/expression.cpp — CallExpr::_mark()
Read-only (default):
No change needed — the default case marks all args Usage::READ.
Write-style (arg[0] = WRITE, rest = READ):
Add to the existing switch:
case CallOp::MY_NEW_OP:
_arguments[0]->mark(Usage::WRITE);
for (size_t i = 1; i < _arguments.size(); i++) {
_arguments[i]->mark(Usage::READ);
}
break;
Custom usage:
Implement arbitrary logic in the switch.
Step 3: Add validation (optional but recommended)
File:src/ast/op.cpp — check_builtin_call_valid()
Add a case to validate argument types and counts at AST construction time:
Step 4: Add helper functions for category detection (optional)
File:include/luisa/ast/op.h
If your op belongs to a new category, add a constexpr helper:
[[nodiscard]] constexprautois_my_category_operation(CallOp op)noexcept{
auto v = to_underlying(op);
return v >= to_underlying(CallOp::MY_CATEGORY_START) &&
v <= to_underlying(CallOp::MY_CATEGORY_END);
}
Step 5: Update each backend codegen
Each backend has a switch on CallOp that emits native code or IR. Add your case to all of them:
Networked backend proxy; no direct AST CallOp switch
Validation
src/backends/validation/
AST validation layer wrapping another backend; no own CallOp switch
Example CUDA addition:
case CallOp::MY_NEW_OP: {
_scratch << "my_new_op(";
for (auto i = 0u; i < args.size(); i++) {
if (i) _scratch << ", ";
emit(args[i]); // use the backend's expression emitter
}
_scratch << ")";
break;
}
Step 6: (Optional) Add DSL helper
If the op should be exposed via the high-level DSL, add a helper in src/dsl/:
Add ser_value and deser_ptr specializations for the new expression type, plus integrate into the Expression base ser_value/deser_value dispatch.
Step 6: Add codegen in each backend
Each backend that processes AST expressions directly (CUDA, Metal, HLSL, SPIR-V LLVM, LLVM/CPU) needs a case Expression::Tag::MY_NEW_EXPR in its visitor switch.
Step 7: Add JSON export (optional)
File:src/ast/ast2json.cpp
Add a conversion method in AST2JSON and wire it into _convert_expr().
How to Add a New Statement
Step 1: Add the statement class
File:include/luisa/ast/statement.h
Add a new Tag enum value to Statement::Tag.
Forward-declare (e.g., class MyNewStmt;).
Add virtual void visit(const MyNewStmt *) = 0; to StmtVisitor.
Implement the class inheriting Statement:
classLUISA_AST_API MyNewStmt final : public Statement {
friendclassCallableLibrary;
private:
// data membersMyNewStmt() noexcept = default;
private:
[[nodiscard]] uint64_t _compute_hash() constnoexceptoverride;
public:
MyNewStmt(/* params */) noexcept
: Statement{Tag::MY_NEW_STMT} /*, init */ {
// mark expression usages here
}
// accessorsLUISA_STATEMENT_COMMON()
};
Step 2: Add hash computation
File:src/ast/statement.cpp
uint64_t MyNewStmt::_compute_hash() constnoexcept {
returnhash_combine({/* member hashes */});
}
Step 3: Add to traverse_expressions
File:include/luisa/ast/statement.h — add a case in the traverse_expressions template function.