Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

x2c The x2c Book

Scopes and Lifetime

The system macros provide $scope() for a retained region, $scope(pointer) for a temporary destination, and $auto for an explicitly owned local. Classes can also supply early free operations.

C makes you answer one question about every allocation: who frees this? x2c does not take that question away. There is no garbage collector and no reference counting. x2c gives you a place to put the answer. A scope holds a group of allocations and frees them together, and some values keep their storage for the life of the process, so not every value needs a matching free.

The standard library overview links to the detailed API reference.

What actually owns the storage

A Var is eight bytes. Integers that fit, doubles, pointers, Symbols, Null, and void are stored in those bytes and allocate nothing. The wide scalars do not fit alongside a tag: long, unsigned long, long long, unsigned long long, and long double. Those go in an immutable box that the allocator owns, and the Var holds its address:

Var small = 42;
long huge = 1L << 60;
Var boxed = huge;
printf("%ld %ld\n", small.integer(), boxed.integer());

Both Vars are eight bytes; only the second one allocated. A boxed Var is valid while the scope that allocated the box is alive. See Values and Var for the encoding itself.

Canonical values are the other case. A non-empty String and every List cell built by cons are interned: equal content becomes one canonical pointer. By default those values stay valid for the rest of the process. You do not free them, and you do not have to keep a scope alive to hold them. They are held in canonical interning pools. The Pool calls described below give temporary String and List structure a shorter lifetime.

That leaves mutable storage: Block, Array, Buffer, wide Var boxes, and anything you request with Scope.malloc, Scope.calloc, or Scope.memdup. All of it belongs to a scope.

A scope is a group of allocations

Scope.retain opens a new region on the currently active scope slot; Scope.release frees everything allocated inside it. Between them, the allocation calls go into the retained region:

Scope.retain();
char *line = Scope.malloc(32);
snprintf(line, 32, "%d bytes", 32);
puts(line);
Scope.release();

Retained regions nest. Each Scope.retain must be paired with exactly one Scope.release, and a nested region is the normal way to say “these temporaries are freed before the surrounding work finishes.”

The active slot tracks the match. An empty retained region still releases normally, but an extra release, or a release from a different pushed slot, aborts instead of destroying the surrounding process root.

Storage returned by these calls is managed memory. Scope.realloc grows or shrinks it in place, and Scope.free ends its life early without waiting for the release. Scope.free shortens a lifetime. Most storage does not need it, because the release frees the region. Scope.malloc_finalized allocates storage that runs a function of your choice when it is reclaimed; see attaching a finalizer.

For work with a longer life, hold a scope in a variable of type Scope and allocate into it directly. Scope.malloc_in, Scope.calloc_in, and Scope.memdup_in take the slot’s address and leave the active scope alone, so intervening allocations cannot be redirected by accident. Scope.destroy ends such a scope:

Scope work = Scope.new_named("request");
char *copy = Scope.memdup_in(&work, "payload", 8);
puts(copy);
Scope.destroy(work);

Scope.new_named copies a diagnostic name. Give one to anything long-lived. At exit the allocator reports what is still alive, and the name tells you which scope. Drop the Scope.destroy line above and the program prints this on standard error before it ends:

Scope leak detected:
	live_scopes: 1
	live_allocations: 1
	scope "forgotten": 1 allocations
	live_backing_allocations: 2

The last line counts the allocator’s own bookkeeping, such as the record holding that name. Scope.stats reports the same counters on demand, so you can check that a routine leaves no allocations behind.

When several operations should allocate in the same scope without passing a slot to every call, Scope.push makes a slot active and Scope.pop restores the previous one. Scope.destroy refuses a scope that is still in use: the active root, a slot still on the pushed stack, or a region with another region attached below it. Those cases print a diagnostic and abort instead of corrupting the allocation lists. A mismatched push/pop or a double destroy aborts with a diagnostic.

Bounding temporary canonical Lists

Canonical identity does not require every List in a batch or request to stay alive until process exit. When a body of work builds substantial List structure and no List from that work escapes, bracket it with Pool.open and Pool.close. String and List share one canonical pool, so the bracket bounds both:


static Var build_left(int input) {
  return input - 1;
}

static Var build_right(int input) {
  return input + 1;
}

static int temporary_score(List values) {
  return values.len();
}

static int evaluate(int input) {
  Pool.open();
  defer Pool.close();

  List temporary = %($input ${build_left(input)} ${build_right(input)});
  return temporary_score(temporary);
}

Every cons inside the bracket remains canonical. Pool.close reclaims cells created in the innermost pool and forgets their identities. Calls nest, and every successful Pool.open requires one matching Pool.close. Use defer when an error or early return can cross the boundary. Pool.open_named gives the bracket a name that appears in Scope diagnostics, and Pool.current returns the pool that is active right now.

If a List must survive, call List.promote before release. Promotion is pointer-stable and recursively preserves nested Lists, interned String cars, and long Atom payloads:


static List build_result(void) {
  return %(result);
}

static List promoted_result(void) {
Pool.open();
List result = build_result();
result.promote();
Pool.close();
return result;
}

An unpromoted List dangles after the release. Use these calls when a task creates substantial temporary List structure. Small amounts of data and values needed for the rest of the process can stay in the default pool.

Context bundles a pool with the rest of a boundary

A pool bracket bounds canonical values and nothing else. A Context is the aggregate: one open Context owns a Scope, Error state, Match state, and, when it is opened isolated, a canonical-value pool of its own. Context.open_isolated calls Pool.open_named to get that pool, and Context.close closes it. So the bracket is the smaller tool underneath Context, not a feature of it. Reach for Pool.open when only canonical String and List structure needs bounding, and for a Context when the Scope and the error and match state must be bounded with it. See Contexts and Threads.

Moving a value out of a scope

Sometimes a temporary region computes one result that has to survive it. Scope.move relinks a single allocation onto another scope without copying it. The pointer does not change:

Scope keep = Scope.new_named("results");
Scope.retain();
char *text = Scope.memdup("survivor", 9);
Scope.move(text, &keep);
Scope.release();
puts(text);
Scope.destroy(keep);

Use this instead of letting a pointer escape. A String or a List needs no move; canonical values outlive the scope that was active when they were built.

Attaching a finalizer

A record that holds something the allocator does not know about, such as a handle from a C library, can release it when the record is reclaimed. Scope.malloc_finalized takes the function to run:

#include <stdio.h>
typedef struct feed_parser feed_parser;
static feed_parser *feed_open(void) { return (feed_parser *) 1; }
static void feed_close(feed_parser *parser) { (void) parser; }
typedef struct Feed { feed_parser *native; } *Feed;

static void _feed_drop(void *ptr) {
  Feed feed = ptr;
  if (feed.native) feed_close(feed.native);
  feed.native = NULL;
}

Feed Feed.open(void) {
  Feed feed = Scope.malloc_finalized(sizeof(struct Feed), _feed_drop);
  feed.native = feed_open();
  return feed;
}
int main(void) {
  Scope.retain();
  Feed feed = Feed.open();
  Scope.release();
  return 0;
}

The finalizer runs exactly once with the record’s pointer: on Scope.free, on a Scope.realloc to size zero, when the owning scope is released or destroyed, or at thread and process shutdown. It follows the record through Scope.move, and Scope.realloc keeps it and passes the resized pointer. Records are reclaimed most recent first, so a finalizer can still read older records in the same scope.

Three rules keep this simple. The record is already unlinked when its finalizer runs, so the finalizer must not free or move the record itself. It must not raise. It may allocate into other scopes, and into the scope being destroyed only for scratch that the same destruction reclaims.

An explicit early release still works: a wrapper that clears its native field in its own free method leaves nothing for the finalizer to do.

Block and Buffer

Block is the fixed-width storage primitive. A Block owns a contiguous run of elements of one width, chosen at construction, and tracks their length and capacity. Its backing store comes from the active scope, so a Block created inside a retained region dies with that region even if you never call Block.free.

Block ids = Block.new(sizeof(int));
for (int i = 0; i < 4; i++) ids.push(&i);
int *values = ids.bytes;
printf("%zu of %zu, last %d\n", ids.len(), ids.capacity(), values[3]);
ids.free();

Look at where values is read. Growth reallocates, so the bytes pointer moves; a copy of it taken before an append can be stale afterwards. The Block handle stays valid across growth, so pass the handle around. If you work through the raw Bytes pointer, use the forms that hand the pointer back, Bytes.append and Bytes.reserve, or recover the handle with Bytes.block. Block.append and Block.reserve raise <bad-arg> on a bad argument; the storage is unchanged and the raise does not return to the call. Capacity and allocation failures raise <size-limit> and <alloc-fail>, which do not return to the raising call. Array is a Block whose element width is sizeof(Var); see Strings, Lists, Arrays, and Maps.

Buffer is a text builder layered on Block. It rejects an embedded NUL, because it builds canonical Strings, and it tracks the byte position and leading indentation of the current line for push, pop, indent, and newline_indent. Use Block for checked storage of some element type, and Buffer for assembling text.

String text;
$scope() {
  Buffer out = $auto(Buffer.new(0));
  out.write("case ").printf("%d", 7).write(":").newline_indent();
  text = out;
}
puts(text);

The String destination calls Buffer.str, which interns the accumulated text, so text is canonical. It is still valid after the Buffer cleanup and the region release that disposed of every byte the builder used. Most x2c code is written this way: mutable storage in a region, canonical result outside it.

defer for cleanup

defer statement schedules a statement for the exit of the enclosing block. It runs on normal exit, on return, on break, and when an Error passes through; multiple defer statements run last-in, first-out.

FILE *log = fopen("/dev/null", "a");
if (log) {
  defer fclose(log);
  fprintf(log, "started\n");
}

Keep the deferred statement small. A defer should release something the block owns, and it puts the release next to the acquisition, where a reader can check them against each other.

Do not use defer to swallow failures. Let Errors propagate and keep expected absence, exhaustion, and parse outcomes in the return value. The interaction with raise, try, filtered catch, and finally is described in Errors and Cleanup, and the exact statement rules in the language reference.

Choosing when to free storage

Free storage when the work that needs it is finished: a request has ended, a file has been parsed, or temporary values have been combined into one result. A scope wrapped around canonical values that already outlive it buys nothing and adds a release you can forget.

A scope does not prevent dangling pointers:

static char *leaked_label(void) {
  Scope.retain();
  char *label = Scope.memdup("temporary", 10);
  Scope.release();
  return label;
}

Scope.release freed that storage. The returned pointer is dangling, and every use of it afterwards is undefined behavior: reading stale bytes, corrupting the allocator, or appearing to work until it does not. Scopes make lifetimes explicit and cheap to end. They do not check at runtime that a value has stopped being used. The same applies to a boxed wide Var: if you keep the Var, you must keep the scope that allocated the box.

The compiler warns about this example because the escape is visible in the source: label is allocated between a retain and its release and read after it. The next two sections give the rule behind that warning and what the check covers.

Three habits keep this out of your code. Return a canonical value, a String, a List, or a Symbol, when a result must cross a scope boundary. Use Scope.move when a mutable allocation has to survive. When in doubt, let the caller create the scope and pass the slot down, so the lifetime is visible where it was chosen.

The region model

Every allocation in this chapter belongs to a region: a span of the program that owns storage and ends at a definite point. A $scope() block is a region, and so are a Scope.retain and Scope.release pair, a $scope(&slot) push, an $auto local, and a Scope local that Scope.destroy ends. A Pool.open bracket is a region over the pool, so a List cell consed inside one belongs to that bracket rather than outliving every scope the way a canonical value normally does.

The model is one rule: a value allocated inside a region must not be reachable after that region ends. Four kinds of place outlive a region, and reaching one of them is how the rule gets broken. They are the function’s result, storage declared outside the region, an object owned by an outer or sibling region, and any pointer whose target the compiler cannot identify. The advice in this chapter follows from that rule. Scope.move and Context.export work because they change which region owns the storage, and returning a String, List, or Symbol works because the pool owns it and no region end frees it.

The compiler checks this rule and warns where it is broken. The next section lists those warnings and the exits they name. The Region Model states the invariant precisely, says what the check cannot see, and explains how it works.

Warnings when a value outlives its region

Translation warns when a value allocated inside a region can still be reached after the region ends. The warning includes the value’s name and the way the value leaves, and its note gives the line that opened the region:

  • returned;
  • assigned to a local declared outside the region;
  • stored through a parameter, through an unknown pointer, or into a static;
  • stored into an object that belongs to another region;
  • held by a List cell, as in cons(a, rest) or %($a), or captured by a closure that leaves;
  • handed to a function in the same unit, or to a runtime operation such as Array.push, that stores it in one of those places.

Two more warnings come from the same pass. unbalanced reports a region with no matching release in the block that opened it, a shape the other warnings cannot track. after-free reports a local read after Scope.free or Array.list_free consumed it.

Code that follows the patterns in this chapter compiles without warnings:

typedef struct Entry { int id; } *Entry;
int main(void) { Scope keep = NULL; Scope.push(&keep); Entry e = adopt(&keep);
  Scope.pop(); Scope.destroy(keep); return e != NULL && !label().len(); }
static Entry adopt(Scope *keep) {
  $scope() {
    Entry entry = Scope.calloc(1, sizeof(struct Entry));
    Scope.move(entry, keep);
    return entry;
  }
  return NULL;
}

static String label(void) {
  $scope() {
    Buffer out = Buffer.new(0);
    out.write("ready");
    return out;
  }
  return NULL;
}

Scope.move moves the storage into a scope the caller owns, and the String return destination calls Buffer.str, which produces a canonical String owned by its pool. Context.export and List.promote end tracking the same way.

These warnings cover the lexical pattern only. They do not cover storage from plain malloc or a C library, raw pointer arithmetic and casts, values reached through a field of a stack struct, callbacks and function pointers, entry points a Lisp binding calls, Context regions, or $auto cleanups of types other than the runtime’s containers and Scope. Each of those can still produce a dangling pointer that translates without a warning. To observe the dangling read itself, link the program against libx2c.a and compile with cc -fsanitize=address,undefined.

Translation continues after a warning, and the program still compiles. When a warning describes a lifetime you have arranged some other way, you may keep the code as written.

Ordinary C storage still works

x2c is a superset of C, and it does not change automatic or static storage. Locals, arrays, and struct values still live on the stack and die at the closing brace; static and file-scope objects still last for the program. Scopes govern only what you explicitly allocate through them.

Many library types take the address of caller-owned state instead of allocating, so a loop can iterate without touching the allocator:

List items = %(1 2 3);
struct Iter storage;
Iter walk = items.iter(&storage);
Var value;
while (walk.try_next(&value)) printf("%ld\n", value.integer());

storage is an automatic variable. Nothing here needs freeing or a scope; see Iteration for the protocol. An automatic local or parameter you assigned directly keeps its value when an Error carries control away, and your source needs no optimization-specific qualifiers for that.

Where to look next

Read the standard library overview and the language reference for the rules. Idioms shows how scopes combine with other x2c features. The Region Model covers the lifetime check itself: what it guarantees, what it leaves to you, and how it compares with the checks in other languages.