Differentiate a function.

Generate derivatives with a compile-time Lisp decorator, then use them to fit a growth curve.

The idea in code

$(import "autodiff.xmacro")

$ad.checkpoint(64)
static double loss(double rate, double capacity) {
  double p = 1.0, t = 0.0, sum = 0.0;
  for (int step = 0; step < 4000; step++) {
    p += 0.0025 * rate * p * (1.0 - p / capacity);
    t += 0.0025;
    if (step % 400 == 399) {
      double residual = p - 50.0 / (1.0 + 49.0 * exp(-0.9 * t));
      sum += residual * residual;
    }
  }
  return sum;
}

int main(void) {
  double rate = 0.5, capacity = 30.0, d_rate, d_capacity, h = 1e-6;
  double value = loss_grad(rate, capacity, &d_rate, &d_capacity);
  for (int i = 1; i <= 200; i++) {
    value = loss_grad(rate, capacity, &d_rate, &d_capacity);
    rate -= 1e-5 * d_rate;
    capacity -= 1e-2 * d_capacity;
    if (i % 50 == 0)
      printf("step %2d  loss %10.6f  rate %.4f  capacity %.4f\n",
             i, value, rate, capacity);
  }
  return 0;
}

Fit a growth curve.

Fit a growth model to observations by adjusting its growth rate and maximum population. The loss function simulates the population over 4,000 small time steps and measures its squared error at ten observation points. This example generates those observations from a known model, so there is a specific answer to recover: a rate of 0.9 and a capacity of 50.

$ad.checkpoint(64) is a decorator implemented with x2c meta functions. It reads the typed syntax tree and generates loss_grad beside the original function. The generated function returns the loss and writes its derivatives into d_rate and d_capacity. Those derivatives account for the entire simulation, including the branch that selects which steps contribute to the error. The compiler itself has no differentiation code.

Improve the fit.

The outer loop uses the gradient to improve the parameter estimates over 200 iterations. Checkpointing saves the simulation state every 64 steps and replays each block when calculating derivatives, reducing the amount of intermediate state it needs to retain. Starting from 0.5 and 30, the fit reaches approximately 0.9006 and 49.9922.

The full program also compares the generated gradient with finite differences before fitting.

Full example / Autodiff macros

Fitting a growth curve.

The excerpt above shows the loss function and fitting loop from autodiff-fit.x. The observations are generated from the analytic solution of the model. The fitted simulation uses Euler integration, so even the correct parameters produce a small numerical error.

The full program first checks its gradient against central finite differences, then takes 200 gradient-descent steps toward the model’s rate of 0.9 and capacity of 50.

Supported operations and limitations.

The transformation supports the documented scalar double subset. Reverse mode supports loops, branches, break, continue, and early returns; it requires unique local names. Checkpointed loops cannot contain return, and fixed-block checkpointing still grows with the number of iterations. The guide explains the tradeoff, supported operations, and what the decorators reject.

Try changing the fit’s initial rate and capacity to see how they affect convergence. The finite-difference comparison lets you check the generated gradient as you experiment.

autodiff-fit.xExpected output

loss 4702.386283
d/drate     -15650.816  finite difference -15650.816
d/dcapacity -118.848  finite difference -118.848
step 50  loss   2.373136  rate 0.9140  capacity 49.1688
step 100  loss   0.014606  rate 0.9016  capacity 49.9274
step 150  loss   0.000256  rate 0.9007  capacity 49.9874
step 200  loss   0.000165  rate 0.9006  capacity 49.9922
observed(4) 21.3779  model(4) 21.4076

Your turn

Run it locally.

Build x2c, then run the fitting example shown in the output above.

You need a GCC- or Clang-compatible C compiler, ar, GNU Make, Python 3, and Bash. The build guide covers setup in detail.

From a new checkout

git clone https://github.com/gwf/x2c.git
cd x2c
git checkout --detach 53c1ed815d792b4a1af2b182a63e32feb1a5e007
make build-safe

./x2c run examples/magic/autodiff-fit.x