Audio & signals

Soundtrack

import { Effect, Layer } from "framefields";

comp.addAudio(
  Layer.audio("assets/audio/score.mp3", {
    durationFrames: totalFrames, // required: without it audio can cut short
    volume: 0.9,
  }).apply(Effect.audioFade({ fadeInDuration: 2, fadeOutDuration: 2, fadeOutCurve: "scurve" })),
);

Layer.audio takes volume, startFrame, durationFrames, muted. Video layers carry their own sound (muted, volume, trimStartSec).

Audio effects attach with .apply(): Effect.audioFade, audioCompressor, audioParametricEq, audioReverb, audioDelay, audioNoiseGate, stereoPanning (or the classes AudioFade, AudioCompressor, …). Props are in the Effects reference.

Procedural SFX

Whooshes, risers, impacts, downshifters and glitches are synthesized on the CPU, seeded, and placed on a bar grid — the same input always gives the same samples.

import fs from "node:fs/promises";
import { encodeStereoWav, Layer, mixSfxInto, renderSfx, softLimit } from "framefields";

const sampleRate = 48_000;
const seconds = totalFrames / FPS;
const bed: [Float32Array, Float32Array] = [
  new Float32Array(Math.ceil(seconds * sampleRate)),
  new Float32Array(Math.ceil(seconds * sampleRate)),
];
const ctx = { sampleRate, secondsPerBar: 2.4 }; // 100 BPM, 4/4

mixSfxInto(bed, [
  renderSfx({ type: "riser", atBar: 0.5, volume: 0.4 }, { ...ctx, seed: 1 }),
  renderSfx({ type: "impact", atBar: 1, volume: 0.8 }, { ...ctx, seed: 2 }),
  renderSfx({ type: "whoosh", atBar: 2.75, volume: 0.5 }, { ...ctx, seed: 3 }),
]);
softLimit(bed);

await fs.writeFile("output/sfx.wav", encodeStereoWav(bed, { sampleRate }));
comp.addAudio(Layer.audio("output/sfx.wav", { durationFrames: totalFrames }));

atBar is where the sound starts (fractional bars allowed); a riser that should land on a cut starts durationSec earlier. mixAudioTracks mixes several tracks headlessly.

Signals

A signal is a value per frame. Bind one to any animatable prop, or pass it to any numeric effect prop.

import { LayerAnimation, Signal } from "framefields";

const pulse = Signal.builder({ type: "sawtooth", frequency: 2, amplitude: 0.08, offset: 1 }); // 120 BPM

card.animate(
  LayerAnimation.create().signal("scale", pulse, { multiplier: 1, offset: 0, smoothing: 2 }),
);

The prop becomes value × multiplier + offset each frame. Signals are sampled at the composition frame.

Constructor	Value
Signal.builder({ type, frequency, amplitude, offset })	sine, triangle, sawtooth, square, bounce, or custom with fn(ctx)
Signal.sine({ frequency, min, max, phase }) / Signal.cosine	periodic wave; frequency in Hz
Signal.fromArray(values, fps)	one sample per frame, e.g. a precomputed kick envelope
Signal.programmatic((ctx) => number)	any function of ctx.frame, ctx.time, ctx.progress
Signal.frame / Signal.time / Signal.progress	the clock itself, as signals (read .value)
Signal.state(v) / Signal.computed(() => …)	reactive values, e.g. Signal.computed(() => 55 - Signal.frame.value * 0.1)

Shared signals by name

comp.addSignal("kick", Signal.fromArray(kickEnvelope, FPS));
LayerAnimation.create().signal("opacity", "kick", { multiplier: 0.5, offset: 0.5 });

A signal object is registered automatically; a string must match an addSignal name. Anything else throws.

Colors

.colorSignal(prop, signal, options) drives a color prop; modes are interpolate, hueRotate and threshold.

Audio-reactive visuals

Extract an envelope from audio with the AudioSignalExtractor effect (modes, attack/release, beat detection), or precompute one in Node and feed it through Signal.fromArray. Precomputing keeps the film a pure function of the frame and is the easiest to test.