This is the abridged developer documentation for framefields # framefields docs > What framefields is, a complete first composition, and where to go next. framefields is a TypeScript library that renders video on the GPU, frame by frame, straight to MP4. A video is a `Composition`: a tree of layers with keyframed animation. It renders natively on WebGPU (Dawn → Metal / Vulkan) in Node.js — no browser, no Puppeteer, no screenshots. It is built to be driven by coding agents. Give yours the skills, then ask for a video: * Claude Code ```text /plugin install framefields ``` * Any other agent ```bash npx skills add gatewai-dev/framefields ``` More in [framefields for agents](/docs/agents/). Or write it yourself — this is a complete film: src/film.ts ```ts import { Composition, FontManager, Layer, LayerAnimation } from "framefields"; export async function buildFilm() { await FontManager.register({ family: "Inter", source: "assets/fonts/Inter.ttf" }); const film = new Composition({ width: 1920, height: 1080, fps: 30, durationMs: 3000, backgroundColor: "#0e0d0c", }); film.add( Layer.text("Hello, framefields", { position: "absolute", x: 0, y: 480, width: 1920, height: 120, fontFamily: "Inter", fontSize: 96, fontWeight: 700, fill: "#eee9e0", align: "center", }).animate( LayerAnimation.create() .fadeIn(0, 20, "power2.out") .fromTo("y", 520, 480, { start: 0, end: 30, ease: "expo.out" }), ), ); return film; } // elsewhere: await (await buildFilm()).renderVideo({ outputPath: "output/hello.mp4" }); ``` ## What it covers [Section titled “What it covers”](#what-it-covers) | Area | You get | | ------------- | ----------------------------------------------------------------------------------------- | | Layout & type | Flexbox (Yoga) boxes, GPU vector text (Slug), rich text, RTL, text animators | | Motion | Keyframes, 13 easing families, springs, wiggles, signals, scene transitions | | Effects | 52 GPU effects: grading, grain, blur, LUTs, keying, glass, relighting, audio DSP | | 3D | Cameras, lights, PBR environment, shadows, bloom, primitives, glTF/OBJ/FBX/STL/PLY models | | Audio | Soundtracks, procedural SFX, audio-reactive signals | | Data | Animated line, area, bar, scatter, candlestick, pie and donut charts | | Vision | On-device detection, segmentation, pose and mattes driving layout and effects | | Delivery | PNG frames, contact-sheet grids, MP4 with built-in QA, live browser preview | ## Start here [Section titled “Start here”](#start-here) [Getting started](/docs/getting-started/)Install, scaffold, and render your first MP4. [framefields for agents](/docs/agents/)Install the skills, the agent workflow, example prompts. [Core concepts](/docs/core-concepts/)Compositions, frames, layer trees, timing and layout. [Authoring rules](/docs/reference/rules/)The short list of rules that prevent most broken renders. ## For agents [Section titled “For agents”](#for-agents) Machine-readable docs See [Docs for agents](/docs/agents/llms/). Every page has a Markdown twin: swap the trailing `/` for `.md` (`/docs/guides/animation/` → `/docs/guides/animation.md`; this page is [`/docs/index.md`](/docs/index.md)). The whole site is in [`/docs/llms-full.txt`](/docs/llms-full.txt), a compact version in [`/docs/llms-small.txt`](/docs/llms-small.txt), and an index in [`/docs/llms.txt`](/docs/llms.txt). Requirements: **Node.js ≥ 22**, an ES module project, and a **WebGPU-capable GPU** (Metal on macOS, Vulkan on Linux/Windows). There is no CPU fallback. framefields is Apache-2.0 and in beta (v2.1). # Getting started > Install framefields, scaffold theme/film/render files, and verify a first render from cheapest check to final MP4. ## 1. Check the machine [Section titled “1. Check the machine”](#1-check-the-machine) ```bash node --version # must be v22 or later ``` * **GPU required.** Rendering uses WebGPU via Dawn: Metal on macOS, Vulkan on Linux/Windows. VMs and CI runners without a GPU cannot render. * **ES modules.** `package.json` needs `"type": "module"`. ## 2. Write `package.json`, then install [Section titled “2. Write package.json, then install”](#2-write-packagejson-then-install) framefields needs exactly one dependency install script: `skia-canvas` downloads its native binary. npm 11, pnpm 10 and bun block install scripts unless approved, so approve that one and deny the rest **before** installing. package.json ```json { "name": "my-film", "private": true, "type": "module", "allowScripts": { "skia-canvas": true, "sharp": false, "webgpu": false, "onnxruntime-node": false, "node-av": false, "esbuild": false }, "pnpm": { "onlyBuiltDependencies": ["skia-canvas"], "ignoredBuiltDependencies": ["sharp", "webgpu", "onnxruntime-node", "node-av", "esbuild"] }, "trustedDependencies": ["skia-canvas"], "scripts": { "typecheck": "tsc --noEmit -p .", "frames": "tsx src/render.ts frames", "grid": "tsx src/render.ts grid", "preview": "tsx src/render.ts preview", "render": "tsx src/render.ts" } } ``` `allowScripts` is read by npm, `pnpm` by pnpm, `trustedDependencies` by bun. Yarn needs none of them. ```bash npm install framefields npm install --save-dev tsx typescript @types/node ``` Caution If you see `Cannot find module '.../skia.node'`, the `skia-canvas` script was skipped. Approve it as above, then run `npm rebuild skia-canvas` (or `pnpm rebuild skia-canvas`). tsconfig.json ```json { "compilerOptions": { "module": "NodeNext", "moduleResolution": "NodeNext", "target": "ES2022", "strict": true, "skipLibCheck": true, "noEmit": true, "types": ["node"] }, "include": ["src/**/*"] } ``` ## 3. Scaffold [Section titled “3. Scaffold”](#3-scaffold) Keep the composition separate from the script that renders it, so tests, preview and export all build the same film. ```plaintext src/ theme.ts sizes, fps, colors, fonts — the only place for magic numbers film.ts buildFilm(): returns the Composition render.ts frames | grid | preview | video assets/ every input file, stored with Git LFS fonts/ .ttf / .otf (text needs a real font file) footage/ .mp4 / .mov / .webm audio/ .mp3 / .wav images/ .png / .jpg / .webp output/ renders (gitignored, never committed) .gitattributes LFS rules for assets/ ``` src/theme.ts ```ts import path from "node:path"; import { fileURLToPath } from "node:url"; import { FontManager } from "framefields"; const HERE = path.dirname(fileURLToPath(import.meta.url)); export const W = 1920; export const H = 1080; export const FPS = 30; export const INK = "#0e0d0c"; export const PAPER = "#eee9e0"; export const SANS = "Inter"; export async function registerFonts(): Promise { // Resolve from the module URL so the browser preview finds the file too. await FontManager.register({ family: SANS, source: path.resolve(HERE, "../assets/fonts/Inter.ttf") }); } ``` src/film.ts ```ts import { Composition, Layer, LayerAnimation } from "framefields"; import { FPS, H, INK, PAPER, registerFonts, SANS, W } from "./theme.js"; const DURATION = 3 * FPS; // frames export async function buildFilm(): Promise { await registerFonts(); // before any text layer is built const film = new Composition({ width: W, height: H, fps: FPS, durationMs: (DURATION / FPS) * 1000, backgroundColor: INK, }); film.add( Layer.text("Hello, framefields", { id: "title", position: "absolute", x: 0, y: H / 2 - 60, width: W, height: 120, fontFamily: SANS, fontSize: 96, fontWeight: 700, fill: PAPER, align: "center", }).animate( LayerAnimation.create() .fadeIn(0, 20, "power2.out") .fromTo("y", H / 2 - 20, H / 2 - 60, { start: 0, end: 30, ease: "expo.out" }), ), ); return film; } ``` src/render.ts ```ts import fs from "node:fs/promises"; import path from "node:path"; import { HeadlessMediaRenderer, startPreview } from "framefields"; import { buildFilm } from "./film.js"; const OUT = path.resolve(import.meta.dirname, "../output"); const [mode = "video", ...args] = process.argv.slice(2); if (mode === "preview") { const session = await startPreview({ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" }); console.log(`Preview at ${session.url}`); await session.closed; process.exit(0); } const film = await buildFilm(); await fs.mkdir(OUT, { recursive: true }); if (mode === "frames") { const renderer = new HeadlessMediaRenderer(); for (const frame of args.map(Number)) { const png = await film.renderFrame({ frame, renderer }); await fs.writeFile(path.join(OUT, `f${String(frame).padStart(4, "0")}.png`), png); } } else if (mode === "grid") { const png = await film.renderFrameGrid({ frames: args.length ? args.map(Number) : [0, 10, 20, 30, 45, 60, 75, 89], columns: 4, cellWidth: 480, }); await fs.writeFile(path.join(OUT, "grid.png"), png); } else { const result = await film.renderVideo({ outputPath: path.join(OUT, "output.mp4"), quality: "high", qa: true }); console.log(`Exported ${result.filePath}`); } // Native GPU and encoder threads keep Node alive: always exit explicitly. process.exit(0); ``` ## 4. Verify, cheapest step first [Section titled “4. Verify, cheapest step first”](#4-verify-cheapest-step-first) Each step catches in seconds what a full render takes minutes to show. | Step | Command | Look for | | ---- | --------------------------- | --------------------------------------------------------------------------------------------------------- | | 1 | `npm run typecheck` | type errors | | 2 | `npm run frames -- 0 30 60` | PNGs in `output/`; a black frame usually means a missing font, an off-canvas layer, or opacity stuck at 0 | | 3 | `npm run grid` | smooth motion across the contact sheet | | 4 | `npm run preview` | a person watches it with sound ([Live preview](/docs/guides/preview/)) | | 5 | `npm run render` | the MP4, plus QA findings ([Rendering](/docs/guides/rendering/)) | ## 5. Store assets with Git LFS [Section titled “5. Store assets with Git LFS”](#5-store-assets-with-git-lfs) Keep every input file under `assets/` and store it with [Git LFS](https://git-lfs.com). Set it up before adding any media: ```bash git lfs install git lfs track "assets/fonts/**" "assets/footage/**" "assets/audio/**" "assets/images/**" \ "assets/luts/**" "assets/models/**" "assets/lottie/**" echo "output/" >> .gitignore git add .gitattributes .gitignore && git commit -m "Track assets with Git LFS" ``` **Push:** `git add assets/…`, commit and `git push` as usual; the media upload with the push. Run `git lfs ls-files` before committing to confirm each new file is listed. **Pull:** in a fresh clone, container or CI runner, run `git lfs pull` before rendering (in GitHub Actions, `actions/checkout` with `lfs: true`). Caution Until pulled, LFS files are \~130-byte pointers. framefields cannot decode them: fonts fail to register and media renders blank. Ignore `output/`, not `*.mp4`, or your footage disappears from Git too. Details, including fixing a file committed without LFS: [Assets and Git LFS](/docs/agents/workflow/#assets-and-git-lfs). ## Next [Section titled “Next”](#next) * [Core concepts](/docs/core-concepts/) — how frames, layers and timing fit together. * [framefields for agents](/docs/agents/) — install the skills so your agent knows all of this. # framefields for agents > Why framefields is built for coding agents, how to give your agent the skills in one command, and the loop it follows to ship a video. framefields is made to be driven by a coding agent. A film is a TypeScript file, every check is a function that returns data, and every review step produces a file the agent can open and look at. You describe the video; the agent writes, renders, inspects and fixes it. ## Install the skills [Section titled “Install the skills”](#install-the-skills) * Claude Code ```text /plugin install framefields ``` * Any other agent ```bash npx skills add gatewai-dev/framefields ``` Works with Codex, Cursor, Gemini CLI, GitHub Copilot, Windsurf, OpenCode, Goose, Cline and 70+ others. Then ask for a video: ```text Set up framefields in this project and render a 3-second title card that says "Hello". ``` The skills are instructions only: no MCP servers, hooks or executables, and they send no data anywhere. Details per agent are on [Install the skills](/docs/agents/install/). ## Why it suits agents [Section titled “Why it suits agents”](#why-it-suits-agents) | | | | ---------------------- | --------------------------------------------------------------------------------------------------------------- | | **Plain code** | A video is a `Composition` in TypeScript. Diffs, reviews, tests and git all work. | | **Deterministic** | `(composition, frame) → pixels`. The same frame renders the same way every time, so checks are repeatable. | | **Checks return data** | `comp.lint()` and `renderVideo({ qa })` return `{ severity, code, message }` objects, not logs to parse. | | **Cheap feedback** | One PNG frame or a contact-sheet grid in seconds, long before a full render. | | **People in the loop** | The live preview saves notes a person pins on the picture to `notes.json` — exact feedback an agent can act on. | | **No creative suite** | One npm package; no browser, no desktop apps, no licenses. | ## The loop [Section titled “The loop”](#the-loop) 1. **Write** `film.ts` exporting `buildFilm()`. 2. **Check** — `tsc --noEmit`, then `comp.lint()`. 3. **Look** — render key frames to PNG and open them. 4. **Look at motion** — render a frame grid. 5. **Show** — start the live preview; read the person’s pinned notes. 6. **Ship** — `renderVideo({ qa: true })`; stop if `qa.passed` is false. Full detail in [The agent workflow](/docs/agents/workflow/). [Install the skills](/docs/agents/install/)Claude Code, Codex, Cursor, Gemini CLI, Copilot and more; project or global; team setup. [The skills](/docs/agents/skills/)What each of the four skills teaches and when it loads. [The agent workflow](/docs/agents/workflow/)Write → check → look → show → ship, with code. [Example prompts](/docs/agents/prompts/)Prompts that work well, from first render to vision effects. [Docs for agents](/docs/agents/llms/)llms.txt, Markdown pages, and an AGENTS.md snippet. # Core concepts > Compositions, the frame clock, layer trees, local timing, layout and coordinates — the model everything else builds on. ## Every frame is a function of its number [Section titled “Every frame is a function of its number”](#every-frame-is-a-function-of-its-number) A composition compiles to a deterministic program: `(composition, frame) → pixels`. There are no timers and no wall clock. Rendering frame 45 always produces the same image, in any order, on any run. This is what makes frames testable and renders parallel. ## Composition [Section titled “Composition”](#composition) ```ts import { Composition } from "framefields"; const comp = new Composition({ width: 1920, height: 1080, fps: 30, // default 60 durationMs: 10_000, // or durationFrames; omitted → computed from media backgroundColor: "#0e0d0c", // default "#000000" antialias3d: true, // 4x MSAA for 3D meshes; false saves GPU memory }); ``` | Method | Does | | ------------------------------------------------- | ------------------------------------------------------------------ | | `add(layer \| layer[])` | adds layers (or arrays, e.g. from `Lighting.studio()`) to the root | | `addAudio(Layer.audio(...))` | adds a soundtrack | | `apply(effect)` | applies an effect to the whole frame | | `addSignal(name, signal)` | registers a named signal layers can bind by name | | `withVision(options)` | runs vision on the composition, returns reactive signals | | `lint()` | document checks, no GPU | | `toSpec()` | the plain-JSON program (validate with `CompositorProgramSchema`) | | `renderFrame` / `renderFrameGrid` / `renderVideo` | output, see [Rendering](/docs/guides/rendering/) | | `computeDuration()` | duration in ms, from options or media | ## Layers [Section titled “Layers”](#layers) Every `Layer.*` builder returns a node you can chain: `.animate()`, `.withEffect()`, `.withMatte()`, `.withVision()`, `.pinToObject()`. Layers nest through `children`. ```ts Layer.box({ width: 720, height: 380, padding: 32, background: "#141721", borderRadius: 24, borderColor: "#262b3d", borderWidth: 1.5, children: [ Layer.flex({ dir: "column", gap: 16, children: [ Layer.text("ENGINE", { fontSize: 16, fontWeight: 700, fill: "#6366f1", letterSpacing: 2 }), Layer.text("WebGPU motion", { fontSize: 48, fontWeight: 700, fill: "#f8fafc" }), ], }), ], }); ``` ## Time [Section titled “Time”](#time) * `startFrame` and `durationFrames` place a layer on its parent’s timeline. * **Keyframes are local**: frame 0 is the layer’s first visible frame. Never add the parent’s `startFrame` to keyframe times. * **Frames are integers.** Keyframe frames, start frames and durations must be whole numbers; `.keys()` rounds for you, other APIs reject decimals. * Keyframes past a layer’s `durationFrames` never play (lint/QA: `animation_truncated`). * Signals are sampled at the **composition** frame, not the layer’s local frame. A beat grid keeps timing musical and readable: ```ts export const FPS = 30; export const BPM = 100; export const BEAT = Math.round((FPS * 60) / BPM); // 18 frames export const beats = (n: number) => Math.round(n * BEAT); ``` ## Layout and position [Section titled “Layout and position”](#layout-and-position) * Containers lay children out with flexbox (Yoga): `Layer.flex({ dir, gap, justify, align, wrap, padding })`. * Setting `x` or `y` without `position: "relative"` makes a layer **absolute** — removed from flex flow, positioned in its parent. * **Animating `x`/`y` overrides layout.** A layer placed at `y: 400` animates with `fromTo("y", 440, 400)`, not `fromTo("y", 40, 0)`. * Borders are `borderColor` + `borderWidth`. A CSS shorthand like `border: "1px solid red"` is ignored. ## Coordinates [Section titled “Coordinates”](#coordinates) * 2D: pixels, origin top-left, y down. * 3D: the camera is calibrated so `z = 0` maps 1:1 onto canvas pixels. y is still down, so “up” is −y. See [3D](/docs/guides/3d/). ## Colors [Section titled “Colors”](#colors) Every color prop takes hex, `rgb()`, `hsl()`, `oklch()`, `oklab()` or a CSS name. Color keyframes blend in OKLab, so red → green passes through yellow. Gradients are CSS strings (`linear-`, `radial-`, `conic-gradient()`) usable as any `background` or shape `fill`. ## Project shape [Section titled “Project shape”](#project-shape) ```plaintext src/theme.ts tokens: size, fps, colors, fonts, beat grid src/scenes/*.ts one function per scene, each returns a full-frame box src/film.ts buildFilm(): fonts, Composition, Layer.sequence(scenes), grade src/render.ts frames | grid | preview | video assets/ fonts, footage, audio, images, LUTs, models — in Git LFS output/ renders, gitignored ``` Media in `assets/` is stored with Git LFS: push it with a normal `git push`, and run `git lfs pull` in any fresh checkout before rendering. See [Assets and Git LFS](/docs/agents/workflow/#assets-and-git-lfs). # Authoring rules > The numbered rules that prevent most broken framefields renders. Read before writing a composition. 1. **Register fonts first.** Call `FontManager.register({ family, source })` for every family before building any text layer. There are no system fonts. Emoji need `FontManager.registerEmojiFont(...)` with a CBDT/CBLC color font. 2. **Keyframes are local.** Frame 0 is the layer’s first frame. Never add the parent’s `startFrame`. 3. **Frames are integers.** Keyframe frames, `startFrame`, `durationFrames` — whole numbers only. 4. **Animating `x`/`y` overrides layout.** Animate to the placed value: placed at `y: 400` → `fromTo("y", 440, 400)`. 5. **`x`/`y` make a layer absolute** unless `position: "relative"` is set. 6. **Borders are `borderColor` + `borderWidth`.** CSS `border` strings are ignored. 7. **Give text a width** when it can wrap; otherwise it may clip. 8. **Every `Layer.audio()` needs `durationFrames`.** 9. **Images are raster.** `.png/.jpg/.webp` → `Layer.image`; `.json/.lottie` → `Layer.lottie`. 10. **Effect strengths are often 0–100**, not 0–1 (`Vignette`, `FilmGrain`, `Blur`). Check the [Effects reference](/docs/reference/effects/). 11. **Grade, then optics, then grain.** `FilmGrain` is the outermost pass. 12. **PBR needs an environment.** `roughness`/`metallic` do nothing without `Layer.environment()` or a `Lighting` rig. 13. **One shadow light**, directional or spot. 14. **Signals sample the composition frame**, not the layer’s local frame. 15. **Vision trails by one frame.** Verify vision with the MP4 or consecutive frames, never with a frame grid. 16. **Resolve assets from `import.meta.url`** so the browser preview can load them. 17. **Assets live in `assets/`, in Git LFS.** `git lfs pull` before rendering in a fresh clone; confirm new media appear in `git lfs ls-files` before committing; never commit `output/`. 18. **Verify cheapest first:** typecheck → `lint()` → `renderFrame` → `renderFrameGrid` → `renderVideo({ qa: true })`. Look at every image you render. 19. **End render scripts with `process.exit()`.** 20. **Tokens live in `theme.ts`.** No magic colors, sizes or timings in scenes. 21. **Ease with intent.** `back.out(1.4–1.7)` entrances, `expo.out`/`spring` decelerations, `power2.in` exits, `linear` only for loops and counters. 22. **Custom shaders output premultiplied alpha** and blend with `srcFactor: "one"`, never `"src-alpha"`. # Install the skills > Install the framefields agent skills in Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot and 70+ other agents — per project, per user, or for a whole team. The skills live in [`plugins/framefields`](https://github.com/gatewai-dev/framefields/tree/main/plugins/framefields) in the framefields repo. They are Markdown instructions only. The engine itself is the `framefields` npm package, which the skills tell your agent to install into your project. ## Install [Section titled “Install”](#install) * Claude Code Inside an interactive `claude` session: ```text /plugin install framefields ``` Or from a shell: ```bash claude plugin install framefields@claude-plugins-official ``` It comes from Anthropic’s official plugin directory, which Claude Code adds for you, and updates automatically. Restart Claude Code or run `/reload-plugins` afterwards. In the desktop app’s Code tab, use the shell form or **+ → Plugins → Add plugin → Framefields**. **Track `main` instead of the directory release:** ```text /plugin marketplace add gatewai-dev/framefields /plugin install framefields@framefields-plugins ``` * Codex ```bash npx skills add gatewai-dev/framefields -a codex ``` * Cursor ```bash npx skills add gatewai-dev/framefields -a cursor ``` * Gemini CLI ```bash npx skills add gatewai-dev/framefields -a gemini-cli ``` * Copilot ```bash npx skills add gatewai-dev/framefields -a github-copilot ``` VS Code / Copilot can also load the portable [`plugin.json`](https://github.com/gatewai-dev/framefields/blob/main/plugins/framefields/plugin.json) through its plugin UI. * Other ```bash npx skills add gatewai-dev/framefields ``` The [`skills`](https://skills.sh) CLI detects the agents on your machine and asks where to install. Agent ids include `windsurf`, `opencode`, `goose`, `cline`, `roo`, `amp`, `kiro-cli` and `hermes-agent`. ## Choose the scope [Section titled “Choose the scope”](#choose-the-scope) | Goal | Command | | --------------------------------------------------- | ----------------------------------------------------------- | | This project only (default) | `npx skills add gatewai-dev/framefields` | | Every project, for your user | `npx skills add gatewai-dev/framefields -g` | | Several agents at once | `npx skills add gatewai-dev/framefields -a codex -a cursor` | | No prompts (CI, scripts, agents installing for you) | add `-y` | | One skill only | add `-s framefields-render` | | See what’s in the package first | `npx skills add gatewai-dev/framefields --list` | | Copy instead of symlink | add `--copy` | ## Set it up for a team [Section titled “Set it up for a team”](#set-it-up-for-a-team) **Claude Code.** Commit this to `.claude/settings.json`; Claude Code offers the plugin to teammates when they trust the folder: .claude/settings.json ```json { "enabledPlugins": { "framefields@claude-plugins-official": true } } ``` Or install with `claude plugin install framefields@claude-plugins-official --scope project`, which writes the same entry. **Other agents.** Install per project (no `-g`) and commit the skill folders the CLI creates (for example `.agents/skills/`), so everyone who clones the repo gets them. ## Keep them current [Section titled “Keep them current”](#keep-them-current) ```bash npx skills update # update installed skills npx skills ls # list what's installed npx skills remove # remove them ``` Claude Code updates plugins from the official directory automatically. The skills’ version follows the `framefields` npm package. ## Install by hand [Section titled “Install by hand”](#install-by-hand) Copy each folder from [`plugins/framefields/skills/`](https://github.com/gatewai-dev/framefields/tree/main/plugins/framefields/skills) into one of: * `.claude/skills/` (Claude Code, project) * `.agents/skills/` (project, most other agents) * `~/.agents/skills/` (your user) ## Check it worked [Section titled “Check it worked”](#check-it-worked) Ask your agent: *“Which framefields skills do you have?”* It should list `framefields`, `framefields-compose`, `framefields-effects` and `framefields-render`. Skills load on their own when a task matches, so you never need to name them. ## Your project still needs the package [Section titled “Your project still needs the package”](#your-project-still-needs-the-package) The skills don’t install the engine. On the first task, the `framefields` skill has your agent add it to your project — Node.js 22+, an ES module `package.json`, and approval for the one install script it needs. You can do it yourself with [Getting started](/docs/getting-started/). # Docs for agents > Machine-readable versions of these docs (llms.txt, full and compact text, per-page Markdown) and an AGENTS.md snippet for your project. These docs are written to be read by agents as much as people. Every page is short, example-first, and available as plain text. ## URLs [Section titled “URLs”](#urls) | URL | Contents | | ---------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | | [`/llms.txt`](/llms.txt) | site-level pointer for agents | | [`/docs/llms.txt`](/docs/llms.txt) | index: summary, requirements, and every page with its Markdown link | | [`/docs/llms-full.txt`](/docs/llms-full.txt) | all pages in one file | | [`/docs/llms-small.txt`](/docs/llms-small.txt) | all pages, compact | | `/docs/.md` | one page as Markdown — swap the trailing `/` for `.md`, e.g. [`/docs/guides/animation.md`](/docs/guides/animation.md) | Each page also has a **Copy as Markdown** button under its title, for pasting into a chat. ## Tell your agent about them [Section titled “Tell your agent about them”](#tell-your-agent-about-them) Add this to your project’s `AGENTS.md` (or `CLAUDE.md`): AGENTS.md ```md ## Video (framefields) This project renders video with the `framefields` npm package. - Docs index: https://framefields.dev/docs/llms.txt — fetch a page's `.md` URL from it when you need details. - Rules to follow: https://framefields.dev/docs/reference/rules.md - Film entry: `src/film.ts` exports `buildFilm()`. Tokens live in `src/theme.ts`. - Input files live in `assets/` and are stored with Git LFS. Run `git lfs pull` before rendering in a fresh checkout; a ~130-byte file starting with `version https://git-lfs` is an un-pulled pointer. Check `git lfs ls-files` lists new media before committing. Renders go to `output/` (gitignored). - Verify cheapest first: `npm run typecheck`, `comp.lint()`, `npm run frames -- `, `npm run grid`, then `npm run render` (QA must pass). - Open every PNG you render before claiming it looks right. - Preview notes from people are in `.framefields/preview-notes/notes.json`. ``` ## Pick the right file [Section titled “Pick the right file”](#pick-the-right-file) * **Starting cold?** `llms-small.txt` fits most context windows and covers everything. * **Working on one area?** Fetch just that page’s `.md` from the index. * **Need effect props?** [`/docs/reference/effects.md`](/docs/reference/effects.md) lists all 52 effects; the skills’ generated catalog has types, ranges and defaults. # Example prompts > Prompts that work well with framefields agents, from a first title card to data stories, 3D product shots and vision effects. Good prompts say **what** (content, length, size), **how it should feel** (pace, palette, type), and **how to check** (which frames to show you). The skills handle the API. ## Getting started [Section titled “Getting started”](#getting-started) ```text Set up framefields in this project and render a 3-second 1080p title card that says "Launch day". Show me frames 0, 30 and 89 before exporting. ``` ```text Add framefields to this repo with a theme.ts, film.ts and render.ts. Use the Inter font in assets/fonts. ``` ```text Set up Git LFS for assets/ (fonts, footage, audio, images, LUTs, models, Lottie), gitignore output/, then add and push the files I just dropped into assets/footage. Confirm they are in git lfs ls-files. ``` ```text This is a fresh clone: pull the LFS assets, check none are still pointers, then render frame 0. ``` ## Motion and type [Section titled “Motion and type”](#motion-and-type) ```text Make a 10-second 1920×1080 intro at 30 fps, 100 BPM. Four scenes on the beat: the headline rises glyph by glyph, three feature cards pop in, a stat counts up, the logo settles. Push transitions between scenes. Render a frame grid of each scene for me. ``` ```text The second card enters too early and feels stiff. Move it one beat later and give it back.out(1.5). ``` ## Data [Section titled “Data”](#data) ```text Turn data/revenue.csv into a 6-second animated bar chart, quarters on x, revenue in $ on y, bars growing in one after another. Use the colors in theme.ts. ``` ## 3D [Section titled “3D”](#3d) ```text Make a product shot: the word "framefields" extruded in yellow, studio lighting with a soft shadow, a slow turn over 4 seconds, light bloom on the edges. Show me the first and last frame. ``` ## Grading and effects [Section titled “Grading and effects”](#grading-and-effects) ```text Give the whole film a warm filmic grade with subtle grain and vignette. Keep it restrained. ``` ```text Key out the green screen in assets/footage/talent.mp4 and put her over the gradient background. ``` ## Vision [Section titled “Vision”](#vision) ```text Put the headline "MOVE" behind the dancer in assets/footage/dancer.mp4. Check it with the exported video, not a frame grid. ``` ```text Reframe assets/footage/interview.mp4 from 16:9 to 9:16, following the speaker. Blur any other faces. ``` ## Sound [Section titled “Sound”](#sound) ```text Add assets/audio/score.mp3 with a 2-second fade in and out, plus a whoosh on every scene change. Target -14 LUFS and fail if QA doesn't pass. ``` ## Review with a person [Section titled “Review with a person”](#review-with-a-person) ```text Start the preview and give me the link. I'll leave notes on the picture — read them and fix each one, then restart the preview so I can tick them off. ``` ## Ship [Section titled “Ship”](#ship) ```text Lint the film, fix any warnings, then export output/final.mp4 at high quality with QA and show me the report. ``` # The skills > What each of the four framefields skills teaches an agent, and the kind of request that loads it. Four skills, each loaded automatically when a request matches its description. They hand off to each other, so an agent starting a project ends up with all four. | Skill | Teaches | Loads on requests like | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | | `framefields` | Checking Node/GPU, writing `package.json` with the right install-script approvals, installing, scaffolding `theme.ts` / `film.ts` / `render.ts`, Git LFS for media, a first verified render | *“Make a video with framefields”*, *“set up framefields in this repo”* | | `framefields-compose` | Compositions and layer trees, flex layout, typography and fonts, keyframes and easing, text animators, scenes on a beat grid, transitions, mattes, 3D looks, charts | *“Add a title sequence”*, *“animate these cards in on the beat”*, *“make the logo spin in 3D”* | | `framefields-effects` | Attaching effects to layers, whole films and regions; grading, grain, blur, LUTs, keying; signals on effect props; on-device vision (tracking, cutouts, pose) | *“Add film grain and a vignette”*, *“put the headline behind the dancer”*, *“blur the faces”* | | `framefields-render` | The verification ladder: lint, single frames, frame grids, pixel tests, live preview and pinned notes, MP4 export with QA, diagnosing blank frames | *“Render a frame grid of the intro”*, *“why is frame 40 black?”*, *“export the final MP4”* | ## What they make an agent do differently [Section titled “What they make an agent do differently”](#what-they-make-an-agent-do-differently) * **Verify cheapest first.** Typecheck and `comp.lint()` before any GPU work; frames and grids before a full render. * **Look at its own output.** Render a PNG, open it, and compare against intent — not assume. * **Keep tokens in one place.** Colors, sizes, fonts and the beat grid live in `theme.ts`. * **Avoid the classic breakages.** Fonts registered first, local integer keyframes, absolute vs flex positioning, `durationFrames` on audio, `process.exit()` after rendering. See [Authoring rules](/docs/reference/rules/). * **Bring a person in before exporting.** Start the preview, share the URL, and act on pinned notes. ## Where they come from [Section titled “Where they come from”](#where-they-come-from) The skills are versioned with the `framefields` package and live in the framefields repo at [`plugins/framefields/skills/`](https://github.com/gatewai-dev/framefields/tree/main/plugins/framefields/skills). `framefields-effects` includes a generated catalog of every effect and prop. If a skill gets something wrong, [open an issue](https://github.com/gatewai-dev/framefields/issues). # The agent workflow > The write → check → look → show → ship loop an agent should follow with framefields, with the code for each step. Every step returns a file or data an agent can inspect. Go in order: each step catches in seconds what the next takes minutes to show. ## 0. Project shape [Section titled “0. Project shape”](#0-project-shape) ```plaintext src/theme.ts tokens: size, fps, colors, fonts, beat grid src/film.ts export async function buildFilm(): Promise src/render.ts frames | grid | preview | video assets/ every input file, tracked with Git LFS fonts/ .ttf .otf footage/ .mp4 .mov .webm audio/ .mp3 .wav images/ .png .jpg .webp luts/ .cube models/ .glb .gltf .obj .fbx lottie/ .json .lottie captions/ .srt (plain text, no LFS needed) output/ renders (gitignored, never committed) .gitattributes LFS rules for assets/ ``` `render.ts` is in [Getting started](/docs/getting-started/#3-scaffold). Keeping `buildFilm()` separate means tests, preview and export all build the same film. Reference assets from the module URL (`path.resolve(HERE, "../assets/…")`) so the [preview](/docs/guides/preview/) can load them too. ## Assets and Git LFS [Section titled “Assets and Git LFS”](#assets-and-git-lfs) Source media goes in `assets/` and is stored with [Git LFS](https://git-lfs.com), so the repo stays small and clones stay fast. Renders go in `output/` and are never committed. ### Set up once per repo [Section titled “Set up once per repo”](#set-up-once-per-repo) ```bash git lfs install # once per machine git lfs track "assets/fonts/**" "assets/footage/**" "assets/audio/**" \ "assets/images/**" "assets/luts/**" "assets/models/**" "assets/lottie/**" echo "output/" >> .gitignore git add .gitattributes .gitignore git commit -m "Track assets with Git LFS" ``` `git lfs track` writes the rules to `.gitattributes` (`assets/footage/** filter=lfs diff=lfs merge=lfs -text`, and so on). Commit that file before adding any media. Ignore `output/`, never `*.mp4` — that would also hide your footage. ### Push assets [Section titled “Push assets”](#push-assets) Add and push as usual; Git uploads the LFS objects with the push. ```bash git add assets/footage/interview.mp4 git lfs ls-files # the new file must be listed — otherwise it is not in LFS git commit -m "Add interview footage" git push ``` If a file was committed before its rule existed, it went into normal Git. Fix it with `git rm --cached `, `git add ` and commit again. ### Pull assets [Section titled “Pull assets”](#pull-assets) A clone with LFS installed downloads assets automatically. Anywhere else — a fresh container, a CI runner, an agent sandbox, or a clone made with `GIT_LFS_SKIP_SMUDGE=1` — fetch them before rendering: ```bash git lfs install git lfs pull # all assets for the current commit git lfs pull --include="assets/fonts/**,assets/footage/intro.mp4" # or just what this render needs ``` Check nothing is still a pointer: in `git lfs ls-files`, `*` means the file is downloaded and `-` means it is still a pointer. Pointers render blank An un-pulled LFS file is a \~130-byte text pointer starting with `version https://git-lfs.github.com/spec/v1`. framefields cannot decode it: fonts fail to register, and videos, images and audio come out blank or throw. If a render shows missing media in a fresh checkout, run `git lfs pull` before changing any code. In GitHub Actions, check out with `lfs: true` (`actions/checkout`) so the runner has the real files. ## 1. Check without the GPU [Section titled “1. Check without the GPU”](#1-check-without-the-gpu) ```bash npx tsc --noEmit -p . ``` ```ts const film = await buildFilm(); for (const d of film.lint()) console.log(`${d.severity} ${d.code}: ${d.message}${d.layerId ? ` [${d.layerId}]` : ""}`); ``` Fix every error and warning before rendering. Codes and fixes: [Lint & QA codes](/docs/reference/qa-codes/). ## 2. Look at key frames [Section titled “2. Look at key frames”](#2-look-at-key-frames) ```bash npm run frames -- 0 30 60 89 ``` Open each PNG in `output/` and compare it to what was asked. A black frame usually means a font that isn’t registered, a layer off canvas, opacity stuck at 0, or an asset that is still a Git LFS pointer (`git lfs pull`). ## 3. Look at motion [Section titled “3. Look at motion”](#3-look-at-motion) ```bash npm run grid -- 0 6 12 18 24 30 36 42 ``` One image of the whole move: easing hitches, text that re-wraps, wrong z-order, missing cuts. Not for vision layers (they need frames in order). ## 4. Show a person [Section titled “4. Show a person”](#4-show-a-person) ```bash npm run preview # run in the background; prints the URL ``` * Give the person the URL, or open it in your own browser pane. * Read the command’s output: page errors and lifecycle lines show up there. * When they say they left notes, read `.framefields/preview-notes/notes.json` and open each open note’s `note-.jpg` before editing. Each note has the frame, the spot (`x`, `y` as fractions) and the text. * Rerun the preview after changes; the open tab reloads at the same moment. Mark notes `"done": true` when addressed. ## 5. Ship with QA [Section titled “5. Ship with QA”](#5-ship-with-qa) ```ts import { formatQaReport } from "framefields"; const result = await film.renderVideo({ outputPath: "output/film.mp4", quality: "high", qa: { targetLufs: -14 } }); console.log(formatQaReport(result.qa!)); process.exit(result.qa!.passed ? 0 : 1); ``` `qa.passed` is false on any error or warning: dropped, black or frozen frames, clipping, loudness off target, plus every lint finding. Treat it as the gate. ## In CI [Section titled “In CI”](#in-ci) Rendering needs a GPU (Metal or Vulkan), so run the full render on a GPU runner. On any runner you can still gate on types and `lint()`: scripts/check.ts ```ts import { CompositorProgramSchema } from "framefields"; import { buildFilm } from "../src/film.js"; const film = await buildFilm(); const bad = film.lint().filter((d) => d.severity !== "info"); const schema = CompositorProgramSchema.safeParse(film.toSpec()); if (bad.length || !schema.success) { for (const d of bad) console.error(`${d.code}: ${d.message}`); process.exit(1); } process.exit(0); ``` ## Habits that keep agents fast [Section titled “Habits that keep agents fast”](#habits-that-keep-agents-fast) * Change one scene at a time and re-render only its frames. * Keep all magic numbers in `theme.ts`; a tweak is then one line. * Name layers with `id` — lint, QA and chart children report by id. * Put asset paths through `import.meta.url` so the preview loads them too. # 3D > Cameras and camera moves, lights and lighting rigs, physically based environments, primitives, extruded text and models. 3D and 2D share one depth buffer. The camera is calibrated so **`z = 0` maps 1:1 onto canvas pixels**: a box at `x: 960, y: 540, z: 0` sits in the centre of a 1920×1080 frame. y points down, so “up” is −y. ## The fast path: a lighting rig [Section titled “The fast path: a lighting rig”](#the-fast-path-a-lighting-rig) ```ts import { Composition, Layer3D, LayerAnimation, Lighting } from "framefields"; const W = 1920, H = 1080; const comp = new Composition({ width: W, height: H, fps: 30, durationMs: 4000, backgroundColor: "#0b0b0c" }); comp.add(Lighting.studio({ target: [W / 2, H / 2, 0] })); // environment + key/rim/fill, soft shadow comp.add( Layer3D.text({ text: "framefields", fontFamily: "Archivo", fontWeight: 800, fontSize: 280, extrusionDepth: 26, fill: "#f5e300", bevelColor: "#6b6300", roughness: 0.35, x: W / 2, y: H / 2, z: 0, }).animate(LayerAnimation.create().keys("rotateY", [[0, -20], [120, 6, "sine.inOut"]])), ); ``` Rigs: `Lighting.studio`, `soft`, `sunset`, `dramatic`. Options: `target`, `distance`, `rotation`, `intensity`, `exposure` (stops), `toneMapping`, `shadows` (default on), `shadowSoftness`, `fog`, `bloom`. A rig returns an array; `comp.add` takes it, or spread it into `children`. ## Camera [Section titled “Camera”](#camera) ```ts import { CameraAnimation, Layer } from "framefields"; comp.add( Layer.camera({ x: 960, y: 540, z: -1000, targetX: 960, targetY: 540, targetZ: 0 }).animate( CameraAnimation.camera() .orbit({ azimuth: { from: -30, to: 30 }, elevation: { from: 15, to: 15 }, radius: { to: 1200 }, start: 0, end: 300, ease: "sine.inOut" }) .rackFocus({ fromDistance: 2000, toDistance: 1200, fromFStop: 1.8, toFStop: 4, start: 20, end: 60 }), ), ); ``` `CameraAnimation.camera()` moves: `orbit`, `dolly`, `truck`, `pedestal`, `pan`, `tilt`, `roll`, `rackFocus`, `handheld({ translationAmplitude, rotationAmplitude })`. Camera `mode` is `"lookAt"` (default), `"free"` or `"orbit"`. ## Lights by hand [Section titled “Lights by hand”](#lights-by-hand) ```ts import { Layer, Light } from "framefields"; comp.add([ Layer.environment("studio", { // presets: studio | soft | sunset | night | neutral exposure: 0.3, toneMapping: "neutral", // default; "agx" filmic, "aces" punchy, "none" rotation: 30, fog: { color: "#0b0b0c", start: 2000, end: 5000 }, bloom: true, // or { intensity, threshold, radius } }), Light.directional({ x: 400, y: -1200, z: -600, targetX: 960, targetY: 540, castShadow: true }), Light.ambient("#ffffff", 0.3), ]); ``` `Light.ambient`, `Light.directional`, `Light.point`, `Light.spot` (also `Layer.ambientLight`, `Layer.directionalLight`, `Layer.pointLight`, `Layer.spotLight`). | Rule | Why | | -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | | Add `Layer.environment(...)` for `roughness` / `metallic` to mean anything | Without it the scene uses classic Blinn-Phong (lint: `pbr_without_environment`) | | One `castShadow` light, directional or spot | Only the first casts (`multiple_shadow_lights`); point/ambient can’t (`shadow_unsupported_light`) | | Tone mapping `"neutral"` keeps brand colors as authored | `"agx"` rolls highlights off filmically | | Bloom only takes light well past white | Use `emissiveIntensity` ≳ 3 to make something glow | | Set `fog.color` to the background | Surfaces fade into it seamlessly | ## Primitives [Section titled “Primitives”](#primitives) ```ts Layer3D.roundedBox({ x: 960, y: 540, z: 0, width: 300, height: 80, depth: 300, radius: 18, taper: 0.9, color: "#ff5a1f", roughness: 0.45 }); // keycap Layer3D.sphere({ x: 700, y: 540, z: 0, radius: 90, color: "#e8e8e8", metallic: 1, roughness: 0.15 }); // chrome Layer3D.cylinder({ x: 1200, y: 540, z: 0, radius: 12, height: 400, color: "#111", emissive: "#4da3ff", emissiveIntensity: 3 }); // neon tube ``` | Builder | Makes | | ------------------------------------------------ | --------------------------------------------------------------------------------- | | `Layer3D.roundedBox / sphere / cylinder` | solid smooth-shaded meshes; equal shapes share GPU buffers, so hundreds are cheap | | `Layer3D.cube({ size, faces })` | a cube whose faces are 2D layers | | `Layer3D.carousel({ radius, items })` | 2D items on a ring | | `Layer3D.prism`, `Layer3D.plane`, `Layer3D.grid` | prisms, flat cards, ground grids | | `Layer3D.text` / `Layer3D.extrudedText` | extruded, bevelled type from your registered fonts | Surface props on meshes: `color`, `tint` (animates as `color`), `roughness`, `metallic`, `emissive`, `emissiveIntensity`, `castShadow`. Flat layers cast shadows only with `castShadow: true`; meshes cast by default. ## Models [Section titled “Models”](#models) ```ts comp.add(Layer.glb("assets/models/character.glb", { x: 960, y: 640, z: 0, scale: 2.5, loop: true })); ``` Loaders: `Layer.glb`, `gltf`, `obj`, `fbx`, `stl`, `ply`, `vox`, `threeDS`, `off` (also on `Layer3D`). glTF brings base color, normal, metallic-roughness, occlusion and emissive maps, and skeletal animation (up to 128 bones). Animate playback with `animationProgress`, `animationTime` or `animationSpeed`. ## Animatable 3D props [Section titled “Animatable 3D props”](#animatable-3d-props) `z`, `rotateX/Y/Z`, `scaleX/Y/Z`, `roughness`, `metallic`, `emissive`, `emissiveIntensity`, `exposure`, `intensity`, plus camera props (`cameraX/Y/Z`, `targetX/Y/Z`, `cameraFov`, `focusDistance`, `fStop`, `orbitRadius`, `orbitAzimuth`, `orbitElevation`). # Animation > Keyframes with LayerAnimation, easing, composing animations, procedural drivers, kinetic typography and 3D transforms on 2D layers. Animations are data. Build one with `LayerAnimation`, attach it with `.animate()`. All frames are **local to the layer** and **integers**. ## Two keyframes: `fromTo` [Section titled “Two keyframes: fromTo”](#two-keyframes-fromto) ```ts import { LayerAnimation } from "framefields"; const entrance = LayerAnimation.create() .fadeIn(0, 20, "power2.out") // opacity 0 → 1, frames 0–20 .fromTo("y", 460, 400, { start: 0, end: 24, ease: "expo.out" }) // to the layout y, not 0 .fromTo("scale", 0.95, 1, { start: 0, end: 30, ease: "back.out(1.4)" }); card.animate(entrance); ``` Shortcuts: `fadeIn(start, end, ease)`, `fadeOut(start, end, ease)`, `slideInX/slideInY(from, to, start, end, ease)`, `rotateX/rotateY/rotateZ`, `translateZ`, `perspective`, `letterSpacing`, `typewriter(start, end)`, `kineticSweep(fromOffset, toOffset, start, end, ease)`. ## Many keyframes: `keys` [Section titled “Many keyframes: keys”](#many-keyframes-keys) Each tuple is `[frame, value, ease?]`; the ease shapes motion **into** that key. Frames are rounded, so beat math can go straight in. ```ts const pop = LayerAnimation.create() .keys("scale", [[0, 0], [10, 1.12, "back.out(1.6)"], [16, 1, "power2.out"]]) .keys("opacity", [[0, 0], [6, 1]]); LayerAnimation.keys("rotation", [[0, -8], [20, 0, "expo.out"]]); // static form ``` Color props (`fill`, `color`, `borderColor`, …) keyframe too, blended in OKLab. ## Easing [Section titled “Easing”](#easing) `.` with optional parameters: `"expo.out"`, `"back.out(1.7)"`, `"power2.inOut"`. | Names | `none` `power1` `power2` `power3` `sine` `circ` `expo` `back` `elastic` `bounce` `spring` `cubic` `hold` | | ---------- | -------------------------------------------------------------------------------------------------------- | | Directions | `in` `out` `inOut` (`"linear"` is an alias for `none`) | | Use | Ease | | ------------------------- | ---------------------------------- | | Snap-overshoot entrances | `back.out(1.4–1.7)` | | Decelerating moves | `expo.out`, `power3.out`, `spring` | | Exits | `power2.in` | | Loops, counters, spinners | `linear` (and nothing else) | ## Composing animations [Section titled “Composing animations”](#composing-animations) `.animate()` **adds** to what a node has. Tracks on the same prop merge into one sorted track; a later key wins on a shared frame. ```ts const rainIn = LayerAnimation.keys("y", [[0, -300], [d, -300], [d + 12, 0, "back.out(1.5)"]]); const press = LayerAnimation.keys("y", [[40, 0], [42, 6, "power2.in"], [47, 0, "power2.out"]]); const glow = LayerAnimation.keys("color", [[42, "#ffd400"], [60, "#2a2b2d", "sine.inOut"]]); key.animate(rainIn).animate(press, glow); // or LayerAnimation.merge(rainIn, press, glow) ``` ## Procedural drivers [Section titled “Procedural drivers”](#procedural-drivers) ```ts LayerAnimation.create() .wiggle("x", { frequency: 1.5, amplitude: 25, seed: 3 }) // seeded noise, deterministic .spring("scale", { damping: 14, stiffness: 190, mass: 1 }); ``` Bind any prop to a signal with `.signal(prop, signal, { multiplier, offset, smoothing })` — see [Audio & signals](/docs/guides/audio-and-signals/). ## Kinetic typography [Section titled “Kinetic typography”](#kinetic-typography) Text animators work per glyph, word or line on the GPU. Add one to the text, then sweep its selection with `kineticSweep`. ```ts import { Layer, LayerAnimation, TextAnimator } from "framefields"; Layer.text("TypeScript in.", { fontFamily: "Archivo", fontWeight: 800, fontSize: 200, fill: "#e6e8e6", animators: [TextAnimator.waveRise({ y: 90, rotationX: 70 })], }).animate(LayerAnimation.create().kineticSweep(-1, 1, 0, 40)); ``` | Preset | Effect | | -------------------------------------------------------------- | -------------------------------- | | `TextAnimator.waveRise({ y, rotationX, opacity, blur, unit })` | glyphs rise and untilt in a wave | | `TextAnimator.wordPop({ scale, y })` | words pop in | | `TextAnimator.blurIn(...)` | glyphs resolve from blur | | `TextAnimator.flip3D(...)` | glyphs flip in around X | | `TextAnimator.colorSweep(...)` | color passes through the text | `unit` is `"character"` (default), `"word"` or `"line"`. For terminal-style typing use `.typewriter(start, end)`. ## 3D transforms on 2D layers [Section titled “3D transforms on 2D layers”](#3d-transforms-on-2d-layers) Any layer can tilt in perspective. ```ts Layer.box({ width: 600, height: 400, background: "#181715", borderRadius: 24, transformStyle: "preserve-3d", perspective: 1200, }).animate( LayerAnimation.create() .fromTo("rotateX", 25, 0, { start: 0, end: 30, ease: "expo.out" }) .fromTo("rotateY", -35, 0, { start: 0, end: 30, ease: "expo.out" }) .fromTo("translateZ", -200, 0, { start: 0, end: 30, ease: "expo.out" }), ); ``` For cameras, lights and meshes, see [3D](/docs/guides/3d/). ## Gotchas [Section titled “Gotchas”](#gotchas) * Keyframes past `durationFrames` never play (`animation_truncated`). * Animating `x`/`y` overrides the layout position. * A track whose keys all hold one value does nothing (`static_track`). # Audio & signals > Soundtracks, audio effects, deterministic procedural SFX, and signals that drive any prop or effect from time, data or sound. ## Soundtrack [Section titled “Soundtrack”](#soundtrack) ```ts import { Effect, Layer } from "framefields"; comp.addAudio( Layer.audio("assets/audio/score.mp3", { durationFrames: totalFrames, // required: without it audio can cut short volume: 0.9, }).apply(Effect.audioFade({ fadeInDuration: 2, fadeOutDuration: 2, fadeOutCurve: "scurve" })), ); ``` `Layer.audio` takes `volume`, `startFrame`, `durationFrames`, `muted`. Video layers carry their own sound (`muted`, `volume`, `trimStartSec`). Audio effects attach with `.apply()`: `Effect.audioFade`, `audioCompressor`, `audioParametricEq`, `audioReverb`, `audioDelay`, `audioNoiseGate`, `stereoPanning` (or the classes `AudioFade`, `AudioCompressor`, …). Props are in the [Effects reference](/docs/reference/effects/). ## Procedural SFX [Section titled “Procedural SFX”](#procedural-sfx) Whooshes, risers, impacts, downshifters and glitches are synthesized on the CPU, seeded, and placed on a bar grid — the same input always gives the same samples. ```ts import fs from "node:fs/promises"; import { encodeStereoWav, Layer, mixSfxInto, renderSfx, softLimit } from "framefields"; const sampleRate = 48_000; const seconds = totalFrames / FPS; const bed: [Float32Array, Float32Array] = [ new Float32Array(Math.ceil(seconds * sampleRate)), new Float32Array(Math.ceil(seconds * sampleRate)), ]; const ctx = { sampleRate, secondsPerBar: 2.4 }; // 100 BPM, 4/4 mixSfxInto(bed, [ renderSfx({ type: "riser", atBar: 0.5, volume: 0.4 }, { ...ctx, seed: 1 }), renderSfx({ type: "impact", atBar: 1, volume: 0.8 }, { ...ctx, seed: 2 }), renderSfx({ type: "whoosh", atBar: 2.75, volume: 0.5 }, { ...ctx, seed: 3 }), ]); softLimit(bed); await fs.writeFile("output/sfx.wav", encodeStereoWav(bed, { sampleRate })); comp.addAudio(Layer.audio("output/sfx.wav", { durationFrames: totalFrames })); ``` `atBar` is where the sound **starts** (fractional bars allowed); a riser that should land on a cut starts `durationSec` earlier. `mixAudioTracks` mixes several tracks headlessly. ## Signals [Section titled “Signals”](#signals) A signal is a value per frame. Bind one to any animatable prop, or pass it to any numeric effect prop. ```ts import { LayerAnimation, Signal } from "framefields"; const pulse = Signal.builder({ type: "sawtooth", frequency: 2, amplitude: 0.08, offset: 1 }); // 120 BPM card.animate( LayerAnimation.create().signal("scale", pulse, { multiplier: 1, offset: 0, smoothing: 2 }), ); ``` The prop becomes `value × multiplier + offset` each frame. Signals are sampled at the **composition** frame. | Constructor | Value | | --------------------------------------------------------------- | ------------------------------------------------------------------------------ | | `Signal.builder({ type, frequency, amplitude, offset })` | `sine`, `triangle`, `sawtooth`, `square`, `bounce`, or `custom` with `fn(ctx)` | | `Signal.sine({ frequency, min, max, phase })` / `Signal.cosine` | periodic wave; `frequency` in Hz | | `Signal.fromArray(values, fps)` | one sample per frame, e.g. a precomputed kick envelope | | `Signal.programmatic((ctx) => number)` | any function of `ctx.frame`, `ctx.time`, `ctx.progress` | | `Signal.frame` / `Signal.time` / `Signal.progress` | the clock itself, as signals (read `.value`) | | `Signal.state(v)` / `Signal.computed(() => …)` | reactive values, e.g. `Signal.computed(() => 55 - Signal.frame.value * 0.1)` | ### Shared signals by name [Section titled “Shared signals by name”](#shared-signals-by-name) ```ts comp.addSignal("kick", Signal.fromArray(kickEnvelope, FPS)); LayerAnimation.create().signal("opacity", "kick", { multiplier: 0.5, offset: 0.5 }); ``` A signal object is registered automatically; a string must match an `addSignal` name. Anything else throws. ### Colors [Section titled “Colors”](#colors) `.colorSignal(prop, signal, options)` drives a color prop; modes are `interpolate`, `hueRotate` and `threshold`. ## Audio-reactive visuals [Section titled “Audio-reactive visuals”](#audio-reactive-visuals) Extract an envelope from audio with the `AudioSignalExtractor` effect (modes, attack/release, beat detection), or precompute one in Node and feed it through `Signal.fromArray`. Precomputing keeps the film a pure function of the frame and is the easiest to test. # Charts & media > Video, image, SVG, Lottie and caption layers, the Media chain, and animated charts built from native nodes. ## Media layers [Section titled “Media layers”](#media-layers) ```ts Layer.video("assets/footage/clip.mp4", { width: 1920, height: 1080, fit: "cover", trimStartSec: 2, muted: true }); Layer.image("assets/images/still.webp", { width: 800, height: 600, fit: "contain" }); // .png .jpg .webp Layer.svg("assets/images/logo.svg", { width: 400, height: 120, fit: "contain" }); Layer.lottie("assets/lottie/badge.json", { width: 400, height: 400, fit: "contain", loop: true }); // .json .lottie Layer.caption("assets/captions/subs.srt", { /* bottom-anchored, auto-wrapping cues */ }); ``` | Builder | Notes | | --------------- | -------------------------------------------------------------------------------------------------------- | | `Layer.video` | decoded with mediabunny/WebCodecs; `fit`, `trimStartSec`, `muted`, `volume` | | `Layer.image` | raster only. Passing `.json`/`.lottie`/video/audio is an error (`unsupported_image_format`) | | `Layer.svg` | vector image | | `Layer.lottie` | Lottie JSON and dotLottie via ThorVG; `fit: "contain" \| "cover" \| "fill"`, `loop`, `speed`, `autoplay` | | `Layer.caption` | SRT-driven subtitles | All of them position, animate, matte, take effects and tilt into 3D like any layer. Asset paths resolve from the working directory; build them from `import.meta.url` if the [preview](/docs/guides/preview/) must load them too. ## Media chains [Section titled “Media chains”](#media-chains) Process a single clip outside the layer tree, e.g. to grade footage or extract a still. ```ts import { ApplyLUT, Media } from "framefields"; const graded = Media.video("assets/footage/raw_log.mp4").apply(new ApplyLUT({ lutUrl: "assets/luts/rec709.cube" })); await graded.renderFrame({ atMs: 1500 }); // PNG Buffer await graded.renderVideo({ outputPath: "output/graded.mp4" }); ``` `Media.video`, `Media.image`, `Media.lottie`, `Media.audio`; each has `apply`, `renderFrame`, `renderFrameGrid`, `renderVideo`. ## Charts [Section titled “Charts”](#charts) `Layer.chart(options, boxProps)` builds a chart out of ordinary boxes, paths and text — d3 does the scales and geometry, framefields draws it with your fonts. ```ts Layer.chart( { type: "bar", // line | area | bar | scatter | candlestick | pie | donut width: 900, height: 480, categories: ["Q1", "Q2", "Q3", "Q4"], series: [ { name: "Revenue", data: [12, 19, 24, 31] }, { name: "Costs", data: [8, 11, 13, 15] }, ], yAxis: { min: 0, ticks: 4, format: "$,.0f" }, // d3-format string or (v) => string fontFamily: "Inter", fontSize: 24, textColor: "#e6e8e6", gridColor: "#2a2b2d", animate: { start: 10, duration: 30, stagger: 6, ease: "power3.out" }, // or false }, { id: "revenue", position: "absolute", x: 120, y: 200 }, ); ``` | Option | Notes | | --------------------------------------------------------------------- | -------------------------------------------------------------------- | | `series: [{ name, data, color, type, curve, pointRadius }]` or `data` | `data` for pie/donut slices and candles | | `stacked` | stack bars/areas | | `xAxis`, `yAxis` | `min`, `max`, `ticks`, `grid`, `format` | | `legend`, `valueLabels` | booleans | | `colors`, `textColor`, `gridColor`, `background`, `cornerRadius` | style | | `innerRadius` | donut hole | | `upColor`, `downColor` | candlesticks | | `animate` | built-in reveal: lines draw on, bars grow, points and slices stagger | Children get stable ids — `-bar--`, `-line-`, `-slice-` — so you can find them in lint output. Keep scatter plots to a few hundred points: each point is a node. # Effects > Attach GPU effects to layers, whole compositions, regions and media chains; order them; drive their props with signals. Effects are typed classes. Construct one with props, attach it, and the GPU runs it each frame. The full list with every prop is in the [Effects reference](/docs/reference/effects/). ```ts import { Blur, Curves, FilmGrain, Vignette } from "framefields"; // or "framefields/effects" ``` ## Where to attach [Section titled “Where to attach”](#where-to-attach) ```ts // 1. One layer (chain as many as you like, in order) Layer.video("assets/footage/plate.mp4", { fit: "cover" }) .withEffect(new Curves({ master: [{ x: 0, y: 0.05 }, { x: 0.5, y: 0.52 }, { x: 1, y: 0.95 }] })) .withEffect(new Vignette({ strength: 40, radius: 0.85, softness: 0.6 })); // 2. The whole frame comp.apply(new FilmGrain({ strength: 12, size: 1.5, animated: true })); // 3. A region of a plate (rectangle, rounded, or a tracked object) comp.section({ source: "assets/footage/interview.mp4", x: 400, y: 200, width: 300, height: 300, borderRadius: 150, effects: [new Blur({ strength: 25, blurType: "Gaussian" })], }); // 4. A clip outside the layer tree import { ApplyLUT, Crop, Media } from "framefields"; const clip = Media.video("assets/footage/raw_log.mp4") .apply(new ApplyLUT({ lutUrl: "assets/luts/cinematic.cube", intensity: 1 })) .apply(new Crop({ cropType: "rectangle", topPercentage: 10, heightPercentage: 80 })); await clip.renderFrame({ atMs: 1500 }); ``` `.withEffects([a, b])` attaches several at once. Lowercase factories work too: `Effect.blur({ strength: 9 })`. ## Value ranges [Section titled “Value ranges”](#value-ranges) Ranges come from each effect’s schema; out-of-range values warn (set `FRAMEFIELDS_STRICT_PROPS=1` to throw). Many “strength” props are **0–100**, not 0–1: | Effect | Prop | Range | Default | Tasteful | | -------------- | ----------------------------------------- | ---------- | -------- | -------- | | `Vignette` | `strength` | 0–100 | 50 | 20–45 | | `FilmGrain` | `strength` | 0–100 | 15 | 8–20 | | `FilmGrain` | `size` | 0.5–4 | 1.5 | 1.2–1.8 | | `Blur` | `strength` | 0–100 | 5 | — | | `ColorBalance` | `cyanRed` / `magentaGreen` / `yellowBlue` | −100–100 | 0 | ±4–12 | | `ColorKey` | `similarity`, `smoothness` | 0–1 | 0.4, 0.1 | — | | `Modulate` | `brightness`, `contrast`, `saturation` | multiplier | 1 | 0.9–1.2 | ## Order [Section titled “Order”](#order) 1. Tonal grading first: `Curves`, `Levels`, `ColorBalance`, `SelectiveColor`, `ApplyLUT`. 2. Then optics: `Blur`, `DepthOfField`, `Vignette`. 3. Grain last, as the outermost pass: `FilmGrain`. For a still export, set `animated: false` on `FilmGrain` so the frame is deterministic across renders of different lengths. ## Drive props with signals [Section titled “Drive props with signals”](#drive-props-with-signals) Any numeric effect prop accepts a signal instead of a number. The uniform updates per frame without rebuilding the pipeline. ```ts import { Blur, Layer, Signal } from "framefields"; const breathe = Signal.sine({ frequency: 0.5, min: 0, max: 20 }); // Hz, sampled at the composition frame Layer.image("assets/images/bg.jpg").withEffect(new Blur({ strength: breathe, blurType: "Gaussian" })); ``` Vision signals plug in the same way, e.g. `new Vignette({ strength: vision.segmentation.subject.coverage })`. See [Vision](/docs/guides/vision/). ## Families [Section titled “Families”](#families) | Family | Effects | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Grading | `Curves`, `Levels`, `ColorBalance`, `SelectiveColor`, `GradientMap`, `ShadowsHighlights`, `ApplyLUT`, `ExtractLUT`, `Modulate`, `ChannelSplitter`, `ChannelMerger` | | Texture & lens | `FilmGrain`, `Vignette`, `Blur`, `MotionBlur`, `UnsharpMask`, `HighPass`, `HalftoneScreen`, `TileOffset`, `TemporalDeflicker` | | Distortion | `DisplacementMap`, `Liquify`, `MeshWarp`, `CornerPin`, `Rotate`, `Flip`, `KenBurns`, `ResizerScaler` | | 3D & light | `Relight3D`, `SSAO`, `PBRGlass`, `DepthOfField`, `RefractionCaustics3D`, `CameraParallax3D` | | Mattes & keying | `ColorKey`, `Crop`, `RefineEdge`, `MaskMath`, `PatchHeal` | | Generators & style | `NoiseGenerator`, `ProceduralVFX`, `Paint`, `LayerStyle` | | Audio | `AudioFade`, `AudioCompressor`, `AudioParametricEq`, `AudioReverb`, `AudioDelay`, `AudioNoiseGate`, `StereoPanning`, `AudioSignalExtractor` | | Vision | `Vision` (prefer `withVision`, see [Vision](/docs/guides/vision/)) | ## Restraint [Section titled “Restraint”](#restraint) Grain and vignette should be felt, not seen. If a viewer notices the effect before the content, halve it. # Layout & text > Boxes, flex layout, shapes, fonts, GPU text, rich text, measurement, colors, gradients and track mattes. ## Boxes and flex [Section titled “Boxes and flex”](#boxes-and-flex) ```ts import { Layer } from "framefields"; Layer.flex({ width: 1920, height: 1080, dir: "column", // "row" | "column" justify: "center", // start | center | end | space-between | space-around align: "center", // start | center | end | stretch gap: 24, padding: 80, wrap: false, background: "#0d0f12", children: [ Layer.box({ padding: 16, background: "#1e232d", borderRadius: 20, children: [/* … */] }), ], }); ``` | Prop | Notes | | -------------------------------------------- | --------------------------------------------------------------------------- | | `width`, `height` | px, `"auto"` (intrinsic), `"fit"` (shrink-wrap) or `"fill"` (parent extent) | | `position: "absolute"` + `x`, `y` | take the layer out of flow; `x`/`y` alone also make it absolute | | `background` | a color or a CSS gradient string | | `borderRadius`, `borderColor`, `borderWidth` | no CSS `border` shorthand | | `overflow: "hidden"` | clip children to the box | | `opacity`, `scale`, `rotation` | static or animated | | `startFrame`, `durationFrames` | when the layer is on screen | | `id` | stable name; used by lint/QA messages and chart children | ## Shapes [Section titled “Shapes”](#shapes) ```ts Layer.shape("circle", { width: 200, height: 200, fill: "#ff5a1f" }); Layer.shape("star", { width: 240, height: 240, fill: "radial-gradient(#fff, #ffb800)" }); Layer.shape("path", { d: "M0 0 L200 0 A100 100 0 0 1 0 0Z", strokeColor: "#fff", strokeWidth: 4, trimEnd: 0 }) .animate(LayerAnimation.create().fromTo("trimEnd", 0, 1, { start: 0, end: 30 })); // draw-on ``` Types: `rect`, `circle`, `ellipse`, `polygon`, `star`, `arrow`, `path`. Animate `trimStart` / `trimEnd` / `trimOffset` for stroke draw-ons. ## Fonts [Section titled “Fonts”](#fonts) There are no system fonts in headless WebGPU. Register every family before building any text layer. ```ts import { FontManager } from "framefields"; await FontManager.register({ family: "Inter", source: "assets/fonts/Inter.ttf" }); await FontManager.register({ family: "Inter", source: "assets/fonts/Inter-Bold.ttf", weight: 700 }); await FontManager.registerEmojiFont("assets/fonts/NotoColorEmoji.ttf"); // emoji need a CBDT/CBLC color font ``` ## Text [Section titled “Text”](#text) ```ts Layer.text("PRECISION MOTION", { fontFamily: "Inter", fontSize: 96, fontWeight: 700, fill: "#ffffff", letterSpacing: 2, lineHeight: 104, width: 1400, // wrap width; set it or text may clip align: "center", // "start" | "center" | "end" (follows text direction) verticalAlign: "middle", // "top" | "middle" | "bottom" inside the box height }); ``` Text is drawn by the Slug algorithm: glyph curves are evaluated per pixel in the shader, so it stays sharp at any size and under 3D camera moves. Arabic, Hebrew and Persian shape right-to-left automatically (`direction: "auto"`; force with `"rtl"` / `"ltr"`). ### Rich text [Section titled “Rich text”](#rich-text) ```ts import { Layer, bold, color, mark, rich } from "framefields"; Layer.text( rich`Scale ${bold("10x", { fill: "#6366f1" })} faster with ${mark("120 FPS", { bg: "#10b98133" })}`, { fontFamily: "Inter", fontSize: 56, fill: "#f8fafc" }, ); ``` Span helpers: `span`, `bold`, `italic`, `color`, `mark`, `size`, `sub`, `sup`. A span takes `fontFamily`, `fontSize`, `fontWeight`, `fill`, `letterSpacing`, `baselineShift`, `opacity`. ### Measure with the real font [Section titled “Measure with the real font”](#measure-with-the-real-font) ```ts const m = FontManager.measure("Hello, world", { fontFamily: "Inter", fontSize: 64 }); m.width; m.height; m.lines; m.caretX(5); m.glyphs[0].x; // px, relative to the text box ``` Use it to place carets, underline a word, or size a pill to its label instead of guessing. ## Color helpers [Section titled “Color helpers”](#color-helpers) Every helper returns a hex string. ```ts import { Color } from "framefields"; Color.oklch(0.72, 0.16, 250); // equal L = equal perceived brightness Color.mix("#ff5a1f", "#1f6bff", 0.5); // perceptual (OKLab) mix Color.ramp(["#3b82f6", "#8b5cf6", "#ec4899"])(0.3); Color.steps(["#0e0d0c", "#ff5a1f"], 5); // 5 swatches Color.lighten(c, 0.1); Color.darken(c, 0.1); Color.alpha("#000", 0.4); Color.contrast(fg, bg); // WCAG ratio: ≥ 4.5 body, ≥ 3 large Color.readableOn(bg); // black or white, whichever reads ``` ## Gradients [Section titled “Gradients”](#gradients) CSS strings, usable in any `background` and any shape `fill`. Linear, radial and conic, up to 8 stops, hard stops allowed. ```ts import { Gradient } from "framefields"; Layer.box({ background: "linear-gradient(135deg, #ff5a1f, #ffb800 60%, #7c3aed)" }); Layer.shape("circle", { fill: Gradient.radial(["#fff", "transparent"]) }); Layer.box({ background: Gradient.conic(["red", "yellow", "lime", "aqua", "blue", "magenta", "red"]) }); ``` A gradient that doesn’t parse draws nothing; `comp.lint()` reports it as `invalid_gradient`. ## Track mattes [Section titled “Track mattes”](#track-mattes) `node.withMatte(matte, mode)` shows `node` only where the matte is opaque (`"alpha"`, default), transparent (`"alpha-inverted"`), bright (`"luma"`) or dark (`"luma-inverted"`). The matte is laid out like a sibling and is never drawn itself. ```ts // Text revealed by a sliding bar Layer.text("LAUNCH", { position: "absolute", x: 560, y: 480, width: 800, height: 120, fontSize: 120 }) .withMatte( Layer.box({ position: "absolute", x: 560, y: 480, width: 800, height: 120, background: "#fff" }) .animate(LayerAnimation.keys("x", [[0, -240], [24, 560, "expo.out"]])), ); // Video inside type Layer.video("assets/footage/surf.mp4", { position: "absolute", x: 0, y: 0, width: 1920, height: 1080, fit: "cover" }) .withMatte(Layer.text("OCEAN", { position: "absolute", x: 0, y: 340, width: 1920, fontSize: 380, align: "center", fill: "#fff" })); ``` # Live preview > Serve a localhost WebGPU player that runs the composition itself, with sound, scrubbing, deep links, scripting and notes pinned on the picture. `startPreview` serves a page that loads your composition’s module and renders it live with WebGPU in the browser — not a video stream. The server only hands over the bundle, your assets, and the soundtrack mixed by the export engine. src/render.ts (preview mode) ```ts import { startPreview } from "framefields"; const session = await startPreview( { entry: new URL("./film.ts", import.meta.url), export: "buildFilm" }, { title: "Launch film" }, ); console.log(`Preview at ${session.url}`); await session.closed; // "idle" | "replaced" | "closed" process.exit(0); ``` `export` names a function returning the `Composition` (may be async) or a `Composition` itself. ## Options [Section titled “Options”](#options) | Option | Default | Notes | | ------------- | ---------------------------------- | ------------------------------------------------------------------------------------------- | | `port` | derived from the project directory | one stable URL per project; a new run replaces the old server and open tabs reload in place | | `host` | `127.0.0.1` | never leaves the machine | | `open` | `false` | open the system browser; otherwise show `session.url` where you are | | `title` | — | page header | | `audio` | `true` | mix and play the soundtrack | | `idleCloseMs` | `5000` | stop after the last tab closes; `false` keeps serving until `close()` | | `root` | project of the entry | extra folders the page may load assets from | | `log` | stderr | where page errors and lifecycle lines go; `false` silences | | `notesDir` | `.framefields/preview-notes` | `false` turns notes off | ## Assets must resolve in the browser [Section titled “Assets must resolve in the browser”](#assets-must-resolve-in-the-browser) The browser runs `film.ts` itself. Build asset paths from the module URL, not `process.cwd()`: ```ts import path from "node:path"; import { fileURLToPath } from "node:url"; const HERE = path.dirname(fileURLToPath(import.meta.url)); const FONTS = path.resolve(HERE, "../assets/fonts"); ``` `node:path`, `fileURLToPath` and synchronous `fs` reads work through shims; writes are skipped. ## For agents [Section titled “For agents”](#for-agents) * Run the preview as a **background** command and read its output: it reports when the page opened, each distinct page error (build failure, a frame that threw, missing font, no WebGPU), and why it stopped. * After editing, run it again — same URL, and the open tab reloads at the same moment. * Deep link: `#t=12.5` (seconds) or `#frame=750`. * Script it: `window.framefieldsPreview` has `seek(seconds)`, `seekFrame(n)`, `play()`, `pause()`, `notes()`, `state()`. `seek` resolves once the frame is drawn — screenshot after it. `` marks the first drawn frame. ## Notes pinned on the picture [Section titled “Notes pinned on the picture”](#notes-pinned-on-the-picture) Paused, a person clicks a spot (or drags an area) and writes what should change. Each note is printed in the preview output and saved: ```plaintext .framefields/preview-notes/ notes.json [{ id, frame, time, x, y, w?, h?, text, done }] note-.jpg the frame with the spot marked ``` To act on notes: read `notes.json`, open each open note’s picture, find the layer from the moment and the spot, edit, rerun the preview. Set `"done": true` yourself or let the person tick them off. ## Playback [Section titled “Playback”](#playback) Play runs in real time on the soundtrack’s clock and skips frames a slow device can’t draw; the page shows the frame rate it reaches. Paused, it draws the exact frame. Requires WebGPU (current Chrome, Edge or Safari). # Rendering > Lint, single frames, contact-sheet grids, pixel tests and MP4 export with built-in QA — in the order that wastes the least time. ## The ladder [Section titled “The ladder”](#the-ladder) | Step | Call | Cost | | ---- | --------------------------------------------------------------------------------- | ------------------- | | 1 | `tsc --noEmit`, `comp.lint()`, `CompositorProgramSchema.safeParse(comp.toSpec())` | no GPU, instant | | 2 | `comp.renderFrame({ frame })` | one frame | | 3 | `comp.renderFrameGrid({ frames })` | a handful of frames | | 4 | Vitest pixel probes | per test | | 5 | `comp.renderVideo({ outputPath, qa: true })` | the whole film | ## 1. Check without rendering [Section titled “1. Check without rendering”](#1-check-without-rendering) ```ts import { CompositorProgramSchema } from "framefields"; const parsed = CompositorProgramSchema.safeParse(comp.toSpec()); if (!parsed.success) throw new Error(JSON.stringify(parsed.error.format())); for (const d of comp.lint()) console.log(`${d.severity} ${d.code}: ${d.message}${d.layerId ? ` (${d.layerId})` : ""}`); ``` `lint()` finds what renders without error but not as intended: unparseable colors and gradients, low-contrast text, off-canvas layers, truncated animations, PBR props without an environment. Codes are in [QA codes](/docs/reference/qa-codes/). ## 2. One frame [Section titled “2. One frame”](#2-one-frame) ```ts import fs from "node:fs/promises"; import { HeadlessMediaRenderer } from "framefields"; const renderer = new HeadlessMediaRenderer(); // reuse across frames const png = await comp.renderFrame({ frame: 45, renderer }); // PNG Buffer await fs.writeFile("output/f0045.png", png); ``` Render the key moments — entrance, peak, exit — and look at them. ## 3. Contact sheet [Section titled “3. Contact sheet”](#3-contact-sheet) ```ts const grid = await comp.renderFrameGrid({ frames: [0, 6, 12, 18, 24, 30, 36, 42], // or startFrame / endFrame / stepFrames, or timesMs columns: 4, cellWidth: 480, // 120–960, default 320 showLabels: true, // frame + time badge per cell (default) outputPath: "output/intro-grid.png", // optional: also write it }); ``` A grid shows easing hitches, wrap jumps, wrong z-order and missing cuts in one image. Default cap is 64 frames (`maxFrames`, hard max 120). Not for vision layers — see [Vision](/docs/guides/vision/#timing-the-one-frame-delay). ## 4. Pixel tests [Section titled “4. Pixel tests”](#4-pixel-tests) src/film.test.ts ```ts import { describe, expect, it } from "vitest"; import { HeadlessMediaRenderer } from "framefields"; import { buildFilm } from "./film.js"; describe("hero", () => { it("draws the title by frame 30", async () => { const comp = await buildFilm(); const png = await comp.renderFrame({ frame: 30, renderer: new HeadlessMediaRenderer() }); expect(png.length).toBeGreaterThan(1000); // decode with skia-canvas and sample pixels at known coordinates }); }); ``` ## 5. Export [Section titled “5. Export”](#5-export) ```ts import { formatQaReport } from "framefields"; const result = await comp.renderVideo({ outputPath: "output/film.mp4", quality: "high", // "very_low" | "low" | "high" | "very_high" (default) codec: "h264", // default, MP4; "vp9" | "vp8" → WebM concurrency: 4, qa: { targetLufs: -14 }, // or true }); console.log(formatQaReport(result.qa!)); if (!result.qa!.passed) process.exitCode = 1; process.exit(); ``` | Option | Notes | | --------------------- | ------------------------------------------------------------------------------------ | | `outputPath` | written there; `result.filePath` is the final path | | `quality` | bitrate preset | | `codec`, `audioCodec` | `h264` → MP4 + AAC; `vp9`/`vp8` → WebM + Opus; `aac`/`mp3`/`opus` alone → audio-only | | `concurrency` | frames rendered in parallel | | `ringCapacity` | GPU→encoder staging depth, 2–8 (or `FRAMEFIELDS_RING_CAPACITY`) | | `qa` | measure frames and audio as they encode; returns `result.qa` | | `renderer` | share a `HeadlessMediaRenderer` | QA thresholds: `minBlackSec` (0.5), `minFrozenSec` (2), `minSilenceSec` (2), `silenceDbfs` (−60), `targetLufs`, `lufsTolerance` (1). `qa.passed` is false on any error or warning. `qa.audio.integratedLufs` is BS.1770 loudness (streaming normalizes to about −14). Always exit Native GPU and encoder threads keep Node alive. End every render script with `process.exit()`. ## Inspect a file from elsewhere [Section titled “Inspect a file from elsewhere”](#inspect-a-file-from-elsewhere) ```bash ffprobe -v error -show_entries stream=codec_name,width,height,duration,r_frame_rate output/film.mp4 ffmpeg -i output/film.mp4 -vf "blackdetect=d=0.5:pix_th=0.10" -f null - ffmpeg -i output/film.mp4 -af volumedetect -f null - ``` ## Production rendering [Section titled “Production rendering”](#production-rendering) A worker uses roughly 200–400 MB of RAM (vs 2–4 GB for headless Chromium), which suits GPU instances and job queues. The framefields repo ships a `Dockerfile.renderer` for a renderer service. # Scenes & transitions > Split a film into scene functions on a beat grid and join them with Layer.sequence and Transition. ## One function per scene [Section titled “One function per scene”](#one-function-per-scene) Each scene returns a full-frame box. Children key from the scene’s own frame 0. src/scenes/hero.ts ```ts import { Layer, LayerAnimation } from "framefields"; import { H, PAPER, SANS, W } from "../theme.js"; export function hero() { return Layer.box({ id: "hero", position: "absolute", x: 0, y: 0, width: W, height: H, background: "#000", children: [ Layer.text("Every frame is a function of its number.", { position: "absolute", x: 140, y: 420, width: W - 280, fontFamily: SANS, fontSize: 120, fontWeight: 800, fill: PAPER, }).animate(LayerAnimation.create().fadeIn(0, 18)), ], }); } ``` ## Join them: `Layer.sequence` [Section titled “Join them: Layer.sequence”](#join-them-layersequence) `Layer.sequence` places scenes end to end, overlaps each pair by its transition, and sets every scene’s `startFrame` / `durationFrames` for you. src/film.ts ```ts import { Composition, Layer, Transition } from "framefields"; import { beats, FPS, H, W } from "./theme.js"; import { features, hero, outro, proof } from "./scenes/index.js"; const film = Layer.sequence( [ { scene: hero(), duration: beats(8) }, { scene: features(), duration: beats(12), transition: Transition.push("left", 18) }, { scene: proof(), duration: beats(8), transition: Transition.wipe("right", 16) }, { scene: outro(), duration: beats(6), transition: Transition.iris(20) }, ], { width: W, height: H }, ); const comp = new Composition({ width: W, height: H, fps: FPS, durationMs: (film.totalFrames / FPS) * 1000, // the sequence knows its length }); comp.add(film); ``` `duration` includes the transition. A missing `transition` is a cut. ## Transitions [Section titled “Transitions”](#transitions) | Call | Look | Defaults | | --------------------------------------- | --------------------------------- | -------------------------- | | `Transition.cut()` | hard cut | — | | `Transition.crossfade(frames, ease)` | new scene fades in over the old | 15, `sine.inOut` | | `Transition.fadeThrough(frames, ease)` | dip to background and back | 20, `sine.inOut` | | `Transition.push(dir, frames, ease)` | new pushes old out | `left`, 18, `power3.inOut` | | `Transition.slide(dir, frames, ease)` | new slides over old, which dims | `left`, 18, `power3.out` | | `Transition.zoom(frames, ease)` | old grows and fades, new settles | 18, `power2.inOut` | | `Transition.whip(dir, frames)` | fast motion-blurred push | `left`, 10 | | `Transition.wipe(dir, frames, ease)` | hard edge sweeps across (matte) | `left`, 18, `power2.inOut` | | `Transition.iris(frames, { at, ease })` | circle opens from a point (matte) | 20, centre | `dir` is `"left" | "right" | "up" | "down"`. `at` is `[x, y]` as fractions of the frame. ## Carry elements across cuts [Section titled “Carry elements across cuts”](#carry-elements-across-cuts) A badge, card or cursor that crosses a scene boundary with continuous position and velocity hides the cut. Put the carrier outside the sequence, on the composition, with its own keyframes spanning both scenes. # Vision > On-device detection, segmentation, pose and mattes as reactive signals — pin layers to subjects, cut them out, reframe, blur, and analyze clips. framefields runs ONNX vision models on the rendered frames and exposes the results as signals that layers and effects consume like any other. All models are Apache-2.0, pinned to an immutable revision, verified by SHA-256, and downloaded lazily on first use. ## Turn it on [Section titled “Turn it on”](#turn-it-on) ```ts // Whole composition → a reactive bundle const vision = comp.withVision({ enableDetection: true, // default enableSegmentation: true, enablePose: true, variant: "s", // "t" | "s" | "m" confidence: 0.35, // default 0.3 classes: ["person"], // COCO filter }); // One layer → render it through a mode Layer.video("assets/footage/dancer.mp4").withVision({ mode: "matte", enableSegmentation: true }); ``` | Option | Model | Gives you | | -------------------- | -------------------------- | ------------------------------------------------------------------- | | `enableDetection` | RTMDet-Ins | COCO-80 boxes, tracked over time → `vision.objects` | | `enableSegmentation` | RTMDet-Ins (same pass) | soft per-instance masks → `vision.masks`, `vision.segmentation` | | `enablePose` | RTMO | 17 COCO keypoints per person → `track.pose`, `vision.poseLandmarks` | | `enableMatte` | MediaPipe Selfie Segmenter | fast person alpha → `vision.segmentation.matte` | Layer modes: `passthrough`, `mask`, `matte`, `crop`, `skeleton`, `boxes`, `tracking`. For cutouts, `matteSource: "instance"` (default; any class, merges attached parts like a dress or instrument) or `"selfie"` (people filling the frame only, fastest). ## Pin, drive, cut out [Section titled “Pin, drive, cut out”](#pin-drive-cut-out) ```ts // Pin a label to the main subject Layer.text("SUBJECT 01", { fontSize: 40, fill: "#f8fafc" }).pinToObject(vision.objects.primary, { anchor: "topCenter", offsetY: -48, smoothFrames: 5, hideWhenLost: true, }); // Drive an effect from mask coverage Layer.box({ width: 1920, height: 1080, background: "#000" }) .withEffect(new Vignette({ strength: vision.segmentation.subject.coverage, radius: 0.9 })); // Editorial one-liners comp.addSubjectSandwich({ source: "assets/footage/dancer.mp4", behind: [headline], feather: 4, fit: "cover" }); // text behind subject comp.addSmartFraming({ source: "assets/footage/action.mp4", target: vision.objects.primary, targetAspect: 9 / 16, damping: 0.15 }); comp.addSubjectOutline(vision.segmentation.subject, { source: "assets/footage/character.mp4", color: "#ff5a1f", width: 6, blur: 12 }); layer.blurRegion(vision.objects.byCategory("person"), { strength: 30 }); // anonymize ``` Anchors: `center`, `topCenter`, `bottomCenter`, `topLeft`, `topRight`, `bottomLeft`, `bottomRight`, `centerLeft`, `centerRight`. `pinToLandmark(coord, { offsetX, offsetY })` pins to a pose keypoint. ## Signals you get [Section titled “Signals you get”](#signals-you-get) | Group | Highlights | | ---------------------- | ---------------------------------------------------------------------------- | | `objects` | `primary`, `get(trackId)`, `byCategory(cat, rank)`, `count`, `hasCategory` | | `objects.*.bounds` | `x`, `y`, `width`, `height`, `screenX/Y/Width/Height`, `area`, `aspectRatio` | | `objects.*.kinematics` | `vx`, `vy`, `speed`, `acceleration`, `headingDeg` | | `objects.*.pose` | 17 keypoints, `hasPose`, `wristSpeed`, `handRaised`, `bodyTiltAngle` | | `masks` | `subject`, `get(trackId)`, `count`; per mask `area`, `coverage`, `solidity` | | `segmentation` | `subject`, `humanSilhouette`, `instanceMasks`, `matte.coverage` | | `classes` | per class `count`, `maxConfidence`, `present` | `vision.summary(frame)` returns a deterministic, serializable snapshot — safe inside a frame hook. ## Analyze before authoring [Section titled “Analyze before authoring”](#analyze-before-authoring) ```ts const report = await comp.analyzeVisionSequence("assets/footage/street.mp4", { tasks: ["detect", "pose"], categories: ["person"] }); report.tracks; // [{ trackId, category, frames: [start, end], centerPath, … }] ``` A zod-serializable report of what is in the clip, without ffmpeg — use it to decide where to place things. ## Timing: the one-frame delay [Section titled “Timing: the one-frame delay”](#timing-the-one-frame-delay) Vision reads each layer’s **previous** rendered frame. Frame 0 has no results, and masks trail the plate by one frame (invisible at speed). * `renderVideo` renders in order and handles this. * For stills, render at least two **consecutive** frames with the same `HeadlessMediaRenderer` and keep the last. * **Don’t verify vision with `renderFrameGrid`**: jumping between frames cuts out pixels from another frame and shows a ghosted double. ## Choosing settings [Section titled “Choosing settings”](#choosing-settings) | Task (CPU, per \~2K frame) | `t` | `s` | `m` | | --------------------------- | ---------- | ---------- | ---------- | | detect + segment (one pass) | 150–300 ms | 200–340 ms | 420–700 ms | | pose | \~50 ms | \~120 ms | \~280 ms | | matte | \~20 ms | \~20 ms | \~20 ms | * Start with `"s"`; use `"m"` for fast motion or busy backgrounds. * Raise `confidence` to \~0.5 on stylized footage; restrict `classes` to stop the subject switching. * Garment drops out of a cutout → `variant: "m"` or `keyBackground: true`. Halo on dark backgrounds → lower `featherRadius` or raise `maskThreshold`. ## Models offline [Section titled “Models offline”](#models-offline) Models cache in `$FRAMEFIELDS_MODELS_DIR` (default `~/.cache/framefields/models`). For CI or air-gapped renders, preload once or point `FRAMEFIELDS_MODELS_BASE_URL` at a mirror. ```ts import { VisionRunner } from "@framefields/vision"; const runner = VisionRunner.create({ variant: "s" }); // zero I/O until used await runner.preload(["detect", "pose"]); // downloads + verifies SHA-256 (~380 MB for all models) runner.close(); ``` # API index > Every public entry point of the framefields package on one page — what it is and where it is documented. Everything imports from `"framefields"`. Subpaths exist for smaller imports: `framefields/effects`, `framefields/audio`, `framefields/signals`, `framefields/renderer`, `framefields/fonts`, `framefields/preview`. Vision runs standalone from `@framefields/vision` (and `@framefields/vision/web` in browsers). ## Composition [Section titled “Composition”](#composition) `new Composition({ width, height, fps = 60, durationMs | durationFrames, backgroundColor = "#000000", fonts, antialias3d = true })` | Member | Returns | Guide | | ----------------------------------------------------------------- | ---------------------------------------- | ----------------------------------------------------------------- | | `add(layer \| layer[], …)` | `this` | [Core concepts](/docs/core-concepts/) | | `addAudio(audioLayer)` | `this` | [Audio](/docs/guides/audio-and-signals/) | | `addSignal(name, signal)` | `this` | [Signals](/docs/guides/audio-and-signals/#shared-signals-by-name) | | `apply(effect)` | `this` | [Effects](/docs/guides/effects/) | | `section({ source, x, y, width, height, borderRadius, effects })` | node | [Effects](/docs/guides/effects/#where-to-attach) | | `withVision(options)` | vision bundle | [Vision](/docs/guides/vision/) | | `addSubjectSandwich` / `addSmartFraming` / `addSubjectOutline` | `this` | [Vision](/docs/guides/vision/#pin-drive-cut-out) | | `analyzeVisionSequence(src, { tasks, categories })` | report | [Vision](/docs/guides/vision/#analyze-before-authoring) | | `registerFont(…)` / `registerEmojiFont(src)` | `this` | [Text](/docs/guides/layout-and-text/#fonts) | | `computeDuration()` | ms | — | | `lint()` | `{ severity, code, message, layerId }[]` | [QA codes](/docs/reference/qa-codes/) | | `toSpec()` | program JSON | [Rendering](/docs/guides/rendering/#1-check-without-rendering) | | `renderFrame({ frame, renderer })` | PNG `Buffer` | [Rendering](/docs/guides/rendering/) | | `renderFrameGrid(options)` | PNG `Buffer` | [Rendering](/docs/guides/rendering/#3-contact-sheet) | | `renderVideo(options)` | `{ filePath, cleanup, qa? }` | [Rendering](/docs/guides/rendering/#5-export) | | `onRequestFrame(ctx => …)` | `this` | per-frame hook | ## Layer builders [Section titled “Layer builders”](#layer-builders) | Builder | Kind | | --------------------------------------------------------------------------------------------- | --------------------------------------------------------- | | `Layer.box(opts)`, `Layer.flex(opts)` | containers | | `Layer.text(content \| spans, opts)` | GPU text | | `Layer.shape(type, opts)` | `rect` `circle` `ellipse` `polygon` `star` `arrow` `path` | | `Layer.video` `image` `svg` `lottie` `audio` `caption` `(src, opts)` | media | | `Layer.chart(options, boxProps)` | charts | | `Layer.sequence(entries, opts)` | scenes + transitions; has `totalFrames` | | `Layer.section(source, target?, opts)`, `Layer.trackedRegion`, `Layer.blurTrackedRegion` | regions | | `Layer.camera(opts)` | 3D camera | | `Layer.environment(preset, opts)` | PBR world | | `Layer.light`, `ambientLight`, `directionalLight`, `pointLight`, `spotLight` | lights | | `Layer.model(opts)`, `glb` `gltf` `obj` `fbx` `stl` `ply` `vox` `threeDS` `off` `(src, opts)` | models | | `Layer.cube` `carousel3d` `plane3d` `prism3d` | 3D layout (also on `Layer3D`) | ## Node methods [Section titled “Node methods”](#node-methods) Every builder’s result has: | Method | Does | | ---------------------------------------------------------------------------------------- | -------------------------------------------- | | `.animate(...animations)` | add keyframes/drivers (merges with existing) | | `.withEffect(fx)` / `.withEffects([...])` / `.apply(fx)` | effects | | `.withMatte(layer, mode)` / `.withMask(layer, mode)` | track mattes | | `.withCrop(crop)` | crop | | `.withVision(opts)` | per-layer vision mode | | `.pinToObject(track, opts)` / `.pinToLandmark(coord, opts)` | follow vision | | `.blurRegion(track, opts)` | blur a tracked object | | `.relight(opts)`, `.withDeflicker(opts)`, `.withPbrGlass(opts)`, `.deformWithAudio(cfg)` | specialised passes | ## 3D [Section titled “3D”](#3d) | Export | Members | | -------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `Layer3D` | `roundedBox` `sphere` `cylinder` `cube` `carousel` `prism` `plane` `grid` `text` `extrudedText` `model` + model loaders | | `Light` | `ambient(color, intensity)` `directional` `point` `spot` `environment` | | `Lighting` | `studio` `soft` `sunset` `dramatic` `(rigOptions)` → array of layers | | `CameraAnimation.camera()` | `orbit` `dolly` `truck` `pedestal` `pan` `tilt` `roll` `rackFocus` `handheld` | ## Animation [Section titled “Animation”](#animation) | Export | Members | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `LayerAnimation` | `create()`, `fromTo`, `keys`, `fadeIn`, `fadeOut`, `slideInX`, `slideInY`, `rotateX/Y/Z`, `translateZ`, `perspective`, `letterSpacing`, `typewriter`, `kineticSweep`, `wiggle`, `spring`, `signal`, `colorSignal`, `followPath`, `merge` | | `TextAnimator` | `waveRise` `wordPop` `blurIn` `flip3D` `colorSweep` | | `Transition` | `cut` `crossfade` `fadeThrough` `push` `slide` `zoom` `whip` `wipe` `iris` | ## Signals [Section titled “Signals”](#signals) `Signal.builder`, `Signal.sine`, `Signal.cosine`, `Signal.fromArray`, `Signal.programmatic`, `Signal.state`, `Signal.computed`, `Signal.frame`, `Signal.time`, `Signal.progress`; also `sineSignal`, `cosineSignal`, `computed`. ## Text, color, fonts [Section titled “Text, color, fonts”](#text-color-fonts) | Export | Members | | ------------- | ----------------------------------------------------------------------------------------------------------------------------- | | `FontManager` | `register({ family, source, weight, style })`, `registerEmojiFont`, `measure(text, opts)`, `get`, `has`, `getAll`, `clear` | | rich text | `rich` tagged template, `span`, `bold`, `italic`, `color`, `mark`, `size`, `sub`, `sup`, `parseMarkup` | | `Color` | `parse` `hex` `rgb` `hsl` `oklch` `mix` `ramp` `steps` `lighten` `darken` `saturate` `rotate` `alpha` `contrast` `readableOn` | | `Gradient` | `linear(stops, angle)` `radial(stops, { at, circle, radius })` `conic(stops)` | ## Audio [Section titled “Audio”](#audio) `renderSfx(trigger, ctx)`, `mixSfxInto(bed, sfx[])`, `softLimit(bed, knee?, ceiling?)`, `encodeStereoWav(channels, { sampleRate })`, `mixAudioTracks(...)`. ## Rendering & media [Section titled “Rendering & media”](#rendering--media) | Export | Notes | | -------------------------------------- | -------------------------------------------------------------------------------- | | `HeadlessMediaRenderer` | share one across `renderFrame` calls | | `Media.video` `image` `lottie` `audio` | single-clip chains with `apply`, `renderFrame`, `renderFrameGrid`, `renderVideo` | | `startPreview(entry, options)` | `{ url, close(), closed }` — [Preview](/docs/guides/preview/) | | `formatQaReport(qa)` | human-readable QA | | `CompositorProgramSchema` | zod schema of `toSpec()` | ## Effects [Section titled “Effects”](#effects) 52 classes, e.g. `Blur`, `Curves`, `FilmGrain`, `Vignette`, `ColorKey`, `ApplyLUT`, plus `Effect.(props)` factories. See the [Effects reference](/docs/reference/effects/). # Effects reference > All 52 effect classes grouped by family, with every prop name. Import from "framefields" or "framefields/effects". ```ts import { Blur, Curves, FilmGrain } from "framefields"; // same classes: import { … } from "framefields/effects"; // factory form: Effect.blur({ strength: 9 }) ``` Attach with `layer.withEffect(fx)`, `layer.withEffects([...])`, `comp.apply(fx)`, `comp.section({ effects })` or `Media.apply(fx)` — see [Effects](/docs/guides/effects/). Every prop is an `EffectProp`: a plain value, a signal, or a function hook. Out-of-range values warn; `FRAMEFIELDS_STRICT_PROPS=1` makes them throw. Types, ranges and defaults for every prop are in the generated [effect catalog](https://github.com/gatewai-dev/framefields/blob/main/plugins/framefields/skills/framefields-effects/references/effects-catalog.md). ## Grading [Section titled “Grading”](#grading) | Effect | Props | | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | `Curves` | `curveType`, `master`, `red`, `green`, `blue`, `hueVsHue`, `hueVsSat`, `lumVsSat`, `satVsSat` | | `Levels` | `master`, `red`, `green`, `blue` | | `ColorBalance` | `shadows`, `midtones`, `highlights`, `preserveLuminosity` | | `SelectiveColor` | `method`, `reds`, `yellows`, `greens`, `cyans`, `blues`, `magentas`, `whites`, `neutrals`, `blacks` | | `GradientMap` | `stops`, `smooth`, `dither`, `opacity` | | `ShadowsHighlights` | `shadowAmount`, `shadowTonalWidth`, `shadowRadius`, `highlightAmount`, `highlightTonalWidth`, `highlightRadius`, `colorCorrection`, `midtoneContrast` | | `ApplyLUT` | `lutUrl`, `intensity` | | `ExtractLUT` | `strategy`, `samplePoints` | | `Modulate` | `hue`, `brightness`, `contrast`, `exposure`, `saturation`, `sepia` | | `ChannelSplitter` | `colorSpace` | | `ChannelMerger` | `colorSpace`, `defaultChannel4` | ## Texture & lens [Section titled “Texture & lens”](#texture--lens) | Effect | Props | | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `FilmGrain` | `strength`, `size`, `monochrome`, `animated`, `speed`, `shadows`, `midtones`, `highlights` | | `Vignette` | `strength`, `radius`, `softness`, `roundness`, `centerX`, `centerY` | | `Blur` | `blurType`, `strength`, `angle`, `sigmaColor`, `centerX`, `centerY`, `partialBlur`, `radius`, `radiusY`, `shape`, `track` | | `MotionBlur` | `shutterAngle`, `samples`, `maxVelocity`, `shutterCurve`, `depthAware` | | `UnsharpMask` | `amount`, `radius`, `threshold` | | `HighPass` | `radius`, `contrastBoost`, `monochrome` | | `HalftoneScreen` | `mode`, `dotShape`, `frequency`, `angle`, `contrast`, `dotColor`, `paperColor`, `smooth`, `invert`, `cyanAngle`, `magentaAngle`, `yellowAngle`, `blackAngle`, `opacity` | | `TileOffset` | `offsetX`, `offsetY`, `wrap`, `edgeMode` | | `TemporalDeflicker` | `blendWeight`, `disocclusionThreshold`, `maxMotionPixels`, `scale`, `windowSize` | ## Distortion & framing [Section titled “Distortion & framing”](#distortion--framing) | Effect | Props | | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `DisplacementMap` | `strengthX`, `strengthY`, `xChannel`, `yChannel`, `wrapMode` | | `Liquify` | `operations` | | `MeshWarp` | `cols`, `rows`, `points` | | `CornerPin` | `points` | | `Rotate` | `angle`, `fit`, `scale` | | `Flip` | `horizontal`, `vertical`, `diagonal`, `mode`, `symmetry` | | `KenBurns` | `keyframes`, `motionBlurSize`, `movementStyle`, `aspectRatio` | | `ResizerScaler` | `aspectRatioPreset`, `resolutionPreset`, `targetWidth`, `targetHeight`, `fitMode`, `zoom`, `offsetX`, `offsetY`, `backgroundMode`, `backgroundColor`, `backgroundColor2`, `blurRadius`, `backgroundBrightness`, `anchorX`, `anchorY` | | `Crop` | `cropType`, `leftPercentage`, `topPercentage`, `widthPercentage`, `heightPercentage`, `roundness`, `pathPoints`, `mode` | ## 3D & light [Section titled “3D & light”](#3d--light) | Effect | Props | | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Relight3D` | `lightType`, `intensity`, `lightPosX`, `lightPosY`, `lightPosZ`, `lightRadius`, `spotConeAngle`, `specularRoughness`, `specularStrength`, `metallic`, `ambientIntensity`, `volumetricDensity`, `depthScale`, `depthInvert`, `lightColor`, `ambientColor`, `normalMap` | | `SSAO` | `radius`, `bias`, `intensity`, `kernelSamples`, `blurRadius` | | `PBRGlass` | `ior`, `roughness`, `transmission`, `dispersion`, `fresnelPower`, `tintColor` | | `DepthOfField` | `focusDistance`, `focalLength`, `aperture`, `maxBlur`, `quality` | | `RefractionCaustics3D` | `ior`, `dispersion`, `refractionScale`, `causticBrightness`, `causticScale`, `fluidRipples`, `rippleSpeed`, `roughness`, `specularIntensity`, `lightPosX`, `lightPosY`, `tintColor`, `tintStrength`, `depthInvert`, `depthScale` | | `CameraParallax3D` | `motionPreset`, `panX`, `panY`, `dollyZ`, `fov`, `parallaxAmount`, `dofAperture`, `focusPlane`, `edgeDilation`, `depthInvert` | ## Mattes & keying [Section titled “Mattes & keying”](#mattes--keying) | Effect | Props | | ------------ | ------------------------------------------------------------------------------------------------------------------------------ | | `ColorKey` | `keyColor`, `similarity`, `smoothness`, `spillSuppression`, `colorSpace`, `spillSuppressionType` | | `RefineEdge` | `decontaminateAmount`, `radius`, `smooth`, `feather`, `shiftEdge`, `matteChannel`, `outputMode` | | `MaskMath` | `operation`, `radius`, `threshold`, `clampMin`, `clampMax`, `channelA`, `channelB`, `binarize`, `invertResult`, `outputFormat` | | `PatchHeal` | `patches`, `centerX`, `centerY`, `offsetX`, `offsetY`, `radius`, `sourceRadius`, `feather`, `opacity`, `mode` | ## Generators & style [Section titled “Generators & style”](#generators--style) | Effect | Props | | ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | `NoiseGenerator` | `noiseType`, `outputType`, `width`, `height`, `scale`, `octaves`, `persistence`, `lacunarity`, `speed`, `colorStart`, `colorEnd`, `durationMs`, `fps` | | `ProceduralVFX` | `effectType`, `outputType`, `width`, `height`, `density`, `scale`, `speed`, `intensity`, `seed`, `colorStart`, `colorEnd`, `durationMs`, `fps` | | `Paint` | `width`, `height`, `maintainAspect`, `aspectRatio`, `backgroundColor`, `strokes` | | `LayerStyle` | `dropShadow`, `innerShadow`, `outerGlow`, `innerGlow`, `stroke`, `bevelEmboss`, `colorOverlay` | ## Audio [Section titled “Audio”](#audio) | Effect | Props | | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `AudioFade` | `fadeInDuration`, `fadeOutDuration`, `fadeInCurve`, `fadeOutCurve` | | `AudioCompressor` | `threshold`, `ratio`, `attack`, `release`, `knee`, `makeupGain` | | `AudioParametricEq` | `type`, `frequency`, `gain`, `q` | | `AudioReverb` | `roomSize`, `damping`, `wet`, `dry`, `width` | | `AudioDelay` | `delayTime`, `feedback`, `wet`, `dry`, `pingPong` | | `AudioNoiseGate` | `threshold`, `attack`, `hold`, `release`, `range` | | `StereoPanning` | `pan` | | `AudioSignalExtractor` | `extractionMode`, `attackMs`, `releaseMs`, `sensitivity`, `noiseFloorDb`, `dynamicRangeDb`, `autoRange`, `smoothing`, `curve`, `beatThreshold`, `beatDecayMs`, `previewMode`, `nodeId` | ## Vision [Section titled “Vision”](#vision) | Effect | Props | | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Vision` | `enableDetection`, `enableSegmentation`, `enablePose`, `enableMatte`, `classes`, `confidence`, `variant`, `mode`, `matteSource`, `maxMissedFrames`, `modelsDir`, `baseUrl`, `maskThreshold`, `featherRadius`, `keyBackground`, `backgroundKeyThreshold`, `visionBundle` | ## Recipes [Section titled “Recipes”](#recipes) ```ts // Warm filmic grade for a whole film comp.apply(new ColorBalance({ shadows: { cyanRed: 4, yellowBlue: -6 }, highlights: { cyanRed: 2, yellowBlue: -4 } })); comp.apply(new Vignette({ strength: 30, radius: 0.9, softness: 0.65 })); comp.apply(new FilmGrain({ strength: 12, size: 1.5, animated: true })); // Greenscreen key Layer.video("assets/footage/green.mp4").withEffect(new ColorKey({ keyColor: "#00ff00", similarity: 0.4, smoothness: 0.1, spillSuppression: 0.3 })); // Print look layer.withEffect(new HalftoneScreen({ mode: "CMYK", dotShape: "Circle", frequency: 30 })); // Duotone layer.withEffect(new GradientMap({ stops: [{ position: 0, color: "#0e0d0c" }, { position: 1, color: "#ff5a1f" }] })); // Audio clean-up Layer.audio("assets/audio/vo.wav", { durationFrames }).apply(new AudioNoiseGate({ threshold: -45 })).apply(new AudioCompressor({ threshold: -18, ratio: 3 })); ``` # Lint & QA codes > Every code comp.lint() and renderVideo({ qa }) can report, its severity, and what to do about it. `comp.lint()` returns document findings before rendering. `renderVideo({ qa })` returns the same document findings plus measurements of the encoded frames and audio, in `result.qa.issues`. Each issue is `{ severity, code, message, layerId?, startSec?, endSec? }`. `qa.passed` is false if any **error** or **warning** is present. ## Document checks (lint and QA) [Section titled “Document checks (lint and QA)”](#document-checks-lint-and-qa) | Code | Severity | Means | Fix | | -------------------------- | -------- | ----------------------------------------------------------- | -------------------------------------------------- | | `invalid_gradient` | error | a `background`/`fill` gradient doesn’t parse; draws nothing | check the CSS syntax, ≤ 8 stops | | `unsupported_image_format` | error | `Layer.image` given `.json`/`.lottie`, video or audio | use `Layer.lottie` / `Layer.video` / `Layer.audio` | | `unknown_color` | warning | a color string that isn’t a color; draws transparent | use hex, `rgb()`, `hsl()`, `oklch()` or a CSS name | | `low_contrast_text` | warning | text below 3:1 against the solid color behind it | use the suggested color, or `Color.readableOn(bg)` | | `offscreen_layer` | warning | a static absolute layer entirely outside the canvas | fix `x`/`y` or remove it | | `animation_truncated` | warning | keyframes past the layer’s `durationFrames` never play | extend the layer or move the keys | | `shadow_unsupported_light` | warning | `castShadow` on a point or ambient light | use a directional or spot light | | `pbr_without_environment` | info | `roughness`/`metallic` set without `Layer.environment()` | add an environment or a `Lighting` rig | | `multiple_shadow_lights` | info | several `castShadow` lights; only the first casts | keep one | | `static_track` | info | every keyframe on a track holds the same value | remove the track or fix the values | ## Render measurements (QA only) [Section titled “Render measurements (QA only)”](#render-measurements-qa-only) | Code | Severity | Means | | --------------------- | -------- | ---------------------------------------------------------------------- | | `frames_missing` | error | fewer frames encoded than the composition has | | `black_frames` | warning | ≥ `minBlackSec` (0.5 s) of black; has `startSec`/`endSec` | | `audio_clipping` | warning | samples at or beyond full scale | | `loudness_off_target` | warning | integrated loudness more than `lufsTolerance` (1 LU) from `targetLufs` | | `frozen_video` | info | picture unchanged ≥ `minFrozenSec` (2 s); fine for a held title | | `audio_silence` | info | ≥ `minSilenceSec` (2 s) below `silenceDbfs` (−60) | | `audio_silent` | info | the whole track is silent | ## Use in CI or an agent loop [Section titled “Use in CI or an agent loop”](#use-in-ci-or-an-agent-loop) ```ts const result = await comp.renderVideo({ outputPath: "output/film.mp4", qa: { targetLufs: -14 } }); const blocking = result.qa!.issues.filter((i) => i.severity !== "info"); for (const i of blocking) console.error(`${i.code}${i.layerId ? ` [${i.layerId}]` : ""}: ${i.message}`); process.exit(result.qa!.passed ? 0 : 1); ``` # Troubleshooting > Symptoms, causes and fixes for install errors, blank frames, missing audio, hanging scripts, vision artifacts and preview problems. ## Install [Section titled “Install”](#install) | Symptom | Cause | Fix | | ------------------------------------ | ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `Cannot find module '.../skia.node'` | `skia-canvas` install script was blocked | approve it ([Getting started](/docs/getting-started/#2-write-packagejson-then-install)), then `npm rebuild skia-canvas` | | Import fails immediately | Node < 22, or not an ES module project | upgrade Node; set `"type": "module"` | | No adapter / GPU error | no WebGPU-capable GPU (VM, plain CI runner) | render on a machine with Metal or Vulkan; there is no CPU fallback | ## Picture [Section titled “Picture”](#picture) | Symptom | Cause | Fix | | --------------------------------------- | ------------------------------------------------- | ---------------------------------------------------------------------- | | Whole frame black | font not registered, or `NaN` from a signal | register every family before building text; keep signal outputs finite | | Text missing or tofu | family name doesn’t match a registered font | match `fontFamily` to the `family` you registered | | Emoji missing | no emoji font | `FontManager.registerEmojiFont(...)` | | Layer invisible | path wrong, opacity 0, off canvas, or LFS pointer | check `comp.lint()`; run `git lfs pull` | | Text clipped | box narrower than the text | set `width` (or `width: "fill"`) | | Animation stops early | keys past `durationFrames` | extend the layer (`animation_truncated`) | | Layer jumps to the corner when animated | animated `y` from 0 instead of its layout value | animate to the placed value | | Fades look murky grey | custom shader with straight alpha | output premultiplied alpha, blend with `srcFactor: "one"` | | Metal/rough looks flat | no environment | add `Layer.environment()` or `Lighting.studio()` | | Effect does nothing | strength on a 0–1 scale where it’s 0–100 | check ranges in the [Effects reference](/docs/reference/effects/) | ## Sound [Section titled “Sound”](#sound) | Symptom | Cause | Fix | | --------------------- | -------------------------------------- | ------------------------------------- | | Audio cuts out early | `Layer.audio` without `durationFrames` | set `durationFrames: totalFrames` | | `audio_clipping` | mix too hot | lower volumes; `softLimit` SFX beds | | `loudness_off_target` | mix level | adjust volume toward the `targetLufs` | ## Process [Section titled “Process”](#process) | Symptom | Cause | Fix | | ---------------------------------- | ------------------------------------------------ | ------------------------------------------ | | Script never exits after rendering | native GPU/encoder threads | `process.exit()` at the end | | Slow exports | too little parallelism or a shallow staging ring | raise `concurrency`; try `ringCapacity: 4` | ## Vision [Section titled “Vision”](#vision) | Symptom | Cause | Fix | | ------------------------------ | ------------------------------------------ | --------------------------------------------------------------------------- | | Cutout empty on a single still | vision reads the previous frame | render two consecutive frames with one renderer, keep the last | | Ghosted double of the subject | frames rendered out of order (grid, jumps) | verify with the MP4 or consecutive frames | | Wrong object cut out | subject choice | `classes: ["person"]` | | Garment drops out | fast motion | `variant: "m"` or `keyBackground: true` | | Halo on dark backgrounds | feathering | lower `featherRadius` or raise `maskThreshold` | | Fails offline / in CI | models not cached | preload into `$FRAMEFIELDS_MODELS_DIR` or set `FRAMEFIELDS_MODELS_BASE_URL` | ## Preview [Section titled “Preview”](#preview) | Symptom | Cause | Fix | | -------------------------------- | ---------------------------------- | ------------------------------------------------------------------------------- | | Font or media 404 in the page | path built from `process.cwd()` | build paths from `import.meta.url`; pass `root` for folders outside the project | | “No WebGPU” | browser without WebGPU | current Chrome, Edge or Safari | | Preview stopped | last tab closed (5 s), or replaced | run it again; same URL | | Cutouts look wrong while playing | device skipped frames | check paused, or in the export | Still stuck? Open an issue at [github.com/gatewai-dev/framefields](https://github.com/gatewai-dev/framefields/issues) or ask on [Discord](https://discord.gg/cbqMGGme5).