The agent workflow

Every step returns a file or data an agent can inspect. Go in order: each step catches in seconds what the next takes minutes to show.

0. Project shape

src/theme.ts      tokens: size, fps, colors, fonts, beat grid
src/film.ts       export async function buildFilm(): Promise<Composition>
src/render.ts     frames | grid | preview | video
assets/           every input file, tracked with Git LFS
  fonts/          .ttf .otf
  footage/        .mp4 .mov .webm
  audio/          .mp3 .wav
  images/         .png .jpg .webp
  luts/           .cube
  models/         .glb .gltf .obj .fbx
  lottie/         .json .lottie
  captions/       .srt (plain text, no LFS needed)
output/           renders (gitignored, never committed)
.gitattributes    LFS rules for assets/

render.ts is in Getting started. Keeping buildFilm() separate means tests, preview and export all build the same film. Reference assets from the module URL (path.resolve(HERE, "../assets/…")) so the preview can load them too.

Assets and Git LFS

Source media goes in assets/ and is stored with Git LFS, so the repo stays small and clones stay fast. Renders go in output/ and are never committed.

Set up once per repo

git lfs install                       # once per machine
git lfs track "assets/fonts/**" "assets/footage/**" "assets/audio/**" \
  "assets/images/**" "assets/luts/**" "assets/models/**" "assets/lottie/**"
echo "output/" >> .gitignore
git add .gitattributes .gitignore
git commit -m "Track assets with Git LFS"

git lfs track writes the rules to .gitattributes (assets/footage/** filter=lfs diff=lfs merge=lfs -text, and so on). Commit that file before adding any media. Ignore output/, never *.mp4 — that would also hide your footage.

Push assets

Add and push as usual; Git uploads the LFS objects with the push.

git add assets/footage/interview.mp4
git lfs ls-files                      # the new file must be listed — otherwise it is not in LFS
git commit -m "Add interview footage"
git push

If a file was committed before its rule existed, it went into normal Git. Fix it with git rm --cached <file>, git add <file> and commit again.

Pull assets

A clone with LFS installed downloads assets automatically. Anywhere else — a fresh container, a CI runner, an agent sandbox, or a clone made with GIT_LFS_SKIP_SMUDGE=1 — fetch them before rendering:

git lfs install
git lfs pull                          # all assets for the current commit
git lfs pull --include="assets/fonts/**,assets/footage/intro.mp4"   # or just what this render needs

Check nothing is still a pointer: in git lfs ls-files, * means the file is downloaded and - means it is still a pointer.

Pointers render blank

An un-pulled LFS file is a ~130-byte text pointer starting with version https://git-lfs.github.com/spec/v1. framefields cannot decode it: fonts fail to register, and videos, images and audio come out blank or throw. If a render shows missing media in a fresh checkout, run git lfs pull before changing any code.

In GitHub Actions, check out with lfs: true (actions/checkout) so the runner has the real files.

1. Check without the GPU

npx tsc --noEmit -p .

const film = await buildFilm();
for (const d of film.lint()) console.log(`${d.severity} ${d.code}: ${d.message}${d.layerId ? ` [${d.layerId}]` : ""}`);

Fix every error and warning before rendering. Codes and fixes: Lint & QA codes.

2. Look at key frames

npm run frames -- 0 30 60 89

Open each PNG in output/ and compare it to what was asked. A black frame usually means a font that isn’t registered, a layer off canvas, opacity stuck at 0, or an asset that is still a Git LFS pointer (git lfs pull).

3. Look at motion

npm run grid -- 0 6 12 18 24 30 36 42

One image of the whole move: easing hitches, text that re-wraps, wrong z-order, missing cuts. Not for vision layers (they need frames in order).

4. Show a person

npm run preview   # run in the background; prints the URL

• Give the person the URL, or open it in your own browser pane.
• Read the command’s output: page errors and lifecycle lines show up there.
• When they say they left notes, read .framefields/preview-notes/notes.json and open each open note’s note-<id>.jpg before editing. Each note has the frame, the spot (x, y as fractions) and the text.
• Rerun the preview after changes; the open tab reloads at the same moment. Mark notes "done": true when addressed.

5. Ship with QA

import { formatQaReport } from "framefields";

const result = await film.renderVideo({ outputPath: "output/film.mp4", quality: "high", qa: { targetLufs: -14 } });
console.log(formatQaReport(result.qa!));
process.exit(result.qa!.passed ? 0 : 1);

qa.passed is false on any error or warning: dropped, black or frozen frames, clipping, loudness off target, plus every lint finding. Treat it as the gate.

In CI

Rendering needs a GPU (Metal or Vulkan), so run the full render on a GPU runner. On any runner you can still gate on types and lint():

scripts/check.ts

import { CompositorProgramSchema } from "framefields";
import { buildFilm } from "../src/film.js";

const film = await buildFilm();
const bad = film.lint().filter((d) => d.severity !== "info");
const schema = CompositorProgramSchema.safeParse(film.toSpec());
if (bad.length || !schema.success) {
  for (const d of bad) console.error(`${d.code}: ${d.message}`);
  process.exit(1);
}
process.exit(0);

Habits that keep agents fast

• Change one scene at a time and re-render only its frames.
• Keep all magic numbers in theme.ts; a tweak is then one line.
• Name layers with id — lint, QA and chart children report by id.
• Put asset paths through import.meta.url so the preview loads them too.