Skip to content

framefields 3: what's new since 2.2

View .mdllms.txt

framefields 2.2 came out on 17 October. Since then we’ve shipped 29 releases, and today’s 3.0 rounds them off. It’s the same engine as before. You still write a composition in TypeScript and render it on WebGPU, and coding agents can still drive it. What’s new is how much you can put in that composition.

This post walks through the main additions in the order you’ll probably meet them. If you’d rather watch than read, the framefields 3 page shows each one in a film the engine rendered.

A Schwarzschild black hole, ray traced in one Layer.shader, from examples/41_hawking_radiation.

New in 3.0, Layer.shader is a layer drawn by your own WGSL fragment function. Each uniform can be a number, a signal or a function of the frame, so a shader can follow the beat or a keyframed value without any extra plumbing.

Layer.shader({
width: 1920,
height: 1080,
uniforms: { zoom: 2, pulse: kick, sweep: (ctx) => ctx.frame / 90 },
fragment: `
fn shade(uv : vec2<f32>) -> vec4<f32> {
let p = (uv - 0.5) * vec2<f32>(u.resolution.x / u.resolution.y, 1.0) * u.zoom;
let ring = smoothstep(0.02, 0.0, abs(length(p) - 0.3 - 0.1 * u.pulse));
return vec4<f32>(vec3<f32>(ring) * u.sweep, 1.0);
}`,
});

Once it’s on the timeline, it behaves like any other layer. You can animate its opacity, put effects on it or use it as a matte. If a fragment doesn’t compile, the error gives line numbers in your own source, not in the code the engine wraps around it. The launch page has an editor where you can try the same contract in your browser.

Most of the work between 2.3 and 2.15 went into 3D. You can now build whole scenes in the same timeline as your type and footage:

  • Worlds. Layer3D.terrain, Layer3D.sdf and Layer3D.scatter build landscapes, smooth sculpted solids and wind-blown foliage. Any layer, including a video, can texture a mesh.
  • Surfaces. glTF models bring their own clearcoat, sheen, anisotropy and iridescence. Skin’s subsurface scattering now conserves energy, so skin looks less washed out, and glass bends the scene behind it using the IOR and thickness from the model file.
  • Characters. Clip blending, two-bone IK, look-at, spring bones, joint drivers and named morph targets. Layer3D.instances plays a skinned crowd in one draw call.
  • Things breaking. world.fracture() shatters a solid, a piece of extruded text or a model into pieces that break on impact.
  • Lenses. Per-object motion blur, outlines and toon lighting, 360° and fisheye cameras, and a relativistic camera that films the scene as it would look near the speed of light.

Terrain, scatter and wind, all from TypeScript.

A box with a fixed size that holds a camera is now its own scene. Its camera and lights see only its layers, and it runs on its own clock. So a Layer.sequence transition between two 3D scenes blends two finished pictures instead of drawing both through one camera. The 3D guide covers the details.

Version 2.3 added rigid-body physics. Physics.world2d and Physics.world3d simulate once while the composition is built, then replay frame by frame, so a render gives the same result every time and you can scrub through it. Each body exposes signals for impact, speed and pose, and you can connect them to anything.

const world = await Physics.world2d({ fps: 24, frames: 150 });
world.walls({ width: W, height: H });
world.body(card, { restitution: 0.35 });
world.simulate();
glow.animate(LayerAnimation.create().signal("opacity", world.impact));

GPU particles arrived in the same release. They run in a compute shader with emitters, flow fields, attractors and floors, and a performer’s silhouette can push them around.

Here the words are physics bodies, and the dancer’s tracked silhouette knocks them about. See the physics guide.

On-device vision got much stronger. comp.vision(clip, { sam2 }) tracks a subject from a point or a box with SAM 2.1. { sam3: { text: "man in white shirt" } } picks out people or objects from plain words, several at once if you need them, and catches new ones as they walk in. matting: true runs ViTMatte over the edge of the mask, so a cutout keeps hair and motion blur instead of a hard outline. Depth Anything lifts footage into a relief that a 3D camera can move around.

All of this runs on your own machine. The models are pinned by SHA-256 and download the first time you use them. SAM 3’s weights download only after you accept Meta’s SAM License. Masks still aren’t perfect: fast movement and thin edges can trip them up, so check your frames. The vision guide shows how.

Thirteen new stylize effects (pixelate, posterize, dither, ASCII, toon, CRT, painterly, contour and more) and Styles presets that stack them into finished looks:

clip.withEffects(Styles.vhs());

There are also gameBoy, ledWall, risograph, comic, thermal, oilPainting, holographic, noir and a dozen others. LayerAnimation.stepped(2) puts any layer on twos. The launch page plays the same clip in each look.

Text now takes a variable font’s axes with fontVariations: { wdth: 68 }. Layout measures the text at those axes, and it draws the same way in 2D and 3D. A few long-standing path bugs are fixed too:

  • strokeDashArray renders.
  • Paths with several subpaths fill with their holes.
  • Keyframing a shape’s d morphs one path into another, even when the two have different point counts.
  • Strokes draw in one call per path, about twice as fast on a map with coastlines and a hundred circles.

The other headline in 3.0 is the style library. It’s creative direction as data: six looks, six formats, six genres, five ways to make the pictures, and presets that combine them. combineStyles turns one mix into a single brief:

import { combineStyles, styleClashes } from "framefields/styles";
const brief = combineStyles({ look: "motion", format: "vertical", genre: "product-launch" });
const watch = styleClashes({ look: "blueprint", format: "ad-spot", genre: "brand-film" });

The brief says which palette, type, pacing, music and grade to use, and how AI-generated clips should match the drawn graphics. It’s also what your agent reads when you ask for a video “in the style of” something.

The agent guides now ship inside the package, at node_modules/framefields/dist/skills/. Each release carries guides written for its own API, so an agent always reads the ones that match the framefields it has installed. Around them:

  • framefields init writes a working project and never prompts or overwrites.
  • framefields add copies any of 26 components into your project, along with the fonts they need.
  • framefields check --json returns schema, lint and timeline findings an agent can act on.
  • Render logs fit an agent’s context: about 15 lines, whatever the film’s length.
  • import { ai } from "framefields" reaches image, video, music and voice models with one key. See the AI guide.

Start with agents to see how it fits together.

A long render can now split into chunks. Each chunk starts on a keyframe and renders on its own GPU, and the chunks are stitched back together without re-encoding. From the CLI it’s one flag, and --plan tells you the workers, time and estimated cost before anything renders:

Terminal window
npx framefields render film --cloud --workers auto --plan

Named renders in framefields.json mean the CLI, renderCloud and the dashboard pick the same cut by name. framefields assets keeps footage out of git on any S3-compatible bucket, pinned by SHA-256. And framefields preview now streams those files and mixes the soundtrack in the browser, sample for sample the same as the export. Read the cloud rendering guide to set it up.

framefields now has a path tracer (from 3.0.1). Add pathTrace: true to the environment and the scene’s 3D layers are traced instead of rasterized. Shadows soften with distance, reflections show things the camera can’t see, glass bends what’s behind it, and light bounces from surface to surface. A glowing mesh lights the room around it.

Layer.environment("night", {
floor: { y: 420, color: "#22252c", reflectivity: 0.35 },
pathTrace: true,
});

Glass, path traced: an iridescent lamp globe, two glasses of flowers and a mosquito in amber, lit by a studio’s window, from examples/43_path_tracer. Models from the Khronos glTF samples (Wayfair and Sketchfab, CC-BY 4.0; vase CC0), light from Poly Haven (CC0).

It runs on WebGPU compute, like the rest of the engine, and uses the same lights, materials and tone mapping as the raster renderer. So a scene looks like itself with tracing on, only with the light worked out properly. A render takes the same number of samples per pixel on any GPU, so frames come out identical every time and split cloud renders join cleanly. The preview adapts to your GPU instead, so scrubbing stays responsive.

It costs time. At the default quality a 1080p frame of a simpler scene, a character and a glowing ring, takes about 12 seconds on an M1 Pro, and this glass still life about 30, so iterate on quality: "draft" and switch back for the final render. Some features aren’t traced yet, including text on a 3D plane, toon shading and instanced crowds, and the render tells you which one it hit. The 3D guide has the presets and the rules.

New in 3.1: Layer3D.clay is a sculpture you write as a list of moves, the way you’d work clay by hand. Solids and brush strokes add or carve material, with smooth joins where they meet. grab, twist and bulge push what’s already there around like an elastic solid. inflate swells a region, and detail adds stone or skin texture. symmetry: "x" mirrors everything you do on one side.

Layer3D.clay({
symmetry: "x",
ops: [
{ type: "sphere", center: [0, -70, 0], radius: 72 },
{ type: "stroke", path: [[0, -48, -48], [0, -38, -100]], radius: 34, smoothness: 20 }, // snout
{ type: "stroke", path: [[40, -122, 10], [72, -160, 30], [86, -200, 72], [72, -228, 110, 4]], radius: 16 }, // horn
{ type: "sphere", center: [30, -74, -66], radius: 15, mode: "carve" }, // eye socket
{ type: "grab", center: [0, -14, -104], radius: 26, delta: [0, 16, -6] }, // pull the chin
{ type: "detail", amplitude: 2.2, scale: 22, ridged: true }, // stone
],
});

The bust is built from the same list of moves, shown a few more at a time as the camera circles. The horns grow along their strokes with progress. The film is examples/45_sculpture, and it renders at 1080p in about 0.2 seconds a frame on an M1 Pro.

The sculpture is never turned into polygons to be drawn. Each pixel is worked out from the moves themselves, so edges and texture stay sharp however close the camera gets, and there’s no polygon count to pick. It casts and catches shadows like any other mesh. Path tracing, glass and physics use a mesh made from the same moves.

A finished sculpture can also leave framefields. comp.exportModel({ frame, outputPath: "bust.glb" }) writes the frame’s 3D scene as glTF, with the camera and lights, and it opens as it rendered in Blender, Unreal, Unity or a web viewer. The 3D guide lists every move.

3.0 is a major release for one visible change: shape strokes now draw their outer half instead of being cut off at the shape’s layout box. If a film relied on that clipping, outlines will look thicker in 3.0. Two earlier changes are also worth a look if you’re coming straight from 2.2:

  • Containers default to overflow: "visible", as in CSS (since 2.14). Set overflow: "hidden" on a box whose children have to stay inside it, such as a rounded card holding a square image.
  • A keyframe track holds its first value until it’s reached (since 2.13.1). A fade that starts at frame 40 is now invisible until frame 40, rather than fully visible and then snapping out.

Render a contact sheet before and after you bump the version, and compare them. The full changelog lists every change, release by release.

Terminal window
npx framefields init my-film

Or give the agent prompt to Claude Code, Codex or Cursor and let it install framefields, read the guides and render a first film. If you’d rather not use a terminal, the desktop app below does the same from a conversation.