motionlab/EFFECTS.md

Writing a MotionLab effect

An effect is one text file: effects/<name>.metal. The app picks it up automatically (also while it is running) and shows it in the panel with a checkbox and its sliders. No other file needs to change.

The contract

The file defines exactly one function, written in Metal Shading Language. It is called once per pixel, every frame, and returns that pixel's colour:

float4 effect(float2 uv, Inputs in) {
    return in.image.sample(in.smp, uv);   // this one changes nothing
}

• uv is the pixel's position: (0,0) is top-left, (1,1) is bottom-right.
• Return float4(r, g, b, 1.0) with components in 0..1. Alpha is not shown on screen; return 1.0 unless you use it as memory (see below).
• Do not write #include, using namespace, a vertex function or a fragment function. The app adds those.

What in gives you

Field	Type	Meaning
in.image	texture	The picture so far: the output of the effect above this one in the chain, or the camera if it is the first. Read this for your input picture.
in.camera	texture	The untouched camera frame.
in.last	texture	The camera frame before this one. length(camera - last) at a pixel tells you how much it changed.
in.prev	texture	This effect's own output from the previous frame. Read it to build trails and feedback.
in.flow	texture	.xy = which way and how far the picture moved at this spot since the last frame, in uv units. Typically tiny (0.001 to 0.02): length(flow) * 50.0 is a rough 0..1 "how fast" value.
in.particles	texture	.r = brightness of particles that spawn where something moves and drift along with the motion.
in.smp	sampler	Pass it to every .sample() call.
in.time	float	Seconds since the app started.
in.dt	float	Seconds since the previous frame. Effects run once per camera frame, so this is about 1/30. Use it to make fades and timers last the same in real time whatever the frame rate: value *= exp(-in.dt / seconds).
in.resolution	float2	Output size in pixels. One pixel is 1.0 / in.resolution in uv units.
in.p.<name>	float	Your sliders (see below).

Reading a texture: in.image.sample(in.smp, uv) returns a float4; take .rgb, .r, .xy and so on. Reading outside 0..1 repeats the edge pixel.

Comment lines the app reads

Put these at the top of the file. Spacing matters: one space between the parts, nothing after the last value.

// slider <name> <min> <max> <default>
// order <number>
// on

• // slider glow 0.0 4.0 1.5 adds a slider; read its value as in.p.glow. Up to 16 per effect.
• // order 60 sets the position in the chain. Any whole number; only the relative order matters, lower runs first, and the default is 50. Existing effects use 10 to 90. The motion inputs (flow, particles) are computed from the camera before any effect runs.
• // on makes the effect enabled when the app starts. Leave it out for new effects.

Slider names must be plain identifiers and must not be Metal built-ins or keywords (mix, length, in, float, step, sample and so on). amount, speed, glow, hue are fine.

Metal Shading Language, for people who know GLSL

GLSL	Metal
vec2, vec3, vec4	float2, float3, float4
texture(tex, uv)	tex.sample(in.smp, uv)
mod(a, b)	fmod(a, b) (keeps the sign of a; use a - b * floor(a / b) for a positive result)
atan(y, x)	atan2(y, x)
mix, clamp, smoothstep, fract, floor, length, dot, sin, cos, pow, abs, min, max	the same

• Write float numbers with a decimal point: 2.0, not 2. Convert loop counters with float(i).
• Helper functions go above effect, must be marked static, and take Inputs in as a parameter if they read textures.
• There is no global state. To remember something between frames, write it into your output and read it back from in.prev. Its .rgb is the picture you showed last frame; its alpha channel (.a) is free for your own use because it is not shown on screen. That gives one private number per pixel, stored with 8-bit precision (256 steps between 0 and 1), which is enough for a glow or heat level but not for a timestamp or a full-colour image.
• To do something every N seconds, test where in.time is inside the current period: fract(in.time / N) < in.dt / N is true for exactly one frame per period.

Two complete examples

A wavy colour split with two sliders:

// Splits red and blue apart in a wave that scrolls down the picture.
// order 60
// slider amount 0.0 0.05 0.012
// slider waves 1.0 80.0 24.0
float4 effect(float2 uv, Inputs in) {
    float2 d = float2(in.p.amount * sin(uv.y * in.p.waves + in.time * 3.0), 0.0);
    return float4(in.image.sample(in.smp, uv + d).r,
                  in.image.sample(in.smp, uv).g,
                  in.image.sample(in.smp, uv - d).b, 1.0);
}

Things that move light up and the glow fades slowly, using in.prev alpha as memory:

// Whatever moves lights up, and the glow lingers.
// order 50
// slider sensitivity 1.0 30.0 10.0
// slider linger 0.5 0.99 0.93
float4 effect(float2 uv, Inputs in) {
    float moved = length(in.camera.sample(in.smp, uv).rgb - in.last.sample(in.smp, uv).rgb);
    float fresh = smoothstep(0.3, 1.0, moved * in.p.sensitivity);
    float heat = max(in.prev.sample(in.smp, uv).a * in.p.linger, fresh);
    float3 base = in.image.sample(in.smp, uv).rgb * 0.5;
    return float4(base + float3(1.0, 0.5, 0.1) * heat, heat);
}

More examples are in effects/.

Checking your work

Build once, then check. Both commands print compiler errors with the line number in your file and exit non-zero on failure.

swift build -c release

# compiles every effect; needs no camera and opens no window
.build/release/MotionLab --check

# runs only your effect on the live camera for 90 frames, saves the last frame, exits
.build/release/MotionLab --only <name> --snapshot /tmp/<name>.png

--check also lists the sliders it found for each effect, and warns when a // slider line was ignored because it is malformed or over the limit of 16.

Checklist

• The file is effects/<name>.metal and defines float4 effect(float2 uv, Inputs in).
• It reads its input picture from in.image, not in.camera, so it works in a chain.
• The few numbers a user would most want to tune are sliders with a sensible range and default. Internal constants can stay in the code.
• The first line is a comment saying what the effect does.
• .build/release/MotionLab --check prints ok for it.