videobeaux/program-docs/docs-lagkage.md

lagkage – JSON‑driven layered compositor for videobeaux

lagkage is a videobeaux program that lets you build complex, multi‑layer video compositions from a single JSON file.

You define:

• A base video (passed via -i as usual)
• A set of layers (images, GIFs, or videos) in a JSON layout file
• Per‑layer properties: position, size, opacity, zoom, crop, audio gain
• Global behavior: sequence ordering, audio mix mode

lagkage then builds an ffmpeg filtergraph that:

• Scales, crops, and blends each layer into the base frame
• Treats GIFs as animated overlays (pre‑processed to loop for the base duration)
• Handles audio according to your chosen mode (base/json/all/external/none)
• Optionally applies per‑layer audio gain, so you can do a rough pre‑mix directly from the JSON without opening a DAW

---

1. CLI usage

From your videobeaux project root:

videobeaux -P lagkage \
  -i media/base.mp4 \
  -o out/composite.mp4 \
  --layout-json lagkage_layouts/layout_ex01_logo_corner.json \
  --force

Arguments

• -i, --input
  Base video to composite onto. Required.

• -o, --output
  Final rendered video. Required.

• --layout-json PATH
  Path to a JSON file that describes all overlay layers. Required.

• --sequence-direction {forward,backward,random}
  Optional override for the order in which layers are applied. If omitted, the layout JSON’s own sequence_direction is used.

• --audio-mode {base,all,json_only,external,none}
  How to construct the final audio track:

  • base (default) – only the base video’s audio. JSON layers are visual only.
  • all – mix base audio + all JSON layer audio together.
  • json_only – ignore the base audio, use only JSON layer audio.
  • external – ignore base + JSON audio, use the file supplied via --audio-src.
  • none – no audio at all (silent output).

• --audio-src PATH
  Path to an external audio file. Only used when --audio-mode=external.

• --force
  From the global videobeaux CLI: forces overwrite of existing outputs and adds -y to ffmpeg under the hood.

---

2. Layout JSON format

A layout JSON looks like this at the top level:

{
  "sequence_direction": "forward",
  "layers": [
    {
      "layer_number": 1,
      "name": "logo",
      "filename": "../media/reem.png",
      "type": "img",
      "mode": "place",
      "place": "top_right",
      "size": 18,
      "opacity": 0.95
    }
  ]
}

2.1 Top level

• sequence_direction (string, optional)

  Controls how lagkage orders the layers before applying them:

  • "forward" – smallest layer_number first (default)
  • "backward" – largest layer_number first
  • "random" – random permutation of layers

  You can override this per‑run using --sequence-direction on the CLI.

• layers (array, required)

  Each element is a layer object describing one overlay media item.

2.2 Per‑layer fields

All layers share a common set of fields. Some are optional depending on mode.

Identity and source

• layer_number (int, recommended)

  Used for ordering and for sequence modes. Lowest layer_number is “first” in forward mode. If omitted, it defaults to 0 for sorting, but it’s best to set explicit numbers.

• name (string, optional)

  Short human‑readable label for the layer. Not used by the code, but useful for documentation.

• filename (string, required)

  Path to the media file for this layer. Resolution rules:

  • Absolute paths (/Users/you/...) are used as‑is.
  • URLs (https://...) are passed directly to ffmpeg.
  • Paths starting with '../media/' are normalized to media/... relative to your project root.
  • Paths starting with 'media/' are also treated as project‑root relative.
  • Any other relative path is treated as layout‑relative: resolved against the folder that contains the JSON.

• type (string, required)

  The kind of media:

  • "img" – PNG/JPEG/etc.; single frame, no audio
  • "gif" – animated GIF; pre‑processed to a looping RGBA video
  • "video" – standard video file; may contain audio

  This mainly affects whether we run the GIF preprocessing pipeline.

Positioning: mode, place, pos_x, pos_y

A layer can be positioned in place mode (logical corners/center) or in free mode (explicit pixel coordinates).

• mode (string, required)

  • "place" – snap the layer into a logical slot:
    • uses place
  • "free" – place the layer at exact coordinates:
    • uses pos_x, pos_y

• place (string, required if mode="place")

  One of:

  • "top_left"
  • "top_right"
  • "bottom_left"
  • "bottom_right"
  • "center"

• pos_x, pos_y (int, required if mode="free")

  Pixel coordinates in the base video’s pixel space. (0,0) is top‑left of the base frame. These are the top‑left of the scaled/cropped layer box.

  You can push a layer partially or fully off‑screen by using negative values or values greater than the base width/height.

Size and opacity

• size (float, optional; default 100)

  Percentage of the base video width that this layer should occupy. The layer is scaled to:

  box_w = base_width * (size / 100)
  box_h = box_w / aspect_ratio
  

  Both are forced to even integers (necessary for some codecs).

• opacity (float, optional; default 1.0)

  Range [0.0, 1.0]. Implemented using colorchannelmixer on the layer’s alpha channel before overlay:

  • 1.0 – fully opaque
  • 0.5 – half transparent
  • 0.0 – fully transparent (invisible)

Zoom and crop

These operate in layer space before the final overlay.

• zoom (float, optional; default 1.0)

  • 1.0 – no zoom (layer fits exactly inside the computed box_w x box_h)
  • >1.0 – zoom in; layer is scaled up to zoom * box then center‑cropped back down to the box. This gives the effect of “zooming inside” a fixed PIP area.

• crop_x, crop_y, crop_w, crop_h (float, optional)

  Percentages of the original source width/height, all in [0,100].

  If all four are present, lagkage applies:

  crop=in_w*crop_w/100 : in_h*crop_h/100 : in_w*crop_x/100 : in_h*crop_y/100
  

  This happens before zoom/scale. Use this to punch out a region from a larger frame (for example, turning a full‑frame video into a “face cam” PIP).

Per‑layer audio gain

These fields only matter if:

• the layer’s media actually has audio (e.g. type: "video" with an audio stream),

• and --audio-mode is all or json_only so layer audio is in the mix.

• audio_gain_db (float, optional)

  Gain in decibels applied to the layer’s audio before mixing:

  • 0 – keep original level
  • -6 – roughly “half as loud” perceptually
  • +3 – a bit louder
  • +12 – very loud / likely to clip unless sources are quiet

• audio_gain (float, optional)

  Linear multiplier, converted internally to dB as:

  audio_gain_db = 20 * log10(audio_gain)
  

  Only used if audio_gain_db is not set.
  For example, audio_gain: 0.5 ≈ -6.02 dB.

---

3. Audio behavior in detail

3.1 Modes

• --audio-mode base (default)

  • Map 0:a? (base audio only).
  • JSON layers are visual only; per‑layer audio gains are ignored.

• --audio-mode all

  • Take audio from:
    • base input (index 0) if it has audio
    • all JSON layers whose preprocessed media has audio
  • Apply per‑layer gains (where specified) on JSON layers.
  • Use amix to combine all sources into a single output stream.

• --audio-mode json_only

  • Ignore base audio completely.
  • Mix only JSON layer audio (with per‑layer gains) via amix.

• --audio-mode external

  • Append --audio-src PATH as an extra ffmpeg input.
  • Map only that input’s audio (N:a?) to the output.
  • JSON layer audio and base audio are ignored.

• --audio-mode none

  • Add -an to ffmpeg.
  • Output video is silent.

3.2 When do per‑layer gains matter?

Per‑layer audio gains (audio_gain_db / audio_gain) are applied:

• only on JSON layers that actually have an audio stream, and
• only when --audio-mode is all or json_only.

They do not affect:

• the base input audio (there’s currently no base_audio_gain_db),
• the external audio when --audio-mode=external,
• GIF overlays (their preprocessed videos are silent),
• image layers (no audio).

---

4. GIF handling

GIFs are pre‑processed before the main pass so the compositor can treat them like normal video streams with alpha.

For each type: "gif" layer:

1. lagkage runs a helper ffmpeg command that:
   • loops the GIF (-stream_loop -1, -ignore_loop 0)
   • clamps to the base duration using -t base_duration
   • forces even dimensions with scale=trunc(iw/2)*2:trunc(ih/2)*2
   • converts to RGBA pixels (format=rgba)
   • writes a .mov with qtrle codec and alpha preserved
2. The resulting .mov is used as the actual overlay source.

This is why animated GIFs loop smoothly across the entire base clip and retain their transparency when overlaid.

---

5. Path resolution rules

Given a layout_exNN.json at:

lagkage_layouts/layout_exNN_*.json

and a layer filename value, lagkage resolves paths as follows:

1. If filename contains "://" → treat as URL and pass directly to ffmpeg.
2. If filename is an absolute path (e.g. /Users/you/file.mp4) → use as‑is.
3. If filename starts with "../media/" → normalize to media/... relative to your project root.
4. If filename starts with "media/" → treat as project‑root relative.
5. Otherwise → resolve relative to the layout JSON’s directory.

---

6. Example layouts & commands

This repo includes a set of example layouts and matching commands that showcase:

• place vs free mode
• cropping and zoom
• animated GIF overlays
• multiple layers with different sizes & opacities
• different sequence_direction values
• all audio modes, including per‑layer gain

See:

• lagkage_layouts/ – 20 example layout JSON files (layout_ex01_.json … layout_ex20_.json)
• lagkage_examples_full.txt – example commands, one per layout

You can copy one example, tweak the filenames to match your media, and then iterate from there.

---

7. Minimal “hello lagkage” example

lagkage_layouts/layout_ex01_logo_corner.json:

{
  "sequence_direction": "forward",
  "layers": [
    {
      "layer_number": 1,
      "name": "logo_corner",
      "filename": "../media/reem.png",
      "type": "img",
      "mode": "place",
      "place": "top_right",
      "size": 18,
      "opacity": 0.95
    }
  ]
}

Command:

videobeaux -P lagkage   -i media/base.mp4   -o out/ex01_logo_corner.mp4   --layout-json lagkage_layouts/layout_ex01_logo_corner.json   --audio-mode base   --force

This is the simplest setup:

• Base video at media/base.mp4
• One PNG logo (reem.png) in the top‑right corner
• Base audio only
• No zoom/crop or audio mixing yet

From here, add more layers, switch to audio-mode=all, and start using audio_gain_db to explore full lagkage functionality.