Foundations: Identity, Versions & Schema Versioning¶

Every Element gets a real, permanent identity separate from each frozen version it produces, and every reference to it points at one specific version. This page owns that three-way shape, the commit rule, what happens when a reference points at a version committed under an older Composer schemaVersion, and the one architectural decision still open above all of it: whether Composer's schema extends the engine's current format or exists alongside it.

Identity, version, instance¶

Three separate identities, easily conflated into one "Element id":

Identity	Table	Carries
Element identity — one row per Element, stable across every version	`retail_elements`	Its own immutable `id`, assigned once per The Change Model's id scheme, at catalogue scope rather than layer scope; a mutable `name`, never doubling as an address; `brandId` / `organisationId` plus a sharing list for the cross-brand case (Brand Scoping & Brand Style); `currentVersionId`, pointing at whichever version is live.
Element version — one row per frozen, immutable version; the thing a Scene's layer instance actually references	`retail_elements_versions`	`id`; `elementId`; `schemaVersion` (below); `parentVersionId`, nullable, null only for an Element's first version — the version-to-version lineage a fork records (The Change Model → Two words, defined once); `status: draft \\| committed`; `content`, the resolved layer list and manifest bindings, frozen once committed — while still a draft it's simply `resolveDocument(parentVersion.content, changes)` computed on demand rather than stored redundantly; `changes`, the `Change[]` diff applied against the parent to produce this version, kept permanently for audit and never mutated once committed; `brandStyleVersionId`, the frozen brand-style version its role references resolve against (Brand Scoping & Brand Style); `committedBy` / `committedAt`.
Element instance	nothing new	A placement of one specific version inside a Scene or Overlay's layer list, with its own instance id scoped to that Scene (Elements, Scenes & Projects), plus whatever `setInstanceField` / `setInstanceLayout` overrides sit on top, and repointed by `rebaseInstance`.

A draft and a committed version are the same row shape at different points in their lifecycle, not two mechanisms — one table with a status flag rather than a separate drafts table, the same "one way to do a thing" reasoning used for assetRefSchema and the brand-style rebase. A draft carries the schemaVersion it is being authored under, the same as a committed version.

Since Composer's content is DB-native only (Storage & Versioning), the identity row is unconditionally authoritative for name and ownership; there is no bundled case where the document would need to be self-describing on its own. This same three-way shape applies unchanged to Scenes, Projects, and campaigns (retail_scenes / _versions, retail_projects / _versions, retail_campaigns / _versions) and to the brand style profile (retail_brand_style_profiles / _versions) — with one column's exception: a style-profile version carries no brandStyleVersionId, since it is the brand-style version everything else pins. A campaign's identity row additionally carries the Project version it forks from (Storage & Versioning).

flowchart LR
    ID["retail_elements<br/>id · name · brandId/orgId · sharing<br/>currentVersionId"]
    V1["version 1<br/>committed · parent: null"]
    V2["version 2<br/>committed · parent: v1"]
    V3["version 3 (draft)<br/>content = resolveDocument(v2.content, changes)"]
    ID -->|currentVersionId| V2
    V1 --> V2 --> V3
    I1["instance in Scene X<br/>→ v1 (stale)"] -.-> V1
    I2["instance in Scene Y<br/>→ v2 (current)"] -.-> V2

Commit rule¶

Committing a draft turns it into the identity's current version: status flips to committed, committedBy / committedAt are stamped, and currentVersionId advances to it. A commit is rejected unless the draft's parentVersionId is the identity's currentVersionId at commit time; otherwise the draft rebases first (Fork, Rebase & Conflict Resolution). This is what keeps two people editing the same thing from destroying each other's work: two drafts from v2 can both exist, but the second to commit finds currentVersionId already at v3 and goes through rebase, conflict resolution, and Approve Migration rather than silently moving the pointer past the first. It protects commits specifically; what stops two concurrent saves into one fork from overwriting each other is a separate, still-open question (The Change Model).

A commit is also rejected if any instance in the document references a draft version. A committed version's content has to be frozen all the way down, or a Scene could be published while an Element inside it is still being edited — precisely the drift the fork model exists to prevent, and the shape of the original BSR incident one level down. References resolve to committed versions only; the sole draft anything ever resolves is the fork currently under edit, for its own author's preview (Storage & Versioning → Resolving a reference).

Commit is one of the lifecycle operations that sit outside Change[] (The Change Model), gated by whichever grant could author the draft in the first place: the composing rung for a Scene or Project draft, the campaign rung for a campaign fork, the Element create/edit permission for an Element draft (Permissions & Enforcement).

Composer's `schemaVersion`¶

schemaVersion is { major, minor }, no patch. Patch has no use case here: nothing in this model needs a distinction finer than "does this break compatibility" (major) versus "does this not" (minor), so it's dropped rather than carried unused.

A separate concept from the engine's schemaVersion

The mechanism below is Composer's own, for Composer's own document versions, and relies on lazy transformer chains the engine has decided against for itself. Whether the two are one axis or two is an open reconciliation point, with the engine's own position recorded there (The Resolver Engine Dependency).

Cross-schema-version resolution: silent upcast at render, explicit migration at placement¶

Two separate triggers, not one, resolve what happens when a reference points at an older schemaVersion than current, and this applies identically across all three altitudes, per direct instruction. The question this answers: a Project on schemaVersion 1.5 references a frozen Scene or Element version committed at 1.0 — what happens.

Trigger one: render/resolve, always silent, unconditional, minor or major. Whenever the storage-layer read the resolver depends on (Storage & Versioning → Resolving a reference) loads a version whose schemaVersion is behind the current one for that altitude, it runs content — and content alone, never changes — through the chain of upgradeVNtoVN+1 transformers up to current before returning it, invoked lazily at lookup time rather than as an eager bulk rewrite. changes stays exactly as committed, permanently: it is the audit record of what happened at authoring time, in whatever shape existed then, and upcasting it would falsify that record. This never rewrites the stored, frozen row; it's computed at lookup time the same way a draft's content already is, so it costs nothing to the immutability guarantee the fork lineage depends on. The resolver stays exactly as specified, unaware this happened; it only ever sees current-shape content. This trigger is what guarantees nothing ever breaks unattended — rendering, preview, incident re-renders, none of it depends on a human being present to approve anything.

A version ahead of Composer's deployed transformer chain fails loudly

If a lookup hits a schemaVersion ahead of what the deployed chain covers — the rollback case — it fails with a typed error rather than guessing at an unknown shape. This is the concrete mechanism by which Blair's go/no-go gate for rollback (Testing, Preview & Incident Response) protects something, not a new gap.

Trigger two: placement, only for a major-version-behind reference, only in the authoring UI. When a user is actively creating a new reference to an existing version — dragging an Element into a Scene, adding a Scene or Overlay to a Project — and that version's schemaVersion.major is behind the current major for its altitude, the UI surfaces a dialog: "This [Element/Scene] was built for an earlier schema version. Run migration to use?", the same pattern as opening a project file built for an older major application version. A minor-behind version never triggers this — by definition a minor bump didn't break compatibility — and resolves through the same silent upcast as trigger one. The dialog itself is a named UI requirement (Editing UI & Canvas).

What "run migration" does: an ordinary fork and commit, and it moves currentVersionId forward. Confirming the dialog forks the stale version through the existing fork mechanism: a new version is created, its content is the already-computed upcast result, parentVersionId records the lineage back to the pre-migration version, committedBy records that this commit was produced by a confirmed migration rather than a hand-authored edit (worth a distinguishing value, for audit granularity), and it commits at current schema. Critically, this is not a side commit only the new placement benefits from: the identity row's currentVersionId advances to point at it, exactly as after any ordinary content edit. The instance being placed then references this freshly committed version. This is the one case where the otherwise ephemeral resolve-time upcast becomes a permanent, committed, auditable version.

Reconciled with staleness: every other existing instance goes stale through the mechanism already built. The moment currentVersionId advances, every instance anywhere that still points at the pre-migration version has a mismatch against its identity's current version — which is already the entire definition of staleness (Fork, Rebase & Conflict Resolution). It surfaces the existing badge and can rebase through the existing confidence-scored auto-resolution and mandatory Approve Migration step, unchanged. This closes the gap the placement dialog alone leaves open — a reference nobody has newly placed, sitting quietly on the silent upcast, has nothing to rebase onto and nothing prompts anyone to look at it. The fix isn't a second staleness mechanism; it's making the migration action reachable from wherever an item's own identity is managed (its catalogue or library view), not only from the placement dialog (Editing UI & Canvas), so a major-behind item can be migrated forward on its own, purely to advance currentVersionId.

	Minor-behind	Major-behind
Render / resolve	Silent upcast	Silent upcast
New placement	Nothing	Dialog offers migration; never blocks
Confirmed migration	n/a — nothing ever forks or commits for a minor gap	Fork + commit at current schema; `currentVersionId` advances; every other instance goes stale via the badge

Confidence scoring needs no special case — the rebase check is agnostic to why a base changed (Fork, Rebase & Conflict Resolution).
The dialog never blocks placement. Dismissing it, or lacking permission to migrate, still lets placement succeed through the plain trigger-one upcast. Blocking placement behind migration would mean a High-tier user with composition privilege but no Element-edit permission — the ordinary BSR case — could be stopped from placing an Element purely because it sits on an old schema, which breaks the boundary that composing never requires reaching into an Element (Permissions & Enforcement). The dialog offers migration; it never gates on it.
Retiring an old transformer, later, is the same mechanism again. Once every reference to a stale version has been migrated forward, that upgradeVNtoVN+1 step becomes safe to drop from the chain; a reference deliberately left pinned on schema N blocks retiring N's transformer, since dropping it would turn that reference's trigger-one upcast into the loud failure above. A bulk "rebase everything still on schema N" job would just be trigger two run in bulk — optional future tooling, not designed now.

Does Composer extend the engine's format, or exist alongside it?¶

Whether schema versioning and a real migration process are a blocking prerequisite for Composer at all depends on an architectural decision that hasn't been made. The engine's own reshape extends the current template format in place under schemaVersion: 2, with schemaVersion: 1 documents rendering through a frozen legacy pipeline and no on-load transform anywhere (Resolver Engine → Schema-Version Gating). Composer's Element, Scene, and Project schema can either extend that v2 format further — the same tables, the same document shape, Composer's altitudes layered onto it — or exist as its own format alongside it, touching nothing the engine's documents depend on. Storage & Versioning treats both paths symmetrically.

Blair's call, not this brief's to resolve

Which of these Composer actually is hasn't been decided. It also decides whether a kill-switch fallback to a non-Composer render path exists at all (Testing, Preview & Incident Response). The resolution behaviour above holds either way; it's about how Composer's own references behave once committed, not which format they're expressed in.

NEO-1657's migration tooling, as it stands¶

A real, built codebase, with one distinction worth keeping honest: built and unit-tested is not the same claim as personally verified. NEO-1657 is a standing migration CLI (dry run by default, --apply to write, per-row failure reporting, resumable), migration run and history tables plus a Postgres RPC for the write path, a rollback CLI with an atomic, updated_at-guarded restore and a --force override, and an advisory, never-blocking visual-diff check built on @remotion/renderer's renderStill plus pixelmatch. Its unit suite passes, including a full dry-run / apply / rollback cycle against real production fixtures (a five-scene, 331-field template among them). Matthew has not personally run or tested any of it. Not yet pushed or opened as a PR; a database integration test exists but is gated off pending a run against a real branch. The engine brief records the ticket itself as backlog with no timeline, and the engine does not depend on it. Its relevance to Composer is as the bulk-migration tooling Composer's own upgradeVNtoVN+1 chain would run through if a stored-row rewrite is ever wanted — trigger two in bulk — not as a prerequisite for either format path.

Worth reconciling, not yet confirmed: this migration-time visual-diff check and the visual-regression system in Testing, Preview & Incident Response should very likely be the same underlying render-and-compare mechanism, reused a third way rather than a second, parallel renderStill + pixelmatch integration. The one deliberate difference is blocking policy, not mechanism: this one is advisory, appropriate for a bulk migration touching many templates at once, where the other is blocking in both of its trigger cases. The code could not be located in this repository checkout during this design pass to confirm the two converge.

Risks¶

The schema-fork decision stays open while foundations get built. Everything on this page holds either way, but storage shape and the incident kill switch both fork on it.
Deploying a new major transformer changes every live render on the old major at once. Trigger one upcasts silently across majors, so a new major transformer rewrites what every un-migrated live version renders as, with no pointer moving and therefore no rebase gate firing; only the fixture-based PR gate stands in the way, and it tests fixtures, not real Projects. Whether such a deploy should run render-and-compare over a sample of real live Projects is open (Testing, Preview & Incident Response).
A rollback that leaves committed versions ahead of the deployed transformer chain fails loudly by design; that's the right failure, but it means rollback is never free of a content-side consequence.

Open questions¶

Extend the engine's v2 format, or a separate format — Blair's call (above).
One schemaVersion axis or two — Composer's { major, minor } against the engine's 1 | 2 pipeline gate (The Resolver Engine Dependency).
Replaying old-shape changes against upcast content. A draft authored at 1.0 whose changes target a field a 2.0 transformer renames, opened after the bump, computes resolveDocument(upcast(parent.content), changes) with 1.0-shaped Changes against 2.0 content; a 1.0 fork rebased onto a 2.0 base replays the same way. changes is never upcast in storage, but the fold needs some rule for a cross-major replay — a Change-shape transformer applied at replay time, or "a draft or un-rebased fork on an older major migrates through trigger two before it can be resolved or rebased." Not decided.
Whether NEO-1657's visual-diff check converges with the visual-regression system — unconfirmed either way until both are built for real.