DailyMark Docs

English

Русский
Srpski

English

Русский
Srpski

Appearance

Restaurant Menu Rich Media Implementation Plan ​

For Codex: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Attach safe, optimized video and 3D candidates to menu items, with external generation providers, explicit editorial approval, static-menu fallbacks, and measurable performance limits.

Architecture: A tenant-scoped rich-media job submits generation to provider-neutral video or 3D adapters and converges webhook/poll results through BullMQ. An independent validation worker probes and normalizes MP4/video posters or self-contained GLB/model posters before creating review candidates in object storage. Approval links assets to the draft; the static renderer emits lazy, accessible players that never block core menu content.

Tech Stack: NestJS, TypeORM/PostgreSQL with FORCE RLS, BullMQ/Redis, S3-compatible object storage, FFmpeg/ffprobe, glTF Transform, Next.js, <model-viewer>, Jest, Playwright, Lighthouse.

Global Constraints ​

  • Run python3 .agent/tools/recall.py "restaurant menu 3D video migration provider media validation performance failing test" before implementation and surface the result.
  • Obtain approval before applying the migration, installing @gltf-transform/core, @gltf-transform/functions, or @google/model-viewer, and adding FFmpeg/ffprobe to the API worker image.
  • Generated media is a candidate until owner/admin approval and a later explicit menu publish.
  • Video delivery format is MP4 with H.264 video, AAC or no audio, faststart, a JPEG/WebP poster, maximum 20 seconds, 1080p, and 15 MiB.
  • 3D delivery format is one self-contained GLB 2.0 plus a JPEG/WebP poster, maximum 12 MiB, 150,000 triangles, 8 materials, 16 textures, and 2048 pixels per texture edge.
  • Reject external URIs, scripts, unsupported extensions, cameras, lights, animation longer than 20 seconds, and malformed scene graphs.
  • Menu text, price, allergens, and primary image remain usable when media is unsupported, disabled, slow, or corrupt.
  • Default rich-media monthly tenant budget is zero and uses the generation budget reservation pattern from the AI-content plan.

Produced and Consumed Interfaces ​

ts
export type RichMediaKind = 'video' | 'model3d';
export type RichMediaJobState =
  | 'queued' | 'submitted' | 'processing' | 'validating'
  | 'needs_review' | 'approved' | 'rejected' | 'failed' | 'cancelled';

export interface RichMediaProvider<TKind extends RichMediaKind> {
  submit(input: {
    kind: TKind;
    sourceAssetIds: string[];
    prompt: string;
    callbackUrl: string;
  }): Promise<{ providerJobId: string; estimatedCostMinor: number }>;
  poll(providerJobId: string): Promise<RichMediaProviderResult>;
}

export interface RichMediaValidationResult {
  accepted: boolean;
  kind: RichMediaKind;
  outputObjectKey?: string;
  posterObjectKey?: string;
  metrics: Record<string, number | string | boolean>;
  violations: string[];
}

export interface RichMediaValidator {
  validate(input: { kind: RichMediaKind; sourceObjectKey: string }): Promise<RichMediaValidationResult>;
}

The workstream consumes approved source assets, ObjectStore, tenant generation budgets, draft asset linking, and publication snapshot contracts. It produces immutable video and model3d assets with required poster relations.

Task 1: Persist Rich-Media Jobs and Asset Metadata ​

Files:

  • Create: apps/api/src/menu/rich-media/entities/menu-rich-media-job.entity.ts

  • Create: apps/api/src/menu/rich-media/menu-rich-media.types.ts

  • Create: apps/api/src/db/migrations/1723700000000-MenuRichMedia.ts

  • Modify: apps/api/src/menu/assets/entities/menu-asset.entity.ts

  • Modify: apps/api/src/db/data-source.ts

  • Modify: apps/api/src/menu/menu.module.ts

  • Test: apps/api/src/menu/rich-media/menu-rich-media.entity.spec.ts

  • [ ] Step 1: Write failing entity and transition tests

Cover state transitions, terminal immutability, tenant idempotency, source/candidate relations, provider lineage, integer costs, media metrics schema, required poster for video/model3d, and FORCE RLS.

  • [ ] Step 2: Run and confirm failure
bash
bun test apps/api/src/menu/rich-media/menu-rich-media.entity.spec.ts
  • [ ] Step 3: Implement entity and migration

Create menu_rich_media_jobs, add asset kinds video, model3d, and poster, add metadata jsonb, and constrain approved rich assets to a same-tenant poster asset. Add tenant/state/created indexes and forced tenant RLS.

  • [ ] Step 4: Register runtime and CLI entities

  • [ ] Step 5: After migration approval, run tests and a migration round trip

bash
bun test apps/api/src/menu/rich-media/menu-rich-media.entity.spec.ts
bun run test:db:up
DATABASE_URL=postgres://app:app@localhost:54332/checklist_test bun run --cwd apps/api migration:run
DATABASE_URL=postgres://app:app@localhost:54332/checklist_test bun run --cwd apps/api migration:revert
DATABASE_URL=postgres://app:app@localhost:54332/checklist_test bun run --cwd apps/api migration:run
bun run test:db:down
  • [ ] Step 6: Commit
bash
git add apps/api/src/menu/rich-media apps/api/src/menu/assets/entities/menu-asset.entity.ts apps/api/src/db/data-source.ts apps/api/src/db/migrations/1723700000000-MenuRichMedia.ts apps/api/src/menu/menu.module.ts
git commit -m "feat(menu): persist rich media jobs"

Task 2: Add Provider-Neutral Video and 3D Adapters ​

Files:

  • Create: apps/api/src/menu/rich-media/providers/rich-media-provider.ts

  • Create: apps/api/src/menu/rich-media/providers/http-video-provider.adapter.ts

  • Create: apps/api/src/menu/rich-media/providers/http-model3d-provider.adapter.ts

  • Create: apps/api/src/menu/rich-media/providers/fake-rich-media-provider.adapter.ts

  • Modify: apps/api/src/config/env.schema.ts

  • Test: apps/api/src/menu/rich-media/providers/rich-media-provider.contract.spec.ts

  • [ ] Step 1: Write failing adapter contract tests

Cover submit, poll, signed callback, duplicate callback, out-of-order callback, provider rejection, timeout, cancellation, unsafe download URL, oversized response, and estimated/actual cost.

  • [ ] Step 2: Run and confirm failure
bash
bun test apps/api/src/menu/rich-media/providers/rich-media-provider.contract.spec.ts
  • [ ] Step 3: Implement adapters and configuration

Require separate base URLs/tokens plus MENU_RICH_MEDIA_WEBHOOK_SECRET. Accept callback timestamps within five minutes, compare HMAC in constant time, allow HTTPS downloads only from configured provider hosts, cap downloads at 50 MiB before validation, and never expose provider URLs to clients.

  • [ ] Step 4: Run contract tests
bash
bun test apps/api/src/menu/rich-media/providers/rich-media-provider.contract.spec.ts
  • [ ] Step 5: Commit
bash
git add apps/api/src/menu/rich-media/providers apps/api/src/config/env.schema.ts
git commit -m "feat(menu): add rich media providers"

Task 3: Normalize and Validate Video ​

Files:

  • Create: apps/api/src/menu/rich-media/validation/video-validator.service.ts

  • Create: apps/api/src/menu/rich-media/validation/media-process-runner.ts

  • Create: apps/api/src/menu/rich-media/fixtures/video-probes.json

  • Modify: apps/api/Dockerfile

  • Test: apps/api/src/menu/rich-media/validation/video-validator.spec.ts

  • [ ] Step 1: Write failing validation tests

Cover codec/container, duration, resolution, pixel format, audio codec, file size, corrupt/truncated file, decompression-bomb timeout, executable/polyglot input, rotation, poster creation, and deterministic metadata.

  • [ ] Step 2: Run and confirm failure
bash
bun test apps/api/src/menu/rich-media/validation/video-validator.spec.ts
  • [ ] Step 3: Add the approved FFmpeg runtime

Pin the package version in the worker image, run as a non-root user, prohibit shell interpolation, and execute argument arrays with CPU, memory, output-size, and wall-clock limits.

  • [ ] Step 4: Implement normalization

Probe first; transcode accepted input to bounded H.264/AAC MP4 with faststart, strip metadata, generate a poster at a representative frame, verify the normalized output again, and write immutable objects only after verification.

  • [ ] Step 5: Run validation tests
bash
bun test apps/api/src/menu/rich-media/validation/video-validator.spec.ts
  • [ ] Step 6: Commit
bash
git add apps/api/src/menu/rich-media/validation apps/api/src/menu/rich-media/fixtures/video-probes.json apps/api/Dockerfile
git commit -m "feat(menu): validate rich media video"

Task 4: Normalize and Validate GLB ​

Files:

  • Create: apps/api/src/menu/rich-media/validation/model3d-validator.service.ts

  • Create: apps/api/src/menu/rich-media/validation/model3d-metrics.ts

  • Create: apps/api/src/menu/rich-media/fixtures/model3d-metrics.json

  • Test: apps/api/src/menu/rich-media/validation/model3d-validator.spec.ts

  • [ ] Step 1: Write failing GLB validation tests

Cover magic/version, JSON/BIN bounds, external URIs, data URIs, unsupported extensions, triangle/material/texture/node/animation limits, missing scene, cyclic graph, NaN transforms, huge accessors, corrupt buffers, and poster requirement.

  • [ ] Step 2: Run and confirm failure
bash
bun test apps/api/src/menu/rich-media/validation/model3d-validator.spec.ts
  • [ ] Step 3: Implement bounded parsing and optimization

Parse GLB without network resolution, reject external references before transform, compute metrics, prune unused data, deduplicate resources, resize textures to the limit, reserialize one self-contained GLB, then revalidate from bytes.

  • [ ] Step 4: Create and validate the poster

Render through an isolated approved renderer with network disabled, or require the provider poster when the renderer is unavailable. Validate poster magic bytes, dimensions, size, and metadata stripping.

  • [ ] Step 5: Run GLB tests
bash
bun test apps/api/src/menu/rich-media/validation/model3d-validator.spec.ts
  • [ ] Step 6: Commit
bash
git add apps/api/src/menu/rich-media/validation apps/api/src/menu/rich-media/fixtures/model3d-metrics.json
git commit -m "feat(menu): validate GLB assets"

Task 5: Orchestrate, Budget, and Review Rich Media ​

Files:

  • Create: apps/api/src/menu/rich-media/menu-rich-media.controller.ts

  • Create: apps/api/src/menu/rich-media/menu-rich-media.service.ts

  • Create: apps/api/src/menu/rich-media/workers/menu-rich-media.processor.ts

  • Create: apps/api/src/menu/rich-media/menu-rich-media-review.service.ts

  • Test: apps/api/src/menu/rich-media/menu-rich-media.service.spec.ts

  • Test: apps/api/src/menu/rich-media/menu-rich-media-review.spec.ts

  • [ ] Step 1: Write failing lifecycle tests

Cover create/get/list/cancel, source-asset ownership, role denial, budget race, retry, callback/poll convergence, validation failure, approval/rejection, stale draft revision, duplicate approval, and cost reconciliation.

  • [ ] Step 2: Run and confirm failure
bash
bun test apps/api/src/menu/rich-media/menu-rich-media.service.spec.ts apps/api/src/menu/rich-media/menu-rich-media-review.spec.ts
  • [ ] Step 3: Implement endpoints and worker
text
POST /menus/:menuId/items/:itemId/rich-media-jobs
GET  /menus/:menuId/rich-media-jobs
GET  /menus/:menuId/rich-media-jobs/:jobId
POST /menus/:menuId/rich-media-jobs/:jobId/cancel
POST /menus/:menuId/rich-media-jobs/:jobId/approve
POST /menus/:menuId/rich-media-jobs/:jobId/reject

Reserve cost before submit, converge on tenant idempotency key, validate independently of provider status, and link asset plus poster to the current draft in one audited transaction on approval.

  • [ ] Step 4: Run lifecycle tests
bash
bun test apps/api/src/menu/rich-media/menu-rich-media.service.spec.ts apps/api/src/menu/rich-media/menu-rich-media-review.spec.ts
  • [ ] Step 5: Commit
bash
git add apps/api/src/menu/rich-media
git commit -m "feat(menu): orchestrate rich media review"

Task 6: Render Lazy, Accessible Players in Static Menus ​

Files:

  • Modify: packages/menu-contracts/src/index.ts

  • Create: apps/api/src/menu/publish/rich-media.ts

  • Modify: apps/api/src/menu/publish/menu-renderer.ts

  • Create: apps/api/src/menu/publish/static/rich-media.js

  • Create: apps/api/src/menu/publish/static/rich-media.css

  • Test: apps/api/src/menu/publish/rich-media.spec.ts

  • Test: apps/api/src/menu/publish/__fixtures__/rich-media-menu.expected.html

  • [ ] Step 1: Write failing renderer and browser tests

Assert poster-first markup, localized labels, video controls, muted/no autoplay, model fallback link, reduced-motion behavior, no media JS for menus without rich assets, lazy import after interaction, and missing/corrupt asset fallback.

  • [ ] Step 2: Run and confirm failure
bash
bun test apps/api/src/menu/publish/rich-media.spec.ts
  • [ ] Step 3: Extend the snapshot contract

Add optional { kind, url, posterUrl, bytes, alt } per item. Reject media without poster/alt and keep the extension backward-compatible for snapshot version 1 readers.

  • [ ] Step 4: Implement lazy players

Render a native <video> shell and load <model-viewer> only after the poster button intersects and the user activates it. Use explicit dimensions, CSP-compatible static files, and an ordinary asset link when WebGL is unavailable.

  • [ ] Step 5: Run renderer tests and size check
bash
bun test apps/api/src/menu/publish/rich-media.spec.ts
bun run --cwd apps/api typecheck

Expected: rich-media-free pages add zero bytes; poster shell CSS/JS stays under 8 KiB compressed, excluding the deferred model viewer.

  • [ ] Step 6: Commit
bash
git add packages/menu-contracts/src/index.ts apps/api/src/menu/publish
git commit -m "feat(menu): render lazy rich media"

Task 7: Build Management and Review UI ​

Files:

  • Create: apps/web/app/[locale]/app/menus/[menuId]/rich-media/page.tsx

  • Create: apps/web/ui/app/menus/rich-media/RichMediaJobForm.tsx

  • Create: apps/web/ui/app/menus/rich-media/RichMediaReview.tsx

  • Create: apps/web/ui/app/menus/rich-media/MediaPreview.tsx

  • Create: apps/web/ui/app/menus/rich-media/menuRichMediaClient.ts

  • Test: apps/web/ui/app/menus/rich-media/RichMediaReview.test.tsx

  • Test: apps/web/e2e/menu-rich-media.spec.ts

  • Modify: apps/web/messages/ru.json

  • Modify: apps/web/messages/en.json

  • Modify: apps/web/messages/sr.json

  • [ ] Step 1: Write failing UI and journey tests

Cover eligible source selection, cost estimate, zero budget, generation progress, video/model preview, validation metrics, warnings, approve/reject, alt text, keyboard flow, unsupported WebGL, and RU/EN/SR strings.

  • [ ] Step 2: Run and confirm failure
bash
bun test --cwd apps/web ui/app/menus/rich-media/RichMediaReview.test.tsx
  • [ ] Step 3: Implement pages and components

Show format and performance metrics before approval, require alt text, never autoplay, and provide poster/download fallback for unsupported clients.

  • [ ] Step 4: Run UI tests
bash
bun test --cwd apps/web ui/app/menus/rich-media/RichMediaReview.test.tsx
bun apps/web/node_modules/playwright/cli.js test apps/web/e2e/menu-rich-media.spec.ts
  • [ ] Step 5: Commit
bash
git add apps/web/app apps/web/ui/app/menus/rich-media apps/web/e2e/menu-rich-media.spec.ts apps/web/messages
git commit -m "feat(menu): add rich media studio"

Task 8: Prove Performance, Resilience, and Operations ​

Files:

  • Create: test/menu-rich-media.e2e.test.ts

  • Create: test/menu-rich-media-performance.e2e.test.ts

  • Create: docs-internal/menu-rich-media-operations.md

  • [ ] Step 1: Write failing end-to-end tests

Prove provider failure/retry, malicious media rejection, worker restart, duplicate callbacks, budget enforcement, tenant isolation, approval/publish separation, no-player fallback, and network failure after poster display.

  • [ ] Step 2: Add performance assertions

For a representative menu on emulated mid-tier mobile, assert rich media does not affect initial LCP by more than 100 ms, no video/GLB bytes load before interaction, and the page remains within the core renderer budgets.

  • [ ] Step 3: Write the runbook

Document provider credentials, FFmpeg/glTF versions, callback rotation, validation limits, job cancellation, provider disable switch, corrupt-asset response, reprocessing, cost reconciliation, and CDN purge rules.

  • [ ] Step 4: Run the acceptance suite
bash
bun test apps/api/src/menu/rich-media
bun test apps/api/src/menu/publish/rich-media.spec.ts
bun apps/web/node_modules/playwright/cli.js test apps/web/e2e/menu-rich-media.spec.ts
bun test test/menu-rich-media.e2e.test.ts test/menu-rich-media-performance.e2e.test.ts
  • [ ] Step 5: Commit
bash
git add test/menu-rich-media.e2e.test.ts test/menu-rich-media-performance.e2e.test.ts docs-internal/menu-rich-media-operations.md
git commit -m "test(menu): prove rich media delivery"

Workstream Acceptance Checklist ​

  • [ ] Provider submit, poll, callback, cancellation, retry, and cost paths converge once.
  • [ ] Video output meets the exact MP4, duration, resolution, size, poster, and metadata rules.
  • [ ] GLB output is self-contained and meets exact geometry, material, texture, animation, size, and poster rules.
  • [ ] Malformed, hostile, or oversized files fail before becoming assets.
  • [ ] No candidate changes a draft until explicit owner/admin approval.
  • [ ] Static pages stay usable without JS, WebGL, media support, or successful asset loading.
  • [ ] Rich media loads only after interaction and passes accessibility/performance budgets.
  • [ ] Tenant isolation, budget, retention, resilience, and runbook checks pass.