feat(mcp,editor): Option A+B storage + 10 agent deliverables (Phase 7)
Ships the combined filesystem/Supabase storage adapter + MCP scene lifecycle tools + Next.js API routes + editor /scene/[id] route, so an MCP save is directly openable at /scene/<id> without any injection hack. End-to-end verified: 10/10 e2e steps pass. Storage (A1/A2/A3): - SceneStore interface + error classes + slug helpers - FilesystemSceneStore at $PASCAL_DATA_DIR (defaults XDG/~/.pascal) with atomic writes, .index sidecar, optimistic locking - SupabaseSceneStore with scenes + scene_revisions tables, RLS migration SQL, mock-backed unit tests - createSceneStore(env) auto-selects based on SUPABASE_URL + SUPABASE_SERVICE_ROLE_KEY MCP tools (A4, A8, A9, A10): - save_scene / load_scene / list_scenes / delete_scene / rename_scene - list_templates / create_from_template (3 seed templates: empty-studio, two-bedroom, garden-house) - generate_variants (7 mutation kinds, seeded RNG, save=true|false) - photo_to_scene (vision sampling → scene graph → save) Editor (A5, A6): - /api/scenes + /api/scenes/[id] with RFC 7232 If-Match locking - /scene/[id] and /scenes route pages with save button, SceneLoader - Removed the window.__pascalScene dev injection hack Security + UX edges (A7, A8): - AssetUrl Zod validator: asset:// blob: data:image/ /path https: (http://localhost for dev) + PASCAL_ALLOWED_ASSET_ORIGINS env allowlist. Hardens scan.url, guide.url, item.asset.src, material.texture.url, MaterialMaps.*Map - Auto-frame camera on empty→non-empty scene transition (camera-controls:fit-scene emitter event) Shared utilities: - rehydrateSiteChildren() extracted to packages/mcp/src/lib/ and used by both create-from-template and generate-variants to work around the SiteNode.children-as-objects vs. ids inconsistency (CROSS_CUTTING §2) - Storage + MCP subpath exports added to packages/mcp/package.json (CROSS_CUTTING §4) Tests: 293 pass / 0 fail across 40 files (was 142 pre-Phase-7). Biome: clean. Phase-7 e2e script at packages/mcp/test-reports/phase7-e2e.ts: MCP HTTP + editor Next.js both point at $PASCAL_DATA_DIR = /tmp/pascal-e2e, save_scene from MCP, GET /api/scenes/<id> from editor server, /scenes list page renders all saved scenes, scene page renders SceneLoader, delete_scene works. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Adrian Perez
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
42bd05db9c
commit
e8d0b13ff5
@@ -0,0 +1,119 @@
|
||||
# Floor-plan photo to Pascal scene
|
||||
|
||||
The `photo_to_scene` orchestrator takes a single floor-plan photo and
|
||||
returns a saved, navigable Pascal scene. It chains vision (via MCP
|
||||
sampling) → scene build → save in one call, so an agent doesn't have to
|
||||
stitch three tools together manually.
|
||||
|
||||
> **Note:** `photo_to_scene` uses MCP sampling to call the host's model.
|
||||
> Hosts that do not advertise `sampling` capability will receive a
|
||||
> structured `sampling_unavailable` error; fall back to the text-only
|
||||
> `from_brief` prompt in that case.
|
||||
|
||||
## The brief
|
||||
|
||||
A user drops a photo of a hand-drawn floor plan into the chat and types:
|
||||
|
||||
> **User:** here's a floor plan photo, turn it into a Pascal scene.
|
||||
|
||||
## The tool call
|
||||
|
||||
The agent reads the attachment as a data URI and issues a single tool call:
|
||||
|
||||
```jsonc
|
||||
// tool: photo_to_scene
|
||||
{
|
||||
"name": "photo_to_scene",
|
||||
"arguments": {
|
||||
"image": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAYABgAAD...",
|
||||
"scaleHint": "1 cm = 1 m, approx 20 m²",
|
||||
"name": "Weekend flat"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Optional knobs:
|
||||
|
||||
- `save` (default `true`) — if `false`, the response includes `graph`
|
||||
inline instead of persisting to the `SceneStore`.
|
||||
- `defaultWallThickness` (default `0.2` m) — used when the vision model
|
||||
doesn't propose a per-wall thickness.
|
||||
- `defaultWallHeight` (default `2.6` m) — applied to every generated wall
|
||||
since the vision schema only captures 2D geometry.
|
||||
|
||||
## What happens under the hood
|
||||
|
||||
1. The orchestrator issues an MCP sampling request to the host with the
|
||||
image and a structured JSON-only system prompt, mirroring
|
||||
`analyze_floorplan_image`. The host's model returns walls, rooms, and
|
||||
approximate dimensions as JSON.
|
||||
2. The reply is validated against a strict Zod schema. Unparseable or
|
||||
schema-failing responses surface as `sampling_response_unparseable` /
|
||||
`sampling_response_invalid` MCP errors.
|
||||
3. A fresh `SceneGraph` is built using the core schema factories: a
|
||||
`site` → `building` → `level 0` skeleton, then one `WallNode` per
|
||||
vision wall and one `ZoneNode` per vision room. Each node is
|
||||
re-parsed with `AnyNode.safeParse`; invalid ones are dropped with a
|
||||
warning appended to `notes`.
|
||||
4. `bridge.setScene(...)` swaps the live scene so any follow-up MCP call
|
||||
(`find_nodes`, `measure`, `apply_patch`, ...) operates on the new
|
||||
geometry.
|
||||
5. If `save: true`, the graph is persisted via `SceneStore.save` and the
|
||||
response carries `sceneId` + `url: /scene/<id>`.
|
||||
|
||||
## The response
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"sceneId": "scene_01hx8a...",
|
||||
"url": "/scene/scene_01hx8a...",
|
||||
"walls": 4,
|
||||
"rooms": 1,
|
||||
"confidence": 0.82
|
||||
}
|
||||
```
|
||||
|
||||
When `save: false` instead:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"walls": 4,
|
||||
"rooms": 1,
|
||||
"confidence": 0.82,
|
||||
"graph": {
|
||||
"nodes": { /* flat id → node dict */ },
|
||||
"rootNodeIds": ["site_..."],
|
||||
"collections": {}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
If any wall or room failed schema validation, the response includes a
|
||||
`notes` string summarising what was dropped.
|
||||
|
||||
## Opening the scene
|
||||
|
||||
The user follows `url` in their browser:
|
||||
|
||||
```
|
||||
https://your-pascal-host/scene/scene_01hx8a...
|
||||
```
|
||||
|
||||
...and lands in the editor with the new scene loaded, camera auto-framed
|
||||
on the building footprint.
|
||||
|
||||
## Follow-up prompts
|
||||
|
||||
Because the bridge now holds the new scene, subsequent agent turns can
|
||||
operate on it without reloading:
|
||||
|
||||
> **User:** add a door on the south wall between Living and Kitchen.
|
||||
|
||||
The agent calls `find_nodes({ type: "wall" })`, picks the appropriate
|
||||
wall, and issues `cut_opening` — no extra wiring needed.
|
||||
|
||||
## Takeaways
|
||||
|
||||
- `photo_to_scene` is a one-shot primitive: one call, one scene.
|
||||
- Vision confidence is surfaced so the agent can warn the user.
|
||||
- v0.1 covers walls + zones; doors, windows, items are follow-up tools.
|
||||
Reference in New Issue
Block a user