docs(mcp): add README, examples, and changelog
- README.md: install/quick start; configs for Claude Desktop, Claude Code, and Cursor; programmatic usage; tables covering all 21 tools, 4 resources, and 3 prompts; limitations; development commands. - CHANGELOG.md: 0.1.0 entry in Keep a Changelog format. - examples/generate-apartment.md: prose transcript using from_brief to build an 80 m² 2-bed apartment, showing apply_patch, set_zone, cut_opening, validate_scene. - examples/renovate-from-photos.md: prose transcript using the vision tools + renovation_from_photos prompt. - examples/embed-in-agent.ts: compilable TypeScript showing programmatic usage via InMemoryTransport. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Adrian Perez
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
441e97b2b6
commit
3406dad8f5
@@ -0,0 +1,29 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to `@pascal-app/mcp` will be documented in this file.
|
||||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [0.1.0] - 2026-04-18
|
||||
|
||||
### Added
|
||||
|
||||
- Initial release.
|
||||
- `SceneBridge` headless adapter for `@pascal-app/core` with RAF polyfill so
|
||||
the Zustand store and Zundo temporal middleware run cleanly in Node.
|
||||
- 19 MCP tools covering scene querying (`get_scene`, `get_node`,
|
||||
`describe_node`, `find_nodes`, `measure`), mutation (`apply_patch`,
|
||||
`create_level`, `create_wall`, `place_item`, `cut_opening`, `set_zone`,
|
||||
`duplicate_level`, `delete_node`), undo/redo (`undo`, `redo`), export
|
||||
(`export_json`, `export_glb`), validation (`validate_scene`,
|
||||
`check_collisions`), plus 2 vision tools (`analyze_floorplan_image`,
|
||||
`analyze_room_photo`) backed by MCP sampling.
|
||||
- 4 MCP resources: `pascal://scene/current`,
|
||||
`pascal://scene/current/summary`, `pascal://catalog/items`, and
|
||||
`pascal://constraints/{levelId}`.
|
||||
- 3 MCP prompts: `from_brief`, `iterate_on_feedback`, and
|
||||
`renovation_from_photos`.
|
||||
- stdio and Streamable HTTP transports.
|
||||
- `pascal-mcp` CLI binary with `--stdio`, `--http --port`, and `--scene`
|
||||
flags.
|
||||
@@ -0,0 +1,205 @@
|
||||
# @pascal-app/mcp
|
||||
|
||||
Model Context Protocol server for the Pascal 3D editor. Drives the
|
||||
`@pascal-app/core` scene graph from any MCP-compatible AI host.
|
||||
|
||||
The server runs headlessly in Node — no browser, no WebGPU, no React — and
|
||||
exposes the same scene mutations used by the editor UI (create walls, place
|
||||
items, cut openings, undo, etc.) as MCP tools, resources, and prompts.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
bun add @pascal-app/mcp # or: npm i @pascal-app/mcp
|
||||
```
|
||||
|
||||
`@pascal-app/core` is a peer dependency; Bun workspaces resolve it automatically.
|
||||
|
||||
## Quick start
|
||||
|
||||
Launch the server over stdio in one line:
|
||||
|
||||
```bash
|
||||
bunx pascal-mcp # or: npx pascal-mcp
|
||||
```
|
||||
|
||||
Load an initial scene from disk:
|
||||
|
||||
```bash
|
||||
pascal-mcp --stdio --scene ./my-scene.json
|
||||
```
|
||||
|
||||
Expose it as HTTP for remote hosts:
|
||||
|
||||
```bash
|
||||
pascal-mcp --http --port 8787
|
||||
```
|
||||
|
||||
## Claude Desktop config
|
||||
|
||||
Edit `~/Library/Application Support/Claude/claude_desktop_config.json`
|
||||
(macOS) or `%APPDATA%\Claude\claude_desktop_config.json` (Windows):
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"pascal": {
|
||||
"command": "bunx",
|
||||
"args": ["pascal-mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
If `bunx` isn't on your PATH, substitute `npx` or point `command` at the
|
||||
absolute path of the `pascal-mcp` binary inside your project.
|
||||
|
||||
## Claude Code config
|
||||
|
||||
Via the CLI:
|
||||
|
||||
```bash
|
||||
claude mcp add pascal bunx pascal-mcp
|
||||
```
|
||||
|
||||
Or add to `.mcp.json` at the repo root:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"pascal": {
|
||||
"command": "bunx",
|
||||
"args": ["pascal-mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Cursor config
|
||||
|
||||
In Cursor settings (`settings.json`):
|
||||
|
||||
```json
|
||||
{
|
||||
"mcp.servers": {
|
||||
"pascal": {
|
||||
"command": "bunx",
|
||||
"args": ["pascal-mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Programmatic use
|
||||
|
||||
Embed the server in your own Node process using the in-memory transport. The
|
||||
example below runs a full client/server pair inside a single script — useful
|
||||
for agent frameworks and tests.
|
||||
|
||||
```ts
|
||||
import { createPascalMcpServer, SceneBridge } from '@pascal-app/mcp'
|
||||
import { Client } from '@modelcontextprotocol/sdk/client/index.js'
|
||||
import { InMemoryTransport } from '@modelcontextprotocol/sdk/inMemory.js'
|
||||
|
||||
const bridge = new SceneBridge()
|
||||
bridge.loadDefault()
|
||||
const server = createPascalMcpServer({ bridge })
|
||||
|
||||
const [srvT, cliT] = InMemoryTransport.createLinkedPair()
|
||||
const client = new Client({ name: 'my-agent', version: '0.1.0' })
|
||||
await Promise.all([server.connect(srvT), client.connect(cliT)])
|
||||
|
||||
const tools = await client.listTools()
|
||||
console.log('available tools:', tools.tools.map((t) => t.name))
|
||||
|
||||
const scene = await client.callTool({ name: 'get_scene', arguments: {} })
|
||||
console.log(scene)
|
||||
```
|
||||
|
||||
See [`examples/embed-in-agent.ts`](./examples/embed-in-agent.ts) for a
|
||||
compilable version.
|
||||
|
||||
## Tools
|
||||
|
||||
All tools validate their inputs and outputs with Zod. Mutation tools are
|
||||
captured by Zundo's temporal middleware as a single undoable step.
|
||||
|
||||
| Name | Purpose | Key input | Output |
|
||||
| --- | --- | --- | --- |
|
||||
| `get_scene` | Return the full scene graph. | — | `{ nodes, rootNodeIds, collections }` |
|
||||
| `get_node` | Fetch a node by id. | `{ id }` | the node, or `InvalidParams` if not found |
|
||||
| `describe_node` | Node summary with ancestry, children count and properties. | `{ id }` | `{ id, type, parentId, ancestry[], childrenCount, properties, description }` |
|
||||
| `find_nodes` | Filter nodes by type / parent / zone / level. | `{ type?, parentId?, zoneId?, levelId? }` | `{ nodes: AnyNode[] }` |
|
||||
| `measure` | Distance between two nodes; area when applicable. | `{ fromId, toId }` | `{ distanceMeters, areaSqMeters?, units: 'meters' }` |
|
||||
| `apply_patch` | Batched create/update/delete/move, validated and dry-run before commit. | `{ patches: Patch[] }` | `{ applied: number }` |
|
||||
| `create_level` | Add a new level to a building. | `{ buildingId, elevation, height, label? }` | `{ levelId }` |
|
||||
| `create_wall` | Add a wall to a level. | `{ levelId, start, end, thickness?, height? }` | `{ wallId }` |
|
||||
| `place_item` | Place a catalog item on a slab, ceiling, or wall with placement validation. | `{ catalogItemId, targetNodeId, position, rotation? }` | `{ itemId }` or `{ error: 'invalid_placement', reason }` |
|
||||
| `cut_opening` | Cut a door or window opening into a wall. | `{ wallId, type: 'door' \| 'window', position, width, height }` | `{ openingId }` |
|
||||
| `set_zone` | Create a zone/room polygon on a level. | `{ levelId, polygon, label, properties? }` | `{ zoneId }` |
|
||||
| `duplicate_level` | Clone a level and all of its descendants. | `{ levelId }` | `{ newLevelId, newNodeIds[] }` |
|
||||
| `delete_node` | Delete a node; cascades when `cascade: true`. | `{ id, cascade? }` | `{ deletedIds: [] }` |
|
||||
| `undo` | Step back through temporal history. | `{ steps? }` | `{ undone: number }` |
|
||||
| `redo` | Step forward through temporal history. | `{ steps? }` | `{ redone: number }` |
|
||||
| `export_json` | Serialize the scene graph as JSON. | `{ pretty? }` | `{ json: string }` |
|
||||
| `export_glb` | Stubbed: GLB export requires the browser renderer. | — | throws `not_implemented` |
|
||||
| `validate_scene` | Zod-validate every node and parent-child integrity. | — | `{ valid, errors: { nodeId, path, message }[] }` |
|
||||
| `check_collisions` | Find overlapping items and out-of-bounds placements. | `{ levelId? }` | `{ collisions: { aId, bId, kind }[] }` |
|
||||
| `analyze_floorplan_image` | Vision tool: extract walls, rooms, and approximate dimensions from a floorplan image. | `{ image, scaleHint? }` | `{ walls, rooms, approximateDimensions, confidence }` |
|
||||
| `analyze_room_photo` | Vision tool: extract approximate dimensions and fixtures from a room photo. | `{ image }` | `{ approximateDimensions, identifiedFixtures, identifiedWindows }` |
|
||||
|
||||
The vision tools require the MCP host to support the sampling capability
|
||||
(`createMessage`). Hosts that don't will see a structured
|
||||
`sampling_unavailable` error.
|
||||
|
||||
## Resources
|
||||
|
||||
| URI | MIME | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `pascal://scene/current` | `application/json` | Full `{ nodes, rootNodeIds, collections }` snapshot. |
|
||||
| `pascal://scene/current/summary` | `text/markdown` | Human-readable summary with node counts, bounding box, and level areas. |
|
||||
| `pascal://catalog/items` | `application/json` | Item catalog; returns `{ status: 'catalog_unavailable', items: [] }` in headless mode if no catalog is provided. |
|
||||
| `pascal://constraints/{levelId}` | `application/json` | Slab footprints and wall polygons for the given level — useful as planner context. |
|
||||
|
||||
## Prompts
|
||||
|
||||
| Name | Args | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `from_brief` | `{ brief: string, constraints?: string }` | Guided workflow for turning a prose brief (e.g. "2-bed apartment in 80 m²") into an incremental sequence of `apply_patch` calls starting from an empty site. |
|
||||
| `iterate_on_feedback` | `{ feedback: string }` | Minimal-diff instructions: examine the current scene, then propose the smallest patch set that satisfies the feedback. |
|
||||
| `renovation_from_photos` | `{ currentPhotos: string[], referencePhotos: string[], goals: string }` | Chains the vision tools with the scene mutation tools to produce a renovation plan grounded in photos. |
|
||||
|
||||
## Limitations
|
||||
|
||||
- `export_glb` returns `not_implemented`. GLB export depends on the Three.js
|
||||
renderer and isn't reachable headlessly without a large additional effort.
|
||||
- Vision tools require MCP host sampling support. Claude Desktop supports
|
||||
this; some MCP clients don't.
|
||||
- Systems (wall mitering, slab triangulation, CSG cutouts, roof / stair
|
||||
generation) run inside React hooks in the editor. Headless mode doesn't
|
||||
regenerate derived geometry — but all node data remains fully manipulable.
|
||||
Consumers that need rendered geometry run `@pascal-app/viewer` in a browser
|
||||
host.
|
||||
- Core's `loadAssetUrl` / `saveAsset` are browser-only; items that reference
|
||||
`asset://<id>` URLs aren't resolvable in Node. Supply absolute URLs or
|
||||
`data:` URLs for item assets if you need them usable outside the browser.
|
||||
- `dirtyNodes` accumulates in headless mode because no renderer consumes it.
|
||||
Call `bridge.flushDirty()` if observability matters to your consumer.
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
bun install
|
||||
bun run --cwd packages/mcp build
|
||||
bun test
|
||||
```
|
||||
|
||||
Smoke-test the stdio binary end-to-end:
|
||||
|
||||
```bash
|
||||
bun run --cwd packages/mcp smoke
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
@@ -0,0 +1,82 @@
|
||||
/**
|
||||
* Programmatic `@pascal-app/mcp` usage.
|
||||
*
|
||||
* Runs a full MCP client/server pair over the in-memory transport inside a
|
||||
* single Node process. Useful for agent frameworks and tests that want to
|
||||
* drive Pascal without spawning a subprocess.
|
||||
*
|
||||
* Compile with the package's `tsc --build`, or run directly with Bun:
|
||||
*
|
||||
* bun run packages/mcp/examples/embed-in-agent.ts
|
||||
*/
|
||||
|
||||
import { Client } from '@modelcontextprotocol/sdk/client/index.js'
|
||||
import { InMemoryTransport } from '@modelcontextprotocol/sdk/inMemory.js'
|
||||
import { createPascalMcpServer, SceneBridge } from '@pascal-app/mcp'
|
||||
|
||||
async function main(): Promise<void> {
|
||||
// 1. Spin up the headless bridge. `loadDefault()` seeds a Site → Building →
|
||||
// Level stack so the client has something to query immediately.
|
||||
const bridge = new SceneBridge()
|
||||
bridge.loadDefault()
|
||||
const server = createPascalMcpServer({ bridge })
|
||||
|
||||
// 2. Link the server to an in-memory client. Exactly the same API surface
|
||||
// as the stdio / HTTP transports, but without any process boundary.
|
||||
const [srvT, cliT] = InMemoryTransport.createLinkedPair()
|
||||
const client = new Client({ name: 'my-agent', version: '0.1.0' })
|
||||
await Promise.all([server.connect(srvT), client.connect(cliT)])
|
||||
|
||||
// 3. Discover available capabilities.
|
||||
const tools = await client.listTools()
|
||||
console.log(
|
||||
'available tools:',
|
||||
tools.tools.map((t) => t.name),
|
||||
)
|
||||
|
||||
// 4. Inspect the current scene.
|
||||
const scene = await client.callTool({ name: 'get_scene', arguments: {} })
|
||||
console.log('scene snapshot:', JSON.stringify(scene, null, 2))
|
||||
|
||||
// 5. Find the default level, create a 5 m wall, and undo it.
|
||||
const levels = await client.callTool({
|
||||
name: 'find_nodes',
|
||||
arguments: { type: 'level' },
|
||||
})
|
||||
const levelId = (levels.structuredContent as { nodes: Array<{ id: string }> }).nodes[0]?.id
|
||||
|
||||
if (levelId) {
|
||||
const created = await client.callTool({
|
||||
name: 'create_wall',
|
||||
arguments: {
|
||||
levelId,
|
||||
start: [0, 0],
|
||||
end: [5, 0],
|
||||
thickness: 0.2,
|
||||
height: 2.5,
|
||||
},
|
||||
})
|
||||
console.log('created wall:', created.structuredContent)
|
||||
|
||||
const undone = await client.callTool({ name: 'undo', arguments: { steps: 1 } })
|
||||
console.log('undone:', undone.structuredContent)
|
||||
}
|
||||
|
||||
// 6. Validate and export.
|
||||
const validation = await client.callTool({ name: 'validate_scene', arguments: {} })
|
||||
console.log('validation:', validation.structuredContent)
|
||||
|
||||
const exported = await client.callTool({
|
||||
name: 'export_json',
|
||||
arguments: { pretty: true },
|
||||
})
|
||||
console.log('export size:', (exported.structuredContent as { json: string }).json.length)
|
||||
|
||||
await client.close()
|
||||
await server.close()
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error(err)
|
||||
process.exit(1)
|
||||
})
|
||||
@@ -0,0 +1,242 @@
|
||||
# Generate a 2-bed apartment from a brief
|
||||
|
||||
This example walks through a realistic session with an MCP host (Claude
|
||||
Desktop, Claude Code, or Cursor) that has `pascal-mcp` configured. The agent
|
||||
uses the `from_brief` prompt to turn a short brief into a concrete scene.
|
||||
|
||||
## The brief
|
||||
|
||||
> **User:** Claude, create a 2-bedroom 1-bath apartment in 80 m² in Spain.
|
||||
|
||||
The host UI lets the user select the **`from_brief`** prompt and fills in:
|
||||
|
||||
```text
|
||||
brief: "2-bedroom 1-bath apartment in 80 m² in Spain, open-plan living /
|
||||
kitchen, bathroom on the interior wall"
|
||||
constraints: "Spanish building regulations; ceiling height 2.5 m"
|
||||
```
|
||||
|
||||
## What the agent does
|
||||
|
||||
The prompt returns a system message instructing the agent to start from an
|
||||
empty site, read the current scene, and emit incremental `apply_patch` calls.
|
||||
The agent proceeds roughly like this:
|
||||
|
||||
### 1. Inspect the current scene
|
||||
|
||||
```jsonc
|
||||
// tool: get_scene
|
||||
{ "name": "get_scene", "arguments": {} }
|
||||
```
|
||||
|
||||
Response (trimmed):
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"nodes": {
|
||||
"site-1": { "type": "site", "id": "site-1", "children": [/* ... */] },
|
||||
"building-1": { "type": "building", "id": "building-1", "parentId": "site-1" },
|
||||
"level-1": { "type": "level", "id": "level-1", "parentId": "building-1",
|
||||
"elevation": 0, "height": 2.5 }
|
||||
},
|
||||
"rootNodeIds": ["site-1"]
|
||||
}
|
||||
```
|
||||
|
||||
The default scene is a Site → Building → Level stack with no walls. The
|
||||
agent decides to work on `level-1` and targets a 10 m × 8 m = 80 m² outline.
|
||||
|
||||
### 2. Create the perimeter walls
|
||||
|
||||
The agent chooses a rectangular outline with its origin at (0, 0):
|
||||
|
||||
```jsonc
|
||||
// tool: apply_patch
|
||||
{
|
||||
"name": "apply_patch",
|
||||
"arguments": {
|
||||
"patches": [
|
||||
{ "op": "create", "parentId": "level-1",
|
||||
"node": { "type": "wall", "start": [0, 0], "end": [10, 0],
|
||||
"thickness": 0.2, "height": 2.5 } },
|
||||
{ "op": "create", "parentId": "level-1",
|
||||
"node": { "type": "wall", "start": [10, 0], "end": [10, 8],
|
||||
"thickness": 0.2, "height": 2.5 } },
|
||||
{ "op": "create", "parentId": "level-1",
|
||||
"node": { "type": "wall", "start": [10, 8], "end": [0, 8],
|
||||
"thickness": 0.2, "height": 2.5 } },
|
||||
{ "op": "create", "parentId": "level-1",
|
||||
"node": { "type": "wall", "start": [0, 8], "end": [0, 0],
|
||||
"thickness": 0.2, "height": 2.5 } }
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```jsonc
|
||||
{ "applied": 4 }
|
||||
```
|
||||
|
||||
### 3. Create interior partitions
|
||||
|
||||
Two bedrooms on the east side, bathroom on the interior wall, open-plan
|
||||
living / kitchen on the west.
|
||||
|
||||
```jsonc
|
||||
// tool: apply_patch
|
||||
{
|
||||
"name": "apply_patch",
|
||||
"arguments": {
|
||||
"patches": [
|
||||
{ "op": "create", "parentId": "level-1",
|
||||
"node": { "type": "wall", "start": [5.5, 0], "end": [5.5, 8],
|
||||
"thickness": 0.15, "height": 2.5 } },
|
||||
{ "op": "create", "parentId": "level-1",
|
||||
"node": { "type": "wall", "start": [5.5, 4], "end": [10, 4],
|
||||
"thickness": 0.15, "height": 2.5 } },
|
||||
{ "op": "create", "parentId": "level-1",
|
||||
"node": { "type": "wall", "start": [5.5, 5.5], "end": [8, 5.5],
|
||||
"thickness": 0.15, "height": 2.5 } },
|
||||
{ "op": "create", "parentId": "level-1",
|
||||
"node": { "type": "wall", "start": [8, 4], "end": [8, 5.5],
|
||||
"thickness": 0.15, "height": 2.5 } }
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Define zones
|
||||
|
||||
The agent declares the rooms so later queries and item placement can target
|
||||
them by name:
|
||||
|
||||
```jsonc
|
||||
// tool: set_zone (called once per zone)
|
||||
{
|
||||
"name": "set_zone",
|
||||
"arguments": {
|
||||
"levelId": "level-1",
|
||||
"label": "Living / Kitchen",
|
||||
"polygon": [[0, 0], [5.5, 0], [5.5, 8], [0, 8]]
|
||||
}
|
||||
}
|
||||
// → { "zoneId": "zone-living" }
|
||||
|
||||
{
|
||||
"name": "set_zone",
|
||||
"arguments": {
|
||||
"levelId": "level-1",
|
||||
"label": "Bedroom 1",
|
||||
"polygon": [[5.5, 0], [10, 0], [10, 4], [5.5, 4]]
|
||||
}
|
||||
}
|
||||
// → { "zoneId": "zone-bed1" }
|
||||
|
||||
{
|
||||
"name": "set_zone",
|
||||
"arguments": {
|
||||
"levelId": "level-1",
|
||||
"label": "Bedroom 2",
|
||||
"polygon": [[5.5, 5.5], [10, 5.5], [10, 8], [5.5, 8]]
|
||||
}
|
||||
}
|
||||
// → { "zoneId": "zone-bed2" }
|
||||
|
||||
{
|
||||
"name": "set_zone",
|
||||
"arguments": {
|
||||
"levelId": "level-1",
|
||||
"label": "Bathroom",
|
||||
"polygon": [[5.5, 4], [8, 4], [8, 5.5], [5.5, 5.5]]
|
||||
}
|
||||
}
|
||||
// → { "zoneId": "zone-bath" }
|
||||
```
|
||||
|
||||
### 5. Cut doors and windows
|
||||
|
||||
The agent uses `cut_opening` to add entry doors on each interior partition
|
||||
and windows on the south and east façades:
|
||||
|
||||
```jsonc
|
||||
// tool: cut_opening (called once per opening)
|
||||
{
|
||||
"name": "cut_opening",
|
||||
"arguments": {
|
||||
"wallId": "wall-south", // perimeter wall [0,0] → [10,0]
|
||||
"type": "window",
|
||||
"position": 0.25, // 25% along centerline
|
||||
"width": 1.2,
|
||||
"height": 1.2
|
||||
}
|
||||
}
|
||||
// → { "openingId": "window-south-1" }
|
||||
```
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"name": "cut_opening",
|
||||
"arguments": {
|
||||
"wallId": "wall-bed1", // partition wall to Bedroom 1
|
||||
"type": "door",
|
||||
"position": 0.4,
|
||||
"width": 0.9,
|
||||
"height": 2.1
|
||||
}
|
||||
}
|
||||
// → { "openingId": "door-bed1" }
|
||||
```
|
||||
|
||||
The agent repeats this for Bedroom 2's door, the bathroom door, and two
|
||||
more windows on the east façade.
|
||||
|
||||
### 6. Validate and report
|
||||
|
||||
```jsonc
|
||||
// tool: validate_scene
|
||||
{ "name": "validate_scene", "arguments": {} }
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```jsonc
|
||||
{ "valid": true, "errors": [] }
|
||||
```
|
||||
|
||||
The agent then reads the scene summary for its response to the user:
|
||||
|
||||
```jsonc
|
||||
// resource: pascal://scene/current/summary
|
||||
{ "uri": "pascal://scene/current/summary" }
|
||||
```
|
||||
|
||||
The host displays the returned Markdown: 1 site, 1 building, 1 level, 8
|
||||
walls, 4 zones, 3 doors, 3 windows; usable area ~78 m²; perimeter ~36 m.
|
||||
|
||||
### 7. Iterate
|
||||
|
||||
The user follows up:
|
||||
|
||||
> **User:** Swap the bathroom and bedroom 2 — I want the bathroom near the
|
||||
> entrance.
|
||||
|
||||
The agent loads the `iterate_on_feedback` prompt and issues a single
|
||||
`apply_patch` that updates the polygon of `zone-bath` and `zone-bed2` and
|
||||
moves the corresponding partition walls. Because mutation goes through the
|
||||
Zustand store, the user can `undo` the change if they dislike it:
|
||||
|
||||
```jsonc
|
||||
{ "name": "undo", "arguments": { "steps": 1 } }
|
||||
// → { "undone": 1 }
|
||||
```
|
||||
|
||||
## Takeaways
|
||||
|
||||
- Mutations batch inside a single `apply_patch` so that `undo` rolls back
|
||||
the whole logical change.
|
||||
- Zones are not walls — they're polygon annotations that make later queries
|
||||
(`find_nodes({ zoneId })`) and planning steps much easier for the agent.
|
||||
- The agent never needs to speak to `@pascal-app/viewer`: everything the
|
||||
host sees flows through tools + resources + prompts.
|
||||
@@ -0,0 +1,202 @@
|
||||
# Renovate an existing flat from photos
|
||||
|
||||
This example shows how an agent can combine the `renovation_from_photos`
|
||||
prompt with the `analyze_floorplan_image` and `analyze_room_photo` vision
|
||||
tools to propose a renovation plan grounded in real photos.
|
||||
|
||||
> **Note:** the vision tools use MCP sampling (`createMessage`), which
|
||||
> Claude Desktop supports today. Hosts without sampling support will get a
|
||||
> structured `sampling_unavailable` error; fall back to the text-only
|
||||
> `from_brief` prompt in that case.
|
||||
|
||||
## The brief
|
||||
|
||||
The user drops four photos into the chat:
|
||||
|
||||
1. A floorplan PDF page (exported as PNG).
|
||||
2. A photo of the current living room.
|
||||
3. A photo of the current kitchen.
|
||||
4. An inspirational photo from a magazine — a minimal Scandinavian loft.
|
||||
|
||||
And types:
|
||||
|
||||
> **User:** Claude, help me plan a renovation. Here's the current plan and
|
||||
> two room photos. I want something like this Scandinavian reference —
|
||||
> open-plan, neutral tones, keep the footprint.
|
||||
|
||||
## What the agent does
|
||||
|
||||
The host loads the **`renovation_from_photos`** prompt:
|
||||
|
||||
```text
|
||||
currentPhotos: ["data:image/png;base64,...", "data:image/jpeg;base64,..."]
|
||||
referencePhotos: ["data:image/jpeg;base64,..."]
|
||||
goals: "Open-plan living/kitchen, neutral tones, keep the footprint."
|
||||
```
|
||||
|
||||
The prompt tells the agent to (1) analyze the floorplan, (2) analyze each
|
||||
room photo, (3) seed a scene from the floorplan, (4) compare against the
|
||||
reference, and (5) propose patches.
|
||||
|
||||
### 1. Extract the floorplan
|
||||
|
||||
```jsonc
|
||||
// tool: analyze_floorplan_image
|
||||
{
|
||||
"name": "analyze_floorplan_image",
|
||||
"arguments": {
|
||||
"image": "data:image/png;base64,iVBORw0KGgoAAAANS...",
|
||||
"scaleHint": "1 m grid, total footprint ~9.5 m × 7 m"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Under the hood, the tool issues an MCP sampling request to the host with
|
||||
the image and a structured prompt asking for walls, rooms, and
|
||||
approximate dimensions. The response is validated against the tool's
|
||||
output schema:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"walls": [
|
||||
{ "start": [0, 0], "end": [9.5, 0], "thickness": 0.25 },
|
||||
{ "start": [9.5, 0], "end": [9.5, 7], "thickness": 0.25 },
|
||||
{ "start": [9.5, 7], "end": [0, 7], "thickness": 0.25 },
|
||||
{ "start": [0, 7], "end": [0, 0], "thickness": 0.25 },
|
||||
{ "start": [4.5, 0], "end": [4.5, 7], "thickness": 0.15 },
|
||||
{ "start": [4.5, 3.5], "end": [9.5, 3.5], "thickness": 0.15 }
|
||||
],
|
||||
"rooms": [
|
||||
{ "label": "Living", "polygon": [[0, 0], [4.5, 0], [4.5, 7], [0, 7]] },
|
||||
{ "label": "Kitchen", "polygon": [[4.5, 0], [9.5, 0], [9.5, 3.5], [4.5, 3.5]] },
|
||||
{ "label": "Bedroom", "polygon": [[4.5, 3.5], [9.5, 3.5], [9.5, 7], [4.5, 7]] }
|
||||
],
|
||||
"approximateDimensions": { "widthMeters": 9.5, "depthMeters": 7, "areaSqMeters": 66.5 },
|
||||
"confidence": 0.82
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Analyze the room photos
|
||||
|
||||
```jsonc
|
||||
// tool: analyze_room_photo
|
||||
{
|
||||
"name": "analyze_room_photo",
|
||||
"arguments": { "image": "data:image/jpeg;base64,/9j/4AAQ..." }
|
||||
}
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"approximateDimensions": { "widthMeters": 4.4, "depthMeters": 5.8, "heightMeters": 2.5 },
|
||||
"identifiedFixtures": [
|
||||
{ "kind": "sofa", "approximatePosition": [2.2, 3.5] },
|
||||
{ "kind": "coffee-table", "approximatePosition": [2.2, 2.4] },
|
||||
{ "kind": "tv-unit", "approximatePosition": [0.3, 2.0] }
|
||||
],
|
||||
"identifiedWindows": [
|
||||
{ "wallHint": "south", "approximateWidth": 1.4, "approximateHeight": 1.5 }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
The kitchen photo is analyzed the same way.
|
||||
|
||||
### 3. Seed the scene
|
||||
|
||||
The agent reads `get_scene`, confirms the default empty Site → Building →
|
||||
Level is present, and then batch-creates walls matching the floorplan:
|
||||
|
||||
```jsonc
|
||||
// tool: apply_patch
|
||||
{
|
||||
"name": "apply_patch",
|
||||
"arguments": {
|
||||
"patches": [
|
||||
{ "op": "create", "parentId": "level-1",
|
||||
"node": { "type": "wall", "start": [0, 0], "end": [9.5, 0],
|
||||
"thickness": 0.25, "height": 2.5 } },
|
||||
/* ...remaining perimeter + partition walls from the vision result... */
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The agent then calls `set_zone` three times to seed the Living / Kitchen /
|
||||
Bedroom polygons from the floorplan rooms.
|
||||
|
||||
### 4. Cut the identified openings
|
||||
|
||||
For each window the vision tool reported, the agent calls `cut_opening`
|
||||
against the corresponding perimeter wall:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"name": "cut_opening",
|
||||
"arguments": {
|
||||
"wallId": "wall-south",
|
||||
"type": "window",
|
||||
"position": 0.5,
|
||||
"width": 1.4,
|
||||
"height": 1.5
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Propose the renovation
|
||||
|
||||
Guided by the reference photo's analysis (bright neutrals, open plan,
|
||||
minimal furnishing), the agent proposes a single logical patch:
|
||||
|
||||
- Remove the partition wall between Living and Kitchen.
|
||||
- Relocate the kitchen island further west.
|
||||
- Delete the bulky TV unit item; leave the sofa and coffee table.
|
||||
- Re-label the merged zone `"Open-Plan Living / Kitchen"`.
|
||||
|
||||
All of that goes into one `apply_patch`:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"name": "apply_patch",
|
||||
"arguments": {
|
||||
"patches": [
|
||||
{ "op": "delete", "id": "wall-partition-living-kitchen", "cascade": false },
|
||||
{ "op": "update", "id": "zone-living", "data": { "label": "Open-Plan Living / Kitchen",
|
||||
"polygon": [[0, 0], [9.5, 0],
|
||||
[9.5, 3.5], [0, 3.5]] } },
|
||||
{ "op": "delete", "id": "zone-kitchen", "cascade": false }
|
||||
/* + item moves / deletes for the TV unit etc. */
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The user can walk back with `undo`; `redo` returns them to the proposal.
|
||||
|
||||
### 6. Sanity-check
|
||||
|
||||
```jsonc
|
||||
// tool: validate_scene
|
||||
{ "name": "validate_scene", "arguments": {} }
|
||||
// → { "valid": true, "errors": [] }
|
||||
|
||||
// tool: check_collisions
|
||||
{ "name": "check_collisions", "arguments": { "levelId": "level-1" } }
|
||||
// → { "collisions": [] }
|
||||
```
|
||||
|
||||
The agent reports a summary of the changes plus the approximate new
|
||||
usable area (from the summary resource), and the user opens the scene in
|
||||
`@pascal-app/viewer` to see the renovated 3D layout.
|
||||
|
||||
## Takeaways
|
||||
|
||||
- The vision tools only return **data**. They don't mutate the scene —
|
||||
the agent is explicit about every structural change via `apply_patch`.
|
||||
- Photos supply priors (approximate dimensions, fixture types) that a
|
||||
brief-only workflow can't. Combine them with `from_brief`-style
|
||||
prompts when the user has both a reference and concrete text goals.
|
||||
- All renovation steps are a single temporal step per patch, so the user
|
||||
can compare before/after with `undo` / `redo`.
|
||||
Reference in New Issue
Block a user