docs(mcp): add README, examples, and changelog
- README.md: install/quick start; configs for Claude Desktop, Claude Code, and Cursor; programmatic usage; tables covering all 21 tools, 4 resources, and 3 prompts; limitations; development commands. - CHANGELOG.md: 0.1.0 entry in Keep a Changelog format. - examples/generate-apartment.md: prose transcript using from_brief to build an 80 m² 2-bed apartment, showing apply_patch, set_zone, cut_opening, validate_scene. - examples/renovate-from-photos.md: prose transcript using the vision tools + renovation_from_photos prompt. - examples/embed-in-agent.ts: compilable TypeScript showing programmatic usage via InMemoryTransport. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Adrian Perez
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
441e97b2b6
commit
3406dad8f5
@@ -0,0 +1,202 @@
|
||||
# Renovate an existing flat from photos
|
||||
|
||||
This example shows how an agent can combine the `renovation_from_photos`
|
||||
prompt with the `analyze_floorplan_image` and `analyze_room_photo` vision
|
||||
tools to propose a renovation plan grounded in real photos.
|
||||
|
||||
> **Note:** the vision tools use MCP sampling (`createMessage`), which
|
||||
> Claude Desktop supports today. Hosts without sampling support will get a
|
||||
> structured `sampling_unavailable` error; fall back to the text-only
|
||||
> `from_brief` prompt in that case.
|
||||
|
||||
## The brief
|
||||
|
||||
The user drops four photos into the chat:
|
||||
|
||||
1. A floorplan PDF page (exported as PNG).
|
||||
2. A photo of the current living room.
|
||||
3. A photo of the current kitchen.
|
||||
4. An inspirational photo from a magazine — a minimal Scandinavian loft.
|
||||
|
||||
And types:
|
||||
|
||||
> **User:** Claude, help me plan a renovation. Here's the current plan and
|
||||
> two room photos. I want something like this Scandinavian reference —
|
||||
> open-plan, neutral tones, keep the footprint.
|
||||
|
||||
## What the agent does
|
||||
|
||||
The host loads the **`renovation_from_photos`** prompt:
|
||||
|
||||
```text
|
||||
currentPhotos: ["data:image/png;base64,...", "data:image/jpeg;base64,..."]
|
||||
referencePhotos: ["data:image/jpeg;base64,..."]
|
||||
goals: "Open-plan living/kitchen, neutral tones, keep the footprint."
|
||||
```
|
||||
|
||||
The prompt tells the agent to (1) analyze the floorplan, (2) analyze each
|
||||
room photo, (3) seed a scene from the floorplan, (4) compare against the
|
||||
reference, and (5) propose patches.
|
||||
|
||||
### 1. Extract the floorplan
|
||||
|
||||
```jsonc
|
||||
// tool: analyze_floorplan_image
|
||||
{
|
||||
"name": "analyze_floorplan_image",
|
||||
"arguments": {
|
||||
"image": "data:image/png;base64,iVBORw0KGgoAAAANS...",
|
||||
"scaleHint": "1 m grid, total footprint ~9.5 m × 7 m"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Under the hood, the tool issues an MCP sampling request to the host with
|
||||
the image and a structured prompt asking for walls, rooms, and
|
||||
approximate dimensions. The response is validated against the tool's
|
||||
output schema:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"walls": [
|
||||
{ "start": [0, 0], "end": [9.5, 0], "thickness": 0.25 },
|
||||
{ "start": [9.5, 0], "end": [9.5, 7], "thickness": 0.25 },
|
||||
{ "start": [9.5, 7], "end": [0, 7], "thickness": 0.25 },
|
||||
{ "start": [0, 7], "end": [0, 0], "thickness": 0.25 },
|
||||
{ "start": [4.5, 0], "end": [4.5, 7], "thickness": 0.15 },
|
||||
{ "start": [4.5, 3.5], "end": [9.5, 3.5], "thickness": 0.15 }
|
||||
],
|
||||
"rooms": [
|
||||
{ "label": "Living", "polygon": [[0, 0], [4.5, 0], [4.5, 7], [0, 7]] },
|
||||
{ "label": "Kitchen", "polygon": [[4.5, 0], [9.5, 0], [9.5, 3.5], [4.5, 3.5]] },
|
||||
{ "label": "Bedroom", "polygon": [[4.5, 3.5], [9.5, 3.5], [9.5, 7], [4.5, 7]] }
|
||||
],
|
||||
"approximateDimensions": { "widthMeters": 9.5, "depthMeters": 7, "areaSqMeters": 66.5 },
|
||||
"confidence": 0.82
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Analyze the room photos
|
||||
|
||||
```jsonc
|
||||
// tool: analyze_room_photo
|
||||
{
|
||||
"name": "analyze_room_photo",
|
||||
"arguments": { "image": "data:image/jpeg;base64,/9j/4AAQ..." }
|
||||
}
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"approximateDimensions": { "widthMeters": 4.4, "depthMeters": 5.8, "heightMeters": 2.5 },
|
||||
"identifiedFixtures": [
|
||||
{ "kind": "sofa", "approximatePosition": [2.2, 3.5] },
|
||||
{ "kind": "coffee-table", "approximatePosition": [2.2, 2.4] },
|
||||
{ "kind": "tv-unit", "approximatePosition": [0.3, 2.0] }
|
||||
],
|
||||
"identifiedWindows": [
|
||||
{ "wallHint": "south", "approximateWidth": 1.4, "approximateHeight": 1.5 }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
The kitchen photo is analyzed the same way.
|
||||
|
||||
### 3. Seed the scene
|
||||
|
||||
The agent reads `get_scene`, confirms the default empty Site → Building →
|
||||
Level is present, and then batch-creates walls matching the floorplan:
|
||||
|
||||
```jsonc
|
||||
// tool: apply_patch
|
||||
{
|
||||
"name": "apply_patch",
|
||||
"arguments": {
|
||||
"patches": [
|
||||
{ "op": "create", "parentId": "level-1",
|
||||
"node": { "type": "wall", "start": [0, 0], "end": [9.5, 0],
|
||||
"thickness": 0.25, "height": 2.5 } },
|
||||
/* ...remaining perimeter + partition walls from the vision result... */
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The agent then calls `set_zone` three times to seed the Living / Kitchen /
|
||||
Bedroom polygons from the floorplan rooms.
|
||||
|
||||
### 4. Cut the identified openings
|
||||
|
||||
For each window the vision tool reported, the agent calls `cut_opening`
|
||||
against the corresponding perimeter wall:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"name": "cut_opening",
|
||||
"arguments": {
|
||||
"wallId": "wall-south",
|
||||
"type": "window",
|
||||
"position": 0.5,
|
||||
"width": 1.4,
|
||||
"height": 1.5
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Propose the renovation
|
||||
|
||||
Guided by the reference photo's analysis (bright neutrals, open plan,
|
||||
minimal furnishing), the agent proposes a single logical patch:
|
||||
|
||||
- Remove the partition wall between Living and Kitchen.
|
||||
- Relocate the kitchen island further west.
|
||||
- Delete the bulky TV unit item; leave the sofa and coffee table.
|
||||
- Re-label the merged zone `"Open-Plan Living / Kitchen"`.
|
||||
|
||||
All of that goes into one `apply_patch`:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"name": "apply_patch",
|
||||
"arguments": {
|
||||
"patches": [
|
||||
{ "op": "delete", "id": "wall-partition-living-kitchen", "cascade": false },
|
||||
{ "op": "update", "id": "zone-living", "data": { "label": "Open-Plan Living / Kitchen",
|
||||
"polygon": [[0, 0], [9.5, 0],
|
||||
[9.5, 3.5], [0, 3.5]] } },
|
||||
{ "op": "delete", "id": "zone-kitchen", "cascade": false }
|
||||
/* + item moves / deletes for the TV unit etc. */
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The user can walk back with `undo`; `redo` returns them to the proposal.
|
||||
|
||||
### 6. Sanity-check
|
||||
|
||||
```jsonc
|
||||
// tool: validate_scene
|
||||
{ "name": "validate_scene", "arguments": {} }
|
||||
// → { "valid": true, "errors": [] }
|
||||
|
||||
// tool: check_collisions
|
||||
{ "name": "check_collisions", "arguments": { "levelId": "level-1" } }
|
||||
// → { "collisions": [] }
|
||||
```
|
||||
|
||||
The agent reports a summary of the changes plus the approximate new
|
||||
usable area (from the summary resource), and the user opens the scene in
|
||||
`@pascal-app/viewer` to see the renovated 3D layout.
|
||||
|
||||
## Takeaways
|
||||
|
||||
- The vision tools only return **data**. They don't mutate the scene —
|
||||
the agent is explicit about every structural change via `apply_patch`.
|
||||
- Photos supply priors (approximate dimensions, fixture types) that a
|
||||
brief-only workflow can't. Combine them with `from_brief`-style
|
||||
prompts when the user has both a reference and concrete text goals.
|
||||
- All renovation steps are a single temporal step per patch, so the user
|
||||
can compare before/after with `undo` / `redo`.
|
||||
Reference in New Issue
Block a user