The Figma MCP Couldn't Read Canvas Feedback — So I Matched It by Coordinates

If you build against Figma with Claude, the first thing you do is connect the Figma MCP. So when the design feedback also started coming through Figma, I did the obvious thing and asked the MCP to read it. It couldn’t — and the reason, plus the fix, is a small technique worth stealing.

This is the third post in a series about redesigning a client’s site with Claude Code. You don’t need the earlier posts for this one. The short version: I was applying design feedback to a marketing site, page after page, and I wanted the agent to read the feedback off the Figma file itself. The second post said the intake step “worked” and moved on — this post is what that one word actually cost to earn.

Why reading feedback off Figma is hard

Here’s the thing I assumed would be easy and wasn’t.

Figma has a clean place for feedback: pinned comments. A comment comes back from the API with coordinates attached, so you know exactly what it points at:

{ "message": "swap this text",
  "client_meta": { "node_id": "2:1873", "node_offset": { "x": 1858, "y": 8660 } } }

If all the feedback had been comments, this post wouldn’t exist. But it wasn’t. Across 25 screens in this project, only 3 had any comments at all — 5 comments total.

The actual feedback came drawn straight onto the canvas. Each screen was pasted in as a flat screenshot, and next to it, in the empty canvas space, sat typed notes and red arrows. “Change this image so it fits on one screen.” “Swap this text.” An arrow pointing from a note to a spot on the screenshot. One feedback file had 54 of these notes across 22 screens, and not one of them was a comment.

That’s the input, and it defeats two obvious approaches:

• The Figma MCP doesn’t return comments at all — and even if it did, comments weren’t where the feedback was. Its tools (get_metadata, get_design_context, get_screenshot) give you the layer tree, the design skeleton, and a render. I ran get_metadata on the feedback page; the output is over 500KB of node geometry, and searching it for comment, message, or author returns zero. The notes drawn on the canvas are in there — but as anonymous text and vector nodes, mixed in with the design, with nothing marking which is which. And there’s no structure to lean on: the whole thing is one flat pile. The screenshots weren’t grouped into per-section frames, the feedback text wasn’t nested under the screen it referred to — screenshot fills, screen-content text, note text, and arrow vectors all sat as siblings at the same level. So you can’t walk the frame tree to find “this section and its notes,” because there is no such tree. Even each screenshot was flat: a pasted image with no inner layers, so there was nothing inside to read either.
• Reading the render visually produces guesses. A screenshot with notes scrawled around it is exactly the input a vision model fumbles: it grabs a note meant for the screen above, or describes an element from a different screenshot, because in a flat image the notes and the design blur together.

So the feedback was all there in the file. Nothing told the agent which note belonged to which screen.

The fix, in one sentence

Feedback sits outside the screen it’s about. People annotate by writing next to a thing, not on top of it — so every note lands in the empty canvas beside its screen. Once you see that, the whole problem turns into arithmetic: each screen is a box, each note is a point, and a note belongs to the box it sits beside. No looking required.

That’s the punchline. Getting to it took a day of being wrong, and the dead ends are worth a paragraph because they’re the ones you’d hit too.

Following the arrows visually went nowhere. The notes had red arrows, so the plan was: find an arrow, read the text by it. From my logs that afternoon: “the text is next to the arrow” → “there’s no text where the arrows point” → “the arrows point off the canvas edge and there’s nothing there.” And matching an arrow to its target by geometry was out too — the REST API gives an arrow only an absoluteBoundingBox, no start or end point, so you can’t even tell which way it points.

The turn was to stop guessing and dump the coordinates. Instead of reasoning about where feedback should be, I printed every text node’s x-position and read down the list: the design text sat in one range of x-values, and a second group sat far to the right of it. Two clear groups. That right-hand group was the feedback — sitting outside the screens, exactly as the one-sentence rule says. Filter for text outside the screen boxes and all 16 notes fell out clean.

The script that came out of it

The implementation is smaller than the story around it. GET /v1/files/:key/nodes returns every node with its absoluteBoundingBox, so a screen and a note both arrive as plain numbers —

{ "id": "149:3", "type": "RECTANGLE",
  "absoluteBoundingBox": { "x": 16156, "y": -7094, "width": 2503, "height": 1298 } }

— and the one rule (feedback sits outside its screen) collapses into three coordinate checks:

units = nodes.filter(n => n.type == 'RECTANGLE' && n.width >= sectionWidth * 0.85)
             .sortBy(n => n.y)              // screens: full-width boxes, stacked

band  = [screen.right, nextScreen.left]     // feedback: to the right of the box

notes = canvasText.filter(n => n.x within band)

for (n of notes)
    attach n to the screen whose y-range contains n.y

The three rules that do the work, each one a bug I hit:

1. A screen is a box wide enough to be one — not a named layer, not a parent frame. With no section frames to walk and no reliable names, the only thing that actually held was size: match by width (≥ 85% of the section). One round had screens named “Screenshot…” except one named “Rectangle 36”; a name filter silently dropped it. Width didn’t care what it was called.
2. A note is feedback if it’s outside the box. Text inside a screen is content; text in the band to its right is a comment about it.
3. A note belongs to the screen whose y-range contains it. Screens are stacked, each owns a vertical slice, a note falls into one.

One extra rule handles the arrows, and it’s the part people get wrong. The rule is three words: nearest node, smallest area wins. Given an arrow tip at (x, y), rank nodes by distance to the point, then break ties by area — because a tie at distance 0 means the point sits inside several stacked boxes, and the smallest is the specific element, not the full-page screenshot behind it. Here’s the function, straight from the script:

// point → box shortest distance (0 if the point is inside the box)
const dist = (b) => {
  const dx = Math.max(b.x - px, 0, px - (b.x + b.w));
  const dy = Math.max(b.y - py, 0, py - (b.y + b.h));
  return Math.round(Math.sqrt(dx * dx + dy * dy));
};

const ranked = nodes
  .filter((n) => n.box)
  .map((n) => ({ ...n, dist: dist(n.box), area: n.box.w * n.box.h }))
  // rank by distance, then by smaller area — a tie at dist 0 means the
  // point sits inside several boxes; the smallest is the most specific
  // target (the element to replace), not the big screenshot behind it.
  .sort((a, b) => a.dist - b.dist || a.area - b.area);

So --near x,y hands back the actual element an arrow points at, and “replace this” resolves to a real target instead of the whole page. The rest of the script is just the REST fetch and arg-parsing around this sort; this is the idea.

What falls out is a plain data structure — each screen with its notes, and a dropped_count so I can see if anything failed to attribute:

{
  "feedback_band": { "left": 18659, "right": 26117 },
  "screen_feedback": [
    { "screen_id": "149:3", "y": [-7094, -5796],
      "feedback": [
        { "x": 20619, "y": -6831, "text": "change image so it fits on one screen" },
        { "x": 18942, "y": -6498, "text": "swap this text" }
      ] }
  ],
  "dropped_count": 0
}

That’s the payload the builder step consumes: screen id, notes, coordinates. On the round I checked, dropped_count was 0 — every note attributed. The step that used to be a coin flip became a lookup.

What it’s worth

The value isn’t the code, it’s what it removed. Reading feedback went from a per-screen guessing game — the thing that made me distrust every automated pass — to a deterministic lookup. On the round I pulled apart, the coordinate rules attributed all 54 canvas notes with dropped_count: 0: no missed notes, nothing mapped to the wrong screen. Every downstream step (which screen, which element, which asset to swap) rests on that attribution being right, and coordinates made it right.

The MCP still pulled its weight, just for the other half of the job — get_screenshot and get_design_context when I needed to see a node, not locate it. REST tells you where things are; the MCP shows you what they look like. Neither reads canvas feedback on its own.

None of this is Claude-specific. The gap is in the Figma tooling, not the agent — any tool reading a Figma file through the MCP hits the same wall, and any agent (Cursor, a custom one) can fall back to the same REST coordinates. The technique travels; the MCP’s blind spot is the constant.

Where it breaks

These rules aren’t a contract anyone signed. They’re what the tool assumes about the input, and real files drift — a rushed round, a different arrangement, a canvas that doesn’t stack cleanly.

So the intake checks itself instead of trusting the result. If it comes back with zero screens or zero notes in the band, that’s the signal the file doesn’t match the shape the tool expects, and the agent falls back — to the render, to --near, or to just asking me — rather than confidently mapping a note to the wrong screen. A broken assumption is detectable: the geometry comes back empty and you know. A bad guess isn’t — it looks exactly like a good one until you ship it.

What this doesn’t touch

Reading the feedback correctly is only the mechanical half of the loop.

The tool nails “shrink that by 5px” and “this note goes with that screen.” It never touches “this looks cheap, fix the hierarchy.” Most of the corrections in this project were that second kind — visual judgment about whether something read right — and no amount of coordinate math helps there. That part stayed mine. Coordinates handle where things are; taste is still a human problem.

Next in the series: where to let the agent run a whole section on its own and where to make it stop for review.

Researched and drafted by me and Claude, from my own project logs. Client, designer, and file details are anonymized.