A11Y Assistant for Figma

ToolJun 15, 2026

A11Y Assistant is a BYOK Figma plugin for turning selected app frames into reviewable VoiceOver and TalkBack labels. It scans the frame image, reads the layer tree, applies accessibility rules, and writes approved labels back into Figma.

I built it after a manual Pocket FM accessibility pass across Android and iOS. Content Designer Sayan Das and I audited core journeys, touch targets, grouping, platform guidance, and the labels a listener would hear through a screen reader.

Scan selected frames

Manual audit set the rules

We aimed for a usable starting point. A listener should be able to browse shows, start playback, understand controls, move across tabs, and avoid dead ends.

We looked at Netflix, Spotify, and YouTube because entertainment products repeat the same accessibility problems: posters need names, playback controls repeat across contexts, carousels create noisy focus paths, and tabs can sound identical if you label them from the layer tree alone.

Sayan and I turned that audit into ground rules, then added labels manually across Figma frames. That worked for a first pass. It also made the tooling gap obvious.

Rules for screen-reader labels

rulesignalresult

Group with intentIcon + text pairsOne label when the pair describes one value or action, such as a rating or comment count.

Keep choices separateTabs and controlsEpisodes, Next Series, play, skip, rewind, and download stay as separate targets.

Prioritise playbackPlayer regionTransport controls get explicit labels because they block the listening experience when wrong.

Name the jobLayer names + visualsLabels describe the function instead of repeating Figma layer names.

Remove chrome noiseStatus barsSystem time, battery, Wi-Fi, and home indicators stay out of the app label pass.

WCAG rules we used

I mapped the manual audit to WCAG 2.2 before I turned it into plugin behavior. A11Y Assistant repeats the label, grouping, reading-order, and target-spacing checks. A designer still reviews contrast, color-only cues, visual density, and anything that needs judgment.

WCAGcheckhow we handled it

1.1.1 + 2.4.6 + 4.1.2Names and statesPosters, icons, tabs, buttons, and player controls needed labels that told a listener what each target did and whether it was selected.

1.3.1 + 1.3.2Structure and orderWe grouped icon-text pairs that formed one value, kept independent actions separate, and checked order against the visual layout.

1.4.1 + 1.4.3 + 1.4.11Visual perceptionWe checked color-only cues, text contrast, icon contrast, and control visibility for low-vision and color-vision users.

2.4.3 + 2.5.8Focus and touchPlayback, tabs, and repeated controls needed logical order, enough size, and spacing from adjacent actions.

Manual labeling did not scale

The manual pass gave us the rules. After that, pasting labels into frame after frame felt wasteful. A model could inspect the screen, apply those rules, and give the designer a reviewable first draft.

I built A11Y Assistant in Antigravity, Google's IDE. I wanted to try a different build environment and model workflow instead of staying inside my usual Claude and Codex loop.

The plugin uses a BYOK model. Paste an API key, select frames, scan, review the suggestions, and apply the labels back into Figma. The key point is review. The plugin proposes labels; the designer still decides what ships.

Prototype with Figma comments

I started with a plain helper: Gemini key at the top, Scan Again, generated labels underneath, and an Insert comment button beside each item. It proved the model could help with the repetitive part.

Comments proved too temporary. The next build needed better grouping rules, frame context, manual override, provider flexibility, and a way to write labels as Accessibility annotations.

First A11Y Assistant build showing a Gemini API key field, generated Pocket FM accessibility labels, and Insert comment buttons

Frame image plus layer tree

The code combines screenshots with structure. The screenshot gives the model visual context. The layer tree gives it names, text, node ids, and geometry. That mix helps it tell two identical icons apart, such as seek backward on the left of the play button and seek forward on the right.

stepcodedetail

Select framesFrame exportThe plugin exports selected frames as JPGs and caps the largest side near 1024 px.

Map layersTree + bboxIt records ids, names, text, types, icon hints, bounding boxes, and spatial signatures.

Bridge payload100 KB chunksLarge frame payloads move across the Figma UI bridge in chunks instead of one silent drop.

Ask the modelBYOK providerIt detects Anthropic, OpenAI, or Gemini from the key format and sends the frames plus rules.

Review labelsTextarea listThe UI shows editable suggestions, single insert actions, apply all, rescan, and resume.

Write backAnnotationsIt creates an Accessibility Dev Mode channel and writes labels back to the selected nodes.

Manual override stays available

The plugin does not need the model for every label. If there is no API key or an export fails, it falls back to the text and icon candidates it can read from the frame. If you select a single child node, the manual override lets you add a label without running a scan.

A11Y Assistant manual override mode for a selected Pocket FM tab, with existing generated Accessibility annotations visible on the canvas

Repeatable accessibility workflow

A11Y Assistant helped turn the first Pocket FM accessibility pass into a repeatable workflow. Select frames, inspect the suggestions, fix the labels that need judgment, apply them as Accessibility annotations, and keep the rules close to the design file.

It did not replace a real accessibility audit. It made the next pass less blank. That was the useful part: we went from one manual cleanup to a tool that could start the next screen with context.