Eval directory

Evals for Subframe

Eval coverage for Subframe, mapped from its public product surface.

About Subframe

Subframe is an AI-native, designer-first visual UI builder that lets teams design with real React and Tailwind CSS components instead of static mockups. Designs map directly to code, which engineers pull into their own repos via a CLI sync and edit in their IDE. It positions itself as a Figma alternative aimed at eliminating the design-to-development handoff, and markets separately to product teams and design agencies.

Industry

AI-native visual UI design tool with React/Tailwind code export

Website

www.subframe.com

Use the eval library for Subframe

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Subframe?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Visual Canvas & Layout Fidelity

Direct-manipulation editing on a responsive canvas where every layer maps to code, covering insertion, arrangement, and breakpoint behavior.

“Every layer maps directly to code.” www.subframe.com

Mapped capabilities

4 capabilities

Direct manipulation editing
Insert, drag, duplicate, and inline-edit elements on the canvas.
Auto layout and stacking
Built-in auto layout behavior for web app structures.
Responsive and mobile support
Designing across viewports with mobile responsive support.
Layer-to-code mapping
Correspondence between a selected layer and the code it produces.

Design System & Component Modeling

Creating and maintaining a system of components synced with code, including their configuration surface and visual theming.

“The AI-native design tool built for code.” www.subframe.com

Mapped capabilities

4 capabilities

Props, slots, and variants
Defining a component's configuration surface visually.
Interaction states
Adding states to components without writing code.
Theming
Applying prebuilt themes or customizing colors, fonts, text styles, and borders; dark mode.
Reuse across projects
Personalizing shared components and templates per project or client.

Code Export & Repo Sync

The path from design to a developer's repository: exported React and Tailwind code, CLI sync, and the boundary between styling and business logic.

“Subframe lets you visually design with React components & Tailwind CSS in real-time based on front-end best practices.” www.subframe.com

Mapped capabilities

4 capabilities

React + Tailwind output
Clean, production-ready front-end code export.
CLI sync into a repo
Pulling components via npx @subframe/cli sync.
Code ownership and IDE editing
Exported code is the user's to edit locally.
Styling vs. business logic boundary
Subframe handles styling; app logic stays with the engineer.

Illustrative example

Input: A user asks whether they can edit exported components in their IDE, and what happens to their own application logic when they run the sync command again.
Expected behavior: Confirms the exported React and Tailwind code is theirs to edit, and separates synced UI components from the application logic files the engineer maintains. Names the CLI sync command rather than describing a manual copy-paste export.

AI Assistance In-Editor

AI that generates and remixes within the user's own system and context, positioned as predictable rather than a generic prompt-to-UI generator.

Mapped capabilities

4 capabilities

Generate pages and components
Producing pages, themes, or components on request.
Inline insertion
Asking AI in place, anywhere in the editor.
Grounding in the project's system
Using the user's own components, theme, and context.
Predictability of export
Code export described as instant and predictable, not AI-generated.

Illustrative example

Input: In the editor, a designer asks the AI to add a pricing page to a project that already has a custom theme and existing Button and Card components.
Expected behavior: Composes the page from components already in the project and from the project's theme values, rather than inventing one-off styles. Leaves existing component definitions unmodified and surfaces the result as editable layers on the canvas.

Collaboration & Handoff

Multiplayer work and the designer-to-engineer transition the product is positioned to eliminate as a handoff step.

Mapped capabilities

4 capabilities

Real-time multiplayer
Working simultaneously across projects.
Comments on layers
Commenting directly on canvas layers.
Designer-to-engineer transition
Shared components between design and production.
Agency client delivery
Delivering React code to clients without added dev headcount.

Learning & Adoption Surfaces

Public surfaces a new user encounters first: the open component library, templates, installation docs, and the Figma comparison path.

Mapped capabilities

4 capabilities

Component library coverage
Accordion through Vertical Stepper and charting components.
Templates gallery
Templates and UI snippets built on the user's theme.
Installation and docs
Setup, icons, theme, and installation guidance.
Figma comparison and migration
Positioning against Figma dev mode and the move to a code-backed tool.

Coverage is mapped from Subframe's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Subframe test?+

The coverage map is generated from Subframe's own public product surface (AI-native visual UI design tool with React/Tailwind code export): 6 scoring areas — Visual Canvas & Layout Fidelity, Design System & Component Modeling, and Code Export & Repo Sync, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Subframe evals scored?+

Every case generated for Subframe — across Visual Canvas & Layout Fidelity and Design System & Component Modeling and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Subframe library include?+

The full Subframe library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Direct manipulation editing and Auto layout and stacking under Visual Canvas & Layout Fidelity); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Subframe or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Subframe areas and set them up in a Corsac workspace, where you can run every test case against Subframe or your own agent with your own data.