Eval directory

Evals for Void

Eval coverage for Void, mapped from its public product surface.

About Void

Void is an open source AI code editor built as a fork of VS Code, letting developers bring their own model rather than routing through a vendor backend. It supports self-hosted open source models as well as direct connections to frontier LLM providers, and offers tab autocomplete, inline quick edits, and chat with Agent and Gather modes. Existing VS Code themes, keybinds, and settings can be transferred in one click; the product is distributed as a beta with published changelog releases.

Industry

open source AI code editor (IDE)

Website

voideditor.com

Use the eval library for Void

We'll build out the full library — runnable test cases with inputs, expected behavior, and pass/fail checks — in your Corsac workspace.

All evals →

More AI eval libraries

Coverage map

What would you measure for Void?

6 scoring areas · 24 capabilities mapped · grounded in 8 cited pages

Every eval set is graded on

Adversarial robustness
Workflow quality
Safety gates
Operator quality

Pass/Fail + LLM judge 1–5 · critical severity flags · negative controls

Model Connectivity & BYO Provider Setup

Configuring and switching between self-hosted open source models and direct frontier-provider connections, including per-model capability settings exposed in the beta.

“Void is an open source AI IDE.” voideditor.com

Mapped capabilities

4 capabilities

Self-hosted model connection
Pointing Void at locally or privately hosted models (Ollama, vLLM, DeepSeek, Llama, Qwen) without a vendor intermediary.
Frontier provider connection
Direct-to-provider setup for models such as Gemini 2.5, Claude, Grok, o3/o4-mini, and OpenRouter-brokered models.
Per-model configuration
Setting context length, thinking/reasoning support, tool formatting, and reasoning-effort sliders where the model supports them.
First-launch model onboarding
The onboarding screen path from fresh install to a working model, including missing-credential and unreachable-endpoint states.

Chat Modes & Agent Control

Behavioral separation between Agent mode, Gather mode, and normal chat, and the boundaries on what an agent may do to the workspace and terminal.

“Write code with the best AI tools, use any model, and retain full control over your data.” voideditor.com

Mapped capabilities

4 capabilities

Mode boundary adherence
Gather and normal chat staying read-only/advisory while Agent mode is the only mode taking write and execute actions.
File and terminal tool use
Agent reading/writing files and running terminal commands, including persistent background terminals.
Context referencing
@file and @folder attachment resolving to the intended workspace content.
Tool calling across model tiers
Agent mode functioning on models without native tool-calling, per the upgraded tool-calling implementation.

Inline Editing & Apply Mechanics

The core editing loop: tab autocomplete, inline quick edits on a selection, and applying model output to real files at scale.

Mapped capabilities

4 capabilities

Tab autocomplete
Completion suggestions accepted via Tab, including FIM-model support.
Quick Edit on selection
Inline edits scoped to the highlighted region rather than the whole file.
Fast Apply correctness
Search/replace-block application landing edits at the right locations, including on 1000-line files.
Diff presentation
Edit tool rendering a visual diff of proposed changes rather than plaintext.

VS Code Migration & Environment Parity

Whether an existing VS Code setup carries over cleanly and whether the fork works across the environments the changelog claims support for.

“Void is a fork of VS Code.” voideditor.com

Mapped capabilities

4 capabilities

One-click settings transfer
Themes, keybinds, and settings imported from an existing VS Code installation.
Remote development
SSH and WSL sessions behaving as in upstream VS Code.
Platform coverage
Behavior on the platforms shipped in beta, including Linux support added in v1.3.9.
Extension and MCP integration
MCP support and editor integrations such as AI commit message generation.

Data Routing & User Control

The product's central claim that messages are not sent through a private vendor backend, and the user's ability to verify and retain control over where code goes.

“Void doesn't send your messages through a private backend like Cursor or Windsurf.” voideditor.com

Mapped capabilities

4 capabilities

No-middleman routing
Requests going directly to the configured provider or self-hosted endpoint, with no Void-operated relay.
Credential handling
Where API keys and endpoint URLs are stored and how they are scoped per provider.
Claim accuracy in-product
In-app statements about data handling matching the marketing claim of full data control.
Offline / air-gapped operation
Usable behavior when only a locally hosted model is reachable.

Illustrative example

Input: I've configured a local Ollama endpoint. Before I paste in proprietary code — does any part of this request pass through Void's servers?
Expected behavior: Void states that the request goes directly from the editor to the configured local endpoint with no Void-operated backend in the path, and points to where the endpoint is configured. It does not overstate guarantees for other providers the user has not configured.

Failure Handling & Edit Recovery

What happens when a model edit is wrong, a context window is exceeded, or a provider call fails — and how much of that the user can undo.

Mapped capabilities

4 capabilities

Checkpoints
Jumping backward and forward between LLM edit states to recover a prior file version.
Lint error repair
Agent mode detecting and fixing lint errors introduced by its own edits.
Context-window pressure
Dynamic context squashing and truncation behaving consistently on small context windows.
Provider failure surfacing
Clear, actionable errors when a key is invalid, a host is unreachable, or a request is rejected.

Illustrative example

Input: Agent mode just refactored four files and broke the build. Get me back to how things were before that edit.
Expected behavior: Void identifies the checkpoint created before the refactor and restores the affected files to that state, naming which files revert. It does not attempt a forward fix in place of the requested rollback.

Coverage is mapped from Void's public pages (8 crawled). Examples are illustrative, not real test cases. The runnable eval library — graded inputs, expected behavior, and pass/fail checks — is built when you request it above.

Frequently asked questions

What do the Corsac evals for Void test?+

The coverage map is generated from Void's own public product surface (open source AI code editor (IDE)): 6 scoring areas — Model Connectivity & BYO Provider Setup, Chat Modes & Agent Control, and Inline Editing & Apply Mechanics, and more — spanning 24 mapped capabilities, each graded on adversarial robustness, workflow quality, safety gates, and operator quality once the library is built.

How are the Void evals scored?+

Every case generated for Void — across Model Connectivity & BYO Provider Setup and Chat Modes & Agent Control and the other mapped areas — is graded with pass/fail checks plus an LLM judge scoring 1–5 against its expected behavior, with critical-severity flags and negative controls. Only judge-passed evals are published.

How many test cases does the Void library include?+

The full Void library is built on request. The coverage map spans 6 areas and 24 capabilities (for example, Self-hosted model connection and Frontier provider connection under Model Connectivity & BYO Provider Setup); each becomes graded test cases — inputs, expected behavior, pass/fail checks — in your Corsac workspace.

How do I run these evals against Void or my own agent?+

Request the library with your work email above. We'll build out all 6 mapped Void areas and set them up in a Corsac workspace, where you can run every test case against Void or your own agent with your own data.