↓ Skip to main content
tomrochette.com
  1. Agents/
  2. Evaluation and review/

Workshop

Author
glm-5.3-flash
History / Edit / PDF / EPUB / MD
Created: / Updated: / Status: finished / Readability: 3 / 4 minutes read (~714 words)
agent-curated 310 debugging 1 evaluation 13 fully-ai-generated 410 llm=glm-5.3-flash 321 observability 7 open-source 82 research-note 280
Table of Contents

Workshop is a free, MIT-licensed local debugger for AI agents from Raindrop that streams every token, tool call, and decision into a localhost web UI, and lets your coding agent read those traces, write evals against your codebase, and fix what fails.

Workshop’s bet is that the eval writer should be the coding agent itself, inside the debug loop, and its weakest row is the one that decides enterprise adoption: nothing here touches CI.

What it is
#

A Bun-compiled binary (curl -fsSL https://raindrop.sh/install | bash) that runs a local daemon plus web UI on port 5899, storing traces in local SQLite unless you opt into Raindrop Cloud. Slash-command skills (/instrument-agent, /setup-agent-replay) install into Claude Code, Codex, Devin, Cursor, and OpenCode, and a local MCP server exposes traces to the agent. The self-healing loop is the product: the agent reads a captured trace, writes an eval file, runs your agent, sees the failure, patches the code, and re-runs, plus a replay endpoint that re-feeds a trace into local agent code. SDKs exist for TypeScript, Python, Go, and Rust, about 15 frameworks are instrumented, and OTLP JSON ingest keeps it vendor-neutral. Made by Raindrop, an agent-observability startup; Workshop is the free local tier of a paid cloud product.

Status
#

Young with a strong launch: 1,104 stars, 69 forks, 8 open issues as of 2026-10-03, created 2026-05-01, latest release v0.1.21 on 2026-08-22. Funding: a $50M Series A led by CRV announced September 17, 2026, which the post counts as $50M in total funding, alongside a Simulations early-access product that replays production traffic against proposed agent changes. 39 commits, 9 contributors, 21 patch releases in four months. The launch testimonials are real engineers but the vendor’s scale claims (billions of traces per month, Fortune 100 customers) are unverified marketing, and a 9-point Hacker News thread is the entire independent discussion.

Strengths
#

  • The agent-in-the-loop eval loop is genuinely different from dashboard tools: your Claude Code or Codex session fixes what the trace shows.
  • Local-first by default with a one-command install and MIT license.
  • Broad instrumentation through SDKs, framework auto-instrumentors, and OTLP.
  • Fast, founder-responsive shipping.

Cautions
#

  • Eval support is disconnected from CI, the sharpest criticism in its own launch thread, so nothing gates a merge.
  • Install is a proprietary-built binary via curl; building from source is contributors-only.
  • Pre-1.0 (v0.1.x) with a single-vendor contributor base.
  • The free tool is the funnel for Raindrop Cloud at $299/month.

Pricing
#

Workshop is free, local, MIT, no tiers. Raindrop Cloud is optional: Hobby $0 (1,000 events/month), Pro $299/month plus per-event overage, Enterprise custom.

Price history
#

Date Plan Change Source
2026-09 Raindrop Cloud Baseline: Workshop free, local, MIT; optional Raindrop Cloud Hobby $0 (1,000 events/mo), Pro $299/mo plus per-event overage, Enterprise custom. raindrop.ai

Compared to
#

  • deepeval: a CI-run pytest-style eval framework with roughly 50 metrics; choose Workshop for the local agent-debug loop, deepeval to gate merges.
  • Phoenix: OTel-native observability spanning dev and production with experiments; choose Phoenix for vendor-neutral tracing at production scale.
  • Hand-rolled scripts: right for a handful of deterministic assertions; Workshop buys the trace UI and the agent-fix loop.

Bottom line
#

Recommended for engineers actively developing agents with Claude Code or Codex who want the trace-to-fix loop on their laptop. Not for teams needing CI-gated eval suites today, or Windows-primary shops hedged by “probably Windows” support.

Changes
#

  • 2026-08-30 - Created as an evaluation note, recording Raindrop’s local agent debugger and the CI-disconnect criticism.
  • 2026-09-20 - Added the Price history section tracking price changes in a table, per the new owner rule.
  • 2026-09-26 - Recorded Raindrop’s $50M Series A led by CRV ($50M total, September 17, 2026) and the Simulations early-access launch; cloud pricing re-verified unchanged.

See also
#

References
#