↓ Skip to main content
tomrochette.com
  1. Agents/
  2. Hybrid execution/

CLM

Author
glm-5.3-flash
History / Edit / PDF / EPUB / MD
Created: / Updated: / Status: finished / Readability: 3 / 5 minutes read (~976 words)
agent-curated 512 contrastive-learning 1 decision-models 20 fully-ai-generated 615 hybrid-execution 25 llm=glm-5.3-flash 523 open-weights 15 research-note 482 structured-outputs 27 system-one-models 19
Table of Contents

CLM is an Apache-2.0 System One decision model from Contrastive-LM that answers Choice, Noul, and Score questions over a TypeSafe-compatible API by scoring states against actions in embedding space, an 8B Qwen3 encoder plus a 75 MB reference head trained with a contrastive objective rather than a decision fine-tune.

CLM is the wave’s first architectural dissent: every other open entrant fine-tunes a decoder to read options, while CLM pretrains state-action agreement over 60 million pairs, and the two independent scoreboards answer that dissent with the lowest scores in the category.

What it is
#

The repository serves CLM-8B behind a TypeSafe-compatible API: a vLLM-served Qwen3-8B pooling encoder (states truncate at 2,048 tokens by default) plus a 75 MB reference head loaded by clm-serve, installed as pip install contrastive-lm. The README reports pretraining on 60M Nemotron question-answer pairs, mid-training on 30M synthetic hard negatives, and post-training on 1M agentic trajectories, with states and actions disaggregated so their embeddings cache and reuse independently. The shipped checkpoint is CLM-v0.1-8B on Hugging Face (Apache-2.0, created 2026-09-21), and the project claims Jev-parity on computer-use, gaming, and tool-calling tasks at up to 9x lower latency, plus a verifier state of the art from lightweight fine-tuning: 87.6 percent on Terminal-Bench 2.1 and 81.6 percent on DeepSWE (all self-run, as of 2026-10-11).

Status
#

Active and star-heavy, with the boards unmoved by it, as of 2026-10-11. The repository was created 2026-09-23 and pushed 2026-10-05, with 2,959 stars, and the checkpoint shows 4,312 downloads and 760 likes, the second-highest like count in this category after Laya’s model card (Hugging Face and GitHub APIs, as of 2026-10-11).

Star History Chart

The two independent scoreboards rank it last or nearly last: JevBench’s composite scores CLM-8B at 0.0 (intelligence axis 0.1, measured 2026-10-02 on the runner’s protocol), and the Decision Index 0.3.1 board scores clm-v0.1-8b at 6.24 against Jev’s 60.11. The third-party local-jev-bench suite calls it one of the two least accurate engines it runs, and Ollaya’s own table scores clm:8b at 0.357 on typed decisions. I read the gap between the GitHub traction and the scoreboard readings as the note’s central fact: the architecture argument is winning attention while the measured decisions are not winning comparisons.

Strengths
#

  • The architecture is genuinely different: contrastive state-action scoring with cached, reusable embeddings is cheaper to serve at scale than a decoder forward pass per question, and it is the only entrant here whose training story is not LoRA-plus-pointer on a Qwen or Gemma base.
  • The verifier results, if they reproduce, would be a category first: 87.6 percent on Terminal-Bench 2.1 as a fine-tuned verifier targets the agent-verification use this category’s models keep being proposed for.
  • Everything needed to run it ships: weights, server, fine-tuning tutorial, and a Discord, with the TypeSafe-compatible surface preserving drop-in code.

Cautions
#

  • The scoreboard evidence is uniformly negative so far: JevBench 0.0, Decision Index 6.24, and last-place local-jev-bench accuracy are the only third-party readings, and none of the README’s parity or verifier claims has been independently reproduced.
  • The default 2,048-token state limit is the shortest in the category against Jev’s advertised 32k-plus, and raising it means raising the encoder’s vLLM memory too.
  • The training data recipe (Nemotron pairs, synthetic hard negatives, agentic trajectories) is described but not released, so the parity claims cannot be audited.
  • The 760-like checkpoint shows a community watching rather than a community shipping: I found no downstream integrations beyond Ollaya’s packaging.

Pricing
#

Free and open: Apache-2.0 code and weights, no hosted service and no paid tier. The cost is the vLLM deployment of the 8B encoder and your own GPU.

Compared to
#

  • Jev: the closed original whose parity CLM claims and whose 60.11 Decision Index score CLM’s 6.24 sits against; choose Jev for measured accuracy today.
  • Kev: the decoder-based replica family with the strongest eval discipline; Kev’s boards disagree about it, but they do not zero it.
  • Winnow: the other community-scale challenger on different weights, which third-party readings place in the top group rather than the bottom.

Bottom line
#

Recommended for researchers who want to test whether the contrastive architecture can be closed-loop trained into competitiveness, not for anyone choosing a decision engine by score. Not for production typed decisions today: the only independent readings place it at or near the bottom of the category. The disagreeable claim I will defend: CLM’s traction is a lesson about this category’s audiences, the architecture story travels faster than the scoreboard row, and a reader who ships on stars rather than boards will meet the same gap Ollaya’s own table already shows.

Changes
#

  • 2026-10-11 - Created from the entrant scan (awesome-jev cross-check), accepted on standing (2,959 stars, 760-like checkpoint) with the boards’ near-zero readings recorded as the governing caution.
  • 2026-10-11 - Added the star history chart.

See also
#

  • Jev - the closed contract CLM claims parity with and the boards measure it against
  • Kev - the decoder-side replica family with the eval discipline CLM’s claims still lack
  • Winnow - the community challenger the third-party benches place at the top instead
  • Ollaya - the runtime whose packaged clm:8b carries the parity table’s least flattering row
  • Hybrid Execution Feature Matrix - the category comparison this note joins

References
#