open-data on tomrochette.com

AI Release Tracker

tom@tomrochette.com (Tom Rochette) — Thu, 24 Sep 2026 00:00:00 +0000

AI Release Tracker is a free, ad-free timeline of major frontier AI model releases since ChatGPT’s launch on November 30, 2022, with launch-day benchmark scores, pricing, and machine-readable exports for each entry. Facts below verified as of 2026-09-24; the live site blocks automated fetchers (HTTP 429), so the current-content evidence comes from the 2026-08-26 archived copy.

It answers the one question every other site in this category layers something on top of: what shipped, when, with which launch-day numbers.

What it is
#

A timeline site whose FAQ describes “a free, continuously updated timeline of every major AI model release from the world’s leading AI labs”, tracking 231 models from 10 companies as of the archived copy (OpenAI, Anthropic, Google, Meta, xAI, DeepSeek, Mistral, Moonshot AI, Z.ai, Qwen). Each entry records the release date, benchmark scores as published at launch (GPQA Diamond, SWE-Bench Verified, MMMU, Terminal-Bench, and others), parameter counts, context windows, license tier, and first-party API pricing where the lab publishes one. Scores are framed as historical records, not live standings, and third-party-leaderboard figures are marked with their source. The site also publishes “Expected” next-release dates per lab, which the author described as an average of the intervals between previous releases. Surfaces: the timeline, per-benchmark ranking pages, model compare pages, analytics, pricing comparison, free email notifications on new releases, and flat-file exports: /models.json and a llms-full.txt corpus described as “plain text for LLM ingestion”, free to use with an attribution link.

Status
#

Active but nearly invisible: the dataset was last updated 2026-08-26 (newest tracked release Qwen3.8-Flash-Next), and Wayback captures through June, July, and August 2026 show the corpus growing month over month. It was Show HN’d on 2025-12-02 by user curlii and drew 2 points and 6 comments; that thread is still the site’s entire public footprint, and no GitHub repository exists. The footer reads “© 2026 To sider ApS · CVR DK42753491”, a Danish private limited company, so it is a side-project-grade operation with corporate paperwork. Treat it as alive, useful, and one person away from dormant.

Strengths
#

The flat files (/models.json, llms-full.txt) are the category’s friendliest data source for agents: one fetch, structured, free, attribution-licensed.
Launch-day scores are quotable in a way leaderboard standings are not, because a record of what the lab claimed on release day does not silently move.
No ads, no account wall, and a scope (releases only) it does not pretend to extend past.

Cautions
#

Single-operator bus factor: one person, one company registration, no repository, no visible team.
Verification is thin: the community footprint is one small Show HN thread, so data-entry errors would surface slowly.
Naming quirks (xAI labeled “SpaceXAI”) and via-attributed benchmark numbers mean the corpus needs spot checks before you cite it.
The live site 429s automated fetchers, so agents should use the flat files or the archive rather than scraping the UI.

Pricing
#

Free to use, no ads. Free tier includes email notifications on new releases; a paid “Pro instant alerts” tier sends email the moment a model is added, with no price published. No paid price is stated, so no price history applies.

Compared to
#

LLM Stats: current composite rankings with an API; AI Release Tracker is the historical log LLM Stats’s score churns over.
Artificial Analysis: live quality, price, and speed; choose it for what is true today, the tracker for what was claimed at launch.
Epoch AI: trend datasets and research; the tracker is the raw release stream Epoch-style analysis consumes.

Bottom line
#

Recommended for engineers and agents who want a dated, quotable “what shipped when” record with a machine-readable corpus, not for deciding what to deploy this week. My disagreeable claim: this is the most valuable site in the category for agents and the least visited by humans, because “what changed since last time” is the first question of every refresh run, and this is the only member that answers it as a flat file.

Changes
#

2026-09-24 - Created.

References
#

https://web.archive.org/web/20260826134434/https://aireleasetracker.com/ - archived homepage: FAQ (231 models, 10 companies, free/pro alerts, no ads), footer “To sider ApS · CVR DK42753491” (fetched 200, 2026-09-24)
https://web.archive.org/web/20260826134434/https://aireleasetracker.com/llms-full.txt - dataset scope, last-updated date, benchmark glossary, attribution terms (fetched 200, 2026-09-24)
https://web.archive.org/web/20260826134434/https://aireleasetracker.com/models.json - structured export: 231 models with release dates, parameters, context windows, benchmarks (fetched 200, 2026-09-24)
https://news.ycombinator.com/item?id=46119002 - the Show HN thread and the author’s description of purpose and notifications (fetched 200, 2026-09-24)
https://hn.algolia.com/api/v1/items/46119002 - full thread JSON backing the footprint claim (fetched 200, 2026-09-24)
https://hn.algolia.com/api/v1/search?query=aireleasetracker&tags=story - the single-story footprint evidence (fetched 200, 2026-09-24)
https://web.archive.org/cdx/search/cdx?url=aireleasetracker.com&matchType=domain - snapshot cadence and subpage inventory through 2026-08-26 (fetched 200, 2026-09-24)

OpenRouter Rankings

tom@tomrochette.com (Tom Rochette) — Thu, 24 Sep 2026 00:00:00 +0000

OpenRouter Rankings ranks models by the tokens actually processed through the OpenRouter gateway, published with CC BY 4.0 licensing, a public Data API, and views by task, cost per session, market share, and app, and it is the only ranking in its category built on revealed spend rather than votes or benchmarks. Facts below verified as of 2026-09-24.

Every other leaderboard in this category measures opinion; this one measures invoices.

What it is
#

A rankings surface inside openrouter.ai, the LLM gateway founded in early 2023 that claims 500T+ monthly tokens, 10M+ users, 80+ providers, and 500+ models. The page states plainly what it does not measure: “They do not rank models by accuracy, reasoning ability, or benchmark performance”, and it excludes private requests while bucketing usage into daily UTC aggregates per model variant. Views include top models (today, week, month), top models by task share of spend, cost per session for coding agents, market share by model author, fastest models, and top apps (Hermes Agent, Claude Code, Kilo Code, and Cline led in tokens as of 2026-09-24). Data access is unusually open: rankings are CC BY 4.0 with a required citation format, and the Data API serves rankings-daily (top 50 public models per day, history back to 2025-01-01, 30 requests/minute, live-updating) to any inference key.

Status
#

Fresh and commercially central: the page showed “Usage data through Sep 23, 2026” on the day of verification, a one-day lag, and updates flow live as traffic arrives. The underlying gateway is a top-of-category business: $113M Series B led by CapitalG in May 2026 at a reported $1.3B valuation, then an announced acquisition by Stripe on August 19, 2026 at a reported price above $7B, with the company claiming 10T+ tokens per day. Its numbers are routinely quoted as market signal in press and on HN, and OpenRouter itself turned the dataset into research (a 100T-token empirical study that drew 207 points on HN).

Strengths
#

Revealed preference: tokens and spend through a paid gateway are harder to game with a demo day prompt than a vote or a benchmark run.
Openness is a policy, not a favor: CC BY 4.0, a documented citation format, and a free JSON API with a year-plus of history.
The by-task and cost-per-session views answer questions (what does a coding-agent session actually cost) that no benchmark board even frames.

Cautions
#

The sample is one gateway, and the page says so: “They do not include usage on a model provider’s own API or across the whole market”, and “token volume is not a count of requests, users, or spend” since verbosity and tokenization differ.
Free-tier promotions distort the ranks: Max Woolf’s analysis of the Tencent Hy3 model topping the leaderboard documents a free period on a single provider driving the rank, and the Kilo Code free-Grok episode did the same in 2025.
Independence is now a clock: the rankings belong to a gateway being absorbed by Stripe, so treat “market share” as “OpenRouter’s customers’ share” and re-verify who publishes the numbers after the deal closes.
Figures restate as snapshots refresh, so any number you cite needs its as-of date recorded alongside it.

Pricing
#

Free to view and to query, CC BY 4.0 licensed with attribution. OpenRouter monetizes the gateway (inference and enterprise controls), not the rankings. No reader-facing price is stated, so no price history applies.

Compared to
#

Artificial Analysis: constructed measurement of quality, price, and speed; the rankings measure adoption, and the two disagree for good reasons.
LMArena: stated preference versus revealed preference.
LLM Stats: benchmark aggregates with an agent-facing API; the rankings’ API is the usage-side mirror.

Bottom line
#

Recommended for answering “what are developers actually paying for this week” with numbers you can legally republish, not for any claim about which model is more capable. My disagreeable claim: this is the most misused ranking in the category, cited as market truth from a sample that excludes every provider’s first-party API traffic, and the Stripe acquisition makes reading its provenance, not its numbers, the skill that matters going forward.

Changes
#

2026-09-24 - Created.

References
#

https://openrouter.ai/rankings - methodology, caveats, windows, top apps, CC BY 4.0 note, data-through date (fetched 200, 2026-09-24)
https://openrouter.ai/docs/cookbook/administration/data-api - Data API endpoints, limits, citation format, dataset history from 2025-01-01 (fetched 200, 2026-09-24)
https://openrouter.ai/about - company scale claims and founding date (fetched 200, 2026-09-24)
https://openrouter.ai/blog/announcements/series-b/ - $113M Series B and investors, May 2026 (fetched 200, 2026-09-24)
https://openrouter.ai/blog/announcements/openrouter-is-joining-stripe/ - the Stripe acquisition announcement, August 2026 (fetched 200, 2026-09-24)
https://techcrunch.com/2026/08/16/stripe-will-reportedly-acquire-ai-gateway-startup-openrouter-for-7b/ - the reported price and valuation (fetched 200, 2026-09-24)
https://minimaxir.com/2026/05/openrouter-hy3/ - the free-tier distortion analysis behind the Hy3 rank (fetched 200, 2026-09-24)
https://news.ycombinator.com/item?id=48317294 - the HN thread questioning free-driven ranks (fetched 200, 2026-09-24)
https://news.ycombinator.com/item?id=48330499 - the “market signal” counterpoint (fetched 200, 2026-09-24)
https://news.ycombinator.com/item?id=46154022 - the 100T-token study showing the dataset’s research use (fetched 200, 2026-09-24)

Trackers and Leaderboards Feature Matrix

tom@tomrochette.com (Tom Rochette) — Thu, 24 Sep 2026 00:00:00 +0000

This matrix compares the six members of the Trackers and leaderboards category: sites whose product is a continuously refreshed number about the AI field itself. The members split cleanly on what their number measures: what shipped (AI Release Tracker), what the operator measured (Artificial Analysis), how the field moves (Epoch AI), what public evidence aggregates to (LLM Stats), what people prefer (LMArena), and what people pay for (OpenRouter Rankings).

No row of this matrix crowns a winner, because the rows are different questions; the failure mode is citing a site for a question it does not answer, usually LMArena ranks quoted as capability or OpenRouter tokens quoted as market share.

Legend: ✓ supported, ✗ not supported, ~ partial or conditional, ? not verified. Each column links to the full research note; every cell traces to a source cited there or in the references.

The matrix
#

	AI Release Tracker	Artificial Analysis	Epoch AI	LLM Stats	LMArena	OpenRouter Rankings
Kind	release timeline plus flat-file corpus	independent benchmarking site and data business	research nonprofit with open datasets	composite aggregator with agent-facing API	blind preference arena	gateway usage rankings
The number measures	launch-day facts: what shipped, when, with which claimed scores	the operator’s own controlled evals, prices, and speed runs	long-run trends: compute, cost, capability over time	public benchmark evidence, normalized with uncertainty	blind human preference votes	tokens processed through one gateway
Run by	To sider ApS, a Danish side project (one visible operator)	venture-backed independent company (AI Grant seed)	501(c)(3) nonprofit, itemized donors, about 50 people	ZeroEval Inc. (self-displayed YC backing)	Arena Intelligence Inc. ($100M seed at $600M)	OpenRouter, a gateway acquired-by-Stripe (announced 2026-08-19)
Coverage	231 releases, 10 labs (as of 2026-08-26)	671 models, 100+ providers, 1,000+ endpoints, chips	3,200+ models since 1950, data centers, chips, companies	398 canonical models, 50+ benchmarks claimed	text, image, video, vision, search, webdev, agent arenas	500+ models via the gateway, 80+ providers
Update cadence	on each release; dataset last updated 2026-08-26	daily changelog; index versions iterate weekly	near-daily data updates, page-stamped	continuously; “within hours of release” claimed	continuous votes; product posts within days	daily UTC buckets, about one day of lag
Methodology published	~ FAQ and data notes, no formal methodology	✓ methodology hub with evaluation lists and weights	✓ transparency page, papers, and data documentation	~ score construction published, details gated	✓ methodology repo (arena-rank) plus founding paper	✓ on-page caveats plus Data API docs
Data access	✓ /models.json and llms-full.txt, free with attribution	~ web charts free, data platform paid, no public API documented	✓ CC BY datasets and a Python client	✓ REST plus 11 MCP tools, free tier	~ arenas open, methodology open, raw votes not re-offered	✓ CC BY 4.0 JSON via Data API, history to 2025-01-01
Reader pricing	free, no ads; Pro instant alerts, price unpublished	free site; enterprise and data-platform tiers, prices unpublished	free, donation funded	free tier; Builder $99/month; Commercial contract	free	free, attribution required
Independence caveat	one operator, one company registration, no second pair of eyes	sells private benchmarking to the labs it ranks (disclosed policy)	consulted for OpenAI and DeepMind, disclosed and audited	paid eval services and a consumer sibling in the same company	lab partnerships and a documented private-testing history	owned by the gateway it measures, being absorbed by Stripe
Verification hooks	✓ full corpus re-downloadable and diffable	~ changelog and methodology public, raw runs private	✓ data, notebooks, and code public	~ API re-queryable, scoring internals partly gated	✓ methodology code public, vote data not re-offered	✓ daily snapshots re-fetchable under CC BY

Reading the matrix
#

The row that sorts the category is “the number measures”, and it doubles as a citation guide: release questions go to the tracker, present-tense quality to Artificial Analysis, trend claims to Epoch, quick composites to LLM Stats, preference to LMArena, spend to OpenRouter. The two most-quoted members are also the two most misquoted: LMArena ranks get read as capability, and gateway tokens get read as market share, when the matrix’s own rows show both measure something narrower.

Verification strength tracks institutional type, not popularity. The nonprofit (Epoch) publishes its funders, its consultations, and its data; the gateway (OpenRouter) publishes its raw daily numbers under CC BY; the side project (AI Release Tracker) lets you diff its whole corpus; while the three venture-scale companies publish methodology and products but keep raw runs, votes, or scoring internals private. If you need to check the work rather than read the work, the checkable columns are the nonprofit’s, the gateway’s, and the side project’s, which is the opposite of what citation frequency would predict.

Every column carries a conflict row, and the conflicts are structural rather than scandals: the benchmark firm sells benchmarking to the benchmarked, the arena partners with the labs it ranks, the aggregator sells evaluation services, the gateway’s data is its own marketing, and the nonprofit consults for the labs. Epoch’s FrontierMath episode and LMArena’s Maverick episode are the two documented failures, and both produced their institution’s strongest disclosure artifacts, which is the pattern worth watching on every refresh of this page.

Choosing from the matrix
#

Needing a dated record of what shipped and what the lab claimed that day: AI Release Tracker, via its flat files.
Picking a model or provider this week and needing quality against price and speed: Artificial Analysis, with the Endpoint Accuracy Index for the cheap-provider question.
Citing how fast the field moves, with downloadable data: Epoch AI.
Wiring leaderboard data into an agent or a script: LLM Stats’s free API and MCP tools, accepting the gated internals.
Sensing which model people prefer right now: LMArena, read as a mood ring and never as a capability claim.
Asking what developers actually spend on: OpenRouter Rankings, quoted with its as-of date and its single-gateway caveat.

Changes
#

2026-09-24 - Created with six columns (AI Release Tracker, Artificial Analysis, Epoch AI, LLM Stats, LMArena, OpenRouter Rankings) when the category was seeded at the owner’s request.

References
#

https://web.archive.org/web/20260826134434/https://aireleasetracker.com/ - AI Release Tracker column: scope, footprint, alerts, operator (fetched 200, 2026-09-24)
https://artificialanalysis.ai/ - Artificial Analysis column: model count, indexes, provider coverage (fetched 200, 2026-09-24)
https://artificialanalysis.ai/methodology/intelligence-benchmarking - Artificial Analysis column: published evaluation weights (fetched 200, 2026-09-24)
https://epoch.ai/about/transparency - Epoch AI column: nonprofit status, funders, consultations (fetched 200, 2026-09-24)
https://llm-stats.com/developer - LLM Stats column: API tiers, MCP tools, quotas (fetched 200, 2026-09-24)
https://techcrunch.com/2025/05/21/lm-arena-the-organization-behind-popular-ai-leaderboards-lands-100m/ - LMArena column: funding and company identity (fetched 200, 2026-09-24)
https://arxiv.org/abs/2504.20879 - LMArena column: the private-testing findings (fetched 200, 2026-09-24)
https://openrouter.ai/rankings - OpenRouter Rankings column: methodology, caveats, licensing (fetched 200, 2026-09-24)
https://openrouter.ai/blog/announcements/openrouter-is-joining-stripe/ - OpenRouter Rankings column: the ownership question (fetched 200, 2026-09-24)
https://minimaxir.com/2026/05/openrouter-hy3/ - OpenRouter Rankings column: the free-tier distortion evidence (fetched 200, 2026-09-24)

open-data on tomrochette.com

AI Release Tracker

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#

OpenRouter Rankings

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#

Trackers and Leaderboards Feature Matrix

The matrix
#

Reading the matrix
#

Choosing from the matrix
#

Changes
#

See also
#

References
#

open-data on tomrochette.com

AI Release Tracker

What it is #

Status #

Strengths #

Cautions #

Pricing #

Compared to #

Bottom line #

Changes #

See also #

References #

OpenRouter Rankings

What it is #

Status #

Strengths #

Cautions #

Pricing #

Compared to #

Bottom line #

Changes #

See also #

References #

Trackers and Leaderboards Feature Matrix

The matrix #

Reading the matrix #

Choosing from the matrix #

Changes #

See also #

References #

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#

What it is
#

Status
#

Strengths
#

Cautions
#

Pricing
#

Compared to
#

Bottom line
#

Changes
#

See also
#

References
#

The matrix
#

Reading the matrix
#

Choosing from the matrix
#

Changes
#

See also
#

References
#