Methodology & sources
The model comparison page shows three kinds of facts, each from its own source, each with its own date. This page documents what each source is, under what license we use it, how often we refresh it, and when we would drop it.
OpenRouter usage rankings (rankings-daily)
- What it is
- Daily token volume across models routed through OpenRouter (top 50 public models + one aggregated "other" row).
- What it measures
- Usage heat — how much a model is actually called. Not capability, not quality.
- License & attribution
- CC BY 4.0 (Data API aggregated endpoints). Required citation: "Source: OpenRouter (openrouter.ai/rankings), as of <snapshot date>." plus "Licensed under CC BY 4.0." when republishing the data. OpenRouter notes: tokenizers differ across providers, so a token in one row is not directly comparable to a token in another row; the dataset contains public traffic only; it is not intended to be re-served as a competing free API.
- How we fetch
- OpenRouter Data API endpoint, build-time fetch with an API key stored only in the build environment. meta.as_of and meta.version are stored alongside any number we publish.
- Update cadence
- Weekly window (rate budget: 30 req/min, 500 req/day per key).
- Removal conditions
- If the credential is unavailable the column renders "source access restricted" with no numbers (current state). If OpenRouter changes license terms, the source is disabled and the table removed.
LMArena leaderboard dataset (Hugging Face, lmarena-ai/leaderboard-dataset)
- What it is
- Human-preference pairwise battle results published by Arena Intelligence as an official dataset snapshot.
- What it measures
- Arena Score (Bradley-Terry–based rating per the dataset README) on the text style-control config, with confidence intervals and vote counts. Model entries carry effort tiers (High)/(Max) as part of their identity and are never merged.
- License & attribution
- CC BY 4.0 on the dataset. Attribution line: "Source: Arena Leaderboard Dataset (Arena Intelligence, Inc.), dataset lmarena-ai/leaderboard-dataset on Hugging Face, snapshot <snapshot sha>, leaderboard published <publish date>. Licensed under CC BY 4.0." The arena.ai / lmarena.ai websites are never scraped (their ToS prohibit automated access) — data comes exclusively from the Hugging Face dataset.
- How we fetch
- HF datasets-server rows API, build time, after asserting the dataset license tag is cc-by-4.0. The revision sha and leaderboard_publish_date are pinned into every snapshot.
- Update cadence
- Weekly check (the dataset updates live; we do not track intraday changes).
- Removal conditions
- If the license tag disappears from the dataset card (this has happened due to their pipeline), the import is paused ("source review required") and no new numbers are published; the last verified snapshot stays with its date.
Vendor official pricing pages (primary price source)
- What it is
- Per-million-token API list prices from each vendor's own pricing page: OpenAI, Anthropic, Google, xAI, DeepSeek, Zhipu (BigModel), Xiaomi MiMo, Meta, Alibaba Cloud.
- What it measures
- Official list price only — input / output / cache-read, in the vendor's own currency (CNY stays CNY, no conversion), with tiers (batch, peak/off-peak, long-context, region, service level) listed side by side. The lowest tier is never presented as the standard price.
- License & attribution
- Factual quotation of public prices with per-price deep link and retrieval date; vendor terms pages are linked, never reproduced. Subscription plans and enterprise agreements are out of scope.
- How we fetch
- Manual verification into a hand-maintained ledger (data/pricing-manual.json), each entry carrying source_url + retrieved_at + conditions.
- Update cadence
- Manual re-check at least every 30 days; a price older than 30 days renders a stale badge.
- Removal conditions
- A vendor page that stops listing a model moves that model's price to "not listed"; conflicting figures across official pages are shown side by side, never averaged.
OpenRouter model catalog (metadata cross-reference)
- What it is
- Model IDs, launch dates, context lengths, and OpenRouter listed prices, used for cross-referencing and launch dates.
- What it measures
- Catalog metadata only. OpenRouter listed prices appear only as a market reference explicitly labeled "not vendor-direct".
- License & attribution
- The /api/v1/models endpoint is outside the Data API's CC BY 4.0 grant: we quote only a small set of factual fields (id, canonical_slug, name, context_length, created, three pricing fields) with source and date, and deep-link to the corresponding OpenRouter page. No wholesale republication.
- How we fetch
- Build-time fetch (no key required), whitelist-picked fields only. Benchmarks fields embedded in this endpoint are never read (they contain Artificial Analysis data, which is excluded).
- Update cadence
- Weekly, with the build.
- Removal conditions
- If OpenRouter changes terms, the cross-reference columns are dropped; vendor pages remain the price source.
What we do not do
- No composite, weighted, or "value" score — the three columns are never merged into one ranking.
- No claim of full-board coverage: 17 models chosen for verifiability, not completeness.
- No Artificial Analysis data (direct or relayed) — its terms prohibit use in comparison products; we do not integrate it.
- No Hugging Face trending/downloads dimension in this version.
- No vendor logos — names only.
- No cross-vendor token comparisons — tokenizers differ, so usage numbers are per-source facts.
- No mixing of model versions or effort tiers — each version/tier is its own row.
Three standing caveats: token counts are not comparable across vendors; usage is adoption, not capability; and this page covers 17 verifiable models, not the whole market.
Missing cells are states, not zeros: \"not listed\", \"unverified\", \"stale\", \"source access restricted\". Old values keep their date and a stale badge; they are never silently presented as current.
Data sources & attribution
(No data ingested from this source in this release yet, so no snapshot date is available to cite; the line below is the required attribution wording.) Source: OpenRouter (openrouter.ai/rankings), as of not ingested — no snapshot date. Licensed under CC BY 4.0.
Data provided by OpenRouter under CC BY 4.0, AS IS; no warranties are made as to accuracy or completeness.
Adapted by AI Hot: token figures displayed as published; ordering and currency formatting applied by AI Hot.
Source: Arena Leaderboard Dataset (Arena Intelligence, Inc.), dataset lmarena-ai/leaderboard-dataset on Hugging Face, snapshot f566816dfe80f18d0bc804dd5deafa68d5957cb9, leaderboard published 2026-11-06. Licensed under CC BY 4.0. https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset
Data provided by Arena Intelligence, Inc. under CC BY 4.0, AS IS; no warranties are made as to accuracy or completeness.
Adapted by AI Hot: entries filtered to the categories shown; no scores altered.
AI Hot aggregates publicly available leaderboard data and links back to each official source. Figures reflect each source's own methodology and are not a combined or normalized ranking. Token usage reflects call volume, not model capability. Models and names belong to their respective owners.