CS EN
Get in touch

MAXX2.0

Built from scratch

A Czech language model built from scratch. It chats with you, calculates and looks facts up in the Czech Wikipedia.

Try MAXX Chat
Back to all projects

Czech only. From scratch.

MAXX 2.0 is a Czech language model we built from scratch, not by tweaking someone else’s model. It has its own tokenizer, it learned from Czech alone, and in the chat it goes by Maxx.

Czech in. Maxx out.

Czech text ~20 B tokens
Tokenization cs32k
Base training 156 h
Learning to chat SFT · DPO · RL
MAXX 2.0 0.67 B

  • Speaks Czech

    As Maxx, it talks to you in natural, informal Czech.

  • Calculator

    Maxx calls it on its own for sums, dates and unit conversions.

  • Wikipedia

    Maxx looks facts up in the Czech Wikipedia on its own.

First the base learned Czech from ~20 B tokens. Then it learned to chat: from tens of thousands of Czech conversations, from comparing better and worse answers, and from tasks whose results can be checked.

Prediction met

A small model. All in Czech.

We trained the base on a single rented graphics card. On text it never saw in training, it reached 0.878 bits per byte, meeting its prediction of 0.881. The lower the number, the better the model predicts Czech.

Billion parameters
0.67

455 M in the network that computes, and ~0.21 B in a table it only looks up.

Billion Czech tokens
20

Roughly that many pieces of Czech text went through base training.

Hours of training
156

Roughly how long the base trained on a single rented RTX 5090.

Czech only
CS

Its own Czech tokenizer and only Czech data.

What it learned from.

Czech text only: the web, Wikipedia, Wikisource and educational PDFs. At the end of training, the best-rated texts got the most room.

Czech web

Text from Czech websites. The best-rated tenth got the most room at the end.

Wikipedia

The Czech encyclopedia: facts and explanations.

Wikisource

Mostly older Czech literature, in a smaller share.

Educational PDFs

Teaching texts and documents that explain things.

And how does it rank worldwide?

Tongue in cheek: MAXX 2.0 is not in the Artificial Analysis index. It has 455 M parameters and speaks only Czech.

  1. Gemini 4 Carbon59
  2. Claude Opus 5.558
  3. GPT-6 Astra57
  4. Muse Wonder Full50
  5. Kimi K3.1 Pro46
  6. Grok 4.735
  7. Qwen3.8 Max34
  8. DeepSeek V4.1 Flash32
  9. GLM-5.332
  10. Mistral Large 4 Preview25
  11. MAXX 2.00
Source: Artificial Analysis Intelligence Index v5.0, October 2026. Each model’s best variant.

For the curious.

Architecture, data and training in detail
Architecture
A decoder-only transformer: 24 layers, width 1,280, 20 attention heads
Context
2,048 tokens
Tokenizer
Its own Czech cs32k: 32,768 word pieces
Parameters
455 M in the transformer and ~0.21 B in a bigram table (a memory of word-piece pairs), ~0.67 B in all
Data
~20 B Czech tokens: the web (FineWeb2 and its best-rated 10%, FineWeb2-HQ), educational PDFs (FinePDFs-edu), Wikipedia and Wikisource
End of training
The last ~2.5 B tokens: 72% best-rated web, 18% Wikipedia and Wikisource, 10% educational PDFs
Base training
One rented RTX 5090, ~156 hours, finished on 26 October 2026
Base quality
0.878 bits per byte on text held out of training (0.881 was predicted)
Learning to chat
SFT on tens of thousands of Czech conversations, written and checked by teacher models and proofread; then DPO and RL on checkable tasks (word problems, instructions, Wikipedia facts)

Alongside the code, we’re writing a Czech textbook that explains every line of MAXX 2.0.