Aleph Alpha's Kolibri-1: 78B Open Weights, 1M Context

Aleph Alpha released Kolibri-1, a German-English open-weight model with 78.1B parameters, 3.46B active, 1M-token context, under Apache 2.0.

@sharedot

Aleph Alpha's Kolibri-1: 78B Open Weights, 1M Context

By @sharedot · · 7 pages

  • AI Frontier
  • Open Weight Models
  • Aleph Alpha
  • Kolibri 1
  • Sovereign AI

Aleph Alpha released Kolibri-1, a German-English open-weight model with 78.1B parameters, 3.46B active, 1M-token context, under Apache 2.0.

A European sovereign model goes open on Unity Day

The model is a bilingual German-English Mixture-of-Experts transformer with 78.1 billion total parameters but only 3.46 billion active per token, a context window the company serves at 1,048,576 tokens, four reasoning settings, and tool calling.

It is pitched squarely at government and regulated industries that want to run inference on their own hardware instead of sending internal data to a third-party cloud.

The release follows the earlier Kolibri Origin and Aleph Alpha's automated Model Factory, and was built in Germany with training run on German and Finnish infrastructure.

Efficiency by design: 384 experts, six per token

The architecture is the story a builder cares about. Kolibri-1 routes each token through six of 384 experts, so it does the compute work of roughly a 3.5-billion-parameter model while carrying the knowledge of a 78B one.

Attention is sparse by design: 40 of 50 layers use a 512-token sliding window while 10 use full attention, a 4-to-1 ratio that keeps the KV cache from ballooning as context grows.

Training took 768 Nvidia B200 GPUs for 21 days in the main phase, with roughly 24 trillion tokens used across pre-training, mid-training, and long-context adaptation.

German accounted for 21.3% of pre-training tokens, with a custom bilingual tokenizer built to handle German compounds rather than relying on machine translation.

Grounding first: trained to abstain

The most distinctive engineering claim is abstention. According to TestingCatalog, Kolibri was trained on abstention examples and Aleph Alpha's Merlin-Arthur procedure, which teaches the model to withhold an answer when the supplied evidence does not support one.

The company reports it avoided a wrong answer on 44% of AA-Omniscience items, up from 14.8% for Kolibri Origin. ForkLog likewise notes the model is trained to refrain from answering when context is insufficient to verify information.

Vendor benchmarks, and the cost of running it

Aleph Alpha's own harnesses report an overall average of 75.5 in English and 70.8 in German, with 96.9 on AIME 2025 and 85.9 on LiveCodeBench v6, and claim parity with models using up to four times the active parameters on math, code, agentic, and long-context tasks — results that are vendor-run and not independently verified.

Pasquale Pillitteri's writeup reports RULER long-context retrieval scores that fall from 72.2% at 64,000 tokens to 63.2% at one million, and notes the native context is 262,144 tokens, with the million coming from extrapolation.

Deployment needs serious hardware: ForkLog reports the FP8 version occupies about 78 GB, with a minimum of one H200/B200 or two A100 80GB cards, served via vLLM with a Kolibri-specific plugin.

Sovereignty stakes and what comes next

The release lands amid upheaval: Pasquale Pillitteri reports Aleph Alpha signed a definitive agreement on September 16 to merge with Canada's Cohere in a deal valued at about $20 billion combined, with the final brand set to be Cohere and Heidelberg remaining a research hub.

For EU buyers, the pitch is supply-chain integrity — data curation, training, evaluation, weights, and deployment all under European control, built with the AI Act, the general-purpose AI Code of Practice, and GDPR in mind, per ForkLog.

Whether Kolibri survives as a distinct line inside a renamed company is the open question, and the open weights now let the community test the claims independently.

Sources

  1. testingcatalog.com › Aleph Alpha releases open-weight Kolibri with 1M context
  2. forklog.com › Germany Unveils Open AI Model for Public Sector
  3. pasqualepillitteri.it › Aleph Alpha Releases Kolibri-1, a 78B Open-Weight Model With 1M Context

More on AI Frontier