Yandex announced on September 11 that it had released Alice AI-T5-35B-A0.6B Base, a model trained from scratch for Alice’s generative answers in Search. External developers can run inference through Hugging Face Transformers, but the optimized production inference system remains available only inside Yandex.
The model combines an Encoder-Decoder design with a sparse mixture-of-experts architecture. Yandex says it was pretrained with the UL2 denoising objective on 15 trillion tokens from the company’s own corpus. Near the end of training, its context length was expanded from 8K to 128K using YaRN.
Yandex selected the architecture to meet the speed and document-processing requirements of search. Its initial auxiliary-loss-based approach to balancing experts proved unstable at this scale, causing routing in the encoder to collapse. The team then adopted the auxiliary-loss-free routing method described in DeepSeek-V3; according to Yandex, the stability problems disappeared after the change.
The search pipeline also uses an information-context extractor: a BERT-like model with 80 million parameters, pretrained from scratch on four trillion tokens. It selects compact passages from retrieved documents that are useful to the answer generator. Yandex estimates that reducing the input-token count increased throughput by about 40% without reducing quality.
For answer alignment, the team used real Search queries, generated multiple candidate responses and selected among them using rule-based and model-based quality signals. Harder examples went through a critique-and-revision pipeline involving a judge model, while RLHF used GSPO, a modification of GRPO. Aggregated behavioral signals—including answer-reading behavior and whether users continued searching—were also incorporated. Yandex says it reduced noise through filtering, repeated-query aggregation, pairwise training and device-based stratification.
After applying the same alignment procedure to all compared models, Yandex reported blind pairwise win rates of 77% against Qwen 3.5-2B, 54% against Qwen 3.5-4B and 69% against T5 Gemma 2.4B-4B. Against Qwen 3.5-35B-A3B, the result was 44%. These are company-reported evaluations: the pairwise judgments were performed blindly by assessors, while the published pretraining benchmarks were calculated on Yandex’s internal infrastructure.
Practical context: The release lets developers examine the combination of an Encoder-Decoder model, sparse expert layers and long-context support. However, the public base model does not reproduce the complete Search product: fast answers also depend on document retrieval, extraction of relevant passages, subsequent alignment and Yandex’s internal optimized inference stack.
Sources
Event date: 2026-09-11. Primary source date: 2026-09-11.
