Research / Project detail

TinyR1: reasoning, safety & model merging

Open 32B models spanning branch-merge distillation, safety control, and distribution-aware model merging for reasoning, instruction following, and safety.

DistillationReasoningOpen research
Problem

How can compact models retain strong reasoning ability while making the method accessible to other researchers?

Method

Preview transfers capabilities through math, coding, and science branches followed by merging. TinyR1-32B uses Control Tokens for reasoning and safety; SCF-RKL controls merging interference through sparse complementary updates and reverse KL.

Outcomes

TinyR1 received coverage from the National Natural Science Foundation of China and reached the Hugging Face Trending Top 5. Preview uses 32B parameters, about 5% of DeepSeek-R1’s 671B, with an AIME 2024 score of 78.1 versus 79.8.

Results & evaluation

Results & evaluation

TinyR1-32B-Preview · Branch-Merge Distillation

Compared with DeepSeek-R1-Distill-Qwen-32B, scores improve by 5.5 points in math, 4.4 in coding, and 2.9 in science. Table 1 reports pass@1; LiveCodeBench uses the 24.08–25.02 subset.

TinyR1-32B-Preview · Branch-Merge Distillation
ModelAIME 2024 ↑LiveCodeBench ↑GPQA-Diamond ↑
DeepSeek-R1-Distill-Qwen-32B72.657.262.1
DeepSeek-R1-Distill-Llama-70B70.057.565.2
DeepSeek-R1 (671B)79.865.971.5
TinyR1-32B-Preview78.161.665.0

TinyR1-32B-Preview · Table 1 / Figure 1 ↗

Branch-Merge Distillation and math, coding, and science results
Branch-Merge Distillation and math, coding, and science results · TinyR1-32B-Preview · Figure 1 ↗

TinyR1-32B · Reasoning, instruction following & safety

The model card describes three SFT rounds on 20K high-quality samples with Control Tokens for task modes. Results below are from the TinyR1-32B card. Constructive Safety averages HarmBench, JBB-Behaviors, S-Eval_attack, and wildjailbreak.

TinyR1-32B · Reasoning, instruction following & safety
ModelSizeAIME24 ↑AIME25 ↑GPQA-Diamond ↑LCB v5 ↑IFEval Prompt Strict ↑Constructive Safety OpenSource ↑
DeepSeek-R1-0528671B91.487.581.073.380.973.0
Qwen3-235B-A22B-Thinking-2507235B—92.381.174.187.765.5
Qwen3-32B32B81.472.968.465.785.064.5
TinyR1-32B32B90.982.769.470.489.289.5

TinyR1-32B · Hugging Face model card ↗

SCF-RKL · Distribution-aware model merging

SCF-RKL achieves a ten-benchmark average of 65.33 when merging TinyR1-Preview-Math-32B and TinyR1-Preview-Science-32B. Selected individual scores and the original ten-task average are from Table 1. Figure 9 shows a separate experiment merging reasoning- and safety-specialized models.

SCF-RKL · Distribution-aware model merging
Method / modelAIME24 ↑GPQA ↑LCB ↑10-task average ↑
Math-32B74.5867.9961.6564.47
Science-32B78.0268.7861.4763.54
Task Arithmetic77.5068.1857.8057.90
DARE Task Arithmetic77.5068.1855.9157.80
TIES Merging74.3867.0850.1855.55
DARE TIES Merging76.1567.0856.9058.22
SCE74.3867.0849.9155.47
SCF-RKL76.9869.1962.4665.33

Beyond Parameter Arithmetic · Table 1 / Figure 9 ↗

SCF-RKL: reasoning, safety, and merged models across benchmarks
SCF-RKL: reasoning, safety, and merged models across benchmarks · Beyond Parameter Arithmetic · Figure 9 ↗
Related publications

Related papers

Related coverage

Related coverage