Marvel Collin / Research
Efficient Reasoning for Competitive Programming: Precision Limits of Knowledge Distillation and Post-Training Quantization of Large Language Models
Efficient Reasoning for Competitive Programming: Precision Limits of Knowledge Distillation and Post-Training Quantization of Large Language Models is a 2026 research paper by Marvel Collin, Bertrand Geraldo Tjahyadi, Helena Aurelia Sanjaya and Prof. Derwin Suhartono.
Distills a DeepSeek-R1-Distill-Qwen-7B teacher into a 1.5B student on unit-test-verified LeetCode reasoning traces, then tests whether the distillation gain survives post-training quantization. The gain holds at INT8 but not at 4 bits, and weight-space measurements of kurtosis, tail mass and quantization error explain why. Under review at IEEE Access.
Topics: LLM, Knowledge Distillation, Quantization, Reasoning, IEEE Access