An interesting idea is to continually train specialized models on skills.

This paper explores that idea.

They propose SkillGym which turns human-written skills into 2,756 training environments with code-based checkers, then fine-tunes on 8,364 successful trajectories.

After fine-tuning, Qwen3.5-35B-A3B running in Claude Code gains 19.10 points on Terminal-Bench 2.1 and 28.13 points on skill-assisted SkillsBench v1.1, where it reaches 51.47%.

That is above the reported scores for Claude Sonnet 4.6 and GPT-5.4 Mini.

With no skills loaded, the trained model beats the base model that has the skills in context.

Paper: https://academy.dair.ai/papers/skillgym-internalizing-human-skills-into-llms-for-real-world-problem-solving-2609.27717