Python
7 stars
Post-train LLMs and agents from one config: SFT, multi-turn RL in tool environments (MCP, Harbor, OpenEnv), on-policy distillation, DPO and merging, with state-of-the-art defaults.
### Overview:
Palingenesis is a comprehensive post-training language model and agent development framework. It leverages state-of-the-art defaults for supervised fine-tuning, multi-turn reinforcement learning, on-policy distillation, and model merging. The project is designed to streamline the process of creating and optimizing language models and agents, making it accessible to developers and researchers alike.
### Key Features:
- **Supervised Fine-Tuning**: Enables the training of language models using a pre-defined configuration, including agentic traces with tool calls and reasoning.
- **Reinforcement Learning**: Supports multi-turn tool use and reasoning through verifiable rewards, providing a robust reinforcement learning environment.
- **Agent Environments**: Offers Python classes, MCP servers, OpenEnv, and Harbor tasks in Docker, with emulated harness dialects.
- **On-Policy Distillation**: Samples from a student model and scores against one or more teachers, using reverse KL for improved performance.
- **Off-Policy Distillation**: Fixed data against one or more teachers' distributions, context distillation, and multi-GPU support.
- **Preference Optimization**: Supports DPO, IPO, and SLiC-HF, with robust and length-normalised DPO for improved accuracy.
- **Model Merging**: Interpolates models to create new architectures, and merges models to improve performance.
### Tech Stack:
- **Python**: The primary programming language used for development and configuration.
- **CUDA**: Utilizes NVIDIA GPUs for optimal performance, especially on the latest CUDA drivers.
### Why It Matters:
Palingenesis simplifies the process