zhengli97/DM-KD
Official PyTorch Code for "Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation?" (https://arxiv.org/abs/2305.12954)
### Overview:
This project is an implementation of "Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation?" by Zheng Li et al. The paper explores the effectiveness of using synthetic data generated by diffusion models for knowledge distillation without access to real images. The results show that synthetic data can significantly improve performance in various benchmarks, particularly in terms of low-fidelity images.
### Key Features:
- **Framework**: The project proposes a framework using diffusion models to synthesize images for distillation.
- **Main Results**: The implementation includes experiments on CIFAR-100 and ImageNet-1K datasets, demonstrating improved performance over existing methods.
- **Datasets**: The project provides offline synthesized datasets for experiments, including low-fidelity and high-fidelity versions.
### Tech Stack:
- Python
### Why It Matters:
The project's findings highlight the potential of synthetic data from diffusion models as a powerful tool for knowledge distillation, offering significant improvements in performance without the need for real images. This opens up new possibilities for training models on limited data and accelerating the development of more efficient distillation methods.