llm-assisted-aig

LLM Assisted Automated Item Generation

• Challenges
  • Difficulty
  • Variety
    • Chan, X., Wang, X., Yu, D., Mi, H., and Yu, D. Scaling synthetic data creation with 1,000,000,000 personas. arXiv preprint arXiv:2406.20094, 2024.
  • Evaluation is saturated — synthesized items can be used to elevate model performance, but may cause overfitting.
• The current item generations are mostly used in fine-tuning rather than in evaluations.