FlagEval, also known as Libra, is a large model evaluation system and open platform designed to establish a scientific, fair, and open benchmarking framework. It provides standardized evaluation methods and toolsets to help researchers comprehensively assess the performance of foundation models and training algorithms. It also explores the use of AI-assisted methods to enhance the efficiency and objectivity of subjective evaluations.FlagEval introduces an innovative “Capability–Task–Metric” three-dimensional evaluation framework that enables fine-grained characterization of the cognitive boundaries of foundation models, with visualized presentation of evaluation results. Currently, it offers tools for evaluating large language models, multilingual vision-language models, and text-to-image generation models, covering a wide range of language and multimodal foundation models.The platform spans four major evaluation domains: Natural Language Processing (NLP), Computer Vision (CV), Audio, and Multimodal, supporting a rich set of downstream tasks.

FlagEval is an important component of Beijing Academy of Artificial Intelligence's open source system of large model technology, FlagOpen. FlagOpen aims to build an open source algorithm system and one-stop basic software platform that fully supports the development of large model technology, supporting collaborative innovation and open competition, jointly building a "Linux" open source ecosystem in the era of shared large models.