Quick Start

Register and Login

By default, visitors have the permission to view the homepage, news, leaderboard, and user manual of the platform. To use the evaluation function, users need to complete platform registration and login, and provide complete personal information. Please ensure the submitted information is accurate and valid. After administrator's review and approval, users can use the platform for evaluation.

Detailed instructions are as follows:

Register

Users click on the "Register Login" button, the following page will pop up. For the first time using the platform, please use WeChat to scan the QR codeto follow "BAAI Community Assistant" WeChat official account, then fill in email, phone number, and verification code online to register.

After registeration is completed, it will automatically jump to the page for personal information collection. After users provide their complete personal information and the platform administrator review and approve according to their personal information, users who got approved can use the platform's evaluation function. The approval results will be notified to users by email.

Parameter	Explanation
Username	Username will be the only identifier on the platform. It is recommended to use the spelling of full name with numbers. After filling in the username, modification is not allowed. Length of 3-32 characters, supports lowercase letters and numbers, starting with a lowercase letter.
Real Name	Users should fill in their real name, and the platform administrator will give priority to real names during the approval process.
Organization	It is recommended to use a combination of organization + department, such as Beijing Academy of Artifical Intelligence Computing Power Platform, Tsinghua University Computer Scence Department. The platform administrator will give priority to real organization during the approval process. Organizations need to be filled in with both Chinese and English.
Task to Register	Select "Online Evaluation" or "Offline Evaluation": Online Evaluation: Users only need to provide the evaluation interface API, and the evaluation platform provides test data for inference evaluation. - Not supported yet. Offline Evaluation: Users need to upload trained models and inference codes. The evaluation platform provides inference computing power and data for inference evaluation.
Whether to evaluate self-developed models	Yes & No, single choice
Agreement statement	Users need to read and provide consent to the agreement before they can use the platform's evaluation function.

The registration process is shown in the following image:

Note:

• Please fill in the personal information carefully, as the administrator will approve it according to the personal information.
• Please fill in a valid personal business email, the approval status will be notified by email, and subsequent evaluation task status will also be notified to the user by email.
• If it is a personal email, the administrator will send an email to the user to modify email. Please modify the personal email and wait for approval again. Each user only has one chance to modify email per month.

Login

The user clicks the "Login" button, and a "Login" page will pop up. The user can choose to login by scanning the QR code at the top of the screen with WeChat, or by selecting the "Mobile Verification Code" method.

After logging in, if the user's personal information is incomplete, it will automatically jump to the personal information collection page.

Create Evaluation

Users click on [SUBMIT] to enter the evaluation management page, which mainly includes model evaluation, algorithms evaluation, and image management.

Users can choose model evaluation or algorithms evaluation according to their own needs. Click "Create Evaluation" to pop up the "Create Evaluation" dialog box, and fill in the corresponding form information according to different evaluation domains and submit to generate an evaluation task.

After creating an evaluation, it will automatically jump to the details page. Users can click to view the "Upload Model & Code" specification and use flageval-serving to upload models and codes. After uploading, click "Inference Verification" to quickly verify whether the inference evaluation code can run. After passing the verification, click "Start Inference Evaluation" to proceed with formal inference evaluation process. Wait for the evaluation to end to view the evaluation results. If there is any problem that causes termination and failure, the error message can be viewed through logs.

Upload Image

Some images are preset in [Images Manage]. If users need to customize images in actual evaluations, they can upload their own images in [Customized Images].

Users click on [Images Manage/Customized Images/Import Images], and the [Import Images] dialog box pops up. Fill in the form and submit it, wait for the platform administrator to approve. After approval, it will be automatically imported. After images are successfully imported, it can be used in evaluation tasks.

Currently, the platform only supports importing images that uses already have. It does not support building images on the platform using dockerfile. The dockerfile filled in by users is only used for platform approvers to review images.