Evaluation Metrics

1. Precision@X or Accuracy@X

The basic evaluation metric of three benchmark datasets(refcoco, refcoco+ and refcocog) is Precision@X or Accuracy@X(short for Prec@X or Acc@X).

Prec@X means that when a referring expression/visual grounding model is performing inference(given a query image and a query sentence), the average ratio of the IOU of the predicted bounding box and the ground truth bounding box lager than the given threshold X(default: X=0.5) on all evaluation instances.

For example: if a dataset split has 1000 referring expression/visual grounding queries, and a referring expression/visual grounding model predicts results with 900 times IOU > 0.5, then the Prec@X equals 900/1000 == 90% == 0.9 .