Size of the preliminary evaluation set
by dymitrruta - Monday, July 5, 2021, 09:05:14

Dear Organizers,

Can I ask how big is the size of the preliminary evaluation set based on which you return the AUC scores on the leaderboard? It would be useful to understand how reliable the leaderboard feedback is. Cheers. Dymitr

RE: Size of the preliminary evaluation set
by andrzej - Tuesday, July 06, 2021, 15:17:53

Hello,

the size of our preliminary test set is 10% of the total test set size (2000 instances).

Best regards,
Andrzej Janusz