3 years, 2 months ago

IEEE BigData 2021 Cup: Predicting Victories in Video Games

Predicting Victories in Video Games is a data mining challenge organized in association with IEEE BigData 2021 (http://bigdataieee.org/BigData2021/) conference. The task is to predict winners in Tactical Troops: Anthracite Shift (http://tacticaltroops.net), based on game logs. The competition is sponsored by QED Software (http://qed.pl/).

Emotions are part of the fun when you play games. One of the greatest excitements is the prediction of who is going to win. QED Software (qed.pl) comes up with the challenge which is related to the Tactical Troops: Anthracite video game (released in April 2021, tacticaltroops.net/) and organized in association with the 2021 IEEE International Conference on Big Data (IEEE BigData 2021).

Find a way to predict victories by analyzing logs from the game! More details regarding the task and the description of the challenge data sets can be found in the Task description section.

Special session at IEEE BigData 2021: A special session devoted to the challenge will be held at the IEEE BigData 2021 conference. We will invite authors of selected challenge reports to extend them for publication in the conference proceedings (after reviews by Organizing Committee members) and presentation at the conference. The publications will be indexed in the same way as regular conference papers. The invited teams will be chosen based on their final rank, innovativeness of their approach, and quality of the submitted report.

 

Terms & Conditions
 
 
Summary of the Challenge

The five-month-long IEEE BigData 2021 Cup: Predicting Victories in Video Games competition came to an end. 125 teams from 23 countries from all over the world competed to predict victories in video games. Congratulations to everyone who took up the challenge and submitted their solutions.

We are pleased to announce the winners of the competition.

Well done to Haitao Xiao, Yuling Liu, Dan Du, Zhigang Lu from the Cyan team who took the first place! The second place was awarded to Quang Hieu Vu from the henryvu team and the third to Kaishen Tseng who participated as the tks team.

The results of their work will be presented during a special session at the IEEE Big Data 2021 Conference.

This year's competition was sponsored by QED Software and the task was to predict victories based on game logs from Tactical Troops: Anthracite Shift (http://tacticaltroops.net), a game created by the sponsor. As promised, the best participants will be given codes, so that they can try playing themselves.

Rank Team Name Is Report   Preliminary Score Final Score Submissions
1
Cyan
True True 0.8930 0.899700 138
2
henryvu
True True 0.8929 0.898000 388
3
tks
True True 0.8920 0.897800 71
4
Dymitr
True True 0.8892 0.896300 363
5
GameOver
True True 0.8901 0.895000 216
6
PKAW
True True 0.8803 0.890800 32
7
Mathurin ACHE
True True 0.8802 0.890000 33
8
MapReduceEverything
True True 0.8809 0.888600 36
9
Rafał Czarnecki
True True 0.8764 0.884700 3
10
ryszard
True True 0.8740 0.883900 19
11
baseline solution
True True 0.8733 0.882200 7
12
player_2
True True 0.8739 0.881700 11
13
grzybola
True True 0.8739 0.881700 23
14
Krzysztof Witczyński
True True 0.8747 0.880600 6
15
Krzysztof Grabiec
True True 0.8731 0.879500 3
16
PFTW
True True 0.8720 0.879000 13
17
bs_dm
True True 0.8681 0.878600 2
18
i_hacked_test_engine
True True 0.8680 0.874500 29
19
DM42
True True 0.8663 0.874400 25
20
JG+AK
True True 0.8508 0.863700 12
21
Kasia
True True 0.8469 0.855200 2
22
Sparked
False True 0.8837 No report file found or report rejected. 54
23
msjj
False True 0.8773 No report file found or report rejected. 4
24
sink
False True 0.8765 No report file found or report rejected. 8
25
AndrewB
False True 0.8720 No report file found or report rejected. 3
26
Aidan
False True 0.8689 No report file found or report rejected. 2
27
Pietruszek team
False True 0.8675 No report file found or report rejected. 15
28
Debojit Mandal
False True 0.8654 No report file found or report rejected. 4
29
td
False True 0.8627 No report file found or report rejected. 6
30
Piotr Grabowski
False True 0.8624 No report file found or report rejected. 2
31
DeepTeam
False True 0.8546 No report file found or report rejected. 27
32
sn_twlo
False True 0.8507 No report file found or report rejected. 10
33
唐牛才是食神
False True 0.8225 No report file found or report rejected. 4
34
CryptoCrackers
False True 0.7829 No report file found or report rejected. 5
35
lgac
False True 0.7796 No report file found or report rejected. 20
36
nowy team
False True 0.7773 No report file found or report rejected. 13
37
RandomPredictorGoBrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
False True 0.7698 No report file found or report rejected. 4
38
DU
False True 0.7673 No report file found or report rejected. 8
39
hubert
False True 0.7659 No report file found or report rejected. 1
40
TJP2
False True 0.7638 No report file found or report rejected. 1
41
berendkatya
False True 0.5000 No report file found or report rejected. 1
42
stograd
False True 0.5000 No report file found or report rejected. 2
43
NehzUx
False True 0.5000 No report file found or report rejected. 2
Please log in to the system!

The task in this challenge is to come up with an efficient algorithm for predicting winners in the Tactical Troops: Anthracite Shift (http://tacticaltroops.net), based on game log data. The available data sets correspond to multiplayer games, played on a variety of maps. In particular, we provide data in three formats, differing in the granularity of game descriptions. Detailed descriptions of each data format are available in separate files in the Data files section.

Tabular format: The simplest representation of played games consists of an aggregated vector description of the situation on a map. Games in this representation are stored in the tabular format, as two CSV files, i.e., training_data_tabular_format.csv and test_data_tabular_format.csv. Each row in those tables corresponds to a single game with an identifier in the id column. Files containing other data representations (in different formats) follow this identifier as a file name. A detailed description of this format can be found in the Representation_1.rtf file.

Flattened logs format: This representation format is more granular than the Tabular Format, yet simpler to parse than the Truncated Logs format. In contrast to the latter, this format represents only the current state of the game, when the prediction is to be made. Each game is represented as a JSON file named ID.json, with many fields being the same as in Truncated Logs format. Refer to that representation description for the description of fields in the files. A detailed description of this format can be found in the Representation_2.rtf file.

Truncated logs format: This data collection contains slightly modified JSON log files generated by the Tactical Troops: Anthracite Shift game. Each file represents one match between two players and its name is the ID of a game, the same as in remaining representations. The files do not contain the full game description but are truncated after a number of turns. Some summary statistics of a match are also removed. Obviously, we removed information about who the winner is. Other game metadata such as player IDs are anonymized. Each JSON file contains a number of data structures, mostly in the form of lists. A detailed description of this format can be found in the Representation_3.rtf file.

Screenshots format: This data format contains screenshots generated by the Replay application of the Tactical Troops: Anthracite Shift game. (see an example at https://sensei.tacticaltroops.net/analysis/replay/126332/). Each JPEG file follows the naming scheme of ID.jpg and represents the state of the game at the decision timestamp. The dimensions of each image are compatible with the binary maps found in Metadata. Additionally, we provide base color map images for each game version, which may be found in the base_color_images.7z file. On the screenshots, players’ units, captured control points, and generators are color-marked in accordance with the values found in the Flattened and Truncated logs format under the ‘Players’ key in the ‘c’ field. For convenience, those values may also be found in the team_colors.csv file, which follows the same scheme as the Tabular format (colors.TEAM_ID.{r,g,b,hex}).

Unit, gadgets, maps, etc. metadata: Each game version comes with slightly different parameters of various items, changed as the game evolved. We provide such metadata for each version of the game. The details might be found in the Metadata.pdf file.

Remarks:

  • There is a large variety of weapons and weapon types with varying properties, e.g., plasma guns are not teleported (they fly over teleports), some guns shoot two bullets at once. Some projectiles have high piercing e.g. sniper bullets can fly through objects inflicting damage (one can kill two units with one shot). Some weapons, e.g., heavy machine guns can restrict the range of movement of a unit.
  • There is a wide array of gadgets in the TT:AS game. Some are destructive and can inflict damage (grenades, mines), some can be selected only once and have a passive effect (armors), some can decrease movement range (armors,  cloaks). Some last only one round (two consequent turn of both players).
  • Units (soldiers) come in four kinds: Assault, Heavy, Support, Scout. They vary in their moving range, initial HP, and special abilities (called ‘perks’). For e.g. Heavy can bash through boxes, Support can supplement HP of any nearby unit.

Solution format: The predictions for test instances from the test_data_tabular_format.csv table should be submitted to the online evaluation system as a textual file. The file should have exactly 20000 lines, and each line should contain exactly one number from the [0,1] interval indicating the chance that player 1 wins the game. The ordering of predictions should be the same as the ordering of instances in the test_data_tabular_format.csv table.

Evaluation: the quality of submissions will be evaluated using the AUC measure. Solutions will be evaluated online and the preliminary results will be published on the public leaderboard. The preliminary score will be computed on a small subset of the test records, fixed for all participants. The final evaluation will be performed after the completion of the competition using the remaining part of the test records. Those results will also be published online. It is important to note that only teams which submit a report describing their approach before the end of the challenge will qualify for the final evaluation.

The competition timeline:

  • April 30, 2021: start of the competition, datasets become available,
  • August 31September 30, 2021 (23:59:59 GMT): deadline for submitting the solutions,
  • September 5 October 3, 2021: deadline for sending the reports, end of the competition,
  • September 12 October 4-6, 2021: online publication of the final results, sending invitations for submitting papers to the associated workshop at the IEEE Big Data 2021 conference,
  • October 10 October 25, 2021: deadline for submitting invited papers,
  • October 25 November 1, 2021: notification of paper acceptance,
  • November 15, 2021: camera-ready versions of accepted papers due.

Authors of the top-ranked solutions (based on the final evaluation scores) will be awarded prizes funded by QED Software:

  • 1000 USD for the winning solution (+ the cost of one IEEE BigData 2021 registration)
  • 500 USD for the 2nd place solution (+ the cost of one IEEE BigData 2021 registration)
  • 250 USD for the 3rd place solution (+ the cost of one IEEE BigData 2021 registration)

The award ceremony will take place during the special track at IEEE BigData 2021 conference (online).

  • Andrzej Janusz, QED Software & University of Warsaw
  • Maciej Świechowski, QED Software & Warsaw University of Technology
  • Maciej Matraszek, QED Software & University of Warsaw
  • Dominik Ślęzak, QED Software & University of Warsaw

In case of any questions please post on the competition forum or write an email at contact {at} knowledgepit.ml

QED Software is an AI products company and a technological enablement partner for their clients - pioneers in their field. Their products reduce the uncertainty of Machine Learning models to align modeling with business understanding, to prioritize enterprise data efforts, to support experts in the ML loop, as well as to reduce data footprint and compactify the models so as to be used by the Internet of Things applications.

This forum is for all users to discuss matters related to the competition. Good manners apply!
  Discussion Author Replies Last post
Template and limit length of the paper Henry 3 by M
Sunday, October 24, 2021, 13:32:35
The end of IEEE BigData 2021 Cup: Predicting Victories in Video Games challenge Andrzej 1 by M
Thursday, October 07, 2021, 06:02:57
submission portal M 0 by M
Thursday, October 07, 2021, 07:05:07
can not upload the report pdf file M 2 by M
Tuesday, October 05, 2021, 05:52:24
Your website SSL certificate M 1 by M
Sunday, October 03, 2021, 08:29:24
about the final 3 submissions M 2 by M
Friday, October 01, 2021, 00:02:59
Deadline for sending the reports Ht 1 by Andrzej
Thursday, September 30, 2021, 12:38:06
sth. wrong with the submissions system and regarding the deadline of submission on Sept 30 M 5 by Andrzej
Thursday, September 30, 2021, 08:43:38
Updated submission system Maciej 0 by Maciej
Monday, September 13, 2021, 10:35:03
Competition schedule update Maciej 0 by Maciej
Monday, August 23, 2021, 16:50:46
Short maintenance break Andrzej 0 by Andrzej
Monday, July 19, 2021, 09:42:08
Size of the preliminary evaluation set Dymitr 1 by Andrzej
Tuesday, July 06, 2021, 15:17:53
Missing data 1 by Andrzej
Saturday, June 26, 2021, 09:44:46
Game ordering Timothy 4 by Timothy
Saturday, June 19, 2021, 11:27:50
New data representation available Maciej 0 by Maciej
Tuesday, June 01, 2021, 22:08:03
Metadata files updated Maciej 0 by Maciej
Wednesday, May 05, 2021, 14:22:05
sample for results. ngọc phương 1 by Maciej
Saturday, May 01, 2021, 17:37:11
Some doubt jayesh 1 by Maciej
Saturday, May 01, 2021, 17:34:27