Predicting Victories in Video Games

2 years, 6 months ago

IEEE BigData 2021 Cup: Predicting Victories in Video Games

Predicting Victories in Video Games is a data mining challenge organized in association with IEEE BigData 2021 (http://bigdataieee.org/BigData2021/) conference. The task is to predict winners in Tactical Troops: Anthracite Shift (http://tacticaltroops.net), based on game logs. The competition is sponsored by QED Software (http://qed.pl/).

Emotions are part of the fun when you play games. One of the greatest excitements is the prediction of who is going to win. QED Software (qed.pl) comes up with the challenge which is related to the Tactical Troops: Anthracite video game (released in April 2021, tacticaltroops.net/) and organized in association with the 2021 IEEE International Conference on Big Data (IEEE BigData 2021).

Find a way to predict victories by analyzing logs from the game! More details regarding the task and the description of the challenge data sets can be found in the Task description section.

Special session at IEEE BigData 2021: A special session devoted to the challenge will be held at the IEEE BigData 2021 conference. We will invite authors of selected challenge reports to extend them for publication in the conference proceedings (after reviews by Organizing Committee members) and presentation at the conference. The publications will be indexed in the same way as regular conference papers. The invited teams will be chosen based on their final rank, innovativeness of their approach, and quality of the submitted report.

Terms & Conditions

Competition Participation Rules for “IEEE BigData 2021 Cup: Predicting Victories in Tactical Troops: Anthracite Shift Game”

By entering this competition you accept these official competition rules.

Organizer

The competition is organized by QED Software sp. z o.o. (qed.pl)
The competition is organized via the KnowledgePit platform (available at knowledgepit.ai or knowledgepit.ml) and any submissions made outside of the platform will not be admissible.

Entry

The competition is open to all interested researchers, specialists, and students. Members of the Contest Organizing Committee (see the dedicated section on the competition page) and employees of QED Software are not allowed to participate.
Participants must be of legal age of majority (Check your country’s law, in Poland and in most countries it is 18 years of age).
Registration for the competition is done via the KnowledgePit platform (knowlegepit.ai).

Timeline

A detailed competition time schedule is available in the Schedule section (below) of the competition page.

Participants and Teams

A person may enroll in the competition with only one KnowledgePit user account. Using multiple accounts constitutes grounds for exclusion from the competition.
Participants submit their solutions as members of teams made up of one or more persons.
Each participant may be a member of only one team enrolled in the challenge.
Each team must designate one of the team members as the team leader responsible for communication with the Organizer.
A single KnowledgePit account can only be associated with one team at a time. It is not possible to withdraw from a team, but teams can be merged.
Merging is done between teams by their leader through the KnowledgePit platform and requires the consent of both teams.
Participation in the competition is free and voluntary.
Privately sharing elements of the solution such as code or data outside of teams is not permitted and may result in disqualification of the persons or teams involved, however, it is permitted to share remarks and ideas with all participants on the Knowledge Pit Competition Forums.
Participants must use their own resources, particularly software and other necessary tools and equipment needed to prepare a solution to be submitted.

Solutions

Participants submit their solutions through the Knowledge Pit platform.
The submitted solution is checked for errors, and if none is found, the solution is immediately included in the public leaderboard.
There is a strict limit for the total number of solutions that can be submitted by a team during the Competition and also, for the number of solutions that can be submitted daily.
For this competition, those limits are set to 400 and 10, respectively.
The daily submission limit resets at 11:59 pm GMT.
When two teams merge, their solutions accumulate, but the limits remain unchanged.
If a team exceeds the limit for the total number of submissions, the team will be unable to submit new solutions until the competition ends.
Team can select up to three solutions for the final evaluation, and the best of them will constitute the team’s final score (final solution).
If a team fails to select their 3 solutions for final evaluation, then this team’s solution with the highest score on the public leaderboard is automatically selected as the team’s final solution.

Evaluation

Each team is obliged to provide a short report describing their final solution. The report must contain information such as the name of the team, the names of all team members, and a brief overview of the approach used in the solution. The description should explain all data pre-processing steps and model construction steps. It should be submitted in the KnowledgePit platform submission system and may require a document in the pdf format by the report submission date specified in the Schedule section of the competition page (below).
The final evaluation takes place after the expiry of the report submission date.
After the final evaluation, three top-ranked teams will be asked to provide the source codes that can be used to reproduce their final solutions and documentation that would allow running the code. If the code requires to be run within a complex environment (e.g. distributed Hadoop cluster), a detailed setup explanation should be provided as well. The source codes will be used to verify the legitimacy of the solutions.
For a team to be eligible for an award in this competition, a team must be one of the top-ranked teams that provided their pdf reports and whose source codes were provided upon request and verified for their legitimacy. From teams that have met the above 3 criteria cumulatively within respective deadlines, the winners of 1^st, 2^nd, and 3^rd prizes respectively will be announced.
Additionally, in this competition, the teams are eligible for money prizes only if their final solution improves the baseline score by at least 10%.
Organizers hold the right to extend the deadlines for submitting solutions and/or reports. In such a case, they will inform participants about such an extension using the KnowledgePit platform Competition Forum.
Organizers are not responsible for any consequences of technical issues related to the participant’s ability to access or submit to the KnowledgePit platform, especially for issues relating to internet connectivity. If due to technical problem within the Organizer’s control, the resulting delays or temporary unavailability of the platform or its components are likely to impact the participant’s ability to timely submit the team’s solution, the challenge deadline will be extended by the Organizer by the time of such unavailability, and participants will be informed of such extension by email.
The final ranking of the competing teams will be published based on the final evaluation results.
In the case of draws in the evaluation scores, the time of the submission will be taken into account.

Prizes

Prizes compose of a monetary amount and the ability for a selected team member to register for the 2021 IEEE International Conference on Big Data (coverage of a registration fee). The prizes are listed in the dedicated section of the competition page.
QED Software will cover the costs of 3 registration fees, one for a representative member of the 3 teams with top solutions. Each winning team must select their representative member and notify their selection to the Organizer.
QED Software will award monetary prizes to the winning teams: each winning team shall notify the Organizer of how the monetary prize will be paid to the team (single payment, split payments, etc.).
In order to receive the prize, the winning participant is required to submit to the Organizer documents necessary for the remittance of such prize, such as their personal detail, including bank account number, their certificate of tax residency issued by the tax authority of their place of residence or other documents required by law. In absence of the necessary documents, the monetary prize cannot be paid out.
The winning participant who fails or refuses to provide the necessary legal documentation such as defined above will be deemed to have forfeited the monetary prize, and such prize will not be paid out. The winning team may select another of its members as the recipient of such prize.
In accordance with the law, the Organizer shall cover the tax payments required on the awarded prizes at the rates and on terms prescribed by law in the country of the Organizer (Poland).
The Organizer is not responsible in any way for any duties, levies, fees or taxes that the winning participant may be subject to in his or her country of residence. It is the sole responsibility of the winning participant to inquire at the appropriate authorities about applicable documentation and/or tax payments that she or he may be required to submit.

Claims

Submissions are not admissible if they are in whole or part illegible, incomplete, damaged, altered, counterfeit, obtained through fraud, or late, or made or submitted in breach of these Rules.
Participant may be disqualified if the Organizers reasonably believes that the participant has attempted to undermine the legitimate operation of the Competition by using multiple accounts, cheating, deception, or other unfair playing practices or abuses, threatens or harassment towards other participants or the Organizer.
Organizer may at its discretion reject any submission or disqualify any participant if the Organizer reasonably beliefs that it was – respectively - produced in an unfair or illegitimate way or submitted by a person who has broken the challenge rules without providing any additional explanation.
If you think you were excluded without reason, or have any questions, please contact the Organizer at contact@knowledgepit.ml.

Data security, Privacy and Copyright

Each participant agrees to use reasonable and suitable measures to prevent persons who have not formally agreed to these Rules from gaining access to the competition data. You agree not to transmit, duplicate, publish, redistribute or otherwise provide or make available the competition data to any party not participating in the competition. You agree to notify the Organizer immediately upon learning of any possible unauthorized transmission of or unauthorized access to the competition data and agree to work with the Organizer to rectify any unauthorized transmission or access.
Each report, paper, and any other type of publication based on the team’s or participant’s research where data from this competition is used should accredit KnowledgePit and QED Software as the institutions that provided data for the study.
The fact of accepting the award is equivalent to granting the organizers a worldwide, non-exclusive, sub-licensable, transferable, royalty-free, perpetual, and irrevocable right to use, reproduce, distribute, create derivative works of, publicly perform, publicly display, digitally perform, make, have made, sell, offer for sale and/or import the winning submission and the source code used to generate it, in any media now known or hereafter developed, for any purpose whatsoever, commercial or otherwise, without further approval or any payment to the participant. By accepting the award the participants also acknowledge that they have full and unrestricted rights to grant the aforementioned rights.
By enrolling in this competition, the participant grants his/her consent for the processing of her/his registration data and the submissions and reports, and grant the Organizer the rights to use such data and submissions for the purpose of evaluation and in post-competition research.
By accepting the award the participant grants the Organizer the right to use the participant's name, affiliation, and/or prize information for the purpose of informational and promotional purposes of the competition and the KnowledgePit platform in any medium without additional compensation.
Participant’s data is administered by QED Software sp. z o.o., and shall be processed in accordance with the European data protection and privacy rules (The General Data Protection Regulation (EU) 2016/679). For more information check Organizer’s Privacy Policy.

Final arrangements

The Organizer reserves the right to modify the Rules of this Competition, including without limitation for the purpose of clarification, correcting obvious editing mistakes, extending the deadlines for the benefit of participants or other minor amendments. In the event of any change to the Rules, participants will be informed of them via the competition forum.
Unless otherwise provided in the competition Rules above, all claims arising out of or relating to these Rules will be governed by Polish law, and will be litigated in Poland. Participant consent to personal jurisdiction in those courts.
If any provision of these Rules is held to be invalid or unenforceable, all remaining provisions of the Rules will remain in full force and effect.

Summary of the Challenge

The five-month-long IEEE BigData 2021 Cup: Predicting Victories in Video Games competition came to an end. 125 teams from 23 countries from all over the world competed to predict victories in video games. Congratulations to everyone who took up the challenge and submitted their solutions.

We are pleased to announce the winners of the competition.

Well done to Haitao Xiao, Yuling Liu, Dan Du, Zhigang Lu from the Cyan team who took the first place! The second place was awarded to Quang Hieu Vu from the henryvu team and the third to Kaishen Tseng who participated as the tks team.

The results of their work will be presented during a special session at the IEEE Big Data 2021 Conference.

This year's competition was sponsored by QED Software and the task was to predict victories based on game logs from Tactical Troops: Anthracite Shift (http://tacticaltroops.net), a game created by the sponsor. As promised, the best participants will be given codes, so that they can try playing themselves.

Final results

Rank	Team Name	Is Report	Preliminary Score	Final Score	Submissions
1	Cyan	True	0.8930	0.899700	138
2	henryvu	True	0.8929	0.898000	388
3	tks	True	0.8920	0.897800	71
4	Dymitr	True	0.8892	0.896300	363
5	GameOver	True	0.8901	0.895000	216
6	PKAW	True	0.8803	0.890800	32
7	Mathurin ACHE	True	0.8802	0.890000	33
8	MapReduceEverything	True	0.8809	0.888600	36
9	Rafał Czarnecki	True	0.8764	0.884700	3
10	ryszard	True	0.8740	0.883900	19
11	baseline solution	True	0.8733	0.882200	7
12	player_2	True	0.8739	0.881700	11
13	grzybola	True	0.8739	0.881700	23
14	Krzysztof Witczyński	True	0.8747	0.880600	6
15	Krzysztof Grabiec	True	0.8731	0.879500	3
16	PFTW	True	0.8720	0.879000	13
17	bs_dm	True	0.8681	0.878600	2
18	i_hacked_test_engine	True	0.8680	0.874500	29
19	DM42	True	0.8663	0.874400	25
20	JG+AK	True	0.8508	0.863700	12
21	Kasia	True	0.8469	0.855200	2
22	Sparked	False	0.8837	No report file found or report rejected.	54
23	msjj	False	0.8773	No report file found or report rejected.	4
24	sink	False	0.8765	No report file found or report rejected.	8
25	AndrewB	False	0.8720	No report file found or report rejected.	3
26	Aidan	False	0.8689	No report file found or report rejected.	2
27	Pietruszek team	False	0.8675	No report file found or report rejected.	15
28	Debojit Mandal	False	0.8654	No report file found or report rejected.	4
29	td	False	0.8627	No report file found or report rejected.	6
30	Piotr Grabowski	False	0.8624	No report file found or report rejected.	2
31	DeepTeam	False	0.8546	No report file found or report rejected.	27
32	sn_twlo	False	0.8507	No report file found or report rejected.	10
33	唐牛才是食神	False	0.8225	No report file found or report rejected.	4
34	CryptoCrackers	False	0.7829	No report file found or report rejected.	5
35	lgac	False	0.7796	No report file found or report rejected.	20
36	nowy team	False	0.7773	No report file found or report rejected.	13
37	RandomPredictorGoBrrrrrrrrrrrrrrrrrrrrrrrrrrrrr	False	0.7698	No report file found or report rejected.	4
38	DU	False	0.7673	No report file found or report rejected.	8
39	hubert	False	0.7659	No report file found or report rejected.	1
40	TJP2	False	0.7638	No report file found or report rejected.	1
41	berendkatya	False	0.5000	No report file found or report rejected.	1
42	stograd	False	0.5000	No report file found or report rejected.	2
43	NehzUx	False	0.5000	No report file found or report rejected.	2

Enroll

Please log in to the system!

Task description

The task in this challenge is to come up with an efficient algorithm for predicting winners in the Tactical Troops: Anthracite Shift (http://tacticaltroops.net), based on game log data. The available data sets correspond to multiplayer games, played on a variety of maps. In particular, we provide data in three formats, differing in the granularity of game descriptions. Detailed descriptions of each data format are available in separate files in the Data files section.

Tabular format: The simplest representation of played games consists of an aggregated vector description of the situation on a map. Games in this representation are stored in the tabular format, as two CSV files, i.e., training_data_tabular_format.csv and test_data_tabular_format.csv. Each row in those tables corresponds to a single game with an identifier in the id column. Files containing other data representations (in different formats) follow this identifier as a file name. A detailed description of this format can be found in the Representation_1.rtf file.

Flattened logs format: This representation format is more granular than the Tabular Format, yet simpler to parse than the Truncated Logs format. In contrast to the latter, this format represents only the current state of the game, when the prediction is to be made. Each game is represented as a JSON file named ID.json, with many fields being the same as in Truncated Logs format. Refer to that representation description for the description of fields in the files. A detailed description of this format can be found in the Representation_2.rtf file.

Truncated logs format: This data collection contains slightly modified JSON log files generated by the Tactical Troops: Anthracite Shift game. Each file represents one match between two players and its name is the ID of a game, the same as in remaining representations. The files do not contain the full game description but are truncated after a number of turns. Some summary statistics of a match are also removed. Obviously, we removed information about who the winner is. Other game metadata such as player IDs are anonymized. Each JSON file contains a number of data structures, mostly in the form of lists. A detailed description of this format can be found in the Representation_3.rtf file.

Screenshots format: This data format contains screenshots generated by the Replay application of the Tactical Troops: Anthracite Shift game. (see an example at https://sensei.tacticaltroops.net/analysis/replay/126332/). Each JPEG file follows the naming scheme of ID.jpg and represents the state of the game at the decision timestamp. The dimensions of each image are compatible with the binary maps found in Metadata. Additionally, we provide base color map images for each game version, which may be found in the base_color_images.7z file. On the screenshots, players’ units, captured control points, and generators are color-marked in accordance with the values found in the Flattened and Truncated logs format under the ‘Players’ key in the ‘c’ field. For convenience, those values may also be found in the team_colors.csv file, which follows the same scheme as the Tabular format (colors.TEAM_ID.{r,g,b,hex}).

Unit, gadgets, maps, etc. metadata: Each game version comes with slightly different parameters of various items, changed as the game evolved. We provide such metadata for each version of the game. The details might be found in the Metadata.pdf file.

Remarks:

There is a large variety of weapons and weapon types with varying properties, e.g., plasma guns are not teleported (they fly over teleports), some guns shoot two bullets at once. Some projectiles have high piercing e.g. sniper bullets can fly through objects inflicting damage (one can kill two units with one shot). Some weapons, e.g., heavy machine guns can restrict the range of movement of a unit.
There is a wide array of gadgets in the TT:AS game. Some are destructive and can inflict damage (grenades, mines), some can be selected only once and have a passive effect (armors), some can decrease movement range (armors, cloaks). Some last only one round (two consequent turn of both players).
Units (soldiers) come in four kinds: Assault, Heavy, Support, Scout. They vary in their moving range, initial HP, and special abilities (called ‘perks’). For e.g. Heavy can bash through boxes, Support can supplement HP of any nearby unit.

Solution format: The predictions for test instances from the test_data_tabular_format.csv table should be submitted to the online evaluation system as a textual file. The file should have exactly 20000 lines, and each line should contain exactly one number from the [0,1] interval indicating the chance that player 1 wins the game. The ordering of predictions should be the same as the ordering of instances in the test_data_tabular_format.csv table.

Evaluation: the quality of submissions will be evaluated using the AUC measure. Solutions will be evaluated online and the preliminary results will be published on the public leaderboard. The preliminary score will be computed on a small subset of the test records, fixed for all participants. The final evaluation will be performed after the completion of the competition using the remaining part of the test records. Those results will also be published online. It is important to note that only teams which submit a report describing their approach before the end of the challenge will qualify for the final evaluation.

Data files

In order to download competition files you need to be enrolled.

Schedule

The competition timeline:

April 30, 2021: start of the competition, datasets become available,
~~August 31~~September 30, 2021 (23:59:59 GMT): deadline for submitting the solutions,
~~September 5~~ October 3, 2021: deadline for sending the reports, end of the competition,
~~September 12~~ October 4-6, 2021: online publication of the final results, sending invitations for submitting papers to the associated workshop at the IEEE Big Data 2021 conference,
~~October 10~~ October 25, 2021: deadline for submitting invited papers,
~~October 25~~ November 1, 2021: notification of paper acceptance,
November 15, 2021: camera-ready versions of accepted papers due.

Awards

Authors of the top-ranked solutions (based on the final evaluation scores) will be awarded prizes funded by QED Software:

1000 USD for the winning solution (+ the cost of one IEEE BigData 2021 registration)
500 USD for the 2nd place solution (+ the cost of one IEEE BigData 2021 registration)
250 USD for the 3rd place solution (+ the cost of one IEEE BigData 2021 registration)

The award ceremony will take place during the special track at IEEE BigData 2021 conference (online).

Competition organizing committee

Andrzej Janusz, QED Software & University of Warsaw
Maciej Świechowski, QED Software & Warsaw University of Technology
Maciej Matraszek, QED Software & University of Warsaw
Dominik Ślęzak, QED Software & University of Warsaw

In case of any questions please post on the competition forum or write an email at contact {at} knowledgepit.ml

Discussion	Author	Replies	Last post
Template and limit length of the paper	Henry	3	by M Sunday, October 24, 2021, 15:32:35
The end of IEEE BigData 2021 Cup: Predicting Victories in Video Games challenge	Andrzej	1	by M Thursday, October 07, 2021, 08:02:57
submission portal	M	0	by M Thursday, October 07, 2021, 09:05:07
can not upload the report pdf file	M	2	by M Tuesday, October 05, 2021, 07:52:24
Your website SSL certificate	M	1	by M Sunday, October 03, 2021, 10:29:24
about the final 3 submissions	M	2	by M Friday, October 01, 2021, 02:02:59
Deadline for sending the reports	Ht	1	by Andrzej Thursday, September 30, 2021, 14:38:06
sth. wrong with the submissions system and regarding the deadline of submission on Sept 30	M	5	by Andrzej Thursday, September 30, 2021, 10:43:38
Updated submission system	Maciej	0	by Maciej Monday, September 13, 2021, 12:35:03
Competition schedule update	Maciej	0	by Maciej Monday, August 23, 2021, 18:50:46
Short maintenance break	Andrzej	0	by Andrzej Monday, July 19, 2021, 11:42:08
Size of the preliminary evaluation set	Dymitr	1	by Andrzej Tuesday, July 06, 2021, 17:17:53
Missing data		1	by Andrzej Saturday, June 26, 2021, 11:44:46
Game ordering	Timothy	4	by Timothy Saturday, June 19, 2021, 13:27:50
New data representation available	Maciej	0	by Maciej Wednesday, June 02, 2021, 00:08:03
Metadata files updated	Maciej	0	by Maciej Wednesday, May 05, 2021, 16:22:05
sample for results.	ngọc phương	1	by Maciej Saturday, May 01, 2021, 19:37:11
Some doubt	jayesh	1	by Maciej Saturday, May 01, 2021, 19:34:27