Could you provide an example how the evaluation metric will be computed based on basline solution from file exemplary_solution.txt? I don't know why there must be 10 lines in submission file, each line means a separate Naive Bayes classifier constructed with features pointed in the line?
You are correct. From the attributes in each line we construct three NB prediction models (for each of the decision attributes). Every model assigns scores to test cases, i.e. probabilities that the case should be classified to the positive decision class. In this way, for every decision attribute and every test case we have ten scores. We create an ensemble of predictions by taking their sum.
As the evaluation metric we use the average AUC of the prediction ensemble for different decision attributes, decreased by a penalty for using a large number of attributes.
I hope that this explanation is clear :-)
thanks! yes it's clear,Piotr
could you provide details of Naive Bayes implementation that you use to compute score? I found that in R language there is a difference between e1071 and klaR packages in Naive Bayes implementation which cause different outputs in prediction.