team removed from leaderboard

« Prev
Topic
» Next
Topic
DavidChudzicki's image
DavidChudzicki
Kaggle Admin
Posts 425
Thanks 106
Joined 21 Nov '10 Email user
From Kaggle

We have determined that a participant in another competition violated our rules by using multiple accounts to submit to that competition. As a result, the user has been stripped of all past accomplishments and disqualified from all competitions that they were participating in. You may notice the leaderboard shift.

There is strong evidence that this user was also submitting to this competition with multiple dummy accounts, and you may notice further leaderboard shifts as those are removed.

Please see the forum post in that competition for more details: https://www.kaggle.com/c/MerckActivity/forums/t/2925/merck-challenge-final-results/15918#post15918.

 
syntax's image Rank 23rd
Posts 7
Thanks 7
Joined 9 Jan '12 Email user

I can't help but wonder a few things.

  1. Was the total amount of submissions by Berry more or less than the sum of submissions allowed by the rules for those who only recently teamed up into these big teams in the top 10? I mean the "berry" account only had fewer than 100 submissions AFAIR and he would have to have used quite a lot of fake accounts to really top the accumulated number of submissions for some of those teams.
  2. Were "A Beautiful Mind" and "Mathcorner" some of his fake accounts? Those two accounts always seemed fishy to me since they seemed to often make submissions in close succession and also jumped up the leaderboard in record time. Also they stopped making submissions back in september, about the same time Berry stopped - presumably to focus on the Merck competition.
  3. Given that team Market Makers in their milestone 1 report mentioned "calibrating to the leaderboard" by using the optimized constant benchmark, Willem used "ridge regression based on the leaderboard scores" in the milestone 1 report, and that team Crescendo documented in their milestone 3 report that they were using "true averages" extracted from the leaderboard - There are no milestone winners who have not used information from the leaderboard to achieve their high scores. Can we have an official statement saying that this is ok? I mean as long as it is "fuzzy" and you do not know whether it is fair game or you will be disqualified at the last minute, it is not really fair - it translates the competition into who can extract the most information and get away with it.
Thanked by Sarkis
 
DavidChudzicki's image
DavidChudzicki
Kaggle Admin
Posts 425
Thanks 106
Joined 21 Nov '10 Email user
From Kaggle

Sorry, I believe that (1) and (2) aren't questions we would address publicly. I apologize. We are looking into those two accounts, however. Thanks for the tip.

On (3): Yes, I can confirm that using the public leaderboard scores as described in those reports is definitely fine.

 
syntax's image Rank 23rd
Posts 7
Thanks 7
Joined 9 Jan '12 Email user

DavidChudzicki wrote:

On (3): Yes, I can confirm that using the public leaderboard scores as described in those reports is definitely fine.

Thanks for clearing that up.

 

Reply

Flag alert Flagging is a way of notifying administrators that this message contents inappropriate or abusive content. Are you sure this forum post qualifies?