Having just submitted a 0.5138 score hopefully due to some bug I haven't tracked down yet, I'm voting in favor of back submittal. :-)

I don't think more than a few are needed.. 2 or 3 maybe?

On the other hand, this will upset the balance that is causing the conversation in this thread: http://www.heritagehealthprize.com/c/hhp/forums/t/815/hidden-rules-for-team-mergers/5338#post5338

Namely that, when merging teams, the combined team can't have more submissions than they would have been allowed if they were one team the whole time.  Queuing submissions or allowing back submissions is going to result in higher submission rates and will further limit the ability of teams to merge later.

i prefer back submissions, or what i immediately thought of as rollover submissions :), over queuing. i actually think it is a great idea. similar to wireless companies that truncate your minutes when you change your plan, the total number of back submissions accrued could be truncated to a limit when teams merge. that would avoid gaming the system by getting non-contributors to join your team to build up submissions. those who submit daily will continue to have the advantage because they will get feedback sooner, however those who cannot submit daily for whatever reason will not be penalized additionally by missing submissions. this setup retains the benefit highlighted by jeremy as well.

The idea of back submissions is nice but when implemented I will almost stop doing any submissions at all until after milestone 3 just to have a lot available when they are most useful. In my opinion making submissions early is not an advantage. A new comer who starts just now after milestone 1 will find a lot of useful information in the milestone 1 papers and can make better submissions now then I could when I started 6 months ago. Therefore I think it is best to view each milestone as a separate competition in terms of submissions. Within such a 6 month period between prizes having back submissions would be great. After a milestone all counters could be reset giving everyone equal chances for the next prize.

That is a very good point and makes total sense to me.

hi, newcomer here. i was wondering what is the rationale for throttling submissions to once a day? is it simply a matter of server resources? or is there some other reason for this?

@Lucidrains, If I had the ability to do infinite submissions, I could hit the server hundreds of times a day with semi-random submissions until I hit on a solution with a great score.

yeah, i understand, but once a day? i'm looking at the number of teams, and it just seems like for only 500 players, a well optimized server shouldn't even sweat if submissions were increased 10x a day.

this is just my perspective as a server administrator and website owner...

@lucidrains, You may be misunderstanding me.  The issue isn't one of server load.  The issue is that I could use a high number of submissions as a modeling technique.

yeah no i get that for sure. i just think once a day is a tad bit too infrequent. but that's just me.

thanks for the prompt response!

Any news regarding this issue? I strongly support the idea of having a submission queue!

I propose a very simple submission queue of 10 days (10 submissions) for the very simple reason of going on vacation and not wanting to lose submission chances.  Also the queue should be editable so that you can change your 10 submissions if your technique radically changes in the meantime.

While I don't think it's necessary to build an API, I think allowing this feature on the webpage would be really helpful (and less sadistic on Kaggle's part... had to log during memorial day weekend....) while not really affecting the nature of what is currently going on in the competition.

thanks

I think even if we didn't have an automatic queue - even having a little side cache of files on kaggle that you could then manually submit would be nice. That way I could use my smart phone when out and choose it instead of driving all the way back to my place to make a submission.

An API would be great.  Is there any chance of this (we can implement our own queues and manage them ourselves!)?

Thanks

We're attracted to the idea of having a 'bank' of extra submissions (besides the daily limit). Among other reasons, this would definitely be nice for people joining the contest late. I think it would be small, maybe 10 or so?

An API or a bank of submissions are also nice ideas -- not sure when we'd get to them, but I like it. But a small bank of submissions might be a smaller/easy quick improvement. (Yeah, I realize they're really two separate issues.)