I was wondering whether it would be possible to create a submission API so that we could submit from software?

It's fairly common for my team to pre-generate a few solutions then take a break. But we still have to log in every day to submit them. If there was an API we could just set up a cron job that uploads the solutions and e-mails us the result.

Failing that, could you provide the ability to upload multiple solutions and auto-submit one every 24 hours?

Queued submissions are an interesting idea.

What if you could fill your submission queue with many submissions (i.e. 2-3 weeks worth) and then at the end of each day if you hadn't used up your submission quota and had X entries left for the day (1 entry in the case of this competition), the next X entries in your queue would become active. It'd be just as if you submitted the entry at the end of the day. If we did it this way, you could still submit during the day and choose to use that submission immediately rather than wait in the the queue.

Alternatively, we could make it such that the concept of submissions is always first-in first-out (FIFO). Thus, if you upload a submission but already had submissions in the queue then your submission would go to the back of the queue. In this approach, the next queued submission would go live at midnight each day (and thus use up your submission for the entire day). This approach is slightly less flexible, but is a bit simpler to understand (and implement :)).

If I take the simpler FIFO approach, would that work for you?

I think a submission queue (either pure FIFO queue, or something that users can reorder) would be a great feature.

Interesting idea Jeff. I'm not sure whether I like it or not... I can see the convenience of it, but I also wonder it might lead to less good results. Forcing people to step back and think for 24 hours before submitting a new answer can be quite useful; maybe others are more disiplined than me, but I can recall many times that this forced me to think more laterally and deeply.

What do others think about this proposed feature?

like netflix :)

hmmm - I am torn on this. 

I almost didn't get in my daily sub yesterday due to a power outage :) ... but,

I don't think I like the idea of people just queuing up 100 submissions, but perhaps a middle ground - such as being able to have three to five in a queue - and if midnight comes with no sub - you can trigger the backups.

However - I don't have a very strong feeling one way or another - I guess I would like a SMALL queue if I were to choose.

I tend to agree, but see some merit to having a backup for when your day just doesn't go well :)

Normally I submit day-by-day because typically I generate a solution based on the previous result. But occasionally I'll be away (or want to be away) from the computer and want to pre-schedule some entries.
As I said earlier, my preference is for a submission API. It would also simplify regular submissions since I could automate some other steps that are currently done manually.
But if we're going the pre-scheduled route, I'd prefer a calendar where we could upload submissions for particular days - and replace or delete them as necessary. A straight up queue is a bit of a hassle to coordinate between multiple team members.

One other thing - if we can pre-schedule submissions, my preference is to be notified of the submission result by e-mail.
I guess a simple way to implement pre-scheduling (from a UI perspective) is to add a calendar to the "Make a submission" page. If a date is selected, the submission is scheduled for that date. And the "Submissions" page could show future as well as past entries.
It might make sense to schedule the time as well - some teams might want to submit ASAP so they can make use of the result, others may want to delay until the last moment in case they come up with something better.

Even a simple queue feature would be useful. Minimally, you need a queue and a way to delete pending (queued) submissions. Everything else is "nice to have".

I think submission queue will be a useful feature, there're times when you'll be away for days but don't want to miss the submission slots.

While we're on this subject, may I suggest (again) increasing the number of submissions per day based on the size of the team ? You can cap it at maximum of 4 or 5 submissions per day.

One submission per day is not a big deal for small teams, but if your team is bigger than a few people, it will be hurting you.

On second thought, the real issue here is your daily submission slot is gone if you don't use it. So perhaps a better solution is to allow unused submission slots to accumulate with a maximum limit (of 10 maybe).

Absolutely untrue: submission limits are handicaps for everyone, regardless of team size.  It would be grossly unfair to let larger teams have more submissions!

Personally, I think there is not much need for a submission queue, though it might help some teams.  What I think would really be beneficial is to change the current system to use a tickets metaphor.  Each team gets a fixed number of submission tickets for each milestone.  When you use them is up to you.  This maintains fair handicapping, but allows teams to match their submissions to their workflow.

As a bonus, such a system naturally lends itself to monetization for Kaggle.  For a small fee, any team could buy a bonus pack of submission tickets.  Obviously the fee should be small, and the maximum number of ticket add-ons should be limited, in the interest of fairness.  Such a system maintains the handicap -- any team can acquire the same total number of submission tickets as any other team -- and allows extra submissions for those that feel they need them.

Andy

Please don't change the rules. Period. This would be unfair to hard working submitters who are regularly submitting and enjoying the thrill of waiting for results.

1 Attachment —

• Untitled.png (20.43 KB)

I support no rule changes until Milestone boundaries.

I like the idea of a short FIFO queue, with the option of deleting queued entries. If the end of the day comes and I have not made a submission, the next one in the queue goes.

Maybe implement this after the milestone?

I think B Yang has the best idea.  Just allow people to back submit.