The papers written by the milestone winners are now available here. As described in section 13 of the rules, if you have any concerns about these papers, you have 30 days from their posting to provide your feedback.
Here are the answers to your questions:
A) The Gini Index has been used, as explained on the following (and follow up) webpages:
I) Yes, it is a subtraction being performed.
M) The update rule is presented in paragraph 3.3.
Thanks, Edward & Willem
@JC36: I don't see any problem, in fact I think that what is causing you confusion is all the work that goes above and beyond the actual algorithm.
All the prize winning entries, as far as I can tell, embody a method where you can take the data set, perform specific manipulations of it, and arrive, deterministically, at the predictions which scored best on the target dataset. This should fit any useful definition of "algorithm".
The difficult part is describing how the teams ARRIVED at these aglorithms, relying on compute-intensive non-deterministic (at least dependant on randomization) algorithms to CREATE the final algorithm. It does make the head hurt, but the resulting algorithm is largely independant of how it was created.
For example, I tend to run two algorithms based on the "gbm" and a "randomForest" packages in "R" and ensemble them. If I published the R code to do that (which is largely what one of the milestone winning teams has done), and rerun it mutliple times on multiple machines, the results will be different. However, if I pick one of those runs, I can SAVE the resulting models and apply them repeatedly to new or existing data sets, using the same ensembling math, and thereby function as a repeatable, deterministic, "single" algorithm. In theory that algorithm could be decomposed to some very basic math, although it's easier to talk about in the rich language of the modeling used to create it.
I hope that viewpoint helps. Basically, the large computational time tends to be used in creating an algorithm, and while the resulting algorithm could be computationally intensive itself, I haven't seen anything yet to indicate that it would be unusably so.
Just my opinion.
Thanked by kit001
Not to beat a dead horse, but a single random forest could in and of itself be considered an ensemble and not a single algo under some very strict definitions. I believe the generic name is something like "ensemble of trees" after all. Also, training time is usually much longer than prediction time. I have algos that take overnight and possibly days to train that will make predictions for new data in under 10 minutes. Further, viewing these as impractical is kind of like looking at Lance Armstrong's gram sized reduction in clothing as unpractical for the average rider, while ignoring all the other advances in understanding the sport & human body that the top tour contestants have made. The less restrictions made upon contestants in my mind the more they can concentrate on what works - practical or not - something that appears impractical now might be more practical in the future. A retina display would have been impractical 10 years ago - now I can't live without it (sligh exaggeration) - without people working on the cutting edge we would be living with mediocrity.
Thanked by Sarkis
Yes I would - as mentioned before - many different things though of as a single algo - or even equation is actually a combination - or linear blend.
Is the the Pythagorean theorem an algo? Or is it a linear ensemble of A^2 plus B^2?
The fact that you are calling Bagged Trees - for example - AN ALGO - shows the problem with this appproach. The R package randomForest is simply a combination of CART trees - therefore - even a single random forest under what you are stating - wouldn't count as a single algo - as it was someone putting together a bunch of CART trees in a clever manner.
I understand what you are saying - and similar objections to practicality were raised during the netflix competition. Google uses over 200 different signals (what we call features) and a combination of algos, but their overall method - as you would call it - is still refered to as "The Google Algorithm". See here for example - the singular is used eight times - the plural never:
I do not disagree that MM and W&E used a combination of algorithims - I just disagree that you can't call that combination an algorithm as well.
I think you can combine four movies (in some cases) - and still consider it A MOVIE - just as you can put up 100s or thousands of orange pieces of cloth in central park and call it A PIECE OF ART. Should a cheeseburger be disqualified as the most delicious piece of food on the planet - simply because it combines cheese and a hamburger? Can the United Kingdom not be considered A COUNTRY, because it contains the countries of England, Scotland, et. al?
Not trying to be a smart ass - ok maybe a little bit :)
Thanked by ChipMonkey