Thanks guys (ChipMonkey, DavidC and Chris Raimondi) for your comments. Let me say right away that, of course, I accept Kaggle Admin's ruling as to whether the milestone winners' methods comply with the rules. (I might use the same techniques myself if my
methods get good enough.)
However, I still disagree that a combination of several algorithms should be considered "AN algorithm". It should be called "a method". Have a look at Market Makers' paper describing their method for their winning round 1 milestone.
Here is what they write:-
There were four underlying algorithms used in our models, all of which are freely available in the R language for statistical computing. Online references for each algorithm are given in the hyperlinks below.
- Gradient Boosting Machines ...
- Neural Networks...
- Bagged Trees...
- Linear Models...
I have deleted the hyperlinks for clarity. Market Makers go on to use "ensembling" which blends the results of the different algorithms.
Surely you guys wouldn't argue that a combination of four such different algorithms is AN algorithm. If you do I would have to give away my geographical location and say you are a mob of "bush lawyers"!
Yes I would - as mentioned before - many different things though of as a single algo - or even equation is actually a combination - or linear blend.
Is the the Pythagorean theorem an algo? Or is it a linear ensemble of A^2 plus B^2?
The fact that you are calling Bagged Trees - for example - AN ALGO - shows the problem with this appproach. The R package randomForest is simply a combination of CART trees - therefore - even a single random forest under what you are stating - wouldn't count as a single algo - as it was someone putting together a bunch of CART trees in a clever manner.
I understand what you are saying - and similar objections to practicality were raised during the netflix competition. Google uses over 200 different signals (what we call features) and a combination of algos, but their overall method - as you would call it - is still refered to as "The Google Algorithm". See here for example - the singular is used eight times - the plural never:
I do not disagree that MM and W&E used a combination of algorithims - I just disagree that you can't call that combination an algorithm as well.
I think you can combine four movies (in some cases) - and still consider it A MOVIE - just as you can put up 100s or thousands of orange pieces of cloth in central park and call it A PIECE OF ART. Should a cheeseburger be disqualified as the most delicious piece of food on the planet - simply because it combines cheese and a hamburger? Can the United Kingdom not be considered A COUNTRY, because it contains the countries of England, Scotland, et. al?
Not trying to be a smart ass - ok maybe a little bit :)