The papers written by the milestone winners are now available as attachments to this forum post. As described in section 13 of the rules, if you have any concerns about these papers, you have 30 days from their posting to provide your feedback.

• crescendo_milestone3.pdf (208.52 KB)
• Milestone3_Willem_Edward 1.pdf (172.16 KB)

Both the docx and pdf files for Edward / Willem's paper appear to be missing formula and symbols at least on all the machine / software combinations I tried it on.

My browser also has a display problem with the Edward+Willem paper, but it works fine for me if I save it (edit: the docx version) to disk and open it with Word.

Thanks Signipinnis. I have downloaded and can open the file - there is text but formula appear to be missing. For example on the bottom of page 5 I have the sentence, "Now we have obtained the following variables: " and then ", , , , and" which suggest that something is missing. If someone could confirm more complete text, then the problem is at my end with the rendering. I've checked the docx xml code and there are only white space between the commas indicated above. The formula do not appear on pdf version, again on multiple systems.

Looks like pdf version corrupted. But docx is ok. Try pdf in attachment (converted from docx)

• Milestone3_Willem_Edward.pdf (573.60 KB)

Hi all, sorry for the trouble with the docx. Here the correct pdf.

• Milestone3_Willem_Edward.pdf (752.80 KB)

Thank you Haru and Willem. The new pdfs are good.

Congratulations, Willem & Edward team on your win.
Besides great modeling, you have solved the puzzle of special paydelay value=0, which is quite a realistic puzzle; it is typical in real life data that there are some special values combining different cases meaning of which is long lost.

We have a question from our team regarding table 1 (Claims distribution of paydelay vs Dsfs) in your paper.

From the paper, on what is counted in the table: "Important is that only claims are used that belong to members of Year3, that have a maximum DSFS value of "11-12" (month), because we then know to which real month all the claims belong."

When we select number of claims with DSFS=12 and Year=3 we get exactly same counts as in the table: 16044 claims with paydelay=0, 152 claims with paydelay=1...10, etc. However, for any other DSFS, we get counts much larger than in the table.
To be specific, let's consider the DSFS="8-9" and paydelay="111...120". That cell in the table has count 0, but we get 9.
Member 35764059 at year=3 has 1 claim with DSFS="11-12", but also at same year=3 has 3 claims with DSFS="8-9" and paydelay="111...120" (all 3 claims have paydelay=111). Member 47834891 at year=3 has a claim with DSFS="11-12", but also has 6 claims with DSFS="8-9" and paydelay="111..120" (these claims have paydelay=111,112,113,114,115 and 116). Thus, altogether we have 9 claims for DSFS="8-9" and paydelay="111...120".

Could you please let us know what do we miss here?
Thanks

Hi Team Crescendo. Congratulations on your milestone achievement.
I have a question about your 'post processing'. You work out the 'true average' values for the 4 categories. What scaling method did you then use? Did you use a multiplicative factor (eg * 1.1), or an additive factor (eg +0.1) to adjust your estimates? Or some other method?
Thanks
Dave Clark

Congrats! Willem & Edward Team on wining the milestone

The modelling on DSFS looks interesting. Yeah, I agree with Oleg Vasilyev the number do not matches with the paper also the papers lacks on details around the simulations used to attain the constants or interpolated values used to evaluate the offset values.

Could you please provide more detail on this?

Thanks

Hi Dave, 
We added a constant to match the average. 
Rie from crescendo

I wonder why my reply above got a rectangle and colors, though.

Congratulations to the milestone winners! Willem & Edward, very impressive improvement yet again. Crescendo, impressive improvement on the private data compared to the public leaderboard!

The crescendo report is very well written, thanks for taking the effort. I do have a few remaining questions though:

• What implementation of random forests did you use?

• The data sets described contain an unusually large number of variables for use in a tree based model (e.g. 14700 providerID count variables) Is this correct? If so, I assume you use some kind of sparse data format to keep all of this in memory?

• Could you please provide us with the weights of the individual runs in each of your 5 blended runs? Also it might be interesting to include the cross-validated rmse of each of these individual runs in the report.

Thanks in advance!

A question for team Crescendo: how did you deal with missing values ? For example, in feature set m1, what did you do for numeric Age value if the member's age is missing ?

Hi Tim,

Tim> What implementation of random forests did you use?

We used our own implementation.

Tim> The data sets described contain an unusually large number of variables for use in a tree based model (e.g. 14700 providerID count variables) Is this correct? If so, I assume you use some kind of sparse data format to keep all of this in memory?

Our code can handle sparse data efficiently.

Tim> Could you please provide us with the weights of the individual runs in each of your 5 blended runs? Also it might be interesting to include the cross-validated rmse of each of these individual runs in the report.

Thanks for suggestions, but we choose not to provide those numbers, which are not needed for replicating the winning entry. In the course of replicating the results based on the current documentation, one would obtain those numbers as side products. It'd be just too tedious to list up or even read so many numbers. I hope you understand.

Rie