There are about 38 thousand records in the Claims dataset that have one or more exact duplicates in all columns. Is this a data error? One possibility is, becaues we don't have the exact date, the same procedure has been done on multiple days of the same DSFS. With the limited number of columns that we have in the Claims data, it would be impossible to tell apart one claim from another if they are less than a month apart, and have everything else equal.

Any thoughts? Which is it, error in data or legitimate duplicates?

regards,

Sassoon

The question really is, is there a difference in the outcome?

I don't know the answer yet. It's a small number anyway, but I would like to know if these are valid duplicates so I don't have to worry about them. I think if the organizers can clairfy then why bother finding out the impact of these records one way or another.

yes there is a difference in outcomes.  If your grandpa said he went to the hospital three times in a month for chest pains as opposed to once... would you be more worried or less....?

My friend had a nerve conduction test and the doctor wanted her to come back for the second part. This seemed odd to me - as he wasn't busy and the other test was only supposed to take a few minutes. When I asked why he couldn't do it then - he admitted it was so he could submit two claims to the insurance companies. it was as close as I ever got to reporting a doctor, and on retrospect I wish I did so as he was CLEARLY optimizing his revenue stream over his patients healthcare.

Two claims from one doctor is not necessarily worse than one claim from another. There are all kinds of weird things that go on with healthcare - there is no reason I know of to assume that the doctors involved in this group are any more or less ethical/efficient than other doctors.

They really could be duplicates - we don't know, but my guess is it means SOMETHING - it is on my mental to do list although I don't think I'll ever have enough time to get to it.

Thanks for the responses. I had totally forgotten about my question. In fact, we deduped those duplicates and forgot about them. Maybe we should revisit though I doubt it would make any noticeable difference.

Bump. Until further notice - I'm assuming that duplicate claim rows are discrete claims with the same characteristics - so I added a "claim_id" field to track the difference. Organiser comments please?