This is the one part of the article that is counterintuitive:
> Worryingly, poor methods still won—albeit more slowly. This was true in even the most punitive version of the model, in which labs received a penalty 100 times the value of the original “pay-off” for a result that failed to replicate, and replication rates were high (half of all results were subject to replication efforts).
How can bad results still confer a net reward on their producers with a penalty like that?
I went back and read the original description of the model, this is what I think is going on:
The average performance of a bad lab is worse then a good lab. However, a bad lab might get lucky and not have any of their false-positives subjected to replication. As a consequence, the top-performing labs tends to be bad labs that got lucky. The selection method heavily favors being the top performer, and thus the poor but lucky lab tends to win out, which is why it takes over the population.
This casts doubt for me on their model, since fitness proportionate selection would probably have quite different results.
> Worryingly, poor methods still won—albeit more slowly. This was true in even the most punitive version of the model, in which labs received a penalty 100 times the value of the original “pay-off” for a result that failed to replicate, and replication rates were high (half of all results were subject to replication efforts).
How can bad results still confer a net reward on their producers with a penalty like that?