Removing just a tiny fraction of the crowdsourced data that informs online ranking platforms can significantly change the results.