Why you can trust it

A reasonable worry is that crowd votes are just noise. The authors checked this carefully and found two reassuring things. The crowdsourced questions were diverse and genuinely discriminating, meaning they actually separate strong models from weak ones rather than being trivially easy. And the crowd's votes agreed well with the judgments of expert raters, suggesting the wisdom of the crowd here is sound, not random.