This article originally appeared on The Conversation, and is republished under a Creative Commons licence.

Bestselling business books promise to teach you the winning formula and reveal the secrets of success. But the inconvenient truth is that exceptional successes in business are largely based on luck. No rule exists for achieving exceptional performance because it usually requires doing something different or novel and there can be no recipe for such innovation.

My new research provides systematic evidence that luck plays a critical role in such performance, not only in business but also in music, movies, science and professional sports. A key finding is that more can be gained by paying more attention to “second best”.

Let’s look at the music industry. If a new band or musician has a top-20 hit, should a music label immediately try to sign them? My analysis of 8,297 acts in the US Billboard 100 from 1980 to 2008 would suggest not. Music label bosses should instead be looking to sign up those reaching positions between 22 and 30, the “second best” in the charts.