Throughout history, people who like to think of themselves as high-minded have sneered at the masses, frittering their days away on “mindless entertainment”. The definition of “mindless” keeps changing: not so long ago, novels were considered a frivolous indulgence; then broadcasting took their place, and novel-reading became something that high-minded people did. For years, I told myself I wasn’t like the Average Person who watched four hours of TV a day (my average must be more like 15 minutes), because I was doing something much more brainy: surfing the internet. Recently, largely thanks to social media, it’s become impossible to ignore the fact that this is often mindless, too. So now, on my more self-disciplined days, I stay off social media, and feel slightly superior about it. And what do I do instead, since I’m far too smart to waste my life on rubbish. Now, I listen to podcasts.

So, naturally, I was intrigued by a recent essay on New York magazine’s website The Cut, by Sirena Bergman: “I listen to 35 hours of podcasts every week. Is that… bad?” Her conclusion: yes, partly. The brain needs silence, and the trouble with audio – like mobile internet, too – is that it doesn’t simply replace other forms of entertainment; rather, it seeps into the gaps (commutes, housework, exercise) that you might previously have used to be alone with your thoughts. Podcasts improve my daily life immensely and I’ve zero intention of abandoning them; but Bergman draws attention to an important truth about the content we incessantly consume: it’s quite possible to get addicted to stuff that seems edifying and intellectual, as well as to brainless nonsense. Indeed, for a certain kind of person, it’s probably easier. You know it’s a distraction to compulsively seek updates on reality shows hosted by Ant and Dec. It’s harder to remember that political news, or fascinating tales from the frontiers of science, might be serving the same function.

The point is that what makes something a distraction isn’t necessarily that it’s stupid or silly. It’s the role it’s playing in your life. If it’s helping you numb out, or put off important but scary tasks, or avoid asking tough questions about how you’re spending your time, it’s a problem, whatever the details. Seemingly productive work can easily be a distraction, if it’s not the work that counts. Even deeply meaningful activities can be distractions. That’s the logic behind a suggestion attributed to the investor Warren Buffett: first, write down your top 25 goals for life; then identify the most important five, focus on them, and avoid the other 20 like the plague – because they’re the seductive ones most likely to distract you, precisely because they do matter. They just don’t matter most.

From this perspective, “mindless entertainment” really isn’t the main danger. Yes, obviously, it’s a waste of time to watch four hours of (most) television a day. But that very obviousness means it’s hard to do by accident. It’s when you catch yourself feeling smug that you’re immune to that sort of thing that you really need to start worrying.

• oliver.burkeman@theguardian.com