THOUGH HE FREQUENTLY WORE a Cartier Tank watch, Andy Warhol didn’t use it to keep the time. “I wear a Tank because it’s the watch to wear,” Mr. Warhol once said, according to a September 2017 article in the New York Times. He would never even wind his wristwatch—it was for looks, not to look at.

Today, it seems, style-conscious men are adopting the Pop artist’s perspective, treating their increasingly redundant wristwatches more like jewelry than a tool—a move that traditionalists might instinctively want to ridicule. Judson Lee, 38, a director at an advertising agency in Fayetteville, Ark., owns three watches but almost never sets them. All of his timepieces are mechanical and therefore require winding for an accurate read, unlike quartz watches which can dependably rely on a battery. He thinks of each watch as “an object of style,” an accessory that rounds out an outfit as a bracelet would. He wasn’t always indifferent to conventional timekeeping: “Pre-iPhone I certainly set them, kept them at the correct time and date, all that jazz,” he said. These days, when he wants to check the hour of day, he simply takes out his iPhone.

With smartphones practically glued to our palms at all times and smartwatches muscling in, traditional timepieces are just no longer as vital as they once were in any practical sense. Fears among watch execs that the Apple Watch, introduced in 2015, would snatch up wrist real estate were confirmed in the fourth quarter of 2017 when industry researcher Canalys and IDC reported that Apple sold more watches than the entire Swiss watch industry combined. Ramsey Hidmi, 28, who works in asset management in Boston, said that few people he knows wear a watch at all, “and if they do, it’s an Apple Watch.” An outlier among them, Mr. Hidmi straps on a 12-year-old Rolex that he inherited from his father. Not that he ever bothers to set the time. “It’s an heirloom piece,” said Mr. Hidmi.

It’s not just casual wearers who rationalize strapping on comatose tickers, though. “I rely on my phone to tell me the time and I’m a mechanical watch collector,” said James Lamdin, the founder of Analog/Shift, a vintage watch retailer in New York City. With an extensive collection at his disposal, Mr. Lamdin sometimes switches watches multiple times a day, and he doesn’t pause to set each one. It’s more accurate just to check his phone. “Any digital time-keeping device, be it a phone or a G-Shock is going to keep much better time than any mechanical watch, no matter how high-grade,” he said, alluding to the fact that mechanical watches tend to speed up or slow down as the day goes on. “One does not wear a vintage watch to tell the time. You wear a vintage watch to experience the passage of time,” declared Mr. Lamdin, obliquely referring to an older watch’s potential to accrue value over the years.

Mr. Lamdin said he’s seen more “fashion forward” types gravitating towards that aged look of a vintage mechanical watch, with little interest in the ticking cogs beneath the glass. “It tends to be some of the super-fashiony maven people who are all about style,” he said of men who go that route, adding that he’s also seen collectors who will buy a nonfunctioning watch and wear it without having it properly serviced simply because they like it as a “statement piece.”