HYATTSVILLE, Md. — An expert panel voted overwhelmingly on Wednesday for the Food and Drug Administration to impose strict new limitations on the multibillion-dollar testosterone drug industry, recommending that the agency tighten labels for the medicines so they are not prescribed to men who only have problems related to aging, such as low energy and libido. The F.D.A. often takes the advice of such panels.

Once used only to treat serious medical conditions, testosterone drugs are now taken by over two million American men, mostly for symptoms that are largely a result of aging, a pattern that has alarmed some medical experts. If the F.D.A. adopts the panel’s recommendations, it could sharply reduce the number of men prescribed the medicines and clamp down on marketing of them by drug companies. It could also limit coverage from insurance companies for their use, experts said.

“The whole idea is to try to rein in the inappropriate advertising and use of these drugs,” said Dr. Michael Domanski, the director of heart failure research at Mount Sinai School of Medicine, who was one of 20 members on the panel who voted to change the labeling. Just one member voted against the change.

Still, some experts were skeptical that usage would change much, because doctors can prescribe the drugs as they see fit, despite indications on the label.