Sweatpants are pajamas or housework clothes. You should not wear them in public unless you are under 14, older than 70, or are visibly ill and just going to the pharmacy. This goes for men and women. Also applies to flip flops, crocs and uggs, which are just high-top slippers.

"Don't leave the house in pajamas" isn't the same as "Oh, I have to dress like a sex goddess every time I leave the house to placate the evil patriarchy."