Dov Charney once tried to buy me a car.

I don’t remember what kind of car it was exactly, but I don’t think it was a particularly fancy one—a Hyundai or something like that. And there were some strings, he would buy it, and at some indefinite point in the future I’d have to take over the payments.

Like I said, I don’t remember the exact specifics, but I remember my response: “That’s very generous of you. I appreciate it, but no, thank you. I’m OK.”

It wasn’t just that I was perfectly happy driving a 1997 Volvo with 160,000 miles on it. It was that I have an aversion to debts and entanglements, and however well-meaning the offer was, an entanglement was certainly part of the intention.

In his biography of Lyndon Johnson, Robert Caro tells the story of Johnson attempting to recruit a man named John Hicks to work for him. At a meeting at a diner in Austin, Johnson made his pitch: “I’m going to lend you ten thousand dollars,” he said, “And I want you to take it and buy yourself a Cadillac car. And I want you to move to a better apartment. I want you to be somebody. Furnish the apartment. Get [your wife] a fur coat. I want you to [join some local clubs] and be somebody here in Austin.”

Hicks was surprised. How would I ever pay you back, he asked Johnson. Johnson simply smiled and said, “Johnny, don’t worry about that. You let me worry about that.”

Certainly offers like this are champagne problems. Most people are struggling to get noticed, to get an opportunity at all. To be able to turn down a gift or a job offer is a privilege. Most of us would kill to have a future president offer us a car, and many people need a car, period. Still, this privileged position is not without its perils.

It’s a dangerous game that goes back further than Lyndon Johnson offering a guy a Cadillac. Seneca, the Roman statesman and writer, spoke often about wealthy Romans who have spent themselves into debt and the misery and dependence this created for them. Slavery, he said, often lurks beneath marble and gold. Yet, his own life was defined by these exact debts. With his own fortune, he made large loans to a colony of Britain at rates so high it eventually destroyed their economy. And what was the source of this fortune? The Emperor Nero was manipulatively generous with Seneca, bestowing upon him numerous estates and monetary awards in exchange for his advice and service. Seneca probably could have said no, but after he accepted the first one, the hooks were in. As Nero grew increasingly unstable and deranged, Seneca tried to escape into retirement but he couldn’t. He pushed all the wealth into a pile and offered to give it back with no luck.

Eventually, death—a forced suicide—was the only option. Money in, blood out.

This is only a slightly more dramatic illustration of the trap we find ourselves in. We take out student loans to pay for an education that will get us a job we hope will make those crushing payments worth it. We go to the bank and ask them how much house they’ll let us buy and then we hope two people working every day for the next forty years will prove them right.

All of us regularly say yes unthinkingly, or out of vague attraction, or out of greed or vanity. Because we can’t say no—because we might miss out on something if we did. We think “yes” will let us accomplish more, will give us more of what we want, when in reality it prevents exactly what we seek. All of us waste precious life doing things we don’t like, to prove ourselves to people we don’t respect, and to get things we don’t want.

I read an article a few weeks ago about a law firm in Houston that pays for a private jet for its associates to fly back and forth to California. It was presented as a perk of the job: Housing prices in San Francisco are steep, so this way the employees can enjoy living in Texas while still benefiting from the brisk technology market in California. This isn’t a perk. It’s a bribe, as Upton Sinclair put it. It’s the normalization of an utterly abnormal status quo—one that to sustain, the associates have to work incredibly long hours in an incredibly unpleasant job. But once the hooks are in? It’s hard to get them out.

The reason we work so hard is for “financial freedom.” Somehow we always seem to end up awfully unfree, don’t we? David “DHH” Heinemeier Hansson has talked about the delusion of “Fuck You Money” (having so much you can say, “Fuck you” to people asking you to do stuff you don’t want to do). How many fuck yous are we hearing from these people, he asks. The truth is: Not many. That’s the trap.

The irony of that offer from Dov, I knew, was that he might be giving me a car but part of the reason was to make sure I wouldn’t go anywhere. Stuck with the payments, grateful for the gift, how could I question things? How could I pursue the life I wanted? The answer was that I wouldn’t be able to. And I saw that happen. Other people who hadn’t been able to say no—for personal reasons, for financial reasons, because they didn’t see the strings—to cars or green cards or apartments or positions of power were stuck when the company began to fall apart. As things spun out of control, and lines—ethical and otherwise—were crossed, they were complicit. They were blinded, too, to what they were doing.

The ancient philosophers understood and warned against this. As Epicurus put it, “Self-sufficiency is the greatest of all wealth.” The Stoic philosopher Epictetus has said that “wealth consists not in having great possessions, but in having few wants.” There is also a story about Socrates. He turned down an invitation from Archelaus, the king of Macedon, because he wanted to “avoid dying a thousand deaths.” Because to him accepting a favor he couldn’t pay back, that created dependency, was worse than death. It was compromising his freedom. It was slavery.

We instinctively grasp the difficulty of Socrates’ position because one of the hardest things to do in life is to say “No.” To invitations, to requests, to obligations, to gifts, and to the stuff that everyone else is doing. Saying yes is so easy…and it feels so good.

Even harder is saying no to less obvious impositions: getting caught up in the status of the job, normalizing yourself at a certain level, the drama, the rush. Why are so many bands from the 70s and 80s still on the road? It’s not only the money, it’s that they need the adulation of the crowd. They can’t go back to regular life. Neither can most of us once we have tasted the forbidden fruits of power or fame or being needed.

Freedom is the most important thing. We’re born with it, and yet many of us wake up one day surprised at the chains we wear. The reason? Because we said yes too many times and never learned how to say no.

Only a free person can decline. Preserving this power is essential.

It’s the difference between a life of subservience and a life of your own, as Lady Bird Johnson, LBJ’s wife knew and often struggled with herself. As Robert Caro wrote, she came to visit John Hicks after he had politely refused her husband’s offer, to let him know she respected, even admired his decision. Because she “had seen other people take their ten thousand dollars and had seen what happened to them.” But Hicks had escaped, as Socrates had escaped, as the brilliant photographer Bill Cunningham escaped and basically all the people who have done truly great work have escaped.

Because if you can’t say no, you’re not powerful or free. You’re a slave.

Like to Read? I’ve created a list of 15 books you’ve never heard of that will alter your worldview and help you excel at your career. Get the secret book list here!