Recently, there has been a lot of talk about how America is on a slow slide to oblivion given our lowest-in-history birthrate and U.S. Census projections showing slower population growth. While it’s true that fewer babies were born last year than in previous years (due to the recession, most likely), our total fertility rate – the number of babies born to the average woman over her lifetime – has barely budged. Most American women want about two kids. That’s been true for decades, and it likely will remain true for decades to come.

But what if the professional fertility doom-mongers are right? What if the United States’ population stops growing and instead remains level? I’d like to shuck common wisdom and offer five reasons why a higher birth rate isn’t a panacea for us or the rest of the planet.

1. Growth Isn’t Enough. Just having a growing population does not ensure prosperity – consider Afghanistan. Afghanistan’s fertility rate is far above ours – the average woman has six children. The nation’s population could rise from 26 million in 2008 to more than 76 million by 2050. Is that a recipe for success? Doubtful. If a nation can’t provide its children a good education, healthy food, a safe place to live and medical care, it’s unlikely that the cycle of poverty can be broken. Right now in the United States, we have nearly 15 million children living in poverty. We have kids who can’t read and go to bed hungry every night. What would happen if instead of worrying about having more babies, we took better care of the children who are already here? I’d wager that would boost our economy more than increasing the birthrate.

2. Growth Isn’t Free. One thing that the “pro-growthers” never consider is that population growth is not free. If American women all of a sudden began having four kids apiece instead of two, we’d need more hospitals. More schools. More roads. More bridges. And we can’t even be bothered to invest adequately in the crumbling ones we’ve already got. While it’s true that those additional children would most likely grow up to be consumers (which seems to be the main goal of the pro-growthers), reaching that goal is not without considerable expense. You have to invest money to make money.

3. Growth Comes At a (Green) Cost. There’s no way around it: You and I are hard on the environment. Some estimates show that if everyone around the world lived like Americans do, we’d need five earths to sustain us. Americans are a world leader in carbon emissions, and if everyone shared our diet, the earth could only support 2.5 billion of us. While it’s true that Americans also contribute a lot to the world, do we really think that we’d stop inventing and producing things just because our growth rate had leveled off? With the world population projected to hit at least 9 billion by 2050, there will be plenty of potential customers both here and abroad without nagging American women to give birth to more of them.

4. It’s Choice, Stupid. One common denominator seems to unite the people wringing their hands over America’s birthrate: They’re usually men. It’s easy to demand that someone else put their life and health on the line creating, gestating and giving birth to another human being when you’ll never be asked to do the same. If women want to have babies, great! More power to them. But if they don’t, it’s their choice. Women are people, too, and they deserve to make their own decisions. Women are not vessels for the creation of consumers to boost your company’s bottom line.

5. Population Growth Doesn’t Equal Happiness or Quality of Life. The Himalayan nation of Bhutan has a unique way of measuring success – in gross national happiness rather than gross domestic product. While I’m not suggesting that the United States chuck economists in favor of comedians, Bhutan might be onto something. Is population growth really necessary for prosperity? And does prosperity really equal happiness? For many people, having choices – especially whether or not to have children and how to raise them – is a much larger determinant of life satisfaction. But life satisfaction is not nearly as easy to measure as is the GDP.

While I’m an optimist, I’m also a realist. I understand that the United States faces challenges as our population grows older. That’s why my organization, Population Connection, will be hosting a symposium this June featuring David Bloom of the Harvard School of Public Health and a number of other distinguished demographers, economists and experts on population dynamics.

We hope this symposium will help spark a nationwide discussion. What kind of a nation do we want? And how can we get there from here? Do we need to buckle down and make more American babies? Or is sustainability our best path forward? The future will be here before we know it. Let’s work together to make it a great one.

John Seager is President of Population Connection, the nation’s largest grassroots population organization. The organization’s website is populationconnection.org.