Thinking about a larger family? Think again. Or, as Pope Francis said, don’t “breed like rabbits.”

Speaking about birth control and population on a flight from Manila to Rome, the increasingly freewheeling pope told reporters, “Some think, excuse me if I use the word, that in order to be good Catholics, we have to be like rabbits — but no,” Reuters reports. The pope said the church promoted responsible parenthood along with its controversial ban on contraception, and cited a woman who had seven children by caesarean section who was putting her life at risk by having another child. “That was an irresponsibility,” he reportedly said.

But many Americans do wish they had more children. Michelle Patterson, 43, had two children 17 months apart. “We ended up having kids right away,” she says. “The plan was to wait a while. That wasn’t in the cards.” At the time of her second pregnancy, she was vice president at a major recruitment firm. She feared that having another child so soon would impact her career and even the company. “I was petrified of telling the president of my company that I was pregnant again,” she says. “I asked myself, ‘How can I be vice president of this company and do this?’”

Patterson agreed on a work plan that meant she was involved in the day-to-day operations during her maternity leave. “I worked throughout the maternity leave and it was very challenging,” she says. “I did not end up jeopardizing my position.”

But she chose to have two children and feels fortunate that she had the economic freedom to do so. “Raising kids is expensive,” she says, adding that the decision on family size is often related to finances. That said, she and her husband had fewer than the previous generation: Patterson’s own parents had three children and her husband’s parents had four.

Many Americans are indeed choosing to have fewer children than their parents did. Despite this, however, recent research finds that many still long to have bigger families. Women in the E.U. have an average of 1.9 children (versus their “ideal” rate of 2.3 children) and women in the U.S. have 2.1 children (versus their ideal rate of 2.4 children), according to new analysis of government data from the U.S. and E.U. by Pew Research Center. “For women, fertility is not one choice, but a lifetime of choices,” says Gretchen Livingston, a senior researcher at Pew Research Center.

Pope Francis draws six million for final Mass in Manila

More surprisingly perhaps, around 40% of U.S. women nearing the end of their childbearing years say they have fewer children than their ideal, according to the General Social Survey carried out by NORC at the University of Chicago, a non-partisan, independent research organization. Among women in the E.U. aged 40 to 54, one-third reported that the number of children they have given birth to is lower than their personal ideal, according to data analyzed by Pew. That gap between the actual and ideal number of kids varies dramatically across Europe: In Denmark, the gap is 45%, but only 18% in Bulgaria.

“More women are also delaying having a family to focus on their career and education and, as time goes on, the window of fertility gets smaller and smaller,” Livingston says. Plus, some women who want to have kids lack a suitable partner, she adds. The birth rate for women aged 15 to 44 years in the U.S. has fallen to 63 births per 1,000 women — a record low — from 71.2 in 1990 and 118 births per 1,000 women in 1960, according to provisional data from the Center for Disease Control. (Foreign-born women in the U.S. have a fertility rate around 13% higher than U.S.-born women.)

Some couples are divided over whether to have another child due to the cost, says Fran Walfish, a psychotherapist in Beverly Hills, Calif. “Many families require two paychecks to make ends meet,” she says, and must postpone buying a house — and starting a family — while they save for a down payment and negotiate with banks’ strict lending requirements post-recession. Middle-income parents will need to spend nearly $300,000 over the first 17 years of their child’s life and $490,000 for high earners, according to the U.S. Department of Agriculture’s Center for Nutrition Policy and Promotion.

Childcare is the biggest household expense in most regions, says Lynette Fraga, executive director of Child Care Aware of America, a non-profit group in Arlington, Va. that works with state and local agencies. The average cost for full-time care for an infant ranges from $4,863 a year in Mississippi to $16,430 in Massachusetts, according to its annual report. “Childcare is a significant consideration on the family budget,” she says. What’s more, the U.S. is one of the few industrialized nations that doesn’t require paid family leave for new parents. Read: “6 of the best countries for new moms.”

Millennials — born between 1983 and 1993 — are also getting married later in life, a separate Pew study found. Just one-quarter of that generation is married. When they were in their early 20s, 36% of Generation X, 48% of baby boomers and 65% of the Silent Generation — born between the Great Depression and World War II — were hitched. While there’s less societal pressure to get married, especially given the high divorce rate, 69% of unmarried millennials say they would like to, but many lack what they deem to be a necessary prerequisite, it found, “a solid economic foundation.”

Raising children can be challenging when juggling a career and running a household, Patterson says, but she’s happy with her decision to have two kids.

Last year, she declined an invitation to speak at the “Power of Collaboration: Women and the Future of Global Leadership” conference at the United Nations because it clashed with her daughter’s 16th birthday. “She’s only going to have a 16th birthday once,” she says. “But they said, ‘Bring her along.’’

She went, thanked them in her speech and everyone sang “Happy Birthday.” More importantly, she says, “I didn’t have to opt out.”

This story was updated on Jan. 20, 2015.