After several years of interviewing, observing, and living among these fathers, I’ve learned that not caring about their children is not the problem. Our 2013 book, Doing the Best I Can: Fatherhood in the Inner City , reveals that these men desperately want to be good fathers, and they are often quite intensively involved in the early years of their children’s lives. Yet they usually fail to stay closely connected as their kids grow older.

Finally, fellow researcher Tim Nelson and I began actually talking to these men—more than 100 low-income noncustodial dads living in poor neighborhoods in the Philadelphia area. As it turns out, “everyone” wasn’t right. We were all dead wrong—me, the country, and even Barack Obama.

I used to brush the question aside. After all, I had spent years living and talking with black, white, and Hispanic single mothers in some of the nation’s toughest urban neighborhoods in Philadelphia, Chicago, the deep South, and the West Coast—10 cities in all. I thought I had learned everything there was to know about these men from the moms. Besides, didn’t everyone know the guys were irresponsible? That they really didn’t care about the kids they conceived? In 2008, even presidential candidate Barack Obama was calling them out, saying they had better stop acting like boys and have the courage to raise a child not just create one.

As the author of two books about low-income single mothers, I often give talks or appear on call-in shows. Audiences always want to know about the men single mothers have children with. They ask me, “Why don’t you talk to the dads? What about the fathers?”

Today, 27% of fathers do not live with their children ­­ for low ­income fathers, that number increases to 40%. A Woman's Nation Pushes Back From The Brink

If lack of caring isn’t the problem, then what is? To answer that question, we have to start with how their relationships form.

Romance in the inner city typically proceeds quickly. Just six or seven months after they first begin “kicking it,” most of these couples “come up pregnant.” Usually neither he nor she explicitly plans to have a baby, but neither of them does much to avoid pregnancy, at least not for long. Inner-city youth often view condoms as a method of disease prevention, not contraception. They believe that ongoing condom use says you don’t trust your partner to be faithful, so as soon as there is a kernel of trust, the condom stays in the drawer—a ritual marking the transition to a more serious relationship.

Pretty soon, the women are skipping doses of the pill or letting the patch or other forms of contraception lapse. Why? In these communities, motherhood often exerts a strong pull on young women’s hearts and minds and weakens their motivation to avoid pregnancy. Being a mom serves as the chief source of meaning and identity in neighborhoods where significant upward mobility is rare. She realizes that her circumstances aren’t ideal, so she doesn’t explicitly “plan” to get pregnant. But she’ll readily admit that it wasn’t exactly an accident either. She’ll say she knew full well where unprotected sex would lead.

For their part, the men typically say they “just weren’t thinking” about the possibility of pregnancy when conception occurs. Yet contrary to the hit-and-run stereotype of the deadbeat dad, 7 times out of 10, men’s reaction to the news of a pregnancy is happiness—even downright joy. In fact, we found they are more likely to be happy than the mothers are! Andre Green, still in high school, told us he shouted “Thank you, Jesus!” when he heard the news, even though he and the would-be mother were no longer together.

What accounts for this strong, latent desire for kids among young people who can ill-afford to support them? Here, context is key. Andre Green and his peers are coming of age in some of the most violent and poverty-stricken neighborhoods in America. Their lives are marked by trauma. Just months before Andre learned that he was about to become a dad, his brother was murdered, and his mother turned to drugs as a salve. Like Andre, many men we spoke with described their lives up to that moment with a single word: “Negativity.”

In this context, a baby—fresh and innocent—is pure potential, a chance to move away from the mistakes of the past and turn to activities that are wholly good. Celebrating those precious first words and first steps. Spending the night soothing a fussy teether. Carefully fixing a little girl’s hair. For middle-class teens coming of age on Philadelphia’s affluent Main Line, early pregnancy ruins lives—a bright future snuffed out, or at least diminished. But if you’re already at the bottom, a baby means something else entirely.

As I’ve said, poor women find meaning in motherhood when sources of meaning are in short supply. But what we often fail to appreciate is how large the rewards of fatherhood also can be for men in extraordinarily challenging circumstances. Seven White, who conceived his first child at 17, told us, “I couldn’t imagine being without them, because when I am spending time with my kids it is like, now that is love! That is unconditional love. … It is like a drug that you got to have. I would never want to be without them.”

These young people know that the right time to have a child is when you are economically ready, but many are afraid that the right time may never arrive. And they’re right. Because of deindustrialization, automation, and outsourcing, there are precious few well-paying or even steady jobs for those without skills these days. Byron Jones, an inner-city dad who is now in his mid-30s, told us that he advised younger men in his neighborhood to hold off on having kids until “you are financially able to take care of the children.” Then he paused and added, “And that’s when nowadays? I have no idea, because—when is it? I mean, shoot, for the average guy, stable employment don’t last long. You might work this week and be out the next week, you know?” So, with little confidence that the right moment to have a child will ever arrive, they allow fate to dictate the timing.

In this corner of America, pregnancy is often the impetus for a relationship, not the outgrowth of one. He and she usually become a “couple” only after a baby is on the way. Shotgun relationships have replaced the shotgun marriage. Yet as the time bomb of pregnancy ticks, men rarely flee. Instead, they try mightily to “get it together for the baby”—the “it” being the relationship with the woman who is about to become their child’s mother. In fact, when the baby enters the world, more than 8 in 10 men are still together with the mother. Yet due to their laissez-faire route to conception, they may not really know their kid’s mom very well when the child is born.

Those first, very tough months of being new parents put these fragile relationships under tremendous strain, made worse by a lack of money. With hardly any shared history to draw on, is it any wonder that half of these couples break up before their child’s first birthday? Even the relationships of middle-class married couples are often tested when a baby comes into the picture. Usually though, they can draw on the trust generated by the years they’ve already shared when those hard times hit.

Some readers might wonder, “Why don’t they get married?” The young couples we interviewed certainly aspire to marriage. In fact, they revere it. But they strongly reject the idea that a hasty wedding is a good idea. Isn’t it better, they reason, to wait until they can get their finances in order and be sure the relationship is strong? Why get married if you’re just going to get a divorce? For them, this would merely make a mockery of a sacred institution. For reasons I’ve outlined above, most of these relationships soon fail their own test.

After the breakup, inner-city dads firmly believe that a shattered couple bond should not get in the way of a father’s relationship with his most precious resource: his kid. They’re not just out to claim status with their peers by getting women pregnant. They long to engage in the father role. But the young men we spoke to have tried to redefine fatherhood to fit their circumstances.

All fathers across America, rich and poor alike, have avidly embraced fatherhood’s softer side. Imparting love, maintaining a clear channel of communication, and spending quality time together are seen as the keys to being a good dad. This “new father” model, which spurred middle-class men to begin changing diapers several decades ago, has gained amazing traction with disadvantaged dads in the inner city, perhaps because it’s the kind of fatherhood they can most easily afford. But while middle-class men now combine these new tasks with being breadwinners, low-income fathers who face growing economic adversity are trying to substitute one role for the other.

Here is the problem: Neither society nor their children’s mothers are willing to go along with this trade-off. Love and affection are all fine and good, but who’s going to pay the light bill? What about keeping the heat on? If a child’s father can’t provide money, the attitude goes that he’s more trouble than he’s worth. Why strive to make sure he stays involved with the kids?

But we’re wrong about that too. From the kid’s point of view, it is hard to make up for the loss of a parent. When a single mom in the inner city feels her kid’s father has failed to provide, there is an enormous temptation to “swap daddies,” pushing the child’s dad aside while allowing a new man—perhaps one with a little more going for him economically—to claim the title of father. These moms are often desperate to find a man who can help with the bills so they can keep a roof over their kid’s head. The problem is that these new relationships may be no more stable than the old ones.

When a mom moves from one relationship to another—playing gatekeeper with the biological father while putting her new boyfriend into the dad’s role—she puts her kids on a “father-go-round.” In the end, will any of these men have the long-term commitment it takes to put these kids through college?

Meanwhile, the biological fathers themselves end up on a family-go-round, having kids by other women in a quest to try to get what they long for—the whole father experience. Each new child with a different mom offers another chance—a clean slate. With eagerness, they once again invest every resource they can muster in service of that new fragile family. But while succeeding with a new child, they often leave others behind. So, while they are good dads to some of their children, they end up being bad dads to others.

As is so often the case, these men often have good intentions. They want to heal their own fatherless childhoods by embracing the father’s role. But good intentions aren’t enough. All of their children—not just some—need them.

If they want to stop the father-go-round, moms will have to do what they can to keep the biological dad involved with his child and not push him aside. It’s up to them, because currently, mothers have most of the power de facto, if not de jure. Almost always, an unmarried mother has presumed custody, even if the dad willingly signs documents to establish legal paternity. When an unmarried dad gets a child support order, there is no corresponding automatic process that grants visitation. Therefore, there is seldom any guarantee that the father will even be able to see his child, regardless of whether he pays child support.

So here is my message to the single moms I’ve come to know and admire over the years, the ones who do so much of the nurturing and caring—those who often struggle mightily just to keep a roof over their kids’ heads: You can’t easily substitute someone else for your kid’s dad. If the two of you don’t manage to stay together, find a way to keep him involved with his kids.

Here is my message to young disadvantaged men: If you really want to be a good dad, wait until you are financially ready to have a child, preferably your mid-to-late 20s, if not beyond. Make sure your relationship with the child’s mother is on a solid footing first.

But the dads aren’t going to listen to my advice unless the rest of America hears what I have to say too: You’ve got to give these men hope, which would mean a real shot at a stable future. Stop locking them up for nonviolent offenses. Make sure they have decent schools to attend, not the broken disgraces that litter our inner cities. Promote programs that instruct them on how to navigate the rocky shoals of their relationships. Increase the supply of decent jobs and, for those who can’t find work, provide jobs of last resort. Make good on the promise upon which America was founded: that here, if you are willing to work hard, you can make it, no matter who you are.

But the most important thing I need you to do—all of you, even Barack Obama—is to change your attitude about these fathers. They care deeply about their kids. They are not lacking the will. Let’s join together to help them find the way.

All names in this essay have been changed in the interest of confidentiality.

This is an excerpt from The Shriver Report: A Woman’s Nation Pushes Back from the Brink, in partnership with the Center for American Progress. Download the full report here for FREE from January 12th – January 15th.