As Doug Andreassen, the chairman of US Soccer’s diversity task force, looks across the game he loves, all he can see is a system broken in America. And he wonders why nobody seems to care.

He sees well-to-do families spending thousands of dollars a year on soccer clubs that propel their children to the sport’s highest levels, while thousands of gifted athletes in mostly African American and Latino neighborhoods get left behind. He worries about this inequity. Soccer is the world’s great democratic game, whose best stars have come from the world’s slums, ghettos and favelas. And yet in the US the path to the top is often determined by how many zeroes a parent can write in their checkbook.

Andreassen watches his federation’s national teams play, and wishes they had more diversity. Like many, he can’t ignore the fact that last year’s Women’s World Cup winners were almost all white, or that several of the non-white players on the US Copa America roster grew up overseas. The talents of some of America’s best young players are being suffocated by a process that never lets them be seen. He sighs.

“People don’t want to talk about it,” he says.

Andreassen used to dance gingerly around the topic, using the same careful code words as the other coaches and heads of leagues, trying not to push or offend only to find that little changed. He has stopped being political. He is frustrated. He is passionate. He is blunt.

“The system is not working for the underserved community,” he says. “It’s working for the white kids.”

But why? How come soccer can’t be more like basketball in America? How come athletes from the country’s huge urban areas aren’t embracing a sport that requires nothing but a ball to play? How have our national soccer teams not found a way to exploit what should be a huge pool of talent?

“We used to say to ourselves: ‘How good would we be if we could just get the kids in the cities,’” one former US official says.

And yet a quarter of a century into soccer’s American boom, that hasn’t happened. Coaches, organizers and advocates say interest is there, especially among immigrants from Mexico and Central and South America, where devotion to soccer runs generations in families. But finding those kids is hard. Money has only hardened the divide between rich and poor, leaving the game to thrive in wealthy communities, where the cost of organized soccer has become outrageous, pricing out those in lower income neighborhoods.

“I don’t think it’s systematic racism,” says Nick Lusson, the director of NorCal Premier Soccer Foundation an organization to grow soccer in California’s underserved communities. “It’s just a system that has been built with blinders to equality.”

Three years ago, Roger Bennett of Men in Blazers and Greg Kaplan, a University of Chicago economics professor, set out to study the effects of the pay-to-play system on American soccer. They compared the background of each US men’s national team member from 1993 to 2013 to that of every NBA all star and NFL pro bowler over the same period, using socio-economic data from their hometown zip codes. They found the soccer players came from communities that had higher incomes, education and employment rankings, and were whiter than the US average, while the basketball and football players came from places that ranked lower than average on those same indicators.

Those numbers have tightened since 2008, reflecting more recent diversity in soccer, but the gap remains.

“I’m – to be honest – surprised that the data is so striking,” Kaplan says.

Briana Scurry (back row, far right) is among a small number of non-white players who have made their name for the US women’s national team. Photograph: David Madison/Getty Images

There are a lot of reasons for this disparity, but mostly everything comes down to perception and economics. “It continues to be seen as a white, suburban sport” says Briana Scurry, who won the Women’s World Cup in 1999 with the US, and was arguably the country’s most prominent black female player.

As a child growing up in Minneapolis, Scurry loved basketball. She assumes it would have been the game she chose to play in college. That changed when her family moved to the suburbs when she was in grade school, and a teacher handed out flyers for a local soccer league. There had been no such activities in her former home. The idea city kids would play suburban soccer was too ludicrous to consider.

People do want to change the pay-to-play system. US Soccer Federation president Sunil Gulati says he believes the organization “has made a lot of strides” before adding: “We have a long ways to go.” Andreassen believes Gulati is committed to the issue, as are a handful of others. But in a world where soccer coaches in wealthy communities can earn decent livings, there are few, like Lusson, who gave up what he calls a “cushy job” running a league in the pricey East Bay suburbs of San Francisco to direct a program in the underserved community of Hayward.

“You are buying skill,” Scurry says of those parents who can spend on their children’s soccer. “But there are some pieces of the game that just can’t be bought.”

Sometimes on weekends Julio Borge, who is director of coaching at the mostly Latino Heritage Soccer Club in Pleasant Hill, California, will spend the day watching games in the ligas Latinas around San Francisco’s East Bay. Most of the area’s English-speaking population knows nothing about these games that are played at local parks in neighborhoods filled with immigrants from Central and South America. But to the families who gather on the fields to barbecue, listen to music and watch soccer, they are the highlight of the week.

“The soccer is amazing quality,” says Borge, who will sometimes try to recruit players from the ligas Latinas for his teams. But with a cost of $1,395 a year, which he uses to cover coaches, fields, insurance and officials, he knows most won’t afford to play on his team. He can occasionally offer a scholarship to a player if someone donates the money to do so, but there is never enough for the children whose families hover above the poverty level.

“In my area, we are missing a ton of these kids,” he says. “A lot of coaches don’t have time to see everybody. It’s expensive to try out for the big programs, so many don’t even go after the opportunity.”

The cost of youth soccer these days is outrageous. Borge’s $1,395 team is a bargain compared to many travel programs where the base fee is $3,000 a year. “How can you charge that for just a year?” Scurry asks. “That’s ridiculous.” And yet two of her friends are paying more than that for their kids in the Virginia suburbs of Washington DC. Across the Potomac River, in Maryland, parents can pay up to $12,000 a year on soccer after adding the cost of travel to out-of-state tournaments.

Most of these teams offer scholarships to kids with financial needs. The gift is kind and usually well-meaning, but they come with their own problems. Usually they go to the very best players in the poor neighborhoods, and they come with the implication that the scholarship kid is expected to help their new team win. They also displace children in the lineup whose parents are paying full price, and wonder why they are spending thousands a year to have their child sit on the bench.

Scholarships often cover the cost of the league but little else. They don’t provide transportation for the player whose parents might work during practice, or don’t have enough money for gas to drive to games. Some families don’t have email and can’t get the club announcements. Resentment builds.

“The parents will say to scholarship kids – and I have seen this countless times – ‘Why did you miss the game on Saturday? We are paying for you to be here,’” Lusson says. “What does a kid say to that? Or what happens if they are late to practice? Or who is going to pay for them to travel to that tournament in San Diego? That’s like the moon to some of these kids. We have kids here in the East Bay who have never seen the beach.

“They drop out of the program and then what happens?” he continues. “They have burned the bridge back to their old club by leaving, and aren’t welcome back. They drop out and drift away and we lose them and that’s terrible because they were really, really talented.”

Economics work against the poor kids in American soccer. Lusson sees this every week as he moves between the teenage girls team he coaches in the wealthy San Francisco enclave of Pacific Heights, and the teams he manages in lower-income Hayward. One night, a few weeks ago, he listened as girls on the Pacific Heights team talked excitedly about applications to elite east coast colleges. The next day, in Hayward, nobody talked about college.

And yet he is amazed by the skill of his Hayward players, who he says would crush the Pacific Heights team in a match. These are the players who could be the future of American soccer, perhaps even rising as high as a national team. But he also knows that the Pacific Heights players will be the ones to play on their college teams and will be identified by US Soccer. They are the ones who will get a chance that the Hayward kids won’t. And this strikes Lusson as very wrong.

The other day, he went to watch a match between a team made up of mostly upper-income San Francisco-area college players against a group of players from poorer neighborhoods in the East Bay and Fresno three hours away. For a few minutes, the college players controlled the game until their untrained opponents deciphered their system, and then ripped it apart by half-time. In the second half, the East Bay-Fresno team trampled the San Francisco team.

“I’ve been seeing that game my whole playing career,” Lusson says. For a time, just out of college, he played professionally in Brazil. It feels rigid in the organized levels here, drained of any life. The difference is as simple as Brazilian children dribbling balls on the streets as they walk to school while American kids lug balls in bags, only pulling them out once they reach the field for a practice or game.

“We are delivering a lot of people who do soccer but not play soccer,” he continues. “I think sometimes we have to be courageous with ourselves and admit when we have a problem.”

One of the biggest complaints about the pay-to-play system is that the over-coaching in suburban programs strips kids of the creativity that comes from playing on the streets. Borge loves his recruiting trips to the underground leagues where kids are free from restrictions. Too often, when a skilled and imaginative player leaves their neighborhood team to join a bigger team in a wealthier community, their gifts are considered a hindrance. The clever dancing with the ball that makes them unique earns them the label of not being a team player. A message is delivered: conform or leave.

Soccer is an inner city sport in most of the world but seen as a suburban preserve in the US. Photograph: The Washington Post/Getty Images

Those who deal in diversity in American soccer know this is a problem. They talk about it all the time. “We are making these little robots,” Lusson says. No one seems sure what to do. How do you tell players to be imaginative while at the same time fitting into the more rigid needs laid out by American coaches? No one knows.

“I don’t know how to structure unstructured,” Gulati says with a chuckle. “I don’t know how else to say it.”

But coaches like Borge are looking for the answer to come from US Soccer. Borge’s voice rises as he begins to talk about the fractured network of leagues around the country, with coaches imposing their own unyielding vision of how soccer should be played. He wants someone in the federation to devise a strategy, not just for a style of play, but for getting the best players to the top.

“We need someone to speak up – [Jürgen] Klinsmann or whatever – and say: ‘Here are the players we are looking for, and here’s how the system should be built,’” Borge says. “We have coaches who need to recruit talent from all over the nation and invest in the ODP and let them be creative and let the let them play freely. It takes huge dedication and it takes money but we have the money at US Soccer.”

“You want to know about diversity in soccer?” Andreassen asks. “Here’s a story about diversity in soccer.”

A few years ago, when Andreassen was head of Washington Youth Soccer, a 15-year-old girl from a lower-income community south of Seattle showed up in the town’s league. Her family was from Mexico, and she had grown up with several older brothers who loved soccer and had developed tremendous skill playing with them. The league organizers wanted her to go to an Olympic Development Program tournament in Arizona that is scouted by college coaches, and because her family had little money, a sponsor was found to pay for the trip.

The girl played brilliantly at the tournament. She was surrounded by college coaches who flooded her with offers, which is unheard of for a sophomore in high school playing at an ODP tournament. But when she returned home her father banned her from playing soccer. He was undocumented, and he feared that if she became a college player, someone would notify the government and he would be deported.

They never saw her again.

The story stayed with Andreassen. It was yet another window into a gap that had been bothering him for years. How many cultural chasms between his white world and the invisible one – to him – around it existed? He wondered what would have happened if she had gotten hurt at the ODP tournament. One of his other great passions is the prevention of head trauma in soccer. What if she had gotten a concussion? Who would have taken care of her? Her family probably didn’t have health insurance. Who would handle the hospital bills? Things that are taken for granted in middle- and upper-class neighborhoods don’t exist in the poor communities, opening the divide even more.

When he took over as head of US Soccer’s diversity committee, he asked these questions on the quarterly conference calls he had with the other members. He pulled together lists of the things that didn’t work and tried to find solutions.

“There are the kids in the diverse communities playing on the street corner, and we have to find them,” he says. “Someone knows they are there, either the church or the school. Maybe it’s a pastor or a principal or someone at the YMCA or Boy’s and Girl’s Club. We have to identify those community leaders.”

Andreassen really likes the Urban Soccer Leadership Academy, a program in San Antonio run by former mayor Ed Garza. Garza is using soccer in building teams similar to those in the suburbs – but at a minimal cost. Garza chose soccer as the foundation for his program because sports like football and basketball already had a strong infrastructure in San Antonio and there was no organization that held together the city’s immigrant families. He saw so much skill going to waste.

“Our city teams were beating the suburban teams,” Garza says. “What if our inner city had a program to develop soccer players? Can you imagine how much better they would become, on a skill level, and get the attention of pro and college scouts?”

The program, which is designed to help send as many kids to college as possible, has recently expanded from largely Latino neighborhoods to include African American communities as well. But what impresses Andreassen the most is it gives hope to the kids who would otherwise be lost in the pay-to-play system the structure of a suburban team without losing their identity or style of play.

A couple of boys compete in a pickup game at Flushing Meadows in Queens, New York. Photograph: Al Bello/Getty Images

He and his task force have written a proposal that has been delivered to US Soccer. He wants to create a national leadership academy to give leaders in underserved neighborhoods the power to build their own San Antonios. The academy would be based in a central location, with the hope to eventually build more around the country and it would exist to show those local principals and pastors and coaches how to make a league that operates like the suburban associations. They would be taught about fundraising and tax filings and field rental.

But what they would get most is admission to the same world as the wealthy suburban clubs. Once underground leagues now would be on the map, in view of college coaches and federation and professional scouts. The kids playing on the street corner would have a greater possibility of being found. They would get a chance.

Gulati has read Andreassen’s proposal but he sees a lot of proposals and a lot of ideas. The pay-to-play model isn’t unique to the US, he says. It exists in more traditional soccer countries. What seems to discourage him is the extreme cost of the American system, as well as the fact that the federation has not been able to identify as many players in inner-city neighborhoods as he would like.

“It’s an important priority,” he says. “Look, we are in some ways still a developing foundation.”

In 10 or 15 years, Gulati hopes the US’s top teams will be more diverse, and that many of the walls that divide the rich and poor in soccer will have come down. He sees great promise in the growth of the MLS where teams like the Philadelphia Union are not only investing in youth development academies but building their own academy facilities as well.

He is certain that the more soccer blossoms in the US, and the professional leagues establish themselves, kids who feel left out in today’s model will see a path for themselves in the game.

Like everyone else, Gulati notices the racial makeup of the women’s national team, but he says the federation is working hard to build the game at the grassroots level. Someday he would love to see the sport grow to such a robust level in underserved neighborhoods that there will be a deep pool of talent that everyone knows about, and that some of those players will rise to the national team.

“Everybody wants to do better,” Gulati says about diversity.

There are signs of improvement. More coaches like Lusson are leaving wealthy suburban programs to help run teams in poorer areas. More programs like Garza’s in San Antonio are starting to grow. And funds are trickling into the neighborhoods that need them.

The US Soccer Foundation (unrelated to the federation) has built several futsal courts in inner cities in an effort to recreate the free play of so many Central and South American children. It has also served 71,000 mostly African American and Latino kids in its Soccer For Success program that provides coaching and mentoring for children in lower-income neighborhoods. The program operates between the 3pm and 6pm, when school ends and parents are less likely to be home.

“Everybody understands the issue,” says the foundation’s president and CEO Ed Foster-Simeon. “Talk about a family living on $25,000 with four kids in a place like Washington DC or even double that, $40,000 in Washington DC. Those kids shouldn’t be barred [from soccer] because their parents don’t make much money.”

And yet how fast will change come? Yes, more than a third of the men’s roster in this month’s Copa América are non-white. But a good number of those learned their soccer overseas, because Klinsmann appears to feel the creativity is still lacking in American soccer. The women’s team going to the Rio Olympics, meanwhile, will almost certainly have a similar racial makeup as the one that played in last summer’s World Cup.

“I don’t see why our team has fewer [black players] than England or France,” Scurry says. “The systems are different here and the people who have the access are the people who have the money.”

Families are still paying thousands of dollars for soccer teams that travel to faraway tournaments, which are often the only places college coaches go to watch players because their recruiting budgets are small. The poorest kids are still shut out. The game is not coming to them in flyers at school the way it did for Scurry in suburban Minneapolis so many years ago. Lusson says non-white parents driving their scholarship kids into wealthy neighborhoods are still getting pulled over by the police who wonder why they are there.

Even MLS teams are motivated by money in opening their academies. Speaking privately, one executive says his team only takes players they believe they can monetize, either by developing for the team itself or selling that player to another club. This limits their reach because they rarely take chances on a borderline prospect, leaving a soup of expensive youth travel and elite soccer clubs to find and train that player. More than likely the system will not allow that to happen.

“I’ve been doing this since 2002, and I don’t see anything that says this will change,” Borge says.

On the phone, Andreassen, who can talk long and fast, is suddenly quiet. He is sure Gulati shares his passion, but he wonders if many others do. “I think I’ve pushed the ball one revolution,” he says. “But my goal is to get it down to the goalline.”

Sometimes he wonders if anyone notices. Is anybody listening? Does anybody care?