At a gathering of more than 100 college leaders and thinkers, U.S. Secretary of Education Arne Duncan says college costs are too high, graduation rates are too low and there is too little accountability

For a room full of academics talking about the future of higher education, the conversation was surprisingly blunt. Yesterday TIME gathered more than 100 college presidents and other experts from across the U.S. to talk about the biggest problems facing higher education, which U.S. Secretary of Education Arne Duncan summed up for the room as “high prices, low completion rates, and too little accountability.”

The day-long summit in New York City, co-sponsored by the Carnegie Corporation of New York and the Bill & Melinda Gates Foundation, brought together the leaders of some of the country’s biggest colleges and universities (the presidents of Miami-Dade College and Arizona State University) and some of the most expensive (the president of New York University and the former presidents of Harvard and Princeton). There were current and former government officials in attendance as well as philanthropists and CEOs, and all were concerned about the rising cost of college and what can be done to curb it without sacrificing quality. There was also a lot of frank discussion about access and how to ensure that more Americans not only have the opportunity to go to college, but have the tools to succeed so they can make it to graduation day. “College completion is a good goal,” said Nancy Zimpher, chancellor of the State University of New York. But, she added, “no one has a collective strategy to get there.”

(MORE: College Is Dead. Long Live College!)

Duncan said the the government needs to increase accountability in higher education, but noted that policymakers have to be careful not to create incentives that make schools run away from high-risk students. “There are no silver bullets,” he said. That was perhaps the one thing everyone at the summit could agree on. As the attendees work to develop policy recommendations in the coming months, here are three big takeaways from the summit:

1. Competition leads to higher costs

In most industries, competition leads to lower prices. But the opposite is true in higher ed, where, as Michael Crow of Arizona State put it, “quality and status is often connected to exclusivity.” Too many schools are trying to compete with Harvard—state-of-the-art gyms, anyone?—and maintain the highest-profile faculty, and those costs trickle down to the students. “We’ve got to stop this arms race,” said Thomas Kean, former governor of New Jersey and president emeritus of Drew University.

Some of the panelists said that colleges need to be run more like businesses, offering degrees based on labor-market demands and focusing on efficiency. “How can we graduate twice as many students for the same cost?” asked Andre Dua, a senior partner at McKinsey & Company. But others worried that all the emphasis on affordability would ultimately result in reduced quality. “If you only talk about cost, you turn higher ed into a commodity,” said Scott Cowen, president of Tulane University. “We can’t think of college as a business. It’s something different.

(MORE: Reinventing College)

2. Innovate or die

No matter the discussion topic, the conversation throughout the day always found its way back to technology. The afternoon session devoted to tech and globalization opened with a presentation by Andrew Ng, an associate professor at Stanford and co-founder of Coursera, the largest provider of massive open online courses, or MOOCs. Some higher-education experts like Dua see Ng’s company, which offers free online courses taught by professors at prestigious universities, as a radical disruption to the traditional college model of having a relatively small number of students spend a lot of money to go to school in one place for a few years.

But the presidents of Rensselaer Polytechnic Institute and other prestigious schools did not view MOOCs as a threat to college campuses. “This is terrific,” Bard’s Leon Botstein said of Ng. “What he’s done is put bad teaching out of business.” Online lecture videos will free up students to spend class time talking with their professors, Botstein said, but technology will never replace the classroom.

Even Ng said his company wouldn’t be competing with top schools like the one where he works. For many students around the world, he said, the choice is not between a MOOC and Stanford—it’s between a MOOC and nothing. But the panel’s moderator, TIME editor at large Fareed Zakaria, warned college leaders to take Coursera and other MOOC providers seriously. “Andrew says he doesn’t want to disrupt you,” he said. “Neither did Alexander Graham Bell.”

(MORE: 8 Ideas to Improve Higher Education)

3. To improve college education, build better K-12 schools

For a summit devoted to higher ed, surprisingly much of the discussions involved the schools years that precede college. Many participants encouraged colleges to take a more active role in K-12 to help close the enormous gap between the skills and knowledge high school graduates have and the skills and knowledge they need to have to be successful in college, where more than a third of undergraduates have to take remedial courses. “You will never make it if the supply coming in is deficient,” said former IBM CEO Louis Gerstner Jr. “Three million kids graduate high school every year. Half of them are unprepared for life. Does that create a sense of urgency for anyone?”

California State University chancellor Charles Reed offered what was perhaps the most radical idea of the day, saying we should “blow up” the 12th grade and use that year for vocational training or, for those who want to go to college, intense preparation to get ready to do college-level work.

Duncan agreed that educators need to come up with a cohesive national strategy. But, he warned, “It’s time to end the buck-passing and blame game, where college leaders blame high schools for sending ill-prepared students, where high school principals blame the elementary schools, where elementary school principals blame the preschool programs, and preschool teachers blame the parents.”

“We don’t need a Kumbaya moment, where everyone joins hands together and sings,” Duncan said. “I’m talking instead about tough-minded partnerships that drive transformational change and deliver a “first-in-the-world” system of higher education opportunities for all Americans.”

But the question is, how do we get there?

MORE: Poll: 89% of Adults Say U.S. Higher Education is in Crisis