James Rogers and Renan Reyes, veterans of the Vietnam war, each made a trip to Washington on Wednesday to see the Vietnam Veterans Memorial for the first time.

“Very impressive,” said Rogers, who is from Madison, Alabama, as a river of parents and children flowed past in bright sunshine.

“It looks like a black mark,” said Reyes, from near Charlotte, North Carolina, disapprovingly.

Divergent opinions over the polished black granite memorial – which lists, chronologically, the names of more than 58,000 Americans who died in the war, from John H Anderson Jr to Jessie C Alba – are peculiarly apt for the Vietnam war itself, a politically and socially polarising episode that shattered the myth of American invincibility.

The war in south-east Asia is now the subject of an epic 10-part, 18-hour series by Ken Burns and Lynn Novick. Burns is America’s premier documentary film-maker, renowned for his 1990 masterpiece on the civil war as well as series on jazz, baseball, the Roosevelts and the second world war. Ten years and millions of dollars in the making, covering the conflict from all sides, The Vietnam War could be the closest thing yet to a definitive account of what Burns believes is the most important event in American history in the second half of the 20th century.

The time for a conversation “about a war we have consciously ignored” has come, Burns, 63, told the National Press Club in Washington earlier this month. “We have said: ‘We don’t want to talk about it. We’re not gonna teach it, we think it’s about this, or my own personal politics at this moment has actually determined what I should say about Vietnam regardless of what I felt when it was taking place.’ We have this dissonance going on.

A preview of the series

“We hope that the film will contribute in some way, shape or form to more courageous conversations about what took place, because let us also be very clear that the divisions that we face today, the lack of civil discourse, the inability to talk with each other but only at each other, had their seeds planted in the Vietnam war, so if we understand it then we also understand our present moment.”

The origins of the conflict are now somewhat foggy in collective memory. A 1954 ceasefire agreement partitioned Vietnam into a communist north and anti-communist south. Trapped in the logic of the cold war, the US backed a series of corrupt regimes against the communist-led Vietcong in the south and their allies in the north who sought to reunite the country.

In 1965, President Lyndon Johnson sent in thousands of air and ground forces in what was initially a popular move. But as the draft expanded and casualties mounted, public opinion turned against him and anti-war protests erupted against a backdrop of social unrest, racial discord and assassinations.

Bill Zimmerman, an anti-war activist, tells the documentary: “People who supported the war were fond of saying, ‘My country, right or wrong,’ [but protesters didn’t] want to live in a country that we’re going to support whether it’s right or wrong. So we began an era where two groups of Americans, both thinking that they were acting patriotically, went to war with each other.”

In the early 1970s, under President Richard Nixon, the war expanded into Cambodia and Laos, but in 1973 US forces quit Vietnam, and in 1975 South Vietnam fell to the communists.

The Vietnam war has gripped popular consciousness with images of Huey transports (helicopters) taking thousands of US troops into battle in thick jungles, river deltas, fields of elephant grass and hamlets of rice paddies and thatched-roof huts. It spawned a genre of movies including Apocalypse Now, Born on the Fourth of July, Coming Home, The Deer Hunter, Full Metal Jacket and Platoon.

But Burns and co-director Novick, who made several trips to Vietnam, aim to strip away US-centric narratives and give a rounded portrait with 80 interviews from both sides. Gen Lo Khac Tam of the North Vietnamese army tells them: “The war was so horribly brutal. I don’t have words to describe it. How can we ever explain to the younger generation the price paid?”

Renan Reyes, 66, a Vietnam war veteran, at the memorial. Photograph: David Smith/The Observer

Speaking a a recent event hosted by the New York Times, Burns reflected: “If you think about The Deer Hunter, there’s not a Vietnamese character that’s real, and that’s our problem, and what we set out to do consciously in this was to say we wish to triangulate this story and understand that for many people [in Vietnam] it was an American war, and not just the Vietnam war.”

Novick told the Guardian: “It was really profound to begin to appreciate the scale of tragedy there, the scale of loss. I think Americans understandably focus on the 58,000-plus Americans who lost their lives in the war, which was a tragedy for the families involved and there’s no way to make that up to them, but when you go to Vietnam you begin to understand that a country of 30 million lost as many as 3 million people; that’s 10% of the population. What that means is that everyone you meet knows somebody who died ... Every single person you ask says, ‘Oh, my uncle, my cousin, my neighbour, my niece’ – someone they knew personally died. So the weight of that for a country, for a people, is indescribable. Feeling it over time is a profound thing.”

Was the war a needless waste, a terrible mistake that could have been avoided? Novick replied: “You will have to watch the film to find out. That is the central question, really, or one of them. There’s a lot of conflict between Americans and even within individual people about the answer to that very question. It’s a very deep one and a very important one but I cannot give you an easy answer because it’s not settled. It’s still very much an open question and our film is an exploration of that.”

The series sweeps from the release of the leaked Department of Defense study about the war known as the Pentagon Papers to the Tet offensive to the anti-war activist Jane Fonda’s visit to Hanoi. Researchers consulted former secretary of state John Kerry, a veteran of the war, and Senator John McCain, who was taken prisoner, and the film tells their stories, although they are not interviewed. Donald Trump received five deferments from the draft: four for university and one for “heel spurs”.

Burns and his research team found their fundamental preconceptions challenged, the film-maker said at the New York Times event. “At every intersection, there was the explosion of myth, there was the humiliation of being just dead wrong about what we thought had happened. And that, at some point, if it’s not about your own self-aggrandisement, is exhilarating and liberating.”

The series, which premieres on the Public Broadcasting Service (PBS) on 17 September in the US and will be released in full on DVD in the UK, includes rarely seen archival footage, photographs, TV broadcasts, home movies and secret audio recordings from the Kennedy, Johnson and Nixon administrations, as well as music of the period from the Beatles, Rolling Stones, Bob Dylan, Jimi Hendrix and Simon & Garfunkel. In a trailer, the army veteran Phil Gioia says: “The Vietnam war drove a stake right into the heart of America.”

Speaking by phone this week from his home near San Francisco, Gioia, 71, told the Guardian: “It was a defining event in the history of the country. It radicalised components of our society and polarised our society in many ways. It affected our decisions in defence strategy ever since.”

Gioia, who comes from a military family, served in Vietnam in 1968 and 1969 and was shot in the left wrist. “I fell like somebody had hit me with a baseball bat,” he recalled. “It was stunning but no pain because the nerves closed down.” He was also present in Huế to witness the Tet offensive launched by the communists, now seen as a crucial turning point. “That was like being at Gettysburg in the civil war.”

By then the war was the target of growing protests at home. Gioia said: “When you’re a soldier you don’t get to pick your war. You go wherever the government sends you.” On his second tour of duty, he added, most of the soldiers were 18, 19 and 20-year-olds. “Even though the war was unpopular and they were mostly draftees, they were terrific soldiers. Most of them were vilified when they went home. A lot of the population transferred their anger to the soldiers. It was a very unfortunate time.”

Two thirds of Americans who served in Vietnam are no longer alive, Gioia noted, while the majority of Vietnamese people were born after the war. “There has not been a Hollywood movie that really accurately portrays Vietnam and the effect of the war. Apocalypse Now, Platoon – it’s all Hollywood. Politics and the anti-war sentiment gets into every movie made by Hollywood. By getting Vietnamese voices, we’re going to see and hear their aspect of the war. They took huge losses; we bombed the hell out of them.”

Gioia, a semi-retired technology and venture investor and student of military history, added: “It’s time the country took a good long look at what happened. This is probably going to be the keystone go to documentary for the Vietnam war. It’ll be, if you want to know what happened and why, watch the Burns film.”

His image of a stake through the heart was echoed by visitors to Maya Lin’s war memorial, completed in 1982, in Washington this week. Reyes, 66, the veteran from North Carolina, accompanied by his three nine-year-old grandchildren, said: “It tears at my heart because some of my friends were killed. We recovered their bodies from a helicopter crash in 1970.”

Amid deep social upheaval and protest, the homecoming was very different from the warmth that generally greets today’s returning veterans from Afghanistan and Iraq. “It was like I didn’t exist,” recalled Reyes, a retired police officer. “The only welcome I got was from my parents and sisters.”

Rogers, 68, the veteran from Alabama who had come to find the names of two comrades, added: “When we came back, it was a different time. We were stoned. We didn’t get much respect and it was painful.”

Burns and Novack are following in the footsteps of the journalist Stanley Karnow, whose award-winning, 13-part PBS series Vietnam: A Television History was one of the most-watched public television documentaries ever when first shown in 1983. But more than three decades later, veterans, historians and journalists welcome a fresh look.

Arnold Isaacs, who covered the last three years of the war for the Baltimore Sun and wrote two books, Without Honor and Vietnam Shadows, said: “It was compelling for me but I think it’s pretty distant for 25-year-olds today. World war two was such a significant event in every respect everywhere in the world and continues to still shape the cultural concept of war – the good war myth remains powerful – whereas Vietnam was inconclusive.”

Isaacs, 76, praised attempts to include Vietnamese voices. “Remembering this war as a part of American history is profoundly distorting of what happened. It’s a part of Vietnamese history we jumped in on. I was back in Vietnam in 1998 and it was obvious to me there were a lot of unhealed wounds. In this country you have a culture war that is still going on; Vietnam was part of that story, but not the whole story.”