Earlier this year, the actor John Cleese, now 80, repeated his claim: “London is no longer an English city.” In 2011, he had told an Australian audience: “I love having different cultures around, but when the parent culture kind of dissipates you’re left thinking: ‘Well, what’s going on?’” He had previously declared: “I love being down in Bath because it feels like the England that I grew up in.”

In May, he doubled down, insisting his foreign friends felt the same way, “so there must be some truth in it”, and describing London (wrongly) as “the UK city that voted most strongly to remain in the EU”.

We will leave aside the fact that Cleese lives in the Caribbean. His meaning was clear: in a familiar, wilful and tiresome confusion of race and place, he was disoriented by the multiracial and multicultural nature of Britain’s capital.

For his own peace of mind and ossified sense of nostalgia, Cleese should steer clear of Tate Britain for a while. Because next week, the artist and film-maker Steve McQueen’s Year 3 will open there – a display of school photographs from almost two-thirds of London’s primaries. There they stand: more than 3,000 photographs, showing about 76,000 children. The project’s photographers have captured the full range of the capital’s seven- and eight-year-old citizenry: from state, private, religious and special education schools, uniformed, non-uniformed, daffy grins, big ears, long braids, scuffed shoes, ironed headscarves and wild afros.

There is a transfixing power to these pictures when laid out together. You are struck by the sheer size of the capital, its diversity in every manner – the full force of its vast and vital nature. It is hard to see how one could look at these kids, standing in rows, wide-eyed and wide-smiled, and come away with a sense of moral panic. But you know it is out there in this moment of parochial anxiety, dislocation and division. In a city with virtually the same number of non-white people as white British people (according to the last census, in 2011, London was home to 45% who described themselves as “white British”, as opposed to 40% who did not describe themselves as white at all), it provides a snapshot not just of the present, but of a future that would leave the likes of Cleese thinking: “Well, what’s going on?”

Year 3 at a state primary in east London. Photograph: ©Steve McQueen & Tate

In a sense, that is precisely the effect McQueen, 50, wants the exhibition to have, even if he takes visitors on a different journey from Cleese’s. “It’s a telescope and microscope situation,” he says. “You’re looking through a telescope and seeing all these people, but it’s also reflecting back on you, making you think: ‘Who am I in all of this?’ It’s exciting and dynamic. I hope the people and the powers that be are aware of that potential and reap from it. I find it very inspiring. The future is in those pictures. There’s going to be the criminal, the banker and the person who may not make it to 21. They are all there.”

Getting them there was no small feat. “We spent a year and a half trying to work out how to do it,” says Clarrie Wallis, the exhibition’s curator. “Very early on it became clear that you couldn’t just do a callout to all the schools in London. There were issues like safeguarding that came into play. We had to make sure we had in place something that was robust and appropriate.”

They needed the consent of every child’s parent or guardian. Some schools had the will but not the wherewithal. As fantastic an opportunity as it was, this was one more demand on a system that was understaffed and overstretched. “Some schools felt they didn’t have the time or the capacity,” says Wallis. “A lot of them are under-resourced anyway and the arts programmes get squeezed.”

So the Tate had to work out an “offer”, providing arts-based workshops with the help of A New Direction, an organisation that specialises in creative work within schools. Facilitators and photographers went into schools to do various exercises with children that related to the broader project.

Every year 3 class that took part has been invited to come and see its photograph in the exhibition. Again, this is no small undertaking, involving everything from employing greeters to usher the children around, including a small tour of the Tate, to portable buildings in the car park where they can eat their lunch. “It’s an opportunity for us to champion why creativity matters,” says Wallis. “The hope is that they come, they’ll fall in love with us and want to come back again.” When the exhibition is over the schools will be given the framed photographs.

Alpha preparatory independent school in Harrow, north-west London. Photograph: ©Steve McQueen & Tate

There is something about the simplicity of the concept that makes the exhibition mesmerising. Row upon row of classic school photographs, all taken with the same format. The teacher in the centre, set back slightly into the class – embedded into their brood – with the teaching assistants towards the back and to the side. Where the class is larger, the front row sits on the floor cross-legged, the second row is on chairs and all the rest are standing up. “We had the photographers all stand a certain distance away, to allow for the variation in the size of classes,” explains Wallis, pointing out that while some classes could go up to 34 children, others, particularly in schools for children with special educational needs, might number eight or nine. However big or small the class, all the children are shown at roughly the same size. All were shot with natural light.

The effect of this rigorous consistency in form is to draw out the difference in content. The pictures were taken wherever the school chose to accommodate them – in gyms and classrooms, under basketball nets, with backdrops of climbing bars, the Virgin Mary, blackboards and artwork. There was little retouching – chipped walls remained chipped.

“I love that fact that there’s a similarity,” says McQueen. “Everyone is regimented in the same way. There’s conformity. But there’s individualism, too. Each one is an individual. They’re trying to make you conform. But, at the same time, there’s still that spark there, that individuality, that questioning.”

This tension, between the energy of the child at this stage of their development and the demands of the system, at this point in the process, McQueen finds particularly intriguing.

“It’s this moment, when things have almost become regimented, just before they have to conform. It’s not that you lose them,” he says. “But it’s a whirlwind and they have to stand up in it. And the ones that can’t stand up often get chopped down and disregarded. And at the same time those that can also try to sustain a bit of themselves within that system. So you see the potential and possibility in these kids’ lives.”

The initial idea, he says, came from his own experience of primary school, which he remembers as a time of wonder, boundless energy and fearless freedom. “It was incredible. The friends I had, and the potential, the possibilities. Playing, drawing, the kinship of friends, experimenting, getting wet in the rain. I had the chance to daydream. But then you think of what could have happened and where people I knew had gone to.”

Year 3 at Northway special school in Barnet, north-west London. Photograph: ©Steve McQueen & Tate

The whole thing makes for quite the rabbit hole. You start by looking at a picture, your eye drawn to the biggest smile, the tallest kid, the goofiest look or the teacher at the centre, who collectively offer a less diverse demographic than the children (they are mostly women), but are varied nonetheless. You scan that class, then another, then another, thinking of how unlike your own school they are, or how similar, and before long you are reaching back to yourself at that age. To how big things were and how small your world was. Trying to remember who your teacher was, or your best friend, or your arch-enemy. Recalling the more troubled kids and wondering what happened to them. What happened to any of them. Some might be grandparents by now; some could be millionaires.

I have scant memories of class photo days, but I remember the sequence of events that made them unique to me and my family. The teachers would give us a letter to take home saying the photographer would be in on a certain day. I would lose it or forget it. I would generally be scruffy (that was and still is ‘“my look”), my hair a mass of tangles and knots.

When the photos came back, I was always easily identifiable. I am the same age as McQueen but grew up in Stevenage in Hertfordshire and was generally the only black kid in the class. I would take the picture home and this would be the first my mum would know that a picture had been taken. She would hit the roof, pointing out how my dishevelment shamed her, sigh, and then say, “I like this one” and remark on my dimples or say I had my father’s smile. She would buy a ridiculous number to “send home” to Barbados and they would take their place along with the dresses, fabric, trousers and linen she hoarded for the same purpose and that never made the journey.

For McQueen, who has two children, it is also a love letter to the city where he was born and raised. He has spent much of the last two decades in Amsterdam, but loves London. “I need this city because it’s given me so much,” he says. “It’s given me my belief system. It’s given me my viewpoint on life.”

It hasn’t always loved him back. As McQueen progressed through the secondary system during the early 80s, he felt the walls closing, seeking to bury that sense of possibility under the dead weight of institutional racism. Simultaneously laughing and shaking his head, he mentions Tam Joseph’s painting UK School Report, which consists of three portraits side by side. The first, of a black youth in school blazer and tie, with a neat afro, is captioned: “Good at Sports”; the second youth is funkier, slightly older with twists and an earring, and reads: “Likes music.” The last has long locks and an open shirt above the words: “Needs surveillance.” “By the time I got into high school things had definitely changed,” says McQueen.

This is partly why he chose year 3. It is at seven or eight, he says, that children start to shed their solipsism and navigate the structures that shape their world. “It’s a limbo. It’s the first time these things are being introduced to them like race, class and gender. How do they grapple with that? I don’t know.”

If that is a challenge for the kids, it is also a challenge for London. With the interview over, McQueen – a man of few, well-chosen words – circles back to finish a thought. “This is the future. They have to make it work. They don’t really have a choice.”

Steve McQueen Year 3 is a partnership between Tate, Artangel and A New Direction. The Tate Britain exhibition runs from 12 November to 3 May 2020. An outdoor exhibition, organised @Artangel, is on billboards across London until 18 November. #Year3Project