Image: Eveliina Matikainen / Yle

Two sociology researchers at the University of Turku say that the oft-quoted results of the OECD's Programme for International Student Assessment (PISA) are a poor indicator of Finnish educational quality.

Sociology professor Osmo Kivinen and researcher Juha Hedman make the claim in an article published in the journal Politiikka. Its English-language abstract is entitled Ambitious PISA results and their problematic interpretations in education policy.

The piece, for which Kivinen and Hedman analysed international critiques of PISA, sternly criticises the assumption that high PISA results indicate that a country's education system is working at its best.

"PISA has not been designed to assess how well students master the contents of the school curricula, yet opinion leaders in Finland keep reading PISA results as schools' report cards indicating the excellence of education policy in a country," the article's abstract begins.

Distorted results

The Organisation for Economic Co-operation and Development (OECD) organises a standardised triennial exam that 15-year-olds take in 72 countries and regions. The test aims to gauge the literary, mathematical and scientific skills of schoolchildren worldwide.

When testing began in 2000 there were only 32 countries in the programme. One issue raised by the researchers about the test's reliability is that some highly performing regions such as Singapore and Shanghai only signed onto the programme at a later stage, skewing earlier comparisons.

"The biggest problem is that PISA doesn't actually describe how well, say, a Finnish school is being directed or how effective the teaching is," says Kivinen. "The OECD has from the very beginning wanted to conduct a test to see how children and teenagers are equipped to handle their adult life after their basic education is over. That isn't the same as measuring how well curricula themselves develop."

Finnish PISA reports are drawn up at the universities of Helsinki and Jyväskylä. The lead professor at the latter, Juhani Rautopuro, defends the test by saying that the OECD's programme has used legitimate research units to help develop a better understanding of national education policy.

"But PISA alone isn't a basis for national legislation," Rautopuro emphasises. "You need domestic research results for that as well."

Boys left behind

Kivinen adds that another indication of the unreliability of PISA results involves the plummeting literacy of Finnish boys. Fifteen-year-old boys, whose average report card grades have declined for years, tend to perform more acceptably when their test results are compared internationally. Girls tend to test better on average in all categories.

Poor grades may preclude students from attending the high school or polytechnic that would best suit them, leading to a rise in social exclusion in the long run, researchers say.

"School needs to be changed. It currently does not work for the good of teenage boys," Kivinen says.

Even Helsinki University's PISA producer, Jarkko Hautamäki, agrees that the education sector is failing many of those it should be elevating.

"Other research has long indicated that boys are testing worse compared with girls. It is an empirical fact that boys on average are doing worse, and denying that will only aggravate the problem. But if we choose to acknowledge the facts, we can help the young people of today sculpt better futures for themselves," Hautamäki says.