Those conducting serious polling, and academic surveys of public attitudes, usually make substantial attempts to ensure that their questions they pose in their surveys are neutral and even-handed. As someone who has been involved in such work for years, I can testify to often prolonged and even agonised attempts to ensure that a question wording is as fair and balanced as we can make it.

Occasionally, however, responsible surveys deviate from this norm: not because they are trying to push respondents to appear to back a particular viewpoint, but simply to gauge public reactions to particular views or political statements. During an elections or referendum campaign, for instance, one might quote the main ‘lines’ from the different campaigns, and ask respondents to indicate the extent to which they agree or disagree with each. Each of the individual statements thus used will very clearly not be balanced; normally, though, various such statements from different viewpoints would be included, thus achieving a broader balance in the round.

This latter form of question was used in the April Welsh Political Barometer poll. To mark the twentieth anniversary of the creation of the National Assembly for Wales, various questions probing public attitudes to devolution was included in the poll. One set of questions was a series of pointed statements, to which respondents were asked to indicate their extent of agreement or disagreement. One of these statements has proven rather controversial:

“The National Assembly has been too dominated by Welsh speakers”

There is a history to this question. The 1997 Welsh Referendum Study, conducted in the immediate aftermath of that year’s referendum, asked their sample of respondents about their expectations of what devolution would mean. Some of their questions were in the form of pointed statements, to which people were asked to agree or disagree, with many of these statements based on the main lines taken by the respective Yes and No campaigns in the referendum. In both the 1979 and 1997 referendums, at least some of the opponents of devolution had sought to suggest that a devolved Welsh Assembly would engender greater domination of life in Wales by speakers of Cymraeg – implying that those who did not speak Welsh had something to fear if there was a Yes vote.

The 1997 study (with which I was not involved, although my Wales Governance Centre colleague Richard Wyn Jones was) therefore included a short battery of statements, based on some of the main concerns raised by No campaigners. The specific question format asked:

“[P]lease say how much you agree or disagree with each of these statements. A Welsh Assembly would…

– Be dominated too much by the Labour party

– Pay too much attention to South Wales

– Simply mean more jobs for politicians

– Cost too much to set up and run

– Be dominated too much by Welsh speakers”

When I was Principal Investigator of the 2016 Welsh Election Study, it seemed to the team conducting that study that it was appropriate to explore whether some of the fears raised in 1997 had been realised. We therefore ran the following battery of questions, which adapted some of the 1997 questions alongside a few others that now seemed to speak to more contemporary concerns:

“To what extent do you agree or disagree with the following:

– The National Assembly for Wales should have the same level of powers as the Scottish Parliament

– Having a National Assembly has improved the way Wales is governed

– Having a National Assembly has improved the way the UK is governed

– The National Assembly has been too dominated by the Labour Party

– The National Assembly has paid too much attention to Cardiff

– Having a National Assembly has simply meant more jobs for politicians

– The National Assembly has cost too much to set up and run

– The National Assembly has been too dominated by Welsh speakers

– It is often difficult to figure out which level of government is responsible for what

When ITV Wales wished to run some questions for the Assembly’s twentieth anniversary, as part of the team that draws up the questions for the Barometer polls I contributed to the discussion by sharing with them some questions that had been run in previous academic surveys. They decided to run the complete battery of 2016 statements that I have just quoted; results from these questions were then used in some of their programmes, and in my recent blog post: http://blogs.cardiff.ac.uk/electionsinwales/2019/05/09/devolution-at-twenty-what-do-the-people-of-wales-think/.

It was rather a surprise that Ifan Morgan Jones appeared to suggest online that the question about Welsh speakers was somehow inappropriate; and disappointing to receive some rather unpleasant comments and messages afterwards from others.

The question on Welsh speakers does touch on a sensitive area. But the politics of the devolution referendums was often unpleasant. Sustained attempts were made by some (though certainly not all) No campaigners in both 1979 and 1997 to stir up divisions and fears around the Welsh language. That is why the 1997 referendum study asked about voter perceptions of such fears: it was important for a serious study of that referendum to explore the extent to which such campaigning methods had gained traction with the public.

For the record, here is the comparison between the 1997 results and those generated by the recent Barometer poll:

Assembly ‘dominated by Welsh speakers’ 1997 2019 % Agree 31% 19% % Neither Agree nor Disagree 20% 33% % Disagree 43% 31% % Don’t Know 5% 18%

The different methods of sampling employed by the two studies may well account for much of the rise in Don’t Know responses. Looking elsewhere at the results, we see equal declines in the percentage both agreeing and disagreeing with the statement, and a rise in those choosing the neutral option. We cannot really conclude from the data that the fears stirred in the referendum campaigns have been comprehensively laid to rest.

The job of social science is to examine society: all of it, including – perhaps especially – the unpleasant parts. The attempts to stir up divisions and fears around Cymraeg in the 1979 and 1997 devolution referendums were, in my view, an ugly and shameful part of our history. That is precisely why it is important that we explore the extent to which such attacks still have any resonance today. Sadly, they still seem to have some.