It was dramatic research that made headlines everywhere. A 17% fall in the number of heart attacks in the year since Scotland stubbed out smoking in public places.

Startling - if true.

Few questioned the research when it was revealed two months ago. Politicians trumpeted the numbers as vindication of a policy introduced a year earlier than in England. Journalists obediently followed suit.

The most arresting finding was that heart attacks among non-smokers had apparently fallen even faster than smokers, suggesting that passive smoking was often to blame.

Then a week ago, with rather less fanfare, routine statistics on hospital activity were published by the official source for health data in Scotland, as they are every year, this time including the time since the ban.

These show a fall in heart attacks for the year from March 2006 - not of 17%, but less than half as much at about 8%.

What's more, taking out the recent trend, this is halved again. Heart attacks have been falling steadily for some years now. The percentage falls in the three years before the ban were 5.1%, 4.7% and 5.7%. So the fall since is still bigger than the trend would lead us to expect, but bigger only by about three or four percentage points - an improvement, but retreating fast from the magnitude of 17.

The latest release also makes clear that even an 8% fall in heart attacks is not unprecedented. There was another, larger drop between 1999 and 2000 of about 11% (see chart).

This seems to demonstrate significant variability around the trend, suggesting that last year's 8% drop might even be the result of chance. It is conceivable, although perhaps unlikely, that the smoking ban had no effect at all. The figures could be a result of no more than the ordinary ups and downs of statistical variation from one year to the next.

There have been several other initiatives designed to reduce heart attacks in Scotland recently, and any or all might also have played a part.

If we look at all heart attacks, rather than just emergency admissions, the fall last year according to the official data was slightly smaller again - about 7% - against falls in the past three years on this measure typically of about 4%.

Rise and fall

Scotland's deputy chief medical officer, Peter Donnelly, said at the time of the study in September: "One of the most important findings is the reduction in heart attacks. We believe that the smoking ban was a large contributory factor to this drop."

Scotland's ban came into force 20 months ago

Such claims are, in fact, positively damaging. What is unquestionably a good news story - the consistent and impressive decline in heart attacks in Scotland over many years - has become overshadowed by squabbles about the smoking ban.

Worse, the true reasons for this success may be overlooked if too much credit is attributed to the ban on unreliable evidence.

So why such a discrepancy between the two numbers?

The first study was presented at a conference in Edinburgh two months ago by a research group called StopIt (Study Of Public Place Intervention on Tobacco Exposure).

The latest statistics come from ISD Scotland, part of the NHS responsible for health data. Dr Colin Fishbacker, an ISD official, says the two studies used data from different sources. "We wouldn't expect the sources to agree exactly. The routine data we publish is based on the discharge summary sent to the GP and medical records for coding. The StopIt study was a specific research exercise."

Numbers game

There are other factors that may might help explain the discrepancy. The first study, unlike the routine data, was a sample, not a total for all Scotland. And although the sample included nine Scottish hospitals with about two-thirds of heart attack admissions, patients did not have to take part.

A patient in cardiac intensive care

But because the data on which the StopIt study was based has never been published, and nor has the study itself, it is impossible to say exactly how it was done. Attempts to obtain it or to talk to the lead researcher have gone unanswered.

Once the number was out, politicians were certainly not about to exercise any scientifically-minded reservations about its reliability. The story became political as much as medical.

None of this rules out the possibility that the ban is having an effect. Such a conclusion would be as reckless as declaring that the ban explains everything. But how much effect it's having - if any - no-one knows.

As for the flurry of excitable headlines, what appeared to be hard medical evidence now looks more like over-hasty and over-confident research, coupled with wishful political thinking and uncritical journalism. Whether the same will happen now attention has shifted to the effect of the bans in England and Wales remains to be seen.



Michael Blastland is the author, with Andrew Dilnot, of The Tiger that Isn't - Seeing Through a World of Numbers.