ECONOMICS is “not a ‘gay science',” wrote Thomas Carlyle in 1849. No, it is “a dreary, desolate, and indeed quite abject and distressing one; what we might call, by way of eminence, the dismal science.”

Carlyle was a fine one to talk. He was a brooding curmudgeon who thundered against industry, progress and the young science that sought to explain them. He found economists dismal not for the obvious reasons, such as their dry arithmetic or their gloomy preoccupation with scarcity and subsistence. Instead, he took against them because they were so wedded to the idea of happiness.

The economists of his day took their cue from Jeremy Bentham and his “utilitarian” philosophy. They calculated happiness, or utility, as the sum of good feelings minus bad, and argued that the pursuit of pleasure and the avoidance of pain were the sole springs of human action. One even looked forward to the invention of a hedonimeter, a “psychophysical machine” that would record the ups and downs of a man's feelings just as a thermometer might plot his temperature. Such people, Carlyle complained, fancied that man was a “dead Iron-Balance for weighing Pains and Pleasures on”.

The hedonimeter was never invented, and for a century or so economists fell silent about both weights on man's scales. They studied outward behaviour, not inward feelings; choices made, not pleasures taken. But in recent years, economists have become newly confident that they can measure utility as Bentham conceived it: as a quantum of pleasure or pain.

How do they do it? Mostly they just ask people. Daniel Kahneman, a psychologist at Princeton University who won the Nobel prize for economics in 2002, reckons people are not as mysterious as less nosy economists supposed. “The view that hedonic states cannot be measured because they are private events is widely held but incorrect,” he and his colleagues argue. Generally, people can say how they feel at a given moment, on a scale of zero to ten.

And if this smacks of hearsay not science, the new “hedonimetrists” can appeal to other kinds of evidence, better calculated to impress. They can look into people's eyes; or better still, their brains. People who confess to feeling happy also grin more than others. And they mean it: they smile with their eyes (a contraction of the orbicularis oculi facial muscles), not just their mouths. People's self-reports also tally roughly with what electrodes planted on their scalp reveal about the frequency and voltage of electrical waves in their left forebrain, which sparks up when they are feeling good.

Mr Kahneman's most notorious experiment took place in a Toronto hospital over a decade ago. He and a colleague asked patients undergoing a colonoscopy (in which a probe is passed up the rectum) to report their level of discomfort minute by minute. Later, they were asked how they felt about the procedure in retrospect. Their answers were surprising. The test left a worse impression on patient A, for whom it lasted less than ten minutes, than on patient B, who suffered for 24 minutes. Patients' recollections were heavily coloured by the procedure's worst moment and its last moment. The duration of the pain did not seem to make much difference. Patients were happier about a colonoscopy that lasted longer but ended better.

Fallible memories

Mr Kahneman, who is not shy of extrapolation, thinks people often choose to repeat experiences that seem better in retrospect than they did at the time. Contrary to Bentham, the “sovereign masters that determine what people will do are not pleasure and pain, but fallible memories of pleasure and pain.”

If people are bad at recalling their feelings, they are worse at predicting them. They fail to anticipate how a person feels after moving to a new city, losing a limb or winning a jackpot. Prisoners imagine that solitary confinement will be worse than it really is; mothers-to-be think the pain of childbirth will be more bearable than it typically proves to be. And it is not just unusual events that trip people up. According to Mr Kahneman, people struggle to predict how their appetite for ice-cream, low-fat yogurt or music might change in the course of a week of enjoying them. If man is an iron-balance that weigh pains and pleasures, the scales are sadly askew.

As a result, many economists now ignore one of the discipline's dreariest maxims: de gustibus non est disputandum, one does not quarrel over tastes. Robert Frank begins his 1999 book “Luxury Fever” with a long, incredulous description of the Viking-Frontgate Professional Grill, a barbecuer's folly, sporting infra-red rotisserie, rangetop burners and brass trimmings. Such purchases would once have gone unquestioned by economists. The consumer was king: if he spent $5,000 on a grill, a $5,000 grill must be what he wanted. Likewise, if he picked X over Y, a colonoscopy over an enema, pushpin over poetry, his choice should be respected. But now economists like Mr Frank and Mr Kahneman delight in second-guessing such choices, citing the evidence of their hedonimeters.

Have fun

What sumptuary advice do they offer? In general, the economic arbiters of taste recommend “experiences” over commodities, pastimes over knick-knacks, doing over having. Mr Frank thinks people should work shorter hours and commute shorter distances, even if that means living in smaller houses with cheaper grills. The appeal of such fripperies palls faster than people expect, they say. David Hume suggested that “the amusements, which are the most durable, have all a mixture of application and attention in them; such as gaming and hunting.”

But as with any argument involving economists, there is more than one side to it. For one thing, many experiences demand a substantial outlay on commodities: horses, hounds and jodhpurs, for example. And as Bryan Caplan, of George Mason University, points out, many trinkets and fripperies themselves provide a stream of experiences.

Adam Smith thought there was pleasure to be had simply in admiring the craftsmanship of a well-made watch, even if its extra accuracy was of little practical benefit. Bentham appreciated his creature comforts: according to Negley Harte, the University of London's historian, his embalmed body wears a pair of knitted underpants, unlike most of his contemporaries, who simply tucked their shirt-tails between their legs.

And before Mr Frank scoffs at Gillette's latest five-blade shaving system, he should recall Benjamin Franklin's belief that teaching a young man to shave, and keeping his blade sharp, would contribute more to his happiness than giving him 1,000 guineas to squander. The money would leave behind only regret. But self-grooming spares a man “the frequent vexation of waiting for barbers, and of their sometimes dirty fingers, offensive breaths, and dull razors.”

Richard Layard, an economist at the London School of Economics, provides one prominent example of the transformation that some dismal scientists have undergone. He made his mark with his 1991 treatise, “Unemployment”, co-authored with Stephen Nickell and Richard Jackman. On its cover, the book featured the painting “L'Absinthe” by Edgar Degas: a dejected woman and a dishevelled man, two “rather sodden” characters, as one reviewer put it at the time, pass the time and ease their sorrows with a tipple in a Paris café. The book was dedicated to the “millions who suffer through want of work”.

Today, Lord Layard argues, unemployment is no longer Britain's biggest social problem. The number of jobless Britons claiming the dole is now about 960,000. But there are over 1m people receiving incapacity benefits because depression and stress have left them unfit to work.

Lord Layard's latest book has a much jauntier image on its cover: a “happy eccentric” with a fez on his head, a monocle in his eye and a bunch of flowers in his hand. A perky character, one might say. Ambitious, policy-minded economists such as Lord Layard are no longer satisfied with raising the rate of employment. They want to lift the rate of enjoyment too.

That, it turns out, is not easy. Happiness, as measured by national surveys, has hardly changed over 50 years. The rich are generally happier than the poor, but rich countries do not get happier as they get richer. The Japanese are much better off now than in 1950, but the proportion who say they are “very happy” has not budged. Americans too have remained much as Alexis de Tocqueville found them in the 19th century: “So many lucky men, restless in the midst of abundance.”

Lord Layard and Mr Frank both blame habit and rivalry for this stagnation of morale. People grow accustomed to what they have—however much of it there is. Moreover, having a lot of things is not enough if other people have more. A rising tide lifts all boats, but not all spirits.

For economists, this is radical stuff. They traditionally argue that people best serve themselves and the public by minding their own business. Indeed, this laissez-faire attitude is one reason Carlyle attacked them. Economics, he wrote, “reduces the duty of human governors to that of letting men alone”. He was afraid this radical idea would “dissever and destroy most existing institutions of society”.

But Lord Layard argues that we cannot help minding other people's business, as well as our own. Doing well is not enough: we also want to do better than our peers. This status anxiety runs deep in our nature, he says. Vervet monkeys at the top of their social tree enjoy more mates and bananas as a result, but they also exult in their position for its own sake. As with monkeys, so with mandarins. Top British civil servants tend to live longer than their underlings, regardless of other differences in lifestyle, according to the “Whitehall II” studies which have been monitoring thousands of Humphreys and Bernards since the 1980s.

Doing well is not enough: we also want to do better than our peers. This status anxiety runs deep

To clamber up the pecking order, some people slave away nights and weekends at the office. They gain in rank at the expense of their free time. But in making that sacrifice they also hurt anyone else who shares their aspirations: they too must give up their weekends to keep up. Mr Frank reckons that many people would like to work less, if only others slackened off also. But such bargains cannot be struck unilaterally. On the contrary, people compete in costly “arms races”, knowing that if they do not work harder, they will lose their standing to someone who does.

These races are motivated by more than just prestige. As Fred Hirsch argued in his 1977 book, “The Social Limits to Growth”, many good things in life are “positional”. You can enjoy them only if others don't. Sometimes, a quick car, fine suit or attractive house is not enough. One must have the fastest car, finest suit or priciest house.

Think of the scramble for schools, Mr Frank says. Only 10% of kids can go to the top 10% of schools. In many countries, wherever the schools are good, the houses will be expensive. Thus parents who want the best education for their child must overwork to afford a house in a good school district. In doing so, however, they raise the bar for everyone else.

Is mutual disarmament possible? Not without government help, Mr Frank and Lord Layard argue. The exchequer should tax earned income heavily enough to deter one-upmanship, they say.

Despite appearances, this is not a naked example of punitive redistribution—the fiscal politics of envy. Mr Frank and Lord Layard do not want to level the social order. Their aim is much more conservative than that. Their taxes would leave the pecking order intact and envy undiminished. But people would be deterred from acting on the green-eyed monster. The problem these economists want to tackle is not inequality per se. It is that people don't know their place and scramble vainly to improve it. Carlyle, who thought man should content himself with being the worthy follower of worthy superiors, would no doubt have approved.

Go with the flow

Not that Carlyle was workshy. On the contrary, he thought that work was the only lasting measure of a man. As he put it, whatever insight, ingenuity and energy a man had in him “will lie written in the work he does”. And the “only happiness a brave man ever troubled himself with asking much about was, happiness enough to get his work done.”

Economics, on the whole, disagrees. It thinks of labour as a chore. People sell it, at the expense of their leisure time, purely as a means to the end of consumption. Indeed, Carlyle first anointed economics the “dismal science” because liberal economists insisted that American slaves be free to sell their labour in the marketplace like everyone else.

For many people, work is—as traditional economics assumes—just a way to pay the rent. But Carlyle is not the only one to see it as much more than that. In a string of experiments, Mihaly Csikszentmihalyi, of Claremont Graduate University, has handed out pagers to thousands of people who agreed to log their mood whenever prompted to do so. People were, unsurprisingly, at their happiest when eating, carousing or pottering around the garden. But some fortunate people also found deep satisfaction from losing themselves in their work: “forgetting themselves in a function”, as W.H. Auden put it.

It is easier to forget yourself in some functions than in others, of course. In Auden's poem, surgeons manage it “making a primary incision”, as do cooks, mixing their sauce, and clerks “completing a bill of lading”. This happy state, which Mr Csikszentmihalyi calls “flow”, arises most often in work that stretches a person without defeating him; work that provides “clear goals”, “unambiguous feedback” and a “sense of control”.

Where these things are lacking, people can sometimes sculpt their jobs to compensate. For example, Amy Wrzesniewski, of New York University, and her colleagues found hospital cleaners who would hold patients' hands and keep them company, brightening their day as well as scrubbing their rooms. Other researchers noted that hairdressers see themselves as more than just scissors for hire. They serve as emotional confidants for clients they like, and “fire” clients they don't.

Mr Csikszentmihalyi is now one of three scholars behind the “Good Work” project, which aims to make “flow” a more common experience in professional life. The project frets about how to square the “competing demands of excellence, ethics, and earnings”. In some fields of endeavour, such as genetic research, it found that good work was rewarded with professional success; but in others, professional pride and corporate profit seemed to tug in opposite directions. Journalism, apparently, is a “prototypically misaligned profession”, staffed by reporters who want to investigate great affairs of state but read by a public more interested in stories that are “scandalous, sensational, superficial”.

Some fortunate people also found deep satisfaction from losing themselves in their work

What to do? The Good Work project tends to blame the “market” for corrupting craftsmanship. But consumers cannot be made to want what producers care to make. Besides, “it is a thrill unique to a market society to find that people are willing to pay for one's product,” writes Deirdre McCloskey in her latest book, “The Bourgeois Virtues”. Payment is a form of applause; all the more convincing because it is costly. Furthermore, when you spend what you have earned in the market, you can enjoy knowing that you have “pulled your own weight”, taking from the national product no more than you have added to it.

If people are determined to pursue their calling rather than simply taking a job, some professions (surgery, cookery, genetics) may become overcrowded, others undersubscribed. But when a job cannot find enough takers, the market finds ways to ennoble it: first pay, and then status, begin to rise. It becomes economical to automate some aspects of the work, employing machines to do the deadening humdrum toil that men and women are no longer willing to put up with. What remains of the job will be the bits only people can do: tasks that require insight, ingenuity and the human touch. Ms McCloskey recalls the Cincinnati sewerman, interviewed a few years ago on National Public Radio, who earned $60,000 a year and liked to tell girls he was an “environmental” worker.

The dismal sage

Did Thomas Carlyle ever make his peace with the dismal science? Even his admirers admit that his “bigoted dislike of Political Economists withheld him from studying their works” or appreciating their advances. Nor did he soften much in his disdain for the fruits of commercial society: cheaper cotton and swifter railways meant nothing to him; and in his opinion, advertising, or “puffing” as he called it, deserved to be taxed out of existence.

But as any economist could have pointed out, he had a lot to thank commercial society for. Having discovered his vocation as a cultural muckraker, he eventually secured an audience, a market and even the offer (which was refused) of Westminster Abbey as a final resting place. In periods of speedy progress, it seems, stubborn reactionaries at least enjoy a certain scarcity value.