I’m not doing a very good job of following my dreams. When I was 16, I wanted to be a writer. I really, really wanted it. It felt tantalisingly, exhilaratingly possible that I might just connect the means – for me, the English language – to some ultimate end: the urgent, as-yet-inarticulate truths I heard whispering to my tender teenage soul.

Instead, I studied physics at university and went to work for a hedge fund. It was supposed to be a temporary job to stockpile some savings. Fourteen years later and pushing 40, I’m still at it. I do a bit of writing when I can, but the Man Booker judges never called. Instead, I have an annual season ticket for my commute, a mortgage and a sensible car – not to mention a wonderful wife and two children. Am I a sell-out? Have I betrayed my vocation?

Many of us ask ourselves such questions. Work is a conundrum. We cannot measure the consequences of our choices against the alternatives that have passed us by. We can only try to be thoughtful and humble, empathise, observe others – sometimes a painful exercise – and speculate about what might have been.

Here, then, is my speculation. Work is something we struggle to get and strive to keep. We love-hate it (usually not in equal measure). Sometimes it seems meaningless. I’m told this is the case even for surgeons, teachers and disaster-relief workers: those with jobs whose worth seems indisputable. For the mere facilitators, the obscure cogs in the machinery of the modern economy whose precise function and value it takes some effort to ascertain, the meaning in what we do often seems particularly elusive (I should know). I contend, however, that while our lives need to be meaningful, our work does not; it only has to be honest and useful. And if someone is voluntarily paying you to do something, it’s probably useful at least to them.

The day before I started work at the hedge fund, I wrote a letter to my future self, sealed in an envelope to be opened exactly two years later. Two years was the most I could give to this lucrative but rather baffling endeavour. In two frugal years I might save enough to write a novel – the novel – gloriously undistracted by pecuniary concerns. But two years might also be long enough for money to corrupt me, and make me forget who I was. Hence, the letter.

I opened it on the appointed date. Its short, plaintive text asserted that I might one day be a good writer – if only I had the courage to try; reminded me of those still-inarticulate truths; and advised me to live sincerely. I read it earnestly enough. Then I did some sums, and told myself: ‘Two more years. There are bills and rent to pay, but two more years should be enough.’

I meant to write myself a second letter, to pass this sacred charge onward into the future, but I never got around to it. Why? Did mammon indeed get its claws into me at that point? Did my creative soul bite the dust? Actually, I was too busy writing the plans for my first novel. The bifurcation of my working life – the compromise – had begun.

Bifurcation is not for everyone. Many people have a passion or at least an enthusiasm for a career that combines a healthy dose of job satisfaction – even of meaning – with a viable wage. For these unified souls, we handwringing bifurcators seem a puzzling breed. They can be excused for being impatient with us, the conflicted, the day-jobbers or the compromisers who pursue our dreams after hours. Our whingeing, our tendency to say one thing while doing another, our procrastination, our inability to commit – these traits are tolerable in youngsters still finding their feet, but tiresome in anyone over the age of 30. It’s not that work-lovers don’t also have doubts and regrets. Work-love is rarely unconditional and unquestioning. But it is at least a solid foundation on which to build a coherent life project.

For any individual, work choices are an experiment with all the variables confounded

Of course, our experiences of work are shaped by our opportunities, which follow from our circumstances, the cards we’re dealt. Any attempt to generalise is dangerous. Wise parents and mentors know this, but typically they’re still moved to give advice when the time comes. (Career advice might be an even graver responsibility than, say, relationship advice, because the latter is likely to be ignored in any case.)

Even the cards we’re dealt can be hard to decipher. My brother and I were the first in our family to go to university. On the other hand, we were privately educated thanks to aspirational parents and generous scholarships. Privilege has many shades of grey. We are often quick to assign credit or blame for our circumstances or those of others, but each situation has a complex heritage: degrees of aptitude, hard work or health, family connections, the bank of Mum and Dad or lack thereof, external political and economic forces, cultural barriers, and the most powerful influence of all: blind chance.

Many influences are no more than conjecture. What might he or she have done, were it not for X? Population studies yield statistics that hint at causal patterns but, for any individual, work choices are an experiment with all the variables confounded. We work with what we have.

What’s certain is that not everyone has the opportunity to do a stint in a well-paid but mildly perplexing job, even if they wanted to. Many school-leavers and graduates face a choice between several uninspiring jobs that are all badly paid, and a passion that pays nothing at all. Some rejected the drudgery of their education, only to find worse drudgery ahead. Some never had much choice in the matter. Still others toiled conscientiously but now find their grades to be debased currency. If the work-devil offers you a decent price, you’re already one of the lucky ones. Indeed, this good fortune is sometimes presented as a reason to accept: ‘Others would give their right arm for this job.’ But the argument is rarely persuasive.

In Dante’s Inferno, the first group of suffering souls encountered by the poet are guilty of no deadly sin. They are the ‘unclassified’, the nothing people, who committed to nothing, stood for nothing, and achieved nothing. They are now destined to wait forever on the shores of the Acheron, neither in Hell nor out of it, wailing with regret for their wasted lives. Crikey – it’s enough to make anyone get out of bed and go to work. But this FONA sentiment (fear of not achieving) is also frequently the source of our doubt and dissatisfaction. Life seems like a pretty special opportunity, after all, and the work we find ourselves doing to pay the bills doesn’t always seem to measure up.

Let’s return to the bifurcators. Among these, some opt for a related day-job, to gather contacts or just breathe the air they love. For would-be writers, this might be journalism, teaching or working in a bookshop. Others, like me, fearing cross-contamination, or with an eye on the bottom line, or having just lost their way, work in a disparate field.

I wrote my two novels during brief spells of part-time work painstakingly negotiated (and generously granted by two different employers). I wrote in the evenings, on the train, and in a bookshop café when colleagues thought I was at the gym. Armed with GSCEs in English language and literature, I rejected any suggestion that I should learn the writer’s trade. If I could read sensitively and well, I reasoned, I should be able to write sensitively and well, too. Besides, pursuing such a self-indulgent long-shot too formally, too publicly, seemed ill-advised. The day-job was cover, of a sort.

But moonlighting is undoubtedly an uncomfortable business. By dividing finite time and energy between two endeavours, bifurcators inevitably feel they aren’t doing either as well as they could. Day-job colleagues might sense that you’re not all there; family, who were supposed to be a decisive part of the equation, find themselves outside it; and the moonlit passion is itself cramped by distraction and fatigue. You don’t get two careers for the price of one; you get half and half. That uncomfortable feeling of two half-failures is the essence of compromise.

You don’t have to lay actual bricks to build something or bandage wounds to help people

In fiction, you really can measure the consequences of work choices against alternatives. In my second novel, Learning to Die (2018), I decided to give my bifurcated life a violent tug and rip its two trouser legs apart: I would have a fictional writer character who really did follow his dreams, and a fictional city trader who followed the money. They would meet, get to know each other and have it out. The results of this experiment were not what I expected.

The novel follows five characters in all, each thirtyish, each questioning very different life and work choices made possible by moderately affluent circumstances. The writer and the financial trader emerge as particularly miserable specimens. In private thoughts they describe themselves in terms such as ‘fraud’, ‘failure’ and ‘waster’, and mope about accumulating future regrets. Yet when their correspondence begins, they find themselves insulting each other while passionately defending their own choices. It takes a series of outside shocks – an illness, a chance reunion, a vengeful act of biblioclasty – to reconcile these internal conflicts.

Of course, my fictional so-called experiment (and this article) could be just an exercise in self-justification. Or, conversely, in self-flagellation. Indeed, our degree of satisfaction with life and work decisions might depend on the outcome of a tussle between two irrational and conflicting biases of the human mind: that powerful, defensive tendency for self-justification, yes – but also an opposing tendency, subversive to our happiness, to undervalue what we possess (an infuriating trait discussed and dissected by writers from Michel de Montaigne to Marcel Proust). Our predominant attitude might depend more on personality than the facts of the case.

Nor should we be fooled into thinking that meaningful jobs are somehow harder to come by now than in the past. This has been a popular grievance for centuries, but you don’t have to lay brick on actual brick to build something, wield a scythe to be productive, or bandage wounds to help people. The creep of abstraction and specialisation is less a conspiracy, more a complex evolution driven by our collective will to achieve more. Not just to consume more (though there is that), but to understand more and to solve more problems. If Bartleby the Scrivener, Ivan Ilyich and David Brent all struggled to find meaning in their work, so too did the New Testament’s labourers in the vineyard.

Today, we have higher expectations of work, and more choices over which we can dither and fret. Tales of bold and triumphant career choices are a media staple. We also have a bigger toolbox in our search for meaning. Technology is a boon for bifurcators, in particular (no need to carry your typewriter on the bus), while the internet has also enabled a growing minority to escape bifurcation by translating their passion directly into a viable income. Circumstances still unfairly constrain opportunities for many but, for plenty of others, whingeing is unwarranted.

Advocates of dream-following, of commitment and career leaps of faith, often say: ‘You’ll regret it if you don’t.’ They might be right about that (actually, they almost certainly are). But here’s the rub: regret is not the sole preserve of the cautious compromiser. A failure to compromise can also beget future unhappiness. Some of your sacrifices might come back to haunt you. I wouldn’t dare to advise young idealists either to follow or not to follow their dreams. At the most, I might advise them to be wary of career preachers, whether radical or conservative, and that, while they shouldn’t fear commitment and leaps of faith, they also shouldn’t fear compromise.

The celebrated achievers are usually those who committed to their passion. But often they sacrificed a lot. There is another group of achievers, less celebrated but perhaps happier on average, whose central accomplishment is a balancing act – between work and family, money and meaning, pursuing their passion at the weekend and getting an early night on Sundays. I’m only a novice bifurcator but I know my role models. They’re not famous and never will be, but you probably know some too.