What happens when you attach several electrodes to your forehead, connect them via wires to a nine-volt battery and resistor, ramp up the current and send an electrical charge directly into your brain? Most people would be content just to guess, but last summer a 33-year-old from Alabama named Anthony Lee decided to find out. "Here we go… oooahh, that stings a little!" he says, in one of the YouTube videos recording his exploits. "Whoa. That hurts… Ow!" The video cuts out. When Lee reappears, the electrodes are gone: "Something very strange happened," he says thoughtfully. "It felt like something popped." (In another video, he reports a sudden white flash in his visual field, which he describes, in a remarkably calm voice, as "cool".) You might conclude from this that Lee is a very foolish person, but the quest he's on is one that has occupied scientists, philosophers and fortune-hunters for centuries: to find some artificial way to improve upon the basic cognitive equipment we're born with, and thus become smarter and maintain mental sharpness into old age. "It started with Limitless," Lee told me – the 2011 film in which an author suffering from writer's block discovers a drug that can supercharge his faculties. "I figured, I'm a pretty average-intelligence guy, so I could use a little stimulation."

The scientific establishment, it's fair to say, remains far from convinced that it's possible to enhance your brain's capacities in a lasting way – whether via electrical jolts, brain-training games, dietary supplements, drugs or anything else. But that hasn't impeded the growth of a huge industry – and thriving amateur subculture – of "neuro-enhancement", which, according to the American Psychological Association, is worth $1bn a year. "Brain fitness technology" has been projected to be worth up to $8bn in 2015 as baby boomers age. Anthony Lee belongs to the sub-subculture of DIY transcranial direct-current stimulation, or tDCS, whose members swap wiring diagrams and cautionary tales online, though if that makes you queasy, you can always pay £179 for Foc.us, a readymade tDCS headset that promises to "make your synapses fire faster" and "excite your prefrontal cortex", so that you can "get the edge in online gaming". Or you could start spending time on a brain-training site such as Lumosity or HappyNeuron, the latter boasting games "scientifically designed to stimulate your cognitive functions". Or start drinking Brain TonIQ or Brain Candy or Nawgan or NeuroPassion, or any of the other "functional drinks" that promise to push you past your cognitive limits.

One problem with Brain TonIQ is that it's disgusting, albeit not as disgusting as Nawgan ("What To Drink When You Want To Think"), which tastes so metallic, it's like drinking the can that it comes in. For the last two weeks, I've been working through a succession of these drinks – and a packet of Focus Formula herbal pills – while wearing a NeuroSky MindWave headset, which thankfully isn't sending current to my brain, but claims to be monitoring my brainwaves via a sensor on my forehead. This is a system of "neurofeedback": the headset is linked to my laptop, which plays the sound of Buddhist chanting through headphones; when my attention wavers, the pitch of the chanting falls, so I'm supposedly being trained to concentrate. I've been playing brain-training games daily. At the start of all this, I took a "culture-neutral" intelligence test, and scored 129, on a scale derived from IQ (which stops being meaningfully measurable around 200). It's not technically an IQ score – and IQ scores are very questionable things, anyway – but if I can boost it by a few points, I'll be willing to declare victory.

Yes, yes, I'm aware that this is all hopelessly unscientific. The intelligence test wasn't a formal one; the placebo effect could be enormous; and even if some of my tactics worked, I'd have no way of identifying which. But in the world of cognitive enhancement, good science regularly takes a back seat to speculative self-experimentation. Dwell on the science and it's liable to make you anxious: according to one study, a key ingredient in Brain TonIQ, dimethylaminoethanol, has been shown to decrease the average lifespan of aged quail. When you're trying to become superhuman at thinking, there are some things it's best not to think about.

The big conundrum at the core of the brain-enhancement debate is this: what counts as "getting smarter"? Many of the claims made by the industry aren't false, but rather boringly true: of course online training games "stimulate your cognitive functions" and "change your brain", since pretty much everything does. And nobody disputes that it's possible to learn new skills, such as speaking German, or riding a bike; nor that taking a substance such as Modafinil or Adderall, now routinely deployed by some students as "study drugs", will temporarily supercharge your focus. It's also pretty easy – relatively speaking – to boost your working memory, for example by learning tricks to remember long strings of digits, as described by Joshua Foer in his bestseller Moonwalking With Einstein. But those tricks aren't transferable: ask a champion digit-memoriser to solve a cryptic crossword, and he'll probably do no better than the rest of us.

The holy grail is to find a way of increasing "fluid intelligence", our underlying capacity to hold information in conscious memory and then manipulate it in order to solve complex problems or come up with new ideas. Fluid intelligence is what IQ tests try to measure – albeit, historically, with all sorts of cultural biases – and the implications of improving it could be huge. "There are approximately 10 million scientists in the world," Nick Bostrom, of Oxford University's Future of Humanity Institute, told Time magazine a while back. "If you improve their cognition by 1%, the gain would hardly be noticeable. But it could be equivalent to instantly creating 100,000 new scientists." But even how to think about this in the first place is a tricky question, as the Imperial College neuroscientist Adam Hampshire points out, because "general intelligence" is a construct: it's an idea we use to group together certain aspects of brainpower, so it's unlikely to be related to just one aspect or system in the brain.

Until only six years ago, when it came to the possibility of increasing fluid intelligence, the verdict was almost uniformly pessimistic. But then, in 2008, a pair of workaholic psychologists from Switzerland, Susanne Jaeggi and her boyfriend Martin Buschkuehl, published a study that sent eyebrows shooting upwards, and that's still being fiercely debated today. "That study was the D-day invasion," says the science writer Dan Hurley, whose book Smarter: The New Science Of Building Brain Power will be published in the UK this month. "That really put down a marker that said: this is real. You can really do this."

The Jaeggi study relied on an especially vicious brain-training game known as the "dual n-back". You can try it for yourself at soakyourhead.com, but I can't recommend it, because it's hellish. "The first time they try it, everybody's impression is, 'Oh, this is impossible, this is crazy, this is awful,'" says Hurley. "It feels like someone just asked you to pick up a car." The game works like this: you hear a voice, slowly reciting a sequence of letters: "B… K… P… K…" Whenever you hear a letter that's the same as the one before last, you press the L key on your computer. So far, so tolerable – but at the same time, you're playing a visual version of the same game, in which one of a set of eight squares lights up in orange; when the illuminated square is the same as the one before last, you press your computer's A key. Doing both these tasks at once feels savagely unpleasant, but if you make it through to the end, something worse is in store: on the next level, you do the same thing, except you're looking for matches two times before last. If you can make it to the next stage – looking for matches three times before last – you're probably a witch.

'I’ve been working through these drinks while wearing a NeuroSky MindWave headset, which claims to be monitoring my brainwaves via a sensor on my forehead.' Photograph: Christopher Lane for the Guardian

Jaeggi and Buschkuehl persuaded undergraduates at the University of Bern, and later other subjects, to submit to the dual n-back for several minutes a day, over weeks. They tested their fluid intelligence using Raven's Progressive Matrices, a widely respected test involving visual pattern manipulations. (Think of those old newspaper ads for Mensa, and you won't be far off.) What they discovered upended the conventional wisdom: after 19 days of training, their subjects recorded a 44% average performance boost on the Raven test. By then, the first generation of commercial brain games had been largely discredited: playing Dr Kawashima's Brain Training on your Nintendo, it's now clear, will only make you better at playing Dr Kawashima's Brain Training on your Nintendo. But playing the dual n-back, it appeared, could truly make people more intelligent.

There are few surer ways to create a firestorm among psychologists and neuroscientists, it turns out, than to claim such impressive changes to an aspect of intelligence long considered fixed. Some in the field compared the Jaeggi findings to cold fusion, which is as close as you can get to accusing a fellow academic of hallucinating while remaining minimally polite. Some prominently reported attempts to replicate the Jaeggi findings failed, but others found similar positive results in schoolchildren and the elderly. In 2013, a meta-analysis based on 23 studies found "no convincing evidence of the generalisation of working memory training to other skills", though there's been debate about the selection criteria involved. An earlier British study, conducted with the BBC show Bang Goes The Theory, reached similar conclusions, but didn't focus on the same kind of game. Interviews conducted for Hurley's book show the scientific establishment to be well and truly divided. It's all "a bit of a mess", Adam Hampshire says, due to the proliferation of numerous small-scale studies, which makes false positives far more likely: "If 1,000 people roll a dice 16 times, some of them are going to get just high numbers" – and those are the studies that get published.

"I know it sounds as if we're just pouring cold water on this, but the thing is, we've been disappointed so many times before," adds James Thompson, a senior honorary lecturer in psychology at University College London, and a prominent sceptic. "About 40 years ago, it was hyperbaric oxygen for pregnant women, so they'd give birth to geniuses. I got transcranial stimulation at Guy's hospital in 1969, as a guinea pig. But then you do the hard research and you don't see much difference."

Yet it would be very strange, ultimately, if it were to prove utterly impossible to modify your brain's basic capacities through any form of training. The brain is a physical organ, and its processes are physical processes; why should the capacities we label "fluid intelligence" be uniquely immune to environmental impacts? Your intelligence is surely heavily influenced by your genes – but so (for example) is your height, and that can be affected by environmental factors, specifically how well you're nourished as a child. "Some people want to assert that it's unchangeable, as if that's hard science," Hurley says. "But it's actually a much more magical way of thinking about the mind to say that the environment can't possibly have any effect."

After four days of 20 minutes doing the dual n-back, I have no idea if it's working, but it's definitely hurting. Sadly, that's probably a good sign, and it's one thing on which researchers do tend to agree: if intelligence can be boosted by brain games – a very big if – they almost certainly won't be enjoyable ones. Unless the task involved keeps getting harder, so that you never quite feel you've got the hang of it, there's no way you'll get more intelligent. When you master a task, your brain becomes more efficient at performing it. And "efficiency is not your friend when it comes to cognitive improvement", as Andrea Kuszewski, a behavioural therapist trained in neuroscience, and a believer in the promise of intelligence-boosting, puts it. She points to studies of people playing Tetris, which showed an increase in cortical activity and cortical thickness as they struggled to get to grips with the game – but a decrease in both once they'd mastered it.

This is the closest thing you're going to get to a solid, science-backed piece of advice, when it comes to exercising your brain: don't let things get too fun. Once you're pretty good at sudoku, stop doing sudoku; switch to something you're worse at. Keep seeking challenges that make your head hurt. Nobody ever said getting smarter was going to be easy.

There could be ways to become smarter more quickly, though – so long as you're willing, like Anthony Lee, to do slightly nerve-racking things with electricity. ("I'm not afraid to experiment," Lee says: as a child, he was always the one to accept dares. "But I'm a relatively responsible adult, so if I felt there was real danger, I don't think I'd do it." Then again, he adds, with a laugh, "I really don't understand all that much about electronics.") At a research lab in New Mexico a few years ago, according to a report in Nature, volunteers wearing small wet sponges on their temples played Darwars Ambush!, a soldier-training game sponsored by the US Defense Advanced Research Projects Agency (Darpa). Darwars Ambush! involves navigating virtual landscapes reminiscent of urban war zones, learning to spot hidden gunmen or deadly explosive devices. After just a few hours' training, players who'd been receiving a 2-milliamp current through the sponges on their heads showed twice as much improvement on the game than those getting a 20th of that.

The idea of electrically stimulating human bodies goes back at least to the 19th century, when it was used to cure "melancholy"; much later, electroconvulsive therapy would be used to induce seizures in psychiatric patients. Since then, studies have demonstrated that a gentler approach, transcranial magnetic stimulation, can alleviate serious depression and perhaps even trigger bursts of "savant" intellectual prowess, reminiscent of the kind depicted in Rain Man. "How long," wondered the New York Times in 2003, "before Americans are walking around with humming antidepression helmets and math-enhancing 'hair-dryers' on their heads?"

The answer: one decade, if you count the Foc.us tDCS headset, now on sale in the US and UK. The Foc.us describes itself as an accessory for gamers, reportedly since it's easier to comply with medical regulations that way. But the implicit promise is the same as for the Darpa initiative, and Lee's home-based tinkering: by temporarily boosting cognitive capacity, tDCS might hugely speed up the learning process. It has also been shown, in one study, to induce "a feeling of anticipated challenge and [a] strong motivation to overcome it", which would presumably aid learning, too.

Precisely why tDCS works remains partly mysterious – though it's not enormously surprising that neurons, which transmit information via electrical signals, might do so faster and better with an electrical boost from outside. Dan Hurley quotes Roy Hoshi Hamilton, director of the Laboratory for Cognition and Neural Stimulation at the University of Pennsylvania: "What is a thought? A thought is what happens when some pattern of firing of neurons has happened in your brain. So if you have a technology that makes it ever so slightly easier for lots and lots of these neurons… to do their thing, then it doesn't seem so far-fetched that such a technology, be it ever so humble, would have an effect on cognition." Repeat the process enough times, and you'd expect the brain's neural pathways to change, too.

All of which is potentially dangerous, if you do it wrong. You might feel inclined to stick to brain games instead, on the rationale that even if they don't work, they can't do any harm. But that position's arguably misguided. Your time is finite, and every hour you spend wrestling with the dual n-back is one you could have spent doing any of the more mundane things that will certainly promote brain health: doing sufficient physical exercise, getting enough sleep, and preparing and eating healthy food. "Live a good clean life, get proper sleep and you'll be at the peak of whatever your potential performance is," James Thompson suggests. "And we use our intelligence to do specific tasks, so don't waste your time remembering numbers backwards – read a good statistics book. Learn about modern genetics. Read a history of intellectual discovery. Whenever people talk about spending 24 hours on the dual n-back, I think, well, yes, but what else could I do with 24 hours?"

I didn't spend 24 hours on the dual n-back, or even 12, but I did spend as long as I ever plan to, pumped up on Brain TonIQ or Brain Candy, both of which seemed to give me mild headaches. (I bought these drinks in the US, and not all are available in the UK: the Neuro range, including Neuro Passion and Neuro Sonic, has been temporarily withdrawn from British sale, because ingredients in some of the range don't have regulators' approval.)

After two weeks, I retook the intelligence test, based on the Raven matrices, and scored four points higher, at 133. Which proves absolutely nothing at all, though it did make me feel briefly smug.

I plan on never doing the dual n-back again, but I might take Andrea Kuszewski's advice and try turning off my smartphone's maps function, forcing myself to navigate the old-fashioned way. "Look, technology like GPS is great," Kuszewski says, "but there are always costs. If you used to walk to work but then you bought a car and you start driving everywhere instead, well, it'd be a lot easier. But everyone knows your body's going to suffer as a result! Why should it be any different with your brain?"

The long-sought secret of boosting intelligence could turn out to be straightforward – wherever possible, do things the harder way. I know, I know: it's not what I wanted to hear, either.