I've probably told you this story before, but a couple of weeks ago I almost destroyed my copy of DmC, Ninja Theory's reboot of the Devil May Cry series. I was stuck on a boss fight with a giant demonic baby, and although I understood what I was supposed to do, I couldn't do it. I just couldn't. I knew I had to avoid the infant's gigantic swipes, I knew I had to attack with aerial combos, I knew a weak spot would eventually open up. But I couldn't do it. So I took the disc out and threw it. It rebounded off two walls and landed behind my desk. If my window had been open, I would have tossed it into the garden.

And I thought to myself as I fumbled for the disc amid the dust and fallen Lego minifigs behind my monitor, was that fun? Am I having fun? Instantly, I recalled the many occasions in which I have abused gaming peripherals in the past, stretching waaaay back to the Commodore 64 – when I was young enough to know no better.

"Dad, the stick has come off my Quickshot II!"

"How on Earth did that happen?"

"I don't know... Can I have another one?"

But I did know. I did. I couldn't beat my friend's high score in California Games so I wrenched the stick off in a bizarre phallic attack. It took me four weeks of paper rounds to buy a new one.

From here I thought, well, is frustration part of game design or a failure of game design? Certainly, frustration has been there from the beginning. Eurogamer writer Christian Donlan once interviewed Eugene Jarvis, the creator of early and immensely difficult arcade titles like Defender and Robotron – he claimed that he would visit arcades and inspect the coin-op cabinets of his titles, feeling immense pride if they had clearly been kicked or punched. In fact, here he is talking about the twisted appeal of Robotron, and he makes a compelling case:

That's fine, you can't argue with Eugene Jarvis. But... But is it also possible that frustration is the evil, undesirable twin of compulsion? That it can only motivate the truly obsessional, the truly hardcore? It certainly exists on the tipping point between needing to continue and wanting to throw the disc at the wall. But the question is, is it on the right side of the tipping point or the wrong one?

Adam Saltsman, the brilliant designer of compulsive games such as Canabalt and Hundreds, sees frustration as a necessary element of certain titles. "I guess for me frustration is a vital part of mastery," he says. "A lot of games are kind of about mastery, one way or another, so it makes sense that this is in there. Super Meat Boy is about a very tactile, physical mastery, a kind of iterative testing and improving of your internal mental model of how the controls map to the results in the world. You have to train your nerves to react and manipulate your fingers according to that model.

"Games like Braid or English Country Tune, I think, are a bit more about mental mastery, as opposed to physical, and you kind of have to push up against a frustrating obstacle for a while in order to have that epiphany – the thing that gives you pleasure, that breakthrough in understanding. I think there is a kind of temporal frustration in grind-based dungeon crawlers too, where you really want Armor X but you have to kill a bunch of dudes to get enough money to get it. And at least within reasonable bounds, that is kind of a pleasurable thing - you're making a plan, and executing on it."

So frustration is not a universal commodity. It's okay in some games, let's say, but it's not necessarily okay in all of them. Indeed, some studios have developed clever ways to sidestep frustration. The Easy mode is the obvious one, and it has become prevalent now that games are a mass entertainment medium. Most narrative adventures will offer an option for players, 'who just want to experience the story'. However, I can't help but wonder if this is a dereliction of duty on their part – if you have produced a game with a win state, there should be a way of challenging inexperienced players without spoon-feeding them narrative sequences in between one-hit kills and dozens of lobotomised enemies.

Nintendo has a better approach. Miyamoto has mastered the art of distraction gameplay, of providing sideshows for any player nearing frustration. In the Legend of Zelda titles, for example, it's very rare that Link is utterly stuck – there are always side quests to attempt, new places to explore at different times, fresh characters to interrogate. Progress is pleasingly scattershot and non-linear. Open world games like GTA are the same; they offer flexible systems of progress and reproach; whenever they shut a door in your face, they usually also open another one just off to the side. In game design terms, side missions are the collateral damage of activity. Sometimes in Zelda, you win something by accident.

Popcap is another master of compulsion subterfuge, but it uses a different approach: it segregates its win states with multi-tiered structures of completion. You can beat a level in Peggle just by meeting the minimal requirements, but there are also combos to pull off to reach high scores and achieving that triggers lots of lovely visual and audio rewards – fireworks, rainbows, onscreen compliments. It's digital dopamine. But once you start chasing that stuff, they've got you, you're into the frustration zone.

Price and expectation can come into this equation. At GameCity last year, I interviewed designer Bennett Foddy, who makes extraordinary physics challenges like Qwop and Girp. He admitted that because his titles are ad-supported and free, he sometimes actively sought to frustrate the player to the point of their despair – it's okay, they can walk away. But he also sees frustration as a necessary motivator. Victory in video games doesn't really matter in the real world and therefore players have no need to learn to play well. But if they're frustrated and angry, they have to continue. As he explains, "players who are trying to play well are typically more engaged in the game. We're trying to build engagement, so players can have that experience — which videogames are so great at providing — of being completely immersed and involved."

It's interesting though, that he uses the term engagement, which I think is the flip side of the compulsion coin. Engagement could be read as the positive state before frustration flows in. Maybe that is the true aim of good design? According to writer and theorist Raph Koster, pacing and variation are the necessary components in a compelling experience that doesn't frustrate – the human brain enjoys learning patterns, but if it finds the process too easy or too hard, it quickly becomes bored. And boredom leads to frustration and anger. I think one of the reasons the real-time strategy genre has been so enormously successful is that it cleverly telegraphs what you need to do – unlock more powerful units – and then provides whole new strata of gameplay when you have them. The compulsion loop is perfect – both predictable and expansive.

What has become clear, however, is that frustration is rarely an accident – not to good game designers anyhow. Those guys know what they're doing. And they don't have to do it. "I list frustration along side 'power fantasy', 'compulsion', and the other commonly evoked emotions in current games," says Chris Hecker, currently working on the fascinating asymmetrical two-player shooter, Spyparty. "But, I don't think it's required for every game any more than sadness is required by every piece of literature. It's a tool, nothing more; it just happens to be one that's easy for designers to wield, so it gets used a lot.

"Of course, the key differentiator of our form is interactivity, and if you are truly empowering the player to succeed, then you're also empowering them to fail. But even failure doesn't have to be frustrating. Failure can be merely useful feedback to the player."

I put this question to Mike Bithell, the Bafta-nominated creator of platforming game, Thomas Was Alone. Platform games are often seething hives of frustration, but Thomas isn't – it has a whimsical charm that deflates those emotions. "I feel like frustration might be a side effect of compelling games," he says. "Whenever I've liked a 'frustrating' game, it's not been for the frustration, it's been for the moment of success which follows. In that sense, I guess frustration is contextually important, as the memory of a frustrating moment makes the win far more satisfying.

"But players are pretty tuned into spotting when the frustration is through bad design rather than challenging gameplay, and it's a very quick way to lose someone. I'd always aim for accessible complexity and challenge, rather than deliberately setting out to frustrate."

So frustration is a tool, but it's a dangerous one: the circular saw of game design. The trick is to use it right. It's about balance. "All the best things in life are addictive," says Foddy. "There are good, meaningful games that are pure engagement-machines, like Terry Cavanagh's Super Hexagon. The dividing line between abusing players and pleasuring them can be very vague and hard to find... just as it is in the case of junk food, sex and drugs."

So why do I think DmC is using it wrong?

Maybe it's this.

All games are systems of abstraction – they remove elements of reality in order to get to the kernel of fun. Need for Speed: Most Wanted would be insufferable if your Porsche 911 engine kept over-heating; Guitar Hero would be less entertaining if you had to spend half an hour tuning your instrument every time you played. Game designers who depict real world activities must remove the peripheral inconveniences we put up with in reality in order to streamline the experience. In games, we just want to get to the fun.

So maybe frustration is a failure of abstraction – it's when a game doesn't quite remove us from the labour of activity. I think DmC is a wonderful game but its combat system, which garners its depth from requiring players to switch quickly between two shoulder button-based modifiers, leads to physiological frustration: I can't get my fingers in the right places fast enough. Compare this to Street Fighter or Virtua Fighter: these games have very smooth button progression systems, designed to ease players through the input. Very good fighting game players are often described as 'piano-ing' the buttons – the challenge is in memorising and timing the moves, and understanding opponents, not in physically carrying out the sequences. That part should be instinctive. It isn't in DmC.

The other possibility, of course, is that I could just be crap at it. I'm prepared to accept that... but then I wasn't crap at Bayonetta! Or God of War! Argh!

Later that night, I wiped the disc and put it back into the Xbox. It was working, which was a relief, but also not a relief. Because I knew what I was heading toward: frustration at the chubby hands of a colossal demon baby. Whatever you think about game design, that is never pretty.