In late October the UK defence secretary, Philip Hammond, indicated that hackers with a criminal past could be recruited to serve in a new military cyber force. While the news has surprised many, it actually points to a significant recent shift where governments and businesses are looking to use hackers and their associated skills to improve security procedures, fight crime and even boost competitive advantage.

Today, look into the heart of every truly successful information security company and you will find hackers. Dig in to most great technical evolutions and you will find hackers. Hackers are now highly cherished commodities throughout the mainstream.

But why has it taken so long for businesses and governments to realise this untapped resource? The simple answer is a combination of negative PR and a fundamental lack of understanding about what motivates a hacker – if you can't understand their motivation, how can you trust them?

The history of the hacker

When I started hacking almost three decades ago, the term hacker was already several decades old. Born within MIT's Tech Model Railway Club in the 1960s, a hacker was a technical wizard who loved to explore electronic systems and lived by a strict ethical code that forbade them from harming the systems they explored.

However, these original hackers were slowly being replaced by a younger, more reckless breed of electronic warrior. Formerly known as crackers, the "new hacker" lived to break security and while some still believed in the ethical principles of their predecessors, many only cared about showing they were better – and more elite – than their peers.

Ultimately, hackers separated into two major subgroups: 'whitehats', who lived by an ethical code much like their predecessors; and the 'blackhats', who simply lived to hack systems.

The condemnation of the hacker

During this period – the late 80s and through the 90s – legitimate whitehat hackers found financial or employment opportunities hard to come by. Some blackhat hackers looked to financially motivated crime; they were prolific at pirating software, stealing telephony access, and credit card fraud.

Unfortunately, it was only stories of the blackhats that caught people's attention. Hackers hit the big screen as skateboarding, bank hacking, missile launching delinquents who threatened the very fabric of society. As far as the public (and ultimately the authorities) were concerned, hackers were a frightening new breed of criminal capable of almost anything. It's little wonder that in this climate, being labelled a "hacker" made otherwise extremely bright young people almost completely unemployable.

It became so bad that when companies identified a candidate with unparalleled expertise in information technology but no qualifications or higher education to justify those skills, suspicions arose and therefore that candidate was perceived as unsuitable for any role. Consequently most hackers either took basic IT roles or sought employment in completely different professions, determined to earn a salary during the day and hack by night.

The outliers move in

However, a small group of hackers found employment in roles that demonstrated their true value, changing the mainstream view of what a hacker could be. This minority group blazed a path that changed the world around them.

Unlike the UK government, which has maintained a strong separation from hackers and hacker groups, the US government had early exposure to hackers when a hacker group known as the L0pht testified in front of the senate on 19 May, 1998. This is arguably one of the reasons why the US government is so advanced in the integration of hackers into cyber defence and other critical parts of the country's military programme.

This integration and acceptance within the US has been painfully slow at times, but it opened doors and allowed for a different, more positive image of hackers to enter the public conscious.

I remember thinking, when my good friend and fellow hacker Jeff Moss was sworn into the US Homeland Security Advisory Council a few years ago, just how far we've come as a group.

As the world at large came to understand hackers and what motivates them in a different light, everything began to change. This brave new world offers many possibilities. Most importantly it allows for industry and government to recognise that sometimes kids will be kids, and that when they grow up these fantastically skilled but now mature individuals could play a pivotal role in shaping our cyber future. Indeed, schools, governments and industry must look to provide a more productive outlet for this group's amazing skill set.

Today the word "hacker" is returning to its original use; that of a technical genius who likes to explore the technical world and reshape it to his or her desires in a non-destructive way.

No longer unemployable, today's hackers are in high demand. Law enforcement sees hackers as valuable assets in fighting today's sophisticated cyber-criminals; companies see hackers as the front-line against complex cyber threats; and nation states see hackers both as a valuable resource and as a marketplace when it comes to building, and defending against, advanced cyber weapons.

Above all, as a UK citizen myself, I find it heartening that in this new world the UK government is finally remembering that sometimes poachers make the best gamekeepers.

Marc Rogers is principal security researcher for Lookout

Get more articles like this sent direct to your inbox by signing up for free membership to the Guardian Media Network – this content is brought to you by Guardian Professional.