This story is part of BBC Earth's "Best of 2016" list, our greatest hits of the year. Browse the full list.

Are you real? What about me?

These used to be questions that only philosophers worried about. Scientists just got on with figuring out how the world is, and why. But some of the current best guesses about how the world is seem to leave the question hanging over science too.

Several physicists, cosmologists and technologists are now happy to entertain the idea that we are all living inside a gigantic computer simulation, experiencing a Matrix-style virtual world that we mistakenly think is real.

Our instincts rebel, of course. It all feels too real to be a simulation. The weight of the cup in my hand, the rich aroma of the coffee it contains, the sounds all around me – how can such richness of experience be faked?

But then consider the extraordinary progress in computer and information technologies over the past few decades. Computers have given us games of uncanny realism – with autonomous characters responding to our choices – as well as virtual-reality simulators of tremendous persuasive power.

It is enough to make you paranoid.

The Matrix formulated the narrative with unprecedented clarity. In that story, humans are locked by a malignant power into a virtual world that they accept unquestioningly as "real". But the science-fiction nightmare of being trapped in a universe manufactured within our minds can be traced back further, for instance to David Cronenberg's Videodrome (1983) and Terry Gilliam's Brazil (1985).

Over all these dystopian visions, there loom two questions. How would we know? And would it matter anyway?

The idea that we live in a simulation has some high-profile advocates.

In June 2016, technology entrepreneur Elon Musk asserted that the odds are "a billion to one" against us living in "base reality".

Similarly, Google's machine-intelligence guru Ray Kurzweil has suggested that "maybe our whole universe is a science experiment of some junior high-school student in another universe".

What's more, some physicists are willing to entertain the possibility. In April 2016, several of them debated the issue at the American Museum of Natural History in New York, US.

There is nothing in principle that rules out the possibility of manufacturing a universe in an artificial Big Bang

None of these people are proposing that we are physical beings held in some gloopy vat and wired up to believe in the world around us, as in The Matrix.

Instead, there are at least two other ways that the Universe around us might not be the real one.

Cosmologist Alan Guth of the Massachusetts Institute of Technology, US has suggested that our entire Universe might be real yet still a kind of lab experiment. The idea is that our Universe was created by some super-intelligence, much as biologists breed colonies of micro-organisms.

There is nothing in principle that rules out the possibility of manufacturing a universe in an artificial Big Bang, filled with real matter and energy, says Guth.

Musk and other like-minded folk are suggesting that we are entirely simulated beings

Nor would it destroy the universe in which it was made. The new universe would create its own bubble of space-time, separate from that in which it was hatched. This bubble would quickly pinch off from the parent universe and lose contact with it.

This scenario does not then really change anything. Our Universe might have been born in some super-beings' equivalent of a test tube, but it is just as physically "real" as if it had been born "naturally".

However, there is a second scenario. It is this one that has garnered all the attention, because it seems to undermine our very concept of reality.

Musk and other like-minded folk are suggesting that we are entirely simulated beings. We could be nothing more than strings of information manipulated in some gigantic computer, like the characters in a video game.

We can simulate human societies using rather simple "agents" that make choices according to certain rules

Even our brains are simulated, and are responding to simulated sensory inputs.

In this view, there is no Matrix to "escape from". This is where we live, and is our only chance of "living" at all.

But why believe in such a baroque possibility? The argument is quite simple: we already make simulations, and with better technology it should be possible to create the ultimate one, with conscious agents that experience it as totally lifelike.

We carry out computer simulations not just in games but in research. Scientists try to simulate aspects of the world at levels ranging from the subatomic to entire societies or galaxies, even whole universes.

Who is to say that before long we will not be able to create computational agents – virtual beings – that show signs of consciousness?

For example, computer simulations of animals may tell us how they develop complex behaviours like flocking and swarming. Other simulations help us understand how planets, stars and galaxies form.

We can also simulate human societies using rather simple "agents" that make choices according to certain rules. These give us insights into how cooperation appears, how cities evolve, how road traffic and economies function, and much else.

These simulations are getting ever more complex as computer power expands. Already, some simulations of human behaviour try to build in rough descriptions of cognition. Researchers envisage a time, not far away, when these agents' decision-making will not come from simple "if…then…" rules. Instead, they will give the agents simplified models of the brain and see how they respond.

Who is to say that before long we will not be able to create computational agents – virtual beings – that show signs of consciousness? Advances in understanding and mapping the brain, as well as the vast computational resources promised by quantum computing, make this more likely by the day.

There is nothing to suggest that truly detailed simulations, in which the agents experience themselves as real and free, are impossible in principle

If we ever reach that stage, we will be running huge numbers of simulations. They will vastly outnumber the one "real" world around us.

Is it not likely, then, that some other intelligence elsewhere in the Universe has already reached that point?

If so, it makes sense for any conscious beings like ourselves to assume that we are actually in such a simulation, and not in the one world from which the virtual realities are run. The probability is just so much greater.

Philosopher Nick Bostrom of the University of Oxford in the UK has broken down this scenario into three possibilities. As he puts it, either:

(1) Intelligent civilisations never get to the stage where they can make such simulations, perhaps because they wipe themselves out first; or

(2) They get to that point, but then choose for some reason not to conduct such simulations; or

(3) We are overwhelmingly likely to be in such a simulation.

The question is which of these options seems most probable.

Astrophysicist and Nobel laureate George Smoot has argued that there is no compelling reason to believe (1) or (2).

The best way would be to search for flaws in the program

Sure, humanity is causing itself plenty of problems at the moment, what with climate change, nuclear weapons and a looming mass extinction. But these problems need not be terminal.

What's more, there is nothing to suggest that truly detailed simulations, in which the agents experience themselves as real and free, are impossible in principle. Smoot adds that, given how widespread we now know other planets to be (with another Earth-like one right on our cosmic doorstep), it would be the height of arrogance to assume that we are the most advanced intelligence in the entire Universe.

What about option (2)? Conceivably, we might desist from making such simulations for ethical reasons. Perhaps it would seem improper to create simulated beings that believe they exist and have autonomy.

But that too seems unlikely, Smoot says. After all, one key reason we conduct simulations today is to find out more about the real world. This can help us make the world better and save lives. So there are sound ethical reasons for doing it.

That seems to leave us with option (3): we are probably in a simulation.

Some scientists argue that there are already good reasons to think we are inside a simulation

But this is all just supposition. Could we find any evidence?

Many researchers believe that depends on how good the simulation is. The best way would be to search for flaws in the program, just like the glitches that betray the artificial nature of the "ordinary world" in The Matrix. For instance, we might discover inconsistencies in the laws of physics.

Alternatively, the late artificial-intelligence maven Marvin Minsky has suggested that there might be giveaway errors due to "rounding off" approximations in the computation. For example, whenever an event has several possible outcomes, their probabilities should add up to 1. If we found that they did not, that would suggest something was amiss.

Some scientists argue that there are already good reasons to think we are inside a simulation. One is the fact that our Universe looks designed.

The constants of nature, such as the strengths of the fundamental forces, have values that look fine-tuned to make life possible. Even small alterations would mean that atoms were no longer stable, or that stars could not form. Why this is so is one of the deepest mysteries in cosmology.

There are limits to the resolution with which we can observe the Universe, and if we try to study anything smaller, things just look "fuzzy"

One possible answer invokes the "multiverse". Maybe there is a plethora of universes, all created in Big Bang-type events and all with different laws of physics. By chance, some of them would be fine-tuned for life – and if we were not in such a hospitable universe, we would not ask the fine-tuning question because we would not exist.

However, parallel universes are a pretty speculative idea. So it is at least conceivable that our Universe is instead a simulation whose parameters have been fine-tuned to give interesting results, like stars, galaxies and people.

While this is possible, the reasoning does not get us anywhere. After all, presumably the "real" Universe of our creators must also be fine-tuned for them to exist. In that case, positing that we are in a simulation does not explain the fine-tuning mystery.

Others have pointed to some of the truly weird findings of modern physics as evidence that there is something amiss.

Quantum mechanics, the theory of the very small, has thrown up all sorts of odd things. For instance, both matter and energy seem to be granular. What's more, there are limits to the resolution with which we can observe the Universe, and if we try to study anything smaller, things just look "fuzzy".

Ultimately, say some physicists, reality might be nothing but mathematics

Smoot says these perplexing features of quantum physics are just what we would expect in a simulation. They are like the pixellation of a screen when you look too closely.

However, that is just a rough analogy. It is beginning to look as though the quantum graininess of nature might not be really so fundamental, but is a consequence of deeper principles about the extent to which reality is knowable.

A second argument is that the Universe appears to run on mathematical lines, just as you would expect from a computer program. Ultimately, say some physicists, reality might be nothing but mathematics.

Max Tegmark of the Massachusetts Institute of Technology argues that this is just what we would expect if the laws of physics were based on a computational algorithm.

We might just be superimposing our current preoccupations onto the laws of physics

However, that argument seems rather circular. For one thing, if some super-intelligence were running simulations of their own "real" world, they could be expected to base its physical principles on those in their own universe, just as we do. In that case, the reason our world is mathematical would not be because it runs on a computer, but because the "real" world is also that way.

Conversely, simulations would not have to be based on mathematical rules. They could be set up, for example, to work randomly. Whether that would result in any coherent outcomes is not clear, but the point is that we cannot use the apparently mathematical nature of the Universe to deduce anything about its "reality".

However, based on his own research in fundamental physics, James Gates of the University of Maryland thinks there is a more specific reason for suspecting that the laws of physics are dictated by a computer simulation.

Gates studies matter at the level of subatomic particles like quarks, the constituents of protons and neutrons in the atomic nucleus. He says the rules governing these particles' behaviour turn out to have features that resemble the codes that correct for errors in manipulating data in computers. So perhaps those rules really are computer codes?

Maybe. Or maybe interpreting these physical laws as error-correcting codes is just the latest example of the way we have always interpreted nature on the basis of our advanced technologies.

At one time Newtonian mechanics seemed to make the universe a clockwork mechanism, and more recently genetics was seen – at the dawn of the computer age – as a kind of digital code with storage and readout functions. We might just be superimposing our current preoccupations onto the laws of physics.

It is likely to be profoundly difficult if not impossible to find strong evidence that we are in a simulation. Unless the simulation was really rather error-strewn, it will be hard to design a test for which the results could not be explained in some other way.

We might never know, says Smoot, simply because our minds would not be up to the task. After all, you design your agents in a simulation to function within the rules of the game, not to subvert them. This might be a box we cannot think outside of.

The Universe can be regarded as a giant quantum computer

There is, however, a more profound reason why perhaps we should not get too worried by the idea that we are just information being manipulated in a vast computation. Because that is what some physicists think the "real" world is like anyway.

Quantum theory itself is increasingly being couched in terms of information and computation. Some physicists feel that, at its most fundamental level, nature might not be pure mathematics but pure information: bits, like the ones and zeros of computers. The influential theoretical physicist John Wheeler dubbed this notion "It From Bit".

In this view, everything that happens, from the interactions of fundamental particles upwards, is a kind of computation.

"The Universe can be regarded as a giant quantum computer," says Seth Lloyd of the Massachusetts Institute of Technology. "If one looks at the 'guts' of the Universe – the structure of matter at its smallest scale – then those guts consist of nothing more than [quantum] bits undergoing local, digital operations."

This gets to the nub of the matter. If reality is just information, then we are no more or less "real" if we are in a simulation or not. In either case, information is all we can be.

Does it make a difference if that information were programmed by nature or by super-intelligent creators? It is not obvious why it should – except that, in the latter case, presumably our creators could in principle intervene in the simulation, or even switch it off. How should we feel about that?

Tegmark, mindful of this possibility, has recommended that we had all better go out and do interesting things with our lives, just in case our simulators get bored.

I think this is said at least half in jest. After all, there are surely better reasons to want to lead interesting lives than that they might otherwise be erased. But it inadvertently betrays some of the problems with the whole concept.

If reality is just information, then we are no more or less "real" if we are in a simulation or not

The idea of super-intelligent simulators saying "Ah look, this run is a bit dull – let's stop it and start another" is comically anthropomorphic. Like Kurzweil's comment about a school project, it imagines our "creators" as fickle teenagers with Xboxes.

The discussion of Bostrom's three possibilities involves a similar kind of solipsism. It is an attempt to say something profound about the Universe by extrapolating from what humans in the 21st Century are up to. The argument boils down to: "We make computer games. I bet super-beings would too, only they'd be awesome!"

In trying to imagine what super-intelligent beings might do, or even what they would consist of, we have little choice but to start from ourselves. But that should not obscure the fact that we are then spinning webs from a thread of ignorance.

It is surely no coincidence that many advocates of the "universal simulation" idea attest to being avid science-fiction fans in their youth. This might have inspired them to imagine futures and alien intelligences, but it may also have predisposed them to cast such imaginings in human terms: to see the cosmos through the windows of the Starship Enterprise.

Perhaps mindful of such limitations, Harvard physicist Lisa Randall is puzzled by the enthusiasm some of her colleagues show for these speculations about cosmic simulation. For her they change nothing about how we should see and investigate the world. Her bafflement is not just a "so what": it is a question of what we choose to understand by "reality".

Plato wondered if what we perceive as reality is like the shadows projected onto the walls of a cave

Almost certainly, Elon Musk does not go around telling himself that the people he sees around him, and his friends and family, are just computer constructs created by streams of data entering the computational nodes that encode his own consciousness.

Partly, he does not do so because it is impossible to hold that image in our heads for any sustained length of time. But more to the point, it is because we know deep down that the only notion of reality worth having is the one we experience, and not some hypothetical world "behind" it.

There is, however, nothing new about asking what is "behind" the appearances and sensations we experience. Philosophers have been doing so for centuries.

Plato wondered if what we perceive as reality is like the shadows projected onto the walls of a cave. Immanuel Kant asserted that, while there might be some "thing in itself" that underlies the appearances we perceive, we can never know it. René Descartes accepted, in his famous one-liner "I think therefore I am", that the capacity to think is the only meaningful criterion of existence we can attest.

The philosopher George Berkeley argued that the world is merely an illusion

The concept of "the world as simulation" takes that old philosophical saw and clothes it in the garb of our latest technologies. There is no harm in that. Like many philosophical conundrums, it impels us to examine our assumptions and preconceptions.

But until you can show that drawing distinctions between what we experience and what is "real" leads to demonstrable differences in what we might observe or do, it does not change our notion of reality in a meaningful way.

In the early 1700s, the philosopher George Berkeley argued that the world is merely an illusion. Dismissing the idea, the ebullient English writer Samuel Johnson exclaimed "I refute it thus" – and kicked a stone.

Johnson did not really refute anything. But he may nevertheless have come up with the right response.

Join over five million BBC Earth fans by liking us on Facebook, or follow us on Twitter and Instagram.

If you liked this story, sign up for the weekly bbc.com features newsletter called "If You Only Read 6 Things This Week". A handpicked selection of stories from BBC Future, Earth, Culture, Capital, Travel and Autos, delivered to your inbox every Friday.