In January 1814, Jane Austen sat down to write a revolutionary novel. Emma, the book she composed over the next year, was to change the shape of what is possible in fiction. Perhaps it seems odd to call Austen “revolutionary” – certainly few of the other great pioneers in the history of the English novel have thought so. From Charlotte Brontë, who found only “neat borders” and elegant confinement in her fiction, to DH Lawrence, who called her “English in the bad, mean, snobbish sense of the word”, many thought her limited to the small world and small concerns of her characters. Some of the great modernists were perplexed. “What is all this about Jane Austen?” Joseph Conrad asked HG Wells. “What is there in her? What is it all about?” “I dislike Jane … Could never see anything in Pride and Prejudice,” Vladimir Nabokov told the critic Edmund Wilson.

Austen left behind no artistic manifesto, no account of her narrative methods beyond a few playful remarks in letters to her niece, Anna. This has made it easy for novelists and critics to follow Henry James’s idea of her as “instinctive and charming”. “For signal examples of what composition, distribution, arrangement can do, of how they intensify the life of a work of art, we have to go elsewhere.” She hardly knew what she was doing, so, implicitly, the innovative novelist like James has nothing to learn from her.

There have been scattered exceptions. The year after he published More Pricks Than Kicks, the young Samuel Beckett told his friend Thomas McGreevy, “Now I am reading the divine Jane. I think she has much to teach me.” (One looks forward to the scholarly tome on the influence of Jane Austen on Samuel Beckett.) Contemporary novelists have been readier to acknowledge her genius and influence. Janeites felt a frisson of satisfaction to see that the most formally ingenious British postmodern novel of recent years, Ian McEwan’s Atonement, opens with a lengthy epigraph from Northanger Abbey. McEwan alerts the reader to the fact that his own novel learns its tricks – about a character who turns fictional imaginings into disastrous fact – from the genteel and supposedly conservative Austen.

Emma, published 200 years ago this month, was revolutionary not because of its subject matter: Austen’s jesting description to Anna of the perfect subject for a novel – “Three or four families in a country village” – fits it well. It was certainly not revolutionary because of any intellectual or political content. But it was revolutionary in its form and technique. Its heroine is a self-deluded young woman with the leisure and power to meddle in the lives of her neighbours. The narrative was radically experimental because it was designed to share her delusions. The novel bent narration through the distorting lens of its protagonist’s mind. Though little noticed by most of the pioneers of fiction for the next century and more, it belongs with the great experimental novels of Flaubert or Joyce or Woolf. Woolf wrote that if Austen had lived longer and written more, “She would have been the forerunner of Henry James and of Proust”. In Emma, she is.

Saoirse Ronan and James McAvoy in Atonement (2007) Photograph: Ronald Grant Archive

To measure the audacity of the book, take a simple sentence that no novelist before her could have written. Our privileged heroine has befriended a sweet, open, deeply naive girl of 17 called Harriet Smith. It is a wholly unequal relationship: Emma is the richest and cleverest woman in Highbury; Harriet is the “natural daughter of someone”, left as a permanent resident of the genteel girls’ boarding school in the town. While cultivating their relationship, Emma knows very well that Harriet is her inferior. “But in every respect as she saw more of her, she was confirmed in all her kind designs.”

The sentence is in the third person, yet we are not exactly being told something by the author. “Kind designs” is Emma’s complacent judgment of herself. Even the rhyme in the phrase makes it sound better to herself. In fact, the kindness is all in the mind of the beholder. Emma has set out to mould Harriet. Emma’s former companion, Miss Taylor, has got married and become Mrs Weston, leaving her solitary and at a loose end. Harriet will be her project. Her plans are kind, she tells herself, because she will improve this uninstructed and wide-eyed young woman. We should be able to hear, however, that her designs are utterly self-serving. Soon she is persuading Harriet to refuse a marriage proposal from a farmer who loves her, and beguiling her with the wholly illusory prospect of marriage to the smooth young vicar, Mr Elton.

Take another little sentence from much later in the novel. By now Emma is convinced that Harriet, scorned by Mr Elton, can be paired off with the highly eligible Frank Churchill. The only impediment seems to be the inflexible Mrs Churchill, Frank’s adoptive mother, who expects him to find a much grander wife. Then news arrives of Mrs Churchill’s sudden death. Emma meets Harriet, who has also heard. “Harriet behaved extremely well on the occasion, with great self-command.” Obviously she is learning self-possession from her patron. “Emma was gratified to observe such a proof in her of strengthened character.”

Except that this is all twaddle. Harriet does not give a fig for Frank and never has. Emma has elaborately deluded herself again. The narration follows the path of Emma’s errors. Indeed, the first-time reader will sometimes follow this path too, and then share the heroine’s surprise when the truth rushes upon her. Yet it is still a third-person narrative; Emma is not telling her own story. We both share her judgments and watch her making them.

Austen was the first novelist to manage this alchemy. She was perfecting a technique that she had begun developing in her first published novel, Sense and Sensibility. It was only in the early 20th century that critics began agreeing on a name for it: free indirect style (a translation from the original French: style indirect libre). It describes the way in which a writer imbues a third-person narration with the habits of thought or expression of a fictional character. Before Austen, novelists chose between first-person narrative (letting us into the mind of a character, but limiting us to his or her understanding) and third-person narrative (allowing us a God-like view of all the characters, but making them pieces in an authorial game). Austen miraculously combined the internal and the external.

Scholars have raked through the fiction of predecessors and contemporaries such as Fanny Burney and Maria Edgeworth, and found a few flickerings of this technique, but nothing more. In our own time, novelists use it almost as second nature, without necessarily giving it a name or thinking that they have learned it from somewhere. Yet, though its pioneer, Austen used it with an assurance that has never been surpassed. David Lodge has observed how odd James’s condescension is, given that he came to specialise in the very technique Austen had pioneered: “Telling the story through the consciousness of characters whose understanding of events is partial, mistaken, deceived, or self-deceived.” It has been easy for sophisticated readers – especially rival novelists – to miss her sophistication.

Romola Garai as Emma in the BBC’s 2009 adaptation. Photograph: BBC

By the time that she began writing Emma, Austen was no longer responding to other novelists, she was in new territory, in dialogue with her own earlier novels. She had been steeped in the fiction of the 18th and early 19th centuries, and in her earliest work she wrote against the novels of sensibility or the gothic fiction that she knew so well. But in the creative furore that saw her complete her last four novels in five years, she left the conventions of existing fiction behind. She began work on Emma before she had even received the proofs of Mansfield Park. That novel’s heroine, Fanny Price, was reticent, self-abnegating, powerless and often silent or absent. As if in response to her own experiment, she now created a heroine who is assertive, dominant, all too powerful. Emma Woodhouse thrusts herself forward in the novel’s title and its very first sentence.

Her viewpoint is so dominant that it takes several readings before you realise how subtly we are invited to imagine how Emma looks to some of the other characters. How Mr Elton imagines that she is egging him on to propose to her (“I think your manners to him encouraging”, warns Mr John Knightley, to no avail). How Jane Fairfax dreads her inquisitiveness and hates her monopolising of Frank Churchill. How the Martin family must regard her as the heartless snob who has torn Harriet away from the man who loves her. All this is intimated through Emma’s own glimmerings of insight – which she duly crushes. Austen’s narrative enacts her heroine’s victories over her own better self.

There is, however, one carefully calculated chapter in the whole novel narrated from another character’s viewpoint. Deep in the third volume, Austen jolts the reader with a chapter from Mr Knightley’s point of view. It comes at a crucial point, where Frank uncharacteristically blunders by mentioning an item of parochial gossip that he can only know from his secret correspondence with Jane: Mr Perry the apothecary is getting a carriage (because he is making so much money from the maladies imaginaries of Highbury). How could he know? “It must have been a dream,” laughs Frank. Emma is “out of hearing”, but Mr Knightley is observing. He watches as all the major characters sit down to play a word game (the novel is full of games and puzzles) and Frank selects the letters for the word “blunder”. Mr Knightley sees and suspects. “Disingenuousness and double-dealing seemed to meet him at every turn.” The spell of Emma’s consciousness has been so powerful that Austen has to wake us up for a moment. But the chapter ends with Mr Knightley suggesting to Emma that there might be some intimate “degree of acquaintance” between Frank and Jane – only to have his suspicions routed by her. “There is no admiration between them, I do assure you.” No one can say she was not given the chance to see the truth.

By the time she began Emma, Austen was in new territory. She had left existing fiction behind

Austen has several different ways of getting us to read through Emma. At key moments, free indirect style becomes something closer to dramatised thought. Austen develops her own system of punctuation for this. Here is our heroine, back home after the Westons’ Christmas Eve dinner party, reflecting on Mr Elton’s marriage proposal (“actually making violent love to her”) in the carriage home. She had persuaded herself that he was amorously interested in Harriet; worse, she had persuaded Harriet of this too.

The hair was curled, and the maid sent away, and Emma sat down to think and be miserable.—It was a wretched business indeed!—Such an overthrow of every thing she had been wishing for!—Such a development of every thing most unwelcome!—Such a blow for Harriet!—that was the worst of all. Every part of it brought pain and humiliation, of some sort or other; but, compared with the evil to Harriet, all was light; and she would gladly have submitted to feel yet more mistaken—more in error—more disgraced by mis-judgment, than she actually was, could the effects of her blunders have been confined to herself.

Austen’s idiosyncratic punctuation, that system of exclamation marks and dashes, allows for a kind of dramatised thought process. Yet because it is still in the third person, we can judge Emma even as we share her thoughts. She is a person worth our sympathy because she is capable of acknowledging and feeling sorry for her mistakes. But, by the unprecedented subtlety of Austen’s narrative technique, we sense that Emma regrets the scotching of her plans (“Such an overthrow of everything she had been wishing for!”) as much as (or more than?) the impending pain for Harriet. We can even hear her trying to persuade herself (“she would gladly have submitted … ”) of her unselfishness.

Alan Cumming (Mr Elton), Juliet Stevenson (Mrs Elton), Polly Walker (Jane Fairfax) and Ewan McGregor (Frank Churchill) in the 1996 film, which starred Gwyneth Paltrow as Emma. Photograph: Allstar/Cinetext/Miramax

The novel’s stylistic innovations allow it to explore not just a character’s feelings, but, comically, her deep ignorance of her own feelings. Out of vanity, encouraged by the promptings of Mr and Mrs Weston, Emma has persuaded herself that Frank, whom she has never met, might be the perfect partner for her. When he finally turns up he proves handsome and humorous and intelligent. Understandably, she soon starts seeing the signs that he must be falling for her; better still, she also starts convincing herself that “she must be a little in love with him”. A few amusing confidences shared with smooth Frank Churchill, and she presumes it is the real thing. “Emma continued to entertain no doubt of her being in love.” Her capacity for self-congratulation deceives her about even the workings of her own heart. Austen does not tell us this, as George Eliot would eloquently tell us: she simply lets us inhabit Emma’s consciousness, simply lets us see the world according to Emma.

Even better is her self-deception about the man whom she does love. When Mrs Weston suggests that Mr Knightley’s evident admiration of Jane presages their likely marriage, the narrative tells us of Emma’s response, but also stages her self-deception.

She could see nothing but evil in it. It would be a great disappointment to Mr. John Knightley; consequently to Isabella. A real injury to the children—a most mortifying change, and material loss to them all;—a very great deduction from her father’s daily comfort—and, as to herself, she could not at all endure the idea of Jane Fairfax at Donwell Abbey. A Mrs. Knightley for them all to give way to!—No—Mr. Knightley must never marry. Little Henry must remain the heir of Donwell.

How natural, then, that when our heroine does realise what love is, it is as a nasty shock. Her erstwhile puppet – now her Frankenstein’s monster – Harriet reveals that she (no longer quite so modest) has her heart set on Mr Knightley and has good reason to think that he returns her affection. Why is the idea of Harriet marrying Mr Knightley so unacceptable? “It darted through her, with the speed of an arrow, that Mr. Knightley must marry no one but herself!” What a brilliant sentence that is! With absolute daring, Austen shows us that love can be a discovery of what a person has unknowingly felt for many a long month or year. Now, suddenly and for the first time, Emma understands the plot of her own story. But even at this moment of self-knowledge Austen lets us hear or feel the character’s imperiousness, her overpowering sense that events “must” meet her desires.

Which is why those who condemn the novel by saying that its heroine is a snob miss the point. Of course she is. But Austen, with a refusal of moralism worthy of Flaubert, abandons her protagonist to her snobbery and confidently risks inciting foolish readers to think that the author must be a snob too. Emma’s snobbery pervades the novel, from that moment when we hear Mrs Goddard, the mistress of the little girls’ boarding school, and Mrs and Miss Bates described as “the most come-at-able” denizens of Highbury (meaning that they are at the beck and call of Emma and her hypochondriac father). Austen has the integrity to make Emma snobbish even when she is in the right. When Mr Elton proposes to her she recognises what the reader has always known: he is vain, cold-hearted and repulsive. But her enlightenment is also affronted dignity:

Those who condemn the novel by saying that its heroine is a snob miss the point. Of course she is

But—that he should talk of encouragement, should consider her as aware of his views, accepting his attentions, meaning (in short), to marry him!—should suppose himself her equal in connexion or mind!—look down upon her friend, so well understanding the gradations of rank below him, and be so blind to what rose above, as to fancy himself shewing no presumption in addressing her!—It was most provoking.

Similarly, her run-ins with Mrs Elton, some of the best comic dialogues in all fiction, show her to be perceptive and socially arrogant in equal measure. Mrs Elton, newly arrived in Highbury, visits Emma and talks of her introduction to Mr Knightley.

“I must do my caro sposo the justice to say that he need not be ashamed of his friend. Knightley is quite the gentleman. I like him very much. Decidedly, I think, a very gentleman-like man.”

Only when Mrs Elton leaves can Emma “breathe” her indignation.

“A little upstart, vulgar being, with her Mr. E., and her caro sposo, and her resources, and all her airs of pert pretension and underbred finery. Actually to discover that Mr. Knightley is a gentleman! I doubt whether he will return the compliment, and discover her to be a lady. I could not have believed it! And to propose that she and I should unite to form a musical club!”

Emma is right – and yet Emma too is full of herself. She even, unconsciously, uses the same vocabulary as her foe, who assures her, “I have quite a horror of upstarts”.

The magnificently ghastly Mrs Elton makes herself known through her voice and, in Emma, Austen discovers new and unprecedented ways of making a human voice live in print. Some of her techniques foresee the ingenuities of modernism. When Mrs Elton picks strawberries at Mr Knightley’s party at Donwell Abbey, a paragraph of fractured monologue brilliantly dramatises what must be at least half an hour’s worth of bossy babble.

“The best fruit in England—everybody’s favourite—always wholesome. These the finest beds and finest sorts.—Delightful to gather for one’s self—the only way of really enjoying them. Morning decidedly the best time—never tired—every sort good—hautboy infinitely superior—no comparison—the others hardly eatable—hautboys very scarce—Chili preferred—white wood finest flavour of all—price of strawberries in London—abundance about Bristol—Maple Grove—cultivation—beds when to be renewed—gardeners thinking exactly different—no general rule—gardeners never to be put out of their way—delicious fruit—only too rich to be eaten much of—inferior to cherries—currants more refreshing—only objection to gathering strawberries the stooping—glaring sun—tired to death—could bear it no longer—must go and sit in the shade.”

A ludicrous progress from know-all enthusiasm to sun-struck exhaustion. For garrulous Miss Bates, Highbury’s good-hearted resident bore, Austen invents a different kind of monologic outpouring that some have called Joycean. Here is just a little sample, as Miss Bates arrives for the ball at the Crown Inn.

“Thank you, my mother is remarkably well. Gone to Mr. Woodhouse’s. I made her take her shawl—for the evenings are not warm—her large new shawl— Mrs. Dixon’s wedding-present.—So kind of her to think of my mother! Bought at Weymouth, you know—Mr. Dixon’s choice. There were three others, Jane says, which they hesitated about some time. Colonel Campbell rather preferred an olive. My dear Jane, are you sure you did not wet your feet?—It was but a drop or two, but I am so afraid:—but Mr. Frank Churchill was so extremely—and there was a mat to step upon—I shall never forget his extreme politeness.”

And so on. There are other people here, not just listening but speaking, or trying to speak. And yet Miss Bates’s voice, self-generating and unstoppable, becomes for a while the only one you can hear.

Emma hardly listens to her “prosing”, and there have been readers who have likewise skipped the details of her speech. But one of Austen’s tricks is to embed many a clue as to the real ruses of other characters in the unsuspicious outpourings of this much-ignored old maid. “What is before me, I see,” she says, typically declaring herself incapable of perceiving what is indirect or implicit. But what she says is truer than what anyone hears: she is the reliable witness to what is really going on. Even that passage above offers clues as to what Frank is really up to. If this is a detective story, then Miss Bates is the foolish bit-part player offering the apparently trivial testimony that is dangerously ignored.

Phyllida Law (Mrs Bates) and Sophie Thompson (Miss Bates) in the 1996 film. Photograph: Rex/Moviestore Collection

Frank is, of course, conducting a covert romance with Jane, Miss Bates’s orphan niece, but he is so clever that it is easy to miss his tricks. Sharing Emma’s perspective, we sometimes get fooled too. Perhaps on a second reading of the novel we are properly suspicious of Frank’s motives in volunteering to mend the rivet in Mrs Bates’s spectacles: Jane is staying at the Bates’s tiny flat and he is always finding excuses to visit. But it will probably take more than two readings for most readers to notice how he has managed to get Miss Bates out of the flat – she bustles over the street to invite Emma in – so that he is alone with Jane and the sleeping (and stone deaf) old Mrs Bates. When we enter the front door with Emma we see through her eyes.

The appearance of the little sitting-room as they entered, was tranquillity itself; Mrs. Bates, deprived of her usual employment, slumbering on one side of the fire, Frank Churchill, at a table near her, most deedily occupied about her spectacles, and Jane, standing with her back to them, intent on her pianoforte.

Emma sees nothing untoward – but what has really been going on? Why is Frank so “deedily occupied” and Jane “intent” on a musical instrument? Surely they have been in a close embrace. It is as if there is a Charlotte Brontë story going on under Emma’s nose.

“The Passions are perfectly unknown to her,” Brontë declared, sounding like a character whom Austen would have delighted in depicting. She had been recommended Pride and Prejudice by George Eliot’s partner, George Henry Lewes, who was partly responsible for Eliot holding Austen in higher regard than most of the other great novelists of the 19th century. Lewes’s 1859 essay in Blackwood’s Magazine is still one of the most perceptive analyses of Austen’s powers.

But instead of description, the common and easy resource of novelists, she has the rare and difficult art of dramatic presentation: instead of telling us what her characters are, and what they feel, she presents the people, and they reveal themselves. In this she has never perhaps been surpassed, not even by Shakespeare himself.

Yet Lewes was rare among serious writers in giving her this status. But then, with Emma, Austen almost seems to be tempting inattentive readers to overlook her technical audacity – to miss her tricks.

None of Austen’s novels is as full of tricks as Emma, and many of them are carefully concealed to reward the rereader. I remember the moment, after many readings over the years, when I finally saw what she was doing with Mr Perry, the apothecary. Everyone is always quoting him, especially Emma’s valetudinarian father Mr Woodhouse: “as Perry says …”; “… This is just what Perry said”; “Ah! my dear, but Perry had many doubts about the sea doing her any good.” Mr Perry is always being spotted passing by (all those lucrative house calls) and his views are always being reported. Yet not a single word that he ever says is actually given us in the novel. Of course not! He is the echo to every person’s existing prejudices; no wonder he is so successful. It is a joke buried by Austen for posterity to discover. As she told her sister Cassandra, she only wrote for those who had “a great deal of ingenuity themselves”.

• What Matters in Jane Austen?: Twenty Crucial Puzzles Solved by John Mullan is published by Bloomsbury.