The man, often hailed as the greatest screen actor of his generation, is famous – some say notorious – for his obsessive attention to detail in building character. His latest role, playing Abraham Lincoln, is no exception

Thick mud and blood mingle in the opening scenes of Steven Spielberg's latest film, Lincoln. In a brutal demonstration of what happens when politics fails, bodies pile up across a boggy battlefield. The rest of the film, also full of dark and muddy tones, looks steadily at how politicians might end or prolong such a grim civil war. And at the heart of the matter, trying to abolish slavery and adorned with a representation of one of the most famous beards of all time, stands Daniel Day-Lewis.

In playing the revered 16th president of the United States, the 55-year old actor adds to the series of New World archetypes he has tackled on screen. He has moved from the fleet-footed, fictional scout Hawkeye in The Last of the Mohicans to the compromised religious settler John Proctor in The Crucible and on to the society figure of Edith Wharton's Newland Archer in The Age of Innocence. Later, he embodied the ferocity of the mobster Bill the Butcher in Gangs of New York and the obsessional frontiersman Daniel Plainview in There Will be Blood.

Lincoln opens in Britain in January, but already the film has put Day-Lewis in contention for his third Oscar, with one critic, AO Scott of the New York Times, suggesting he "eases into a role of epic difficulty as if it were a coat he had been wearing for years".

The plot begins a good while after we left Henry Fonda, the other notable Honest Abe, in John Ford's 1939 film The Young Mr Lincoln. Gone is the straightforward lawyer from a Kentucky cabin. Spielberg's mature Lincoln has just been re-elected, yet faces a Capitol in revolt. The Democratic opposition regards him as a dictator and his Republican party is riven with feuds.

The British film critic David Thomson once said that Day-Lewis "knows how to grasp an audience without noticing them" and in Lincoln the actor again delivers what could be called a total performance. He adopts an odd, light, cracked voice that has divided audiences so far, although Spielberg claims he won the role by sending him a recording of the way he felt Lincoln ought to speak. Day-Lewis also surprises by bringing out the folksy manner and homespun humour that apparently belied the intellect of the great man. Rather like TV detective Colombo in a stovepipe hat, he is full of anecdotes and shaggy dog stories.

One shaggy dog story not included in the screenplay is the old one about an actor who is so determined to play the part of Abraham Lincoln in an upcoming show that he grows a beard and walks about in costume for weeks. At the risk of spoiling the Spielberg film for newcomers to American history, the punchline inevitably involves the actor, who fails to get the part, not making it home from a night out at the theatre. It is a black joke that could well have been on the mind of Spielberg's cast as they watched Day-Lewis between scenes. He stayed in his Kentucky accent at all times and even, according to Sally Field, who plays his wife, texted her silly limericks in the character of Abe.

Each role Day-Lewis has played since his first success as a gay, fascist, punk in My Beautiful Laundrette in 1985 has been trailed with news of his extraordinary attempts to immerse himself in character. In the Oscar-winning guise of Christy Brown in My Left Foot, in 1989, Day-Lewis lived in a wheelchair and learned to paint with his toes.

As Hawkeye for Michael Mann in 1992, he studied survival techniques, skinning rabbits and hollowing out canoes. While making In the Name of the Father in 1993, he rehearsed his portrayal of the incarcerated Gerry Conlon by eating prison food and sleeping in a cell, yet in the same year he also donned Victorian garb to walk Manhattan's sidewalks in preparation for going back to the 1870s in The Age of Innocence. By the time Nicholas Hytner cast him in The Crucible the crew cannot have been surprised to hear he wanted to help them build Salem.

If a reputation for seeking ludicrous authenticity follows Day-Lewis, it is probably because his performances are so strong they demand explanation. As Thomson has pointed out: "Day-Lewis is uncanny. Watch his Newland Archer or his Hawkeye and it is not easy to believe we are seeing the same person."

When audiences learn that the actor sharpened knives between takes on Scorsese's set for Gangs of New York in 2002, it helps them understand why he dominated the screen. "Day-Lewis gives a gargantuan performance as the mad, charismatic Bill, a cross between Fagin and Bill Sikes, inevitably overshadowing DiCaprio and Diaz as the Oliver and Nancy figures," Philip French noted in this newspaper at the time.

And the legends are not confined to the film set. Before he appeared in The Crucible, there were reports he had become a nomad, wandering Europe and learning how to cobble.

He has been linked to a string of famous women, too, from Madonna and Sinead O'Connor to Winona Ryder and Julia Roberts, and is most infamously supposed to have dumped Isabelle Adjani, the French actress and mother of his first child, by fax. Then, while working on Arthur Miller's The Crucible, he became close not only to the late playwright, but to his daughter, Rebecca, and headlines swiftly ensued. Day-Lewis's live-in girlfriend, fitness trainer Deya Pichardo, told the press of her horror at the news he had married the 32-year-old painter and film-maker in a secret ceremony in Vermont.

Day-Lewis was born the son of the Irish-born British poet laureate Cecil Day-Lewis and the actress Jill Balcon, herself the daughter of the renowned film producer Sir Michael Balcon. According to his elder sister, Tamasin, a chef and film-maker, home life was not warm. "We didn't come down for dinner," she recalled last week. "We had tea in the nursery with nanny and were thrown together into a solitary world, but for each other, which led us straight into the landscape of the imagination."

Describing her parents as "emotionally distant", she added that at boarding school in Hampshire, they spent their time rebelling. "I egged Dan on and, he has told me since, was an evil influence. He insists I made him steal for me," she said. But her brother's former English teacher, the film producer David Thompson, does not remember a trouble-maker.

"He was a little wild perhaps, but he was one of those we all wanted to teach. He was staggeringly talented in school plays even at 14. He had incredible charisma," he told the Observer. "Smoking was the issue, I remember. It was a sackable offence and we had a long talk about whether he should really confess, when such a small misdemeanour would get him expelled."

After school, Day-Lewis considered taking up furniture-making, but returned to acting, to the relief of his mother. "She probably feared for me much more than she ever let on, because all I ever got from her, no matter what I was doing, was encouragement – so much so that I think I became quite a harsh judge of myself to try to restore some kind of balance," he has recalled.

At 25, he appeared in the West End in Another Country, the same play that gave Kenneth Branagh, Colin Firth and Rupert Everett their big break. Seven years later, he met his match on stage in Hamlet at the National Theatre when he collapsed during a performance in the ghost scene. The play was thought to have recalled his sorrow for his lost father, who had died 10 years earlier.

The actor has recently explained he did not see the ghost of his father on stage "that dreadful night", although he admits the content of the play was hard for him. "I probably saw my father's ghost every night, because of course if you're working in a play like Hamlet you explore everything through your own experience."

Twenty years ago, Day-Lewis wrote a piece for this newspaper about his difficult feelings for his father. He recalled the moment family members had urged him to hold his dying father's hand.

"Since that bizarre, alienated, emotionless first encounter with the great scythe, which left me reeling from my own indifference, my sense of loss has grown, soured, devoured, belched and finally purified into what is now the eternal certainty of grief, ignorance and the mystery of love," he wrote bleakly.

His sister now argues that life, even with success and his own family, gets more difficult daily for her brother as "the stakes get higher".

If so, in the weekend that Stephen Fry, the troubled star of that other infamous theatrical walkout, courageously returns to the London stage, it is impossible not to wonder whether an actor who is hailed by many as the best working in film may one day return to the scene of his Shakespearean crime.