In a roll call of the pioneers of modern photography, one name is never invoked. From the late 1920s to the 1950s, William Mortensen was one of the most famous and celebrated photographers in America. However, his subject matter – which veered towards the savage, indecorous, gothic and grotesque – as well as his use of montage and illustration, made him a pariah among the puritanical new guard in photography, led by Ansel Adams, who tried to write him out of history.

For decades, Mortensen’s work and ideas were kept alive by a small coterie of adherents. Only in recent years has he been recognised as a unique and innovative visual stylist, leading to the publication this month of American Grotesque, the first major survey of Mortensen’s work and career, which will be marked by retrospectives in New York, Los Angeles and Seattle.

Mortensen was the last of the great pictorialist photographers, the movement that dominated early 20th-century photography. Working in Hollywood, he shot many of the leading stars of his day: Rudolph Valentino, Lon Chaney, Fay Wray, Jean Harlow, Clara Bow and Peter Lorre all submitted themselves to the gaze of his lens. At the height of his fame, his images were published in Vanity Fair, he had a weekly photography column in the Los Angeles Times, and he wrote a series of bestselling instructional books.

While much pictorialist photography drew on Romanticism, especially the landscape and portrait paintings of Caspar David Friedrich and John William Waterhouse, Mortensen disdained this, seeing it as belonging to the school of “fuzzy-wuzzy” photography. His work was more in tune with the popular taste, unashamedly exploring primal images of sex and violence that took their cue from the Universal monster movies that reigned supreme in the late 1920s and 30s, and the dark, nightmarish German expressionist style that inspired them. L’Amour, one of Mortensen’s most iconic images, features a semi-naked woman lying on the ground, possibly dead, as a monstrous ape leers over her with a club, clearly a reference to King Kong, which had been released two years earlier in 1933.

He also produced explicit images of torture and death, long before the public was exposed to that kind of material daily through war photography and televised images of conflict. The Glory of War, from 1927, depicted a young woman lying among rubble and detritus, dirtied and bloodied, seemingly felled by a wooden beam, clutching a crucifix to her breast. Religious persecution, especially crucifixion, was a repeated theme, usually featuring naked women, chained, shackled and tormented by hooded figures. Historical figures, such as Machiavelli and Paganini, were also favourites, often personified by Hollywood stars; in one such image, Peter Lorre played Napoleon.

Mortensen’s methods often made it hard to distinguish whether the results were photographs or not. He used traditional printmaking techniques, such as bromoiling, and developed many of his own. He would create composite images, scratch, scrape and draw on his prints, then apply a texture that made them look like etchings, thereby disguising his manipulations. Consequently, every print was unique. Ultimately, Mortensen’s aim was to create something that, for all intents and purposes, appeared to be a photograph, yet portrayed scenes so fantastic they caused wonder and astonishment in the viewer.

L’Amour (1935). Photograph: William Mortensen

A native of Park City, Utah, Mortensen had trained in various artistic disciplines, including painting, illustration and etching, both there and at the Art Students League of New York, before taking off for the west coast in 1922 at the age of 25. In LA, he worked with director Ferdinand Pinney Earle, painting matte backgrounds for The Rubaiyat of Omar Khayyam, a 1922 silent epic starring Ramon Navarro. He also persuaded Earle to hire 14-year-old Fay Wray for the film, her first screen appearance.

Mortensen had brought Wray, the younger sister of his then fiancée, with him to Los Angeles, acted as her chaperone and guardian, and introduced her to people he thought could further her career. He also took photographs of Wray that she would later credit as being influential in the development of her own self-image. “There was a person in those pictures I hadn’t seen before,” she wrote in her 1989 memoir, “even if I had suspected she could be there.” Her mother thought otherwise and – suspecting an improper relationship between the two, which Wray denied – angrily confronted Mortensen and destroyed, in his presence, all the glass negatives he had taken of her daughter. They were separated, but Wray would continue to consider Mortensen her most significant mentor.

Through Earle, Mortensen was introduced to Cecil B DeMille, with whom he would work as a set decorator, costumier and set photographer. DeMille, an enthusiastic patron of Mortensen’s work, privately published a monograph of his photographs from the set of The King of Kings. Mortensen also made batik, costume jewellery and papiermâché masks. He produced a number of alien-looking tribal masks for the Tod Browning movie West of Zanzibar, which starred Lon Chaney as a paraplegic magician living in an African village who takes to dressing as a god to maintain control over the tribe. Mortensen learned much from Chaney’s extraordinary use of special-effects makeup. He was also employed as a photographer by the Western Costume Company, who supplied clothing and props to all major Hollywood productions. In his 11th floor studio, Mortensen shot costume tests and it is this that brought him into contact with many stars. All the techniques he had learnt – matte painting, costuming, props, special effects makeup – fed into his working methods for making a picture.

Despite the subject matter he was drawn to, Mortensen was far from a vulgarian preoccupied with outré material. He was remarkably learned and inquisitive, not just about art and culture, but science and religion. Fay Wray recalled that, during those early days, he possessed a leather-bound edition of Plutarch’s Lives and asked her to read from it. His intellectual curiosity led to a friendship with Manly P Hall, a chiselled Canadian philosopher and mystic living in LA. Through studying Hall’s voluminous library of books about the occult, Mortensen was inspired to begin a series of remarkable images (over a hundred and fifty in number), numbering over 150, intended for a pictorial history of witchcraft. With its witches flying over rooftops and its portraits of demons, succubi, gargoyles and vampires, it owed much to Goya and Bosch.

William Mortensen self-portrait in the guise of the Mad Hatter, c.1928. Photograph: William Mortensen/PR

In his own writings, Mortensen invoked Hogarth, Beardsley, Daumier and Goya as his forebears. But he also had much in common – in technique, style and approach – with European outlier artists of the dadaist and surrealist movements. In a series of articles for Camera Craft in 1934, Mortensen categorically rejected the use of realism for its own sake, calling it a “blind alley”. Photographs, he said, should be more than just objects of aesthetic beauty to be admired; they should have an effect on the viewer, exploring extreme emotions and inspiring extreme reactions.

His love of the fantastic and the grotesque was, then, partly an outward expression of his love to shock, but it had another purpose: by giving form to such emotions as fear and hatred, Mortensen, a Christian Scientist, believed “we are enabled to lessen their power over us”. He added: “When the world of the grotesque is known and appreciated, the real world becomes vastly more significant.”

It was these kinds of ideas that so angered Adams and his Group f/64 brethren, who published their own manifesto in Camera Craft that same year, devoted to photography that depicted a pure, unmediated reality. This began a spirited debate with Mortensen within the pages of the magazine that became ever more vitriolic. However, Adams did not stop there, suggesting in a personal letter to Mortensen that he “negotiate oblivion”. When fellow photographer Edward Weston wrote telling of his excitement at photographing a “fresh corpse”, Adams replied: “My only regret is that the identity of said corpse is not our Laguna Beach colleague.”

The critics Beaumont Newhall and his wife Nancy held the same view: Beaumont consciously excluded Mortensen from his grandiosely titled 1949 book The History of Photography, From 1839 to the Present Day. Their distaste would not even allow them to acknowledge Mortensen’s mastery of his craft, although Adams would later concede he was “embarrassed” to find out Mortensen had “anticipated some of my pet ideas of technique: controlled exposure and development of the negative”. The critic AJ Coleman even contends that Adams’s “zone system” for exposure was taken largely from articles written by John L Davenport, who himself relied greatly on Mortensen.

Even after Mortensen’s death in 1965 from leukaemia, Group f/64 and their flunkies the Newhalls could not stop talking of their loathing for him. Beaumont described his work as “perverse”; Willard Van Dyke, a founder of Group f/64, said “his work was disgusting”; and Adams summed him up with the words, “For us, he was the antichrist.”

Ultimately though, for all the griping of Adams and f/64, it turns out that Mortensen was the true modernist all along, not them. For today, we are surrounded by images of the fantastic and unreal. In comic-book movies such as Spider-Man and Rise of the Planet of the Apes, special effects merge seamlessly into the action and the monsters appear as real as humans. A photograph is rarely just a photograph these days, seen without filters or retouching. And, thanks to sites like Instagram, many of Mortensen’s painstaking techniques can now be applied with the touch of a button.

• American Grotesque, edited by Larry Lytle and Michael Moynihan, will be published this month by Feral House; it has also republished William Mortensen and George Dunham’s The Command to Look. Read more at whmortensen.com