"If you have your camera up to your eye, you can't keep track of what's going on," says photographer Mark Cohen. "By holding my camera down here" – he gestures to his waist – "I can suddenly take pictures." Cohen has a peculiar style of shooting: he does it secretly, and always at hip level. Working like a sniper, he gets close, snaps low, then moves away before anyone has the chance to bristle. "There's no conversation," he says. "I'm not interested in having to explain myself. I'm just using people on the street in the most transitory way."

Cohen has been pursuing scrappy street photography in Wilkes-Barre, Pennsylvania, for 55 years, ever since he received a plastic camera at the age of 13. "I don't take my camera everywhere," he says. "I go for designated walks where I'm just taking pictures." A selection of images, culled from his years of walking around the industrial town, is now on display with Dark Knees, an exhibition at LE BAL in Paris, and available as a photobook.

Cohen likes to keep his wits about him while he walks, and finds that holding the camera low allows him to be extra watchful for antagonists. But his furtive shooting technique has always been laced with danger. One of his images is of a man, angered by the invasion of his personal space, swinging a punch at Cohen (Man Flinching, 1969). "That type of interaction took a psychological toll over the years," he says. "I made a lot of nifty pictures by being that close to people. But after a while, I went to a wider lens. 28mm. Then 35mm. Now I'm at 50mm, so I feel very safe."

But isn't getting audaciously close, almost predatory, integral to his work? "The trespass makes it happen, yes," he says. "When you're trying to make a new object, you've got to make something happen. And you learn to read people's reactions quickly." After all these years, honing in on details to find images has become automatic. "Here's a wonderful button – I love to see the buttons come out," he says, examining his silver print of a lady's coat. Or, of Seedpod in the Snow (1978), he comments that the orderly row of kernels "look like they're on a bus". The titles of the images – Wisp of Hair, Red Bow/Bare Back, Shirtless Boy with Chain – emphasise his powerful fixations.

Mark Cohen's Boy in Yellow Shirt Smoking Photograph: Mark Cohen

For years, Cohen's approach was to shoot three rolls of film over a two-hour walk, develop the rolls directly, have dinner, then go back to the darkroom, develop eight to nine prints directly from the negatives, and cast aside the rest. Cohen did this several times a week for decades. He estimates he has 600,000-800,000 images that he's never seen or developed, not even on contact sheets.

"It's something I've never encountered before," marvels Diane Dufour, the curator of the LE BAL show. "And it's something I have trouble understanding. It's almost vertiginous to think of the number of photos we could have selected just from the negatives Mark has never seen." Cohen has recently revisited some of his overlooked images. He's even compiled a dummy book of rediscovered pictures, tentatively called No Contact No Print, which is how he classifies the forsaken negatives.

The 1970s were a notable era for Cohen. His photos were showcased in an expo at MoMA in 1973 under John Szarkowski, and he regularly showed new work at galleries, though he always retreated back to Wilkes-Barre. Removing himself from the New York scene gave him a "purity", he says, by virtue of "not having a personality so involved in the dissemination of work". But by his own admission, he "dropped out" in the late 80s.

Mark Cohen's Two Black Women at Phone Photograph: Mark Cohen

"Gallerists couldn't sell my stuff," he says matter-of-factly. "My work's not the most optimistic. It's not like Yosemite." The framing is unexpected and the subjects sometimes gritty. Cohen often photographed the poorer neighbourhoods in his area because they were "more exposed": children playing outside, people lingering in the streets. "This guy's teeth are so terrible," he says, looking at the craggy, not-so-pearly-whites in Laughing Man's Teeth (1976). "This", he says,"is not right for someone's living room". Though gallery interest waned, it didn't put a dint in his productivity.

He moved to Philadelphia six months ago, and is still acclimatising to living in a metropolitan space for the first time. But he still operates in exactly the same way, going on single-minded photographic missions: "I get on a trolley and go to a specific intersection. I like to go to the same one 10 times, so I understand the texture of the neighbourhood," he says. As for Wilkes-Barre, he sees no need to dwell on it any more: "The slice of America I've been looking at is everywhere."

Mark Cohen's exhibition Dark Knees is at LE BAL until 8 December.