"It's the buzz," Paul Smith says. "When you capture somebody else's bird, you get a buzz. That's the whole point." Smith is a dooman. "Doo" is a Scottish word for male pigeon, and doomen keep pigeons.

But that's only the half of it. Doomen also keep other people's pigeons. The pigeons are a special type, known as horseman thief pouters. They look quite different from your average feral pigeon – prettier, prouder, more erect, as if standing to attention at a passing-out parade. They are often bleached peroxide-blond and puff their chests out like Orson Welles in his heyday.

Male and female pouters are hot, and they know it – they attract other pouters back to their hut, and that's where the doomen come in. As soon as their pigeon lands on the roof with its prospective new partner, the dooman whips down the net and bags the bird. The bird is then legitimately theirs and is often sold on. It's kidnapping as competition. Traditional pigeon fanciers often regard doomen as pirates.

Flying doos is a working-class sport in Scotland. Smith lives in Muirhouse, a part of Edinburgh far removed from the cosy intellectualism of the festival. Doomen tend to live tough, hard-drinking, sometimes lawless lives. While flying doos does not necessarily keep them on the straight and narrow, it does keep them on a path that is a bit straighter and narrower than it might have been.

It's 9am on a hot Sunday in August, and Smith is hungover. He's looking towards the sky and it's hurting his eyes. He was on the beer last night, he says, as he pours us coffee.

"My earliest memory is of my grandad flying them 40 years ago. He and my dad used to take me to the wee pet shop, and the guy used to wear a long white coat, like a doctor. Back then, you could buy pigeons for 50p. That's how I got hooked." The most he's paid for a pigeon is £75.

Smith says human skill plays a part in his sport, but most of it is down to the bird. "There are birds who will catch a hen when they meet it, nae trouble, and there are other birds I've had for five years who have never caught a single pigeon."

He opens the hut and shows me his favourite, a Dutch bird he calls the Tillinator – plump and primped. He takes a hen out of her cage and holds it out towards the Tillinator, to demonstrate his reaction. "They blow themselves up to try to make themselves the Don Juan of the pigeon world." Sure enough, the Tillinator's chest becomes even more plumped. The birds are kept from mating for as long as possible before being released in order to "build up steam".

"That's the one they're all trying to catch, but naebody can catch him. When he gets a hen, he's so sharp, so quick." How many birds has the Tillinator caught? "About 25." How long has he had him? "Five years."

You need to be patient in the doo business. Smith says he's out there flying two to three hours a night, maybe all day at the weekend, and sometimes he can go weeks without a sniff of a pigeon. There are few doowomen or girls. It has always been regarded as a man's sport, and every man has a story about the domestic strife the birds have caused. Smith says his grandmother used to throw pigeons out of the window when his grandad kept them in the house. Nowadays, Smith's own wife tells him he cares more for his pigeons than he does for her. Is it true? "Probably 50%. The pigeons never give me grief!" He laughs. "If it's a nice day, are you going to sit in there, or stand outside, fly your birds and get a tan?"

Smith talks about his relationship with fellow doomen – they can be friendly, but there is always an intense rivalry. Do they ever congratulate each other when a rival has lured away a good bird?

"Pfff! Never heard of that yet. Never."

Six years ago, Smith's teenage son Kevin was murdered – stabbed by three young men after they stole his scooter. When Kevin died, Smith lost interest in his pigeons. "I couldn't feed them – my brother came down and fed them for me. I couldn't do anything. It takes time, and it never goes away." Now at least, he says, his love of flying has returned. All those hours outside with the birds is thinking time.

Robert Ormerod started taking photographs of doomen more than a year ago. Everything about them is fascinating, he says – the passion, the lifestyle, the battles, the birds themselves. Many doomen live unpredictable lives and are hard to track down – they might agree to a photograph one day and not want to know you the next. Ormerod is driving us to the Niddrie estate, which became infamous for gangs and knife crime, and has now been largely demolished. We are here to speak to two boys he has photographed – one 18-year-old who already has two children, and his younger brother, who's nine. Ormerod warns about their two huge dogs, which can be intimidating.

Father and son Ian and Mark Wilson dye their newest pigeons yellow in the Southhouse area of Edinburgh. The yellow colour helps the bird attract members of the opposite sex. Photograph: Robert Ormerod

As we arrive at the estate, a football smashes into the boot of the car. Only the nine-year-old is home, but he doesn't want to talk to us today and we are sent packing. Another young man says he will talk only for money. So we drive to the south of Edinburgh, to meet 56-year-old Iain Wilson and his daughter, Sinead. It's 11am and he offers me a beer. Wilson is a plumber and runs one of the two shops operating out of Edinburgh where doomen trade pigeons. Flying doos, he says, is an addiction. "I caught a curse. It's like smoking. You cannae give them up." Which is more dangerous? "The same degree," he says. The pigeons produce dust that is easily inhaled. "I've had part of one of my lungs removed because of dust-related injuries."

His obsession with the birds dates back to his boyhood. "My mother and father had nae love of animals at all, so, much to their annoyance, I used to bring injured pigeons home."

How did they react? "They looked at it as a hobby and were relieved I wasn't going about the streets throwing stones at windows." Does he think he'd have got into trouble otherwise? "Probably, aye. I had good friends, not hardened criminals; we'd just do a wee bit of shoplifting now and then."

Why was he so taken with pigeons? "I was fascinated by something that could fly that comes back to you. You get an affinity with them. Certainly with the horseman. I used to have a pigeon that followed me all the way to the pub."

He feeds them only the best food, he says. "Sinead, show Simon." Sinead returns with a bucket of food that could pass for nibbles at an upmarket party. "Sunflower seeds, barley, maize, maple peas, wheat, barley – I used to eat it when I was waiting for the racing pigeons. So they get the best grub; no rubbish."

As he talks, he keeps an eye on the sky and catches sight of a hen coming in to land. "Bop. Bohop bhop. Ooop. Cooo," he warbles gently.

Can you be a good dooman without talking to the pigeons? "No, you've got to talk the language. You've got tae have some affinity with the birds. Good pigeons respond."

These days it's Sinead who plays Florence Nightingale to hurt birds. "If my mates find an injured pigeon, they'll always bring it to me because I cannae turn it away," she says. "They wouldnae take it to my dad, though, because he'd kill it. He's only interested in pigeons who work."

"I'll be perfectly honest with you," Wilson says. "I'm brutal with killing pigeons. If pigeons dinnae perform for you, you've got to kill them. Either that or you'd have thousands. I either pull their neck or tap them on the back of the head."

He hopes that if Sinead becomes a fully-fledged doogirl, it will keep her out of trouble. "Well, it doesn't matter if it's pigeons or whatever – you want your children to have some sort of thing, because there's absolutely nothing for children to do here. It is quite a drugs-dominated place, a lot of coke. Very high unemployment. Everybody that flies in this scheme beyond me and my son dinnae work." He pauses. Mind you, he says, you can get into plenty of trouble flying doos. Sinead smiles knowingly and says nothing. "In the old days, there'd be murder," Wilson continues. "Huts burnt down. Glasgow was the worst. You strive for years to get good pigeons, then if someone comes into your hut and steals them, it makes your blood boil. Once you get a good pigeon, everybody knows about it."

Has he ever got into trouble over the birds? "Touch wood I've never had that scenario, because I think people know, were that to happen, there'd be repercussions. The majority in the scheme are frightened of me." Physically frightened? "Aye." Because he's tough? "I don't know whether I'm tough now, but I used to be. I'm getting a bit old. I've hung up my gun belt. I mostly keep the peace. My son, he's a hot-headed one."

It's funny, Wilson says, sometimes he wonders why he still dedicates so much of his life to the birds – there's his lungs, the time, the money. "The worst thing is that it brings out emotions that it shouldn't bring out." Such as? "Anger. There are times you want to kick people around the street. I think it triggers off emotions that you wouldn't normally have in everyday life. Extreme emotions, aye. I could be sitting in the pub having a laugh. Instead, I'm standing in this garden going off my head at the pigeons."

So why does he keep doo-flying? "Pure addiction. Aye, it ruins your life." Then he sees a pigeon coming in, smiles with relief, and welcomes it home with a warble.