Not many men belong to a stitching group, but Tony Casdagli picked up his enthusiasm for the craft from his father, who kept himself sane by fashioning subversive messages as a PoW

After six months held by the Nazis in a prisoner of war camp, Major Alexis Casdagli was handed a piece of canvas by a fellow inmate. Pinching red and blue thread from a disintegrating pullover belonging to an elderly Cretan general, Casdagli passed the long hours in captivity by painstakingly creating a sampler in cross-stitch. Around decorative swastikas and a banal inscription saying he completed his work in December 1941, the British officer stitched a border of irregular dots and dashes. Over the next four years his work was displayed at the four camps in Germany where he was imprisoned, and his Nazi captors never once deciphered the messages threaded in Morse code: "God Save the King" and "Fuck Hitler".

This subversive needling of the Nazis was a form of defiance that Casdagli, who was not freed from prison until 1945, believed was the duty of every PoW. "It used to give him pleasure when the Germans were doing their rounds," says his son, Tony, of his father's rebellious stitching. It also stopped him going mad. "He would say after the war that the Red Cross saved his life but his embroidery saved his sanity," says Tony. "If you sit down and stitch you can forget about other things, and it's very calming."

Tony should know. The 79-year-old picked up his father's stitching habit after a lifetime at sea serving in the Royal Navy, and from 6 September two of his pieces will feature in a new exhibition opening at the Victoria and Albert Museum called Power of Making. Tony is thrilled, but the relationship between father, son, needlework and suffering is complex and occasionally ambiguous.

The son of a family of Greek cotton merchants with operations in Manchester and Egypt, Tony and his mother, Joyce, were separated from his father when war broke out. During the disastrous allied campaign in Crete, Casdagli was captured. For a month, Joyce had no idea whether he was alive or dead; for a year, Casdagli did not receive any letters or parcels.

Most of Casdagli's confinement was spent in a German castle. Life for a British officer was not as brutal as it was in Japanese camps but it still involved terror, hunger and deprivation. Casdagli scrupulously wrote down and crossed out every day in pencil in a small black notebook. "He was very meticulous," recalls Tony, more than once. Casdagli made lists of everything – every window pane broken in bombing raids, every letter sent and received. He recorded "Improvisations", such as making a "watch stand" from a "broom handle & incendiary bomb" and "Reflections" on hunger: "Unable to remember in which hand to use knife and fork on arrival of first Red Cross parcel."

Most of all, though, Casdagli recorded his anger and frustration in cross-stitch. He had picked up sewing skills from elderly relatives and, when Red Cross parcels began arriving (containing hairbrushes with secret compartments that concealed maps, which the prisoners annotated with intelligence and smuggled out), he acquired materials. He also borrowed more threads from his old Cretan general friend – this time from his pyjamas.

When Tony was 11, he received a stitched letter through the post. "It is 1,581 days since I saw you last but it will not be long now. Do you remember when I fell down the well? Look after Mummy till I get home again," Casdagli laboriously spelled out with finely stitched letters.

In a bleak, claustrophobic part-map and part-diagram, his father created a needlework of "Room 13, Spangenberg castle". The stitching depicted inmates' cells, a few lumps of coal, a sign saying "bath every 14 days", and a menu: "soup, potatoes, wurst, bread, semolina". At the bottom was a Union flag. National flags were forbidden in the camp, so Casdagli sewed a canvas flap over it with "do not open" written on it in German. "Each week the same officer would open the flap and say, 'This is illegal,' and Pa said, 'You're showing it, I'm not showing it.'"

Captured officers played cricket and other games to pass the time, but needlework proved surprisingly popular: Casdagli ran a class for 40 officers. Was his "Fuck Hitler" gesture a great risk? "It would certainly have been torn down and he would've been put in solitary confinement or worse," says Tony. But he does not believe his father would have been executed. Despite seeing a fellow inmate shot in the back for accidentally tripping an alarm, Casdagli stuck to his policy of being unrelentingly unco-operative. One Christmas, a senior British officer struck a deal with his German counterpart that no one would try to escape, in exchange for a comfortable Christmas. Casdagli stayed in bed and refused to eat. "Pa was very cross about that. One of the few duties a PoW had was to make life as uncomfortable as possible for his captors by trying to escape," says Tony.

Among his father's works hanging above the stairs in the London home that Tony shares with his second wife, Sally, is a small, sad piece. It lists the years 1939 to 1943 alongside Joyce's initials and the words: "Any day now." It was to be another two years before Casdagli saw his wife and son again. In April 1945, in an "absolute daze", he was flown back to Britain, given a cursory medical and £10. Then he caught two buses to find his way home. Joyce had gone to pick up Tony from school. "At 12 noon, they arrived, and my cup of happiness was FULL," wrote Casdagli in his diaries.

Sadly, his joy could not so simply erase four traumatic years in captivity. Tony describes his father as "very frustrated" when he returned. His time in prison unsettled him, and soon afterwards he went to Greece as part of a British military mission during the civil war..

Meanwhile, Tony entered the navy, and hardly saw his father. He "half-heartedly" stitched as a teenager, but at sea he was always too busy to do it. When he retired from the navy, however, he and Sally, with their daughter Lucy, moved to Highgate, north London near where his father kept a flat.

Then on holiday in Cornwall, the retired son and his elderly father began stitching together. "We used to sit alongside each other doing it. Pa didn't talk very much, but we would sit and talk a bit while we did it. There were so many questions I should've asked and didn't," says Tony. "I never asked him why he pinched the old general's wool."

Stitching requires discipline and patience, two qualities Tony must have inherited from his father, but the two men developed unique styles. Tony's father created intricate symmetrical patterns. "He didn't have an enormous imagination, Pa. He liked doing things rather than inventing things," says Tony. In contrast, Tony enjoys designing his needlework. Six years ago, the wife of an old naval friend introduced Tony to the Chelsea Women's Cross-stitch group. Tony became the only male member, mentored by Joyce Conwy Evans, whose work is displayed in Canterbury cathedral and the V&A.

Tony is self-deprecating about his work, but not self-conscious. He used to enjoy stitching while waiting at airports, but cannot any longer because his needles are banned airside. "I'd sit and do my needlework after going through the gate, and people would gradually move away from me," he jokes. Now he tends to stitch in the evenings, when Sally is reading. Most of his works get sent to his five children, who live all around the world. Each grandchild receives a special piece; sons get the poem If by Rudyard Kipling. He is currently stitching one for his latest grandchild, Griffin, which depicts the mythical creature with the body of a lion and an eagle's head and wings.

Major Casdagli died in 1996, aged 90. Did he approve of his son taking on his passion? "Towards the end of his life not quite so much, because he thought we were in competition," says Tony. Having said that, Tony and Sally agree he "would be so thrilled" that Tony's work is to be exhibited alongside the creations of professional craftsmen and women in the V&A. Major Casdagli's stitching was born out of suffering, but later it became a particularly fiercely pursued habit. "He did it for defiance to start with, then he did it because he did it," says Tony. "He hated finishing them because it meant he had to do something else. He loved doing something slavishly. He was a great slave."

Power of Making is at the V&A from Tuesday until 2 January 2012, www.vam.ac.uk. A Stitch in Time: God Save the King – Fu*k Hitler! by Captain A Casdagli, available from lulu.com. Tony Casdagli is participating in a free workshop at the V&A. Crafting the Collection: Power of Making, 17 September, 11am-4pm.

• This article was amended on 4 November 2011 to clarify that Tony Casdagli went to Greece during the civil war as part of a British military mission.