When Eurosceptic politicians sneer about unelected foreign bureaucrats, one person they surely cannot have had in mind was Sir Julian Priestley, one of the most senior Britons in the European Union, the secretary general of its parliament for 10 years and a man who devoted his whole career to promoting the European ideal. To see the Brexit vote last June must have been particularly galling for him, as he had been at the centre of British involvement from the start in 1973, first as a junior administrator in Brussels and then as youth organiser for the successful Britain in Europe referendum campaign in 1975.

Priestley, who has died aged 66 of cancer, was a tall, sardonic figure who never made any secret of his socialist principles. He was central to the development of the parliament and its work over three decades. The secretary generalship is somewhat analogous to the role of clerk of the Commons at Westminster: advising on procedure and rules, overseeing the 4,000 staff who work at the parliament, and organising the plenary sittings in Brussels and Strasbourg.

No one ever suggested that he was anything other than impartial in his dealings with the political groups, though he may have grown a little jaded with the constant baggage-trundling across Belgium and France: as the Guardian’s correspondent I once suggested that it was fun to visit Strasbourg, one of the most beautiful cities in Europe, every month. “Not after 20 years it isn’t,” he replied wryly.

During the late 90s, he and another Briton, David Williamson (Lord Williamson of Horton), who was secretary general of the European commission, held two of the three most senior administrative and bureaucratic jobs in the EU – so much for Britain having no influence in Brussels.

Priestley was born in Croydon, south London, the only son of Arthur, the chief accountant for an Anglo-French car components company, and his wife, Patricia (nee Maynard), but he grew up in Plymouth after his father’s firm moved to the West Country. His parents were Labour party members and early supporters of Britain joining the common market. Priestley was educated at St Boniface’s college, Plymouth, a Catholic direct grant school, and won a place at Balliol College, Oxford, to read philosophy, politics and economics. There he chaired the university Labour club and became president of the Oxford Union debating society.

In the 1970s and early 80s, he fought three general elections for Plymouth constituencies: Plymouth Sutton in October 1974 and 1979, where he came second to the Conservative Alan Clark, and Plymouth Devonport in 1983, where he trailed third behind David Owen, the SDP leader who had formerly held the seat for Labour, and the Tories’ Ann Widdecombe. As Labour grew more hostile to Europe, he concentrated his career across the Channel, rising through the sometimes stultifying international bureaucracy of the parliament, serving as secretary to the budget committee – where he drafted MEPs’ first rejection of the commission’s proposed budget in 1980 – then the staff committee, where he led a strike against the antisocial hours the parliament was imposing unnecessarily on its employees.

In 1994 he might have returned to British politics, when the newly elected Labour leader, Tony Blair, invited him to become his chief of staff, but Priestley decided to remain in Europe as chief of staff to Klaus Hänsch, the German Social Democrat president of the European parliament. He had previously been secretary general of the socialist group of MEPs for five years and in 1997 became secretary general of the parliament itself.

This period was a turbulent one, encompassing the rapid expansion in the number of member states from 15 to 27, the rise of Eurosceptic parties including Ukip, and MEPs’ increasing flexing of muscles as the parliament’s powers of consultation and decision-making grew. This culminated in the resignation of the entire European commission in 1999, under pressure from MEPs following the French commissioner Edith Cresson’s refusal to resign after she gave a grace and favour job to her dentist. The period was perhaps the EU’s high water mark as boundaries seemed limitless, integration inevitable and a common currency obvious.

Priestley took early retirement from the parliament in 2007 and was knighted, but in 2014 he ran the campaign of his friend Martin Schulz, the German Social Democrat president of the European parliament, in the latter’s unavailing bid to challenge Jean-Claude Juncker for the presidency of the commission.

He lived in Luxembourg with his partner of 33 years, Jean Schons, a property adviser, whom he married in 2015 and who survives him. Following his retirement, he wrote five books with European themes, mainly about the institutions but including most recently a novel, Putsch! (2016), subtitled Principle, Ambition, Compromise, Intrigue, Threats, Sex – Well That’s Politics, about internecine struggles within a post-Blairite Labour party. He also lectured at the College of Europe in Bruges.

Following the Brexit vote, Priestley told friends that he had taken Luxembourgeois citizenship and joined the Luxembourg Socialist Workers’ party (actually social democrat rather than Trotskyite).

Recent blogs on his website indicate the depths of his disillusionment, but he saw hope in the candidacy of Emmanuel Macron for the French presidency: “It would be a turning point in the defence of the European Union, but more widely in support of European values in a cold climate of populism and authoritarianism. Macron would be the candidate least acceptable to the Putins and the Trumps … the candidate who best embodies our hopes for a revitalised Europe.”

• Julian Gordon Priestley, civil servant, born 26 May 1950; died 22 April 2017