An embassy building is an ambassador for the country it represents. Ideally it should be gracious, well mannered and welcoming. It should offer something of the culture of its home country while nodding appreciatively to that of its host. It need not be old-fashioned, but nor should it attempt to be "cool". Passing fads are just that.

What embassy architecture really shouldn't be, is like the new US embassy in Baghdad. Opened this week, this monster of a modern fortress designed by Berger Devine Yaeger , has been described by at least one US commentator as the "imperial mother ship dropping into Baghdad." It could, I suppose, be seen as a latter-day, Wild East-version of a 7th Cavalry fort, with the Iraqis playing the role of Native Americans.

Whatever it might be in our imaginations, it spells "undiplomatic" in the kind of rough and ready capital letters found stencilled on to the side of tanks, armoured cars and military trucks. And given that this vast compound – 10 times bigger than the UN complex in Manhattan and comprising over 20 buildings – has been built for the most part by humbly-paid "guest" workers from Bangladesh, the Philippines and elsewhere, it looks and feels like the work of a regime battling for first place with the galactic empire itself.

The Baghdad embassy is a very long way indeed, in time, space and political culture from the US embassies of the 1950s and 60s. Designed at the height of the cold war, these were nevertheless fine, civilised buildings by modern architects. Walter Gropius, founding director of the Bauhaus school of architecture, designed the US embassy all light and grace in Athens; Richard Neutra, best known for his glamorous modern houses on the Californian coast, was responsible for the US consulate in Karachi ;Eero Saarinen, of JFK airport's TWA-terminal fame, shaped the controversial and soon-to-be-replaced US embassy in London; while John Johansen designed the dramatic US embassy in Dublin.

The east African embassy bombings of 1998 and the infamous events of September 11 encouraged the US to rebuild its embassies and consulates as castles. While other countries have had to think hard, too, about terrorism, there have been alternatives to the 21st-century 7th Cavalry look. Take the new British embassy in Sana'a, by Design Engine Architects. Set in the Yemeni highlands against the stunning backdrop of the Jebel Nuqum and Jebel Aiban mountains, it is a brave attempt to shape a building that is secure, good to look at and that, while clearly of our times, nods intelligently to local design traditions and the powerful, mud-coloured landscape it hunkers down on.

This thoughtful British building is, however, quietly upstaged by the particularly fine Royal Netherlands embassy in Addis Ababa. Designed by Dick van Gameren and Bjarne Mastenbroek, it has been dug into the earth like the famous Ethiopian rock churches at Lalibela. Its rough-hewn, reddish concrete walls are at once part-and-parcel of the landscape and powerful modern architecture in their own right. The roof, modelled on an abstraction of the Dutch polder landscape, has watercourses helping to keep the building cool while delighting the wandering eye. Here is a building designed to be secure and yet as far in terms of character from the new US embassy in Baghdad as Addis Ababa itself is from Amsterdam.

Many of the best new embassy buildings fit into their host cities happily while offering something a little different to local design. Two contrasting examples from the 1970s show how this has been done. Arne Jacobsen's Royal Danish embassy in London's Sloane Street [1977] is a highly popular design if the capital's annual Open House weekend is anything to go by.Here is a showcase of some of the best of late 20th-century Danish design and an institution that, as yet, feels no need to abandon central London in favour of a fortified campus in the suburbs (as the new US embassy in London will be when complete somewhere between 2012 and 2016).

Basil Spence's British embassy in Rome, is a second example of how to build in a slightly exotic - yet diplomatic - fashion in a great foreign city. As the debonair Spence told the Daily Mail, "I have as a neighbour on this site, a building by Michelangelo, setting an embarrassingly high standard for any architect to meet." The embassy, is a thing of well-crafted travertine, oak and yellow marble, raised on "piloti" [modern concrete columns]. With its rooms and corridors shaded from the Roman sun, it remains a powerful yet sensitive work by an underrated British architect.

Perhaps, though, the most intriguing embassies of all are those shoehorned into existing buildings. Who today would ever guess that the Nazi German embassy in London was housed at 8-9 Carlton House Terrace, an elegant building by John Nash, today home to the Royal Society?

Ultimately, though, as both the German experience in the 1940s and British-American involvement in Iraq and elsewhere in the Middle East prove, foreign policy and intelligent diplomacy count, in the long term, for more than guns. The world is unlikely to be one, in which people of all classes, creeds, colours and gun calibres dance to music at charming embassy balls . Even so, the new US embassy in Iraq, and its future sibling in London, look like one thing, and one thing only: the architecture of failed diplomacy.