A great deal has been written recently about the frustrations of publishing a book with Julian Assange, mainly in a widely discussed, marathon article for the London Review of Books by Andrew O'Hagan. O'Hagan relates his experiences when working as a ghostwriter on an autobiography of the WikiLeaks leader that ended up being published in opposition to its subject's wishes. I'm the co-publisher of Assange's most recent book (Cypherpunks: Freedom and the Future of the Internet) and I, too, have found the experience frequently exasperating.

Let me give an illustration. It's June of last year and I'm at a party in New York when a friendly, youngish man with a beard and a beer engages me in conversation. He tells me he is a journalist on one of the city's listings magazines and asks what I do for a job. I reply that I'm a publisher and he asks whose books I'm working on. I pick the one writer of whom I'm pretty certain he will have heard. "Well," I say, shouting to make myself heard above the music, "I've just published Julian Assange." The young man's demeanour changes abruptly and he fixes me with a sneer. "Assange," he echoes, "he's a bit of a cunt isn't he?"

I've become wearily accustomed to this over my time working with Assange: the vituperation heaped on my author, the scorn directed at me for giving him a platform. I know the general script that will follow. And, sure enough, here it so often comes, as if read from the page: "I mean, he's a weirdo isn't he? That massive ego. And the sex offences in Sweden."

It's almost impossible to counter this kind of attitude, with its shallow presumptions about the character of someone never met and the guilt of someone never tried. I'm aware of what feeds it: there's plenty in the Annals of Assange about fallings-out with past collaborators – enough to suggest a "burn that bridge when we get to it" approach to life. And the accusations of the women in Sweden (which I believe, if charges are formalised, should be heard in front of a judge) do remain unanswered, in court at least.

But I also know that it is especially dangerous to pass casual judgment on the character of people who confront the powerful, because our perceptions of them are open to manipulation by those to whom they present a threat. This point is underscored by Glenn Greenwald's recent publication of documents from the Joint Threat Research Intelligence Group, a hitherto secret unit within GCHQ. The papers, part of the cache acquired by Edward Snowden, show the way that covert agents are developing internet techniques to destroy the personal reputation of targets. Appearing under the heading "The Art of Deception", tactics discussed include "'false flag operations' (posting material to the internet and falsely attributing it to someone else), fake victim blog posts (pretending to be a victim of the individual whose reputation they want to destroy), and posting 'negative information' on various forums". Furthermore, another Greenwald report reveals that Assange and WikiLeaks have been the specific target of operatives in GCHQ and the NSA.

O'Hagan's LRB piece is no part of an organised dirty tricks campaign. But by focusing as it does on Assange's character defects, it ends up serving much the same purpose. Here is a man who eats with his hands and is paranoid about assassins in roadside bushes, whose lascivious gaze is directed towards teenagers and who is infatuated with the thrill of arriving at the Hay literary festival in a helicopter. Meanwhile, his achievements in uncovering the misdemeanors of the secret state are almost entirely passed over.

O'Hagan portrays Assange as a Walter Mitty-like fantasist whose absorption with grand and unrealisable schemes prevents him from ever achieving anything practical. Yet this is someone who can number among his achievements the founding of WikiLeaks, the publication of the Iraq and Afghanistan war logs, and the smuggling of Edward Snowden to safety. During the time that O'Hagan writes about, Assange was managing the ongoing Cablegate releases, preparing for his own extradition hearings and a US grand jury investigation, and assembling the Guantanamo documents from Chelsea (then Bradley) Manning that would appear in April 2011.

I have direct experience of Assange's ability to get things done. The publication of Cypherpunks at the beginning of 2012 involved an intensive editorial and promotional effort by its author. The book is based on substantially edited transcripts of conversations led by Assange and including fellow internet theorist/activists Jacob Appelbaum, Andy Müller-Maghun and Jérémie Zimmermann. Melding these various inputs required close attention to detail and diplomatic flexibility. When it came to promotion, media interviews were agreed to with little fuss; a video parody of Bruno Ganz's portrayal of Hitler in Downfall was quickly assembled for use as a book trailer; and an op-ed for the New York Times was delivered to a tight deadline, the editorial process smooth and consensual.

Cypherpunks was a big hit for OR Books; it soon became our bestselling title. In its foreword, Assange describes a world "not sliding, but galloping into a new transnational dystopia" in which "the Internet, our greatest tool for emancipation, has been transformed into the most dangerous facilitator of totalitarianism we have ever seen". He proceeds to analyse the political and philosophical implications of the state having unlimited access to data passing through our computers and mobile phones. It is all eerily and precisely predictive of the revelations about NSA surveillance that would emerge eight months later with Snowden's whistle blowing.

It's also worth saying that during the months O'Hagan's essay covers, Assange was in a tough place. He had only recently been released from Wandsworth prison, and was now required to wear an electronic anklet while reporting to the local police station on a daily basis. The most powerful politicians on earth were demanding his head; calls for his assassination by Fox News and Time magazine pundits were in circulation. Funding for WikiLeaks had dried up in the face of a financial blockade implemented by Bank of America, Pay Pal and the credit card companies.

In O'Hagan's account, collaborators such as journalists and the head of Assange's erstwhile publisher Canongate come in for much fiercer criticism from Assange than more powerful and malevolent opponents in corporations and government. But if this charge of narcissism of small differences has any purchase when directed at Assange, it can be levelled too against O'Hagan, who largely ignores the bigger issues about which Assange and WikiLeaks have consistently sounded alarm.

I'm Julian Assange's publisher, not his friend. I'm now working on another book with him, an account of his exchanges with Eric Schmidt, the executive chairman of Google. I've visited him on a regular basis in the Ecuadorean embassy and have always enjoyed our conversations, which are generally focused on matters of politics and publishing. Though his enthusiasm and relentless optimism are striking, I don't know much about what he is like as a person. I am, however, acutely aware of his achievements, which seem to me to be both substantial and generally on the side of justice. Each time I leave the embassy and pass in front of the stern gaze of the armed special branch officer who sits at the reception desk outside, I feel a pang of sadness for my author, trapped in a small, overheated room for more than a year and half. It seems to me he deserves better.