As a member of the media covering the tragedy in Haiti, it's with a sense of alarm and astonishment that I've witnessed how some senior aid officials have argued for withholding aid of the utmost urgency because of sensational claims about violence and insecurity, which appear to be based more on fantasy than reality.

John O'Shea, who runs the well-known Irish aid agency Goal, has joined this chorus, telling the Guardian he couldn't get his trucks from the Dominican Republic to Haiti because he had no guarantees his drivers wouldn't be "macheted to death on the way down". He added that Goal has no plans to deploy its much-needed doctors and nurses on the streets of Port-au-Prince.

From what I've observed, such chilling claims do not match the reality on the ground; and by trumpeting a distorted and sensational picture about the violence, some senior aid officials may be culpable of undermining the very aid effort they are supposed to be promoting. When I traveled into Haiti's disaster zone last week from the Dominican Republic, I did so alone and on a bus, whose passengers were mostly Haitians, including some living in the US. Since then, whether on the road to Port-au-Prince or within the city, I have not witnessed anyone wielding a gun, a machete or a club of any kind. Nor have I witnessed an act of violence. (I have seen one badly wounded man who had been shot in circumstances which were unclear and who was eventually rescued by US soldiers after an American reporter sought help.)

Any violence is localised and sporadic; the situation is desperate yet not dangerous in general. Crucially, it's not a war zone; it's a disaster zone – and there appears to have been little attempt to distinguish carefully between destructive acts of criminality and the behaviour of starving people helping themselves to what they can forage. For Haitians and many of those trying to help them, the overriding sentiment is that a massive catastrophe on this scale shouldn't have to wait for aid because blanket security is the absolute priority.

Moreover hundreds of journalists, volunteers from churches and private individuals have traveled in from the Dominican Republic, some of them bringing in desperately needed aid. From what I know, not a single person who has attempted to provide assistance to the people of Port-au-Prince, including medics, has run into any serious trouble.

En route to Port-au-Prince I met David Pierre-Louis, a 31-year-old Haitian-American, who had come to find his mother and bring much needed medical supplies. Happily, he found her alive and well, and later, she and a local nurse used the medicine David brought all the way from Seattle to set up a makeshift clinic on a street near her shattered home.

David, who runs a jazz club in Seattle, is now trying to fill the void by sending in his own medical supplies paid for by donations from the Haitian-American community and other concerned Americans. He told me:

"Haitians here cannot understand why they're not getting help, especially as the way the violence is portrayed is not right. The people are unhappy that there's been no assistance but do you see them rioting in the streets? No.

"People are hungry and needy and yet they're being portrayed as savages. Aid is not getting there quick enough and that's sad because the solution is right there and we have the power to do it."

John O'Shea has shown in the past that his aid agency has the power to do it. Yet this time, while the Irish people have generously donated more than 1m euros to Goal for their Haiti operations, the agency has yet to swing into action. While announcing that they hope to start some limited food distribution in one location in Port-au-Prince, O'Shea is insisting on a change in the security situation first before their operation can be rolled out, medics and all.

There are some real security issues in Port-au-Prince but some of the more alarming images and incidents portrayed in the media must be seen for what they are, and in context. Reports about marauding, machete-wielding gangs taking over Port-au-Prince are very wide of the mark. The people are welcoming and helpful to those who come to help them and, if anything, go out of their way to ensure you are safe.

Last weekend, in the park near the destroyed presidential palace, which has become a makeshift refugee camp for tens of thousands, we meet three Cuban doctors and nurses. They were working alone, without an escort, and they were treating a large group of injured men women and children, who calmly waited their turn. That night, on the other side of the park, I saw a group of homeless queueing patiently to collect water in plastic containers. No one was harassing them, and there was no sign of any of the criminal gangs that supposedly now rule the streets.

I can see no reason why, with some concerted pressure and a little coordination, aid agencies like Goal cannot deploy securely into the heart of Port–au-Prince, with their clinics and food distribution outlets. With thousands of the injured living in close quarters at makeshift camps, the rapid deployment of medical care is still paramount.

Of all the disasters I've covered in recent years, the response to this has perhaps been the most perplexing, and disastrous in itself. From the Haitian perspective, if anyone is dragging their feet it's the aid agencies. One thing is clear: if aid agencies do not quickly roll out a coordinated and comprehensive response, then not only will many more die, but the deteriorating security situation, which is being talked up so much, may perhaps become a self-fulfilling prophecy.

So John O'Shea, if you are reading this, I put down this challenge to you: if you are prepared, in the next few days, to bring an aid van or truck to the Dominican/Haitian border, I will travel with it into Port-au-Prince. I will even help you to distribute the aid.

The Haitian people need help now, not excuses.