As any hormonal Catholic teenage boy knows, it’s perfectly possible to believe something is a grievous sin and yet repeatedly, almost compulsively, do it anyway. Almost 90% of the UK population should know the feeling, with the main difference that any guilt is more justly deserved. That’s how many of us knowingly and willingly buy from a company that has a terrible record on employee terms and working conditions, avoids paying tax as much as possible and shares responsibility for boarded-up shops in decimated high streets across the country.

That company, Amazon, is 25 years old. I’m not sorry for pooping its party. Nor will Amazon be worried about the dearth of warm birthday wishes. It’s always the winner in its year-round game of pass the parcel and every day it gets a bigger and bigger slice of the world’s wealth cake.

Why on earth do so many of us continue to feed this monstrous beast? The answer looks obvious: price and convenience. If that were true, it would seem the only things bought more cheaply and easily than the latest Dan Brown for Kindle are our own consciences.

But if you look beyond the few seconds it takes to one-click order it’s clear that Amazon is not genuinely cheap or convenient. One lesson we have been slow to learn is that in badly regulated markets, the price you pay at the point of purchase does not necessarily reflect the true cost. Amazon can only sell at the prices it does because it doesn’t pay for many of the things that make its very existence possible. From the roads its delivery vans drive along to the healthcare and benefits its employees rely on to make up for their inadequate pay, Amazon relies on things others pay tax for.

One way or another, it is we who ultimately have to pick up the tab for this. Either we pay by other means what Amazon avoids in corporation tax or we simply get worse public services. It seems we’ve forgotten the old adage that if a bargain looks too good to be true, it probably isn’t a bargain.

As for convenience, I find it much more convenient to actually see what I’m going to buy before I pay for it. Nor can I be the only one who actually wastes huge amounts of time whenever I do try to research what the best option is online. And don’t forget the hassle of returning stuff we would never have bought if we had seen it first.

It is all too convenient for Amazon to reduce the concept of convenience to how little time it takes for us to get things. But there’s a richer sense of “convenient” that Amazon rates one star at most. The older meaning of “convenient” is “befitting” or “suitable”. In this sense it is much more convenient to buy things from a person than it is from an HTML interface. It is more convenient for me to have a rich diversity of shops in my area than it is to have only betting shops, hairdressers, cafes and nail bars. Often it turns out that what is most convenient in this sense is actually very convenient in the narrow sense, too. Phone your local bookshop and they’ll usually have the title you want in store the next day. Go to your local hardware store and they’ll advise you on what you need more quickly and more authoritatively than you can advise yourself by poring over dubious reviews.

John Oliver on Amazon: 'A system that squeezes people lowest on the ladder' Read more

But you’re lucky if you still have a local hardware store or bookshop. Because every time we clicked on “order” we dug their graves a little deeper.

I’m not saying that no one should ever shop with Amazon. Sometimes it’s either the only way to get something or so much cheaper than the alternative that it would be heroic to go elsewhere. I’m not a complete refusenik myself. But too many of us are lazily using the company by default. I was taken aback recently when a liberal, middle-class, ethically conscious friend said casually that “everyone” had an Amazon Prime account. That turned out to be not much of an exaggeration in some socioeconomic groups. Nationally, 40% have access to its Prime subscription, meaning they or someone in their household has signed up.

It won’t do. Yes, government should create a level playing field by making such companies pay their fair share of tax. That could be the one silver lining of the mother of all rain clouds that is a Boris Johnson prime ministership, in the unlikely event that he means what he says for once. But that doesn’t excuse us feeding the beast in the meantime.

You can’t have any pretence of being an ethical consumer if you routinely use Amazon when you do have good alternatives. It may be Amazon’s birthday but it’s time to burst its bloated balloon.

• Julian Baggini is a writer and philosopher