I look at the camera, hold a bucket of ice water over my head, tip it upside down, post the video on social media and then nominate two others to do the same. Along the way, my nominees and I use the opportunity to donate to the ALS Association, a charity that fights amyotrophic lateral sclerosis (also called Lou Gerhig’s disease), a fatal neurodegenerative disease. Multiply this activity 70,000 times, and the result is that the ALS Association has received $3 million in additional donations. Via the ice bucket challenge, celebrities and the general public have fun and receive publicity; at the same time, millions of dollars are raised for a good cause. It’s a win-win, right?

Sadly, things are not so simple.

The key problem is funding cannibalism. That $3 million in donations doesn’t appear out of a vacuum. Because people on average are limited in how much they’re willing to donate to good causes, if someone donates $100 to the ALS Association, he or she will likely donate less to other charities.

This isn’t just speculation. Research from my own non-profit, which raises money for the most effective global poverty charities, has found that, for every $1 we raise, 50¢ would have been donated anyway. Giving What We Can fundraises for global poverty charities by encouraging people to pledge at least 10% of their income. For everyone who joins, we ask them to estimate what proportion they would have donated otherwise. Averaged among all our members, that amount is greater than 50% (or $150mn out of $300mn). Given our fundraising model, which asks for commitments much larger than the amount people typically donate, we have reason to think that this is a lower proportion than is typical for fundraising drives. So, because of the $3 million that the ALS Association has received, I’d bet that much more than $1.5 million has been lost by other charities.

A similar phenomenon has been studied in the lab by psychologists. It’s called moral licensing: the idea that doing one good action leads one to compensate by doing fewer good actions in the future. In a recent experiment, participants either selected a product from a selection of mostly “green” items (like an energy-efficient light bulb) or from a selection of mostly conventional items (like a regular light bulb). They were then told to perform a supposedly unrelated task. However, in this second task, the results were self-reported, so the participants had a financial incentive to lie; and they were invited to pay themselves out of an envelope, so they had an opportunity to steal as well.

What happened? People who had previously purchased a green product were significantly more likely to both lie and steal than those who had purchased the conventional product. Their demonstration of ethical behavior subconsciously gave them license to act unethically when the chance arose.

Amazingly, even just saying that you’d do something good can cause the moral self-licensing effect. In another study, half the participants were asked to imagine helping a foreign student who had asked for assistance in understanding a lecture. They subsequently gave significantly less to charity when given the chance to do so than the other half of the participants, who had not been asked to imagine helping another student.

The explanation behind moral licensing is that people are often more concerned about looking good or feeling good rather than doing good. If you “do your bit” by buying an energy-efficient lightbulb, then your status as a good human being is less likely to be called into question if you subsequently steal.

In terms of the conditions for the moral licensing effect to occur, the ice bucket challenge is perfect. The challenge gives you a way to very publicly demonstrate your altruism via a painful task, despite actually accomplishing very little (on average, not including those who don’t donate at all, a $40 gift, or 0.07% of the average American household’s income): it’s geared up to make you feel as good about your actions as possible, rather than to ensure that your actions do as much good as possible.

This why Caitlin Dewey, a blogger for the Washington Post who claims that we should praise the challenge for raising so much money, gets it all wrong. The ice bucket challenge has done one good thing, which is raise $3 million for the ALS Association. But it’s also done a really bad thing: take money and attention away from other charities and other causes. That means that, if we want to know whether the ice bucket challenge has been on balance a good thing for the world, we’ve got to assess how effective the ALS Associations is compared with other charities. If 50% of that $3 million would have been donated anyway, and if the ALS association is less than half as effective at turning donations into positive impact on people’s wellbeing than other charities are on average, then the fundraiser would actively be doing harm. It’s perfectly possible that this is the case: even though some charities are fantastically effective, many achieve very little. You just can’t know without doing some serious investigation.

This isn’t to object to the ALS Association in particular. Almost every charity does the same thing — engaging in a race to the bottom where the benefits to the donor have to be as large as possible, and the costs as small as possible. (Things are even worse in the UK, where the reward of publicizing yourself all over social media comes at a suggested price of just £3 donated to MacMillan Cancer Support.) We should be very worried about this, because competitive fundraising ultimately destroys value for the social sector as a whole. We should not reward people for minor acts of altruism, when they could have done so much more, because doing so creates a culture where the correct response to the existence of preventable death and suffering is to give some pocket change.

Cannibalism of funding among charities is a major problem. However, there is a solution. The moral licensing phenomenon doesn’t always happen: there is a countervailing psychological force, called commitment effects. If in donating to charity you don’t conceive of it as “doing your bit” but instead as taking one small step towards making altruism a part of your identity, then one good deed really will beget another. This means that we should tie new altruistic commitments to serious, long-lasting behavior change. Rather than making a small donation to a charity you’ve barely heard of, you could make a commitment to find out which charities are most cost-effective, and to set up an ongoing commitment to those charities that you conclude do the most good with your donations. Or you could publicly pledge to give a proportion of your income.

These would be meaningful behavior changes: they would be structural changes to how you live your life; and you could express them as the first step towards making altruism part of your identity. No doubt that, if we ran such campaigns, the number of people who would do these actions would be smaller, but in the long term the total impact would be far larger.

So, sure, pour a bucket of water over yourself, or go bungee jumping, or lie in a bathtub of beans, whatever. But only do these things if you connect these fundraisers with meaningful behavior change, otherwise your campaign, even if seemingly fantastically successful, could be doing more harm than good.

We welcome your comments at ideas@qz.com.

You might also like: This week, let’s dump a few ice buckets to wipe out malaria too