The problem is heuristics. All mentioned tools are built on heuristics and the only way to avoid them is to change how you live completely. You can be fingerprinted by the modules installed in your browser. By the programs you use and the frequency you use them.

These days you're going further than just online behavior. Shops know what you buy in what amounts, because nobody buys all the same brands you are getting fingerprinted constantly. This is used for targeted advertising, but it can also theoretically be used to track you.

MIT's Reality Mining project proved the same using smartphones. You prefer certain apps, you use your phone at certain intervals, you move around certain places. This all contributes to a somewhat unique pattern (back when I did some research on it during my internship we were getting 91% certainty in simulations, even when people changed their SIM card every few days we were still able to track them based on the SSIDs they encountered, places they went, apps they installed and used, when they checked their phones, Bluetooth devices they connect to, cell towers they passed at a certain moment in time and what smileys they use in text messages).

Avoiding heuristics means changing everything you do completely. Stop using the same apps, accounts, go live somewhere else and do not buy the same food from the same brands. The problem here is that this might also pop up as a special pattern because it is so atypical.

Changing your identity is the first step. The second one is not being discovered. As Thomas said the internet doesn't forget. This means that photos of you will remain online, messages you posted, maybe even IDs you shared will remain on the net. So even when changing your behavior it only will need one picture which might expose you.