IOTA SNAPSHOT

What is a IOTA snapshot?

A IOTA snapshot is a process where all the transaction history gets reset. After the snapshot processing resumes and a new Tangle grows on this basic ledger. Until it grows so big that another snapshot is required.

IOTA‘s nodes are receiving transactions in a staggering amount. They need to keep track of them all and sort through them. This means ever increasing memory and CPU requirements. To combat this every now and then a snapshot occurs. This essentially throws away all the transaction history and resets the IOTA ledger to a list of all the addresses that have a nonzero balance and of course their balances.

After the snapshot processing resumes and a new Tangle grows on this basic ledger. Until it grows so big that another snapshot is required.

In the future nodes will decide for themselves when to do a snapshot, but at the moment this is still a synchronized operation. The snapshot ledger is prepared by the IOTA team, then it is verified that it contains all the transaction history correctly by a number of nodes, and only when the consensus is that the snapshot data contains no errors the green light is given and every node out there will switch to this new ledger at the same time.

Do I care and why?

Well you definitely should. At least as long as your wallet takes its state information from the Tangle, like the light wallets are doing at the moment. The wallet stores absolutely no state information on your device. That means it has to use the seed you provide at login to generate addresses one by one, check if these addresses are used in a transaction already, and if they are it knows that it already gave out those addresses and can retrieve the balances of these addresses to give you your total wallet balance.

However, since the snapshot removes all transaction history from the Tangle, the wallet loses all its state information and therefore will show you a zero balance after the snapshot! Don’t worry. This is expected. All you need to do is reattach all previously used receive addresses to the tangle manually.

Wait, what?



Yep, because IOTA was designed for the Internet of Things, and those devices follow simple rules with extreme precision and patience, this is no problem for the IoT devices. They will happily and patiently reattach every address they ever used to the Tangle. Our problem is that we are no IoT devices. And the wallets at the moment don’t hide this tedious process behind the scenes. So we will have to do this manually so that we can see our balance being restored.

One additional important reason to do this is to avoid re-using any addresses that we have used before the snapshot, so we don’t end up with multi-spends from these addresse and won’t be subjected to hackers intercepting out transactions.

I know, WTF, right? But this is what happens when beta software gains large acceptance before the hurdles get ironed out. We can complain all we want about this but fact it we will have to deal with this until the next generation of wallets come out that do this stuff automatically behind the scenes.

IOTA Snapshot Preparations

There are a few things you can do in preparation of a snapshot. One is to log into your wallet using your seed, go to the history section, then click on addresses. You now want to make a note of the amount of addresses used, plus the what the first and last address are.

That way, when you want to restore the wallet after the snapshot, you have some data that can assist you in knowing:

1. whether you are in the correct seed. (You can compare the first receive address you re-attach with the one you wrote down)

2. how many times you will have to re-attach receive addresses for your funds to show up.

Knowing this is having won half the battle. You know immediately if you are in the correct seed. And you know exactly how many re-attachments it will take to regain your balance. Fort the rest it is just a tedious process.

Follow the steps, if you have any doubts, after the snapshot

First of all, don’t panic, your IOTAs are most likely perfectly safe.

Second, no matter who or what, *never* give your seed to anyone. Especially not if they say they are an IOTA developer who will help you recover your balance. These people are scammers and will steal your IOTA.

I will show the steps you can take to recover your balance. I will also try to explain to you why this happened. I recommend you take the time to read and understand the how and why. Understanding what you invested in is the best thing you can do to keep your investment safe.

Steps to find your balance

First, make sure you upgrade to the latest version of the *desktop* wallet. You can safely skip older versions if you did not install those before. These older versions will no longer work. The process for the Android wallet should be similar. Just make sure you update to the latest version first. You can find the latest version of the desktop wallet for Windows, Mac, and Linux here (don’t get confused by the title of the page, this is the latest wallet):

https://github.com/iotaledger/wallet/releases Next, determine if you need to reclaim your funds. Do not start the reclaim tool yet. If you used the previous wallets and you had any funds that were visible in that wallet you do not need to reclaim these funds using the reclaim tool. Just continue with step 3 first to find your balance again. If you last used your wallet before the transition in August, or if you already know that the IOTA foundation took custody of some of your funds, then you need to take some extra steps to reclaim your funds after you restored your balance. I will discuss those later. If you did start the reclaim tool unnecessarily there’s no problem. You can still use your old seed and use the next steps to find your balance. After installing the new wallet you need to set up your wallet correctly. What you need to do is go to Tools->Edit Node Configuration. Make sure that a Host node is selected. If you see ‘Select Your Host’ in the Host field, use the drop down to select one. Usually any one will do. However, keep in mind that a host may be temporarily offline, so if after selecting one the wallet indicates No Connection you may want to select another Host. My experience shows that the iotatoken.nl hosts are pretty reliable and fast. Make sure that Min Weight Magnitude is set to 14. Higher values should be no problem but will just cause the Proof of Work to take longer unnecessarily. Finally, you see a field that says Curl Implementation. The default is WebGL, which is a new setting that allows the wallet to use your graphics card to do the Proof of Work, thereby speeding up the process. However, some people will find that they get an Invalid Transaction Hash when they use this setting. The solutions for that are to either select CCURL instead, or to follow the instructions found in this link to make sure the wallet uses the correct graphics card:

https://steemit.com/iota/@tector/iota-light-wallet-2-5-1-dealing-with-invalid-transaction-hash Note that it may be necessary to close and restart the wallet after changing any of those settings. I would do that just to be sure that the changes are active. Now you can use your seed to log into the wallet, so you can start finding your balance again. Make absolutely sure that you are using the correct seed. Any typo in the seed will result in a different wallet, and trying to find your balance in there is not going to work, obviously. There is a helpful checksum field next to the field where you can enter your seed. The wallet will tell you if your seed is too short or too long. And if you have entered exactly 81 characters it will show you a 3 character checksum that should always be the same for every version of the wallet. It will allow you to very quickly see if you made a typo, since any typo will result in a different checksum. If you are not sure if you used the correct seed there are instructions further down to help you make sure. After logging into the wallet you click on the Receive section. It will show you a new receive address. Now click Attach To Tangle. It will show ‘Attaching to Tangle’ for a while. Just wait for that to complete. If it takes too long you could try a different, faster node as outlined in step 4. Once it says ‘Address Attached’ you may see your total balance get updated with the balance of that address. If not, then click ‘Address Attached’. It will now say ‘Generate New Address’. Click that again. That will cause the next address to be generated and it will show ‘Attaching to Tangle’ again. Keep repeating this loop until you have recovered your balance completely. You can click the History section to see how many and which addresses you have generated so far. Addresses can be in pending state but that does not matter for finding your balance. You will need to generate all addresses you have used before for your entire balance to show up again. So if you have an idea of how many you used before that will be helpful. Most people should be able to find some or all of their balance within the first 20-30 addresses. But I know of people who needed to do 150+ re-attachments because they had been very active with their wallet. If your balance started to show up but never completely shows, and does not change any more after re-attaching don’t bother continuing. The funds are usually clustered at the end of the address list. Any funds still missing have probably been taken into custody. See below for details on how to verify that. If you did not manage to find your balance using these steps, I suggest you take the time to read through the explanations below. You need to be able to understand why it still does not show and what you can do about it. If you did find your balance? Congratulations! But I still suggest you educate yourself so you know what you are invested in. I know it’s a drag to have to go through such a long document, but we’re still in beta software stage with the wallet and IOTA as a whole. Understanding how it works is equal to protecting yourself from losing your funds.

How does an IOTA wallet work?

Contrary to what most people believe your IOTA wallet does *not* contain your IOTAs. Instead, your IOTAs are safely stored in one or more addresses in the Tangle. You can see your wallet as a browser that knows how to retrieve the balance of an address. To do that it contacts a so-called node, and asks it to find the balance of that address in the distributed ledger database that the Tangle actually is. To be able to retrieve your entire balance, the wallet will usually have to retrieve the balances of multiple addresses that belong to your wallet and add them all up. The end result is displayed as your current balance.

Now here is an important thing: the wallet does not know at first how many and which addresses exactly to retrieve to accumulate your balance. The only starting point it has is the 81-character seed you provided when you logged into the wallet. This seed is the key that enables the wallet to always re-generate the addresses used by your wallet deterministically. Deterministically means that given the same seed it will always be able to generate the exact same sequence of addresses one by one. To determine your total accumulated balance the wallet will ask the node for each address balance in succession until the node indicates that an address cannot be found because it is not present in a transaction in the Tangle. This indicates to the wallet that it has not generated/used that address before, so it can safely stop accumulating address balances. This fact is important to remember, because it will shortly explain why your balance is currently zero.

The wallet will generate new addresses as needed in two important ways. First, and most visible, when you want to receive IOTAs in your wallet you will need to generate a new receive address and attach that to the Tangle. Addresses are not supposed to be re-used because the signing process used to spend funds from an address uses one-time signatures. Because of the one-time nature of the signing process, the private key for that address gets partially exposed by every spend from the same address, opening it to a brute force attack. And since anyone that has the private key to an address can spend the funds in that address it’s not very wise to open yourself up to such an attack. That’s why the wallet tries to defend itself against such attacks by automatically moving the remaining funds after spending to a new address in the wallet. It will also give you a ‘private key reuse’ error whenever you try to spend from an address you have already spent from.

So the second way in which the wallet will generate new addresses is by moving the remainder of a spent address to a new address. When you instruct it to send a certain amount of IOTAs to another address, what it will do is go through the known addresses in your wallet one by one, starting from the first, by generating the addresses in sequence from the seed. The wallet accumulates the amount of IOTAs it finds along the way on addresses with a non-zero balance, until it has accumulated enough IOTAs to cover the amount you want to send. The contents of the non-zero addresses it has accumulated in this way will be sent as a single transfer to the receiving address and -if there is a remainder- the remainder will be sent as a second transfer to a newly generated address in your wallet. This will cause the latter address to be attached to the Tangle, so it lives there now, and the wallet will be able to see it as in use already whenever it scans for your balance as outlined above. All addresses used as the input for the transaction will end up having a zero balance, so they are not likely to be re-used any more, unless you make the mistake of not creating a new receive address but re-using a previous one.

It is important for the recovery process that you have an idea of how many addresses were in use before you ‘lost’ your balance (before the Snapshot happened, see below). For most people that will be up to one or two dozen addresses. But for people who used the wallet regularly before the snapshot happened that can easily go up in the hundreds. I personally have helped someone retrieve his balance who had 150+ addresses in use in his wallet.

Why is my IOTA balance zero?

To understand what happened you need to know what happens when the IOTA foundation generates a new Snapshot of the IOTA ledger database. This is done every now and then for maintenance reasons. The last Snapshot was created on Tuesday October 24th, 2017. One reason for taking a Snapshot is that the Tangle is constantly expanding, and therefore taking up more and more memory. To help nodes in keeping the amount of memory they need for storing the database limited, the Tangle is pruned by removing the history of transactions. Remember that after every transaction you end up with no longer used addresses with a zero balance. The transactions that caused these only take up space and are no longer necessary for correct functioning of the nodes. There are even more ways that can cause zero balance transactions to occur, like when a wallet generates and attaches a new address to the Tangle. There are also transactions that are used as messages between systems and do not contain any balance.

So after a Snapshot is taken the end result is a simple list of all addresses that contain a non-zero balance and their respective balances, and no history of how those balances happened is preserved on the regular nodes. By the way, this history *does* get preserved on so called perma-nodes, like ioatsear.ch, which will keep track of the entire history all the way back to the first transaction ever. But for most use cases this history is not necessary. It’s only the non-zero end balances that are important, since they are the only ones that can be spent.

Note that the Snapshot process is such that any transactions that were not confirmed yet while the Snapshot was being taken will be canceled and it will be as if the funds never left the wallet of the sender. So that means some transactions may have to be performed again once you have restored your wallet. A deposit from your wallet to Bitfinex that was canceled will have never left your wallet. A withdrawal from Bitfinex to your wallet that was canceled will have never left the Bitfinex wallet. Just send those transactions again when that happened. They should pose no problem.

Now, remember how the wallet will figure out its balance? It will ask a node to retrieve the balances of each address it generates in succession from the seed, starting with the first one, that has been used in a transaction, until it finds a non-used address, which indicates that the wallet did not generate that address before. However, since the Snapshot has removed all historical transactions the first address it checks will not be found to be used by a transaction in the Tangle, and so the wallet stops scanning for balances immediately, because it thinks it has not generated that address yet, ultimately resulting in a zero accumulated balance.

So that’s the reason you see a zero balance. The wallet simply stopped scanning for addresses with balances too soon, because it does not know that the Snapshot process has removed the transaction history. This means that the remedy for finding your balance is simple. Start re-generating and attaching your previously used receive addresses one by one. This will cause zero-balance transactions using that address to be connected to the Tangle. Which means the wallet now can see the address is in use in the Tangle and ask the node for the corresponding balance. In particular, you need to do this as many times as you had used addresses in the past, so that they can all be found in the Tangle again. To be sure, this can be a tedious process and it would have been nice if the wallet designers had included a command that allows you to tell it to generate the next X addresses until it finds one with a balance. But that is not the case at the moment, so there is no use in complaining right now. I am sure this feature will show up in a next incarnation of the wallet.

Why is the wallet still not able to find my balance?

The Sep 22nd and Oct 24th Snapshot processes have done two additional maintenance tasks that may result in you still not being able to successfully retrieve your correct balance. First, since the implementation of the Proof of Work function has changed from kerl to curl in August there was a need to transition balances to the use of the new curl implementation. A lot of people already did this transition as part of their upgrading their wallet from an older version. Since it turned out that there are people who managed to lose their seed and also many people who did not transition yet, the IOTA foundation has taken the precaution to remove those old-style kerl addresses from the Tangle and has taken custody of them. So if your previous wallet was from before version you may fall into that category. You will need to use the reclaim tool as explained further down to reclaim those funds.

Second, because it was found that quite a number of people had spent multiple times from the same address, thereby exposing their funds to being stolen, the IOTA foundation decided to take similar custody of those addresses so they are no longer vulnerable. That means if you accidentally spent from the same address more than once in the past, and this address was still holding a balance, you will need to use the reclaim tool as explained further down to reclaim those funds.

In both cases one or more of your address balances may still be ‘missing’ even after the wallet has re-generated the missing address in the Tangle. The balance of such addresses was moved to the IOTA foundation custody address. You can check for those addresses if you think that may be what has happened to them here:

https://goo.gl/FGtXgp

Allow for the page to load before searching through it, as it is a huge list.

You just copy an address you want to find out about to the clipboard, and then use control-F and paste to search on that page. Make sure you only use the first 81 characters of the address, as the 9 character checksum is not present in this list. I just paste the address and delete the last 9 characters.

If your address shows a category of KEY_REUSE or KEY_REUSE_OCT in this list then you have executed more than one spend from that address and the balance is now in custody of the IOTA foundation until you reclaim it.

If your address shows a category of CURL_UNUSED then your address was not transitioned to the new curl implementation yet and the balance is also in custody of the IOTA foundation until you reclaim it.

For more details, see this announcement from the foundation:

https://forum.iota.org/t/snapshot-public-validation-22-09-2017/4256

How do I make sure I am using the correct seed?

If you don’t remember the 3-character checksum or in any other way are unsure if you are using the correct seed, here are ways to find out:

If you know roughly how many addresses you used to have in your wallet, just go ahead and re-generate at least that amount of new receive addresses (detailed steps below). Along the way, usually towards the end, you should see your balance increasing. If your balance becomes non-zero that is a clue that you are definitely in your own wallet with the correct seed. If you cannot seem to get the entire balance you may either need to generate even more addresses, or some of your funds were taken into custody by the IOTA foundation for the reasons described above. In that case you will have to check the addresses which you think should contain a non-zero balance against the snapshot list. And if you find them there, you will have to reclaim them using the reclaim tool.

Making double sure that you are using the correct wallet can be easily done by using the iotasear.ch Tangle browser to check the history of an address. This means you can copy the first address and see if in the past funds were there. Assuming the first address was the first one to receive funds of course. If it did contain funds the amount is probably recognizable as your first IOTA withdrawal ever. Anyway, when any funds are there in the iotasear.ch history you can be sure you have used this seed before.

Finally,

How do I use the reclaim tool?

If you are absolutely sure that the IOTA foundation took custody of some of your funds after checking the above, you will need to use the reclaim tool to reclaim your funds. Note that the reclaim process is a two-step process currently.

NOTE: to make sure that you don’t make any typos when entering your old and new seeds with this tool i highly suggest you copy/paste them from a proven working copy. It would be silly to finally reclaim your funds to a seed that contains a typo and therefore the reclaimed funds would become irretrievable.

UPDATE: The reclaim tool in phase 2 will ask once again for the new seed, so unless you make the same typo twice this seems not really to be a problem.

In case multiple reclaims were put in, the one confirmed in the lowest milestone will be the one processed.

For more details on how to use the reclaim tool see this link for phase 1:

https://blog.iota.org/gui-v2-5-2-latest-release-with-iota-reclaim-tool-32d364d6241a

And this link for phase 2:

https://blog.iota.org/gui-wallet-phase-two-of-the-reclaim-process-f5913109cf46