_ilap: _ilap: Every solution has its own drawbacks and benefits

@_ilap,thank you for your answer! As always it is a true pleasure to discuss implementation details with you!

_ilap: _ilap: These files are created at the installation time

Sorry, I didn’t yet had time to check the actual code of this. But how is it possible to create these files at installation time, if there should be a file per block, and until you’re downloading a chain you don’t know how much blocks there are at all?

_ilap: _ilap: and when the blocks are downloaded, then the global utxo must be create

I hoped that “global” UTxO-index shoulda be used (for all addresses), but didn’t know if it’s implemented in reality.

_ilap: _ilap: I think, the wallet restoration is a quite tricky and time-consuming process, cos when the root key-pair is created from the seeds then that HD Wallet can have 2*2^31 (hardened, unhardened) addresses. During the restoration, we need to go through on all utxos and check each individual address whether it belongs to that HD wallet or not. It’s theoretically, in the worst case is O(m * n), where m the number of unspent addresses in the utxos and n is 4 billion.

It should definitely not take O(m * n) , now that would be mad to iterate thru all possible 2^64 wallet addresses

If I understand it correctly, as described in the HD address payload doc - they are storing the encrypted “hierarchy path” of the address in the address itself, so when Daedalus has the private key of the wallet (at restoration) - it can try to decrypt the address, and if it’s decrypts successfully - then it means that this address belongs to this wallet.

So address check takes constant time, and the whole restoration should take only O(m)

_ilap: _ilap: So, I do not think that it reads the blocks during the wallet restoration, but only the RocskDB’s utxo database, but I am not sure. I will check this theory when I get home.

If this is really the case - then my ramble is, sadly, wrong

Sadly, because it would mean that the O(m) check of all positive balances itself (without I\O overhead) takes this much time at restoration, and it would mean that there’s not that much algorithmic room for optimisation, as I would hope.

In that case, we would have to just accept that at all times full blockchain contains hella lot of active addresses (with positive balances) and even linear complexity iteration of those takes ages

Thank you, again, for keeping the discussion open and interesting! I will need to look into the details too, but I hope you won’t withhold any interesting findings

Previously I thought, epoch-snapshots (afaik, planned) could help speed-up the restoration process, but now I’m not that sure about it, since, as I understand, it would only help to create the UTxO index faster, but the eventual scan of this index would take the same time.