This week’s newsletter requests testing for release candidates of Bitcoin Core and LND, describes a discussion about UTXO snapshots for fast initial syncing of nodes, and provides regular coverage of bech32 sending support and notable merges in popular Bitcoin infrastructure projects.

Action items

● Help test Bitcoin Core 0.18.0 RC3: The third Release Candidate (RC) for the next major version of Bitcoin Core has been released. This may be the final test version, so we encourage organizations and expert users to test it promptly if they want to ensure any possible regressions are caught before release. Please use this issue for reporting feedback.

● Help test LND 0.6-beta RC3: the first, second, and third RCs for the next major version of LND were released this week. Testing by organizations and experienced LN users is encouraged to catch any regressions or serious problems that could affect users of the final release. Open a new issue if you discover any problems.

News

● Discussion about an assumed-valid mechanism for UTXO snapshots: When Bitcoin Core developers are preparing a new major release, one developer selects the hash of a recent block on the best block chain. Other well-known contributors check their personal nodes and ensure that hash is indeed part of the best block chain, and then add that hash to the software as the “assumed valid” block. When new users start Bitcoin Core for the first time, the program then defaults to skipping script evaluation in all transactions included in blocks before the assumed valid block. The program still keeps track of the bitcoin ownership changes produced by each transaction in a index called the Unspent Transaction Output (UTXO) set. Although reviewing each historic ownership change still takes time, simply skipping script checking reduces initial sync time by about 80% according to tests. Gregory Maxwell, who implemented the assumed valid feature, has argued that, “Because the validity of a chain history is a simple objective fact, it is [easy] to review this setting.” This week James O’Beirne started a thread on the Bitcoin-Dev mailing list about taking a hash of the UTXO set at a particular block, having multiple well-known contributors verify they get the same hash, and then having freshly installed Bitcoin Core nodes default to using that hash to download the exact same UTXO set. This would allow a newly-started node to skip not only scripts but all block chain data before the assumed valid block, perhaps reducing the time requirements to start a node today by 95% or more (and certainly more as the block chain continues to grow). The verification of older blocks and transactions could then be done in the background after the user is already using their node, eventually giving them the same security as a user who disabled this feature. This is an old idea and is part of the motivation for research into other techniques such as fast updatable UTXO commitments and automatic levelDB backups. Discussion mainly revolved around whether or not this is a good idea. Arguments in favor of it include it making starting a new node much easier and that it doesn’t seem to change the trust model for anyone who already trusts the peer review of their development team. Arguments against it include a fear that fast initial syncs with an assumed valid UTXO set would disguise the fact that block size increases make complete initial syncing from scratch much more expensive; if block sizes increased too much, it could become impossible for anyone of modest means to ever trustlessly verify Bitcoin’s UTXO state, forcing new users to trust existing users.

Bech32 sending support

Week 4 of 24. Until the second anniversary of the segwit soft fork lock-in on 24 August 2019, the Optech Newsletter will contain this weekly section that provides information to help developers and organizations implement bech32 sending support—the ability to pay native segwit addresses. This doesn’t require implementing segwit yourself, but it does allow the people you pay to access all of segwit’s multiple benefits.

In last week’s newsletter, we used the Python reference library for bech32 to decode an address into a scriptPubKey that you could pay. However, sometimes the user provides an address containing a typo. The code we suggested would detect the typo and ensure you didn’t pay a wrong address, but bech32 is also able to help detect the location of typos for your users. This week, we’ll demonstrate this capability using the Javascript sample code.

The code is written using Node.js-style module inclusion syntax, so the first step is to compile it into code we can use in the browser. For that, we install a browserify tool:

sudo apt install node-browserify-lite

Then we compile it into a standalone file:

browserify-lite ./segwit_addr_ecc.js --outfile bech32-demo.js --standalone segwit_addr_ecc

Followed by including it in our HTML:

<script src= "bech32-demo.js" ></script>

For convenience, we’ve included that file on the web version of this newsletter, so you can follow along with the rest of this example by simply opening the developer console in your web browser. Let’s start by checking a valid address. Recall from last week that we provide the network identifier when checking an address ( bc for Bitcoin mainnet):

>> segwit_addr_ecc.check('bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4', 'bc') error: null program: Array(20) [ 117, 30, 118, … ] version: 0

We see above that, just like last week, we get back the witness version and the witness program. The presence of the version field, plus the lack of an error, indicate that this program decoded without any checksum failure.

Now we replace one character in the above address with a typo and try checking that:

>> segwit_addr_ecc.check('bc1qw508d6qejxtdg4y5r4zarvary0c5xw7kv8f3t4', 'bc') error: "Invalid" pos: Array [ 21 ]

This time we get back the description of the error (the address is invalid because it doesn’t match its checksum) and a position. If we place the addresses above each other with each position marked, we see that this “21” identifies the location of the specific error:

1x 2x 0123456789012345678901 >> good='bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4' >> typo='bc1qw508d6qejxtdg4y5r4zarvary0c5xw7kv8f3t4' ^

What if we make an additional replacement to the typo address and try again?

>> segwit_addr_ecc.check('bc1qw508d6qejxtdg4y5r4zarvary0c5yw7kv8f3t4', 'bc') error: "Invalid" pos: Array [ 32, 21 ]

We get two locations. Once again, when we compare the addresses to each other, we see this has identified both incorrect characters:

1x 2x 3x 012345678901234567890123456789012 >> good='bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4' >> typo='bc1qw508d6qejxtdg4y5r4zarvary0c5yw7kv8f3t4' ^ ^

Pieter Wuille’s interactive demo of this Javascript code includes a few lines of additional code (view source on that page to see the function) that uses the position of the typo characters to emphasize them in red:

There’s a limit to how many errors the check() function can specifically identify. After that it can still tell that an address contains an error, but it can’t identify where to look in the address for the error. In that case, it’ll still return the address as invalid but it won’t return the position details:

>> segwit_addr_ecc.check('bc1qw508z6qejxtdg4y5r4zarvary0c5yw7kv8f3t4', 'bc') error: "Invalid" pos: null

In the case where there are other problems with the address, the error field will be set to a more descriptive message that may or may not include a position of the error. For example:

>> segwit_addr_ecc.check('bc1zw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4yolo', 'bc') error: "Invalid character" pos: Array [ 43 ]

You can review the source for a complete list of errors.

Although we spent a lot of time looking at errors in this mini tutorial, we’ve hopefully shown how easy it is to provide nice interactive feedback to users entering bech32 addresses on a web-based platform. We encourage you to play around with the interactive demo to get an idea of what your users might see if you make use of this bech32 address feature.

Notable code and documentation changes

Notable changes this week in Bitcoin Core, LND, C-Lightning, Eclair, libsecp256k1, and Bitcoin Improvement Proposals (BIPs). Note: all merges described for Bitcoin Core and LND are to their master development branches; some may also be backported to their pending releases.