Bitcoin Core’s Development Priorities in 2023

Developing for Bitcoin Core can be a difficult path. Developers often forgo high paying software engineering jobs in order to advance the state of bitcoin technology. Additionally, the long delivery cycles can be demotivating; it takes years of effort to get a large feature or upgrade merged into the code base. If the feature involves a consensus change then it must clear the almost impossibly high bar of a soft fork activation.

But being a Bitcoin Core developer can be very rewarding as well. For most, the prospect of advancing the state of freedom technology is highly motivating. Some Bitcoin Core developers have also shared that they are attracted to the intellectual challenge of doing meaningful work in a scientific domain that solves real world problems. They seek the opportunity to collaborate with an interesting and diverse group of peers who self organize according to meritocratic principles.

Most of these developers pursue open source grants, which incentivizes them to commit to a particular project or area of focus in order to “pitch themselves” to grant committees. In order for a developer to get a code change merged, other developers must understand, test, and approve of the change. Now consider that Bitcoin Core developers are an amorphous, leaderless group. Each developer pursues their own goals and they loosely organize according to the principles of voluntarism. Given this, it can be very difficult to coordinate around large projects and features. What’s more, there is a slow but steady trickle of contributors leaving for other projects or pursuits. In order for bitcoin to grow and thrive as the global monetary layer it needs to attract more developers than it loses and Bitcoin Core serves as a leading indicator in this regard.

Starting this year at an in person meeting a group of developers began discussing these issues. The discussion continued in the weekly IRC meetings and, judging from the results, it has been very fruitful. This group started an initiative for the upcoming Bitcoin Core v26.0 release to prioritize a few large projects that will have an outsized impact on the scalability, flexibility, privacy, and usability of the bitcoin node software. Each project has a dedicated owner who is responsible for advancing the project with check-ins at the weekly IRC meetings and dedicated tracking issues in github. These developers are cognizant of the maintenance work that goes into keeping Bitcoin Core running smoothly: attestations of hard-coded data values, security backports, website changes, package management, and release process work. This maintenance work cannot be deprioritized but developers have agreed to pull more of their time and focus away from individual priorities to push forward four large and impactful projects.

Package relay aims to correct a fundamental problem with the design of the mempool. If you aren’t familiar with the mempool, think of it as a storage area for unconfirmed bitcoin transactions. Each bitcoin node sends and receives new transactions to and from other nodes on the network. When a node receives a new transaction it validates the transaction using a set of rules called mempool policy, or just policy for short. If the transaction pays a high enough fee and doesn’t break any of the policy rules, the node will store it in memory and forward it to other nodes it is peered with. These policy rules have to be carefully balanced to prevent DoS (denial of service) attacks on the bitcoin network.

The primary problem with the current design is that it considers each bitcoin transaction individually for inclusion in the mempool. This worked fine in the early days of bitcoin before any off-chain protocols saw significant usage. With the growth of lightning and continuing research into other off-chain protocols the cracks in this system are starting to show.

The security of lightning relies on node operators being able to get a transaction confirmed on chain within a limited time frame, even against the wishes of a motivated adversary. In an ideal mempool, a bitcoin user should be able to have their transaction confirmed in time simply by paying an appropriate fee. In trustless off-chain protocols such as lightning when a peer goes offline, or worse, if they turn malicious, the only way to unilaterally increase the fee of a presigned transaction is to use child pays for parent (CPFP) fee bumping. This requires the user to create a child transaction that spends an output from an unconfirmed parent transaction.

Since the mempool only considers one transaction at a time for inclusion, this opens a lot of attack vectors. A savvy adversary can play games against their channel partner using the convoluted DoS-prevention policy rules. This is called transaction pinning because the attackers' goal is to 'pin' the transaction in the mempool at a low fee rate, in a way where their channel peer can’t bump the fee. If a transaction is successfully pinned, it will be stuck in the mempool until time runs out. If these games are successful then a malicious lightning node operator could steal from their channel peer. This is clearly a big problem for a 'trustless' protocol.

Package relay is a redesign of the mempool policy logic and the peer-to-peer messaging protocol to allow multiple related transactions (a package) to be relayed and considered together for inclusion in the mempool. This is a much needed overhaul that will fundamentally strengthen the security guarantees of lightning and similar off-chain protocols. It doesn’t completely solve the pinning problem, however. There are two more upgrades under discussion (version three transactions and ephemeral anchors) that build on package relay to eliminate even more pinning vectors.

Package relay is an ambitious project--the mempool inclusion rules are complicated, conservative, and thorough for good reason; they protect the entire bitcoin network from being DoS attacked. It is not a simple task to update mempool policy to apply equally well to a package of transactions. Fortunately, we have a crack team on the job. Gloria Zhao, the Bitcoin Core maintainer focused on Mempool and Policy is the owner of this project.

Software libraries are a powerful tool to create modular software stacks with strictly defined interfaces. Libbitcoinkernel is a proposed software library that aims to isolate the bitcoin consensus rules into a software library with its own release cycle. The bitcoin consensus rules determine which bitcoin blocks are accepted by the network as valid and which are rejected. This code is exceedingly difficult to change because bugs or regressions can result in a hard fork in the bitcoin blockchain, which would be disastrous for the stability, reliability, and public confidence in bitcoin.

The consensus rules are currently spread around the codebase and touch a lot of areas that they don’t need to. Centralizing these rules into a dedicated library that can be segregated from the rest of the codebase will open the door to far more development and experimentation with bitcoin node software.

Libbitcoinkernel aims to ensure a high degree of confidence in the consensus logic, allowing node and wallet developers to focus on new design concepts and features that make different tradeoffs. For example, some bitcoin developers have expressed interest in writing a node implementation using rust, a very popular and modern systems programming language. Some Bitcoin Core developers are interested in designing a new peer-to-peer messaging stack, or a different wallet, or implementing a bitcoin node with no wallet. A consensus library will also unlock better security checks and features for hardware and mobile wallets.

Libbitcoinkernel promises to open the door to explore a variety of different bitcoin node implementations and supercharge the development of the single most important piece of software in the bitcoin ecosystem. After being spearheaded by Carl Dong for years, Sebastian Kung has volunteered to become the owner of this project.

BIP 324 is a proposed upgrade to the bitcoin peer-to-peer (P2P) protocol. This is the language bitcoin nodes use to talk to each other over the internet. Currently all P2P traffic is unencrypted and unauthenticated which opens the bitcoin network to a number of security vulnerabilities. It is possible to spy on a node’s P2P traffic to compromise the identity of the node runner. An attacker can use a man-in-the-middle attack to intercept all of a node’s traffic and potentially censor or tamper with the messages that node sends and receives. Without encrypted P2P messages it is not possible to conceal a bitcoin node from network analysis without additional software such as a VPN or a tor service.

BIP 324 is intended to solve these problems by encrypting the P2P messages and adding random obfuscations that make it very difficult to identify whether the bitcoin protocol is even being used. The randomness will frustrate attempts to identify nodes through traffic analysis and in the future it could allow bitcoin P2P traffic to mimic other protocols such as HTTPS or SSH. This effort is complicated by the fact that P2P traffic must be backwards compatible and permissionless, which is not possible with an authenticated protocol. To maintain backward compatibility, the new encryption protocol must be opt-in, meaning all nodes will still be able to speak the unencrypted protocol. If two nodes both want to use BIP324 rules they can upgrade their connection, otherwise they will fall back to the old plaintext protocol. Preferential peering is not planned so node runners who do not want to use the new protocol won’t have to.

BIP 324 takes aim at a big hole in the security of the bitcoin network. It is a long awaited and much needed upgrade that will cement bitcoin’s position as the world’s most secure and sovereignty-preserving form of money for centuries to come. Bitcoin Core developer Dhruv Mehta worked on BIP324 for years but recently stepped away from Bitcoin Core development. Now, the inimitable Pieter Wuille is owner of this project. Pieter Wuille created the secp256k1 library and was a critical contributor to both the segwit and taproot network upgrades. On October 3rd 2023 the BIP324 code was merged into the Bitcoin Core codebase meaning that it will be available for use in the next software release, v26.0.

AssumeUTXO is a new feature that aims to decrease the waiting period before you can start to use a new bitcoin node. When you first start a bitcoin node you must download and verify the entire blockchain. Before you can use your new node to transact on the network you need to compute the UTXO set, or the set of “coins” that are able to be spent. Your node does this by going through every bitcoin transaction in the blockchain and ensuring that it follows the rules. This process is called the initial block download, or IBD, and as anyone who runs a node can tell you, it is very time consuming, especially on old or underpowered hardware.

AssumeUTXO is a proposed configuration setting that would allow a new node to import a file containing the UTXO set at a certain block height. The UTXO set changes with each block, so as part of the Bitcoin Core release process a block will be chosen that was known to exist for several months before the projected release date to ensure that there is ample proof of work burying the block. This makes it exceedingly unlikely for the chosen block to be reorganized out of the blockchain. The UTXO set corresponding to the chosen block height will be serialized (or converted) into a file, hashed, and the hash value will be attested to by a number of Bitcoin Core developers and included in the source code. AssumeUTXO will be a feature flag, so the truly paranoid can simply not download the UTXO set file or disable this feature before syncing their node.

This proposal does not include a plan for how to package and distribute the UTXO set. Originally, when assumeUTXO was proposed it was scoped into several phases and this task was relegated to one of the later phases. But it is currently unclear who, if anyone, will own the latter phases of the project.

When a new node starts up with AssumeUTXO enabled the UTXO set file will first be validated against the hash value included in the source code, then it will be loaded into memory and used to sync from the assumeUTXO block height to the most recent bitcoin block. Once the node has synced to the chain tip, it can be used to trustlessly send, receive, or validate new bitcoin transactions. After this, the node will start a background process that rewinds to the start of history and validates the beginning of the blockchain up to the assumeUTXO block height.

With assumeUTXO, initial block download could decrease from weeks or days to hours or minutes, which should encourage many more people to run a bitcoin node. More nodes equals a stronger and more resilient bitcoin network. Bitcoin Core developer James O’Beirne, creator of the OP_VAULT proposal, is the owner of this project. On October 2nd 2023 the final PR was merged into the main code repository, meaning this feature will be available for developer testing in the next release, v26.0.

With two of the four priority projects merged into the bitcoin core master branch for the next release, it looks like this initiative has been a resounding success. The developers seem to agree! They are already voting on priorities for the next release. As a bitcoiner and software developer, watching this loose collective of cypherpunks crush through obstacles, both technical and social, fills me to the brim with conviction. We are winning. We will change the money and, in doing so, we will change the world. Let’s fucking go.

The author would like to thank Murch and especially Fabian Jahr for helping to improve this article.

This article was updated on October 31st to clarify that assumeUTXO will not be available for mainnet use in the next release.