Messages

Once two nodes are connected, they talk in short, typed messages: “here’s a new block”, “send me that transaction”, “what headers do you have after this one?”. Pearl’s vocabulary is btcd’s: Bitcoin’s, word for word, minus the compact-block messages. No new message types were added.

What changed is what some messages carry. Pearl blocks travel behind a ~60 KB certificate, and nodes check those certificates while syncing, so the messages that carry headers had to be redesigned around them.

How is a message packaged?

Every message travels inside a BIP-324 encrypted packet. Inside the encryption, a message is just two things:

1. Which message it is. Usually one byte, a short ID: 2 is block, 12 is getheaders, 13 is headers, 14 is inv, 21 is tx. A few rarer messages use a 00 byte followed by their name spelled out in 12 bytes: version, verack, getaddr, sendheaders, sendaddrv2, wtxidrelay, reject.
2. The payload, in the same byte format as Bitcoin’s, except for the messages below.

Bitcoin’s older v1 header, with magic bytes, a command name, a length and a checksum, is never used: pearld’s peer code always reads and writes the encrypted v2 form.

How does a conversation start?

Right after the encrypted handshake, each side sends a version message: its protocol version (2), its services, its user agent. Each answers the other’s with verack. That’s the version handshake.

pearld refuses peers whose protocol version is below 2. The protocol also has a few optional preference messages: sendheaders (announce new blocks as headers), sendaddrv2 (use the newer address format), wtxidrelay (name transactions by wtxid), feefilter (don’t send me transactions cheaper than this). pearld itself sends only sendaddrv2 during the handshake, but it understands the others.

How do new blocks and transactions spread?

By announcement, so nobody receives the same data twice:

1. Announce

   A node that has something new sends an inv listing hashes: “I have block X, transactions Y and Z.”

2. Ask

   A peer that lacks one replies with getdata for those hashes.

3. Send

   The node sends the block or tx. If it no longer has it, notfound.

To peers that asked for it with sendheaders, pearld announces a new block as a one-entry headers message instead, and that announcement carries the block’s certificate. (pearld never asks for this itself, so between pearld nodes new blocks are announced with inv.)

There are no compact blocks. Bitcoin’s BIP-152 lets a node send a block as short transaction IDs that the peer rebuilds from its own mempool. Pearl’s message table simply leaves those IDs out (sendcmpct, cmpctblock, getblocktxn, blocktxn), so blocks always travel whole.

How does a new node download the chain?

Headers first, with certificates, a checkpoint at a time. This is headers-first sync.

The node sends getheaders. Its payload has Bitcoin’s fields, plus one more byte at the end:

Here is a real-format getheaders, built with Pearl’s own wire code (not captured from the network): “I only have genesis; send what comes next, with certificates.” 70 bytes:

The peer answers with a headers message: a count, then up to 100 entries, each a certificate followed by a 108-byte header. Without certificates, each entry starts with 00000000, the empty “version 0” certificate. Here is a one-header reply carrying the genesis header without its certificate, 113 bytes:

With the genesis block’s (placeholder) certificate included, the same reply is 317 bytes. With real ones it’s much bigger:

That’s why the cap is 100, not Bitcoin’s 2,000. A single headers message can’t mix entries with and without certificates.

What does a syncing node do with each header?

It verifies it, fully, before asking for the next batch:

1. The header must link to the previous one; otherwise the peer is disconnected.
2. The certificate must follow the rules for that height: right version, dense-only, rank penalty.
3. The header and certificate must pass the sanity checks, including the full zero-knowledge proof.

This repeats up to the next checkpoint; then the node fetches the blocks. Past the last checkpoint it switches to ordinary inv and getdata. Two safety rules round it off. Outside headers-first sync, a peer that sends headers carrying certificates is disconnected. And when a low-quality peer announces a block, the node first probes with a cheap certificate-less getheaders, and only fetches the block if the answer checks out.

What does a block message look like?

Exactly like a block on disk, certificate first:

The header and transactions may total 4,000,000 bytes; with a certificate of up to 65,000 bytes in front, a block message tops out at 4,065,000 bytes. The full layout, byte by byte, is on blocks.

merkleblock, the reply to a BIP-37 bloom-filter client, carries the 108-byte header without its certificate. A client fed only merkleblocks can’t check the proof of work for itself. Pearl’s own light client uses compact block filters and fetches headers with certificates instead.

// 2000 is too many for our Block Header size, so we limit it to 100.
const MaxBlockHeadersPerMsg = 100

type MsgHeader struct {
	MsgCertificate MsgCertificate
	BlockHeader    BlockHeader
}