bolt7

Routing gossip protocol, per BOLT #7.
git clone git://git.ppad.tech/bolt7.git
Log | Files | Refs | README | LICENSE

commit e1c410743f7c24488045c7ba8c6347c84c45c8e8
parent b93640a812a1f57867b89299ce9c08179804a20b
Author: Jared Tobin <jared@jtobin.io>
Date:   Sun, 25 Jan 2026 14:43:40 +0400

Add implementation plan and cache BOLT #7 spec

- plans/IMPL1.md: 8-phase implementation plan covering types, codec,
  signature hashing, validation, tests, benchmarks, and docs
- etc/07-routing-gossip.md: cached BOLT #7 specification

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Diffstat:
Aetc/07-routing-gossip.md | 1123+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Aplans/IMPL1.md | 240+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 1363 insertions(+), 0 deletions(-)

diff --git a/etc/07-routing-gossip.md b/etc/07-routing-gossip.md @@ -0,0 +1,1123 @@ +# BOLT #7: P2P Node and Channel Discovery + +This specification describes simple node discovery, channel discovery, and channel update mechanisms that do not rely on a third-party to disseminate the information. + +Node and channel discovery serve two different purposes: + + - Node discovery allows nodes to broadcast their ID, host, and port, so that other nodes can open connections and establish payment channels with them. + - Channel discovery allows the creation and maintenance of a local view of the network's topology, so that a node can discover routes to desired destinations. + +To support channel and node discovery, three *gossip messages* are supported: + +- For node discovery, peers exchange `node_announcement` + messages, which supply additional information about the nodes. There may be + multiple `node_announcement` messages, in order to update the node information. + + - For channel discovery, peers in the network exchange + `channel_announcement` messages containing information regarding new + channels between the two nodes. They can also exchange `channel_update` + messages, which update information about a channel. There can only be + one valid `channel_announcement` for any channel, but at least two + `channel_update` messages are expected. + +# Table of Contents + + * [Definition of `short_channel_id`](#definition-of-short_channel_id) + * [The `announcement_signatures` Message](#the-announcement_signatures-message) + * [The `channel_announcement` Message](#the-channel_announcement-message) + * [The `node_announcement` Message](#the-node_announcement-message) + * [The `channel_update` Message](#the-channel_update-message) + * [Query Messages](#query-messages) + * [Rebroadcasting](#rebroadcasting) + * [HTLC Fees](#htlc-fees) + * [Pruning the Network View](#pruning-the-network-view) + * [Recommendations for Routing](#recommendations-for-routing) + * [References](#references) + +## Definition of `short_channel_id` + +The `short_channel_id` is the unique description of the funding transaction. +It is constructed as follows: + 1. the most significant 3 bytes: indicating the block height + 2. the next 3 bytes: indicating the transaction index within the block + 3. the least significant 2 bytes: indicating the output index that pays to the channel. + +The standard human readable format for `short_channel_id` is created +by printing the above components, in the order: +block height, transaction index, and output index. +Each component is printed as a decimal number, +and separated from each other by the small letter `x`. +For example, a `short_channel_id` might be written as `539268x845x1`, +indicating a channel on the output 1 of the transaction at index 845 +of the block at height 539268. + +### Rationale + +The `short_channel_id` human readable format is designed +so that double-clicking or double-tapping it will select the entire ID +on most systems. +Humans prefer decimal when reading numbers, +so the ID components are written in decimal. +The small letter `x` is used since on most fonts, +the `x` is visibly smaller than decimal digits, +making it easy to visibly group each component of the ID. + +## The `announcement_signatures` Message + +This is a direct message between the two endpoints of a channel and serves as an opt-in mechanism to allow the announcement of the channel to the rest of the network. +It contains the necessary signatures, by the sender, to construct the `channel_announcement` message. + +1. type: 259 (`announcement_signatures`) +2. data: + * [`channel_id`:`channel_id`] + * [`short_channel_id`:`short_channel_id`] + * [`signature`:`node_signature`] + * [`signature`:`bitcoin_signature`] + +The willingness of the initiating node to announce the channel is signaled during channel opening by setting the `announce_channel` bit in `channel_flags` (see [BOLT #2](02-peer-protocol.md#the-open_channel-message)). + +### Requirements + +The `announcement_signatures` message is created by constructing a `channel_announcement` message, +corresponding to the newly confirmed channel funding transaction, and signing it with the secrets +matching an endpoint's `node_id` and `bitcoin_key`. + +A node: + - If the `open_channel` message has the `announce_channel` bit set AND a `shutdown` message has not been sent: + - After `channel_ready` has been sent and received AND the funding transaction has enough confirmations to ensure that it won't be reorganized: + - MUST send `announcement_signatures` for the funding transaction. + - Otherwise: + - MUST NOT send the `announcement_signatures` message. + - Upon reconnection (once the above timing requirements have been met): + - If it has NOT previously received `announcement_signatures` for the funding transaction: + - MUST send its own `announcement_signatures` message. + - If it receives `announcement_signatures` for the funding transaction: + - MUST respond with its own `announcement_signatures` message. + +A recipient node: + - If the `short_channel_id` is NOT correct: + - SHOULD send a `warning` and close the connection, or send an + `error` and fail the channel. + - If the `node_signature` OR the `bitcoin_signature` is NOT correct: + - MAY send a `warning` and close the connection, or send an + `error` and fail the channel. + - If it has sent AND received a valid `announcement_signatures` message: + - If the funding transaction has at least 6 confirmations: + - SHOULD queue the `channel_announcement` message for its peers. + - If it has not sent `channel_ready`: + - SHOULD defer handling the `announcement_signatures` until after it has sent `channel_ready`. + +### Rationale + +Channels must not be announced before the funding transaction has enough +confirmations, because a blockchain reorganization would otherwise invalidate +the `short_channel_id`. + +## The `channel_announcement` Message + +This gossip message contains ownership information regarding a channel. It ties +each on-chain Bitcoin key to the associated Lightning node key, and vice-versa. +The channel is not practically usable until at least one side has announced +its fee levels and expiry, using `channel_update`. + +Proving the existence of a channel between `node_1` and `node_2` requires: + +1. proving that the funding transaction pays to `bitcoin_key_1` and + `bitcoin_key_2` +2. proving that `node_1` owns `bitcoin_key_1` +3. proving that `node_2` owns `bitcoin_key_2` + +Assuming that all nodes know the unspent transaction outputs, the first proof is +accomplished by a node finding the output given by the `short_channel_id` and +verifying that it is indeed a P2WSH funding transaction output for those keys +specified in [BOLT #3](03-transactions.md#funding-transaction-output). + +The last two proofs are accomplished through explicit signatures: +`bitcoin_signature_1` and `bitcoin_signature_2` are generated for each +`bitcoin_key` and each of the corresponding `node_id`s are signed. + +It's also necessary to prove that `node_1` and `node_2` both agree on the +announcement message: this is accomplished by having a signature from each +`node_id` (`node_signature_1` and `node_signature_2`) signing the message. + +1. type: 256 (`channel_announcement`) +2. data: + * [`signature`:`node_signature_1`] + * [`signature`:`node_signature_2`] + * [`signature`:`bitcoin_signature_1`] + * [`signature`:`bitcoin_signature_2`] + * [`u16`:`len`] + * [`len*byte`:`features`] + * [`chain_hash`:`chain_hash`] + * [`short_channel_id`:`short_channel_id`] + * [`point`:`node_id_1`] + * [`point`:`node_id_2`] + * [`point`:`bitcoin_key_1`] + * [`point`:`bitcoin_key_2`] + +### Requirements + +The origin node: + - MUST set `chain_hash` to the 32-byte hash that uniquely identifies the chain + that the channel was opened within: + - for the _Bitcoin blockchain_: + - MUST set `chain_hash` value (encoded in hex) equal to `6fe28c0ab6f1b372c1a6a246ae63f74f931e8365e15a089c68d6190000000000`. + - MUST set `short_channel_id` to refer to the confirmed funding transaction, + as specified in [BOLT #2](02-peer-protocol.md#the-channel_ready-message). + - Note: the corresponding output MUST be a P2WSH, as described in [BOLT #3](03-transactions.md#funding-transaction-output). + - MUST set `node_id_1` and `node_id_2` to the public keys of the two nodes + operating the channel, such that `node_id_1` is the lexicographically-lesser of the + two compressed keys sorted in ascending lexicographic order. + - MUST set `bitcoin_key_1` and `bitcoin_key_2` to `node_id_1` and `node_id_2`'s + respective `funding_pubkey`s. + - MUST compute the double-SHA256 hash `h` of the message, beginning at offset + 256, up to the end of the message. + - Note: the hash skips the 4 signatures but hashes the rest of the message, + including any future fields appended to the end. + - MUST set `node_signature_1` and `node_signature_2` to valid + signatures of the hash `h` (using `node_id_1` and `node_id_2`'s respective + secrets). + - MUST set `bitcoin_signature_1` and `bitcoin_signature_2` to valid + signatures of the hash `h` (using `bitcoin_key_1` and `bitcoin_key_2`'s + respective secrets). + - MUST set `features` based on what features were negotiated for this channel, according to [BOLT #9](09-features.md#assigned-features-flags) + - MUST set `len` to the minimum length required to hold the `features` bits + it sets. + - If the funding transaction has less than 6 confirmations: + - MUST NOT send `channel_announcement`. + +The receiving node: + - MUST verify the integrity AND authenticity of the message by verifying the + signatures. + - if there is an unknown even bit in the `features` field: + - MUST NOT attempt to route messages through the channel. + - if the `short_channel_id`'s output does NOT correspond to a P2WSH (using + `bitcoin_key_1` and `bitcoin_key_2`, as specified in + [BOLT #3](03-transactions.md#funding-transaction-output)) OR the output is + spent: + - MUST ignore the message. + - if the specified `chain_hash` is unknown to the receiver: + - MUST ignore the message. + - if the `short_channel_id`'s output does NOT have at least 6 confirmations: + - MAY accept the message if the output is close to 6 confirmations, in case + the receiving node hasn't received the latest block(s) yet. + - otherwise: + - SHOULD ignore the message. + - otherwise: + - if `bitcoin_signature_1`, `bitcoin_signature_2`, `node_signature_1` OR + `node_signature_2` are invalid OR NOT correct: + - SHOULD send a `warning`. + - MAY close the connection. + - MUST ignore the message. + - otherwise: + - if `node_id_1` OR `node_id_2` are blacklisted: + - SHOULD ignore the message. + - otherwise: + - if the transaction referred to was NOT previously announced as a + channel: + - SHOULD queue the message for rebroadcasting. + - MAY choose NOT to for messages longer than the minimum expected + length. + - if it has previously received a valid `channel_announcement`, for the + same transaction, in the same block, but for a different `node_id_1` or + `node_id_2`: + - SHOULD blacklist the previous message's `node_id_1` and `node_id_2`, + as well as this `node_id_1` and `node_id_2` AND forget any channels + connected to them. + - otherwise: + - SHOULD store this `channel_announcement`. + - once its funding output has been spent OR reorganized out: + - SHOULD forget a channel after a 72-block delay. + - SHOULD NOT rebroadcast this `channel_announcement` to its peers. + +### Rationale + +Both nodes are required to sign to indicate they are willing to route other +payments via this channel (i.e. be part of the public network); requiring their +Bitcoin signatures proves that they control the channel. + +The blacklisting of conflicting nodes disallows multiple different +announcements. Such conflicting announcements should never be broadcast by any +node, as this implies that keys have leaked. + +While channels should not be advertised before they are sufficiently deep, the +requirement against rebroadcasting only applies if the transaction has not moved +to a different block. + +In order to avoid storing excessively large messages, yet still allow for +reasonable future expansion, nodes are permitted to restrict rebroadcasting +(perhaps statistically). + +New channel features are possible in the future: backwards compatible (or +optional) features will have _odd_ feature bits, while incompatible features +will have _even_ feature bits +(["It's OK to be odd!"](00-introduction.md#glossary-and-terminology-guide)). + +A delay of 72 blocks is used when forgetting a channel on funding output spend +as to permit a new `channel_announcement` to propagate which indicates this +channel was spliced. + +## The `node_announcement` Message + +This gossip message allows a node to indicate extra data associated with it, in +addition to its public key. To avoid trivial denial of service attacks, +nodes not associated with an already known channel are ignored. + +1. type: 257 (`node_announcement`) +2. data: + * [`signature`:`signature`] + * [`u16`:`flen`] + * [`flen*byte`:`features`] + * [`u32`:`timestamp`] + * [`point`:`node_id`] + * [`3*byte`:`rgb_color`] + * [`32*byte`:`alias`] + * [`u16`:`addrlen`] + * [`addrlen*byte`:`addresses`] + +`timestamp` allows for the ordering of messages, in the case of multiple +announcements. `rgb_color` and `alias` allow intelligence services to assign +nodes colors like black and cool monikers like 'IRATEMONK' and 'WISTFULTOLL'. + +`addresses` allows a node to announce its willingness to accept incoming network +connections: it contains a series of `address descriptor`s for connecting to the +node. The first byte describes the address type and is followed by the +appropriate number of bytes for that type. + +The following `address descriptor` types are defined: + + * `1`: ipv4; data = `[4:ipv4_addr][2:port]` (length 6) + * `2`: ipv6; data = `[16:ipv6_addr][2:port]` (length 18) + * `3`: Deprecated (length 12). Used to contain Tor v2 onion services. + * `4`: Tor v3 onion service; data = `[35:onion_addr][2:port]` (length 37) + * version 3 ([prop224](https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.txt)) + onion service addresses; Encodes: + `[32:32_byte_ed25519_pubkey] || [2:checksum] || [1:version]`, where + `checksum = sha3(".onion checksum" || pubkey || version)[:2]`. + * `5`: DNS hostname; data = `[1:hostname_len][hostname_len:hostname][2:port]` (length up to 258) + * `hostname` bytes MUST be ASCII characters. + * Non-ASCII characters MUST be encoded using Punycode: + https://en.wikipedia.org/wiki/Punycode + +### Requirements + +The origin node: + - MUST set `timestamp` to be greater than that of any previous + `node_announcement` it has previously created. + - MAY base it on a UNIX timestamp. + - MUST set `signature` to the signature of the double-SHA256 of the entire + remaining packet after `signature` (using the key given by `node_id`). + - MAY set `alias` AND `rgb_color` to customize its appearance in maps and + graphs. + - Note: the first byte of `rgb_color` is the red value, the second byte is the + green value, and the last byte is the blue value. + - MUST set `alias` to a valid UTF-8 string, with any `alias` trailing-bytes + equal to 0. + - SHOULD fill `addresses` with an address descriptor for each public network + address that expects incoming connections. + - MUST set `addrlen` to the number of bytes in `addresses`. + - MUST place address descriptors in ascending order. + - SHOULD NOT place any zero-typed address descriptors anywhere. + - SHOULD use placement only for aligning fields that follow `addresses`. + - MUST NOT create a `type 1`, `type 2` or `type 5` address descriptor with + `port` equal to 0. + - SHOULD ensure `ipv4_addr` AND `ipv6_addr` are routable addresses. + - MUST set `features` according to [BOLT #9](09-features.md#assigned-features-flags) + - SHOULD set `flen` to the minimum length required to hold the `features` + bits it sets. + - SHOULD not announce a Tor v2 onion service. + - MUST NOT announce more than one `type 5` DNS hostname. + +The receiving node: + - if `node_id` is NOT a valid compressed public key: + - SHOULD send a `warning`. + - MAY close the connection. + - MUST NOT process the message further. + - if `signature` is NOT a valid signature (using `node_id` of the + double-SHA256 of the entire message following the `signature` field, including +any future fields appended to the end): + - SHOULD send a `warning`. + - MAY close the connection. + - MUST NOT process the message further. + - if `features` field contains _unknown even bits_: + - SHOULD NOT connect to the node. + - Unless paying a [BOLT #11](11-payment-encoding.md) invoice which does not + have the same bit(s) set, MUST NOT attempt to send payments _to_ the node. + - MUST NOT route a payment _through_ the node. + - SHOULD ignore the first `address descriptor` that does NOT match the types + defined above. + - if `addrlen` is insufficient to hold the address descriptors of the + known types: + - SHOULD send a `warning`. + - MAY close the connection. + - if `port` is equal to 0: + - SHOULD ignore `ipv6_addr` OR `ipv4_addr` OR `hostname`. + - if `node_id` is NOT previously known from a `channel_announcement` message, + OR if `timestamp` is NOT greater than the last-received `node_announcement` + from this `node_id`: + - SHOULD ignore the message. + - otherwise: + - if `timestamp` is greater than the last-received `node_announcement` from + this `node_id`: + - SHOULD queue the message for rebroadcasting. + - MAY choose NOT to queue messages longer than the minimum expected length. + - MAY use `rgb_color` AND `alias` to reference nodes in interfaces. + - SHOULD insinuate their self-signed origins. + - SHOULD ignore Tor v2 onion services. + - if more than one `type 5` address is announced: + - SHOULD ignore the additional data. + - MUST not forward the `node_announcement`. + +### Rationale + +New node features are possible in the future: backwards compatible (or +optional) ones will have _odd_ `feature` _bits_, incompatible ones will have +_even_ `feature` _bits_. These will be propagated normally; incompatible +feature bits here refer to the nodes, not the `node_announcement` message +itself. + +New address types may be added in the future; as address descriptors have +to be ordered in ascending order, unknown ones can be safely ignored. +Additional fields beyond `addresses` may also be added in the future—with +optional padding within `addresses`, if they require certain alignment. + +### Security Considerations for Node Aliases + +Node aliases are user-defined and provide a potential avenue for injection +attacks, both during the process of rendering and during persistence. + +Node aliases should always be sanitized before being displayed in +HTML/Javascript contexts or any other dynamically interpreted rendering +frameworks. Similarly, consider using prepared statements, input validation, +and escaping to protect against injection vulnerabilities and persistence +engines that support SQL or other dynamically interpreted querying languages. + +* [Stored and Reflected XSS Prevention](https://www.owasp.org/index.php/XSS_(Cross_Site_Scripting)_Prevention_Cheat_Sheet) +* [DOM-based XSS Prevention](https://www.owasp.org/index.php/DOM_based_XSS_Prevention_Cheat_Sheet) +* [SQL Injection Prevention](https://www.owasp.org/index.php/SQL_Injection_Prevention_Cheat_Sheet) + +Don't be like the school of [Little Bobby Tables](https://xkcd.com/327/). + +## The `channel_update` Message + +After a channel has been initially announced, each side independently +announces the fees and minimum expiry delta it requires to relay HTLCs +through this channel. Each uses the 8-byte channel shortid that matches the +`channel_announcement` and the 1-bit `channel_flags` field to indicate which end of the +channel it's on (origin or final). A node can do this multiple times, in +order to change fees. + +Note that the `channel_update` gossip message is only useful in the context +of *relaying* payments, not *sending* payments. When making a payment + `A` -> `B` -> `C` -> `D`, only the `channel_update`s related to channels + `B` -> `C` (announced by `B`) and `C` -> `D` (announced by `C`) will + come into play. When building the route, amounts and expiries for HTLCs need + to be calculated backward from the destination to the source. The exact initial + value for `amount_msat` and the minimal value for `cltv_expiry`, to be used for + the last HTLC in the route, are provided in the payment request + (see [BOLT #11](11-payment-encoding.md#tagged-fields)). + +1. type: 258 (`channel_update`) +2. data: + * [`signature`:`signature`] + * [`chain_hash`:`chain_hash`] + * [`short_channel_id`:`short_channel_id`] + * [`u32`:`timestamp`] + * [`byte`:`message_flags`] + * [`byte`:`channel_flags`] + * [`u16`:`cltv_expiry_delta`] + * [`u64`:`htlc_minimum_msat`] + * [`u32`:`fee_base_msat`] + * [`u32`:`fee_proportional_millionths`] + * [`u64`:`htlc_maximum_msat`] + +The `channel_flags` bitfield is used to indicate the direction of the channel: it +identifies the node that this update originated from and signals various options +concerning the channel. The following table specifies the meaning of its +individual bits: + +| Bit Position | Name | Meaning | +| ------------- | ----------- | -------------------------------- | +| 0 | `direction` | Direction this update refers to. | +| 1 | `disable` | Disable the channel. | + +The `message_flags` bitfield is used to provide additional details about the message: + +| Bit Position | Name | +| ------------- | ---------------| +| 0 | `must_be_one` | +| 1 | `dont_forward` | + +The `node_id` for the signature verification is taken from the corresponding +`channel_announcement`: `node_id_1` if the least-significant bit of flags is 0 +or `node_id_2` otherwise. + +### Requirements + +The origin node: + - MUST NOT send a created `channel_update` before `channel_ready` has been received. + - MAY create a `channel_update` to communicate the channel parameters to the + channel peer, even though the channel has not yet been announced (i.e. + the `announce_channel` bit was not set or the `channel_update` is sent before + the peers exchanged [announcement signatures](#the-announcement_signatures-message)). + - MUST set the `short_channel_id` to either an `alias` it has + received from the peer, or the real channel `short_channel_id`. + - MUST set `dont_forward` to 1 in `message_flags` + - MUST NOT forward such a `channel_update` to other peers, for privacy + reasons. + - Note: such a `channel_update`, one not preceded by a + `channel_announcement`, is invalid to any other peer and would be discarded. + - MUST set `signature` to the signature of the double-SHA256 of the entire + remaining packet after `signature`, using its own `node_id`. + - MUST set `chain_hash` AND `short_channel_id` to match the 32-byte hash AND + 8-byte channel ID that uniquely identifies the channel specified in the + `channel_announcement` message. + - if the origin node is `node_id_1` in the message: + - MUST set the `direction` bit of `channel_flags` to 0. + - otherwise: + - MUST set the `direction` bit of `channel_flags` to 1. + - MUST set `htlc_maximum_msat` to the maximum value it will send through this channel for a single HTLC. + - MUST set this to less than or equal to the channel capacity. + - MUST set this to less than or equal to `max_htlc_value_in_flight_msat` it received from the peer. + - MUST set this to greater than or equal to `htlc_minimum_msat`. + - MUST set `must_be_one` in `message_flags` to 1. + - MUST set bits in `channel_flags` and `message_flags` that are not assigned a meaning to 0. + - MAY create and send a `channel_update` with the `disable` bit set to 1, to + signal a channel's temporary unavailability (e.g. due to a loss of + connectivity) OR permanent unavailability (e.g. prior to an on-chain + settlement). + - MAY send a subsequent `channel_update` with the `disable` bit set to 0 to + re-enable the channel. + - MUST set `timestamp` to greater than 0, AND to greater than any + previously-sent `channel_update` for this `short_channel_id`. + - SHOULD base `timestamp` on a UNIX timestamp. + - MUST set `cltv_expiry_delta` to the number of blocks it will subtract from + an incoming HTLC's `cltv_expiry`. + - MUST set `htlc_minimum_msat` to the minimum HTLC value (in millisatoshi) + that the channel peer will accept. + - MUST set `htlc_minimum_msat` to less than or equal to `htlc_maximum_msat`. + - MUST set `fee_base_msat` to the base fee (in millisatoshi) it will charge + for any HTLC. + - MUST set `fee_proportional_millionths` to the amount (in millionths of a + satoshi) it will charge per transferred satoshi. + - SHOULD NOT create redundant `channel_update`s + - If it creates a new `channel_update` with updated channel parameters: + - SHOULD keep accepting the previous channel parameters for 10 minutes + +The receiving node: + - if the `short_channel_id` does NOT match a previous `channel_announcement`: + - MUST ignore `channel_update`s that do NOT correspond to one of its own + channels. + - if the channel output has been spent: + - MUST ignore `channel_update`s, unless they have the `disable` bit set to 1. + - SHOULD NOT rebroadcast `channel_update`s to its peers, unless they have the + `disable` bit set to 1. + - SHOULD accept `channel_update`s for its own channels (even if non-public), + in order to learn the associated origin nodes' forwarding parameters. + - if `signature` is not a valid signature, using `node_id` of the + double-SHA256 of the entire message following the `signature` field (including + unknown fields following `fee_proportional_millionths`): + - SHOULD send a `warning` and close the connection. + - MUST NOT process the message further. + - if the specified `chain_hash` value is unknown (meaning it isn't active on + the specified chain): + - MUST ignore the channel update. + - if the `timestamp` is equal to the last-received `channel_update` for this + `short_channel_id` AND `node_id`: + - if the fields below `timestamp` differ: + - MAY blacklist this `node_id`. + - MAY forget all channels associated with it. + - if the fields below `timestamp` are equal: + - SHOULD ignore this message + - if `timestamp` is lower than that of the last-received + `channel_update` for this `short_channel_id` AND for `node_id`: + - SHOULD ignore the message. + - otherwise: + - if the `timestamp` is unreasonably far in the future: + - MAY discard the `channel_update`. + - otherwise: + - SHOULD queue the message for rebroadcasting. + - MAY choose NOT to for messages longer than the minimum expected length. + - if `htlc_maximum_msat` < `htlc_minimum_msat`: + - SHOULD ignore this channel during route considerations. + - if `htlc_maximum_msat` is greater than channel capacity: + - MAY blacklist this `node_id` + - SHOULD ignore this channel during route considerations. + - otherwise: + - SHOULD consider the `htlc_maximum_msat` when routing. + +### Rationale + +The `timestamp` field is used by nodes for pruning `channel_update`s that are +either too far in the future or have not been updated in two weeks; so it +makes sense to have it be a UNIX timestamp (i.e. seconds since UTC +1970-01-01). This cannot be a hard requirement, however, given the possible case +of two `channel_update`s within a single second. + +It is assumed that more than one `channel_update` message changing the channel +parameters in the same second may be a DoS attempt, and therefore, the node responsible +for signing such messages may be blacklisted. However, a node may send a same +`channel_update` message with a different signature (changing the nonce in signature +signing), and hence fields apart from signature are checked to see if the channel +parameters have changed for the same timestamp. It is also important to note that +ECDSA signatures are malleable. So, an intermediate node who received the `channel_update` +message can rebroadcast it just by changing the `s` component of signature with `-s`. +This should however not result in the blacklist of the `node_id` from where +the message originated. + +The recommendation against redundant `channel_update`s minimizes spamming the network, +however it is sometimes inevitable. For example, a channel with a +peer which is unreachable will eventually cause a `channel_update` to +indicate that the channel is disabled, with another update re-enabling +the channel when the peer reestablishes contact. Because gossip +messages are batched and replace previous ones, the result may be a +single seemingly-redundant update. + +When a node creates a new `channel_update` to change its channel parameters, +it will take some time to propagate through the network and payers may use +older parameters. It is recommended to keep accepting older parameters for +at least 10 minutes to improve payment latency and reliability. + +The `must_be_one` field in `message_flags` was previously used to indicate +the presence of the `htlc_maximum_msat` field. This field must now always +be present, so `must_be_one` is a constant value, and ignored by receivers. + +## Query Messages + +Understanding of messages used to be indicated with the `gossip_queries` +feature bit; now these messages are universally supported, that feature has +now been slightly repurposed. Not offering this feature means a node is not +worth querying for gossip: either they do not store the entire gossip map, or +they are only connected to a single peer (this one). + +There are several messages which contain a long array of +`short_channel_id`s (called `encoded_short_ids`) so we include an encoding byte +which allows for different encoding schemes to be defined in the future, if they +provide benefit. + +Encoding types: +* `0`: uncompressed array of `short_channel_id` types, in ascending order. +* `1`: Previously used for zlib compression, this encoding MUST NOT be used. + +This encoding is also used for arrays of other types (timestamps, flags, ...), +and specified with an `encoded_` prefix. For example, `encoded_timestamps` is +an array of timestamps with a `0` prefix. + +Query messages can be extended with optional fields that can help reduce the number of messages needed to synchronize routing tables by enabling: + +- timestamp-based filtering of `channel_update` messages: only ask for `channel_update` messages that are newer than the ones you already have. +- checksum-based filtering of `channel_update` messages: only ask for `channel_update` messages that carry different information from the ones you already have. + +Nodes can signal that they support extended gossip queries with the `gossip_queries_ex` feature bit. + +### The `query_short_channel_ids`/`reply_short_channel_ids_end` Messages + +1. type: 261 (`query_short_channel_ids`) +2. data: + * [`chain_hash`:`chain_hash`] + * [`u16`:`len`] + * [`len*byte`:`encoded_short_ids`] + * [`query_short_channel_ids_tlvs`:`tlvs`] + +1. `tlv_stream`: `query_short_channel_ids_tlvs` +2. types: + 1. type: 1 (`query_flags`) + 2. data: + * [`byte`:`encoding_type`] + * [`...*byte`:`encoded_query_flags`] + +`encoded_query_flags` is an array of bitfields, one bigsize per bitfield, one bitfield for each `short_channel_id`. Bits have the following meaning: + +| Bit Position | Meaning | +| ------------- | ---------------------------------------- | +| 0 | Sender wants `channel_announcement` | +| 1 | Sender wants `channel_update` for node 1 | +| 2 | Sender wants `channel_update` for node 2 | +| 3 | Sender wants `node_announcement` for node 1 | +| 4 | Sender wants `node_announcement` for node 2 | + +Query flags must be minimally encoded, which means that one flag will be encoded with a single byte. + +1. type: 262 (`reply_short_channel_ids_end`) +2. data: + * [`chain_hash`:`chain_hash`] + * [`byte`:`full_information`] + +This is a general mechanism which lets a node query for the +`channel_announcement` and `channel_update` messages for specific channels +(identified via `short_channel_id`s). This is usually used either because +a node sees a `channel_update` for which it has no `channel_announcement` or +because it has obtained previously unknown `short_channel_id`s +from `reply_channel_range`. + +#### Requirements + +The sender: + - SHOULD NOT send this to a peer which does not offer `gossip_queries`. + - MUST NOT send `query_short_channel_ids` if it has sent a previous `query_short_channel_ids` to this peer and not received `reply_short_channel_ids_end`. + - MUST set `chain_hash` to the 32-byte hash that uniquely identifies the chain + that the `short_channel_id`s refer to. + - MUST set the first byte of `encoded_short_ids` to the encoding type. + - MUST encode a whole number of `short_channel_id`s to `encoded_short_ids` + - MAY send this if it receives a `channel_update` for a + `short_channel_id` for which it has no `channel_announcement`. + - SHOULD NOT send this if the channel referred to is not an unspent output. + - MAY include an optional `query_flags`. If so: + - MUST set `encoding_type`, as for `encoded_short_ids`. + - Each query flag is a minimally-encoded bigsize. + - MUST encode one query flag per `short_channel_id`. + +The receiver: + - if the first byte of `encoded_short_ids` is not a known encoding type: + - MAY send a `warning`. + - MAY close the connection. + - if `encoded_short_ids` does not decode into a whole number of `short_channel_id`: + - MAY send a `warning`. + - MAY close the connection. + - if it has not sent `reply_short_channel_ids_end` to a previously received `query_short_channel_ids` from this sender: + - MAY send a `warning`. + - MAY close the connection. + - if the incoming message includes `query_short_channel_ids_tlvs`: + - if `encoding_type` is not a known encoding type: + - MAY send a `warning`. + - MAY close the connection. + - if `encoded_query_flags` does not decode to exactly one flag per `short_channel_id`: + - MAY send a `warning`. + - MAY close the connection. + - MUST respond to each known `short_channel_id`: + - if the incoming message does not include `encoded_query_flags`: + - with a `channel_announcement` and the latest `channel_update` for each end + - MUST follow with any `node_announcement`s for each `channel_announcement` + - otherwise: + - We define `query_flag` for the Nth `short_channel_id` in + `encoded_short_ids` to be the Nth bigsize of the decoded + `encoded_query_flags`. + - if bit 0 of `query_flag` is set: + - MUST reply with a `channel_announcement` + - if bit 1 of `query_flag` is set and it has received a `channel_update` from `node_id_1`: + - MUST reply with the latest `channel_update` for `node_id_1` + - if bit 2 of `query_flag` is set and it has received a `channel_update` from `node_id_2`: + - MUST reply with the latest `channel_update` for `node_id_2` + - if bit 3 of `query_flag` is set and it has received a `node_announcement` from `node_id_1`: + - MUST reply with the latest `node_announcement` for `node_id_1` + - if bit 4 of `query_flag` is set and it has received a `node_announcement` from `node_id_2`: + - MUST reply with the latest `node_announcement` for `node_id_2` + - SHOULD NOT wait for the next outgoing gossip flush to send these. + - SHOULD avoid sending duplicate `node_announcements` in response to a single `query_short_channel_ids`. + - MUST follow these responses with `reply_short_channel_ids_end`. + - if does not maintain up-to-date channel information for `chain_hash`: + - MUST set `full_information` to 0. + - otherwise: + - SHOULD set `full_information` to 1. + +#### Rationale + +Future nodes may not have complete information; they certainly won't have +complete information on unknown `chain_hash` chains. While this `full_information` +field (previously and confusingly called `complete`) cannot be trusted, a 0 does indicate that the sender should search +elsewhere for additional data. + +The explicit `reply_short_channel_ids_end` message means that the receiver can +indicate it doesn't know anything, and the sender doesn't need to rely on +timeouts. It also causes a natural ratelimiting of queries. + +### The `query_channel_range` and `reply_channel_range` Messages + +1. type: 263 (`query_channel_range`) +2. data: + * [`chain_hash`:`chain_hash`] + * [`u32`:`first_blocknum`] + * [`u32`:`number_of_blocks`] + * [`query_channel_range_tlvs`:`tlvs`] + +1. `tlv_stream`: `query_channel_range_tlvs` +2. types: + 1. type: 1 (`query_option`) + 2. data: + * [`bigsize`:`query_option_flags`] + +`query_option_flags` is a bitfield represented as a minimally-encoded bigsize. Bits have the following meaning: + +| Bit Position | Meaning | +| ------------- | ----------------------- | +| 0 | Sender wants timestamps | +| 1 | Sender wants checksums | + +Though it is possible, it would not be very useful to ask for checksums without asking for timestamps too: the receiving node may have an older `channel_update` with a different checksum, asking for it would be useless. And if a `channel_update` checksum is actually 0 (which is quite unlikely) it will not be queried. + +1. type: 264 (`reply_channel_range`) +2. data: + * [`chain_hash`:`chain_hash`] + * [`u32`:`first_blocknum`] + * [`u32`:`number_of_blocks`] + * [`byte`:`sync_complete`] + * [`u16`:`len`] + * [`len*byte`:`encoded_short_ids`] + * [`reply_channel_range_tlvs`:`tlvs`] + +1. `tlv_stream`: `reply_channel_range_tlvs` +2. types: + 1. type: 1 (`timestamps_tlv`) + 2. data: + * [`byte`:`encoding_type`] + * [`...*byte`:`encoded_timestamps`] + 1. type: 3 (`checksums_tlv`) + 2. data: + * [`...*channel_update_checksums`:`checksums`] + +For a single `channel_update`, timestamps are encoded as: + +1. subtype: `channel_update_timestamps` +2. data: + * [`u32`:`timestamp_node_id_1`] + * [`u32`:`timestamp_node_id_2`] + +Where: +* `timestamp_node_id_1` is the timestamp of the `channel_update` for `node_id_1`, or 0 if there was no `channel_update` from that node. +* `timestamp_node_id_2` is the timestamp of the `channel_update` for `node_id_2`, or 0 if there was no `channel_update` from that node. + +For a single `channel_update`, checksums are encoded as: + +1. subtype: `channel_update_checksums` +2. data: + * [`u32`:`checksum_node_id_1`] + * [`u32`:`checksum_node_id_2`] + +Where: +* `checksum_node_id_1` is the checksum of the `channel_update` for `node_id_1`, or 0 if there was no `channel_update` from that node. +* `checksum_node_id_2` is the checksum of the `channel_update` for `node_id_2`, or 0 if there was no `channel_update` from that node. + +The checksum of a `channel_update` is the CRC32C checksum as specified in [RFC3720](https://tools.ietf.org/html/rfc3720#appendix-B.4) of this `channel_update` without its `signature` and `timestamp` fields. + +This allows querying for channels within specific blocks. + +#### Requirements + +The sender of `query_channel_range`: + - SHOULD NOT send this to a peer which does not offer `gossip_queries`. + - MUST NOT send this if it has sent a previous `query_channel_range` to this peer and not received all `reply_channel_range` replies. + - MUST set `chain_hash` to the 32-byte hash that uniquely identifies the chain + that it wants the `reply_channel_range` to refer to + - MUST set `first_blocknum` to the first block it wants to know channels for + - MUST set `number_of_blocks` to 1 or greater. + - MAY append an additional `query_channel_range_tlv`, which specifies the type of extended information it would like to receive. + +The receiver of `query_channel_range`: + - if it has not sent all `reply_channel_range` to a previously received `query_channel_range` from this sender: + - MAY send a `warning`. + - MAY close the connection. + - MUST respond with one or more `reply_channel_range`: + - MUST set with `chain_hash` equal to that of `query_channel_range`, + - MUST limit `number_of_blocks` to the maximum number of blocks whose + results could fit in `encoded_short_ids` + - MAY split block contents across multiple `reply_channel_range`. + - the first `reply_channel_range` message: + - MUST set `first_blocknum` less than or equal to the `first_blocknum` in `query_channel_range` + - MUST set `first_blocknum` plus `number_of_blocks` greater than `first_blocknum` in `query_channel_range`. + - successive `reply_channel_range` message: + - MUST have `first_blocknum` equal or greater than the previous `first_blocknum`. + - MUST set `sync_complete` to `false` if this is not the final `reply_channel_range`. + - the final `reply_channel_range` message: + - MUST have `first_blocknum` plus `number_of_blocks` equal or greater than the `query_channel_range` `first_blocknum` plus `number_of_blocks`. + - MUST set `sync_complete` to `true`. + +If the incoming message includes `query_option`, the receiver MAY append additional information to its reply: +- if bit 0 in `query_option_flags` is set, the receiver MAY append a `timestamps_tlv` that contains `channel_update` timestamps for all `short_channel_id`s in `encoded_short_ids` +- if bit 1 in `query_option_flags` is set, the receiver MAY append a `checksums_tlv` that contains `channel_update` checksums for all `short_channel_id`s in `encoded_short_ids` + +#### Rationale + +A single response might be too large for a single packet, so multiple replies +may be required. We want to allow a peer to store canned results for (say) +1000-block ranges, so replies can exceed the requested range. However, we +require that each reply be relevant (overlapping the requested range). + +By insisting that replies be in increasing order, the receiver can easily +determine if replies are done: simply check if `first_blocknum` plus +`number_of_blocks` equals or exceeds the `first_blocknum` plus +`number_of_blocks` it asked for. + +The addition of timestamp and checksum fields allow a peer to omit querying for redundant updates. + +### The `gossip_timestamp_filter` Message + +1. type: 265 (`gossip_timestamp_filter`) +2. data: + * [`chain_hash`:`chain_hash`] + * [`u32`:`first_timestamp`] + * [`u32`:`timestamp_range`] + +This message allows a node to constrain future gossip messages to +a specific range. A node which wants any gossip messages has +to send this, otherwise no gossip messages would be received. + +Note that this filter replaces any previous one, so it can be used +multiple times to change the gossip from a peer. + +#### Requirements + +The sender: + - MUST set `chain_hash` to the 32-byte hash that uniquely identifies the chain + that it wants the gossip to refer to. + - If the receiver does not offer `gossip_queries`: + - SHOULD set `first_timestamp` to 0xFFFFFFFF and `timestamp_range` to 0. + +The receiver: + - SHOULD send all gossip messages whose `timestamp` is greater or + equal to `first_timestamp`, and less than `first_timestamp` plus + `timestamp_range`. + - MAY wait for the next outgoing gossip flush to send these. + - SHOULD send gossip messages as it generates them regardless of `timestamp`. + - Otherwise (relayed gossip): + - SHOULD restrict future gossip messages to those whose `timestamp` + is greater or equal to `first_timestamp`, and less than + `first_timestamp` plus `timestamp_range`. + - If a `channel_announcement` has no corresponding `channel_update`s: + - MUST NOT send the `channel_announcement`. + - If the funding output of the `channel_announcement` has been spent: + - SHOULD NOT send the `channel_announcement`. + - Otherwise: + - MUST consider the `timestamp` of the `channel_announcement` to be the `timestamp` of a corresponding `channel_update`. + - MUST consider whether to send the `channel_announcement` after receiving the first corresponding `channel_update`. + - If a `channel_announcement` is sent: + - MUST send the `channel_announcement` prior to any corresponding `channel_update`s and `node_announcement`s. + +#### Rationale + +Since `channel_announcement` doesn't have a timestamp, we generate a likely +one. If there's no `channel_update` then it is not sent at all, which is most +likely in the case of pruned channels. + +Otherwise the `channel_announcement` is usually followed immediately by a +`channel_update`. Ideally we would specify that the first (oldest) `channel_update`'s +timestamp is to be used as the time of the `channel_announcement`, but new nodes on +the network will not have this, and further would require the first `channel_update` +timestamp to be stored. Instead, we allow any update to be used, which +is simple to implement. + +In the case where the `channel_announcement` is nonetheless missed, +`query_short_channel_ids` can be used to retrieve it. + +Nodes can use `timestamp_filter` to reduce their gossip load when they +have many peers (eg. setting `first_timestamp` to `0xFFFFFFFF` after the +first few peers, in the assumption that propagation is adequate). +This assumption of adequate propagation does not apply for gossip messages +generated directly by the node itself, so they should ignore filters. + +### Requirements + +A node: + - MUST NOT relay any gossip messages it did not generate itself, unless explicitly requested. + +## Rebroadcasting + +### Requirements + +A receiving node: + - upon receiving a new `channel_announcement` or a `channel_update` or + `node_announcement` with an updated `timestamp`: + - SHOULD update its local view of the network's topology accordingly. + - after applying the changes from the announcement: + - if there are no channels associated with the corresponding origin node: + - MAY purge the origin node from its set of known nodes. + - otherwise: + - SHOULD update the appropriate metadata AND store the signature + associated with the announcement. + - Note: this will later allow the node to rebuild the announcement + for its peers. + +A node: + - MUST not send gossip it did not generate itself, until it receives `gossip_timestamp_filter`. + - SHOULD flush outgoing gossip messages once every 60 seconds, independently of + the arrival times of the messages. + - Note: this results in staggered announcements that are unique (not + duplicated). + - SHOULD NOT forward gossip messages to peers who sent `networks` in `init` + and did not specify the `chain_hash` of this gossip message. + - MAY re-announce its channels regularly. + - Note: this is discouraged, in order to keep the resource requirements low. + +### Rationale + +Once the gossip message has been processed, it's added to a list of outgoing +messages, destined for the processing node's peers, replacing any older +updates from the origin node. This list of gossip messages will be flushed at +regular intervals; such a store-and-delayed-forward broadcast is called a +_staggered broadcast_. Also, such batching forms a natural rate +limit with low overhead. + +## HTLC Fees + +### Requirements + +The origin node: + - SHOULD accept HTLCs that pay a fee equal to or greater than: + - fee_base_msat + ( amount_to_forward * fee_proportional_millionths / 1000000 ) + - SHOULD accept HTLCs that pay an older fee, for some reasonable time after + sending `channel_update`. + - Note: this allows for any propagation delay. + +## Pruning the Network View + +### Requirements + +A node: + - SHOULD monitor the funding transactions in the blockchain, to identify + channels that are being closed. + - if the funding output of a channel is spent and received 72 block confirmations: + - SHOULD be removed from the local network view AND be considered closed. + - if the announced node no longer has any associated open channels: + - MAY prune nodes added through `node_announcement` messages from their + local view. + - Note: this is a direct result of the dependency of a `node_announcement` + being preceded by a `channel_announcement`. + +### Recommendation on Pruning Stale Entries + +#### Requirements + +A node: + - if the `timestamp` of the latest `channel_update` in either direction is + older than two weeks (1209600 seconds): + - MAY prune the channel. + - MAY ignore the channel. + - Note: this is an individual node policy and MUST NOT be enforced by + forwarding peers, e.g. by closing channels when receiving outdated gossip + messages. + +#### Rationale + +Several scenarios may result in channels becoming unusable and its endpoints +becoming unable to send updates for these channels. For example, this occurs if +both endpoints lose access to their private keys and can neither sign +`channel_update`s nor close the channel on-chain. In this case, the channels are +unlikely to be part of a computed route, since they would be partitioned off +from the rest of the network; however, they would remain in the local network +view would be forwarded to other peers indefinitely. + +The oldest `channel_update` is used to prune the channel since both sides need +to be active in order for the channel to be usable. Doing so prunes channels +even if one side continues to send fresh `channel_update`s but the other node +has disappeared. + +## Recommendations for Routing + +When calculating a route for an HTLC, both the `cltv_expiry_delta` and the fee +need to be considered: the `cltv_expiry_delta` contributes to the time that +funds will be unavailable in the event of a worst-case failure. The relationship +between these two attributes is unclear, as it depends on the reliability of the +nodes involved. + +If a route is computed by simply routing to the intended recipient and summing +the `cltv_expiry_delta`s, then it's possible for intermediate nodes to guess +their position in the route. Knowing the CLTV of the HTLC, the surrounding +network topology, and the `cltv_expiry_delta`s gives an attacker a way to guess +the intended recipient. Therefore, it's highly desirable to add a random offset +to the CLTV that the intended recipient will receive, which bumps all CLTVs +along the route. + +In order to create a plausible offset, the origin node MAY start a limited +random walk on the graph, starting from the intended recipient and summing the +`cltv_expiry_delta`s, and use the resulting sum as the offset. +This effectively creates a _shadow route extension_ to the actual route and +provides better protection against this attack vector than simply picking a +random offset would. + +Other more advanced considerations involve diversification of route selection, +to avoid single points of failure and detection, and balancing of local +channels. + +### Routing Example + +Consider four nodes: + + +``` + B + / \ + / \ +A C + \ / + \ / + D +``` + +Each advertises the following `cltv_expiry_delta` on its end of every +channel: + +1. A: 10 blocks +2. B: 20 blocks +3. C: 30 blocks +4. D: 40 blocks + +C also uses a `min_final_cltv_expiry_delta` of 18 (the default) when requesting +payments. + +Also, each node has a set fee scheme that it uses for each of its +channels: + +1. A: 100 base + 1000 millionths +2. B: 200 base + 2000 millionths +3. C: 300 base + 3000 millionths +4. D: 400 base + 4000 millionths + +The network will see eight `channel_update` messages: + +1. A->B: `cltv_expiry_delta` = 10, `fee_base_msat` = 100, `fee_proportional_millionths` = 1000 +1. A->D: `cltv_expiry_delta` = 10, `fee_base_msat` = 100, `fee_proportional_millionths` = 1000 +1. B->A: `cltv_expiry_delta` = 20, `fee_base_msat` = 200, `fee_proportional_millionths` = 2000 +1. D->A: `cltv_expiry_delta` = 40, `fee_base_msat` = 400, `fee_proportional_millionths` = 4000 +1. B->C: `cltv_expiry_delta` = 20, `fee_base_msat` = 200, `fee_proportional_millionths` = 2000 +1. D->C: `cltv_expiry_delta` = 40, `fee_base_msat` = 400, `fee_proportional_millionths` = 4000 +1. C->B: `cltv_expiry_delta` = 30, `fee_base_msat` = 300, `fee_proportional_millionths` = 3000 +1. C->D: `cltv_expiry_delta` = 30, `fee_base_msat` = 300, `fee_proportional_millionths` = 3000 + +**B->C.** If B were to send 4,999,999 millisatoshi directly to C, it would +neither charge itself a fee nor add its own `cltv_expiry_delta`, so it would +use C's requested `min_final_cltv_expiry_delta` of 18. Presumably it would also add a +_shadow route_ to give an extra CLTV of 42. Additionally, it could add extra +CLTV deltas at other hops, as these values represent a minimum, but chooses not +to do so here, for the sake of simplicity: + + * `amount_msat`: 4999999 + * `cltv_expiry`: current-block-height + 18 + 42 + * `onion_routing_packet`: + * `amt_to_forward` = 4999999 + * `outgoing_cltv_value` = current-block-height + 18 + 42 + +**A->B->C.** If A were to send 4,999,999 millisatoshi to C via B, it needs to +pay B the fee it specified in the B->C `channel_update`, calculated as +per [HTLC Fees](#htlc-fees): + + fee_base_msat + ( amount_to_forward * fee_proportional_millionths / 1000000 ) + + 200 + ( 4999999 * 2000 / 1000000 ) = 10199 + +Similarly, it would need to add B->C's `channel_update` `cltv_expiry_delta` (20), C's +requested `min_final_cltv_expiry_delta` (18), and the cost for the _shadow route_ (42). +Thus, A->B's `update_add_htlc` message would be: + + * `amount_msat`: 5010198 + * `cltv_expiry`: current-block-height + 20 + 18 + 42 + * `onion_routing_packet`: + * `amt_to_forward` = 4999999 + * `outgoing_cltv_value` = current-block-height + 18 + 42 + +B->C's `update_add_htlc` would be the same as B->C's direct payment above. + +**A->D->C.** Finally, if for some reason A chose the more expensive route via D, +A->D's `update_add_htlc` message would be: + + * `amount_msat`: 5020398 + * `cltv_expiry`: current-block-height + 40 + 18 + 42 + * `onion_routing_packet`: + * `amt_to_forward` = 4999999 + * `outgoing_cltv_value` = current-block-height + 18 + 42 + +And D->C's `update_add_htlc` would again be the same as B->C's direct payment +above. + + +![Creative Commons License](https://i.creativecommons.org/l/by/4.0/88x31.png "License CC-BY") +<br> +This work is licensed under a [Creative Commons Attribution 4.0 International License](http://creativecommons.org/licenses/by/4.0/). diff --git a/plans/IMPL1.md b/plans/IMPL1.md @@ -0,0 +1,240 @@ +# IMPL1: Core BOLT #7 Implementation + +## Overview + +Complete implementation of BOLT #7 routing gossip protocol with full +encode/decode support, validation, and signature verification. + +## Phase 1: Fix Skeleton and Verify Build + +**Goal:** Ensure current skeleton compiles and tests pass. + +**Tasks:** +1. Enter nix develop shell +2. Run `cabal build` and fix any compilation errors +3. Run `cabal test` and verify skeleton tests pass +4. Ensure benchmarks compile + +**Dependencies:** None (can start immediately) + +## Phase 2: Complete Types Module + +**Goal:** Ensure all types are correct and complete. + +**Tasks:** +1. Verify ShortChannelId parsing functions (block height, tx index, output) +2. Add `mkShortChannelId` constructor from components +3. Add human-readable SCID formatting (`539268x845x1` style) +4. Add Bitcoin mainnet chain hash constant +5. Verify Address type covers all descriptor types: + - Type 1: IPv4 (4 bytes + 2 port) + - Type 2: IPv6 (16 bytes + 2 port) + - Type 4: Tor v3 (35 bytes + 2 port) + - Type 5: DNS (1 len + hostname + 2 port) +6. Add Ord instances where needed for lexicographic comparison + +**Dependencies:** Phase 1 + +## Phase 3: Complete Codec Module + +**Goal:** Fully correct encode/decode for all 9 message types. + +**Tasks:** +1. Review and fix `channel_announcement` codec: + - Signature order: node_sig_1, node_sig_2, bitcoin_sig_1, bitcoin_sig_2 + - Features are u16-length-prefixed + - Verify field order matches spec exactly + +2. Review and fix `node_announcement` codec: + - Features are u16-length-prefixed (flen) + - Addresses are u16-length-prefixed (addrlen) + - Verify address type encoding (type byte + data + port) + - Handle unknown address types gracefully (skip by length) + +3. Review and fix `channel_update` codec: + - `message_flags` bit 0 MUST be 1 (htlc_maximum_msat always present) + - Remove Maybe wrapper from htlc_maximum_msat (always required now) + - Verify field order and sizes + +4. Review and fix `announcement_signatures` codec: + - Straightforward fixed-size fields + +5. Review and fix query message codecs: + - `query_short_channel_ids`: chain_hash + u16-prefixed encoded_short_ids + TLV + - `reply_short_channel_ids_end`: chain_hash + full_information byte + - `query_channel_range`: chain_hash + first_blocknum + number_of_blocks + TLV + - `reply_channel_range`: chain_hash + first/num blocks + sync_complete + + u16-prefixed encoded_short_ids + TLV + - `gossip_timestamp_filter`: chain_hash + first_timestamp + timestamp_range + +6. Add TLV type definitions: + - `query_flags` (type 1) for query_short_channel_ids + - `query_option` (type 1) for query_channel_range + - `timestamps_tlv` (type 1) and `checksums_tlv` (type 3) for reply_channel_range + +7. Add encoded_short_ids helpers: + - `encodeShortChannelIds :: [ShortChannelId] -> ByteString` + - `decodeShortChannelIds :: ByteString -> Either DecodeError [ShortChannelId]` + - Encoding type 0 only (uncompressed, ascending order) + +**Dependencies:** Phase 2 + +## Phase 4: Signature Hash Computation + +**Goal:** Support signature verification for gossip messages. + +**Tasks:** +1. Add `channelAnnouncementHash` function: + - Double-SHA256 of message starting at offset 256 (after 4 signatures) + - Returns 32-byte hash for signature verification + +2. Add `nodeAnnouncementHash` function: + - Double-SHA256 of message starting after signature field + +3. Add `channelUpdateHash` function: + - Double-SHA256 of message starting after signature field + +4. Add checksum computation for channel_update: + - CRC32C per RFC3720 + - Excludes signature and timestamp fields + - Used in reply_channel_range checksums_tlv + +**Dependencies:** Phase 3 +**External dependency:** May need ppad-crc32c or similar + +## Phase 5: Validation Module + +**Goal:** Implement message validation rules from spec. + +**Tasks:** +1. Create `Lightning.Protocol.BOLT7.Validate` module + +2. Add `validateChannelAnnouncement`: + - node_id_1 < node_id_2 (lexicographic ordering) + - All signatures present and correct length + - Feature bits validation (unknown even bits = error) + +3. Add `validateNodeAnnouncement`: + - Timestamp must be valid + - Feature bits validation + - Address list validation (no duplicate type 5 DNS entries) + - Alias is valid UTF-8 (or at least doesn't crash) + +4. Add `validateChannelUpdate`: + - message_flags bit 0 must be 1 + - htlc_minimum_msat <= htlc_maximum_msat + - Timestamp validation + +5. Add `validateQueryChannelRange`: + - first_blocknum + number_of_blocks doesn't overflow + +6. Add `validateReplyChannelRange`: + - Encoded short_channel_ids are in ascending order + - Timestamp/checksum counts match SCID count + +**Dependencies:** Phase 3 + +## Phase 6: Tests + +**Goal:** Comprehensive test coverage. + +**Tasks:** +1. Add encode/decode roundtrip tests for all message types +2. Add validation tests (valid and invalid cases) +3. Add property tests: + - Roundtrip: decode(encode(x)) == x + - SCID component extraction consistency + - Lexicographic ordering preservation +4. Add edge case tests: + - Empty feature bits + - Empty address lists + - Maximum-size messages + - Boundary values for numeric fields + +**Dependencies:** Phases 3, 5 + +## Phase 7: Benchmarks + +**Goal:** Performance baselines for all operations. + +**Tasks:** +1. Complete criterion benchmarks for all encode/decode operations +2. Complete weigh benchmarks for allocation tracking +3. Add benchmarks for: + - Signature hash computation + - SCID list encoding/decoding + - Validation functions + +**Dependencies:** Phases 3, 4, 5 + +## Phase 8: Documentation + +**Goal:** Complete Haddock documentation. + +**Tasks:** +1. Document all exported types with examples +2. Document all exported functions with examples +3. Add module-level documentation explaining BOLT #7 protocol +4. Verify `cabal haddock` produces clean output + +**Dependencies:** All previous phases + +--- + +## Dependency Graph + +``` +Phase 1 (build fix) + | + v +Phase 2 (types) + | + v +Phase 3 (codec) + | + +---> Phase 4 (sig hash) --+ + | | + +---> Phase 5 (validate) --+ + | + v + Phase 6 (tests) + | + v + Phase 7 (benchmarks) + | + v + Phase 8 (docs) +``` + +Phases 4 and 5 can proceed in parallel after Phase 3. + +--- + +## Open Questions + +1. **CRC32C dependency:** Need to decide whether to add ppad-crc32c or + implement inline. Required for channel_update checksums in + reply_channel_range. + +2. **Signature verification:** This library provides hash computation only. + Actual verification requires ppad-secp256k1 and is left to the caller. + Should we add optional verification helpers? + +3. **Zlib compression:** Encoding type 1 for encoded_short_ids uses zlib. + Spec says MUST NOT use, but we may need to decode it. Add zlib dependency + or reject? + +--- + +## Estimated Complexity + +| Phase | Complexity | Notes | +|-------|------------|-------| +| 1 | Low | Just build fixes | +| 2 | Low | Minor type additions | +| 3 | Medium | Core codec work | +| 4 | Medium | Requires careful hash construction | +| 5 | Medium | Many validation rules | +| 6 | Medium | Comprehensive test writing | +| 7 | Low | Mechanical benchmark setup | +| 8 | Low | Documentation pass |