IMPL1.md (7702B)
1 # IMPL1: Core BOLT #7 Implementation 2 3 ## Overview 4 5 Complete implementation of BOLT #7 routing gossip protocol with full 6 encode/decode support, validation, and signature verification. 7 8 ## Phase 1: Fix Skeleton and Verify Build 9 10 **Goal:** Ensure current skeleton compiles and tests pass. 11 12 **Tasks:** 13 1. Enter nix develop shell 14 2. Run `cabal build` and fix any compilation errors 15 3. Run `cabal test` and verify skeleton tests pass 16 4. Ensure benchmarks compile 17 18 **Dependencies:** None (can start immediately) 19 20 ## Phase 2: Complete Types Module 21 22 **Goal:** Ensure all types are correct and complete. 23 24 **Tasks:** 25 1. Verify ShortChannelId parsing functions (block height, tx index, output) 26 2. Add `mkShortChannelId` constructor from components 27 3. Add human-readable SCID formatting (`539268x845x1` style) 28 4. Add Bitcoin mainnet chain hash constant 29 5. Verify Address type covers all descriptor types: 30 - Type 1: IPv4 (4 bytes + 2 port) 31 - Type 2: IPv6 (16 bytes + 2 port) 32 - Type 4: Tor v3 (35 bytes + 2 port) 33 - Type 5: DNS (1 len + hostname + 2 port) 34 6. Add Ord instances where needed for lexicographic comparison 35 36 **Dependencies:** Phase 1 37 38 ## Phase 3: Complete Codec Module 39 40 **Goal:** Fully correct encode/decode for all 9 message types. 41 42 **Tasks:** 43 1. Review and fix `channel_announcement` codec: 44 - Signature order: node_sig_1, node_sig_2, bitcoin_sig_1, bitcoin_sig_2 45 - Features are u16-length-prefixed 46 - Verify field order matches spec exactly 47 48 2. Review and fix `node_announcement` codec: 49 - Features are u16-length-prefixed (flen) 50 - Addresses are u16-length-prefixed (addrlen) 51 - Verify address type encoding (type byte + data + port) 52 - Handle unknown address types gracefully (skip by length) 53 54 3. Review and fix `channel_update` codec: 55 - `message_flags` bit 0 MUST be 1 (htlc_maximum_msat always present) 56 - Remove Maybe wrapper from htlc_maximum_msat (always required now) 57 - Verify field order and sizes 58 59 4. Review and fix `announcement_signatures` codec: 60 - Straightforward fixed-size fields 61 62 5. Review and fix query message codecs: 63 - `query_short_channel_ids`: chain_hash + u16-prefixed encoded_short_ids + TLV 64 - `reply_short_channel_ids_end`: chain_hash + full_information byte 65 - `query_channel_range`: chain_hash + first_blocknum + number_of_blocks + TLV 66 - `reply_channel_range`: chain_hash + first/num blocks + sync_complete + 67 u16-prefixed encoded_short_ids + TLV 68 - `gossip_timestamp_filter`: chain_hash + first_timestamp + timestamp_range 69 70 6. Add TLV type definitions: 71 - `query_flags` (type 1) for query_short_channel_ids 72 - `query_option` (type 1) for query_channel_range 73 - `timestamps_tlv` (type 1) and `checksums_tlv` (type 3) for reply_channel_range 74 75 7. Add encoded_short_ids helpers: 76 - `encodeShortChannelIds :: [ShortChannelId] -> ByteString` 77 - `decodeShortChannelIds :: ByteString -> Either DecodeError [ShortChannelId]` 78 - Encoding type 0 only (uncompressed, ascending order) 79 80 **Dependencies:** Phase 2 81 82 ## Phase 4: Signature Hash and Checksum Computation 83 84 **Goal:** Support signature verification and checksums for gossip messages. 85 86 **Tasks:** 87 1. Add `channelAnnouncementHash` function: 88 - Double-SHA256 of message starting at offset 256 (after 4 signatures) 89 - Returns 32-byte hash for signature verification 90 91 2. Add `nodeAnnouncementHash` function: 92 - Double-SHA256 of message starting after signature field 93 94 3. Add `channelUpdateHash` function: 95 - Double-SHA256 of message starting after signature field 96 97 4. Add internal `Lightning.Protocol.BOLT7.CRC32C` helper module: 98 - Implement CRC32C (Castagnoli polynomial 0x1EDC6F41) per RFC3720 99 - Pure Haskell implementation, no external dependencies 100 - Can be refactored into ppad-crc32c later if needed elsewhere 101 102 5. Add `channelUpdateChecksum` function: 103 - CRC32C of channel_update excluding signature and timestamp fields 104 - Used in reply_channel_range checksums_tlv 105 106 **Dependencies:** Phase 3 107 108 ## Phase 5: Validation Module 109 110 **Goal:** Implement message validation rules from spec. 111 112 **Tasks:** 113 1. Create `Lightning.Protocol.BOLT7.Validate` module 114 115 2. Add `validateChannelAnnouncement`: 116 - node_id_1 < node_id_2 (lexicographic ordering) 117 - All signatures present and correct length 118 - Feature bits validation (unknown even bits = error) 119 120 3. Add `validateNodeAnnouncement`: 121 - Timestamp must be valid 122 - Feature bits validation 123 - Address list validation (no duplicate type 5 DNS entries) 124 - Alias is valid UTF-8 (or at least doesn't crash) 125 126 4. Add `validateChannelUpdate`: 127 - message_flags bit 0 must be 1 128 - htlc_minimum_msat <= htlc_maximum_msat 129 - Timestamp validation 130 131 5. Add `validateQueryChannelRange`: 132 - first_blocknum + number_of_blocks doesn't overflow 133 134 6. Add `validateReplyChannelRange`: 135 - Encoded short_channel_ids are in ascending order 136 - Timestamp/checksum counts match SCID count 137 138 **Dependencies:** Phase 3 139 140 ## Phase 6: Tests 141 142 **Goal:** Comprehensive test coverage. 143 144 **Tasks:** 145 1. Add encode/decode roundtrip tests for all message types 146 2. Add validation tests (valid and invalid cases) 147 3. Add property tests: 148 - Roundtrip: decode(encode(x)) == x 149 - SCID component extraction consistency 150 - Lexicographic ordering preservation 151 4. Add edge case tests: 152 - Empty feature bits 153 - Empty address lists 154 - Maximum-size messages 155 - Boundary values for numeric fields 156 157 **Dependencies:** Phases 3, 5 158 159 ## Phase 7: Benchmarks 160 161 **Goal:** Performance baselines for all operations. 162 163 **Tasks:** 164 1. Complete criterion benchmarks for all encode/decode operations 165 2. Complete weigh benchmarks for allocation tracking 166 3. Add benchmarks for: 167 - Signature hash computation 168 - SCID list encoding/decoding 169 - Validation functions 170 171 **Dependencies:** Phases 3, 4, 5 172 173 ## Phase 8: Documentation 174 175 **Goal:** Complete Haddock documentation. 176 177 **Tasks:** 178 1. Document all exported types with examples 179 2. Document all exported functions with examples 180 3. Add module-level documentation explaining BOLT #7 protocol 181 4. Verify `cabal haddock` produces clean output 182 183 **Dependencies:** All previous phases 184 185 --- 186 187 ## Dependency Graph 188 189 ``` 190 Phase 1 (build fix) 191 | 192 v 193 Phase 2 (types) 194 | 195 v 196 Phase 3 (codec) 197 | 198 +---> Phase 4 (sig hash) --+ 199 | | 200 +---> Phase 5 (validate) --+ 201 | 202 v 203 Phase 6 (tests) 204 | 205 v 206 Phase 7 (benchmarks) 207 | 208 v 209 Phase 8 (docs) 210 ``` 211 212 Phases 4 and 5 can proceed in parallel after Phase 3. 213 214 --- 215 216 ## Open Questions 217 218 1. **Signature verification:** This library provides hash computation only. 219 Actual verification requires ppad-secp256k1 and is left to the caller. 220 Should we add optional verification helpers? 221 222 2. **Zlib compression:** Encoding type 1 for encoded_short_ids uses zlib. 223 Spec says MUST NOT use, but we may need to decode it. Add zlib dependency 224 or reject? 225 226 ## Resolved 227 228 1. **CRC32C:** Implement in internal helper module 229 `Lightning.Protocol.BOLT7.CRC32C`. Refactor to ppad-crc32c later if 230 needed elsewhere. 231 232 --- 233 234 ## Estimated Complexity 235 236 | Phase | Complexity | Notes | 237 |-------|------------|-------| 238 | 1 | Low | Just build fixes | 239 | 2 | Low | Minor type additions | 240 | 3 | Medium | Core codec work | 241 | 4 | Medium | Requires careful hash construction | 242 | 5 | Medium | Many validation rules | 243 | 6 | Medium | Comprehensive test writing | 244 | 7 | Low | Mechanical benchmark setup | 245 | 8 | Low | Documentation pass |