IMPL3.md (14170B)
1 # IMPL3: Benchmark Suite Expansion 2 3 ## Overview 4 5 Implementation plan for expanding benchmark coverage per ARCH3. 6 7 ## Step 0: NFData Instances 8 9 Add NFData instances for all types that will be benchmarked. These are 10 required for both criterion (`nf`) and weigh (`func`). 11 12 Add to `bench/Main.hs` and `bench/Weight.hs`: 13 14 ```haskell 15 -- Transaction types 16 instance NFData CommitmentTx where 17 rnf (CommitmentTx v l i s o f) = 18 rnf v `seq` rnf l `seq` rnf i `seq` rnf s `seq` rnf o `seq` rnf f 19 20 instance NFData HTLCTx where 21 rnf (HTLCTx v l i s ov os) = 22 rnf v `seq` rnf l `seq` rnf i `seq` rnf s `seq` rnf ov `seq` rnf os 23 24 instance NFData ClosingTx where 25 rnf (ClosingTx v l i s o f) = 26 rnf v `seq` rnf l `seq` rnf i `seq` rnf s `seq` rnf o `seq` rnf f 27 28 -- Output types 29 instance NFData TxOutput where 30 rnf (TxOutput v s t) = rnf v `seq` rnf s `seq` rnf t 31 32 instance NFData OutputType where 33 rnf OutputToLocal = () 34 rnf OutputToRemote = () 35 rnf OutputLocalAnchor = () 36 rnf OutputRemoteAnchor = () 37 rnf (OutputOfferedHTLC e) = rnf e 38 rnf (OutputReceivedHTLC e) = rnf e 39 40 -- Primitives 41 instance NFData Script where 42 rnf (Script bs) = rnf bs 43 44 instance NFData Witness where 45 rnf (Witness items) = rnf items 46 47 instance NFData Outpoint where 48 rnf (Outpoint t i) = rnf t `seq` rnf i 49 50 instance NFData Sequence where 51 rnf (Sequence x) = rnf x 52 53 instance NFData Locktime where 54 rnf (Locktime x) = rnf x 55 56 instance NFData TxId where 57 rnf (TxId bs) = rnf bs 58 59 instance NFData ToSelfDelay where 60 rnf (ToSelfDelay x) = rnf x 61 62 instance NFData CommitmentNumber where 63 rnf (CommitmentNumber x) = rnf x 64 65 -- Parsing types 66 instance NFData RawTx where 67 rnf (RawTx v i o w l) = 68 rnf v `seq` rnf i `seq` rnf o `seq` rnf w `seq` rnf l 69 70 instance NFData RawInput where 71 rnf (RawInput o scr seq) = rnf o `seq` rnf scr `seq` rnf seq 72 73 instance NFData RawOutput where 74 rnf (RawOutput v s) = rnf v `seq` rnf s 75 76 -- Context types (for env setup verification) 77 instance NFData CommitmentContext where 78 rnf ctx = rnf (cc_funding_outpoint ctx) `seq` 79 rnf (cc_commitment_number ctx) `seq` 80 rnf (cc_htlcs ctx) `seq` 81 rnf (cc_keys ctx) 82 83 instance NFData CommitmentKeys where 84 rnf keys = rnf (ck_revocation_pubkey keys) `seq` 85 rnf (ck_local_delayed keys) `seq` 86 rnf (ck_local_htlc keys) `seq` 87 rnf (ck_remote_htlc keys) 88 89 instance NFData HTLCContext where 90 rnf ctx = rnf (hc_commitment_txid ctx) `seq` 91 rnf (hc_htlc ctx) 92 93 instance NFData ClosingContext where 94 rnf ctx = rnf (clc_funding_outpoint ctx) `seq` 95 rnf (clc_local_amount ctx) `seq` 96 rnf (clc_remote_amount ctx) 97 98 -- Key types 99 instance NFData LocalDelayedPubkey where 100 rnf (LocalDelayedPubkey p) = rnf p 101 102 instance NFData RemoteDelayedPubkey where 103 rnf (RemoteDelayedPubkey p) = rnf p 104 105 instance NFData LocalHtlcPubkey where 106 rnf (LocalHtlcPubkey p) = rnf p 107 108 instance NFData RemoteHtlcPubkey where 109 rnf (RemoteHtlcPubkey p) = rnf p 110 111 instance NFData LocalPubkey where 112 rnf (LocalPubkey p) = rnf p 113 114 instance NFData RemotePubkey where 115 rnf (RemotePubkey p) = rnf p 116 117 instance NFData PaymentBasepoint where 118 rnf (PaymentBasepoint p) = rnf p 119 120 instance NFData DelayedPaymentBasepoint where 121 rnf (DelayedPaymentBasepoint p) = rnf p 122 123 instance NFData HtlcBasepoint where 124 rnf (HtlcBasepoint p) = rnf p 125 126 instance NFData FundingPubkey where 127 rnf (FundingPubkey p) = rnf p 128 129 instance NFData PerCommitmentSecret where 130 rnf (PerCommitmentSecret bs) = rnf bs 131 132 -- Secret storage 133 instance NFData SecretStore where 134 rnf store = rnf (length store) -- approximate; avoids deep traversal 135 136 instance NFData SecretEntry where 137 rnf (SecretEntry i idx sec) = rnf i `seq` rnf idx `seq` rnf sec 138 ``` 139 140 **Note**: Some of these may already exist or need adjustment based on 141 actual constructor visibility. Check exports and adjust. 142 143 **Independent**: Yes, can be done first before other steps. 144 145 ## Step 1: Test Fixtures 146 147 Define shared test fixtures in `bench/Main.hs`. Use `env` for expensive 148 setup to exclude from timing. 149 150 ```haskell 151 -- Sample pubkeys (33-byte compressed, from BOLT #3 test vectors) 152 samplePubkey1, samplePubkey2, samplePubkey3 :: Pubkey 153 samplePubkey1 = Pubkey $ BS.pack [0x03, 0x6d, 0x6c, ...] 154 -- ... (use actual BOLT #3 test vector keys) 155 156 -- Funding outpoint 157 sampleFundingOutpoint :: Outpoint 158 sampleFundingOutpoint = Outpoint 159 (TxId $ BS.replicate 32 0x01) 160 0 161 162 -- Sample HTLCs 163 mkHtlc :: HTLCDirection -> Word64 -> Word32 -> HTLC 164 mkHtlc dir amtMsat expiry = HTLC 165 { htlc_direction = dir 166 , htlc_amount_msat = MilliSatoshi amtMsat 167 , htlc_payment_hash = PaymentHash (BS.replicate 32 0x00) 168 , htlc_cltv_expiry = CltvExpiry expiry 169 } 170 171 htlcs0, htlcs10, htlcs100 :: [HTLC] 172 htlcs0 = [] 173 htlcs10 = [mkHtlc (if even i then HTLCOffered else HTLCReceived) 174 (5000000 + i * 100000) 175 (500000 + i) 176 | i <- [0..9]] 177 htlcs100 = [mkHtlc (if even i then HTLCOffered else HTLCReceived) 178 (5000000 + i * 10000) 179 (500000 + i) 180 | i <- [0..99]] 181 182 -- CommitmentKeys fixture 183 sampleCommitmentKeys :: CommitmentKeys 184 sampleCommitmentKeys = CommitmentKeys 185 { ck_revocation_pubkey = RevocationPubkey samplePubkey1 186 , ck_local_delayed = LocalDelayedPubkey samplePubkey1 187 , ck_local_htlc = LocalHtlcPubkey samplePubkey1 188 , ck_remote_htlc = RemoteHtlcPubkey samplePubkey2 189 , ck_local_payment = LocalPubkey samplePubkey1 190 , ck_remote_payment = RemotePubkey samplePubkey2 191 , ck_local_funding = FundingPubkey samplePubkey1 192 , ck_remote_funding = FundingPubkey samplePubkey2 193 } 194 195 -- CommitmentContext builder 196 mkCommitmentContext :: [HTLC] -> ChannelFeatures -> CommitmentContext 197 mkCommitmentContext htlcs features = CommitmentContext 198 { cc_funding_outpoint = sampleFundingOutpoint 199 , cc_commitment_number = CommitmentNumber 42 200 , cc_local_payment_bp = PaymentBasepoint samplePubkey1 201 , cc_remote_payment_bp = PaymentBasepoint samplePubkey2 202 , cc_to_self_delay = ToSelfDelay 144 203 , cc_dust_limit = DustLimit (Satoshi 546) 204 , cc_feerate = FeeratePerKw 5000 205 , cc_features = features 206 , cc_is_funder = True 207 , cc_to_local_msat = MilliSatoshi 500000000 208 , cc_to_remote_msat = MilliSatoshi 500000000 209 , cc_htlcs = htlcs 210 , cc_keys = sampleCommitmentKeys 211 } 212 ``` 213 214 **Independent**: Yes, after Step 0. 215 216 ## Step 2: Transaction Building Benchmarks 217 218 Add to `bench/Main.hs`: 219 220 ```haskell 221 bgroup "tx building" [ 222 bench "build_commitment_tx (0 htlcs, no anchors)" $ 223 whnf build_commitment_tx (mkCommitmentContext htlcs0 noAnchors) 224 , bench "build_commitment_tx (10 htlcs, no anchors)" $ 225 whnf build_commitment_tx (mkCommitmentContext htlcs10 noAnchors) 226 , bench "build_commitment_tx (100 htlcs, no anchors)" $ 227 whnf build_commitment_tx (mkCommitmentContext htlcs100 noAnchors) 228 , bench "build_commitment_tx (10 htlcs, anchors)" $ 229 whnf build_commitment_tx (mkCommitmentContext htlcs10 withAnchors) 230 , bench "build_htlc_timeout_tx" $ 231 whnf build_htlc_timeout_tx sampleHtlcContext 232 , bench "build_htlc_success_tx" $ 233 whnf build_htlc_success_tx sampleHtlcContext 234 , bench "build_closing_tx" $ 235 whnf build_closing_tx sampleClosingContext 236 ] 237 ``` 238 239 Define `sampleHtlcContext` and `sampleClosingContext` fixtures. 240 241 **Independent**: Yes, after Step 1. 242 243 ## Step 3: Script Generation Benchmarks 244 245 Add to `bench/Main.hs`: 246 247 ```haskell 248 bgroup "script generation" [ 249 bench "funding_script" $ 250 whnf (funding_script (FundingPubkey samplePubkey1)) 251 (FundingPubkey samplePubkey2) 252 , bench "to_local_script" $ 253 whnf (to_local_script (RevocationPubkey samplePubkey1) 254 (ToSelfDelay 144)) 255 (LocalDelayedPubkey samplePubkey2) 256 , bench "to_remote_script (no anchors)" $ 257 whnf (to_remote_script (RemotePubkey samplePubkey1)) noAnchors 258 , bench "to_remote_script (anchors)" $ 259 whnf (to_remote_script (RemotePubkey samplePubkey1)) withAnchors 260 , bench "anchor_script" $ 261 whnf anchor_script (FundingPubkey samplePubkey1) 262 , bench "offered_htlc_script" $ 263 whnf (offered_htlc_script (RevocationPubkey samplePubkey1) 264 (RemoteHtlcPubkey samplePubkey2) 265 (LocalHtlcPubkey samplePubkey3) 266 (PaymentHash $ BS.replicate 32 0)) 267 noAnchors 268 , bench "received_htlc_script" $ 269 whnf (received_htlc_script (RevocationPubkey samplePubkey1) 270 (RemoteHtlcPubkey samplePubkey2) 271 (LocalHtlcPubkey samplePubkey3) 272 (PaymentHash $ BS.replicate 32 0) 273 (CltvExpiry 500000)) 274 noAnchors 275 ] 276 ``` 277 278 **Independent**: Yes, after Step 1. 279 280 ## Step 4: Serialization Benchmarks 281 282 Add to `bench/Main.hs`: 283 284 ```haskell 285 bgroup "serialization" [ 286 env (pure $ build_commitment_tx $ mkCommitmentContext htlcs0 noAnchors) 287 $ \tx -> bench "encode_tx (0 htlcs)" $ whnf encode_tx tx 288 , env (pure $ build_commitment_tx $ mkCommitmentContext htlcs10 noAnchors) 289 $ \tx -> bench "encode_tx (10 htlcs)" $ whnf encode_tx tx 290 , env (pure $ build_commitment_tx $ mkCommitmentContext htlcs100 noAnchors) 291 $ \tx -> bench "encode_tx (100 htlcs)" $ whnf encode_tx tx 292 , bench "encode_htlc_tx" $ 293 whnf encode_htlc_tx (build_htlc_timeout_tx sampleHtlcContext) 294 , bench "encode_closing_tx" $ 295 whnf encode_closing_tx (build_closing_tx sampleClosingContext) 296 ] 297 ``` 298 299 **Independent**: Yes, after Step 2 (needs tx fixtures). 300 301 ## Step 5: Parsing Benchmarks 302 303 Add to `bench/Main.hs`: 304 305 ```haskell 306 bgroup "parsing" [ 307 env (pure $ encode_tx $ build_commitment_tx $ 308 mkCommitmentContext htlcs0 noAnchors) 309 $ \bs -> bench "decode_tx (0 htlcs)" $ whnf decode_tx bs 310 , env (pure $ encode_tx $ build_commitment_tx $ 311 mkCommitmentContext htlcs10 noAnchors) 312 $ \bs -> bench "decode_tx (10 htlcs)" $ whnf decode_tx bs 313 , env (pure $ encode_tx $ build_commitment_tx $ 314 mkCommitmentContext htlcs100 noAnchors) 315 $ \bs -> bench "decode_tx (100 htlcs)" $ whnf decode_tx bs 316 ] 317 ``` 318 319 **Independent**: Yes, after Step 4 (needs encoded bytes). 320 321 ## Step 6: Validation Benchmarks 322 323 Add to `bench/Main.hs`: 324 325 ```haskell 326 bgroup "validation" [ 327 env (pure $ build_commitment_tx $ mkCommitmentContext htlcs10 noAnchors) 328 $ \tx -> bench "validate_commitment_tx (valid)" $ 329 whnf (validate_commitment_tx dust noAnchors) tx 330 , env (pure $ build_htlc_timeout_tx sampleHtlcContext) 331 $ \tx -> bench "validate_htlc_tx" $ 332 whnf (validate_htlc_tx dust noAnchors) tx 333 , env (pure $ build_closing_tx sampleClosingContext) 334 $ \tx -> bench "validate_closing_tx" $ 335 whnf validate_closing_tx tx 336 , env (pure $ ctx_outputs $ build_commitment_tx $ 337 mkCommitmentContext htlcs10 noAnchors) 338 $ \outs -> bench "validate_output_ordering" $ 339 whnf validate_output_ordering outs 340 ] 341 ``` 342 343 **Independent**: Yes, after Step 2. 344 345 ## Step 7: Secret Storage Benchmarks 346 347 Add to `bench/Main.hs`: 348 349 ```haskell 350 bgroup "secret storage" [ 351 bench "insert_secret (first)" $ 352 whnf (insert_secret empty_store 281474976710655) 353 (PerCommitmentSecret $ BS.replicate 32 0xFF) 354 , env setupFilledStore $ \store -> 355 bench "derive_old_secret (recent)" $ 356 whnf (derive_old_secret store) 281474976710654 357 , env setupFilledStore $ \store -> 358 bench "derive_old_secret (old)" $ 359 whnf (derive_old_secret store) 281474976710600 360 ] 361 where 362 setupFilledStore = pure $ foldl insertOne empty_store [0..99] 363 insertOne store i = 364 let idx = 281474976710655 - i 365 sec = PerCommitmentSecret $ BS.replicate 32 (fromIntegral i) 366 in case insert_secret store idx sec of 367 Just s -> s 368 Nothing -> store 369 ``` 370 371 **Independent**: Yes, after Step 0. 372 373 ## Step 8: Output Sorting Benchmarks 374 375 Add to `bench/Main.hs`: 376 377 ```haskell 378 bgroup "output sorting" [ 379 env (pure $ ctx_outputs $ build_commitment_tx $ 380 mkCommitmentContext htlcs10 noAnchors) 381 $ \outs -> bench "sort_outputs (10)" $ nf sort_outputs outs 382 , env (pure $ ctx_outputs $ build_commitment_tx $ 383 mkCommitmentContext htlcs100 noAnchors) 384 $ \outs -> bench "sort_outputs (100)" $ nf sort_outputs outs 385 ] 386 ``` 387 388 **Independent**: Yes, after Step 2. 389 390 ## Step 9: Mirror in Weight.hs 391 392 Replicate all new benchmarks in `bench/Weight.hs` using `weigh`: 393 394 ```haskell 395 -- Transaction building 396 func "build_commitment_tx (0 htlcs)" 397 build_commitment_tx (mkCommitmentContext htlcs0 noAnchors) 398 func "build_commitment_tx (10 htlcs)" 399 build_commitment_tx (mkCommitmentContext htlcs10 noAnchors) 400 -- ... etc 401 ``` 402 403 Use same fixtures. Structure allocation tracking to match criterion 404 groups. 405 406 **Independent**: Can proceed in parallel with Steps 2-8. 407 408 ## Step 10: Verify and Clean Up 409 410 - Build and run benchmarks: `cabal bench` 411 - Verify all benchmarks execute without error 412 - Check for reasonable timing/allocation values 413 - Remove any duplicate NFData instances if they conflict with library 414 - Ensure fixture setup is excluded from measurement via `env` 415 416 **Depends on**: All previous steps. 417 418 ## Parallelization Summary 419 420 Independent work items: 421 422 1. **Step 0** (NFData instances) — must be first 423 2. **Step 1** (fixtures) — after Step 0 424 3. **Steps 2, 3, 7** — after Step 1, independent of each other 425 4. **Step 4** — after Step 2 426 5. **Steps 5, 6, 8** — after Step 4, independent of each other 427 6. **Step 9** — can run in parallel with Steps 2-8 428 7. **Step 10** — final validation 429 430 Recommended parallel execution: 431 432 ``` 433 Step 0 → Step 1 → ┬→ Step 2 → Step 4 → ┬→ Step 5 434 │ ├→ Step 6 435 │ └→ Step 8 436 ├→ Step 3 437 ├→ Step 7 438 └→ Step 9 (parallel throughout) 439 ↓ 440 Step 10 441 ``` 442 443 ## Deliverables 444 445 - Expanded `bench/Main.hs` with all benchmark groups 446 - Expanded `bench/Weight.hs` with matching allocation tracking 447 - NFData instances for all benchmarked types 448 - Shared fixture definitions 449 - All benchmarks passing `cabal bench`