bolt1

Base Lightning protocol, per BOLT #1 (docs.ppad.tech/bolt1).
git clone git://git.ppad.tech/bolt1.git
Log | Files | Refs | README | LICENSE

IMPL5.md (4347B)

      1 # IMPL5: Benchmark Suite Population
      2 
      3 Populate the empty criterion (wall-time) and weigh (allocation) benchmark
      4 suites with comprehensive coverage of encoding/decoding operations.
      5 
      6 ## Overview
      7 
      8 Current state:
      9 - `bench/Main.hs`: empty criterion suite (just `defaultMain []`)
     10 - `bench/Weight.hs`: empty weigh suite (just `pure ()`)
     11 - All types have NFData instances (ready for benchmarking)
     12 
     13 ## Benchmark Categories
     14 
     15 ### 1. Primitive Encoding (Prim.hs)
     16 
     17 Fixed-size unsigned:
     18 - `encodeU16`, `encodeU32`, `encodeU64`
     19 
     20 Fixed-size signed:
     21 - `encodeS8`, `encodeS16`, `encodeS32`, `encodeS64`
     22 
     23 Truncated unsigned (variable-size):
     24 - `encodeTu16`, `encodeTu32`, `encodeTu64`
     25 - Test multiple size classes (0, small, medium, max)
     26 
     27 Special encodings:
     28 - `encodeMinSigned` (boundary cases: -128, 127, 128, -129, large)
     29 - `encodeBigSize` (0, 252, 253, 65535, 65536, large)
     30 
     31 ### 2. Primitive Decoding (Prim.hs)
     32 
     33 Fixed-size unsigned:
     34 - `decodeU16`, `decodeU32`, `decodeU64`
     35 
     36 Fixed-size signed:
     37 - `decodeS8`, `decodeS16`, `decodeS32`, `decodeS64`
     38 
     39 Truncated unsigned:
     40 - `decodeTu16`, `decodeTu32`, `decodeTu64`
     41 
     42 Special decodings:
     43 - `decodeMinSigned`
     44 - `decodeBigSize`
     45 
     46 ### 3. TLV Operations (TLV.hs)
     47 
     48 Encoding:
     49 - `encodeTlvRecord` (single record)
     50 - `encodeTlvStream` (varying sizes: 1, 5, 20 records)
     51 
     52 Decoding:
     53 - `decodeTlvStreamRaw` (varying sizes)
     54 - `decodeTlvStream` (with init validation)
     55 - `decodeTlvStreamWith` (custom predicate)
     56 
     57 Init TLV:
     58 - `parseInitTlvs`
     59 - `encodeInitTlvs`
     60 
     61 ### 4. Message Encoding (Codec.hs)
     62 
     63 Individual messages:
     64 - `encodeInit` (minimal, with TLVs)
     65 - `encodeError` (minimal, with data)
     66 - `encodeWarning`
     67 - `encodePing` (minimal, with padding)
     68 - `encodePong`
     69 - `encodePeerStorage`
     70 - `encodePeerStorageRetrieval`
     71 
     72 Envelope:
     73 - `encodeMessage` (each message type)
     74 - `encodeEnvelope` (with/without extension TLVs)
     75 
     76 ### 5. Message Decoding (Codec.hs)
     77 
     78 Individual messages:
     79 - `decodeInit`
     80 - `decodeError`
     81 - `decodeWarning`
     82 - `decodePing`
     83 - `decodePong`
     84 - `decodePeerStorage`
     85 - `decodePeerStorageRetrieval`
     86 
     87 Envelope:
     88 - `decodeMessage`
     89 - `decodeEnvelope`
     90 - `decodeEnvelopeWith`
     91 
     92 ### 6. Round-Trip Benchmarks
     93 
     94 Encode then decode for each message type:
     95 - Measures combined codec performance
     96 - Verifies no accumulation overhead
     97 
     98 ## Implementation Steps
     99 
    100 Steps 1-3 can be done in parallel (independent files).
    101 
    102 ### Step 1: Create test fixtures module
    103 
    104 Create `bench/Fixtures.hs` with:
    105 - Sample ByteStrings of various sizes
    106 - Pre-constructed message values (Init, Error, Ping, etc.)
    107 - Sample TlvStream values
    108 - Encoded message bytes for decode benchmarks
    109 
    110 ### Step 2: Implement criterion benchmarks (bench/Main.hs)
    111 
    112 Structure:
    113 ```haskell
    114 main = defaultMain
    115   [ bgroup "prim/encode" [...]
    116   , bgroup "prim/decode" [...]
    117   , bgroup "tlv/encode" [...]
    118   , bgroup "tlv/decode" [...]
    119   , bgroup "message/encode" [...]
    120   , bgroup "message/decode" [...]
    121   , bgroup "envelope" [...]
    122   , bgroup "roundtrip" [...]
    123   ]
    124 ```
    125 
    126 Use `bench`, `nf`, `whnf` appropriately:
    127 - `nf` for functions returning data structures
    128 - `whnf` for functions returning strict values
    129 
    130 ### Step 3: Implement weigh benchmarks (bench/Weight.hs)
    131 
    132 Structure:
    133 ```haskell
    134 main = mainWith $ do
    135   func "encodeU16" encodeU16 0x1234
    136   func "encodeU32" encodeU32 0x12345678
    137   ...
    138   func "encodeInit/minimal" encodeInit minimalInit
    139   func "encodeInit/with-tlvs" encodeInit initWithTlvs
    140   ...
    141 ```
    142 
    143 Track allocations for:
    144 - Primitive encoders (to verify Builder overhead)
    145 - Message encoders (to establish baselines)
    146 - TLV stream operations (to verify accumulator behavior)
    147 
    148 ### Step 4: Update cabal file if needed
    149 
    150 Verify `bench/Fixtures.hs` is included in both benchmark stanzas.
    151 Add any missing `other-modules` entries.
    152 
    153 ### Step 5: Run and validate
    154 
    155 ```
    156 cabal bench bolt1-bench
    157 cabal bench bolt1-weigh
    158 ```
    159 
    160 Verify:
    161 - All benchmarks run without error
    162 - Results are reasonable (no obvious performance cliffs)
    163 - Allocation tracking captures expected patterns
    164 
    165 ## Test Data Sizes
    166 
    167 Use consistent sizing for comparison:
    168 - "minimal": smallest valid message
    169 - "small": ~64 bytes payload
    170 - "medium": ~1KB payload
    171 - "large": ~16KB payload (approaching protocol limits)
    172 
    173 ## Notes
    174 
    175 - Keep fixture generation pure (no IO in benchmark loops)
    176 - Use `env` combinator for expensive setup if needed
    177 - Consider adding `NOINLINE` to fixtures to prevent constant folding
    178 - Document any surprising results in comments