commit c9aef2076cf97347841c16de38657caa531c1622
parent e94d28f4bcbf99fe3e6d74f615ee60e9507d3463
Author: Jared Tobin <jared@jtobin.io>
Date: Sun, 25 Jan 2026 10:44:22 +0400
docs: add README and implementation plans
- README.md: library overview, usage example, dev instructions
- plans/: architecture notes, implementation plans (IMPL1-*), reviews
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Diffstat:
10 files changed, 326 insertions(+), 0 deletions(-)
diff --git a/README.md b/README.md
@@ -0,0 +1,75 @@
+# ppad-bolt1
+
+[](https://hackage.haskell.org/package/ppad-bolt1)
+
+[](https://docs.ppad.tech/bolt1)
+
+A Haskell implementation of the Lightning Network base protocol (BOLT
+#1), providing message and TLV encoding/decoding with validation.
+
+## Usage
+
+A sample GHCi session:
+
+```
+ > :set -XOverloadedStrings
+ >
+ > import qualified Data.ByteString as BS
+ > import Lightning.Protocol.BOLT1
+ >
+ > let msg = MsgPingVal (Ping 10 "")
+ > let ext = TlvStream [TlvRecord 101 "ext"]
+ >
+ > Right enc = encodeEnvelope msg (Just ext)
+ > enc
+ "\NUL\DC2\NUL\n\NUL\NULe\ETXext"
+ >
+ > decodeEnvelope enc
+ Right (Just (MsgPingVal (Ping {pingNumPongBytes = 10, pingIgnored = ""})),
+ Just (TlvStream [TlvRecord {tlvType = 101, tlvValue = "ext"}]))
+```
+
+## Documentation
+
+Haddocks (API documentation, etc.) are hosted at
+[docs.ppad.tech/bolt1](https://docs.ppad.tech/bolt1).
+
+## Performance
+
+The aim is best-in-class performance for message encoding/decoding.
+Benchmarks are available under `bench/` and can be run with:
+
+```
+$ cabal bench
+```
+
+## Security
+
+This library targets safe, validated parsing of Lightning BOLT #1
+messages. If you discover any vulnerabilities, please disclose them via
+security@ppad.tech.
+
+## Development
+
+You'll require [Nix][nixos] with [flake][flake] support enabled. Enter a
+development shell with:
+
+```
+$ nix develop
+```
+
+Then do e.g.:
+
+```
+$ cabal repl ppad-bolt1
+```
+
+to get a REPL for the main library.
+
+## Attribution
+
+This library implements the Lightning Network BOLT #1 specification:
+https://github.com/lightning/bolts/blob/master/01-messaging.md
+
+[nixos]: https://nixos.org/
+[flake]: https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake.html
diff --git a/plans/ARCH1.md b/plans/ARCH1.md
@@ -0,0 +1,72 @@
+# ARCH1 - ppad-bolt1 BOLT #1 Library Architecture
+
+## Goals
+
+- Provide a safe, total, and performant Haskell implementation of BOLT #1
+ message encoding/decoding.
+- Encode protocol invariants in types where practical, with smart
+ constructors for validation.
+- Keep dependencies minimal (base/bytestring/primitive, ppad-*).
+- Provide clear error reporting for parse and protocol violations.
+
+## Scope
+
+- BOLT #1 message envelope, TLV stream, and defined messages:
+ init, error, warning, ping, pong, peer_storage, peer_storage_retrieval.
+- Fundamental types used by BOLT #1: big-endian u/s integers, truncated
+ unsigned integers, BigSize, and fixed-size byte fields.
+
+## Module Layout (proposed)
+
+- Lightning.Protocol.BOLT1
+ - High-level API and re-exports.
+
+- Lightning.Protocol.BOLT1.Prim
+ - Encoding/decoding primitives for:
+ u16/u32/u64, s16/s32/s64, truncated unsigned ints, BigSize.
+ - Minimal encoding checks, bounded size validation.
+
+- Lightning.Protocol.BOLT1.TLV
+ - TLV record and stream types.
+ - Encode/decode and validation (ordering, minimal encoding, length bounds,
+ unknown even behavior).
+
+- Lightning.Protocol.BOLT1.Message
+ - ADTs for BOLT #1 messages and message envelope.
+ - Feature bitset types and init TLVs.
+ - Smart constructors for messages with validation.
+
+- Lightning.Protocol.BOLT1.Codec
+ - Encode/decode for messages and envelopes.
+ - Error types and mapping from decode failures to protocol errors.
+
+## Error Model
+
+- Parse errors:
+ - non-minimal encoding, insufficient data, length mismatch, invalid TLV
+ ordering, unknown even TLV, invalid extension.
+- Protocol errors:
+ - unknown even message type, invalid message length for known type.
+
+Errors should be structured so callers can decide when to drop/close
+connections vs. ignore a message, per spec.
+
+## Performance Strategy
+
+- Strict fields with UNPACK where it pays off.
+- INLINE small encode/decode helpers.
+- Prefer ByteString builders with manual sizing for small frames.
+- Avoid intermediate allocations in TLV parsing by slicing input.
+
+## Public API
+
+- Total encode/decode functions returning Either error message.
+- Types re-exported from a single module for consumers:
+ message ADTs, TLV types, and common primitives.
+
+## Testing and Benchmarking
+
+- Unit tests from BOLT #1 vectors (BigSize, signed ints).
+- Property tests for round-trip and minimal encodings.
+- Benchmarks for encode/decode hot paths and allocation tracking.
+
diff --git a/plans/IMPL1-1.md b/plans/IMPL1-1.md
@@ -0,0 +1,20 @@
+# IMPL1-1 - Core Primitives
+
+## Scope
+
+- Big-endian unsigned/signed integers (u16/u32/u64, s16/s32/s64).
+- Truncated unsigned integers (tu16/tu32/tu64) with minimal encoding.
+- BigSize encode/decode with minimal checks.
+
+## Work
+
+- Implement encode/decode functions with total APIs.
+- Add minimality and bounds validation.
+- Provide strict, small helpers for hot paths.
+
+## Tests
+
+- Appendix A BigSize vectors.
+- Appendix D signed integer vectors.
+- Negative tests for non-minimal encodings.
+
diff --git a/plans/IMPL1-2.md b/plans/IMPL1-2.md
@@ -0,0 +1,20 @@
+# IMPL1-2 - TLV Streams
+
+## Scope
+
+- TLV record and TLV stream types.
+- Encode/decode with BOLT #1 validation rules.
+
+## Work
+
+- Strictly increasing type ordering.
+- Minimal encoding checks for type/length.
+- Length bounds validation.
+- Unknown even type fails; unknown odd skipped.
+
+## Tests
+
+- Appendix B vectors if available.
+- Property tests for ordering/minimality.
+- Negative tests for invalid lengths and unknown even types.
+
diff --git a/plans/IMPL1-3.md b/plans/IMPL1-3.md
@@ -0,0 +1,23 @@
+# IMPL1-3 - Message Types
+
+## Scope
+
+- ADTs for BOLT #1 messages:
+ - init (including init_tlvs: networks, remote_addr)
+ - error, warning
+ - ping, pong
+ - peer_storage, peer_storage_retrieval
+- Message envelope with type + payload + optional extension TLV.
+- Feature bitset modeling and helpers.
+
+## Work
+
+- Define strict, UNPACKed fields where helpful.
+- Add smart constructors for validated fields.
+- Enforce padding/byte-aligned feature lengths.
+
+## Tests
+
+- Unit tests for constructors and invariants.
+- Roundtrip tests for type-level encodings.
+
diff --git a/plans/IMPL1-4.md b/plans/IMPL1-4.md
@@ -0,0 +1,21 @@
+# IMPL1-4 - Message Codec Integration
+
+## Scope
+
+- Encode/decode for each BOLT #1 message type.
+- Envelope encoding with optional extension TLV.
+- Enforce unknown-type/extension behavior per spec.
+
+## Work
+
+- Decode known types with strict length checks.
+- Unknown odd type: ignore.
+- Unknown even type: fail (close connection).
+- Invalid extension TLV: fail (close connection).
+
+## Tests
+
+- Known-good vectors per message type.
+- Negative tests for invalid lengths and invalid TLV extension.
+- Property tests for encode/decode invariants.
+
diff --git a/plans/IMPL1-5.md b/plans/IMPL1-5.md
@@ -0,0 +1,12 @@
+# IMPL1-5 - Test Suite Expansion
+
+## Scope
+
+- Comprehensive test coverage for BOLT #1 codecs.
+
+## Work
+
+- Message roundtrip tests for all defined types.
+- Negative tests for minimal encoding and length violations.
+- Property tests for TLV ordering and BigSize minimality.
+
diff --git a/plans/IMPL1-6.md b/plans/IMPL1-6.md
@@ -0,0 +1,14 @@
+# IMPL1-6 - Benchmarks
+
+## Scope
+
+- Criterion benchmarks for encode/decode hot paths.
+- Weigh benchmarks for allocation tracking.
+
+## Work
+
+- Bench BigSize encode/decode.
+- Bench TLV stream encode/decode.
+- Bench full message encode/decode.
+- Add NFData instances where required.
+
diff --git a/plans/IMPL2.md b/plans/IMPL2.md
@@ -0,0 +1,29 @@
+# IMPL2 - Replace ByteString Builders
+
+## Goal
+
+Remove ByteString.Builder usage and replace with unsafe ByteString
+construction primitives, with careful bounds/length handling.
+
+## Scope
+
+- Library code that currently uses `Data.ByteString.Builder` for encoding.
+- Update tests if they depend on builder behavior or output ordering.
+
+## Plan
+
+1) Identify builder usage in `lib/` (e.g., `encodeU16/U32/U64`).
+2) Replace with unsafe ByteString creation primitives:
+ - Use `BS.create` or `BS.unsafeCreate` with explicit writes.
+ - Prefer `poke`/`pokeByteOff` for big-endian layout.
+ - Ensure bounds correctness and totality.
+3) Remove `bytestring` builder imports and any now-unused deps.
+4) Add/adjust tests for encode functions to ensure exact bytes.
+5) Run tests to confirm no regressions.
+
+## Notes
+
+- Keep new helpers small and INLINE.
+- Validate length fields before unsafe writes to avoid overflow.
+- Avoid introducing new dependencies.
+
diff --git a/plans/REVIEW-80d0966.md b/plans/REVIEW-80d0966.md
@@ -0,0 +1,40 @@
+# Review: 80d0966
+
+## Findings (ordered by severity)
+
+- High: `encodeEnvelope` can append an extension TLV, but `decodeEnvelope` never
+ parses or surfaces extensions, so extension data is lost and invalid
+ extensions can’t be detected; this breaks roundtrips for any envelope using
+ `mext` and contradicts the API comment about invalid extensions.
+ (`lib/Lightning/Protocol/BOLT1.hs:512-517`,
+ `lib/Lightning/Protocol/BOLT1.hs:638-655`)
+
+- High: `decodeTlvStream` is hard-wired to treat only TLV types 1 and 3 as
+ “known,” so any even TLV in other contexts (including future extension TLVs)
+ will be rejected even when it should be accepted. This makes the exported TLV
+ decoder unusable for anything besides `init_tlvs`.
+ (`lib/Lightning/Protocol/BOLT1.hs:264-275`)
+
+- Medium: Length fields are encoded with `fromIntegral (BS.length ...)` and no
+ bounds checks, so payloads longer than 65535 bytes will wrap and produce
+ invalid encodings instead of failing fast. This affects every `u16` length
+ field encoder.
+ (`lib/Lightning/Protocol/BOLT1.hs:435-487`)
+
+- Medium: `decodeMessage` treats `MsgUnknown` odd types as
+ `DecodeInsufficientBytes`, which is misleading and can cause clients calling
+ `decodeMessage` directly to close/abort when the spec says unknown odd should
+ be ignored.
+ (`lib/Lightning/Protocol/BOLT1.hs:624-636`)
+
+- Low: `unhex` uses `error` on invalid input, which is a partial failure path in
+ tests; consider total helpers in tests for consistency with safety guidance.
+ (`test/Main.hs:328-331`)
+
+## Open questions / assumptions
+
+- Should `decodeEnvelope` return extension TLVs (e.g., via `Envelope`), or should
+ there be a separate decode API for validating and extracting extensions?
+- Should `decodeTlvStream` accept a known-type predicate or map so it can be
+ reused across message-specific TLVs?
+
